Lightweight robust unsupervised zipper defect detection method

Abstract

The invention provides a lightweight robust unsupervised zipper defect detection method. The method comprises the following steps: preprocessing an acquired zipper image data set to obtain a grey-scale map data set; a defect detection model is constructed, the defect detection model is trained according to the grey-scale map data set, in the training process, a difficult case learning mechanism is introduced, features which are most difficult to align are focused, constraints of an abnormal mode are increased, features of normal and abnormal samples stored in a difficult case memory bank are dynamically updated, and a student network feature space is optimized through comparative learning; obtaining a normal zipper image fine tuning generation model, generating defect description in combination with a large language model to construct a pseudo-defect sample set, and analyzing output distribution of the trained defect detection model according to the pseudo-defect sample set to obtain a discrimination threshold value; obtaining a to-be-detected gray zipper graph, inputting the to-be-detected gray zipper graph into the trained defect detection model to obtain an output result, and performing defect detection according to the output result and the judgment threshold; therefore, the defect detection effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual defect detection, and in particular to a lightweight and robust unsupervised zipper defect detection method, a computer readable storage medium and a computer device. BACKGROUND

[0002] In related technologies, as a basic component widely used in clothing, luggage, home furnishing and other fields, the zipper has a sustained demand in the market due to its simple structure, low cost, convenient use and strong irreplaceability. In the high-strength and long-period production process, defects such as missing teeth, broken lines, cloth belt wrinkles, color difference and pollution often occur, which greatly affect the use of the zipper and cause economic losses to the zipper manufacturers and users. Therefore, the zipper needs to go through a strict defect detection link before leaving the factory to ensure its use performance and appearance quality.

[0003] The existing zipper defect detection method is mostly based on a traditional supervised deep learning framework, and the model training process needs a large number of manually labeled defect samples as support. The existing unsupervised detection method still needs real defect samples as the basis when determining the detection threshold. When the types of defects and product types continue to increase, the number of required defect samples increases exponentially, which seriously limits the deployment efficiency and economy of the visual detection system, and it is difficult to meet the requirements of rapid online and low-cost detection for large-scale industrial production.

[0004] In the actual detection process, some defect types are difficult to obtain sufficient labeled samples for model training due to their extremely low occurrence frequency or unknown morphology, which are called "difficult example defects". The existing supervised models or rule-based algorithms have weak recognition ability for such difficult example defects, and often can only rely on a large number of labeled samples for training or rely on experience-based rules for inference. This lack of difficult example recognition strategy causes potential quality problems in the production process, which may lead to serious production accidents or quality problems, causing great economic and brand risks to enterprises.

[0005] When processing zipper image samples, the existing technology uses a gray scale processing method, which aims to generalize the model to zippers with different tooth parts and cloth belt color combinations. However, under the same lighting conditions, the brightness value of the gray scale image is also affected by the color of the zipper. Simple gray scale processing cannot effectively simulate the actual situation of different tooth parts and cloth belt color combinations, resulting in the model being unable to generalize to zippers with a wide range of color combinations. SUMMARY

[0006] The present application aims to at least solve one of the above technical problems in the art. To this end, one object of the present application is to provide a lightweight and robust unsupervised zipper defect detection method, which can realize the training of the detection model and the adaptive estimation of the discrimination threshold without real defect samples, effectively alleviating the high dependence on real defect data in the model training stage or the threshold setting process before inference.

[0007] A second object of the present application is to provide a computer-readable storage medium.

[0008] A third object of the present application is to provide a computer device.

[0009] To achieve the above object, the first aspect of the present application provides a lightweight and robust unsupervised zipper defect detection method, which comprises the following steps: obtaining a zipper image dataset; preprocessing each zipper image in the zipper image dataset to obtain a grayscale zipper image and corresponding tooth mask, tape mask and background mask; performing geometric data enhancement on the grayscale zipper image, and performing brightness data enhancement on the grayscale zipper image according to the tooth mask and the tape mask to obtain an enhanced grayscale image dataset; constructing a defect detection model, the defect detection model comprising a pre-trained teacher network and a student network, inputting the enhanced grayscale image dataset into the defect detection model to train the student network to learn to align the teacher feature and extract normal sample features, and perform image reconstruction, wherein, in the training process, a hard example learning mechanism is introduced, focusing on the most difficult to align features and increasing the constraint of abnormal patterns, and a dynamic update hard example memory is used to store normal and abnormal sample features, and the student network feature space is optimized through contrastive learning, and the feature difference between normal and abnormal samples is enlarged; obtaining a normal zipper image fine-tuning text image generation model, and automatically generating defect descriptions combined with a large language model to construct a diversified pseudo-defect sample set, analyzing the output distribution of the trained defect detection model according to the pseudo-defect sample set to obtain a discrimination threshold; obtaining a grayscale zipper image to be detected and inputting it into the trained defect detection model to obtain an output result, and determining whether the detected grayscale zipper image has defects according to the output result and the judgment threshold; thus, without real defect samples, the training of the detection model and the adaptive estimation of the discrimination threshold can be realized, effectively alleviating the high dependence on real defect data in the model training stage or the threshold setting process before inference.

[0010] In addition, the lightweight and robust unsupervised zipper defect detection method according to the above embodiments of the present application can also have the following additional technical features:

[0011] Optionally, each of the zip image data sets is pre-processed to obtain a grayscale zip image and corresponding tooth mask, tape mask and background mask, including: performing illumination component and reflection component separation on each of the zip image data sets, and targeted enhancement to obtain a brightness uniform RGB image; converting the brightness uniform RGB image into a grayscale zip image, and inputting the pre-trained segmentation model to obtain the corresponding tooth mask, tape mask and background mask of each grayscale zip image.

[0012] Optionally, the illumination component and reflection component separation and targeted enhancement of each of the zip image data sets to obtain a brightness uniform RGB image includes: performing multi-scale Retinex decomposition on the zip image to separate it into a reflection component and an illumination component; performing equalization adjustment on the illumination component; and recombining the adjusted illumination component and the reflection component to obtain a brightness uniform RGB image.

[0013] Optionally, the brightness data of the grayscale zip image is enhanced according to the following formula:

[0014] I out (x,y)=I in (x,y)γ

[0015] wherein I in and I out are pixel values before and after adjustment, respectively, and γ is a random brightness adjustment coefficient generated based on the Gamma correction principle, with γ<1 indicating that the image is brightened, and γ>1 indicating that the image is darkened. Different γ values are used for the tooth region and the tape region using the tooth mask and the tape mask to perform brightness enhancement, so as to obtain a grayscale image with different color combinations.

[0016] Optionally, the training process includes the following steps:

[0017]

[0018] L total =L hard +αL p +βL rec +γL con

[0019] wherein L total is the total loss of the student network, L hard is the difficult feature loss, L p is the penalty loss, L rec is the reconstruction loss, L con is the contrast loss, the feature representation output by the encoder of the student network is F S , and the high-dimensional feature representation output by the teacher network is F T, the reconstructed image output by the decoder of the student network is represented as N represents the number of input dataset images, and the nth input image is represented as x n Ω is the selected "hard feature" index set, C'xH'xW' is the dimension of the feature output by the student encoder, represents the high-dimensional feature at the cth channel, position (i, j), represents the feature at the cth channel, position (i, j), and the feature map obtained only by inputting the student network encoder is represented as F S (x'), x' represents a penalty sample, s is the feature of the student network, t + is the positive sample feature from the memory bank, is the negative sample feature.

[0020] Optionally, a normal zipper image fine-tuning text image generation model is obtained, and a large language model is combined to automatically generate defect descriptions to construct a diversified pseudo-defect sample set. The output distribution of the trained defect detection model is analyzed according to the pseudo-defect sample set to obtain a discrimination threshold, including: when fine-tuning the text image generation model, registering a special token in the text encoder to realize semantic guidance of a specific subject part; when generating a defect description, constructing a defect text description template, and using a large language model to generate a variety of template defect description sentences conforming to physical laws; when generating a defect image, based on the template defect description sentence and the subject mask corresponding to the description sentence, jointly constrain the defect generation process; when estimating the threshold, use pseudo-defect samples of different defect types and different severity to form a simulation test set together with normal samples, and the threshold corresponding to the maximum F1 score of the defect detection model on the simulation test set is set as the discrimination threshold during inference.

[0021] Optionally, a structure preservation mechanism, a frequency domain disturbance constraint, and a latent space disturbance strategy are also introduced when generating a defect image. The structure preservation mechanism constrains defect injection to only occur in the zipper subject area specified by the mask. The frequency domain disturbance constraint is to apply a loss function based on frequency characteristics in the latent space of the text image generation model to limit the deviation of the synthesized image from the original image in the frequency domain. The latent space disturbance strategy uses differentiated latent variable manipulation methods according to different defect types and visual features, including directional offset of text representation, directional disturbance of initial noise, replacement and guidance of local attention area, to guide the text image generation model to generate a defect area with specific semantic features and spatial distribution in the latent space.

[0022] Optionally, the gray-scale zipper image to be detected is obtained and input into the trained defect detection model to obtain an output result, and whether the detected gray-scale zipper image has defects is determined according to the output result and the judgment threshold, including: inputting the gray-scale zipper image to be detected into the teacher network and the student network of the trained defect detection model to obtain corresponding features; comparing the corresponding features of the teacher network and the corresponding features of the student network to obtain a difference in structure of the gray-scale zipper image to be detected; performing feature matching detection on the gray-scale zipper image to be detected and the difficult example memory library to obtain a memory library matching image; fusing the difference image and the memory library matching image to obtain an abnormal score image; performing smoothing processing on the abnormal score image using a Gaussian filter, and comparing the processed abnormal score image with a discrimination threshold, and determining that the abnormal score image is abnormal when the abnormal score image is greater than the discrimination threshold, and determining that the abnormal score image is normal when the abnormal score image is less than the discrimination threshold, so as to obtain a binary image of normal and abnormal and a final detection result.

[0023] To achieve the above object, the second aspect of the present application provides a computer readable storage medium, which stores a lightweight robust unsupervised zipper defect detection program, and the lightweight robust unsupervised zipper defect detection program is executed by a processor to implement the lightweight robust unsupervised zipper defect detection method as described above.

[0024] To achieve the above object, the third aspect of the present application provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the lightweight robust unsupervised zipper defect detection method as described above when executing the program. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A flowchart of the lightweight robust unsupervised zipper defect detection method according to the embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and should not be understood as a limitation of the present application.

[0027] In order to better understand the above technical solutions, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0028] For better understanding of the above technical solutions, the above technical solutions will be described in detail below in combination with the drawings and specific embodiments of the specification.

[0029] Reference Figure 1 As shown in the figure, the lightweight robust unsupervised zipper defect detection method of the embodiment of the application includes the following steps:

[0030] S101, acquiring a zipper image dataset.

[0031] It should be noted that the zipper image can be collected by an industrial camera to obtain the zipper image dataset.

[0032] S102, preprocessing each zipper image in the zipper image dataset to obtain a gray zipper image and corresponding tooth mask, tape mask and background mask.

[0033] That is, through adaptive illumination processing and gray scale conversion operation, the interference of illumination variation is eliminated, providing a stable image basis for subsequent defect detection, and through structure segmentation, the accurate masks of tooth, tape and background area are obtained, providing clear structure information for subsequent analysis.

[0034] As an embodiment, preprocessing each zipper image in the zipper image dataset to obtain a gray zipper image and corresponding tooth mask, tape mask and background mask includes: separating the illumination component and the reflection component of each zipper image in the zipper image dataset, and enhancing it to obtain a brightness uniform RGB image; converting the brightness uniform RGB image into a gray zipper image, and inputting the pre-trained segmentation model to obtain the tooth mask, tape mask and background mask corresponding to each gray zipper image.

[0035] That is, adaptive illumination processing based on Retinux is performed on the zipper image collected by the industrial camera, the local illumination intensity deviation is eliminated, the processed RGB image is converted into a single-channel gray image, and then the structure segmentation model is inputted to obtain the masks of the tooth, tape and background, which are the basis for the subsequent steps.

[0036] As a specific embodiment, first, adaptive lighting processing is performed: based on the Retinux theory, by separating the lighting component and the reflection component and targeted enhancement, adaptive lighting processing is realized to eliminate the interference of uneven lighting. Then, gray processing is performed: the RGB image with uniform brightness is converted into a single-channel gray image to further improve the clarity and contrast, and the gray processing adopts the weighted average method (the red, green and blue channel weights are 0.299, 0.587 and 0.114 respectively) to convert the color image into a gray image; the histogram equalization maps the cumulative pixel ratio of each gray level to the gray range of 0-255 to enhance the contrast of the image and provide a more stable image basis for subsequent detection. Next, structure segmentation is performed: a zipper sample is collected and preprocessed by the above steps to obtain a gray image, and the tooth mask and the tape mask of the zipper are manually labeled for training the personalized segmentation model PerSAM-F; in the test stage, the zipper image is input into the trained PerSAM-F to obtain the tooth mask, the tape mask and the background mask. The background mask is not directly output by the PerSAM-F, but is defined as the part of the entire image area excluding the tooth area and the tape area.

[0037] As an embodiment, the lighting component and the reflection component of each zipper image in the zipper image dataset are separated and targetedly enhanced to obtain an RGB image with uniform brightness, including: performing multi-scale Retinex decomposition on the zipper image to separate the reflection component and the lighting component; performing equalization adjustment on the lighting component; and recombining the adjusted lighting component and the reflection component to obtain the RGB image with uniform brightness.

[0038] As a specific embodiment, first, multi-scale Retinex decomposition is performed: the collected zipper image is subjected to multi-scale Retinex decomposition, and the image is separated into a reflection component (reflecting the characteristics of the object itself) and an illumination component (reflecting the influence of environmental illumination); wherein the image is subjected to convolution operation through a plurality of Gaussian filter functions of different scales, the perception of different detail scales by the human eye is simulated, and the reflection component is obtained through weighted calculation; in the experiment, three scales of 15, 80 and 250 are selected, and weight distribution (the total weight is 1) is used, which can achieve good decomposition effect, and the illumination component is obtained through the ratio of the input image to the reflection component. Then, adaptive illumination adjustment is performed: according to the local illumination characteristics of the zipper image, the illumination component is adaptively adjusted; specifically, the intensity of the area with too strong illumination is reduced, and the intensity of the area with too weak illumination is enhanced; by calculating statistical information such as the mean and variance of the local area, the adjustment parameter is determined, that is, the ratio of the area mean value to the target mean value is taken as the adjustment coefficient, and the illumination component of the area is scaled by the coefficient to realize the balanced adjustment of the illumination. Then, image reconstruction is performed: the adjusted illumination component and the reflection component are combined again to reconstruct the preprocessed image, specifically, by multiplying the two, the image that eliminates the illumination interference and retains the characteristics of the object itself is restored.

[0039] S103, the gray scale zipper image is subjected to geometric data enhancement, and the gray scale zipper image is subjected to brightness data enhancement according to the tooth mask and the belt mask to obtain an enhanced gray scale image data set.

[0040] That is, through the geometric transformation and brightness enhancement operation, the detection model can adapt to various spatial position changes and illumination conditions, and the robustness of the model to the zipper in different states is improved.

[0041] Specifically, by adding basic changes such as rotation and translation and nonlinear brightness adjustment to the preprocessed data, the robustness of the model to spatial position and attitude changes of the target object is achieved.

[0042] As a specific embodiment, first, geometric data augmentation is performed: rotation and translation transformations are applied to the input image; for the rotation transformation, the center of the image is taken as the origin, and a rotation angle θ is randomly selected, θ is in the range of [-15°, +15°], and the edge blank area generated after rotation is filled by a mirror filling strategy, i.e., by copying the mirror value of the image edge pixel to avoid introducing invalid pixels (such as black filling); for the translation transformation, the image is randomly translated in the direction perpendicular to the zipper movement, and the translation distance is within ±10% of the image height, and the blank area after translation is also filled by a mirror filling. Then, brightness data augmentation is performed: the pixel values of the input image are subjected to nonlinear brightness adjustment; based on the Gamma correction principle, a random brightness adjustment coefficient γ is generated, γ is in the range of [0.7, 1.3], and the brightness channel (or linear combination of RGB channels) of each pixel of the input image is adjusted according to the following formula:

[0043] I out (x,y)=I in (x,y)γ

[0044] where I in and I out are the pixel values before and after adjustment, respectively, and when γ < 1, the image becomes brighter, and when γ > 1, the image becomes darker. If the input is an RGB image, different γ values are applied to each color channel independently to simulate changes in light color temperature.

[0045] It should be noted that under the same lighting conditions, the brightness value of a grayscale image is also affected by the color of the zipper, and simply adjusting the brightness of the entire image cannot effectively simulate the actual situation of different tooth parts and belt color combinations. For example, if the tooth part region is bright and the belt region is dark in the original grayscale image, the relative brightness relationship of "bright tooth part and dark belt" will still be maintained after uniform adjustment. Therefore, the tooth part mask and the belt mask obtained in the foregoing steps are used to perform brightness enhancement on the tooth part region and the belt region respectively using different γ values to simulate grayscale images with different color combinations.

[0046] S104, a defect detection model is constructed, the defect detection model includes a pre-trained teacher network and a student network, and the enhanced grayscale image dataset is input into the defect detection model to train the student network to learn to align the teacher feature extraction normal sample features and perform image reconstruction, wherein, in the training process, a hard example learning mechanism is introduced, focusing on the most difficult to align features and increasing the constraint of abnormal patterns, and a dynamic update of a hard example memory bank stores normal and abnormal sample features, and the student network feature space is optimized through contrastive learning to increase the feature difference between normal and abnormal samples.

[0047] That is, unsupervised learning is performed through a teacher-student network architecture, the teacher network provides stable normal mode features, the student network learns to extract normal sample features by aligning the teacher features, and image reconstruction is performed; during training, a hard example learning mechanism is introduced, focusing on the most difficult to align features and increasing the constraints of abnormal patterns, thereby improving the detection ability of detailed abnormalities; at the same time, a dynamically updated hard example memory stores normal and abnormal features, further enhancing the model's ability to distinguish abnormal patterns; finally, the student network feature space is optimized through contrastive learning, and the feature difference between normal and abnormal samples is enlarged; thus, the model can learn normal mode features without labeled data, and improve the detection ability of detailed abnormalities through hard example learning technology, thereby effectively distinguishing the micro defects in the zipper.

[0048] As a specific embodiment, first, a teacher-student network architecture is designed: a double-branch structure is built by building a teacher network and a student network to achieve unsupervised knowledge distillation and feature alignment, so as to complete the training of the defect detection model without using labeled data.

[0049] Specifically, the teacher network adopts a deep convolutional neural network or a visual Transformer model pre-trained on a large-scale image dataset, and the parameters are kept frozen during training to provide stable and reliable feature representation in the normal mode; the student network is built by combining convolutional layers and pooling layers to construct an efficient feature extraction network, which is aligned with the features output by the teacher to guide the student to learn the features and paradigm of normal samples; the features extracted by the student network are aligned with the features extracted by the pre-trained teacher model pixel by pixel, so that the student model can distill the knowledge learned by the teacher model in advance, and through element-by-element matching and alignment, the student model can obtain the ability of detailed detection, so as to detect specific structural abnormalities. Preferably, the pre-training weights of the teacher network are used to partially initialize the student network before training (such as using the same weights as the corresponding layers of the teacher for the student encoder), and the un-initialized part is randomly assigned to accelerate convergence and ensure stable training. The above architecture design ensures that the teacher branch provides high-quality feature benchmarks, and the student branch has reconstruction capability and is ready to approach the teacher feature distribution through training.

[0050] Next, feature alignment and knowledge distillation are performed: in the teacher-student network architecture design, since in the real world, the types of abnormalities cannot be exhausted, and labeling the types of abnormalities also requires a lot of expert knowledge, therefore, supervised anomaly detection with labeled data is very time-consuming and labor-intensive, therefore, the teacher-student model is preliminarily trained under unsupervised conditions, and the training set only contains normal samples, so that the student network learns the reconstruction of normal samples and approaches the teacher features.

[0051] Specifically, a batch of enhanced normal sample images is input into the teacher network and the student network: the teacher network outputs a high-dimensional feature representation F T , the student encoder outputs a corresponding feature representation F S , and the student decoder outputs a reconstructed image Then, two-part loss is defined: one is the feature distillation loss, which calculates the mean square error between the corresponding feature maps of the teacher and the student, which is expressed in the formula as Minimizing this loss can make the features extracted by the student encoder as close as possible to the teacher network, thereby obtaining similar representation ability to the teacher. The second is the reconstruction loss, which measures the pixel difference between the input image and the reconstructed image , preferably using L2 norm calculation: Encourage the student network to restore normal sample details to the greatest extent. In summary, the two losses are weighted to form the target function L stage1 of the preliminary training distill = L rec , where λ0is the balance coefficient. The Adam or SGD optimization algorithm is used to iteratively update the student network parameters (the teacher network parameters remain unchanged). After several rounds of training, the student encoder gradually masters the feature extraction ability similar to the teacher, and the decoder can reconstruct an image similar to the input, thereby establishing a preliminary representation of the normal mode. However, the sensitivity of the model to subtle abnormalities is limited, and further improvement is needed through the difficult example mechanism.

[0052] Then, introduce difficult example learning: in feature alignment and knowledge distillation, introduce difficult example learning mechanism, focus on the most difficult difference features between teacher and student, and increase abnormal mode constraints to improve the recognition ability of student network to subtle abnormalities.

[0053] Specifically, by calculating the difference Δ = |F T -F S | between the corresponding feature maps of the teacher and the student for each training sample, where Δ c,i,j represents the difference amplitude of the two at channel c, position (i, j). Usually, most of the normal area student features can approach the teacher features, and only a small number of areas have significant differences. Therefore, this embodiment only selects the most error-prone areas to update the student network, that is, defines the difficult feature loss L hard Only the sum of squares of errors for the elements with the largest value in Δ is calculated. As shown in the formula:

[0054]

[0055] Ω is the set of selected "hard features" indexes, containing the positions of the highest error values in Δ (e.g. the top 0.1% of the total elements), and Ω is the number of elements in Ω. By minimizing the error of only these most difficult areas, the student network will focus on learning the details of the features that are difficult to simulate, avoiding the overemphasis on easy-to-learn features caused by average error training, thereby improving the sensitivity to subtle abnormal patterns.

[0056] Then, to prevent the student network from overfitting the normal data distribution and responding strongly to abnormal patterns, an abnormal penalty term is added to the hard example training.

[0057] Specifically, an input without normal patterns is constructed as a penalty sample x' (e.g. a random noise image or a large range of abnormal patterns superimposed on a normal image), and only the student network encoder is input to obtain the feature map F S (x'). The student network is expected to extract features close to zero vectors for x', so the penalty loss L p is defined as the average energy of the feature map, as shown in the formula:

[0058]

[0059] where C' x H' x W' is the dimension of the student encoder output feature. This penalty term limits the amplitude of the student network's feature activation for abnormal patterns, prompting it to be "insensitive" to unseen abnormal inputs.

[0060] The hard feature loss and abnormal penalty term are weighted together with the reconstruction loss to form the new training objective function of the student network, as shown in the formula:

[0061] L student = L hard + αL p + βL rec

[0062] where α and β are weight hyperparameters. Based on the above loss, the student network is further trained: the hard feature loss guides the student to focus on tackling feature differences that have not yet been learned well, and the abnormal penalty term avoids the student network from adapting to abnormal patterns incorrectly. The combination of these two aspects enables the model to maintain good reconstruction of normal patterns while significantly enhancing its sensitivity to abnormal patterns.

[0063] Then, a hard example memory bank is introduced: after completing the introduction of hard example learning, the implementation constructs a hard example memory bank to store normal and abnormal feature patterns, and dynamically updates it during the training process. A contrast learning mechanism is introduced to further optimize the feature space of the student network to make the normal and abnormal feature distributions more separable.

[0064] Specifically, the difficult example memory bank includes two parts: a normal feature bank and an abnormal feature bank. The former stores feature prototypes of various normal samples, and the latter stores feature representations of various abnormal (defect) patterns. After the difficult example memory bank is established, it is updated in real time during the training process according to a preset strategy. The updating process is as follows: every several epochs, the current student network is used to infer part of the training samples, and the features extracted by the student are compared with the normal feature bank prototypes. If it is found that the features of some normal samples exceed the threshold distance from any normal prototype in the student representation, these features are considered as new difficult example normal patterns and added to the normal feature bank to expand the representation range of the normal patterns. At the same time, new pseudo-defect samples are generated regularly, and the features extracted by the teacher network are added to the abnormal feature bank to make the abnormal patterns more diverse. Through the construction and rolling update of the difficult example memory bank, rich and discriminative normal and abnormal feature references are always obtained during the model training process, which is beneficial to further improve the model's ability to distinguish abnormalities.

[0065] Finally, a contrast learning mechanism is introduced: a contrast learning strategy is introduced based on the difficult example memory bank to further optimize the feature space of the student network to make the normal and abnormal feature distributions more separable.

[0066] Specifically, for each training batch, the normal sample features F S (x) extracted by the student encoder are compared with the samples in the memory bank: a feature vector is selected from the normal feature bank as the “positive sample” v n (For example, the closest normal prototype to F S (x) is selected), and a feature vector is selected from the abnormal feature bank as the “negative sample” v a (For example, the most dissimilar abnormal feature to F S (x) is selected). And by constructing a triplet loss, the student features are required to be closer to the positive sample and farther from the negative sample: ||F S (x)-v n || 2 +δ<||F S (x)-v a || 2 , if not, a loss is generated and is optimized in the reverse direction to ensure that the distance between the student features and the normal prototype is significantly smaller than the distance between the student features and the abnormal features.

[0067] The goal of contrast learning is to widen the distance between normal and abnormal features in the representation space. Specifically, the invention further introduces a contrast learning mechanism in the model training to optimize the feature representation of difficult samples. The contrast loss maximizes the similarity of samples of the same class and minimizes the similarity of samples of different classes, further improving the discrimination ability of the model. The specific formula is as follows:

[0068]

[0069] where s is the feature of the student network, t + is the positive sample feature from the memory bank, is the negative sample feature. After introducing the contrastive learning, the contrastive loss is denoted as L con and added to the total loss of the student network, and the total loss of the student network is updated as L total = L hard + aL p + bL rec + gL con After this stage of optimization, the feature distribution extracted by the student encoder will present clearer cluster separation, so that the normal sample features will be closer to the normal memory bank prototype, and any feature containing abnormal patterns will be pushed away to the direction of abnormal feature distribution, thereby greatly improving the separability and recognition accuracy of the model for abnormal samples.

[0070] S105, obtaining a normal zipper image fine-tuned text image generation model, and automatically generating defect descriptions combined with a large language model to construct a diversified pseudo-defect sample set, analyzing the output distribution of the trained defect detection model based on the pseudo-defect sample set to obtain a discrimination threshold.

[0071] That is, the Stable Diffusion model is fine-tuned using the available normal zipper image, combined with the large language model to automatically generate defect descriptions, and a diversified pseudo-defect sample set is constructed by a structure-controlled image synthesis method, and the output distribution of the unsupervised detection model is analyzed based on the sample set to automatically estimate the optimal discrimination threshold, thereby improving the adaptability of the method.

[0072] As an embodiment, a normal zipper image fine-tuned text image generation model is obtained, and defect descriptions are automatically generated combined with a large language model to construct a diversified pseudo-defect sample set, and the output distribution of the trained defect detection model is analyzed based on the pseudo-defect sample set to obtain a discrimination threshold, including: when fine-tuning the text image generation model, registering a special token in the text encoder to realize semantic guidance of specific subject parts; when generating defect descriptions, constructing a defect text description template, and using a large language model to generate a variety of template defect description sentences conforming to physical laws; when generating defect images, based on the template defect description sentence and the subject mask corresponding to the description sentence, jointly constrain the defect generation process; when estimating the threshold, use pseudo-defect samples of different defect types and different severity to form a simulation test set together with normal samples, and the threshold corresponding to the maximum F1 score of the defect detection model on the simulation test set is set as the discrimination threshold during inference.

[0073] As an embodiment, a structure preservation mechanism, a frequency domain disturbance constraint, and a latent space disturbance strategy are introduced during defect image generation. The structure preservation mechanism restricts defect injection to only occur in the designated zipper body area specified by the mask. The frequency domain disturbance constraint applies a loss function based on frequency characteristics in the latent space of the text image generation model to limit the deviation of the synthesized image from the original image in the frequency domain. The latent space disturbance strategy uses differentiated latent variable manipulation methods based on different defect types and visual features, including directional offset for text representation, directional disturbance of initial noise, replacement and guidance of local attention areas, to guide the text image generation model to generate defect areas with specific semantic features and spatial distribution in the latent space.

[0074] As a specific embodiment, first, the generation model is fine-tuned: 50 normal zipper image samples of different types are collected, and the pre-trained Stable Diffusion v2.1 model is fine-tuned using DreamBooth to enable it to learn the distribution rules of zipper structures and textures. During the fine-tuning process, exclusive tokens such as "[ZIP-T]" (corresponding to the zipper teeth) and "[ZIP-F]" (corresponding to the fabric tape) are registered in the text encoder to achieve semantic guidance for specific parts. Structural constraints are applied to the UNet attention layer to separate material information and structural parts, enhancing controllability and structural consistency in the generation process.

[0075] Next, defect description generation is performed: a large language model (such as GPT-4) is used to construct standardized defect text description templates, and multiple defect description sentences conforming to physical laws are automatically generated based on these templates. The template structure includes six fields: material, main body, defect type, defect degree, defect location, and defect size. The defect type includes but is not limited to tooth loss, tooth deformation, tooth burr, fabric tape tearing, etc. The defect degree includes three levels: light, medium, and severe. The defect location can include random, concentrated, and edge areas. The size controls the proportion of the defect area in the image. By combining information in multiple dimensions, defect text descriptions with diversity and semantic clarity can be automatically generated, such as "fabric tape light tear, location near edge, area about 5% of image" or "metal zipper tooth middle has obvious missing teeth".

[0076] Then, the defect image generation is performed: based on the template defect description obtained in the defect description generation step and the mask of the corresponding main body (tooth area or tape area) obtained in the zipper structure segmentation step, the defect generation process of the model obtained in the text-guided and mask-controlled joint constraint model fine-tuning step is controlled to realize the synthesis of the defect image with structural consistency and semantic accuracy. Further, the structural preservation mechanism, frequency domain disturbance constraint and latent space disturbance strategy are introduced to enhance the authenticity and physical rationality of the generated defects. The structural preservation mechanism only allows defect injection to occur in the main body area of the zipper specified by the mask, maintains the background information and overall structure of the image unchanged, ensures the consistency of the non-defect area, and improves the discriminability and synthesis quality of the image. The frequency domain disturbance constraint, i.e. applying a loss function based on frequency features in the latent space of the Stable Diffusion model, limits the deviation of the synthesized image from the original image in the frequency domain, so that the defect has natural texture transition and physical continuity, avoiding artifacts such as edge breakage or texture mutation. The latent space disturbance strategy uses differentiated latent variable manipulation methods according to different defect types and visual features, including but not limited to directional offset of text representation, directed disturbance of initial noise, replacement and guidance of local attention area, so as to guide the model to generate defect areas with specific semantic features and spatial distribution in the latent space. This strategy not only improves the consistency and diversity of defect appearance, but also enables more fine-grained defect morphology control for different materials and main body areas of the zipper (such as teeth or tape), enhancing the usability and generalization ability of the synthesized image.

[0077] Finally, threshold estimation is performed: using the pseudo-defect samples of different defect types and different severity levels synthesized in the defect generation step, together with normal samples, to form a simulated test set. Using the score distribution output by the current unsupervised detection model on the test set, the precision and recall values under different thresholds are calculated, and the final discrimination threshold is determined based on the F1 score maximization principle, so as to obtain an adaptive abnormality judgment standard. In this way, a representative pseudo-defect dataset can be constructed without introducing real defect samples, a more reasonable model discrimination threshold can be estimated, and the stability and robustness of the unsupervised defect detection method in actual application can be improved.

[0078] S106, obtaining a gray zipper image to be detected and inputting it into the trained defect detection model to obtain an output result, and determining whether the detected gray zipper image has defects according to the output result and the judgment threshold.

[0079] As an embodiment, the gray scale zipper image to be detected is obtained and input into the trained defect detection model to obtain an output result, and whether the detected gray scale zipper image has defects is determined according to the output result and a judgment threshold, including: inputting the gray scale zipper image to be detected into the teacher network and the student network of the trained defect detection model to obtain corresponding features; comparing the corresponding features of the teacher network and the corresponding features of the student network to obtain differences at the Patch level, and obtaining a difference image of the structure of the gray scale zipper image to be detected; performing feature matching detection on the gray scale zipper image to be detected and the difficult example memory library to obtain a memory library matching image; fusing the difference image and the memory library matching image to obtain an abnormal score image; using a Gaussian filter to perform smoothing processing on the abnormal score image, and comparing the processed abnormal score image with a judgment threshold, and determining that the abnormal score image is abnormal when the abnormal score image is greater than the judgment threshold, and determining that the abnormal score image is normal when the abnormal score image is less than the judgment threshold, so as to obtain a binary image of normal and abnormal and a final detection result.

[0080] That is, the detection image is respectively sent into the teacher and student models after training to obtain respective features, differences at the Patch level are obtained by comparing the features between the two, and a detection result of the structure of the detection image is obtained; at the same time, the detection image also performs feature matching detection on the difficult example feature library, so as to have better performance on some small and difficult-to-detect abnormalities. After the above comparison and detection, an abnormal score image in the inference stage can be obtained, and then a Gaussian filter is used to perform smoothing processing on the abnormal score image to improve the continuity and stability of the abnormal area. The processed abnormal score image is compared with a judgment threshold, and it is determined that the abnormal score image is abnormal when the abnormal score image is greater than the threshold, and it is determined that the abnormal score image is normal when the abnormal score image is less than the threshold, so as to obtain a binary image of normal and abnormal and a final detection result. In addition, the abnormal score image is normalized to generate an abnormal heat map, so as to facilitate the user to quickly locate the abnormal position.

[0081] In summary, according to the lightweight robust unsupervised zipper defect detection method of the embodiment of the present application, first, the zipper image dataset is obtained; then, each zipper image in the zipper image dataset is preprocessed to obtain a grayscale zipper image and corresponding tooth mask, tape mask and background mask; then, the grayscale zipper image is subjected to geometric data enhancement, and the grayscale zipper image is subjected to brightness data enhancement according to the tooth mask and the tape mask to obtain an enhanced grayscale image dataset; then, a defect detection model is constructed, the defect detection model includes a pre-trained teacher network and a student network, and the enhanced grayscale image dataset is input into the defect detection model to train the student network to learn to align the teacher feature to extract normal sample features and to perform image reconstruction, wherein, in the training process, a hard example learning mechanism is introduced, the most difficult to align features are focused and the constraint of abnormal patterns is increased, and a dynamic hard example memory is updated to store normal and abnormal sample features, and the student network feature space is optimized through contrast learning to widen the feature difference between normal and abnormal samples; then, a normal zipper image fine-tuning text image generation model is obtained, and a defect description is automatically generated in combination with a large language model to construct a diversified pseudo defect sample set, the output distribution of the trained defect detection model is analyzed according to the pseudo defect sample set to obtain a judgment threshold; finally, a grayscale zipper image to be detected is obtained and input into the trained defect detection model to obtain an output result, and it is determined whether the detected grayscale zipper image has defects according to the output result and the judgment threshold; thus, without real defect samples, the training of the detection model and the adaptive estimation of the judgment threshold can be realized, effectively alleviating the high dependence on real defect data in the model training stage or the threshold setting process before inference; the grayscale processing operation and the structure decoupled illumination enhancement algorithm enable the detection model to have strong generalization ability for zippers with different tooth and tape color combinations. For zipper images with various color combinations, the method can effectively reduce the number of model training times and data dependence, improve the training efficiency and practical application adaptability; the hard example memory library technology realizes the dynamic updating of hard example samples during training, so that the model can always obtain rich normal features and abnormal features during the training process, which is conducive to further improving the discrimination ability of the model to abnormal features; by introducing the contrast learning mechanism, a new loss function is constructed, the separability of the student network feature space is optimized, the distance between the normal sample and the abnormal sample in the feature representation space is more obvious, and thus the discrimination ability and detection accuracy of the model are improved.

[0082] In addition, the present application also provides a computer readable storage medium having a lightweight robust unsupervised zipper defect detection program stored thereon, and the lightweight robust unsupervised zipper defect detection program is executed by a processor to implement the lightweight robust unsupervised zipper defect detection method as described above.

[0083] In addition, the embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the light-weight robust unsupervised zip defect detection method is realized.

[0084] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0085] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0086] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0087] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0088] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0089] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0090] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0091] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0092] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0093] In the present application, unless otherwise explicitly specified and limited, a first feature is "on" or "under" a second feature can mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature is "over", "above" and "on top of" the second feature can mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is horizontally higher than the second feature. The first feature is "under", "below" and "underneath" the second feature can mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is horizontally lower than the second feature.

[0094] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0095] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A lightweight robust unsupervised chain defect detection method characterized in that, The method comprises the following steps: obtaining a zip image dataset; preprocessing each zip image in the zip image dataset to obtain a grayscale zip image and corresponding tooth mask, tape mask and background mask; performing geometric data enhancement on the grayscale zip image, and performing brightness data enhancement on the grayscale zip image according to the tooth mask and the tape mask to obtain an enhanced grayscale image dataset; constructing a defect detection model, the defect detection model comprising a pre-trained teacher network and a student network, inputting the enhanced grayscale image dataset into the defect detection model to train the student network to learn to align the teacher feature and extract normal sample features, and perform image reconstruction, wherein, in the training process, a hard example learning mechanism is introduced to focus on the most difficult to align features and increase the constraint of abnormal patterns, and a dynamic update hard example memory is used to store the features of normal and abnormal samples, and the student network feature space is optimized through contrastive learning to enlarge the feature difference between normal and abnormal samples; obtaining a normal zip image fine-tuning text image generation model, and automatically generating defect descriptions combined with a large language model to construct a diversified pseudo defect sample set, analyzing the output distribution of the trained defect detection model according to the pseudo defect sample set to obtain a judgment threshold; obtaining a grayscale zip image to be detected and inputting it into the trained defect detection model to obtain an output result, and determining whether the detected grayscale zip image has defects according to the output result and the judgment threshold.

2. The lightweight robust unsupervised tab defect detection method of claim 1, wherein, The preprocessing of each zip image in the zip image dataset to obtain a grayscale zip image and corresponding tooth mask, tape mask and background mask comprises: separating the illumination component and the reflection component of each zip image in the zip image dataset, and enhancing them to obtain an RGB image with uniform brightness; converting the RGB image with uniform brightness into a grayscale zip image, and inputting it into a pre-trained segmentation model to obtain the tooth mask, tape mask and background mask corresponding to each grayscale zip image.

3. The lightweight robust unsupervised tab defect detection method of claim 2, wherein, The preprocessing of each zip image in the zip image dataset to obtain a grayscale zip image and corresponding tooth mask, tape mask and background mask comprises: performing multi-scale Retinex decomposition on the zip image to separate it into a reflection component and an illumination component; performing equalization adjustment on the illumination component; recombining the adjusted illumination component and the reflection component to obtain an RGB image with uniform brightness.

4. The lightweight robust unsupervised tab defect detection method of claim 1, wherein, The brightness data enhancement of the grayscale zip image according to the structural decoupling comprises: I out (x,y) = I in (x,y)γ where I in and I out are the pixel values before and after adjustment, respectively, and γ is a random brightness adjustment coefficient generated based on the Gamma correction principle. When γ < 1, the image becomes brighter, and when γ > 1, the image becomes darker. Different γ values are used for brightness enhancement of the tooth region and the cloth belt region by using the tooth mask and the cloth belt mask, respectively, to obtain a gray-scale image with different color combinations.

5. The lightweight robust unsupervised tab defect detection method as claimed in claim 1, wherein, The training process comprises the following steps: L total = L hard + αL p + βL rec + γL con where L total is the total loss of the student network, L hard is the hard feature loss, L p is the penalty loss, L rec is the reconstruction loss, L con is the contrastive loss, the feature representation of the encoder output of the student network is F S , the high-dimensional feature representation of the teacher network output is F T , and the reconstructed image representation of the decoder output of the student network is N represents the number of input dataset images, the nth input image is represented as x n , Ω is the selected "hard feature" index set, C' x H' x W' is the dimension of the student encoder output feature, represents the high-dimensional feature at the cth channel, position (i, j), represents the feature at the cth channel, position (i, j), and the feature map obtained only by inputting the student network encoder is represented as F S (x'), x' represents the penalty sample, s is the feature of the student network, t + is the positive sample feature from the memory bank, is the negative sample feature.

6. The lightweight robust unsupervised tab defect detection method of claim 1, wherein, obtaining a normal zip image fine-tuning text image generation model, and automatically generating defect descriptions combined with a large language model to construct a diversified pseudo defect sample set, analyzing the output distribution of the trained defect detection model according to the pseudo defect sample set to obtain a judgment threshold, comprising: registering a special token in the text encoder when fine-tuning the text image generation model to realize semantic guidance of specific body parts; In the defect description generation, a defect text description template is constructed, and a large language model is used to generate a variety of templated defect description sentences conforming to physical laws; In the defect image generation, the templated defect description sentences and the subject mask corresponding to the description sentences are used to jointly constrain the generation of the defect; In the threshold estimation, pseudo-defect samples of different defect types and different severity levels are used together with normal samples to form a simulation test set, and the threshold corresponding to the maximum F1 score of the defect detection model on the simulation test set is set as the discrimination threshold during inference.

7. The lightweight robust unsupervised tab defect detection method of claim 6, wherein, In the defect image generation, a structure preservation mechanism, a frequency domain disturbance constraint, and a latent space disturbance strategy are introduced. The structure preservation mechanism constrains the defect injection to occur only in the zipper subject area specified by the mask. The frequency domain disturbance constraint applies a loss function based on frequency characteristics in the latent space of the text image generation model to limit the deviation of the synthesized image from the original image in the frequency domain. The latent space disturbance strategy uses differentiated latent variable manipulation methods, including directional offset of text representation, directional disturbance of initial noise, replacement and guidance of local attention area, to guide the text image generation model to generate defect areas with specific semantic features and spatial distribution in the latent space.

8. The lightweight robust unsupervised tab defect detection method as claimed in claim 1, wherein, The gray zipper image to be detected is input into the trained defect detection model to obtain an output result, and whether the detected gray zipper image has defects is determined according to the output result and the judgment threshold, including: The gray zipper image to be detected is input into the teacher network and student network of the trained defect detection model to obtain corresponding features; The features corresponding to the teacher network and the features corresponding to the student network are compared to obtain a difference graph in the structure of the gray zipper image to be detected; The gray zipper image to be detected is matched with the feature of the difficult example memory bank to obtain a memory bank matching graph; The difference graph and the memory bank matching graph are fused to obtain an anomaly score graph; The anomaly score graph is smoothed using a Gaussian filter, and the processed anomaly score graph is compared with the discrimination threshold. If the anomaly score graph is greater than the discrimination threshold, it is determined to be abnormal, and if the anomaly score graph is less than the discrimination threshold, it is determined to be normal, thereby obtaining a binary graph of normal and abnormal and a final detection result.

9. A computer-readable storage medium, characterized in that, A lightweight robust unsupervised zipper defect detection program is stored thereon, and the lightweight robust unsupervised zipper defect detection program is executed by a processor to implement the lightweight robust unsupervised zipper defect detection method of any one of claims 1-8.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the lightweight robust unsupervised zipper defect detection method of any one of claims 1-8.

Images