Defect feature identification method based on clothing visual image and quality inspection system

Abstract

The invention discloses a defect feature recognition method based on a clothing visual image and a quality inspection system, and aims to solve the problems that traditional manual quality inspection is low in efficiency and poor in precision and an existing automatic system is insufficient in robustness and generalization ability. The method comprises the following steps: after image preprocessing, segmenting by using an SAM network to obtain a clothes main body and an initial flaw; constructing multi-scale features by the FPN, inputting the multi-scale features into the improved YOLOv11 network, and outputting an initial candidate box; correcting the candidate frame according to IoU / Dice to obtain a refined frame; extracting texture and morphological characteristics and calculating abnormal scores; carrying out weighted fusion on multiple indexes to obtain a comprehensive confidence coefficient to identify flaws; and carrying out secondary training to update the model after manually annotating the low-confidence sample. The system comprises eight corresponding modules, realizes full-process automatic detection, can significantly improve the precision, robustness and automation level of garment defect identification, and has important practical application value.

Description

Technical Field

[0001] This invention belongs to the field of garment quality inspection and image processing technology, specifically, it relates to a method and quality inspection system for identifying defects in garment visual images. Background Technology

[0002] With the rapid development of the garment industry, product quality control has become one of the core competencies of enterprises. Traditional garment quality inspection mainly relies on manual visual inspection, which suffers from problems such as low efficiency, high susceptibility of inspection accuracy to subjective factors, high labor intensity, and high cost. Especially when dealing with small, hidden, or complex defects, manual inspection has a high rate of missed detection and false detection, making it difficult to meet the demands of modern production for high-efficiency and high-precision quality inspection.

[0003] In recent years, with the rapid development of computer vision and deep learning technologies, machine vision-based automated quality inspection systems have been gradually applied in the apparel industry. Existing automated inspection systems typically employ image processing techniques combined with machine learning or deep learning models for defect identification. However, these systems still face many challenges: First, the diversity and complexity of defects are significant issues. Clothing defects are numerous, including but not limited to loose threads, stains, tears, weaving defects, and sewing defects, with significant differences in shape, size, and texture, making unified and efficient identification difficult. Second, background interference and lighting variations have a significant impact. Clothing images have complex backgrounds, and in actual production environments, lighting conditions may be uneven or changeable, making defect features less obvious and easily masked by background noise, affecting detection accuracy and robustness. Third, the ability to detect small defects is limited. Many key defects (such as small loose threads and tiny holes) are small in size and occupy a very small proportion in the image. Existing target detection algorithms have limited ability to identify small targets and are prone to missed detections. Fourth, the localization accuracy is insufficient. Existing detection methods typically output rectangular bounding boxes. For irregularly shaped defects, the bounding box may contain a large number of non-defect areas, resulting in insufficient localization and affecting subsequent defect analysis and repair. Fifth, the model's generalization ability is poor. Existing models often rely on large amounts of labeled data for training, but obtaining real defect samples is difficult and the types are unbalanced. When faced with novel defects or clothing of different materials and styles that have not been trained on before, the model lacks generalization ability and is prone to false positives or false negatives. Finally, it lacks an adaptive optimization mechanism. After deployment, most systems rely on manual data collection, labeling, and model retraining for performance optimization, lacking a mechanism that can continuously learn and self-optimize based on actual detection feedback.

[0004] Therefore, developing an advanced method and system that can significantly improve the accuracy, robustness, and automation level of garment defect identification, and has adaptive optimization capabilities, is a key technical problem that urgently needs to be solved in the field of intelligent garment manufacturing. Summary of the Invention

[0005] In view of this, the technical problem to be solved by the present invention is to provide a method and quality inspection system for identifying defects in clothing based on visual images, which aims to overcome the problems of low efficiency and poor accuracy of manual inspection in the prior art, as well as the insufficient robustness and limited generalization ability of automated inspection systems, and to achieve high-precision, high-robustness, and automated identification and evaluation of clothing defects.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for identifying defects in clothing based on visual images includes the following steps: Step S1: Image Preprocessing and Clothing Mask Segmentation. First, the input clothing image is preprocessed, including but not limited to noise reduction, color correction, and size normalization, to improve image quality and the stability of subsequent processing. Next, a pre-trained segmentation network, SAM (Segment All), is used to perform precise mask segmentation of the clothing region in the preprocessed image. Through the clothing mask, background interference can be effectively removed, limiting the detection range to the main clothing area, and a preliminary set of potential clothing defects (i.e., pixel-level masks of the clothing region) is obtained.

[0007] Step S2: Multi-scale Feature Construction and Initial Defect Detection. Based on clothing mask segmentation, multi-scale features are constructed to extract multi-scale information from feature maps of different resolutions to adapt to the detection needs of defects of different sizes. These multi-scale features are input into an improved YOLOv11 detection network, and structural optimization is performed for small targets (no higher than 20×20 pixels) and irregularly shaped defects, introducing an attention mechanism and improving the anchor box generation strategy. The detection network outputs preliminary defect candidate boxes and their corresponding defect category prediction results (e.g., loose threads, stains, holes, etc.).

[0008] Step S3: Fusion and Correction of Defect Candidate Boxes. To improve the accuracy of defect localization, this step performs mask-box matching analysis based on the candidate defect mask set obtained in Step S1 and the initial defect candidate boxes output in Step S2. The matching degree can be calculated based on metrics such as IoU (Intersection over Union) or the Dice coefficient. For initial defect candidate boxes with high matching degrees, the initial defect candidate boxes are fused and corrected by combining their corresponding pixel-level mask information. For example, the precise contour information of the mask can be used to adjust the size and position of the bounding box, making it more closely fit the true shape of the defect, thereby obtaining a refined defect box. This step effectively solves the problem of inaccurate traditional bounding box localization.

[0009] Step S4: Multidimensional Feature Extraction and Anomaly Score Calculation. For each refined defect box obtained in Step S3, multidimensional features are extracted from the corresponding local region (i.e., the defect region). These features include, but are not limited to: texture features, such as Gray-Level Co-occurrence Matrix (GLCM), Gabor filter response, Local Binary Pattern (LBP), etc., used to describe the texture anomalies of the defect region; and morphological similarity features, such as those based on Shape Context, Hu invariant moments, or Fourier descriptors, used to describe the difference in shape between the defect region and the normal region or the similarity to a known defect template. Based on these multidimensional features, an anomaly score is calculated for each defect region. The anomaly score can quantify the degree of anomaly using statistical methods (such as Gaussian Mixture Model, One-Class SVM) or deep feature-based anomaly detection models (such as autoencoders, GANs). A higher score indicates a more significant or more abnormal defect.

[0010] Step S5: Weighted Fusion of Overall Confidence and Final Defect Recognition. The detection confidence score output from Step S2 (the confidence score of the YOLOv11 network for candidate boxes), the mask consistency score calculated in Step S3 (the matching degree between the refined defect box and the mask), and the anomaly score calculated in Step S4 are weighted and fused to obtain the overall confidence score for each refined defect box. The weights of the weighted fusion can be adjusted according to actual application requirements and the importance of different features. Based on the overall confidence score, defect type classification (using the YOLOv11 network or an independent classifier) ​​and confidence evaluation are performed on the retained defect boxes to obtain the final defect recognition result, including defect type, precise location (refined defect box), and overall confidence score.

[0011] Step S6: Secondary Training and Model Update for Low-Confidence Samples. To achieve continuous optimization and adaptive capability of the model, this invention introduces a closed-loop feedback mechanism. For low-confidence samples (overall confidence below a preset threshold) or samples with inconsistent labeling results identified in step S5 (e.g., samples where model predictions differ from human experience judgments), a set to be labeled is constructed. Samples in this set are submitted to humans for confirmation and accurate labeling. High-quality labeled data after human confirmation is used for secondary training to update the segmentation network in step S1 and the detection network in step S2. In this way, the model can continuously learn new defect features, improve its ability to identify and generalize complex and diverse defects, and form a continuously iteratively optimized intelligent quality inspection system.

[0012] In addition, the present invention also provides a quality inspection system based on a method for identifying defects in clothing visual images, comprising: The image acquisition module, including an industrial camera and light source, is used to acquire high-resolution images of the garment to be inspected. The image preprocessing module includes a noise reduction unit, a color correction unit, and a size normalization unit, used to preprocess the clothing image; The clothing mask segmentation module integrates a semantic segmentation network based on the SAM architecture to perform pixel-level mask segmentation of clothing areas in preprocessed images, outputting the main clothing mask and an initial set of defects; it achieves high-precision pixel-level segmentation of clothing areas and adapts to different clothing types and complex backgrounds through parameter adjustment; The initial defect detection module integrates the Feature Pyramid Network (FPN) and an improved YOLOv11 detection network containing PANet and attention mechanisms. It is used to construct multi-scale features and output initial defect candidate boxes, category prediction results, and detection confidence. This improves the detection capability for multi-scale and small-target defects and enhances detection robustness. The defect box fusion and correction module is used to calculate the IoU or Dice coefficient between the initial defect candidate box and the initial defect set mask, and correct the candidate box based on the mask contour with a matching degree ≥0.7 to generate the refined defect box and mask consistency; thus achieving accurate correction of defect position. The multidimensional feature extraction and anomaly scoring module includes a texture feature extraction unit, a morphological similarity feature extraction unit, and an anomaly scoring calculation unit; effectively improving the ability to identify complex and novel defects. The comprehensive confidence assessment module, including a weighted fusion unit, calculates the comprehensive confidence score using the formula: Comprehensive Confidence Score = w1 × Detection Confidence Score + w2 × Mask Consistency Score + w3 × Anomaly Score, and outputs the final defect identification result; achieving high-precision defect identification and confidence assessment. The model adaptive optimization module includes a unit for constructing the set of objects to be labeled, a manual confirmation interface, and a secondary training unit, enabling continuous optimization and improvement of the system's generalization ability.

[0013] As a preferred embodiment of the present invention, the light source of the image acquisition module is a backlight or a uniformly diffused light source to avoid image reflection and shadows; the loss function of the improved YOLOv11 detection network adopts GIoU Loss or CIoULoss as the bounding box regression loss.

[0014] Compared with the prior art, the present invention can achieve the following technical effects: 1) Significantly improve recognition accuracy. By introducing clothing mask segmentation, defect box fusion correction, and multi-dimensional feature extraction and anomaly scoring mechanisms, pixel-level precise segmentation and more accurate localization of defects are achieved, effectively reducing false detection and false negative rates.

[0015] 2) Enhanced system robustness. The fusion of multi-dimensional features (texture, morphology) and the weighted fusion of multiple confidence levels (detection confidence, mask consistency, anomaly score) enable the system to more comprehensively evaluate defects and effectively cope with the challenges posed by complex backgrounds, lighting variations, and diverse defect types.

[0016] 3) Improve automation. The entire process from image acquisition to defect identification, location, classification and evaluation has been automated, which has greatly reduced the reliance on manual quality inspection and improved production efficiency.

[0017] 4) Possesses adaptive optimization capabilities. A closed-loop feedback mechanism for secondary training using low-confidence samples is introduced, enabling the system to continuously learn and optimize itself based on actual production data, thereby continuously improving the model's generalization ability and its ability to identify novel defects, solving the problem of poor generalization ability in traditional models.

[0018] 5) Reduced production costs. Automated quality inspection reduces labor costs, improves inspection efficiency, and reduces the risk of recalls and compensation due to defective products entering the market, thereby reducing overall production costs.

[0019] 6) High scalability. The modular system design allows each module to be upgraded and optimized independently, facilitating the introduction of new segmentation networks, detection networks, or feature extraction methods, and exhibits excellent scalability.

[0020] Of course, any product implementing this invention does not necessarily need to achieve all of the technical effects described above at the same time. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic diagram of the process for identifying defects in clothing based on visual images as described in this invention; Figure 2 This is a schematic diagram of the defect candidate box fusion and correction process described in this invention; Figure 3 This is a schematic diagram of the low-confidence sample secondary training and model update process described in this invention; Figure 4 This is a schematic diagram of the quality inspection system based on the method for identifying defects in clothing visual images as described in this invention. Detailed Implementation

[0022] The following will describe in detail the implementation of the present invention with reference to the accompanying drawings and embodiments, so that the process of how the present invention uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0023] The hardware configuration of the system of this invention is as follows: it can be deployed on a high-performance computing platform, the image acquisition device is an industrial-grade high-resolution camera equipped with a suitable light source, and the storage device includes a high-speed SSD and a large-capacity HDD.

[0024] The software environment for this invention is as follows: the system supports the Linux operating system, uses the mainstream deep learning framework PyTorch, and the programming language is Python.

[0025] This invention provides a method for identifying defects based on visual images of clothing, such as... Figure 1 As shown, the main steps include: Step S1: Image preprocessing and clothing mask segmentation Image preprocessing: Three preprocessing operations are performed on the raw clothing images captured by the industrial camera: Denoising: Gaussian filtering (with a kernel size of 5×5, dynamically adjusted according to the image noise intensity) is used to remove high-frequency random noise, and median filtering (with a window size of 3×3) is used to eliminate salt-and-pepper noise, thus avoiding noise interference with subsequent defect feature extraction. Color correction: By adjusting the white balance (using the gray world algorithm or perfect reflection algorithm), the color space of the image is unified, eliminating the color deviation of clothing under different lighting conditions, and ensuring that the same type of defect presents consistent color characteristics in different images; Size normalization: All preprocessed images are uniformly scaled to a fixed resolution (640×640 pixels) to adapt to the input requirements of subsequent deep learning networks, while reducing the impact of image size differences on detection accuracy.

[0026] Clothing Mask Segmentation: A pre-trained SAM (Segment Anything Model) network is used to perform pixel-level semantic segmentation on the preprocessed image. By fine-tuning the SAM network parameters, optimizing only the last two layers with a learning rate of 0.0001 and 200 iterations, the network is adapted to different clothing types (e.g., dark jeans, light-colored T-shirts, fleece jackets) and complex backgrounds (e.g., dark conveyor belts, multi-colored workshop backgrounds). This accurately segments the main clothing area and the background area, generating a binary mask image (with pixel values ​​of 1 for the clothing area and 0 for the background area). Simultaneously, based on the grayscale and texture differences of pixels within the main clothing area, a preliminary set of potential clothing defects (i.e., pixel masks within the clothing area that significantly differ from normal fabric features) is extracted, completely eliminating background interference and limiting subsequent detection to the main clothing area.

[0027] Step S2: Multi-scale feature construction and initial defect detection Multi-scale feature construction: Based on the clothing subject mask output in step S1, a feature pyramid network (FPN) is used to construct multi-scale feature maps. Specifically, the clothing image is input into the feature extraction network to extract feature maps at different levels (such as C2, C3, C4, and C5 feature layers, corresponding to different resolutions). Through top-down sampling and bottom-up feature fusion, multi-scale feature maps containing low-resolution, high semantic information (suitable for large-size defect detection) and high-resolution, high-detail information (suitable for small-size defect detection) are generated to ensure full coverage detection of defects of different sizes.

[0028] Initial defect detection network: Multi-scale feature maps are input into an improved YOLOv11 detection network, which performs three key optimizations on the original YOLOv11 architecture: Introducing Path Aggregation Network (PANet): The PANet structure is added to the Neck part of YOLOv11. Through lateral connections and top-down path aggregation, the fusion effect of low-level features and high-level features is enhanced, and the feature extraction capability for small object defects is improved. Add an attention mechanism module: After the PANet structure, at the end of the Neck part, i.e. after feature fusion, a convolutional attention module (CAM) and a spatial attention module (SAM) are embedded, so that the network automatically focuses on potential defect areas in clothing images, suppresses background noise interference, and improves detection robustness; Anchor box optimization: The K-means++ clustering algorithm is used to cluster the size and aspect ratio of the defect bounding boxes in the large-scale clothing defect training dataset (a total of different categories, about 10,000 labeled defect samples) to generate anchor boxes adapted to the features of clothing defects (such as 10×5 and 15×8 anchor boxes for loose threads, and 8×8 and 12×12 anchor boxes for holes), which replace the default anchor boxes of YOLOv11 and improve the recall rate of the initial detection.

[0029] Initial detection results output: After detecting the multi-scale feature map, the improved YOLOv11 detection network outputs the initial defect candidate box (represented by coordinates (x,y,w,h) for each potential defect, where x and y are the horizontal and vertical coordinates of the candidate box center point, and w and h are the width and height of the candidate box), defect category prediction results (such as "thread end", "oil stain", "hole", "broken yarn", etc.), and detection confidence (value range 0-1, representing the probability assessment of the network that the candidate box is a real defect, the higher the value, the stronger the reliability).

[0030] Step S3: Fusing and correcting defect candidate boxes. Please refer to [link / reference]. Figure 2 .

[0031] Mask-Boundary Matching Degree Calculation: To improve defect localization accuracy, the matching degree between the pixel-level mask of the initial defect set output in step S1 and the initial defect candidate bounding box output in step S2 is calculated. Two quantification metrics are used for the matching degree: IoU (Intersection over Union): Calculates the ratio of the intersection area to the union area of ​​the initial defect candidate box and the corresponding defect mask. The formula is IoU=(A∩B) / (A∪B), where A is the candidate box region and B is the defect mask region. Dice coefficient: Calculates the similarity between the candidate bounding box and the mask. The formula is Dice = 2 × (A ∩ B) / (total number of pixels in A + total number of pixels in B). The two metrics can be selected based on the characteristics of the clothing material (e.g., for defects in plush fabrics, the Dice coefficient is preferred to avoid interference from fabric texture on IoU calculation).

[0032] Candidate box fusion and correction: Set a preset threshold for matching degree (based on multiple cross-validation experiments, the optimal value of this threshold is set to 0.7): If the matching degree between the initial defect candidate box and the corresponding defect mask is ≥0.7, it indicates that the candidate box has a high degree of overlap with the real defect area. At this time, the initial candidate box is corrected by combining the accurate contour information of the defect mask (such as obtaining the edge pixels of the mask through the contour extraction algorithm): if the defect is a regular shape (such as a circular hole), the size and position of the candidate box are adjusted by the minimum bounding rectangle of the mask; if the defect is an irregular shape (such as an irregular stain), a convex polygon bounding box that fits the contour of the mask is generated by the convex hull algorithm to replace the original rectangular candidate box, and finally the "refined defect box" is generated. If the matching degree is <0.7, it means that the initial candidate box deviates significantly from the actual defect area. The candidate box is discarded directly or marked as a "to be reviewed" sample, which will then be manually confirmed.

[0033] Mask consistency record: After the correction is completed, the final matching degree (IoU or Dice coefficient) between the refined defect box and the corresponding defect mask is calculated and defined as "mask consistency", which is used for subsequent comprehensive confidence assessment.

[0034] Step S4: Multidimensional Feature Extraction and Anomaly Score Calculation Local region extraction: For each refined defect box output in step S3, the local region corresponding to the box (i.e. the precise defect region) is cropped from the original clothing image and used as the object of feature extraction to avoid interference from non-defect region features on the analysis results.

[0035] Multidimensional feature extraction: Two types of core features are extracted from local defect areas to comprehensively describe the appearance and morphological characteristics of the defects. Texture features: Gray-level co-occurrence matrix (GLCM): In a local gray-level image, the co-occurrence probability of gray values ​​under different directions (0°, 45°, 90°, 135°) and different lengths (such as 1 pixel, 2 pixels) is calculated, and four statistical features are extracted from them: contrast (reflecting texture sharpness), energy (reflecting texture uniformity), homogeneity (reflecting texture smoothness), and correlation (reflecting texture directionality). Gabor filter response: A Gabor filter bank with 8 directions (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°) and 5 scales (wavelengths of 2, 4, 6, 8, and 10 pixels respectively) is used to filter the local area, and the response value of each filter is extracted to form a texture direction and frequency feature vector. Local Binary Pattern (LBP): The gray value of each pixel in a local region is compared with the gray values ​​of its eight neighboring pixels to generate an 8-bit binary code. The histogram distribution of the code is statistically analyzed as a feature of texture roughness.

[0036] Morphological similarity features: Shape Context Descriptor: Samples the contour points of the local defect area, calculates the relative polar coordinates (angle, distance) of each contour point with all other contour points, generates a shape histogram, and matches it with predefined typical defect shape templates (such as "thread end template" and "hole template") to obtain the shape similarity; Hu invariant moments: Calculate 7 Hu invariant moments for the local defect region (generated by combining second and third central moments). These moments are invariant to translation, rotation and scaling and can quantitatively describe the overall shape characteristics of the defect. Fourier descriptor: Perform a Fourier transform on the coordinate sequence of the defect contour, and take the first 10-20 low-frequency components as the Fourier descriptor to reflect the main shape information of the defect contour, while ignoring high-frequency detail noise.

[0037] Anomaly score calculation: Based on the extracted multidimensional features, two methods are used to calculate the "anomaly score" of the defective area (the score ranges from 0 to 1, with higher scores indicating a more significant difference between the defect and normal fabric, and a more severe defect): Statistical anomaly detection methods include: Gaussian Mixture Model (GMM): Models the multidimensional features of a large number of normal clothing fabric areas, constructs a GMM probability distribution model, and calculates the probability density of the defective area features in the model. The lower the probability density, the higher the anomaly score. One-Class SVM: The One-Class SVM model is trained using normal fabric features as positive samples. A classification hyperplane is constructed in the feature space, and the distance from the defective region features to the hyperplane is calculated. The greater the distance, the higher the anomaly score.

[0038] Anomaly detection models based on deep features include: Autoencoder: An autoencoder (consisting of an encoder and a decoder) is trained on a large number of normal clothing images, enabling the model to accurately reconstruct normal images. The defective region is input into the trained autoencoder, and the mean squared error (MSE) between the reconstructed image and the original defective region is calculated. The larger the reconstruction error, the higher the anomaly score. Generative Adversarial Networks (GANs): GAN models (consisting of a generator and a discriminator) are trained so that the generator can generate realistic images of normal fabric. Defective regions are input into the discriminator, which outputs the probability that the region is "abnormal," which is the anomalous score.

[0039] Step S5: Comprehensive confidence-weighted fusion and final defect identification Comprehensive Confidence Calculation: To comprehensively evaluate the reliability of the defect identification results, the "Detection Confidence" output from step S2, the "Mask Consistency" output from step S3, and the "Anomaly Score" output from step S4 are weighted and fused together. The calculation formula is as follows: Overall confidence score = w1 × detection confidence score + w2 × mask consistency score + w3 × anomaly score Where w1, w2, and w3 are weighting coefficients, satisfying w1 + w2 + w3 = 1. The weighting coefficients are determined as follows: Cross-validation optimization: On the validation set containing 50,000+ labeled defective samples, a grid search method is used to traverse different weight combinations (e.g., w1 takes values ​​of 0.3-0.5, w2 takes values ​​of 0.2-0.4, w3 takes values ​​of 0.2-0.4) and select the weight combination that maximizes the detection accuracy (F1 score). Dynamic adjustment: The weights are adjusted according to the characteristics of the clothing material. For example, for wool fabrics with complex textures, the weight of "anomaly score" is increased (w3 is increased to 0.4); for light-colored T-shirts with simple backgrounds, the weight of "detection confidence" is increased (w1 is increased to 0.5). Experiments have verified that the default optimal weight combination is w1=0.4, w2=0.3, w3=0.3.

[0040] Final defect screening and identification: Set a comprehensive confidence threshold (default is 0.6, which can be adjusted according to the company's quality control requirements; for example, for high-end clothing, the threshold can be increased to 0.7): If the overall confidence of a certain refined defect box is greater than or equal to the threshold, the box is determined to correspond to a real defect, and defect type classification is performed (if the confidence of the category prediction result in step S2 is low, an additional independent classifier (such as support vector machine) can be introduced to perform secondary classification on multi-dimensional features to improve the accuracy of category judgment). If the overall confidence level is less than the threshold, the box is judged as a "suspected defect" and added to the set to be labeled for subsequent manual review.

[0041] Final output: For samples determined to be true defects, output complete defect identification results, including: Types of defects: such as "loose threads", "oil stains", "holes", "broken yarn", etc.; Precise location: Coordinates of the defect frame (for rectangular frames, coordinates are (x, y, w, h); for irregular defects, coordinates are the vertex coordinates of polygons). Overall confidence level: A quantitative assessment of the reliability of the identification results; Anomaly score: reflects the severity of defects and provides a basis for quality grading (e.g., anomaly score ≥0.8 is judged as "serious defect", 0.5-0.8 is "general defect", and <0.5 is "minor defect").

[0042] Step S6: Secondary training and model update for low-confidence samples. Please refer to [link / reference]. Figure 3 .

[0043] Construction of the unlabeled set: To enable continuous model optimization, an "unlabeled set" is constructed, automatically collecting two types of samples: Low-confidence samples: suspected defective samples with a comprehensive confidence level less than the threshold in step S5, including areas where the model cannot determine whether they are defects; Inconsistent label samples: Samples where the system identification results differ from the manual review results, such as samples that the system judges as "defective" but are confirmed to be defective by humans (false alarm samples), samples that the system does not identify but are found to be defective by humans (missed alarm samples), and samples where the system's category prediction is incorrect.

[0044] Manual verification and precise annotation: The samples in the set to be annotated are presented to professional quality inspectors through a human-computer interaction interface (such as a web-based annotation platform), and the quality inspectors complete three operations: Defect Presence Confirmation: Determining whether there are actual defects in the sample; Defect type labeling: Clearly state the category of the actual defect (e.g., "dye residue" or "uneven stitches"); Precise boundary annotation: The pixel-level annotation tool Labelme is used to annotate the precise masks and bounding boxes of defects, ensuring the accuracy of the annotation data.

[0045] Secondary training data preprocessing: Preprocessing the manually labeled samples, including: Data cleaning: Remove invalid samples with incorrect labeling or blurry images; Data augmentation: The sample size is expanded by random flipping, rotation (0°-360°), scaling (0.8-1.2 times), brightness adjustment (±10%), and contrast adjustment (±10%) to avoid model overfitting; Data format conversion: Convert labeled data into a format suitable for training deep learning networks (e.g., convert mask labels to PNG format, and bounding box labels to XML or JSON format).

[0046] Secondary training and update of the model: The preprocessed high-quality labeled data is used as incremental training data to perform secondary training on the SAM segmentation network in step S1 and the improved YOLOv11 detection network in step S2. Segmentation network update: An incremental learning strategy is adopted, loading the pre-trained weights of the SAM network, inputting the newly labeled clothing mask sample into the network, and using the cross-entropy loss function as the optimization objective to fine-tune the network parameters (learning rate set to 0.0001, iteration 10-15 rounds) to improve the network's segmentation accuracy for new clothing materials and complex backgrounds. Detection network update: Incremental learning is also used. The existing weights of the improved YOLOv11 are loaded, and the newly labeled defect samples are input into the network. The optimization objectives are CIoU Loss (boundary box regression loss) + cross-entropy loss (class classification loss) + target existence loss. The network parameters are fine-tuned (learning rate is set to 0.0001, iterations are 15-20 times) to improve the network's ability to identify new defects and the detection accuracy. Model Deployment: After training, the updated model is deployed to the quality inspection system, replacing the original model and achieving iterative optimization of system performance. Through this closed-loop feedback mechanism, the model can be updated every 1-2 weeks, continuously adapting to changes in the production environment and new defect challenges.

[0047] Through this continuous "human-machine" collaborative learning mechanism, the system can continuously adapt to new production environments, new clothing styles, and new defect types, achieving continuous performance improvement and adaptive optimization.

[0048] In response, this invention also provides a quality inspection system based on a method for identifying defects in clothing visual images, such as... Figure 4 As shown, it includes: Module 1: Image Acquisition Module, comprising: an industrial-grade high-resolution camera (resolution ≥ 5 megapixels, such as Hikvision MV-CA050-10GM), a dedicated light source (using backlight or uniform diffused light, such as the CCS LDR2 series, with adjustable brightness range of 500-1500 lux), a camera bracket, and a conveyor belt synchronization triggering device. Specifically, on the garment production line, the camera's shooting rhythm is controlled by the conveyor belt synchronization triggering device (matched to the conveyor belt speed; for example, when the conveyor belt speed is 0.3 m / s, the shooting interval is set to 0.5 s), ensuring that the front, back, and key areas such as cuffs and collars of each garment are clearly captured. This yields high-definition, reflective, and shadow-free garment images, which are then transmitted in real-time via a high-speed data interface (USB 3.0) to the subsequent image preprocessing module, providing high-quality raw data for subsequent defect detection and segmentation.

[0049] Module 2: Image Preprocessing Module, consisting of three units: a denoising unit, a color correction unit, and a size normalization unit. The denoising unit uses Gaussian filtering and median filtering algorithms to remove noise from the acquired raw images, eliminating image noise caused by factors such as dust and equipment vibration in the production environment. The color correction unit uses grayscale world algorithm or perfect reflection algorithm to adjust the white balance of images captured under different lighting conditions, unifying the image color space and ensuring the consistency of blemish color features. The size normalization unit scales the image to a preset resolution (e.g., 640×640 pixels) and performs padding (filling the edges with black pixels when the image aspect ratio is inconsistent with the preset resolution) to adapt to the input requirements of subsequent deep learning networks. It also stores the original image size information for later mapping the detection results back to the original image coordinates.

[0050] Module 3: Clothing Mask Segmentation Module, based on a semantic segmentation network (SAM) architecture deployed on a GPU server and a parameter fine-tuning unit. This module loads the pre-trained SAM network model, performs pixel-level segmentation on the pre-processed image, and outputs binary masks of the main clothing area and the background area. The parameter fine-tuning unit adjusts the segmentation threshold of the SAM network according to the characteristics of different clothing types (e.g., dark clothing, light clothing, plush clothing) (e.g., a segmentation threshold of 0.5 for dark clothing and 0.4 for light clothing) to improve segmentation accuracy. Based on the pixel feature differences within the main clothing area, it initially extracts pixel-level masks of potential defect sets and transmits the segmentation results (clothing mask, initial defect mask) to the initial defect detection module and the defect box fusion and correction module.

[0051] Module 4: Initial Flaw Detection Module, which includes Feature Pyramid Network (FPN), Improved YOLOv11 Detection Network, Anchor Box Generation Unit, and Loss Function Calculation Unit; The FPN unit receives the main image of the clothing from the clothing mask segmentation module, extracts multi-scale feature maps, and provides feature support at different resolutions for subsequent detection. The improved YOLOv11 detection network integrates the PANet structure and attention mechanism module to perform defect detection on the multi-scale feature maps. The anchor box generation unit uses the K-means++ algorithm to cluster defect samples in the training set and generate anchor boxes that are adapted to clothing defects, replacing the default anchor boxes of YOLOv11. The loss function calculation unit uses GIoU Loss or CIoU Loss as the bounding box regression loss, and combines cross-entropy loss (class classification loss) and target existence loss to calculate the network training loss and guide the optimization of network parameters. The system outputs the initial defect candidate boxes, defect category prediction results, and detection confidence scores, which are then transmitted to the defect box fusion and correction module.

[0052] Module 5: Defective bounding box fusion and correction module, which includes a matching degree calculation unit, a candidate bounding box correction unit, and a mask consistency recording unit; The matching degree calculation unit receives the initial candidate box output by the initial defect detection module and the initial defect mask output by the clothing mask segmentation module, and calculates the IoU or Dice coefficient (matching degree) between the two. The candidate box correction unit corrects candidate boxes with a matching degree ≥ 0.7 based on the matching degree threshold (0.7) and combines them with the defect mask contour to generate refined defect boxes; for candidate boxes with a matching degree < 0.7, they are marked as "to be reviewed" or directly discarded. The mask consistency recording unit calculates the final matching degree (mask consistency) between the refined defect box and the corresponding defect mask, and transmits it to the comprehensive confidence evaluation module.

[0053] Module 6: Multidimensional feature extraction and anomaly scoring module, which includes a local region cropping unit, a texture feature extraction unit, a morphological similarity feature extraction unit, and an anomaly scoring calculation unit; The local area cropping unit cropped out the local area of ​​the defect from the original clothing image based on the coordinates of the refined defect box, and removed the interference of the non-defect area. The texture feature extraction unit extracts GLCM features, Gabor filter responses, and LBP features of local regions to form a texture feature vector. The morphological similarity feature extraction unit extracts the shape context descriptor, Hu invariant moments, and Fourier descriptors of local regions to form morphological feature vectors; The anomaly scoring unit calculates anomaly scores based on texture and morphological feature vectors, using statistical methods (GMM, One-Class SVM) or deep models (autoencoders, GANs), and then transmits the scores to the comprehensive confidence assessment module.

[0054] Module 7: Comprehensive confidence assessment module, including weight configuration unit, weighted fusion unit, defect screening and classification unit, and result output unit; The weight configuration unit provides a weight adjustment interface, which supports setting the values ​​of w1, w2, and w3 through cross-validation or human experience, and can be dynamically adjusted according to the clothing material. The weighted fusion unit receives the detection confidence, mask consistency, and anomaly score, and substitutes them into the formula to calculate the overall confidence. The defect screening and classification unit filters out real defect samples based on the comprehensive confidence threshold, performs secondary verification on the category prediction results (if necessary, calls an independent classifier), and determines the final defect type. The results output unit outputs information such as defect type, refined defect box location, overall confidence level, and anomaly score in JSON format or a visual interface, which is fed back to the results output and alarm module. At the same time, it transmits low-confidence samples to the model adaptive optimization module.

[0055] Module 8: Model Adaptive Optimization Module, including the unit for constructing the set of objects to be labeled, the manual labeling interface unit, the data preprocessing unit, and the secondary training unit; The unlabeled set construction unit collects low-confidence samples and samples with inconsistent labels to establish an unlabeled sample library; The manual annotation interface unit provides a web-based annotation platform that allows quality inspectors to identify defects, annotate types, and annotate bounding boxes on samples, and record the annotation results. The data preprocessing unit cleans, enhances, and converts the format of manually labeled samples to generate an incremental training dataset. The secondary training unit loads the existing weights of the segmentation and detection networks, uses the incremental dataset to fine-tune the model, and updates the model parameters in the system after training to achieve continuous model optimization.

[0056] This invention's system can be applied to visual quality inspection of various clothing products, such as T-shirts, shirts, jeans, and jackets. The workflow is as follows: First, high-resolution images of the garment to be inspected are acquired using image acquisition modules such as industrial cameras. These images are then fed into an image preprocessing module for denoising and size normalization to eliminate environmental interference and standardize the image format, providing high-quality input for subsequent processing. The preprocessed image then enters the garment mask segmentation module, where a deep learning segmentation network performs pixel-level analysis to accurately segment the main garment region and generate a garment mask. Simultaneously, it initially identifies a set of potential defects within the garment region. The segmented image or its features are then sent to the initial defect detection module, which uses an improved YOLOv11 detection network to quickly scan the garment region, outputting preliminary defect candidate boxes, their category predictions, and detection confidence scores. The defect box fusion and correction module matches the candidate boxes output from the initial defect detection with the pixel-level defect set generated by the garment mask segmentation module. By calculating the mask-box matching degree and based on the matched defect mask information, the candidate boxes are fused and corrected to generate more accurate, refined defect boxes that better fit the true shape of the defects, and the mask consistency is recorded. For each local defect area defined by the refined defect bounding box, the multi-dimensional feature extraction and anomaly scoring module performs in-depth analysis, extracting texture features and morphological similarity features. Based on these multi-dimensional features, it calculates the anomaly score for each defect area, quantifying its degree of anomaly. The comprehensive confidence assessment module weights and fuses the detection confidence of the initial defect detection, the mask consistency of the defect bounding box fusion correction, and the anomaly score from the multi-dimensional feature extraction and anomaly scoring to calculate the comprehensive confidence of each defect. Based on this, it performs the final defect type classification and confidence assessment on the retained defect bounding boxes, outputting the final defect recognition result, including defect type, precise location, and comprehensive confidence. The result output and alarm module is responsible for presenting the final recognition result to the inspectors in an intuitive way, such as marking the defect location and type on the image, and triggering log alarms when severe defects are detected or the number of defects exceeds a preset threshold. Finally, the model adaptive optimization module automatically collects low-confidence samples or samples whose labeling results are inconsistent with manual review, constructs a set to be labeled, submits it for accurate manual labeling and confirmation, and uses high-quality data for secondary training and model updates of the clothing mask segmentation and initial defect detection modules. This enables the system to continuously adapt to new defect types, clothing styles and production environments, and achieve continuous iterative optimization of performance.

[0057] In Turing enterprise identification tasks, the system of this invention, deployed within the enterprise's built-in program, can accurately identify various defect types such as loose threads, stains, and holes. The system's detection accuracy reaches over 95%, with a false detection rate controlled below 3%, and labor costs reduced by 70%. This fully verifies the effectiveness and practicality of the invention.

[0058] The foregoing description illustrates and describes several preferred embodiments of the present invention. However, as previously stated, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for identifying defects in a garment based on visual image features, characterized in that, The method comprises the following steps: S1, image preprocessing and garment mask segmentation: after the input garment image is preprocessed by denoising, color correction and size normalization, a SAM network is used for pixel-level mask segmentation of the garment region to obtain a mask of the main garment region and an initial defect set; S2, multi-scale feature construction and initial defect detection: multi-scale feature maps are constructed through a feature pyramid network FPN, and the multi-scale feature maps are input into an improved YOLOv11 detection network, the improvement including introducing a PANet and an attention mechanism module, and outputting initial defect candidate boxes, defect category prediction results and detection confidence; S3, fusion and correction of defect candidate boxes: based on the pixel-level mask of the initial defect set and the initial defect candidate boxes, a mask-box matching degree is obtained by calculating an IoU or a Dice coefficient, the initial defect candidate boxes that meet a preset threshold in the matching degree are fused and corrected in combination with a minimum bounding rectangle or a convex hull of the mask to obtain refined defect boxes, and a mask consistency degree of the refined defect boxes and the mask is recorded; S4, multi-dimensional feature extraction and abnormal score calculation: multi-dimensional features including texture features and morphological similarity features are extracted for a local region corresponding to the refined defect box, and an abnormal score is calculated based on the multi-dimensional features through a statistical method or a deep feature anomaly detection model; S5, comprehensive confidence weighted fusion and final defect recognition: a comprehensive confidence is calculated by a formula: comprehensive confidence=w1xdetection confidence+w2xmask consistency+w3xabnormal score, and weighted fusion is performed, wherein w1, w2 and w3 are weight coefficients, and w1+w2+w3=1; Based on the comprehensive confidence, refined defect boxes that meet a threshold are retained, defect type classification and confidence evaluation are performed, and final defect recognition results are obtained, the final defect recognition results including defect types, refined defect box positions and comprehensive confidences; S6, secondary training of low-confidence samples and model updating: low-confidence samples with a comprehensive confidence lower than a preset threshold or label inconsistent samples with inconsistent system recognition results and artificial re-inspection results are constructed into a set to be labeled; After artificial confirmation and accurate labeling, the set to be labeled is used for secondary training to update the SAM network and the improved YOLOv11 detection network. 2.The clothing visual image defect feature recognition method based on clothing according to claim 1, characterized in that, In step S1, the SAM network is adjusted and adapted to different garment types and complex backgrounds through parameters to realize pixel-level differentiation between garment regions and backgrounds. 3.The clothing visual image defect feature recognition method based on clothing according to claim 1, characterized in that, In step S2, the improved YOLOv11 detection network further includes anchor box optimization: K-means++ algorithm is used to cluster garment defect sizes and aspect ratios in a training set to generate anchor boxes adapted to garment defect features. 4.The clothing visual image defect feature recognition method based on clothing according to claim 1, characterized in that, In step S3, the preset threshold is 0.7, that is, when the mask-box matching degree is greater than or equal to 0.7, fusion and correction are performed; otherwise, the initial defect candidate box is discarded or enters an artificial re-inspection process. 5.The clothing visual image defect feature recognition method based on clothing according to claim 1, characterized in that, In step S4, the texture features include a gray level co-occurrence matrix GLCM or a Gabor filter response, a local binary pattern LBP feature, and the morphological similarity features include a shape context descriptor, a Hu invariant moment and a Fourier descriptor. 6.The clothing visual image defect feature recognition method based on clothing according to claim 1, characterized in that, In step S4, the statistical method includes Gaussian Mixture Model or One-Class SVM, and the deep feature anomaly detection model includes autoencoder or generative adversarial networks (GANs). 7.The clothing visual image based defect feature identification method according to claim 1, characterized in that, In step S5, the weight coefficients w1, w2 and w3 are determined by cross-validation or expert experience optimization, and can be dynamically adjusted according to the clothing material. 8.The clothing visual image based defect feature identification method according to claim 1, characterized in that, In step S6, the artificial confirmation includes labeling the existence, real type, accurate pixel-level mask and bounding box of the defect, and the secondary training updates the network parameters in an incremental training manner.

9. A quality inspection system based on the garment visual image defect feature recognition method according to claim 1, characterized in that, The system comprises: an image acquisition module including an industrial camera and a light source, which is used to acquire high-resolution images of the clothing to be detected; an image preprocessing module including a denoising unit, a color correction unit and a size normalization unit, which is used to preprocess the clothing images; a clothing mask segmentation module integrating a semantic segmentation network based on a SAM architecture, which is used to perform pixel-level mask segmentation on the clothing region of the preprocessed images, and output a clothing main mask and an initial defect set; an initial defect detection module integrating a feature pyramid network (FPN) and an improved YOLOv11 detection network including a PANet and an attention mechanism module, which is used to construct multi-scale features and output initial defect candidate boxes, class prediction results and detection confidence; a defect box fusion and correction module, which is used to calculate the IoU or Dice coefficient of the initial defect candidate box and the initial defect set mask, and correct the candidate box based on the mask outline with a matching degree ≥0.7 to generate a refined defect box and mask consistency degree; a multi-dimensional feature extraction and anomaly scoring module including a texture feature extraction unit, a morphological similarity feature extraction unit and an anomaly scoring calculation unit; a comprehensive confidence evaluation module including a weighted fusion unit, which is used to calculate the comprehensive confidence by the formula: comprehensive confidence=w1×detection confidence+w2×mask consistency+w3×anomaly score, and output the final defect recognition result; a model self-adaptive optimization module including a to-be-labeled set construction unit, an artificial confirmation interface and a secondary training unit. 10.The quality inspection system based on the clothing visual image flaw feature recognition method according to claim 9, characterized in that, The light source of the image acquisition module is a backlight or a uniform diffuse light source, which is used to avoid image reflection and shadow; the loss function of the improved YOLOv11 detection network uses GIoU Loss or CIoU Loss as the bounding box regression loss.

Images