Mask flaw detection model construction method, medium and mask flaw detection system

Abstract

The invention discloses a mask flaw detection model construction method, a medium and a mask flaw detection system. The method comprises the following steps: acquiring different types of flaw mask images and flawless normal mask images; performing data annotation on the defective mask image, and annotating the position and the category of a defective area; performing standardization processing on all the mask images, and normalizing pixel values to a [0, 1] interval or a [-1, 1] interval; a defect detection model is trained based on the standardized mask image, and the defect detection model is a YOLOv81 model; converting the trained flaw detection model from a PyTorch format to an ONNX format; and the ONNX format is converted into a TensorRT format. According to the invention, the detection capability and the detection speed of facial mask flaws are greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a facial mask defect detection model construction method, a medium, and a facial mask defect detection system. Background Art

[0002] Traditional facial mask defect detection mostly relies on manual visual inspection, which is inefficient in large-scale production and is easily affected by visual fatigue and subjective judgment, resulting in unstable detection results.

[0003] With the development of deep learning methods, machine vision has begun to replace the human eye for defect detection. However, existing defect detection methods are mostly used for cloth defect detection or workpiece defect detection. When applied to facial mask defect detection, these technologies face the following technical issues: 1. Insufficient identification of defect types, some defects cannot be identified.

[0004] 2. The defect recognition accuracy is insufficient, and some minor defects cannot be identified.

[0005] 3. Unable to meet the speed requirements of real-time recognition. It is unable to meet the production line's demand for a response time of less than 200ms, especially when hardware performance is relatively limited. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the deficiencies in the above-mentioned prior art and provide a method for constructing a facial mask defect detection model, a medium, and a facial mask defect detection system. By using the YOLOv8l model, background interference is effectively filtered out, the accuracy of defect detection is improved, and the diversity of defect samples is achieved through P-graph technology, reducing the fluctuations in model accuracy and recall rate when processing different types of defect detection, greatly improving the detection capability of facial mask defects, and optimizing the inference speed through TensorRT to meet the speed requirements of real-time recognition.

[0007] The first aspect of the present invention discloses a method for constructing a facial mask defect detection model, comprising the following steps: Step 1: Collect different types of blemish mask images and flawless normal mask images; Step 2: Label the blemish mask image and mark the location and category of the blemish area; Step 3: Normalize all mask images and normalize pixel values to the [0, 1] interval or the [-1, 1] interval; Step 4: Based on the standardized mask image, train a defect detection model, which is a YOLOv8l model. Step 5: Use PyTorch's torch.onnx.export function to convert the trained defect detection model from PyTorch format to ONNX format. Then use NVIDIA TensorRT to optimize and accelerate the model and convert ONNX format to TensorRT format.

[0008] The above-mentioned facial mask defect detection model construction method, when training the defect detection model, jointly applies the Focal loss function and the UIoU loss function.

[0009] The above-mentioned facial mask defect detection model construction method adopts the gradient descent optimization algorithm Adam during training, combined with adaptive learning rate decay and early stopping strategy for training.

[0010] In the above-mentioned method for constructing a facial mask defect detection model, in step 2, when data annotation of the defective facial mask image is performed, it is divided into loose mode annotation and strict mode annotation; In strict mode, both minor and obvious defect areas are marked, while in loose mode, only obvious defect areas are marked. Minor defect areas and obvious defect areas are distinguished based on the threshold of the defect area.

[0011] In the above-mentioned method for constructing a facial mask defect detection model, step 2 also includes: performing P-image processing on all defective facial mask images, wherein the P-image processing refers to synthesizing virtual defects on the original image, and the virtual defects have their own annotation information.

[0012] The above-mentioned method for constructing a facial mask defect detection model also includes step 6, encrypting the trained model and performing software authorization management.

[0013] The above-mentioned method for constructing a facial mask defect detection model also includes step 7, performing cyclic iterative optimization of the defect detection model based on false detection and missed detection samples.

[0014] The above-mentioned method for constructing a facial mask defect detection model, before performing the standardization process in step 3, further includes detecting tailing images through variance analysis, and removing the detected tailing images or the tailing parts in the tailing images.

[0015] A second aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the deployment or operation of the defect detection model described in the first aspect.

[0016] The third aspect of the present invention discloses a facial mask defect detection system, comprising an input device, a training machine, a host computer, a facial mask machine and a rejecting knife; The input device includes two high-resolution cameras deployed on the facial mask machine, and the two high-resolution cameras are used to collect the front image and the back image of the facial mask respectively; The training machine is used to execute the facial mask defect detection model construction method described in the first aspect above to construct a defect detection model; The host computer is equipped with a defect detection model output by the training machine to screen out unqualified masks and control the rejection knife; The facial mask machine is used for continuously transporting facial masks; The rejecting blade is used to reject unqualified facial masks on the facial mask machine.

[0017] Compared with the prior art, the present invention has the following advantages: 1. Collecting multiple defect samples in step 1 and standardizing them in step 3 ensures that the model can adapt to different defect forms and environmental changes, reducing the risk of overfitting. The refined annotation of defect locations and categories in step 2, combined with the powerful target detection capabilities of the YOLOv8l model, achieves high-precision defect location and classification, which is particularly suitable for identifying tiny defects.

[0018] 2. YOLOv8l, as a model that balances lightweight and precision, optimizes computational efficiency while ensuring detection accuracy, making it suitable for real-time detection scenarios. By converting the model to TensorRT format (step 5) and leveraging optimization technologies such as layer fusion and precision calibration (FP16 / INT8 quantization), the inference speed is significantly improved, meeting the millisecond-level response requirements of the production line.

[0019] 3. From data preprocessing to model format conversion, standardized operations throughout the entire process ensure model compatibility with industrial environments and reduce hardware adaptation costs. ONNX, used as an intermediate representation (step 5), supports multi-framework interoperability, and TensorRT further optimizes GPU utilization, making it suitable for edge computing devices (such as NVIDIA Jetson) or cloud servers, enabling flexible deployment.

[0020] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the method of Example 1.

[0022] Figure 2 This is a flow chart of the operating principle of Example 3. DETAILED DESCRIPTION

[0023] Example 1: Figure 1 As shown, a method for constructing a facial mask defect detection model includes the following steps: Step 1: Data collection: A high-resolution industrial camera (1920×1920 pixels) was used to capture images on the facial mask production line, covering all defect types (including discolored spots, missing edges, creases, defects, hair, etc.) as well as samples without defects. Image acquisition was performed under constant lighting conditions to avoid environmental interference.

[0024] Step 2: Data annotation: Use the annotation tool to mark the defective area in strict mode and loose mode: Strict mode: Mark all defect areas (including minor defects, the area threshold is set to 0.5mm 2 ).

[0025] Loose mode: only mark areas ≥ 1mm 2 obvious flaws.

[0026] The annotation information includes bounding box coordinates, defect categories (different-color stains, missing edges, etc.), and is saved as a label file in YOLO format.

[0027] This stage is the basis of model training. The accuracy of annotation directly affects the final detection effect. Common image annotation tools are used to annotate images. These annotation information will affect the subsequent model threshold setting and detection strategy.

[0028] To improve the model's detection accuracy for small and discolored stains, this paper uses P-image technology. P-image involves synthesizing virtual defects on the original image, enhancing the diversity of the dataset and improving the robustness of the model.

[0029] Randomly select certain areas on the original image (such as the edge or center of the mask) to simulate stains or small blemishes.

[0030] Use different colors, shapes, and textures to simulate defects such as off-color stains and missing edges, and increase the diversity of these defects.

[0031] Different defect samples are generated by rotation, scaling, color change, etc., further enhancing the diversity of the dataset.

[0032] In this way, the model can learn more diverse defect features, especially having better detection capabilities for difficult-to-detect defects such as small stains and discolored stains.

[0033] Step 3: Image preprocessing: Before the data is input into the network, necessary preprocessing is required to ensure that the image quality is suitable for model training and inference.

[0034] Normalization: Normalize image pixel values to the range [0, 1] to accelerate network training. Resize the captured images to the YOLO model input size of 1920x1920. This step involves scaling and cropping the images to preserve key features. Data augmentation techniques such as random cropping, rotation, flipping, and color changes can also be used to increase the diversity of the training set and prevent overfitting.

[0035] Smear detection: Calculate the local variance of the image. If the line scan camera's smear phenomenon is detected (such as a sudden drop in variance in a continuous area), the image is rejected.

[0036] Step 4: Model training: Network structure: YOLOv8l model is used, and the input size is adjusted to 1920×1920.

[0037] Loss function: Focal Loss (γ=2, α=0.25) is used for classification tasks to solve the problem of category imbalance.

[0038] Detection box regression uses UIoU Loss to improve positioning accuracy.

[0039] It's important to note that the Focal Loss function is primarily used to address class imbalance, particularly when dealing with small objects and hard-to-classify defects. Focal Loss can reduce the weight of easily classified samples and focus training on difficult-to-classify samples. This allows the model to better learn the characteristics of hard-to-detect categories such as small spots and discolored stains. The latest UIoU loss function optimizes the position and size of the detection box to ensure precise positioning. Especially when dealing with fuzzy edges or irregular shapes of defects, the UIoU loss helps the model better fit the defect area.

[0040] Optimization strategy during training: Use the Adam optimizer with an initial learning rate of 1e-4 and a cosine annealing schedule. If the validation set loss does not decrease for three consecutive epochs, reduce the learning rate to 10% of the current value. If the validation set accuracy does not improve within 10 epochs, terminate training.

[0041] Step 5: Model conversion: To improve inference speed, especially in production environments, this paper converts the trained PyTorch model into a model that can run efficiently on production line hardware through the following steps: Convert the trained .pt model to ONNX format through PyTorch's torch.onnx.export.

[0042] Using FP16 precision and layer fusion technology, the ONNX model is converted to the TensorRT engine, and the inference speed is improved from 210ms to 110ms.

[0043] Step 6: Model encryption and authorization: Model files are encrypted using common encryption algorithms such as AES to ensure that only authorized devices can load them. This measure can effectively prevent the model from being obtained by unauthorized third parties.

[0044] A license is generated based on the device's hardware fingerprint to ensure the legitimacy of the software system. Models can only be run for inference on devices that have passed authorization verification. In this example, the host computer's NVIDIA 3050 GPU and CPU are bound.

[0045] Step 7: Loop iteration optimization: The effectiveness of the system needs to be ensured through real-time feedback and improvement. The present invention uses a cyclic iterative optimization mechanism to continuously improve the detection model through false detection and missed detection analysis.

[0046] False detection and missed detection feedback mechanism: The system records the false detection (detecting a normal mask as a defect) and missed detection (not detecting the actual defect) information generated during each detection process and analyzes it regularly.

[0047] False positives and missed detections are annotated: False positives and missed detections are annotated, manually reviewed, and added to the training dataset. This new data is used for secondary model training, further improving detection accuracy and model robustness.

[0048] Cyclic training and updating: Each round of new training data is updated based on feedback information (false detections, missed detections) from actual applications, allowing the system to be continuously optimized during the production process and avoiding long-term performance degradation.

[0049] Through the above steps, the entire facial mask defect detection system is able to efficiently perform data collection, annotation, training, inference acceleration, and optimization. Innovations in data annotation, spot and defect re-image processing, Focal Loss, and IoU loss functions in particular give the system significant advantages in detection accuracy and real-time performance. Furthermore, model encryption and software authorization ensure system security, while an iterative optimization mechanism further enhances the system's long-term performance.

[0050] Example 2: The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the deployment or operation of the defect detection model described in Example 1.

[0051] Example 3: Figure 2As shown, the present invention also discloses a facial mask defect detection system, including an input device, a training machine, a host computer, a facial mask machine and a rejection knife; The input device includes two high-resolution cameras deployed on the facial mask machine, and the two high-resolution cameras are used to collect the front image and the back image of the facial mask respectively; The training machine is used to execute the facial mask defect detection model construction method described in Example 1 to construct a defect detection model; The host computer is equipped with a defect detection model output by the training machine to screen out unqualified masks and control the rejection knife; The facial mask machine is used for continuously transporting facial masks; The rejecting blade is used to reject unqualified facial masks on the facial mask machine.

[0052] During specific use, the mask machine inputs the mask, and the high-resolution camera deployed on the mask machine captures the front and back images of the mask. The captured images are input into the host computer, and the host computer performs defect detection based on the built-in defect detection model. If a defect is detected, a rejection instruction is sent to the PLC controller, and the PLC controller controls the rejection knife to reject the defective mask.

[0053] The specific structure of the facial mask machine and the reject knife can be found in Chinese invention patent CN119869965A, an automatic detection mechanism for facial mask defects.

[0054] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for constructing a facial mask defect detection model, characterized in that: The following steps are involved: Step 1: Collect different types of blemish mask images and flawless normal mask images; Step 2: Label the blemish mask image and mark the location and category of the blemish area; Step 3: Normalize all mask images and normalize pixel values to the [0, 1] interval or the [-1, 1] interval; Step 4: Based on the standardized mask image, train a defect detection model, which is a YOLOv8l model. Step 5: Use PyTorch's torch.onnx.export function to convert the trained defect detection model from PyTorch format to ONNX format. Then use NVIDIA TensorRT to optimize and accelerate the model and convert ONNX format to TensorRT format.

2. The method for constructing a facial mask defect detection model according to claim 1, wherein: When training the defect detection model, the Focal loss function and the UIoU loss function are jointly applied.

3. The method for constructing a facial mask defect detection model according to claim 2, wherein: The defect detection model is trained using the gradient descent optimization algorithm Adam, combined with adaptive learning rate decay and early stopping strategy.

4. The method for constructing a facial mask defect detection model according to claim 1, wherein: In step 2, the data annotation of the blemish mask image is divided into loose mode annotation and strict mode annotation; In strict mode, both minor and obvious defect areas are marked, while in loose mode, only obvious defect areas are marked. Minor defect areas and obvious defect areas are distinguished based on the threshold of the defect area.

5. The method for constructing a facial mask defect detection model according to claim 1, wherein: Step 2 also includes: performing P-image processing on all blemish mask images, wherein the P-image processing refers to synthesizing virtual blemishes on the original image, and the virtual blemishes have their own annotation information.

6. The method for constructing a facial mask defect detection model according to claim 1, wherein: It also includes step 6, encrypting the trained model and managing software authorization.

7. The method for constructing a facial mask defect detection model according to claim 6, wherein: The method also includes step 7, performing cyclic iterative optimization on the defect detection model based on the falsely detected or missed samples.

8. The method for constructing a facial mask defect detection model according to claim 1, wherein: Before the standardization process is performed in step 3, the method further includes detecting the tailing picture by variance analysis, and removing the detected tailing picture or the tailing part in the tailing picture.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the deployment or operation of the defect detection model described in any one of claims 1 to 8 is realized.

10. A facial mask defect detection system, characterized in that: Including input device, training machine, host computer, facial mask machine and rejection knife; The input device includes two high-resolution cameras deployed on the facial mask machine, and the two high-resolution cameras are used to collect the front image and the back image of the facial mask respectively; The training machine is used to execute the facial mask defect detection model construction method according to any one of claims 1 to 8 to construct a defect detection model; The host computer is equipped with a defect detection model output by the training machine to screen out unqualified masks and control the rejection knife; The facial mask machine is used for continuously transporting facial masks; The rejecting blade is used to reject unqualified facial masks on the facial mask machine.

Images