Contact lens defect detection model training method and device
By acquiring real sample data and customizing defect types, and using targeted sample data to generate a model to train a contact lens defect detection model, the problems of data scarcity and defect quantification are solved, achieving efficient and accurate defect detection and quantitative analysis, thereby improving production efficiency and product quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BRIGHTVIEW MEDICAL TECHNOLOGIES (NANJING) CO LTD
- Filing Date
- 2024-10-17
- Publication Date
- 2026-04-17
AI Technical Summary
Existing contact lens defect detection models face the problem of scarce specific defect samples during training, resulting in an extremely unbalanced dataset distribution. This makes it difficult to effectively cover the complexity and diversity of defects in real production environments, and the lack of quantitative analysis of defects affects production efficiency and product quality control.
By acquiring real sample data and customizing defect types, a pre-trained directional sample data generation model is used to generate directional sample data. Information fusion is then performed using a cross-attention method to train a contact lens defect detection model, enabling the identification and segmentation of defects and automatic quantitative evaluation.
It significantly improves the model's detection accuracy and generalization ability, enabling rapid and accurate identification and quantification of defects in contact lenses, improving production efficiency, reducing human error, and ensuring the objectivity and accuracy of detection results.
Smart Images
Figure CN121883348A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for training a contact lens defect detection model. Background Technology
[0002] In modern industrial manufacturing, the widespread application of automated production lines has greatly improved production efficiency, but the resulting product quality control issues have also become increasingly prominent. Among these, the automatic detection of defective samples, as a crucial link in ensuring product pass rates, is undeniably important. An ideal automatic detection model needs to possess high accuracy, high sensitivity, and strong robustness to cope with complex and ever-changing production environments. However, training such a model faces the problem of scarce specific defect samples, especially on efficient and low-error-rate production lines, where the probability of generating specific defective samples is extremely low, leading to a highly imbalanced distribution of the training dataset.
[0003] Currently, to alleviate the problems caused by data scarcity, data augmentation techniques are often employed. These techniques aim to increase the diversity of defect samples through human manipulation, thereby improving the generalization ability of the model. These techniques include, but are not limited to, adjusting the visual attributes of defective images such as hue, saturation, and brightness, as well as applying image transformations such as inversion, rotation, and morphological transformations. In addition, there are methods that superimpose known defects onto other image samples to simulate more diverse defect scenarios.
[0004] While the aforementioned data augmentation techniques have alleviated the data scarcity problem to some extent, they still reveal many limitations in practical applications. Most of these techniques are based on pre-defined transformation rules, making it difficult to fully capture and simulate the complexity and diversity of defects in real-world production environments. Especially in few-shot learning scenarios, due to the limited and discrete number of available defect samples, existing data augmentation methods often struggle to effectively cover all possible defect morphologies and distribution characteristics.
[0005] Moreover, most existing contact lens defect detection models only detect whether the contact lens contains defects, without further quantitative analysis of the defects. As a result, they cannot identify problems in the production line, which is not conducive to upgrading and improving the production line and increasing production efficiency. Summary of the Invention
[0006] To address the above problems, this application provides a method and apparatus for training a contact lens defect detection model, comprising the following:
[0007] Firstly, this application provides a method for training a contact lens defect detection model, the method comprising:
[0008] Acquire real sample data and real sample annotation data, wherein the real sample data includes real contact lens images with defects, and the real sample annotation data includes the types of defects in the real sample data;
[0009] Obtain custom defect types, and use a pre-trained directional sample data generation model to generate directional generated sample data, wherein the samples in the directional generated sample data include one or more types of custom defects;
[0010] The targeted generated sample data is labeled to obtain targeted generated sample labeled data;
[0011] The contact lens defect detection model is trained using the real sample data, the defect types of the real sample data, the targeted generated sample data, and the targeted generated sample annotation data.
[0012] Optionally, the contact lens defect detection model can be trained after enhancing the real sample data, the defect types of the real sample data, the targeted generated sample data, and the targeted generated sample annotation data.
[0013] Optionally, the training process of the pre-trained directional sample data generation model includes:
[0014] The image features of the real sample data are obtained after processing the real sample data, and are used as real image features;
[0015] Based on the defect types in the real sample data, a binary vector of length N is generated as defect type data, where N is the number of defect types.
[0016] Obtain defect features, wherein the defect features are image features with N trainable channels;
[0017] The real image features, the defect type data, and the defect features are fused to obtain reconstructed sample data, which is used to train the directional sample data generation model.
[0018] Optionally, the real image features, the defect type data, and the defect features are fused using a cross-attention method. The fused data is then input into the decoder of the directional sample data generation model, and the output of the decoder is used as the reconstructed sample data.
[0019] Optionally, generating targeted sample data using a pre-trained targeted sample data generation model for the customized defect types specifically includes:
[0020] Based on the customized defect types, a binary vector of length N is generated as defect type data, where N is the number of defect types. Each value in the vector is set to 0 or 1 to represent a combination of customized defect types.
[0021] Obtain defect features, wherein the defect features are image features with N trainable channels;
[0022] Obtain random noise features, wherein the random noise features are image features in which random noise is added to the defect features;
[0023] The defect type data, defect features, and random noise features are fused together and input into the targeted sample data generation model to obtain generated sample data for the customized defect type.
[0024] Optionally, the customized defect type, defect features, and random noise features are fused using a cross-attention method. The fused data is then input into the decoder of the directional sample data generation model. Based on the output of the decoder, generated sample data for the customized defect type is obtained.
[0025] Optionally, the information fusion using the cross-attention combination method includes:
[0026] Expand the defect type data into a defect type mask with N channels;
[0027] The defect features and the defect type mask are used to calculate the defect features of the defect type using the Hadamard product;
[0028] The image features for input to the decoder are obtained by combining the defect features of the defect type with the real image features, or by combining the defect features of the defect type with the random noise features, through a cross-attention mechanism.
[0029] Optionally, the trained contact lens defect detection model is used to identify and segment the contact lens image to be detected. The segmented image has the same pixel size as the contact lens image to be detected, and each pixel represents either no defect or a certain type of defect. Different types of defects are displayed through different grayscale values.
[0030] Optionally, the real sample annotation data and the directed generated sample annotation data are obtained through pixel-level annotation. The trained contact lens defect detection model is used to identify and segment the contact lens image to be detected, and the segmentation results of the contact lens defect detection model are automatically quantified and evaluated using a pre-trained evaluation model.
[0031] Optionally, the automatic quantification and evaluation of the detection results of the contact lens defect detection model includes:
[0032] (1) Obtain the number and total area of each type of defect;
[0033] For defects caused by the production environment, assess whether the number and total area of defects exceed the warning standard in order to monitor the cleanliness of the production environment;
[0034] And / or (2) For defects caused by the manufacturing process, detect the location, number and / or total area of the defects in the optical and non-optical areas of the lens, and determine whether the lens is qualified;
[0035] And / or (3) Statistically analyze the morphological characteristics of each defect to distinguish between random defects and systematic failures;
[0036] And / or (4) to count and track the frequency of various types of defects, determine whether there is an abnormality in the production line, and issue an early warning when the frequency exceeds the preset non-conforming frequency threshold.
[0037] Secondly, this application provides a training device for a contact lens defect detection model, the device comprising:
[0038] The first acquisition unit is used to acquire real sample data and real sample annotation data. The real sample data includes real contact lens images with defects, and the real sample annotation data includes the types of defects in the real sample data.
[0039] The second acquisition unit is used to acquire customized defect types;
[0040] A directional sample data generation unit is used to generate directional sample data from the customized defect types using a pre-trained directional sample data generation model, wherein the samples in the directional sample data include one or more types of customized defects.
[0041] The first processing unit is used to annotate the targeted generated sample data to obtain targeted generated sample labeled data.
[0042] The second processing unit is used to train the contact lens defect detection model using the real sample data, the defect types of the real sample data, the targeted generated sample data, and the targeted generated sample annotation data.
[0043] Optionally, the device further includes a third processing unit, used to perform enhancement processing on the real sample data, the defect types of the real sample data, the targeted generated sample data, and the targeted generated sample annotation data, and input the processed data into the second processing unit to train the contact lens defect detection model.
[0044] Optionally, the device includes a training unit for a directional sample data generation model, and the training process of the pre-trained directional sample data generation model is as follows:
[0045] The image features of the real sample data are obtained after processing the real sample data, and are used as real image features;
[0046] Based on the defect types in the real sample data, a binary vector of length N is generated as defect type data, where N is the number of defect types.
[0047] Obtain defect features, wherein the defect features are image features with N trainable channels;
[0048] The real image features, the defect type data, and the defect features are fused to obtain reconstructed sample data, which is used to train the directional sample data generation model.
[0049] Optionally, the device further includes a decoding unit, used to fuse the real image features, the defect type data, and the defect features using a cross-attention combination method, input the fused data into a decoder of the directional sample data generation model, and use the output of the decoder as reconstructed sample data.
[0050] Optionally, the targeted sample data generation unit generates targeted sample data for the customized defect types using a pre-trained targeted sample data generation model, specifically including:
[0051] Based on the customized defect types, a binary vector of length N is generated as defect type data, where N is the number of defect types. Each value in the vector is set to 0 or 1 to represent a combination of customized defect types.
[0052] Obtain defect features, wherein the defect features are image features with N trainable channels;
[0053] Obtain random noise features, wherein the random noise features are image features in which random noise is added to the defect features;
[0054] The defect type data, defect features, and random noise features are fused together and input into the targeted sample data generation model to obtain generated sample data for the customized defect type.
[0055] Optionally, the decoding unit is specifically used to, after fusing the customized defect type, defect features, and random noise features using a cross-attention combination method, input the fused data into the decoder of the directional sample data generation model for decoding, and obtain the generated sample data for the customized defect type based on the output of the decoder.
[0056] Optionally, the apparatus further includes an information fusion unit for performing information fusion using a cross-attention combination method, including:
[0057] Expand the defect type data into a defect type mask with N channels;
[0058] The defect features and the defect type mask are used to calculate the defect features of the customized defect type using the Hadamard product;
[0059] The image features for inputting the decoder are obtained by combining the defect features of the customized defect type with the real image features, or by combining the defect features of the customized defect type with the random noise features, and processing them through a cross-attention mechanism.
[0060] Optionally, the device further includes a fourth unit, used to identify and segment defects in the contact lens image to be detected using the trained contact lens defect detection model. The segmented image has the same pixel size as the contact lens image to be detected, and each pixel represents either no defect or a certain type of defect. Different types of defects are displayed through different grayscale values.
[0061] Optionally, the device further includes a fifth processing unit, wherein the real sample annotation data and the directed generated sample annotation data are obtained through pixel-level annotation. The fifth processing unit is used to use the trained contact lens defect detection model to identify and segment the contact lens image to be detected, and to use a pre-trained evaluation model to automatically quantify and evaluate the segmentation results of the contact lens defect detection model.
[0062] Optionally, the automatic quantification and evaluation of the detection results of the contact lens defect detection model includes:
[0063] (1) Obtain the number and total area of each type of defect;
[0064] For defects caused by the production environment, assess whether the number and total area of defects exceed the warning standard in order to monitor the cleanliness of the production environment;
[0065] And / or (2) For defects caused by the manufacturing process, detect the location, number and / or total area of the defects in the optical and non-optical areas of the lens, and determine whether the lens is qualified;
[0066] And / or (3) Statistically analyze the morphological characteristics of each defect to distinguish between random defects and systematic failures;
[0067] And / or (4) to count and track the frequency of various types of defects, determine whether there is an abnormality in the production line, and issue an early warning when the frequency exceeds the preset non-conforming frequency threshold.
[0068] Thirdly, this application provides an apparatus comprising a memory and a processor, the memory for storing instructions or code, and the processor for executing the instructions or code to cause the apparatus to perform the contact lens defect detection model training method described in any of the implementations of the first aspect.
[0069] Fourthly, this application provides a computer-readable storage medium storing code, wherein when the code is executed, a device running the code implements the contact lens defect detection model training method described in any of the implementations of the first aspect.
[0070] This application provides a training method for a contact lens defect detection model. This method significantly improves the model's performance and generalization ability, with specific beneficial effects including: (1) More comprehensive defect coverage: Customized defect data allows for the targeted generation of contact lens samples containing specific or multiple defects. These targeted samples can cover various defect types that may be encountered in actual production, ensuring that the model can fully learn the characteristics of these defects during training, thereby improving the detection accuracy and sensitivity. (2) Improved model generalization ability: By introducing enhanced real sample data and targeted defect sample data, the model can access more diverse data distributions during training, thereby enhancing its generalization ability. This means that the model can still maintain a high detection accuracy when faced with unseen contact lens images. (3) Optimized detection efficiency: A fully trained contact lens defect detection model can quickly and accurately identify defects in contact lenses and perform quantitative analysis and evaluation, which is of great significance for improving production efficiency and product quality. (4) Furthermore, the automated detection achieved in this application can reduce human error and ensure the objectivity and accuracy of the detection results. Attached Figure Description
[0071] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0072] Figure 1 A flowchart illustrating a method for training a contact lens defect detection model, as provided in this application embodiment;
[0073] Figure 2 This is a schematic diagram of the generated sample data provided in an embodiment of this application;
[0074] Figure 3 A training flowchart for a directional sample data generation model provided in this application embodiment;
[0075] Figure 4 A schematic diagram of the cross-attention combination method provided in the embodiments of this application;
[0076] Figure 5 A schematic diagram illustrating the training process of another contact lens defect detection model provided in this application embodiment;
[0077] Figure 6 This is a schematic diagram of an image segmentation result provided in an embodiment of this application;
[0078] Figure 7 This is a schematic diagram of a contact lens defect detection model training device provided in an embodiment of this application. Detailed Implementation
[0079] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0080] Figure 1 This is a flowchart illustrating a method for training a contact lens defect detection model, provided in an embodiment of this application. (Combined with...) Figure 1 As shown, the contact lens defect detection model training method provided in this application embodiment may include:
[0081] S101. Obtain real sample data and real sample annotation data, wherein the real sample data includes real contact lens images with defects, and the real sample annotation data includes the types of defects in the real sample data.
[0082] The authentic sample data consists of real contact lens images, both defective and non-defective, obtained during actual production. The defective contact lens images include various types of defects, specifically surface defects, edge defects, and air bubbles / pores. Surface defects include impurities, scratches, and splits, while edge defects include polygons, curled edges, and missing edges. The authentic sample images may contain one or more types of defects simultaneously.
[0083] Real sample annotation can be performed using existing methods. This embodiment utilizes manual or existing artificial intelligence-assisted pixel-level annotation of real sample data. Each image in the annotation result has detailed annotation information, such as whether defects exist, the specific type of defect (e.g., impurities, scratches, splits, polygons, curled edges, missing edges, bubbles, pores, etc.), and the location and size of each defect. Obtaining pixel-level annotation results is beneficial for subsequent quantitative analysis and evaluation of defects.
[0084] S102. Obtain the types of customized defects, and use a pre-trained directional sample data generation model to generate directional generated sample data, wherein the samples in the directional generated sample data include one or more types of customized defects.
[0085] Custom defect types are obtained, and then a pre-trained model (such as a Generative Adversarial Network, GAN) is used to generate targeted sample data with specific defect types based on the custom defect type data. The custom defect type data is a binary vector of length N generated based on the custom defect types. Here, N represents the number of defect types, specifically referring to all possible types of defects contained in contact lenses, such as impurities, scratches, splits, polygons, rolled edges, missing edges, bubbles, and pores. Each value in the vector takes the value 0 or 1; 0 indicates that defects are not expected to be included in the generated result, and 1 indicates that defects are expected to be included in the generated result. In this way, any number and any type of defect combination can be specified, improving the diversity of the generated defect samples.
[0086] The pre-trained directional sample data generation model is trained by combining images of contact lenses with real defects and images of contact lenses with customized defects. It should be noted that the process of generating directional sample data using the pre-trained directional sample data generation model can produce generated sample images that are similar in overall shape to real sample images but contain different types of defects, such as... Figure 2 The above, Figure 2This illustration shows a sample data generation method provided in this application. The left side shows an image of a real sample obtained during the production process (with bubble defects), and the right side shows a directional sample image generated by the pre-trained directional sample data generation model in this application (with splitting defects). This sample image can include one or more custom defects set by the user. This method not only enriches the diversity of sample data but also significantly improves the model's ability to identify and respond to various defects, providing strong support for subsequent defect detection and classification tasks. During model generation, different random image features are obtained through random sampling, allowing the model to generate diverse contact lens images, thereby improving the model's generalization ability when training the defect detection model. The specific training process of the directional sample data generation model is described in detail below and will not be repeated here.
[0087] S103. Label the directed sample data to obtain directed sample labeled data.
[0088] The annotation of targeted generated sample data can be carried out using existing methods. This embodiment involves manual or AI-assisted annotation of the targeted generated sample data. Specific annotation content includes detailed information annotation for each image, including the presence of defects, the specific type of defect (e.g., impurities, scratches, splits, polygons, curled edges, missing edges, bubbles, pores, etc.), and the location and size of each defect. Obtaining pixel-level annotation results facilitates subsequent quantitative analysis and evaluation of defects.
[0089] S104. The contact lens defect detection model is trained using the real sample data, the defect types of the real sample data, the directional generated sample data, and the directional generated sample annotation data.
[0090] The real sample data, the defect types in the real sample data, the targeted generated sample data, and the targeted generated sample annotation data are integrated into a complete training set. This integrated training set is then used to train the contact lens defect detection model. During training, it may be necessary to adjust the model's parameters and structure to achieve optimal performance.
[0091] In one implementation of this application, the real sample data, the defect types of the real sample data, the directed generated sample data, and the directed generated sample labeled data are augmented before the contact lens defect detection model is trained. The data augmentation method is not limited here. Specifically, traditional data augmentation methods can be used to process the real sample data, the defect types of the real sample data, the directed generated sample data, and the directed generated sample labeled data. For example, this includes scaling, rotation, projection transformation, changing hue, saturation, brightness, adding noise, blurring, and random splicing to increase the diversity and quantity of sample data, thereby improving the model's generalization ability.
[0092] The above embodiments introduce a method for training a contact lens defect detection model. During training, this model combines real sample data, defect types from the real sample data, targeted generated sample data, and targeted generated sample annotation data. The targeted generated sample data is generated through a pre-trained targeted sample data generation model. The training process of this targeted sample data generation model is described below with reference to specific embodiments. In one implementation method of this application embodiment, the training process of the pre-trained targeted sample data generation model includes:
[0093] After processing the real sample data, image features of the real sample data are obtained as real image features. Based on the defect types in the real sample data, a binary vector of length N is generated as defect type data, where N is the number of defect types. Specifically, it refers to all possible defect types contained in contact lenses. For example, there are 8 possible defect types in contact lenses: impurities, scratches, splits, polygons, curled edges, missing edges, bubbles, and pores. The real sample data contains impurities, scratches, and splits, so N is 8, and the binary vector of defect type data of length N is represented as [11100000]. Defect features are obtained, which are trainable image features with N channels, specifically the features corresponding to all possible defect types. For example, if there are 8 defects, then the number of channels N is 8. The real image features, the defect type data, and the defect features are fused to obtain reconstructed sample data, which is used to train the directional sample data generation model.
[0094] Specifically, real sample data is input into an encoder (which is a trainable deep neural network). The encoder extracts and outputs image features from the real sample data, which are the real image features. Based on the defect types present in the real sample annotation data, a binary vector of length N is generated as the real defect type data. Defect features are obtained, which are trainable image features with N channels, and their height and width are consistent with the real image features. The real image features, the real defect type data, and the defect features are fused using a cross-attention method. The fused data is input into a decoder, and the output of the decoder is used as the reconstructed sample data.
[0095] In one implementation method of this application embodiment, generating targeted generated sample data using a pre-trained targeted sample data generation model for the customized defect type specifically includes:
[0096] Obtain customized defect type data, which is a binary vector of length N, where each value is randomly set to 0 or 1 to represent a randomly customized combination of defect types. For example, the defect types include eight types: impurities, scratches, splits, polygons, rolled edges, missing edges, bubbles, and pores. The customized defect types are missing edges, bubbles, and pores. In this case, the binary vector of defect type data of length N is represented as [00000111]. Obtain random noise features, which are randomly generated image features with dimensions consistent with the real image features. Combine the defect features, the customized defect type data, and the random noise features using a cross-attention method to fuse information. Input the fused data into a decoder. Based on the output of the decoder, obtain the generated sample data for the customized defect types.
[0097] like Figure 3 As shown, Figure 3 The flowchart provided in this application embodiment shows a training process for a directional sample data generation model. The directional sample data generation model is a GAN-based model for generating custom defect types from defect samples. It includes real sample data 1, real defect type data 2, encoder 3, defect features 4, real image features 5, decoder 6, reconstructed samples 7, custom defect types 8, random noise features 9, and generated samples 10.
[0098] In this dataset, real sample data 1 consists of images of contact lenses with and without defects obtained during actual production; real defect type data 2 is a binary vector of length N obtained based on the defect types labeled in the real samples; encoder 3 is a trainable deep neural network that extracts and outputs the image features of real sample data 1 (i.e., real image feature 5) from the input of real sample data 1. Defect feature 4 is a trainable image feature with N channels, whose height and width are identical to real image feature 5. Real defect type data 2, defect feature 4, and real image feature 5 are integrated as follows: Figure 4 The cross-attention combination method shown, Figure 4 This is a schematic diagram of a cross-attention combination method provided in an embodiment of this application. The cross-attention combination method can perform information fusion. After the information fusion is completed, it is input into the decoder 6 of the trainable deep neural network structure to obtain the reconstructed sample 7.
[0099] Customized defect type data 8 is a binary vector of length N generated based on customized defect types, where each value is randomly set to 0 or 1, thus obtaining a customized combination of defect types. Random noise feature 9 is an image feature with random noise added to the defect features, and its dimension is consistent with the real image feature 5. Defect feature 4, customized defect type data 8, and random noise feature 9 are obtained through methods such as... Figure 4 The cross-attention combination method shown is used for information fusion.
[0100] The loss function of the targeted sample data generation model during training includes:
[0101] 1) Reconstruction loss function: By minimizing the difference between the real sample data 1 and the reconstructed sample 7 (which can be calculated using mean absolute error or mean squared error), the model is guided to learn the features of normal samples and the defect features 4 corresponding to each defect type.
[0102] 2) Adversarial Loss Function: First, an independent defect type classifier is trained, which learns to determine whether real sample data 1 contains each type of defect based on the real defect types. When training the generative model, for reconstructed sample 7, the difference between the classifier's judgment of the defect type of reconstructed sample 7 and the defect type of the real sample, and the difference between the classifier's judgment of the defect type of generated sample 10 and the customized defect type, are minimized. This guides the generative model to learn to generate samples that conform to the customized defect type for different customized defect types.
[0103] In one implementation of this application, the information fusion using the cross-attention combination method includes:
[0104] The defect type data 11 is expanded into a defect type mask 12 containing N channels. The defect features 4 and the defect type mask 12 are then calculated using the Hadamard product to obtain the defect features 13 of each defect type. These defect features 13 are combined with the real image features 5 or random noise features 9, and processed by the cross-attention module 14 to obtain the image features 15 used as input to the decoder. The specific implementation process is as follows... Figure 4 As shown, Figure 4 This is a schematic diagram of a cross-attention combination method provided in an embodiment of this application. The defect type data 11 is a binary vector of length N, which is expanded into a defect type mask 12 containing N channels. The expansion process includes broadcasting, copying and pasting, and padding operations. By expanding the defect type data into a defect type mask, the aim is to expand the single-dimensional defect type data into multi-dimensional image data, facilitating the calculation of the Hadamard product. Each channel is filled with 0 or 1 according to the value in the corresponding defect type data 11. The defect feature 4 is a matrix with the same dimension as the defect type mask 12, and the defect feature 13 of the defect type is obtained through the Hadamard product, i.e., multiplying the values at each corresponding position in the two matrices pairwise. The defect feature 13 of the defect type and the real image feature 5 / random noise feature 9 are combined through a cross-attention mechanism to obtain the image feature 15 used as input to the decoder, and finally, the image corresponding to the defect type data 11 is generated. Optionally, the cross-attention module 14 can be a transformer structure, in which the defect features 13 of the defect type and the real image features 5 / random noise features 9 are reshaped into vectors, which are then passed through linear layers and subjected to matrix multiplication and normalization; after that, they are reshaped again into a matrix of the desired image feature shape, which is used as the input of the decoder.
[0105] Through the methods described in the above embodiments, this application can train a contact lens defect detection model using a contact lens defect detection model training method. In particular, the method for generating contact lens defect patterns with customized defect types can set multiple defects that coexist, increasing the diversity of generated samples, and specifically generating and amplifying the data required for training the defect detection model, thereby improving the accuracy of the detection model and ensuring the stability of the production line and inspection efficiency.
[0106] In one implementation of this application, the method further includes: using the trained contact lens defect detection model to identify and segment defects in the contact lens image to be detected, and using a pre-trained evaluation model to automatically quantify and evaluate the segmentation results of the contact lens defect detection model. Specifically, the trained contact lens defect detection model is used to identify and segment defects in the contact lens image to be detected. The segmented image has the same pixel size as the contact lens image to be detected, and each pixel represents no defect or a certain type of defect. Different types of defects are displayed through different grayscale values.
[0107] This application further incorporates cutting-edge technologies from deep learning and computer vision, achieving not only high-precision recognition of surface patterns and textures in minute products such as contact lenses, but more importantly, by constructing an advanced pixel-level defect detection model, it enables meticulous post-processing of these detection results. For example... Figure 5 As shown, Figure 5 This is a schematic diagram illustrating another process for training a contact lens defect detection model according to an embodiment of this application. The process includes real sample data, labeled real sample data, traditional data augmentation, customized defect types, targeted generated sample data, targeted generated sample data labeling, defect detection model training, and defect result analysis and quantification. The specific implementation process is the same as the contact lens defect detection model training process described in the above embodiments, using the real sample data, the defect types in the real sample data, the targeted generated sample data, and the targeted generated sample labeled data to train the contact lens defect detection model. The specific implementation process of defect detection model training and defect result analysis and quantification is described in detail below.
[0108] Defect segmentation model training involves training a pre-trained defect detection model to segment the contact lens image to be detected. The segmentation process includes semantic segmentation, instance segmentation, and panoramic segmentation. The segmented image and the contact lens image to be detected have the same pixel size, with each pixel representing either no defect or a specific type of defect. Different grayscale values are used to display different types of defects. Figure 6 As shown, Figure 6 This is a schematic diagram of an image segmentation result provided in an embodiment of this application, where the left side is an image of the contact lens to be detected, and the right side is the result of defect identification and segmentation. The segmentation result and the image to be detected have the same pixel size, and each pixel is classified into normal or a certain type of defect. Different types are displayed with different grayscale values, making it more intuitive.
[0109] The defect result analysis and quantification process involves pixel-level image feature extraction and analysis for each type of defect, based on the actual production pass rate and the technical requirements of the production environment, to automate and standardize the production process. Specifically, the automatic quantification and evaluation of defect segmentation results includes:
[0110] (1) Obtain the quantity and total area of each type of defect. For each type of defect, identify and separate the connected components of the defect (this step is optional for instance segmentation and panoramic segmentation). For each connected component of each type of defect, calculate the area of that connected component. For all connected components of each type of defect, calculate their quantity and total area. For defects caused by the production environment, assess whether the quantity and total area of the defects exceed the warning standard to monitor the cleanliness of the production environment. For example, for dust-related defects, assess whether the quantity and total area exceed the warning standard to monitor the cleanliness of the production environment. If they exceed the standard, it indicates that the cleanliness of the production environment is abnormal, and it is necessary to investigate the cause of the problem or even suspend production.
[0111] (2) For defects caused by the manufacturing process, the location, number, and / or total area of the defects in the optical and non-optical areas of the lens are detected to determine whether the lens is qualified; based on the location and standards of defects such as bubbles, pores, and scratches in the optical and non-optical areas of the lens, the lens is judged to meet the standards of non-compliance (NG, not good), review (RW, review), and acceptance (OK). If the severity of the defect judgment results differs, the most severe one shall prevail. For example, if a scratch is detected in the optical area, the lens is judged to be non-compliance (NG, not good); if a bubble is detected at the edge of the non-optical area, the lens is judged to be review (RW, review); if no defects are detected in either the optical or non-optical areas, the lens is judged to be acceptance (OK); if a scratch is detected in the optical area and a bubble is detected at the edge of the non-optical area, the lens is judged to be non-compliance (NG, not good).
[0112] (3) Statistically analyze the morphological characteristics of each defect (including area, perimeter, eccentricity, etc.) to help investigate the cause of the defect and distinguish between random defects and systematic failures. For example, when the size and location of the scratch defects on the lens are the same, it can be determined to be a systematic failure. If the size and location of the scratch defects on the lens are random, it can be determined to be a random defect.
[0113] And / or (4) to count and track the frequency of various types of defects, determine whether there is an abnormality in the production line, and issue an early warning when the frequency exceeds the preset non-conforming frequency threshold.
[0114] By automatically quantifying and evaluating the defect segmentation results, the severity of defects is systematically and multidimensionally measured. In addition to ensuring product quality itself, it also provides a monitoring and early warning system for whether the production environment and production line equipment are malfunctioning, which is conducive to upgrading and improving the production line and increasing production efficiency.
[0115] The above are some specific implementations of a contact lens defect detection model training method provided in the embodiments of this application. Based on this, this application also provides a corresponding device. The device provided in the embodiments of this application will be described below from the perspective of functional modularity. Figure 7 This is a schematic diagram of a contact lens defect detection model training device provided in an embodiment of this application. (Combined with...) Figure 7 As shown, the contact lens defect detection model training device 700 provided in this application embodiment includes:
[0116] The first acquisition unit 710 is used to acquire real sample data and real sample annotation data. The real sample data includes real contact lens images with defects, and the real sample annotation data includes the types of defects in the real sample data.
[0117] The second acquisition unit 720 is used to acquire customized defect types;
[0118] The directional sample data generation unit 730 is used to generate directional sample data from the customized defect types using a pre-trained directional sample data generation model, wherein the samples in the directional sample data include one or more types of customized defects.
[0119] The first processing unit 740 is used to annotate the directed generated sample data to obtain directed generated sample annotation data.
[0120] The second processing unit 750 is used to train the contact lens defect detection model using the real sample data, the defect types of the real sample data, the directional generated sample data, and the directional generated sample annotation data.
[0121] In one implementation method of this application embodiment, the apparatus further includes a third processing unit, used to perform enhancement processing on the real sample data, the defect types of the real sample data, the directional generated sample data, and the directional generated sample annotation data, and input the data after data processing into the second processing unit to train the contact lens defect detection model.
[0122] In one implementation method of this application embodiment, the apparatus includes a training unit for a directional sample data generation model, and the training process of the pre-trained directional sample data generation model is specifically as follows:
[0123] The image features of the real sample data are obtained after processing the real sample data, and are used as real image features;
[0124] Based on the defect types in the real sample data, a binary vector of length N is generated as defect type data, where N is the number of defect types.
[0125] Obtain defect features, wherein the defect features are image features with N trainable channels;
[0126] The real image features, the defect type data, and the defect features are fused to obtain reconstructed sample data, which is used to train the directional sample data generation model.
[0127] Optionally, the device further includes a decoding unit, used to fuse the real image features, the defect type data, and the defect features using a cross-attention combination method, input the fused data into a decoder of the directional sample data generation model, and use the output of the decoder as reconstructed sample data.
[0128] In one implementation method of this application embodiment, the targeted sample data generation unit generates targeted sample data for the customized defect type using a pre-trained targeted sample data generation model, specifically including:
[0129] Based on the customized defect types, a binary vector of length N is generated as defect type data, where N is the number of defect types. Each value in the vector is set to 0 or 1 to represent a combination of customized defect types.
[0130] Obtain defect features, wherein the defect features are image features with N trainable channels;
[0131] Obtain random noise features, wherein the random noise features are image features in which random noise is added to the defect features;
[0132] The defect type data, defect features, and random noise features are fused together and input into the targeted sample data generation model to obtain generated sample data for the customized defect type.
[0133] In one implementation method of this application embodiment, the decoding unit is specifically used to, after fusing the customized defect type, defect features, and random noise features using a cross-attention combination method, input the fused data into the decoder of the directional sample data generation model for decoding, and obtain the generated sample data for the customized defect type according to the output result of the decoder.
[0134] In one implementation of the method described in this application, the apparatus further includes an information fusion unit, used for information fusion using a cross-attention combination method, including:
[0135] Expand the defect type data into a defect type mask with N channels;
[0136] The defect features and the defect type mask are used to calculate the defect features of the customized defect type using the Hadamard product;
[0137] The image features for inputting the decoder are obtained by combining the defect features of the customized defect type with the real image features, or by combining the defect features of the customized defect type with the random noise features, and processing them through a cross-attention mechanism.
[0138] In one implementation method of this application embodiment, the device further includes a fourth unit, used to identify and segment defects in the contact lens image to be detected using the trained contact lens defect detection model. The segmented image has the same pixel size as the contact lens image to be detected, and each pixel represents no defect or a certain type of defect. Different types of defects are displayed through different grayscale values.
[0139] In one implementation method of this application embodiment, the device further includes a fifth processing unit, wherein the real sample annotation data and the directed generated sample annotation data are obtained through pixel-level annotation. The fifth processing unit is used to use the trained contact lens defect detection model to identify and segment the contact lens image to be detected, and to use a pre-trained evaluation model to automatically quantify and evaluate the segmentation results of the contact lens defect detection model.
[0140] In one implementation method of this application embodiment, the automatic quantification and evaluation of the detection results of the contact lens defect detection model includes:
[0141] (1) Obtain the number and total area of each type of defect;
[0142] For defects caused by the production environment, assess whether the number and total area of defects exceed the warning standard in order to monitor the cleanliness of the production environment;
[0143] And / or (2) For defects caused by the manufacturing process, detect the location, number and / or total area of the defects in the optical and non-optical areas of the lens, and determine whether the lens is qualified;
[0144] And / or (3) Statistically analyze the morphological characteristics of each defect to distinguish between random defects and systematic failures;
[0145] And / or (4) to count and track the frequency of various types of defects, determine whether there is an abnormality in the production line, and issue an early warning when the frequency exceeds the preset non-conforming frequency threshold.
[0146] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.
[0147] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform the method described in any embodiment of this application.
[0148] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of this application.
[0149] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0150] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0151] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0152] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for training a contact lens defect detection model, characterized in that, The method includes: Acquire real sample data and real sample annotation data, wherein the real sample data includes real contact lens images with defects, and the real sample annotation data includes the types of defects in the real sample data; Obtain custom defect types, and use a pre-trained directional sample data generation model to generate directional generated sample data, wherein the samples in the directional generated sample data include one or more types of custom defects; The targeted generated sample data is labeled to obtain targeted generated sample labeled data; The contact lens defect detection model is trained using the real sample data, the defect types of the real sample data, the targeted generated sample data, and the targeted generated sample annotation data.
2. The method according to claim 1, characterized in that, After enhancing the real sample data, the defect types of the real sample data, the targeted generated sample data, and the targeted generated sample annotation data, the contact lens defect detection model is then trained.
3. The method according to claim 1, characterized in that, The training process of the pre-trained directional sample data generation model includes: The image features of the real sample data are obtained after processing the real sample data, and are used as real image features; Based on the defect types in the real sample data, a binary vector of length N is generated as defect type data, where N is the number of defect types. Obtain defect features, which are image features with N trainable channels; The real image features, the defect type data, and the defect features are fused to obtain reconstructed sample data, which is used to train the directional sample data generation model.
4. The method according to claim 3, characterized in that, The real image features, the defect type data, and the defect features are fused using a cross-attention method. The fused data is then input into the decoder of the directional sample data generation model, and the output of the decoder is used as the reconstructed sample data.
5. The method according to claim 1, characterized in that, The step of generating targeted sample data for the customized defect type using a pre-trained targeted sample data generation model specifically includes: Based on the customized defect types, a binary vector of length N is generated as defect type data, where N is the number of defect types. Each value in the vector is set to 0 or 1 to represent a combination of customized defect types. Obtain defect features, which are image features with N trainable channels; Obtain random noise features, wherein the random noise features are image features in which random noise is added to the defect features; The defect type data, defect features, and random noise features are fused together and input into the targeted sample data generation model to obtain generated sample data for the customized defect type.
6. The method according to claim 5, characterized in that, After fusing the customized defect type, defect features, and random noise features using a cross-attention combination method, the method further includes inputting the fused data into the decoder of the directional sample data generation model, and obtaining generated sample data for the customized defect type based on the output of the decoder.
7. The method according to claim 4 or 6, characterized in that, The information fusion method utilizing cross-attention includes: Expand the defect type data into a defect type mask with N channels; The defect features and the defect type mask are used to calculate the defect features of the defect type using the Hadamard product; The image features for input to the decoder are obtained by combining the defect features of the defect type with the real image features, or by combining the defect features of the defect type with the random noise features, through a cross-attention mechanism.
8. The method according to claim 1, characterized in that, The method further includes: The trained contact lens defect detection model is used to identify and segment the contact lens image to be detected. The segmented image has the same pixel size as the contact lens image to be detected. Each pixel represents no defect or a certain type of defect. Different types of defects are displayed by different gray values.
9. The method according to claim 1, characterized in that, The method further includes: The real sample annotation data and the directed generated sample annotation data are obtained through pixel-level annotation. The trained contact lens defect detection model is used to identify and segment defects in the contact lens image to be detected, and the segmentation results of the contact lens defect detection model are automatically quantified and evaluated using a pre-trained evaluation model.
10. The method according to claim 9, characterized in that, The automatic quantification and evaluation of the detection results of the contact lens defect detection model includes: (1) Obtain the number and total area of each type of defect; For defects caused by the production environment, assess whether the number and total area of defects exceed the warning standard in order to monitor the cleanliness of the production environment; And / or (2) For defects caused by the manufacturing process, detect the location, number and / or total area of the defects in the optical and non-optical areas of the lens, and determine whether the lens is qualified; And / or (3) Statistically analyze the morphological characteristics of each defect to distinguish between random defects and systematic failures; And / or (4) to count and track the frequency of various types of defects, determine whether there is an abnormality in the production line, and issue an early warning when the frequency exceeds the preset non-conforming frequency threshold.
11. A training device for a contact lens defect detection model, characterized in that, The device includes: The first acquisition unit is used to acquire real sample data and real sample annotation data. The real sample data includes real contact lens images with defects, and the real sample annotation data includes the types of defects in the real sample data. The second acquisition unit is used to acquire customized defect types; A directional sample data generation unit is used to generate directional sample data from the customized defect types using a pre-trained directional sample data generation model, wherein the samples in the directional sample data include one or more types of customized defects. The first processing unit is used to annotate the targeted generated sample data to obtain targeted generated sample labeled data. The second processing unit is used to train the contact lens defect detection model using the real sample data, the defect types of the real sample data, the targeted generated sample data, and the targeted generated sample annotation data.