Contact lens printed pattern defect detection method and system based on semi-supervised anomaly detection

Abstract

The present application relates to the technical field of machine vision, and in particular to a contact lens printing pattern defect detection method and system based on semi-supervised anomaly detection, which performs anomaly detection on color mold pad printing patterns based on a deep learning method of semi-supervised technology, and for each new pattern, only needs to input dozens of qualified product images for training, and can be online detected, which can make up for the limitations of the current method in adapting to various patterns and detecting weak defects. The semi-supervised deep learning method can break through the dependence of traditional deep learning on large-scale labeled data, use a semi-supervised learning strategy to mine potential information in unlabeled data, and improve the recognition ability of the model for complex and sparse defects. At the same time, according to the characteristics of the color mold pad printing pattern, a specific deep learning network structure and loss function are designed to ensure the sensitivity and generalization ability of the model to specific defect types.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically a method and system for detecting defects in printed patterns on contact lenses based on semi-supervised anomaly detection. Background Technology

[0002] For defect detection in colored contact lens pattern printing, current methods are based on manual visual inspection and machine vision inspection. Among them, machine vision-based methods can be further divided into traditional image processing methods and deep learning-based target detection methods. The two are generally used in combination. The former mainly detects defects that need to be compared with qualified products, while the latter mainly supplements the detection of defects with clear characteristics such as missing ink and stains.

[0003] However, among the aforementioned detection methods, manual visual inspection is highly subjective, lacks quantitative basis, and is prone to over- and under-detection due to human fatigue. Machine vision-based methods, using traditional image processing, require qualified products as reference templates. Therefore, templates need to be entered whenever a new pattern is designed and produced. Since patterns in the contact lens industry are frequently updated, each manufacturer's pattern library may contain thousands of patterns, leading to high costs for template information maintenance. Furthermore, due to the characteristics of flexographic printing, subtle differences inevitably exist between qualified products. In this case, if the threshold setting is strict, traditional methods are prone to excessive over-detection; if the threshold is lenient, minor defects are easily missed. Therefore, a compromise is usually chosen to achieve a balance between over- and under-detection. Deep learning-based target detection methods require pre-labeling and training of data. When new patterns appear, the detection results are often unpredictable, requiring continuous retraining to reduce over- and under-detection, consuming significant human and material resources. Moreover, it is often difficult to collect large quantities of defect images in industrial settings, making dataset construction challenging. Some newly emerging atypical defects (such as texture deformation and missing colors in underlying patterns) also present challenges. Figure 1 This is a diagram illustrating easily missed defects in color mold printing, such as... Figure 1 As shown, existing algorithms still have limitations in detection accuracy and robustness. These defects, which are difficult for existing algorithms to effectively identify, often cause hidden damage to brand image and user experience in actual product use. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method and system for detecting defects in printed patterns of contact lenses based on semi-supervised anomaly detection, so as to solve the problems in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The present invention provides a method for detecting defects in printed patterns of contact lenses based on semi-supervised anomaly detection, comprising the following steps:

[0007] Acquire multiple normal sample images and images to be inspected from the production line;

[0008] Preprocessing and anomaly simulation are performed on the multiple normal sample images to obtain simulated abnormal images;

[0009] A training sample set is constructed based on multiple simulated abnormal images and multiple normal sample images. An input image is selected from the training sample set, and multi-scale features of the input image are extracted based on a preset encoder. The multi-scale features of the input image are then fused and attention-enhanced based on a feature pyramid to obtain enhanced fused features. Reconstructed features are generated based on the enhanced fused features and a pre-constructed dynamic memory bank. The reconstruction error between the fused features and the reconstructed features is calculated. The reconstructed features are composed of multiple memory items in the memory bank that have the highest similarity to the fused features. The reconstruction error is used to represent the difference between the fused features and the reconstructed features.

[0010] The total loss is calculated based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank. The encoder and the feature pyramid are then adjusted based on the total loss. The input image is selected from the training sample set until the total loss is minimized, and training is completed when the total loss is minimized. The loss function includes reconstruction loss, edge loss, and contrast loss.

[0011] Based on the trained encoder, the trained feature pyramid, and the dynamic memory, the reconstruction error of the image to be detected is extracted. Based on the pre-built probability distribution model, the probability anomaly score of the reconstruction error of the image to be detected is calculated. Based on the probability anomaly scores of multiple locations of the image to be detected, defect localization detection is performed.

[0012] In one embodiment of this application, the plurality of normal sample images are preprocessed and anomaly simulations are performed to obtain simulated anomaly images, including:

[0013] Locate the center and edge of the color mold printing pattern in the normal sample image, and fit a circle based on the center and edge of the color mold printing pattern to obtain the diameter of the color mold printing pattern;

[0014] The region of interest is obtained by expanding outward by multiple pixels based on the diameter of the printed pattern; the region of interest is then scaled and normalized to obtain a preprocessed image.

[0015] The preprocessed image is subjected to severe defect simulation to obtain a severe defect simulation image; the preprocessed image is subjected to weak defect simulation to obtain a weak defect simulation image; and the severe defect simulation image and the weak defect simulation image are combined to obtain a simulated abnormal image.

[0016] In one embodiment of this application, a severe defect simulation is performed on the preprocessed image to obtain a severe defect simulation image, including:

[0017] A two-dimensional noise map is generated based on the simulated anomaly image, and the two-dimensional noise map is binarized to obtain a noise mask. The preprocessed image is then binarized to obtain the foreground mask. ;

[0018] The noise mask and the foreground mask By superimposing and combining the layers, the final mask is obtained. ;

[0019] Extracting real defects from a pre-built library of color mold printing defects. and the actual defects Image enhancement processing is performed to obtain a defect mask image. The image enhancement process includes rotation, scaling, and brightness adjustment.

[0020] The defect mask image Apply the texture to the noise mask Obtain the abnormal mask image The abnormal mask image The expression is:

[0021]

[0022] Based on the final mask and the preprocessed image Extract background , ;

[0023] Based on the aforementioned background and the abnormal mask image Generate abnormal images , ,in, The transparency coefficient is used for the abnormal image. The background described in the text and the abnormal mask image Gaussian blur fusion is performed on the edges to obtain a simulated image of severe defects.

[0024] In one embodiment of this application, a weak defect simulation is performed on the preprocessed image to obtain a weak defect simulation image, including:

[0025] The preprocessed image is converted into a LAB image in LAB space;

[0026] Extract the mask region of the printed area from the LAB image, and cut out multiple regions of interest of random shapes from the mask region. Perform color clustering on the regions of interest to obtain dark cluster centers, light cluster centers and background cluster centers.

[0027] Based on the dark cluster centers, the brightness range is expanded to obtain the dark region;

[0028] Based on the light-colored cluster centers, the dark-colored regions are transformed to a light-colored range to obtain a brightness transformation image;

[0029] The brightness transformation image is converted into an HSV image, the saturation of the HSV image is reduced by a target value, and then converted into an RGB image to obtain the region of interest for the defect.

[0030] The preprocessed image and the region of interest of the defect are combined using Gaussian fusion mapping to obtain a simulated image of the weak defect.

[0031] In one embodiment of this application, the method for constructing the dynamic memory includes:

[0032] A feature encoder based on freezing parameters extracts multi-scale feature maps from multiple normal sample images. The feature encoder is pre-trained using an image dataset of color mold transfer printing.

[0033] For the multi-scale feature map L2 normalization is performed to obtain normalized feature maps. The normalized feature map The mathematical expression is:

[0034]

[0035] In the formula, To prevent division by zero;

[0036] For the normalized feature map Vectorization is performed to obtain the feature vector. Based on multiple feature vectors Construct a set of feature vectors , where the feature vector and the set of feature vectors The mathematical expressions are as follows:

[0037]

[0038]

[0039] In the formula, Let C represent the feature vector corresponding to each spatial location (x, y), where C is the dimension of the feature channel. Normalized feature map of high, Normalized feature map width, It is a natural number;

[0040] Based on the feature vector The feature vector Feature weights and the feature vector Last update timestamp Constructing memory items And construct memory information based on N memory items. ,in, , The feature weights Pre-allocation;

[0041] Introducing a time decay mechanism to update the memory item Feature weights at the current time The memory item Feature weights at the current time The formula for calculation is:

[0042]

[0043] In the formula, For memory items The feature weights at the previous time step, The attenuation coefficient is... For the current moment With the last update timestamp The difference;

[0044] The feature weights of all memory items at the current time are compared with a preset weight threshold, and memory items whose feature weights at the current time are lower than the weight threshold are removed, thereby modifying the memory information. Update;

[0045] An update mechanism is introduced to extract the feature vector of a new normal sample image when it is input. And calculate the feature vector With updated memory information The cosine similarity of all memory items is used to filter out the items with the highest cosine similarity, and the feature vector is calculated based on the highest cosine similarity. The novelty value Wherein, the smaller the value of the maximum cosine similarity, the stronger the feature vector. The novelty value The larger;

[0046] In the feature vector The novelty value If the novelty threshold is greater than the preset threshold, determine whether the number of memory items in the current memory information is full. If not, change the feature vector. Add to the memory information If so, calculate the elimination score of all memory items in the current memory information, and based on the feature vector. Replace the memory entry with the lowest elimination score and update the timestamp to update the memory information. The update is performed, wherein the elimination score is calculated using the following formula:

[0047]

[0048]

[0049] In the formula, Indicates the first The elimination score for each memory item.

[0050] In one embodiment of this application, the total loss is calculated based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank, including:

[0051] The reconstruction loss is calculated based on the reconstruction error of the reconstruction features and enhancement fusion features of the input image. The edge loss is calculated based on the enhanced fusion features of the input image and the dynamic memory. Calculate the contrastive loss based on the training sample set. and based on the reconstruction loss The edge loss And the comparison loss Calculate total loss The reconstruction loss The edge loss The contrast loss The total loss mentioned above The mathematical expressions are as follows:

[0052]

[0053]

[0054]

[0055]

[0056] In the formula, The total number of feature vectors in the memory. To enhance fusion features, For the first in the dynamic memory bank One memory item, For edge threshold, This represents a set of positive samples consisting of multiple normal sample images. Represents the training sample set, Represents samples in the positive sample set The feature vectors are the feature vectors of the samples used in the training. Represents the samples in the total sample set eigenvectors, This represents the feature vector of the sample participating in the training. As the first weight, As the second weight, It is the third weight.

[0057] In one embodiment of this application, the method for calculating the reconstruction features and reconstruction errors includes:

[0058] Calculate query features Cosine similarity to all memory items in the dynamic memory bank The query feature is either an enhanced fusion feature of the input image or an enhanced fusion feature of the image to be detected.

[0059] Cosine similarity of all memory items We take the K largest terms from the set and weight them to obtain the reconstructed features. The reconstructed features The mathematical expression is:

[0060]

[0061] in, The weighting coefficients decrease linearly according to the ranking and can be calculated using the following formula:

[0062]

[0063] Calculate the reconstructed features With the query features Reconstruction error The reconstruction error The mathematical expression is: .

[0064] In one embodiment of this application, calculating the probability anomaly score of the reconstruction error of the image to be detected based on a pre-built probability distribution model includes:

[0065] The reconstruction error of the image to be detected is fed into a pre-constructed probability distribution model to obtain a probability anomaly score. , ,in, This represents a pre-constructed probability distribution model, wherein the method for constructing the probability distribution model includes:

[0066] Construct a mixture model consisting of K Gaussian distributions. The mathematical expression of the mixture model is:

[0067]

[0068] In the formula, e This represents the scalar value of the reconstruction error. Indicates the first The mixing coefficients of a Gaussian distribution. Indicates the first The mean reconstruction error of a Gaussian distribution, Indicates the first The reconstruction error variance of a Gaussian distribution;

[0069] The reconstruction error of multiple normal sample images is estimated by combining the EM algorithm. The mixing coefficient of a Gaussian distribution , No. Mean reconstruction error of a Gaussian distribution and the Reconstruction error variance of a Gaussian distribution Thus, a probability distribution model is obtained.

[0070] In one embodiment of this application, defect localization detection is performed based on probability anomaly scores at multiple locations in the image to be detected, including:

[0071] The probability anomaly score Compare with the set anomaly score threshold and assign the probability anomaly score. Locations with an abnormal score exceeding the set threshold are identified as defect locations.

[0072] This application also provides a contact lens printing pattern defect detection system based on semi-supervised anomaly detection, including:

[0073] The acquisition module is used to acquire multiple normal sample images and images to be detected in the production line;

[0074] An anomaly simulation module is used to preprocess and simulate anomalies in the multiple normal sample images to obtain simulated anomaly images;

[0075] The feature extraction module is used to construct a training sample set based on multiple simulated abnormal images and multiple normal sample images, select an input image from the training sample set, extract multi-scale features of the input image based on a preset encoder, fuse the multi-scale features of the input image based on a feature pyramid to obtain enhanced fused features, generate reconstructed features based on the enhanced fused features and a pre-built dynamic memory bank, and calculate the reconstruction error between the fused features and the reconstructed features. The reconstructed features are based on multiple memory items in the memory bank that have the highest similarity to the fused features, and the reconstruction error is used to represent the difference between the fused features and the reconstructed features.

[0076] The training module is used to calculate the total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank; adjust the parameters of the encoder and the feature pyramid based on the total loss; select an input image from the training sample set until the total loss is minimized; and complete the training when the total loss is minimized. The loss function includes reconstruction loss, edge loss, and contrast loss.

[0077] The defect detection module is used to extract the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid and the dynamic memory, calculate the probability anomaly score of the reconstruction error of the image to be detected based on the pre-built probability distribution model, and perform defect localization detection based on the probability anomaly scores of multiple locations of the image to be detected.

[0078] The beneficial effects of this invention are as follows: This invention provides a method and system for detecting defects in printed contact lens patterns based on semi-supervised anomaly detection. This application utilizes a semi-supervised deep learning method for anomaly detection in color mold transfer printing patterns. For each new pattern, only a few dozen frames of qualified product images need to be recorded for training before online detection can begin. This overcomes the limitations of current methods in adapting to various patterns and detecting minor defects. The semi-supervised deep learning method overcomes the dependence of traditional deep learning on large-scale labeled data. It utilizes semi-supervised learning strategies to mine potential information in unlabeled data, improving the model's ability to identify complex and sparse defects. Furthermore, considering the characteristics of color mold transfer printing patterns, a targeted deep learning network structure and loss function are designed to ensure the model's sensitivity and generalization ability to specific defect types. Attached Figure Description

[0079] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0080] Figure 1 A schematic diagram illustrating defects that are easily missed in color mold printing;

[0081] Figure 2This is a flowchart illustrating the construction process of a dynamic memory bank in one embodiment of this application;

[0082] Figure 3 This is a flowchart illustrating a method for detecting defects in printed contact lens patterns based on semi-supervised anomaly detection in one embodiment of this application;

[0083] Figure 4 This is a flowchart illustrating a severe defect simulation in one embodiment of this application;

[0084] Figure 5 This is a schematic flowchart illustrating the simulation of a weak defect in one embodiment of this application;

[0085] Figure 6 The image shows a simulated image of a weak defect as illustrated in the embodiments of this application;

[0086] Figure 7 This is a flowchart illustrating the training phase in one embodiment of this application;

[0087] Figure 8 This is a schematic diagram of the pre-training of the ConvNeXt-T model in one embodiment of this application;

[0088] Figure 9 This is a flowchart illustrating the reasoning process in one embodiment of this application;

[0089] Figure 10 This is a diagram illustrating the effect of weak defect detection in one embodiment of this application;

[0090] Figure 11 This is a schematic diagram of the structure of a multi-scale feature pyramid fusion strategy in one embodiment of this application;

[0091] Figure 12 This is a schematic diagram of the attention mechanism structure in one embodiment of this application;

[0092] Figure 13 This is a flowchart illustrating a method for detecting defects in printed contact lens patterns based on semi-supervised anomaly detection in one embodiment of this application. Detailed Implementation

[0093] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0094] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0095] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of the invention; however, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details.

[0096] In real-world industrial quality inspection scenarios, obtaining a large number of accurately labeled anomalous samples is typically expensive and difficult. Conversely, collecting a large amount of unlabeled data and a small number of known normal or mostly normal samples is relatively easy. Semi-supervised learning-based anomaly detection techniques are an important class of methods developed to address this core challenge. They only require a small amount of labeled normal sample data combined with a large amount of unlabeled data to jointly learn a more robust and accurate model to identify anomalies.

[0097] Current mainstream semi-supervised anomaly models can be divided into three categories: reconstruction-based approaches, feature embedding-based approaches, and vision-language model-based approaches. Among them, feature embedding-based technology is one of the most advanced and mainstream technologies in the field of industrial anomaly detection. Its core idea is to utilize the powerful representational capabilities of pre-trained deep neural networks to map input data into a high-dimensional feature embedding space rich in semantic information. In this space, the features of normal data will cluster into a compact and clearly patterned distribution, while the features of abnormal data, due to their unknown patterns, will deviate from this normal feature distribution area. By modeling the "normal" distribution in the feature space and calculating the degree of deviation between the test sample features and this distribution, anomalies can be detected and located. The advantages of this method are its powerful feature representation capabilities, its focus on semantic deviation rather than attempting to reconstruct pixel-level details, its high-precision localization capability, its efficient training, its fundamental avoidance of reconstruction pitfalls, and its excellent performance on industrial inspection datasets. Therefore, we chose this approach to improve the performance of color mold printing defect detection and fine-tuned the model for the characteristics of the color mold printing dataset.

[0098] Meanwhile, a module for simulating abnormal data is introduced, using algorithms to generate data instead of manual collection, thus solving the "abnormal sample shortage" dilemma in industrial scenarios. Furthermore, parameters can be controlled to cover a wide variety of anomalies, narrowing the distribution gap between simulated and real defects and improving the model's generalization ability, enabling the identification of even unseen defects. Compared to models without anomaly simulation, models incorporating the anomaly simulation module can explicitly learn the distinguishing features between normal and abnormal conditions, avoiding missed detections caused by modeling only normal patterns and relying on reconstruction errors or feature deviations to judge anomalies. Additionally, anomalies can be customized to suit specific scene characteristics, such as being insensitive to changes in the rotation or translation of colored models.

[0099] In summary, by learning from differences and commonalities, and combining a dynamic memory database of artificially simulated abnormal and normal samples, anomaly detection is transformed into a semantic segmentation task, as described in the following text.

[0100] Before formally starting semi-supervised training, this application needs to build a dynamic memory in advance. The memory is usually fixed and used to store the feature information of normal samples. Considering that the consistency of flexible printing is not high enough, there are usually slight differences between normal samples of color molds. Moreover, as the production time increases, the features of normal samples will gradually change and deviate from the initial feature distribution. Therefore, we adopt a dynamically updatable memory to achieve more accurate detection.

[0101] Figure 2 This is a flowchart illustrating the construction process of the dynamic memory in one embodiment of this application, as follows: Figure 2 As shown, the construction process of the dynamic memory is as follows:

[0102] (1) Extracting multi-scale feature maps from multiple normal sample images using a feature encoder based on freeze parameters The feature encoder is pre-trained using an image dataset of color mold transfer printing.

[0103] First, N normal samples are randomly selected, and ConvNeXt-T with frozen parameters is used as the feature encoder to extract multi-scale features. In our application, N is set to 100. Furthermore, a feature pyramid can be used to fuse the multi-scale feature maps F.

[0104] The feature map extracted by ConvNeXt-T is denoted by F, where the height, width, and number of channels are denoted by H, W, and C, respectively.

[0105] (2) For the multi-scale feature map L2 normalization is performed to obtain normalized feature maps. The normalized feature map The mathematical expression is:

[0106]

[0107] In the formula, To prevent division by zero, a parameter is introduced. To prevent division by zero.

[0108] (3) The normalized feature map Vectorization is performed to obtain the feature vector. Based on multiple feature vectors Construct a set of feature vectors , where the feature vector and the set of feature vectors The mathematical expressions are as follows:

[0109]

[0110]

[0111] In the formula, Let C represent the feature vector corresponding to each spatial location (x, y), where C is the dimension of the feature channels, which is 768 dimensions in ConvNeXt-T. Normalized feature map of high, Normalized feature map width, It is a natural number; compared with feature maps, the advantage of storing feature vectors is that they can be stored and retrieved independently, and can be matched locally, saving storage space.

[0112] (4) Based on the feature vector The feature vector Feature weights and the feature vector Last update timestamp Constructing memory items And construct memory information based on N memory items. ,in, , The feature weights Pre-allocation;

[0113] The i-th memory item in the dynamic memory bank Three types of information need to be stored, including feature vectors. Feature weights and the last updated timestamp Therefore, each memory term contains a feature vector. Feature weights and the last updated timestamp Multiple memory items constitute the initial dynamic memory bank.

[0114] (5) A time decay mechanism is introduced. The automatic updating of the dynamic memory bank follows the principles of time decay and feature novelty to ensure that feature changes in normal samples can update the memory bank in real time. (Update the memory items) Feature weights at the current time The memory item Feature weights at the current time The formula for calculation is:

[0115]

[0116] In the formula, For memory items The feature weights at the previous time step, The attenuation coefficient is... For the current moment With the last update timestamp The difference;

[0117] The feature weights of all memory items at the current time are compared with a preset weight threshold, and memory items whose feature weights at the current time are lower than the weight threshold are removed, thereby modifying the memory information. Update;

[0118] (6) Introduce an update mechanism to extract the feature vector of the new normal sample image when a new normal sample image is input (e.g., during subsequent training). And calculate the feature vector With updated memory information The cosine similarity of all memory items is used to filter out the items with the highest cosine similarity, and the feature vector is calculated based on the highest cosine similarity. The novelty value Wherein, the smaller the value of the maximum cosine similarity, the stronger the feature vector. The novelty value The larger;

[0119] In the feature vector The novelty value If the novelty threshold is greater than the preset threshold, it is necessary to further determine whether the number of memory items in the current memory information is full. If not, the feature vector is... Add to the memory information If so, calculate the elimination score of all memory items in the current memory information, and based on the feature vector. Replace the memory entry with the lowest elimination score and update the timestamp to update the memory information. The update is performed, wherein the elimination score is calculated using the following formula:

[0120]

[0121] In the formula, Indicates the first The elimination score for each memory item.

[0122] Update the timestamp afterwards.

[0123]

[0124]

[0125] Based on the above process, this application constructs a dynamic memory library that is dynamically updated based on time decay and novelty judgment to adapt to the situation where the characteristics of normal samples gradually change as production time increases. It can effectively adapt to the pattern updates and iterations in the colored contact lens industry and has the advantage of strong adaptability.

[0126] Figure 3 This is a flowchart illustrating a method for detecting defects in contact lens printing patterns based on semi-supervised anomaly detection, as shown in one embodiment of this application. Figure 3 As shown, the method for detecting defects in printed contact lens patterns based on semi-supervised anomaly detection in this embodiment includes a training phase and an inference phase. The specific process includes:

[0127] (a) Training Phase

[0128] A. Acquire multiple normal sample images and the image to be detected from the production line;

[0129] First, 100 normal sample batch images were collected from the production line as normal sample images. The images to be tested are those that will be used later when it is unknown whether they contain defects.

[0130] B. Preprocessing and anomaly simulation are performed on the multiple normal sample images to obtain simulated abnormal images;

[0131] Normal images undergo preprocessing, which includes the following steps:

[0132] Preprocessing and anomaly simulation methods include:

[0133] (1) Pretreatment

[0134] B1, Locate the center and edge of the color mold printing pattern in the normal sample image, and fit a circle based on the center and edge of the color mold printing pattern to obtain the diameter of the color mold printing pattern;

[0135] B2, using the diameter of the printed pattern as a reference, expand outward by multiple pixels to obtain the region of interest; perform scaling and normalization processing on the region of interest to obtain a preprocessed image.

[0136] During the preprocessing process, the center and edge of the color mold printing pattern are located, the diameter of the pattern is calculated using the method of fitting a circle, and the padding is expanded by 10 pixels according to the diameter of the circle to extract the ROI. The size of the ROI is reduced to 256×256 and normalized as input data.

[0137] (2) Abnormal simulation

[0138] For semi-supervised anomaly detection models, since only normal samples are input, the quality of the simulated anomaly samples plays a decisive role in the final model performance. If the simulated anomaly image can well represent the real anomaly, then the model's ability to detect the real anomaly will be good. Conversely, if the simulated anomaly differs greatly from the real anomaly, then the model may perform poorly in detecting the real anomaly. Therefore, we focused on optimizing the simulation method for color mold printing defects, especially weak defects. A common method is to add Perlin noise to the image. This defect has very strong contrast, which is not conducive to the detection of weak defects. Therefore, it is necessary to optimize the anomaly simulation scheme based on the characteristics of color mold printing defects.

[0139] Therefore, the abnormal simulation in this application includes severe defect simulation and minor defect simulation. The execution order can be changed, but it needs to be combined with severe defect simulation and minor defect simulation.

[0140] (2-1) Severe Defect Simulation

[0141] Figure 4 This is a flowchart illustrating a severe defect simulation in one embodiment of this application. The specific process is as follows: Figure 4 As shown, it includes:

[0142] B3. Generate a two-dimensional noise map based on the simulated abnormal image, and binarize the two-dimensional noise map to obtain a noise mask. The preprocessed image is then binarized to obtain the foreground mask. ;

[0143] Generate a 2D Perlin noise map of the same size as the original image, binarize it, and obtain the mask. The input color model image is binarized to obtain the foreground mask. .

[0144] B4, the noise mask and the foreground mask By superimposing and combining the layers, the final mask is obtained. ;

[0145] B5, extracting real defects from a pre-built library of color mold printing defects. and the actual defects Image enhancement processing is performed to obtain a defect mask image. The image enhancement process includes rotation, scaling, and brightness adjustment.

[0146] Sample real defects from a real color mold printing defect database. Furthermore, by employing image enhancement methods, random operations such as rotation, scaling, and brightness adjustment are performed on the defect texture to obtain a defect mask image. .

[0147] B6, the defect mask image Apply the texture to the noise mask Obtain the abnormal mask image The abnormal mask image The expression is:

[0148] B7, based on the final mask and the preprocessed image Extract background , The transparency coefficient δ∈[0.15, 1] ​​is used to control the blending intensity.

[0149] B8, based on the aforementioned background and the abnormal mask image Generate abnormal images , ,in, The transparency coefficient is used for the abnormal image. The background described in the text and the abnormal mask image Gaussian blur is applied to the edges to obtain a simulated image of severe defects. Gaussian blur edge blending makes the transition more realistic and avoids synthetic artifacts.

[0150] (2-2) Simulation of weak defects

[0151] Based on the characteristics of the color mold pattern, the simulation of minor defects was added. The purpose was to replace the dark spots in the pattern with light colors to simulate the abnormality of missing single-layer ink. Figure 5 This is a schematic diagram of a weak defect simulation process in one embodiment of this application, as shown below. Figure 5 As shown, the specific process is as follows:

[0152] B9, convert the preprocessed image into a LAB image in LAB space; in LAB space, dimension L represents brightness, and a and b represent color-opposite dimensions.

[0153] B10, extract the mask region of the printing area from the LAB image, and cut out multiple regions of interest of random shapes within the mask region. Perform brightness clustering on the regions of interest to obtain dark cluster centers, light cluster centers and background cluster centers.

[0154] After converting sample image I to a LAB image, the mask area of ​​the printed area is extracted, and several small ROIs of random shapes are cut out within the mask area for color clustering. Cluster centers are obtained. Assuming that color points can be divided into dark and light colors, and adding the whitish background color of the unprinted color points, the number of clusters is set to 3. The three cluster centers are arranged in ascending order of brightness L. The brightness value of the first cluster center represents the brightness of the dark color, the second represents the brightness of the light color center, and the third represents the background color.

[0155] B11, based on the dark cluster centers, the brightness range is expanded to obtain dark regions;

[0156] The dark regions R within a brightness range of ±20 are obtained by using a fixed threshold operation to identify cluster centers in a small ROI.

[0157] B12, based on the light-colored cluster centers, the dark-colored regions are transformed to a light-colored range to obtain a brightness transformation image;

[0158] The brightness of R is transformed as follows: (R / Dark center brightness) * Light center brightness. By transforming the color, the brightness of the dark area is transformed to the light area to simulate the abnormality of missing single layer of ink.

[0159] B13, convert the brightness transformation image into an HSV image, reduce the saturation of the HSV image by a target value, and convert it into an RGB image to obtain the region of interest for the defect;

[0160] B14. Perform Gaussian fusion mapping between the preprocessed image and the region of interest of the defect to obtain a simulated image of the weak defect.

[0161] Finally, the brightness-transformed image is converted to an HSV image, the saturation S is reduced by 50, and it is converted to an RGB image to obtain the defect ROI. Then, Gaussian fusion is used to map the sample image to the defect ROI.

[0162] Figure 6 The image shown in the embodiments of this application is a simulated image of a weak defect. Figure 6 The right side shows regions of interest with multiple random shapes cut out from the mask area, and the right side shows a simulated image of weak defects.

[0163] The two methods mentioned above need to be used together to achieve a simulation effect that takes into account both obvious severe anomalies and weak anomalies.

[0164] After obtaining the simulated abnormal images, they can be combined with normal images to form a training sample set, and then training can be performed. Figure 7 This is a schematic diagram of the training phase in one embodiment of this application, as shown below. Figure 7 As shown, it includes:

[0165] C. A training sample set is constructed based on multiple simulated abnormal images and multiple normal sample images. An input image is selected from the training sample set, and multi-scale features of the input image are extracted based on a preset encoder. The multi-scale features of the input image are fused based on a feature pyramid to obtain enhanced fused features. Reconstructed features are generated based on the enhanced fused features and a pre-constructed dynamic memory bank. The reconstruction error between the fused features and the reconstructed features is calculated. The reconstructed features are based on multiple memory items in the memory bank that have the highest similarity to the fused features. The reconstruction error is used to represent the difference between the fused features and the reconstructed features.

[0166] A training sample set is constructed by mixing normal sample images and simulated abnormal images. The normal sample images and simulated abnormal images in the training sample set can be labeled to distinguish between normal and abnormal.

[0167] In this application, an encoder is used for multi-scale feature extraction. The encoder used for feature extraction is ConvNeXt-T, which was pre-trained using an image dataset of color mold transfer printing. Figure 8 This is a schematic diagram of the pre-training of the ConvNeXt-T model in one embodiment of this application, as shown below. Figure 8 As shown, the image dataset for color mold transfer printing contains 150 patterns and a total of 60,000 images.

[0168] The extracted multi-scale features are then fused using feature pyramids and attention enhancement to obtain enhanced fused features.

[0169] A pre-built dynamic memory is used to generate reconstructed features of the enhanced fusion features, and the reconstruction error between the enhanced fusion features and the reconstructed features is calculated. Since feature pyramid fusion, reconstructed features, and reconstruction errors also need to be calculated in subsequent inference processes, to reduce repetition and avoid redundancy, the specific processes of feature pyramid fusion, calculation of reconstructed features, and reconstruction errors are statistically explained after the inference process.

[0170] The reconstructed features are obtained by weighting multiple memory entries in the dynamic memory bank that are closest to the enhanced fusion features. In other words, the reconstruction error represents the difference between the enhanced fusion features and the feature maps of normal samples. This difference is used in conjunction with backpropagation updates to update the parameters of the encoder and the feature pyramid. The training process primarily aims to train the feature extractor (encoder), enabling it to approximate features from normal samples and de-apply features from anomalous samples, thus constructing a discriminative embedding space.

[0171] D, calculate the total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank, adjust the parameters of the encoder and the feature pyramid based on the total loss, and return to select the input image from the training sample set until the total loss is minimized, and complete the training when the total loss is minimized, wherein the loss function includes reconstruction loss, edge loss and contrast loss;

[0172] Specifically, the reconstruction loss is calculated based on the reconstruction error of the reconstruction features and the enhanced fusion features of the input image. The edge loss is calculated based on the enhanced fusion features of the input image and the dynamic memory. Calculate the contrastive loss based on the training sample set. and based on the reconstruction loss The edge loss And the comparison loss Calculate total loss The reconstruction loss The edge loss The contrast loss The total loss mentioned above The mathematical expressions are as follows:

[0173]

[0174]

[0175]

[0176] In the formula, The number of pixels in the input image. To enhance fusion features, Let m be the i-th memory item in the dynamic memory bank, and m be the edge threshold. This represents a set of positive samples consisting of multiple normal sample images. Represents the training sample set, Represents samples in the positive sample set eigenvectors, Represents the samples in the total sample set eigenvectors, This represents the feature vector of the sample participating in the training. As the first weight, As the second weight, It is the third weight.

[0177] in, , , The values ​​were set to 1.0, 0.5, and 0.8 respectively. Reconstruction loss dominated the entire process, while contrast loss could accelerate convergence in the early stages.

[0178] The reconstruction loss aims to minimize the difference between the features of normal samples and the features reconstructed in the memory bank; the edge loss aims to ensure that the features of normal samples can find sufficiently similar memory entries in the memory bank, while abnormal samples cannot, and hopes that their similarity with the nearest neighbor normal features in the memory bank is less than the edge threshold m, thereby pushing abnormal image features away from normal features; the contrast loss aims to learn feature representations by bringing samples of the same class closer together and pushing samples of different classes apart.

[0179] Based on the above loss, the encoder and feature pyramid are trained in this application so that the encoder can bring the features of normal samples closer together and push away the features of abnormal samples, making the features of normal samples more compact (low intra-class divergence) and the features of abnormal samples more dispersed (high inter-class divergence), thus constructing a discriminative embedding space.

[0180] Furthermore, during the training phase, the reconstruction errors of all normal samples at various locations on the feature map constitute an error set. This error set is used to fit a Gaussian Mixture Model (GMM) for probabilistic modeling. We assume that the reconstruction errors of normal samples follow a mixture model consisting of K Gaussian distributions, while the reconstruction errors of abnormal samples deviate from this model. The probability density function of the GMM is...

[0181] Where e represents the scalar value of the reconstruction error, and there is a corresponding e for each position (x, y) on the feature map; K represents the number of Gaussian distributions, with a value of 3. Let the mixing coefficients of the k-th Gaussian distribution satisfy the following condition:

[0182]

[0183] The mean is The variance is The probability density function of the Gaussian distribution is used. The parameters are estimated using the EM algorithm. , and This means the modeling was successful.

[0184] (II) Reasoning Process

[0185] After training is complete, the trained encoder and feature pyramid can be used to perform the inference process. Figure 9 This is a flowchart illustrating the reasoning process in one embodiment of this application, as shown below. Figure 9 As shown, the specific process is as follows.

[0186] E. Based on the trained encoder, the trained feature pyramid, and the dynamic memory, the reconstruction error of the image to be detected is extracted, and the probability anomaly score of the reconstruction error of the image to be detected is calculated based on the pre-built probability distribution model. Defect localization detection is performed based on the probability anomaly scores of multiple locations of the image to be detected.

[0187] The process of extracting the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid, and the dynamic memory can be referred to the preceding text, including:

[0188] E1 extracts multi-scale features of the image to be detected based on the trained encoder;

[0189] E2, based on the trained feature pyramid, fuses multi-scale features of the image to be detected to obtain enhanced fused features;

[0190] E3, calculates the reconstructed features of the enhanced fusion features based on the dynamic memory bank;

[0191] E4, calculates the reconstruction error between the reconstructed features and the enhanced fusion features;

[0192] E5, then the reconstruction error of the image to be detected is fed into the pre-built probability distribution model to obtain the probability anomaly score. , ;

[0193] E6, based on probability anomaly scores The anomaly score map is obtained, and then threshold segmentation is performed on the anomaly score map to obtain the defect mask. The result is then output. Figure 10 This is a diagram illustrating the effect of weak defect detection in one embodiment of this application. The final output result obtained from the segmentation is as follows: Figure 10 As shown.

[0194] In one specific embodiment, the scheme of this application is used to verify and test 20 different pattern color mold images. Each time the pattern is changed, it is necessary to retrain with the normal image of the new pattern. The model can meet the requirements of online learning. The collection and training of 100 images on the production line takes no more than 10 minutes.

[0195] By setting a reasonable threshold for outlier scores, defective products can be directly identified and removed. The final outlier score graph A is obtained by normalizing using the sigmoid function.

[0196]

[0197] in, , representing coordinates The probability of an anomaly, finally, take The pixel was identified as abnormal.

[0198] This application constructs an error set, uses this error set to fit a Gaussian mixture model (GMM) for probabilistic modeling, and uses this probabilistic model to convert the reconstruction error of the image to be detected into the defect probability at each location, thereby extracting the region with a higher defect probability and realizing defect localization.

[0199] (III) Introduction to the specific calculation process of reconstructed features and reconstruction errors:

[0200] Both the training and inference phases involve calculating reconstructed features and reconstruction errors. Reconstruction error represents the difference between the features of a normal sample and the features reconstructed from the memory. The dynamic memory is automatically updated while the reconstruction error is being calculated.

[0201] Calculate query features Cosine similarity to all memory items in the dynamic memory bank , wherein the query features The enhanced fusion features are either the input image or the image to be detected.

[0202]

[0203] Cosine similarity of all memory items We take the K largest terms from the set and weight them to obtain the reconstructed features. The reconstructed features The mathematical expression is:

[0204]

[0205] in, The weighting coefficients decrease linearly according to the ranking and can be calculated using the following formula:

[0206]

[0207] Calculate the reconstructed features With the query features Reconstruction error The reconstruction error The mathematical expression is:

[0208]

[0209] (iv) Introduction to Feature Pyramid Integration:

[0210] The feature information stored in the dynamic memory has the problem of feature redundancy, and it increases the computational load of the model and reduces the inference speed. Therefore, visual and semantic information can be globally fused by introducing attention mechanism and multi-scale feature pyramid fusion strategy. Figure 11 This is a schematic diagram of the structure of a multi-scale feature pyramid fusion strategy in one embodiment of this application, as shown below. Figure 11 As shown:

[0211] in, , , , These are the outputs of stages 1, 2, 3, and 4 of the feature encoder, respectively. , , , For the corresponding pyramid features, 1×1 convolution is used to compress the channel dimension, and after upsampling to align the resolution, the elements are added one by one to fuse semantics and details, achieving multi-scale feature fusion of 1 / 4, 1 / 8, and 1 / 16, while optimizing the matching efficiency of the memory bank.

[0212]

[0213] Furthermore, by introducing a channel- and spatial attention mechanism and fusing it with the feature pyramid, key features can be further enhanced. At the same time, cross-scale attention can achieve information complementarity, achieving a balance between computational efficiency and accuracy. Figure 12 This is a schematic diagram of the attention mechanism structure in one embodiment of this application, as shown below. Figure 12 As shown, channel attention enhancement is performed first. :

[0214] Further enhance spatial attention :

[0215]

[0216] Original features This represents the features after channel attention enhancement. This represents the features enhanced by spatial attention.

[0217] This invention discloses a method for detecting defects in printed contact lens patterns based on semi-supervised anomaly detection. This application utilizes a semi-supervised deep learning approach for anomaly detection in color-molded transfer printing patterns. For each new pattern, only a few dozen frames of qualified product images need to be recorded for training before online detection can begin. This overcomes the limitations of current methods in adapting to various patterns and detecting subtle defects. The semi-supervised deep learning method overcomes the dependence of traditional deep learning on large-scale labeled data, utilizing semi-supervised learning strategies to mine potential information in unlabeled data, improving the model's ability to identify complex and sparse defects. Furthermore, a targeted deep learning network structure and loss function are designed specifically for the characteristics of color-molded transfer printing patterns, ensuring the model's sensitivity and generalization ability to specific defect types.

[0218] like Figure 13 As shown, this application also provides a contact lens printing pattern defect detection system based on semi-supervised anomaly detection, comprising:

[0219] The acquisition module is used to acquire multiple normal sample images and images to be detected in the production line;

[0220] An anomaly simulation module is used to preprocess and simulate anomalies in the multiple normal sample images to obtain simulated anomaly images;

[0221] The feature extraction module is used to construct a training sample set based on multiple simulated abnormal images and multiple normal sample images, select an input image from the training sample set, extract multi-scale features of the input image based on a preset encoder, fuse the multi-scale features of the input image based on a feature pyramid to obtain enhanced fused features, generate reconstructed features based on the enhanced fused features and a pre-built dynamic memory bank, and calculate the reconstruction error between the fused features and the reconstructed features. The reconstructed features are based on multiple memory items in the memory bank that have the highest similarity to the fused features, and the reconstruction error is used to represent the difference between the fused features and the reconstructed features.

[0222] The training module is used to calculate the total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank; adjust the parameters of the encoder and the feature pyramid based on the total loss; select an input image from the training sample set until the total loss is minimized; and complete the training when the total loss is minimized. The loss function includes reconstruction loss, edge loss, and contrast loss.

[0223] The defect detection module is used to extract the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid and the dynamic memory, calculate the probability anomaly score of the reconstruction error of the image to be detected based on the pre-built probability distribution model, and perform defect localization detection based on the probability anomaly scores of multiple locations of the image to be detected.

[0224] This invention discloses a defect detection system for contact lens printing patterns based on semi-supervised anomaly detection. This application utilizes a semi-supervised deep learning method for anomaly detection in color mold transfer printing patterns. For each new pattern, only a few dozen frames of qualified product images need to be recorded for training before online detection can begin. This overcomes the limitations of current methods in adapting to various patterns and detecting subtle defects. The semi-supervised deep learning method overcomes the dependence of traditional deep learning on large-scale labeled data, utilizing semi-supervised learning strategies to mine potential information in unlabeled data, improving the model's ability to identify complex and sparse defects. Furthermore, a targeted deep learning network structure and loss function are designed specifically for the characteristics of color mold transfer printing patterns, ensuring the model's sensitivity and generalization ability to specific defect types.

[0225] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0226] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0227] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0228] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0229] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0230] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0231] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended description.

[0232] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the present invention.

Claims

1. A method for contact lens printed pattern defect detection based on semi-supervised anomaly detection, characterized in that, The method comprises the following steps: Obtain a plurality of normal sample images and a to-be-detected image in a production line; Preprocess and simulate abnormalities for the plurality of normal sample images to obtain simulated abnormal images; Preprocess and simulate abnormalities for the plurality of normal sample images to obtain simulated abnormal images, including: locating the center and edge of a color module printing pattern in the normal sample image, fitting a circle based on the center and edge of the color module printing pattern to obtain the diameter of the color module printing pattern; expanding a plurality of pixels outward based on the diameter of the color module printing pattern to obtain a region of interest; performing scaling and normalization processing on the region of interest to obtain a preprocessed image; simulating severe defects on the preprocessed image to obtain a severe defect simulation image; simulating weak defects on the preprocessed image to obtain a weak defect simulation image; and combining the severe defect simulation image and the weak defect simulation image to obtain a simulated abnormal image; Based on a plurality of simulated abnormal images and a plurality of normal sample images, a training sample set is constructed, and an input image is selected from the training sample set. Based on a pre-set encoder, multi-scale features of the input image are extracted, and the multi-scale features of the input image are fused and attention-enhanced based on a feature pyramid to obtain enhanced fusion features. Based on the enhanced fusion features and a pre-constructed dynamic memory bank, reconstruction features are generated, and a reconstruction error between the enhanced fusion features and the reconstruction features is calculated. The reconstruction features are composed of a plurality of memory items in the memory bank that have the highest similarity to the enhanced fusion features. The reconstruction error represents the difference between the enhanced fusion features and the reconstruction features. The construction process of the dynamic memory bank includes: extracting a vector of multi-scale features from a plurality of normal sample images to obtain memory information; introducing a time decay mechanism to determine the weight of each memory item in the memory information, and removing memory items with a weight lower than a weight threshold. The earlier the timestamp, the lower the weight of the memory item. When a new normal sample image is input, the maximum similarity between the feature vector of the new normal sample image and the memory information is calculated. Based on the maximum similarity, novelty is calculated. Based on the feature vector of the new normal sample image, the state of whether the number of memory items in the current memory information is full, and the weight of the memory item corresponding to the maximum similarity, the memory information is added or replaced. Based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank, a total loss is calculated. Based on the total loss, the parameters of the encoder and the feature pyramid are adjusted. The input image is selected from the training sample set until the total loss is minimized. When the total loss is minimized, the training is completed. The total loss includes reconstruction loss, edge loss, and contrast loss. The reconstruction error of the to-be-detected image is extracted based on the trained encoder, the trained feature pyramid, and the dynamic memory bank, and a probability anomaly score of the reconstruction error of the to-be-detected image is calculated based on a pre-constructed probability distribution model, and defect positioning detection is performed based on the probability anomaly scores of multiple positions of the to-be-detected image.

2. The method of claim 1, wherein the method is based on semi-supervised anomaly detection. The preprocessed image is subjected to severe defect simulation to obtain a severe defect simulation image, including: generating a two-dimensional noise map based on the simulated abnormal image, and binarizing the two-dimensional noise map to obtain a noise mask ; and binarizing the pre-processed image to obtain a foreground mask ; superimposing the noise mask and the foreground mask in combination to obtain a final mask ; Extracting real defects from a pre-constructed color mode printing defect library and performing image enhancement processing on the real defects to obtain a defect mask image wherein the image enhancement processing includes rotation, scaling and brightness adjustment; applying the defect mask image to the noise mask , to obtain an anomaly mask image wherein the anomaly mask image has an expression of: based on the final mask and the pre-processed image extracting the background , ; based on the background and the anomaly mask image generating an anomaly image , wherein, is a transparency coefficient; performing Gaussian blur fusion on edges of the background in the anomaly image and the anomaly mask image to obtain a severe defect simulation image.

3. The method of claim 1, wherein the method is based on semi-supervised anomaly detection. The preprocessed image is subjected to weak defect simulation to obtain a weak defect simulation image, including: The preprocessed image is converted into an LAB image in LAB space; A mask area of a printing area is extracted from the LAB image, and a plurality of random-shaped regions of interest are cut out in the mask area, color clustering is performed on the regions of interest to obtain dark color clustering centers, light color clustering centers, and background clustering centers; Based on the dark color clustering centers, the luminance range is expanded to obtain a dark color region; Based on the light color clustering centers, the dark color region is subjected to luminance transformation to the light color range to obtain a luminance transformation image; The luminance transformation image is converted into an HSV image, the saturation of the HSV image is reduced by a target value, and the HSV image is converted into an RGB image to obtain a defect region of interest; The preprocessed image and the defect region of interest are subjected to Gaussian fusion mapping to obtain a weak defect simulation image.

4. The method of claim 1, wherein the method is based on semi-supervised anomaly detection. The construction method of the dynamic memory bank includes: A feature encoder based on frozen parameters extracts multi-scale feature maps of a plurality of normal sample images wherein the feature encoder is pre-trained on a dataset of images previously printed by a digital pad printer; perform L2 normalization processing on the multi-scale feature map to obtain a normalized feature map wherein a mathematical expression of the normalized feature map is: ​​ wherein to prevent zero parameters; normalizing the feature map to obtain a feature vector ; and constructing a feature vector set based on a plurality of feature vectors , wherein the feature vector and a mathematical expression of the feature vector set are respectively wherein, represents a feature vector corresponding to each spatial position (x, y), C is the dimension of the feature channel, is the height of the normalized feature map , is the width of the normalized feature map , is a natural number; Based on the feature vector The feature vector Feature weights and the feature vector Last update timestamp Constructing memory items And construct memory information based on N memory items. ,in, , The feature weights Pre-allocation; introducing a time decay mechanism to update the memory item feature weight at the current time , the memory item feature weight at the current time is calculated as In the formula, For memory items The feature weights at the previous time step, The attenuation coefficient is... For the current moment With the last update timestamp The difference; The feature weight of all memory items at the current time is compared with a preset weight threshold, and the memory items with the feature weight lower than the weight threshold at the current time are removed to update the memory information ; An update mechanism is introduced, when a new normal sample image is input, a feature vector of the new normal sample image is extracted , and a cosine similarity between the feature vector and all memory items in the updated memory information is calculated, the maximum cosine similarity is screened out, and a value of novelty of the feature vector is calculated based on the maximum cosine similarity , wherein the smaller the value of the maximum cosine similarity is, the larger the value of novelty of the feature vector is; the larger the value of the maximum cosine similarity is, the smaller the value of novelty of the feature vector is. In the feature vector The novelty value of the feature vector is greater than the preset novelty threshold value, it is judged whether the number of memory items in the current memory information is full, if not, the feature vector is added to the memory information , if yes, the elimination score of all memory items in the current memory information is calculated, and the memory item with the lowest elimination score is replaced based on the feature vector , and the timestamp is updated to update the memory information , wherein the elimination score calculation formula is: In the formula, indicates the elimination score of the memory item.

5. The method of claim 1, wherein the method is based on semi-supervised anomaly detection. The total loss is calculated based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank, including: calculate a reconstruction loss based on a reconstruction error of the reconstruction feature and the enhanced fusion feature of the input image , calculate an edge loss based on the enhanced fusion feature of the input image and the dynamic memory bank , calculate a contrast loss based on the training sample set , and calculate a total loss based on the reconstruction loss , the edge loss , and the contrast loss , wherein mathematical expressions of the reconstruction loss , the edge loss , the contrast loss , and the total loss are respectively​ In the formula, Indicates reconstruction error, The total number of feature vectors in the memory. To enhance fusion features, For the first in the dynamic memory bank One memory item, For edge threshold, This represents a set of positive samples consisting of multiple normal sample images. Represents the training sample set, Represents samples in the positive sample set eigenvectors, Represents the samples in the total sample set eigenvectors, This represents the feature vector of the sample participating in the training. As the first weight, As the second weight, It is the third weight.

6. The method of claim 5, wherein the method is based on semi-supervised anomaly detection. The calculation method of the reconstruction features and the reconstruction error includes: Computing query features Cosine similarity with all memory items in the dynamic memory wherein the query features are enhanced fusion features of the input image or are enhanced fusion features of the image to be detected. The top K items with the maximum cosine similarity from all the memory items are weighted to obtain reconstructed features The mathematical expression of the reconstructed features is:​ wherein, are weight coefficients, linearly decreasing with rank, and can be calculated by the following formula: denotes the probability density function of a Gaussian distribution with mean and variance , the reconstruction error of the reconstructed feature and the query feature , the mathematical expression of the reconstruction error is: .​ 7. The method of claim 6, wherein the method is based on semi-supervised anomaly detection. The probability anomaly score of the reconstruction error of the to-be-detected image is calculated based on a pre-constructed probability distribution model, including: Bringing the reconstruction error of the image to be detected into a pre-constructed probability distribution model to obtain a probability anomaly score , wherein, indicates a pre-constructed probability distribution model, and a construction method of the probability distribution model comprises: A mixture model composed of K Gaussian distributions is constructed, and the mathematical expression of the mixture model is: where e denotes the reconstruction error scalar value, denotes the mixing coefficient of the th Gaussian distribution, denotes the mean of the reconstruction error of the th Gaussian distribution, denotes the variance of the reconstruction error of the th Gaussian distribution; reconstruction errors of the plurality of normal sample images in combination with an EM algorithm to estimate a mixing coefficient of a mixture of first second third and fourth gaussian distributions to obtain a probability distribution model.

8. The method of claim 7, wherein the method is based on semi-supervised anomaly detection. Defect positioning detection is performed based on the probability anomaly scores of multiple positions of the to-be-detected image, including: comparing the probability anomaly score to a set anomaly score threshold, and identifying locations having a probability anomaly score greater than the set anomaly score threshold as defect locations.

9. A contact lens printed pattern defect detection system based on semi-supervised anomaly detection, characterized in that, including: An acquisition module is configured to acquire a plurality of normal sample images and a to-be-detected image in a production line; An anomaly simulation module is configured to preprocess and simulate anomalies of the plurality of normal sample images to obtain simulation anomaly images. The plurality of normal sample images are preprocessed and simulated for anomalies to obtain simulation anomaly images, including: locating the center and edge of a color module printing pattern in the normal sample image, fitting a circle based on the center and edge of the color module printing pattern to obtain the diameter of the color module printing pattern; expanding the color module printing pattern outward by a plurality of pixels based on the diameter to obtain a region of interest; performing scaling and normalization processing on the region of interest to obtain a preprocessed image; performing severe defect simulation on the preprocessed image to obtain a severe defect simulation image; performing weak defect simulation on the preprocessed image to obtain a weak defect simulation image; and combining the severe defect simulation image and the weak defect simulation image to obtain a simulation anomaly image; The feature extraction module is configured to construct a training sample set based on a plurality of simulated abnormal images and a plurality of normal sample images, select an input image from the training sample set, extract multi-scale features of the input image based on a preset encoder, fuse the multi-scale features of the input image based on a feature pyramid to obtain enhanced fusion features, generate reconstruction features based on the enhanced fusion features and a pre-constructed dynamic memory bank, and calculate a reconstruction error between the enhanced fusion features and the reconstruction features, wherein the reconstruction features are composed of a plurality of memory items in the memory bank that have the highest similarity with the enhanced fusion features, and the reconstruction error is used to represent the difference between the fusion features and the reconstruction features; the construction process of the dynamic memory bank includes: extracting a vector of multi-scale features from a plurality of normal sample images to obtain memory information; introducing a time decay mechanism to determine the weight of each memory item in the memory information, and removing memory items with a weight lower than a weight threshold, wherein the earlier the timestamp of a memory item, the lower the weight of the memory item; when a new normal sample image is input, calculating the maximum similarity between the feature vector of the new normal sample image and the memory information, calculating novelty based on the maximum similarity, and adding or replacing the memory information based on the feature vector of the new normal sample image, the state of whether the number of memory items in the current memory information is full, and the weight of the memory item corresponding to the maximum similarity; The training module is configured to calculate a total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory bank, adjust parameters of the encoder and the feature pyramid based on the total loss, and return the input image selected from the training sample set until the total loss is minimized, and complete training when the total loss is minimized, wherein the total loss includes a reconstruction loss, an edge loss, and a contrast loss; The defect detection module is configured to extract the reconstruction error of the to-be-detected image based on the trained encoder, the trained feature pyramid, and the dynamic memory bank, calculate the probability anomaly score of the reconstruction error of the to-be-detected image based on a pre-constructed probability distribution model, and perform defect positioning detection based on the probability anomaly scores of a plurality of positions of the to-be-detected image.

Images