Contact lens printing pattern defect detection method and system based on semi-supervised anomaly detection

Abstract

The invention relates to the technical field of machine vision, in particular to a contact lens printing pattern defect detection method and system based on semi-supervised anomaly detection.According to the contact lens printing pattern defect detection method and system based on the semi-supervised anomaly detection, anomaly detection of color mold transfer printing patterns is carried out through a deep learning method based on the semi-supervised technology, and for each new pattern, only dozens of frames of qualified product images need to be input for training; and the limitation of the current method when the current method is adaptive to various patterns and weak flaws are detected can be made up. The semi-supervised deep learning method can break through the dependence of traditional deep learning on large-scale labeled data, utilizes a semi-supervised learning strategy to mine potential information in unlabeled data, improves the recognition capability of a model on complex and sparse defects, and meanwhile, aims at the characteristics of a color mold transfer printing pattern, and improves the recognition efficiency of the color mold transfer printing pattern. A targeted deep learning network structure and a loss function are designed, and the sensitivity and generalization ability of the model to specific defect types are ensured.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a contact lens printing pattern defect detection method and system based on semi-supervised anomaly detection. Background Art

[0002] For the detection of printed defects in cosmetic contact lens patterns, the current methods are based on manual visual inspection and machine vision inspection. Among them, the machine vision-based methods can be divided into traditional image processing methods and target detection methods based on deep learning. The two are generally used in combination. The former mainly detects defects that need to be compared with qualified products, and the latter mainly supplements the detection of defects with clear characteristics such as ink shortage and stains.

[0003] However, in the above-mentioned detection schemes, manual visual quality inspection is highly subjective, lacks quantitative basis, and is prone to fatigue, which can lead to over-detection and missed detection. In the machine vision-based methods, traditional image processing methods require qualified products as reference templates. Therefore, templates need to be entered every time a new pattern is designed and produced. Since the patterns in the cosmetic contact lens industry are often updated and iterated, each manufacturer may have thousands of pattern libraries, resulting in high costs for template information maintenance. At the same time, due to the characteristics of flexible printing, there are inevitably certain subtle differences between qualified products. At this time, if the threshold is set strictly in the traditional method, it is easy to cause a large number of over-detections. If the threshold is set loosely, it is easy to cause minor defects to be missed. Therefore, usually only a compromise solution can be selected to achieve a balance between over-detection and missed detection. The target detection method based on deep learning requires data labeling and training in advance, and when new patterns appear, the detection results are easily unpredictable. In order to reduce over-detection and missed detection, continuous supplementary training is required, which consumes a lot of manpower and material resources. At the same time, defect images in industrial scenarios are often difficult to collect in large quantities, and the construction of data sets also encounters difficulties. Some newly emerging atypical defects (such as texture deformation, color loss of the underlying pattern, etc.) Figure 1 This is a schematic diagram of defects that are easily missed in color die printing, such as Figure 1 As shown in the figure, existing algorithms still have limitations in detection accuracy and robustness. These defects, which are difficult for existing algorithms to effectively identify, often cause hidden damage to brand image and user experience during actual product use. Summary of the Invention

[0004] In view of this, an object of the present invention is to provide a contact lens printed pattern defect detection method and system based on semi-supervised anomaly detection to solve the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A method for detecting defects in printed patterns of contact lenses based on semi-supervised anomaly detection of the present invention comprises the following steps: Acquire multiple normal sample images and images to be tested in the production line; Preprocessing and simulating abnormalities on the multiple normal sample images to obtain simulated abnormal images; A training sample set is constructed based on multiple simulated abnormal images and multiple normal sample images, and an input image is selected from the training sample set. Multi-scale features of the input image are extracted based on a preset encoder, and the multi-scale features of the input image are fused and attention-enhanced based on a feature pyramid to obtain enhanced fused features. A reconstructed feature is generated based on the enhanced fused feature and a pre-constructed dynamic memory library, and a reconstruction error between the fused feature and the reconstructed feature is calculated, wherein the reconstructed feature is composed of multiple memory items in the memory library that have the highest similarity to the fused feature, and the reconstruction error is used to represent the difference between the fused feature and the reconstructed feature; Calculating a total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, adjusting parameters of the encoder and the feature pyramid based on the total loss, and returning to select an input image from the training sample set until the total loss is minimized, and completing training when the total loss is minimized, wherein the loss function includes reconstruction loss, edge loss, and contrast loss; The reconstruction error of the image to be detected is extracted based on the trained encoder, the trained feature pyramid, and the dynamic memory library, and the probability anomaly score of the reconstruction error of the image to be detected is calculated based on a pre-built probability distribution model, and defect location detection is performed based on the probability anomaly scores of multiple positions of the image to be detected.

[0006] In one embodiment of the present application, preprocessing and abnormality simulation are performed on the plurality of normal sample images to obtain simulated abnormal images, including: Locating the center and edge of the color mold printing pattern in the normal sample image, and fitting a circle based on the center and edge of the color mold printing pattern to obtain the diameter of the color mold printing pattern; Expanding a plurality of pixels outward based on the diameter of the color mold printing pattern to obtain a region of interest; performing scaling and normalization processing on the region of interest to obtain a preprocessed image; Performing severe defect simulation on the preprocessed image to obtain a severe defect simulation image; performing faint defect simulation on the preprocessed image to obtain a faint defect simulation image; and combining the severe defect simulation image and the faint defect simulation image to obtain a simulated abnormality image.

[0007] In one embodiment of the present application, performing severe defect simulation on the pre-processed image to obtain a severe defect simulation image includes: Generate a two-dimensional noise map based on the simulated abnormal image, and binarize the two-dimensional noise map to obtain a noise mask ; and binarize the preprocessed image to obtain a foreground mask ; The noise mask and the foreground mask Superimpose and combine to obtain the final mask ; Extract real defects from a pre-built color die printing defect library , and for the actual defects Perform image enhancement processing to obtain the defect mask image , wherein the image enhancement processing includes rotation, scaling and brightness adjustment; The defect mask image Map to the noise mask , get the abnormal mask image , wherein the abnormal mask image The expression is:

[0008] Based on the final mask and the preprocessed image Extract Background , ; Based on the background and the abnormal mask image Generate abnormal images , ,in, is the transparency coefficient; for the abnormal image Background and the abnormal mask image Gaussian blur fusion is performed on the edges to obtain a severe defect simulation image.

[0009] In one embodiment of the present application, performing a faint defect simulation on the pre-processed image to obtain a faint defect simulation image includes: Converting the preprocessed image into a LAB image in a LAB space; Extracting a mask area of ​​the printing area from the LAB image, cutting out a plurality of randomly shaped regions of interest within the mask area, and performing color clustering on the regions of interest to obtain a dark cluster center, a light cluster center, and a background cluster center; Expanding the brightness range based on the dark cluster center to obtain a dark area; Based on the light-colored cluster center, the dark-colored area is subjected to brightness transformation to a light-colored range to obtain a brightness transformation image; Converting the brightness transformed image into an HSV image, reducing the saturation of the HSV image by a target value, and converting the HSV image into an RGB image to obtain a defect region of interest; The pre-processed image and the defect region of interest are Gaussian-fused and mapped to obtain a faint defect simulation image.

[0010] In one embodiment of the present application, the method for constructing the dynamic memory library includes: Extracting multi-scale feature maps of multiple normal sample images based on feature encoder with frozen parameters , wherein the feature encoder is pre-trained using a color mold pad printing image dataset; For the multi-scale feature map Perform L2 normalization to obtain the normalized feature map , wherein the normalized feature map The mathematical expression is:

[0011] Where, To prevent division by zero parameters; The normalized feature map Perform vectorization processing to obtain the feature vector ; and based on multiple feature vectors Constructing a set of feature vectors , where the feature vector and the feature vector set The mathematical expressions are:

[0012]

[0013] Where, Represents the feature vector corresponding to each spatial position (x, y), C is the dimension of the feature channel, is the normalized feature map of high, is the normalized feature map width, is a natural number; Based on the feature vector , the feature vector The feature weight and the eigenvector Last updated timestamp Constructing memory items , and build memory information based on N memory items ,in, , , the feature weight pre-allocation; Introduce a time decay mechanism to update the memory item Feature weight at the current moment , the memory item Feature weight at the current moment The calculation formula is:

[0014] Where, For memory items The feature weight at the previous moment, is the attenuation coefficient, For the current moment With the last updated timestamp The difference between The feature weights of all memory items at the current moment are compared with the preset weight threshold, and the memory items whose feature weights at the current moment are lower than the weight threshold are removed to improve the memory information. Make updates; Introducing an update mechanism, when a new normal sample image is input, extracting the feature vector of the new normal sample image , and calculate the eigenvector and updated memory information The cosine similarity of all memory items in , the maximum cosine similarity is selected, and the feature vector is calculated based on the maximum cosine similarity The value of novelty , wherein the smaller the value of the maximum cosine similarity is, the greater the feature vector The value of novelty The bigger; In the feature vector The value of novelty When the novelty value is greater than the preset threshold, it is determined whether the number of memory items in the current memory information is full. If not, the feature vector Add to the memory information If so, calculate the elimination scores of all memory items in the current memory information, and based on the feature vector The memory item with the lowest elimination score is replaced and the timestamp is updated to update the memory information. Update, wherein the calculation formula of the elimination score is:

[0015]

[0016] Where, Indicates the Elimination score for each memory item.

[0017] In one embodiment of the present application, the total loss is calculated based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, including: The reconstruction loss is calculated based on the reconstruction error between the reconstruction features of the input image and the enhanced fusion features. , calculating edge loss based on the enhanced fusion features of the input image and the dynamic memory library , calculate the contrast loss based on the training sample set , and based on the reconstruction loss , the edge loss and the contrast loss Calculate total loss , where the reconstruction loss , the edge loss , the contrast loss 、the total loss The mathematical expressions are:

[0018]

[0019]

[0020]

[0021] Where, is the total number of eigenvectors in the memory bank, To enhance the fusion features, The dynamic memory library memory items, is the edge threshold, represents a positive sample set consisting of multiple normal sample images, represents the training sample set, Represents samples in the positive sample set The feature vector of is the feature vector of the sample participating in the training, Indicates the sample in the total sample set The eigenvector of Represents the feature vector of the samples participating in the training, is the first weight, is the second weight, The third weight.

[0022] In one embodiment of the present application, the method for calculating the reconstruction feature and the reconstruction error includes: Calculating query features Cosine similarity with all memory items in the dynamic memory library , wherein the query feature is the enhanced fusion feature of the input image or the enhanced fusion feature of the image to be detected; Cosine similarity from all memory items Take the largest top K items and weight them to get the reconstructed features , the reconstruction feature The mathematical expression is:

[0023] in, is the weight coefficient, which decreases linearly according to the ranking and can be calculated by the following formula:

[0024] Calculate the reconstruction feature With the query characteristics The reconstruction error , the reconstruction error The mathematical expression is: .

[0025] In one embodiment of the present application, calculating the probability anomaly score of the reconstruction error of the image to be detected based on a pre-built probability distribution model includes: The reconstruction error of the image to be detected is brought into the pre-built probability distribution model to obtain the probability anomaly score , ,in, represents a pre-built probability distribution model, wherein the method for building the probability distribution model includes: A mixture model consisting of K Gaussian distributions is constructed, and the mathematical expression of the mixture model is:

[0026] Where, e represents the reconstruction error scalar value, Indicates the The mixing coefficient of the Gaussian distribution, Indicates the The mean of the reconstruction error of a Gaussian distribution, Indicates the The reconstruction error variance of a Gaussian distribution; The reconstruction error of multiple normal sample images combined with the EM algorithm is used to estimate the The mixing coefficient of a Gaussian distribution , No. The mean of the reconstruction error of a Gaussian distribution Hedi The reconstruction error variance of a Gaussian distribution , and obtain the probability distribution model.

[0027] In one embodiment of the present application, defect location detection is performed based on probability anomaly scores of multiple positions of the image to be detected, including: The probability anomaly score Compare it with the set anomaly score threshold and convert the probability anomaly score Positions with anomaly score greater than the set threshold are located as defect locations.

[0028] The present application also provides a contact lens printing pattern defect detection system based on semi-supervised anomaly detection, comprising: An acquisition module is used to acquire multiple normal sample images and images to be tested in the production line; an abnormality simulation module, configured to preprocess and simulate abnormalities on the plurality of normal sample images to obtain simulated abnormal images; a feature extraction module, configured to construct a training sample set based on a plurality of simulated abnormal images and a plurality of normal sample images, select an input image from the training sample set, extract multi-scale features of the input image based on a preset encoder, fuse the multi-scale features of the input image based on a feature pyramid to obtain an enhanced fused feature, generate a reconstructed feature based on the enhanced fused feature and a pre-constructed dynamic memory library, and calculate a reconstruction error between the fused feature and the reconstructed feature, wherein the reconstructed feature is composed of a plurality of memory items in the memory library that have the highest similarity to the fused feature, and the reconstruction error is used to represent the difference between the fused feature and the reconstructed feature; a training module, configured to calculate a total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, adjust parameters of the encoder and the feature pyramid based on the total loss, and return to select an input image from the training sample set until the total loss is minimized, and complete the training when the total loss is minimized, wherein the loss function includes reconstruction loss, edge loss, and contrast loss; The defect detection module is used to extract the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid and the dynamic memory library, calculate the probability anomaly score of the reconstruction error of the image to be detected based on a pre-built probability distribution model, and perform defect location detection based on the probability anomaly scores of multiple positions of the image to be detected.

[0029] The beneficial effects of the present invention are as follows: a method and system for detecting defects in contact lens printing patterns based on semi-supervised anomaly detection is provided. This application uses a deep learning method based on semi-supervised technology to detect anomalies in color-printed patterns. For each new pattern, only dozens of frames of qualified product images need to be recorded for training before online detection can be performed, which can make up for the limitations of current methods in adapting to multiple patterns and detecting subtle defects. Semi-supervised deep learning methods can break through the traditional deep learning's reliance on large-scale labeled data, utilize semi-supervised learning strategies to mine potential information in unlabeled data, and enhance the model's ability to recognize complex and sparse defects. At the same time, based on the characteristics of color-printed patterns, targeted deep learning network structures and loss functions are designed to ensure the model's sensitivity and generalization capabilities to specific defect types. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of defects that are easily missed in color die printing; Figure 2 A flowchart of constructing a dynamic memory bank in one embodiment of the present application; Figure 3 This is a flow chart of a contact lens printed pattern defect detection method based on semi-supervised anomaly detection shown in one embodiment of the present application; Figure 4 Schematic diagram of a severe defect simulation process in one embodiment of the present application; Figure 5 Schematic diagram of a process for simulating a weak defect in one embodiment of the present application; Figure 6 This is a simulated image of a faint defect shown in the embodiments of this application; Figure 7 Schematic diagram of the training process in one embodiment of the present application; Figure 8 Schematic diagram of ConvNeXt-T model pre-training in one embodiment of the present application; Figure 9 Schematic diagram of the flow of the reasoning process in one embodiment of the present application; Figure 10 This is a diagram specifically illustrating the effect of weak defect detection in one embodiment of the present application; Figure 11 This is a schematic diagram of the structure of a multi-scale feature pyramid fusion strategy in one embodiment of the present application; Figure 12 Schematic diagram of the attention mechanism structure in one embodiment of the present application; Figure 13This is a flowchart of a contact lens printing pattern defect detection method based on semi-supervised anomaly detection shown in one embodiment of the present application. DETAILED DESCRIPTION

[0031] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0032] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer can be changed arbitrarily, and the layer layout type may also be more complicated.

[0033] In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present invention; however, it is apparent to one skilled in the art that the embodiments of the present invention may be practiced without these specific details.

[0034] In real-world industrial quality inspection scenarios, obtaining a large number of accurately labeled anomaly samples is often expensive and difficult. Conversely, collecting large amounts of unlabeled data and a small number of known normal or mostly normal samples is relatively easy. Anomaly detection technology based on semi-supervised learning is an important approach developed to address this core challenge. It utilizes only a small amount of labeled normal samples combined with a large amount of unlabeled data to jointly learn a more robust and accurate model for identifying anomalies.

[0035] Current mainstream semi-supervised anomaly models can be categorized into three categories: reconstruction-based, feature embedding-based, and vision-language model-based. Feature embedding-based techniques are currently one of the most advanced and mainstream approaches in industrial anomaly detection. Their core concept is to leverage the powerful representation capabilities of pre-trained deep neural networks to map input data into a high-dimensional, semantically rich feature space (Feature Embedding Space). In this space, the features of normal data cluster into a compact, well-defined distribution, while the features of anomaly data, due to their unknown patterns, deviate from this normal distribution. By modeling the "normal" distribution in the feature space and calculating the degree of deviation of the test sample's features from this distribution, anomalies can be detected and localized. This approach offers advantages in terms of powerful feature representation, a focus on semantic deviations rather than attempting to reconstruct pixel-level details, high-precision localization capabilities, efficient training, fundamental avoidance of the reconstruction trap, and excellent performance on industrial inspection datasets. Therefore, we chose this approach to improve the performance of color-stamp pad printing defect detection and fine-tuned the model specifically for the specific characteristics of the color-stamp pad printing dataset.

[0036] At the same time, a module for simulating abnormal data is introduced. By replacing manual data collection with algorithmic generation, this module addresses the "abnormal sample shortage" problem in industrial scenarios. Parameter control allows for coverage of diverse anomalies, narrowing the distribution gap between simulated and real defects and improving the model's generalization capabilities, enabling identification of even unseen defects. Compared to models without anomaly simulation, models with the simulated anomaly module explicitly learn the distinguishing features between normal and abnormal conditions, avoiding missed detections caused by modeling only normal patterns and relying on reconstruction errors or feature deviations to identify anomalies. Furthermore, anomalies can be customized based on scenario characteristics, such as being insensitive to changes in the color model's rotation or translation.

[0037] In summary, through difference and commonality learning, combined with a dynamic memory library of artificially simulated abnormal samples and normal samples, anomaly detection is transformed into a semantic segmentation task. The specific process is described below.

[0038] Before officially starting semi-supervised training, this application needs to build a dynamic memory bank in advance. The usual memory bank is fixed and is used to store the feature information of normal samples. Considering that the consistency of flexible printing is not high enough, there are usually slight differences between normal samples of color molds. As the production time increases, the features of normal samples will gradually change and deviate from the initial feature distribution. Therefore, we use a dynamically updateable memory bank to achieve more accurate detection.

[0039] Figure 2 This is a flow chart of constructing a dynamic memory library in an embodiment of the present application, such as Figure 2 As shown in the figure, the construction process of the dynamic memory library is as follows: (1) A feature encoder based on frozen parameters extracts multi-scale feature maps of multiple normal sample images , wherein the feature encoder is pre-trained using a color mold pad printing image dataset; First, randomly select N normal samples and use ConvNeXt-T with frozen parameters as the feature encoder to extract multi-scale features. In our application, N is set to 100. In addition, the feature pyramid can be used to fuse the multi-scale feature map F.

[0040] The feature map extracted by ConvNeXt-T is represented by F, where the height, width and number of channels are represented by H, W, and C respectively.

[0041] (2) For the multi-scale feature map Perform L2 normalization to obtain the normalized feature map , wherein the normalized feature map The mathematical expression is:

[0042] Where, To prevent zero parameters, introduce Prevent division by zero.

[0043] (3) The normalized feature map Perform vectorization processing to obtain the feature vector ; and based on multiple feature vectors Constructing a set of feature vectors , where the feature vector and the feature vector set The mathematical expressions are:

[0044]

[0045] Where, Represents the feature vector corresponding to each spatial position (x, y), C is the dimension of the feature channel, which is 768 in ConvNeXt-T. is the normalized feature map of high, is the normalized feature map width, is a natural number. Compared with feature maps, the advantage of storing feature vectors is that they can be stored and retrieved independently, and can be matched locally, saving storage space.

[0046] (4) Based on the feature vector , the feature vector The feature weight and the eigenvector Last updated timestamp Constructing memory items , and build memory information based on N memory items ,in, , , the feature weight pre-allocation; The i-th memory item in the dynamic memory bank Three types of information need to be stored, including feature vectors , feature weight and the last updated timestamp , so each memory item contains the eigenvector , feature weight and the last updated timestamp , multiple memory items constitute the initial dynamic memory library.

[0047] (5) Introducing the time decay mechanism, the automatic update of the dynamic memory library is carried out according to the principle of time decay and feature novelty to ensure that the feature changes of normal samples can always update the memory library. Update the memory items Feature weight at the current moment , the memory item Feature weight at the current moment The calculation formula is:

[0048] Where, For memory items The feature weight at the previous moment, is the attenuation coefficient, For the current moment With the last updated timestamp The difference between The feature weights of all memory items at the current moment are compared with the preset weight threshold, and the memory items whose feature weights at the current moment are lower than the weight threshold are removed to improve the memory information. Make updates; (6) Introducing an update mechanism to extract the feature vector of a new normal sample image when a new normal sample image is input (e.g., in a subsequent training process) , and calculate the eigenvector and updated memory information The cosine similarity of all memory items in , the maximum cosine similarity is selected, and the feature vector is calculated based on the maximum cosine similarity The value of novelty , wherein the smaller the value of the maximum cosine similarity is, the greater the feature vector The value of novelty The bigger; In the feature vector The value of novelty When the novelty value is greater than the preset threshold, it is necessary to further determine whether the number of memory items in the current memory information is full. If not, the feature vector Add to the memory information If so, calculate the elimination scores of all memory items in the current memory information, and based on the feature vector The memory item with the lowest elimination score is replaced and the timestamp is updated to update the memory information. Update, wherein the calculation formula of the elimination score is:

[0049]

[0050] Where, Indicates the Elimination score for each memory item.

[0051] Then update the timestamp.

[0052]

[0053] Based on the above process, this application constructs a dynamic memory library that is dynamically updated based on time decay and novelty judgment to adapt to the situation where the characteristics of normal samples gradually change as production time increases. It can effectively adapt to the pattern updates and iterations in the cosmetic contact lens industry and has the advantage of strong adaptability.

[0054] Figure 3 FIG. 1 is a flow chart of a contact lens printing pattern defect detection method based on semi-supervised anomaly detection shown in an embodiment of the present application. Figure 3 As shown, a contact lens printed pattern defect detection method based on semi-supervised anomaly detection in this embodiment includes a training phase and an inference phase. The specific process includes: (1) Training phase A, obtain multiple normal sample images and images to be tested in the production line; First, 100 normal sample images are collected from the production line as normal sample images. The images to be tested are the images whose defects are unknown in the future.

[0055] B. Preprocessing and simulating abnormalities on the plurality of normal sample images to obtain simulated abnormal images; Preprocess the normal image. The preprocessing process includes: Preprocessing and exception simulation methods include: (1) Preprocessing B1, locating the center and edge of the color mold printing pattern in the normal sample image, and fitting a circle based on the center and edge of the color mold printing pattern to obtain the diameter of the color mold printing pattern; B2, expanding a plurality of pixels outward based on the diameter of the color mold printing pattern to obtain a region of interest; performing scaling and normalization processing on the region of interest to obtain a preprocessed image.

[0056] During the preprocessing process, the center and edge of the color mold printing pattern are located, and the diameter of the pattern is calculated using the circle fitting method. The ROI is extracted by expanding the padding by 10 pixels according to the diameter of the circle. The size of the ROI is reduced to 256×256 and normalized as the input data.

[0057] (2) Abnormal simulation For semi-supervised anomaly detection models, since only normal samples are input, the quality of the simulated anomaly samples plays a decisive role in the final model performance. If the simulated anomaly images are a good representation of the real anomalies, the model will also be able to detect the real anomalies well. Conversely, if the simulated anomalies differ significantly from the real anomalies, the model may not be able to detect the real anomalies effectively. Therefore, we focused on optimizing the simulation method for color die printing defects, especially subtle defects. The common method is to add Perlin noise to the image, which has a very high contrast and is not conducive to the detection of subtle defects. Therefore, it is necessary to optimize the anomaly simulation scheme based on the characteristics of color die printing defects.

[0058] Therefore, the abnormality simulation in this application includes severe defect simulation and weak defect simulation, and the execution order can be swapped, but severe defect simulation and weak defect simulation need to be combined.

[0059] (2-1) Severe defect simulation Figure 4 This is a flow chart of a severe defect simulation in an embodiment of the present application. The specific process is as follows: Figure 4 As shown, including: B3, generating a two-dimensional noise map based on the simulated abnormal image, and binarizing the two-dimensional noise map to obtain a noise mask ; and binarize the preprocessed image to obtain a foreground mask ; Generate a two-dimensional Perlin noise map of the same size as the original image and binarize it to obtain a mask , binarize the input color model image to obtain the foreground mask .

[0060] B4, the noise mask and the foreground mask Superimpose and combine to obtain the final mask ; B5, extracting real defects from a pre-built color die printing defect library , and for the actual defects Perform image enhancement processing to obtain the defect mask image , wherein the image enhancement processing includes rotation, scaling and brightness adjustment; Sample real defects in the real color die printing defect library And through the image enhancement method, the defect texture is randomly rotated, scaled, and brightness adjusted to obtain the defect mask image .

[0061] B6, the defect mask image Map to the noise mask , get the abnormal mask image , wherein the abnormal mask image The expression is:

[0062] B7, based on the final mask and the preprocessed image Extract Background , ; Transparency coefficient δ∈[0.15, 1], used to control the fusion strength.

[0063] B8, based on the background and the abnormal mask image Generate abnormal images , ,in, is the transparency coefficient; for the abnormal image Background and the abnormal mask image Gaussian blur is used to blend the edges of the image to obtain a simulated image with severe defects. Gaussian blur is used to blend the edges to make the transition more realistic and avoid synthetic artifacts.

[0064] (2-2) Weak defect simulation According to the characteristics of the color mold pattern, the simulation of faint defects is added, with the purpose of replacing the dark spots in the pattern with light colors to simulate the abnormality of missing a single layer of ink. Figure 5 This is a schematic diagram of a weak defect simulation process in an embodiment of the present application. Figure 5 As shown, the specific process is as follows: B9, converting the preprocessed image into a LAB image in a LAB space; the dimension L in the LAB space represents brightness, and a and b represent color opposition dimensions.

[0065] B10, extracting a mask area of ​​the printing area from the LAB image, cutting out a plurality of randomly shaped regions of interest within the mask area, and performing brightness clustering on the regions of interest to obtain a dark color cluster center, a light color cluster center, and a background cluster center; After converting sample image I to a LAB image, we take the printed area mask and cut out several small, randomly shaped ROIs within the mask for color clustering. We then determine the cluster centers, assuming that the color dots can be divided into dark and light categories. Furthermore, we consider the white background color of the unprinted dots and set the number of clusters to 3. Arrange the three cluster centers in ascending order of brightness L. The brightness value of the first cluster center represents the brightness of the dark color, the second represents the brightness of the light color center, and the third represents the background color.

[0066] B11, expanding the brightness range based on the dark cluster center to obtain a dark area; A fixed threshold operation is performed to obtain the dark area R with a brightness range of ±20 in the cluster center in the small ROI.

[0067] B12, performing brightness conversion on the dark area to a light range based on the light cluster center to obtain a brightness conversion image; The brightness of R is transformed as follows: (R / dark center brightness)*light center brightness. Through color transformation, the brightness of the dark area is transformed to the light area to simulate the abnormality of a single layer of ink missing.

[0068] B13, converting the brightness transformed image into an HSV image, reducing the saturation of the HSV image by a target value, and converting the image into an RGB image to obtain a defect region of interest; B14, performing Gaussian fusion mapping on the pre-processed image and the defect region of interest to obtain a faint defect simulation image.

[0069] Finally, the image after brightness transformation is converted into HSV image, the saturation S is reduced by 50, and then converted into RGB image to obtain the defect ROI. Then Gaussian fusion is used to map the sample image and the defect ROI.

[0070] Figure 6 This is a simulated image of a weak defect shown in the embodiment of this application. Figure 6 The right side shows multiple randomly shaped regions of interest cut out from the mask area, and the right side shows a simulated image of a faint defect.

[0071] The above two solutions need to be used together to achieve the simulation effect of taking into account both obvious severe abnormalities and weak abnormalities.

[0072] After obtaining the simulated abnormal image, it can be used together with the normal image to form a training sample set and perform training. Figure 7 This is a flow chart of the training phase in one embodiment of the present application. Figure 7 Shown, including: C. Constructing a training sample set based on multiple simulated abnormal images and multiple normal sample images, selecting an input image from the training sample set, extracting multi-scale features of the input image based on a preset encoder, fusing the multi-scale features of the input image based on a feature pyramid to obtain enhanced fused features, generating a reconstructed feature based on the enhanced fused features and a pre-constructed dynamic memory library, and calculating a reconstruction error between the fused feature and the reconstructed feature, wherein the reconstructed feature is composed of multiple memory items in the memory library that have the highest similarity to the fused feature, and the reconstruction error is used to represent the difference between the fused feature and the reconstructed feature; A training sample set is constructed by mixing normal sample images and simulated abnormal images. The normal sample images and simulated abnormal images in the training sample set can be labeled to distinguish between normal and abnormal.

[0073] In this application, an encoder is used to extract multi-scale features. The encoder used for feature extraction adopts ConvNeXt-T. This model is pre-trained using a color pad printing image dataset. Figure 8 This is a schematic diagram of the ConvNeXt-T model pre-training in one embodiment of the present application, as shown in FIG. Figure 8 As shown in Figure 1, the image dataset of color pad printing contains 150 patterns and a total of 60,000 images.

[0074] The extracted multi-scale features are then fused and attention enhanced using the feature pyramid to obtain enhanced fused features.

[0075] A pre-built dynamic memory library is used to generate reconstructed features from the enhanced fused features, and the reconstruction error between the enhanced fused features and the reconstructed features is calculated. Since the subsequent inference process also requires the calculation of feature pyramid fusion, reconstructed features, and reconstruction error, to reduce repetition and avoid redundant description, the detailed process of feature pyramid fusion, calculation of reconstructed features, and reconstruction error is statistically described after the inference process.

[0076] The reconstructed features are obtained by weighting the multiple memory items in the dynamic memory bank that are closest to the enhanced fusion features. In other words, the reconstruction error represents the difference between the feature maps of the enhanced fusion features and those of normal samples. This difference is used in conjunction with backpropagation updates to update the parameters of the encoder and feature pyramid. The training process primarily trains the feature extractor (encoder) so that it can focus on the features of normal samples and push away the features of abnormal samples, thereby constructing a discriminative embedding space.

[0077] D, calculating a total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, adjusting parameters of the encoder and the feature pyramid based on the total loss, and returning to select an input image from the training sample set until the total loss is minimized, and completing the training when the total loss is minimized, wherein the loss function includes reconstruction loss, edge loss, and contrast loss; Specifically, the reconstruction loss is calculated based on the reconstruction error of the reconstruction features of the input image and the enhanced fusion features. , calculating edge loss based on the enhanced fusion features of the input image and the dynamic memory library , calculate the contrast loss based on the training sample set , and based on the reconstruction loss , the edge loss and the contrast loss Calculate total loss , where the reconstruction loss , the edge loss , the contrast loss 、the total loss The mathematical expressions are:

[0078]

[0079]

[0080]

[0081] Where, is the number of pixels in the input image, To enhance the fusion features, is the i-th memory item in the dynamic memory bank, m is the edge threshold, represents a positive sample set consisting of multiple normal sample images, represents the training sample set, Represents samples in the positive sample set The eigenvector of Indicates the sample in the total sample set The eigenvector of Represents the feature vector of the samples participating in the training, is the first weight, is the second weight, The third weight.

[0082] in, 、 、 They are set to 1.0, 0.5, and 0.8 respectively. The reconstruction loss is dominant throughout the process, and the contrast loss can achieve the effect of accelerating convergence in the early stage.

[0083] Among them, the purpose of reconstruction loss is to minimize the difference between normal sample features and their reconstructed features in the memory bank; the purpose of edge loss is to ensure that the features of normal samples can find memory items in the memory bank that are sufficiently similar to them, while abnormal samples cannot, and it is hoped that their similarity with the normal features of the nearest neighbors in the memory bank is less than the edge threshold m, thereby pushing abnormal image features away from normal features; the purpose of contrast loss is to learn feature representation by shortening the distance between samples of the same class and pushing the distance between samples of different classes.

[0084] Based on the above loss, this application trains the encoder and feature pyramid so that the encoder can bring the features of normal samples closer and push the features of abnormal samples away, making the features of normal samples closer (low intra-class divergence) and the features of abnormal samples more dispersed (high inter-class divergence), thereby constructing a discriminative embedding space.

[0085] In addition, during the training phase, the reconstruction errors of all normal samples at various locations on the feature map form an error set. This error set is used to fit the Gaussian mixture model (GMM) for probabilistic modeling. We assume that the reconstruction error of normal samples follows a mixture model composed of K Gaussian distributions, while the reconstruction error of abnormal samples will deviate from this model. The probability density function of GMM is,

[0086] Among them, e represents the reconstruction error scalar value. For each position (x, y) on the feature map, there is a corresponding e. K represents the number of Gaussian distributions, which is 3. represents the mixing coefficient of the kth Gaussian distribution, which satisfies,

[0087] Indicates the mean , the variance is The probability density function of the Gaussian distribution is obtained by using the EM algorithm to estimate the parameters. 、 and , that is, the modeling is successful.

[0088] (2) Reasoning Process After the training is completed, the trained encoder and feature pyramid can be used to perform the inference process. Figure 9 This is a flowchart of the reasoning process in one embodiment of the present application. Figure 9 As shown, the specific process is as follows.

[0089] E. Extract the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid, and the dynamic memory library, calculate the probability anomaly score of the reconstruction error of the image to be detected based on a pre-built probability distribution model, and perform defect location detection based on the probability anomaly scores of multiple positions of the image to be detected.

[0090] The process of extracting the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid, and the dynamic memory library can refer to the above, including: E1, extracts multi-scale features of the image to be detected based on the trained encoder; E2, based on the trained feature pyramid, fuses the multi-scale features of the image to be detected to obtain enhanced fusion features; E3, calculating the reconstructed features of the enhanced fusion features based on the dynamic memory library; E4, calculate the reconstruction error between the reconstructed features and the enhanced fusion features; E5, then bring the reconstruction error of the image to be detected into the pre-built probability distribution model to obtain the probability anomaly score , ; E6, based on probability anomaly score Obtain an anomaly score map, then perform threshold segmentation on the anomaly score map to obtain a defect mask and output the result. Figure 10 This is a diagram showing the effect of weak defect detection in an embodiment of the present application. The output result obtained by the final segmentation is as follows: Figure 10 shown.

[0091] In a specific embodiment, the solution of the present application is used to conduct verification tests on color model images of 20 different patterns. Each time the pattern is changed, it is necessary to re-train with normal images of the new pattern. The model can meet the requirements of online learning, and the collection and training of 100 images on the production line takes no more than 10 minutes.

[0092] By setting a reasonable threshold for the anomaly score, defective products can be directly judged and eliminated. The final anomaly score graph A is obtained by normalizing it with the sigmoid function:

[0093] in, , indicating the coordinates The abnormal probability of Determined as abnormal pixel.

[0094] This application constructs an error set, uses this error set to fit a Gaussian mixture model (GMM) for probability modeling, and uses this probability model to convert the reconstruction error of the image to be detected into the defect probability of each position, thereby extracting areas with a higher defect probability and realizing defect location.

[0095] (III) Introduction to the specific calculation process of reconstruction features and reconstruction errors: During both the training and inference phases, the reconstructed features and reconstruction errors are calculated. The reconstruction error represents the difference between the features of a normal sample and the features reconstructed in the memory bank. While calculating the reconstruction error, the dynamic memory bank is automatically updated.

[0096] Calculating query features Cosine similarity with all memory items in the dynamic memory library , wherein the query feature is the enhanced fusion feature of the input image or the enhanced fusion feature of the image to be detected;

[0097] Cosine similarity from all memory items Take the largest top K items and weight them to get the reconstructed features , the reconstruction feature The mathematical expression is:

[0098] in, is the weight coefficient, which decreases linearly according to the ranking and can be calculated by the following formula:

[0099] Calculate the reconstruction feature With the query characteristics The reconstruction error , the reconstruction error The mathematical expression is:

[0100] (IV) Introduction to Feature Pyramid Fusion: The feature information stored in the dynamic memory library has the problem of feature redundancy, which increases the model's computational complexity and reduces the inference speed. Therefore, we can globally fuse visual and semantic information by introducing an attention mechanism and a multi-scale feature pyramid fusion strategy. Figure 11 This is a schematic diagram of the structure of the multi-scale feature pyramid fusion strategy in one embodiment of the present application. Figure 11 As shown: in, , , , are the outputs of the 1st, 2nd, 3rd, and 4th stages of the feature encoder, respectively. , , , For the corresponding pyramid features, 1×1 convolution is used to compress the channel dimension, and the resolution is upsampled and aligned before element-by-element addition to fuse semantics and details, achieving 1 / 4, 1 / 8, and 1 / 16 multi-scale feature fusion, while optimizing the matching efficiency of the memory bank.

[0101]

[0102] In addition, by introducing a channel-based and spatial attention mechanism and fusing it with a feature pyramid, key features can be further enhanced. At the same time, cross-scale attention can achieve information complementarity and achieve a balance between computational efficiency and accuracy. Figure 12 This is a schematic diagram of the attention mechanism structure in one embodiment of the present application. Figure 12 As shown, channel attention enhancement is performed first :

[0103] Then enhance spatial attention :

[0104] is the original feature, represents the features after channel attention enhancement, Represents the features after spatial attention enhancement.

[0105] The present invention discloses a method for detecting defects in contact lens printed patterns based on semi-supervised anomaly detection. This application uses a deep learning method based on semi-supervised technology to detect anomalies in color-printed patterns. For each new pattern, only dozens of frames of qualified product images need to be recorded for training before online detection can be performed. This can overcome the limitations of current methods in adapting to multiple patterns and detecting subtle defects. Semi-supervised deep learning methods can overcome the traditional deep learning's reliance on large-scale labeled data, using semi-supervised learning strategies to mine potential information in unlabeled data and enhance the model's ability to recognize complex and sparse defects. At the same time, targeted deep learning network structures and loss functions are designed based on the characteristics of color-printed patterns to ensure the model's sensitivity and generalization capabilities to specific defect types.

[0106] like Figure 13 As shown, the present application also provides a contact lens printing pattern defect detection system based on semi-supervised anomaly detection, comprising: An acquisition module is used to acquire multiple normal sample images and images to be tested in the production line; an abnormality simulation module, configured to preprocess and simulate abnormalities on the plurality of normal sample images to obtain simulated abnormal images; a feature extraction module, configured to construct a training sample set based on a plurality of simulated abnormal images and a plurality of normal sample images, select an input image from the training sample set, extract multi-scale features of the input image based on a preset encoder, fuse the multi-scale features of the input image based on a feature pyramid to obtain an enhanced fused feature, generate a reconstructed feature based on the enhanced fused feature and a pre-constructed dynamic memory library, and calculate a reconstruction error between the fused feature and the reconstructed feature, wherein the reconstructed feature is composed of a plurality of memory items in the memory library that have the highest similarity to the fused feature, and the reconstruction error is used to represent the difference between the fused feature and the reconstructed feature; a training module, configured to calculate a total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, adjust parameters of the encoder and the feature pyramid based on the total loss, and return to select an input image from the training sample set until the total loss is minimized, and complete the training when the total loss is minimized, wherein the loss function includes reconstruction loss, edge loss, and contrast loss; The defect detection module is used to extract the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid and the dynamic memory library, calculate the probability anomaly score of the reconstruction error of the image to be detected based on a pre-built probability distribution model, and perform defect location detection based on the probability anomaly scores of multiple positions of the image to be detected.

[0107] The present invention provides a contact lens printing pattern defect detection system based on semi-supervised anomaly detection. This application uses a deep learning method based on semi-supervised technology to detect anomalies in color mold pad printing patterns. For each new pattern, only dozens of frames of qualified product images need to be recorded for training before online detection can be performed. This can overcome the limitations of current methods in adapting to multiple patterns and detecting subtle defects. Semi-supervised deep learning methods can break through the traditional deep learning's reliance on large-scale labeled data, using semi-supervised learning strategies to mine potential information in unlabeled data, improving the model's ability to recognize complex and sparse defects. At the same time, based on the characteristics of color mold pad printing patterns, targeted deep learning network structures and loss functions are designed to ensure the model's sensitivity and generalization capabilities to specific defect types.

[0108] This embodiment also provides an electronic terminal, including: a processor and a memory; The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0109] Regarding the computer-readable storage medium in this embodiment, those skilled in the art will appreciate that all or part of the steps in the aforementioned method embodiments can be implemented using hardware associated with the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the aforementioned method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0110] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used for communication, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes the various steps of the above method.

[0111] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0112] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0113] In the above embodiments, although the present invention has been described in conjunction with specific embodiments of the present invention, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such replacements, modifications and variations that fall within the appended broad scope.

[0114] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be encompassed by the present invention.

Claims

1. A contact lens printing pattern defect detection method based on semi-supervised anomaly detection, characterized in that: include: Acquire multiple normal sample images and images to be tested in the production line; Preprocessing and simulating abnormalities on the multiple normal sample images to obtain simulated abnormal images; A training sample set is constructed based on multiple simulated abnormal images and multiple normal sample images, and an input image is selected from the training sample set. Multi-scale features of the input image are extracted based on a preset encoder, and the multi-scale features of the input image are fused and attention-enhanced based on a feature pyramid to obtain enhanced fused features. A reconstructed feature is generated based on the enhanced fused feature and a pre-constructed dynamic memory library, and a reconstruction error between the enhanced fused feature and the reconstructed feature is calculated, wherein the reconstructed feature is composed of multiple memory items in the memory library that have the highest similarity to the fused feature, and the reconstruction error is used to represent the difference between the fused feature and the reconstructed feature; Calculating a total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, adjusting parameters of the encoder and the feature pyramid based on the total loss, and returning to select an input image from the training sample set until the total loss is minimized, and completing training when the total loss is minimized, wherein the total loss includes reconstruction loss, edge loss, and contrast loss; The reconstruction error of the image to be detected is extracted based on the trained encoder, the trained feature pyramid, and the dynamic memory library, and the probability anomaly score of the reconstruction error of the image to be detected is calculated based on a pre-built probability distribution model, and defect location detection is performed based on the probability anomaly scores of multiple positions of the image to be detected.

2. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 1, characterized in that: Preprocessing and simulating abnormalities on the multiple normal sample images to obtain simulated abnormal images includes: Locating the center and edge of the color mold printing pattern in the normal sample image, and fitting a circle based on the center and edge of the color mold printing pattern to obtain the diameter of the color mold printing pattern; Expanding a plurality of pixels outward based on the diameter of the color mold printing pattern to obtain a region of interest; performing scaling and normalization processing on the region of interest to obtain a preprocessed image; Performing severe defect simulation on the preprocessed image to obtain a severe defect simulation image; performing faint defect simulation on the preprocessed image to obtain a faint defect simulation image; and combining the severe defect simulation image and the faint defect simulation image to obtain a simulated abnormality image.

3. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 2, characterized in that: Performing severe defect simulation on the preprocessed image to obtain a severe defect simulation image includes: Generate a two-dimensional noise map based on the simulated abnormal image, and binarize the two-dimensional noise map to obtain a noise mask ; and binarize the preprocessed image to obtain a foreground mask ; The noise mask and the foreground mask Superimpose and combine to obtain the final mask ; Extract real defects from a pre-built color die printing defect library , and for the actual defects Perform image enhancement processing to obtain the defect mask image , wherein the image enhancement processing includes rotation, scaling and brightness adjustment; The defect mask image Map to the noise mask , get the abnormal mask image , wherein the abnormal mask image The expression is: Based on the final mask and the preprocessed image Extract Background , ; Based on the background and the abnormal mask image Generate abnormal images , ,in, is the transparency coefficient; for the abnormal image Background and the abnormal mask image Gaussian blur fusion is performed on the edges to obtain a severe defect simulation image.

4. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 2, characterized in that: Performing a faint defect simulation on the pre-processed image to obtain a faint defect simulation image includes: Converting the preprocessed image into a LAB image in a LAB space; Extracting a mask area of ​​the printing area from the LAB image, cutting out a plurality of randomly shaped regions of interest within the mask area, and performing color clustering on the regions of interest to obtain a dark cluster center, a light cluster center, and a background cluster center; Expanding the brightness range based on the dark cluster center to obtain a dark area; Based on the light-colored cluster center, the dark-colored area is subjected to brightness transformation to a light-colored range to obtain a brightness transformation image; Converting the brightness transformed image into an HSV image, reducing the saturation of the HSV image by a target value, and converting the HSV image into an RGB image to obtain a defect region of interest; The pre-processed image and the defect region of interest are Gaussian-fused and mapped to obtain a faint defect simulation image.

5. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 1, characterized in that: The method for constructing the dynamic memory library includes: Extracting multi-scale feature maps of multiple normal sample images based on feature encoder with frozen parameters , wherein the feature encoder is pre-trained using a color mold pad printing image dataset; For the multi-scale feature map Perform L2 normalization to obtain the normalized feature map , wherein the normalized feature map The mathematical expression is: Where, To prevent division by zero parameters; The normalized feature map Perform vectorization processing to obtain the feature vector ; and based on multiple feature vectors Constructing a set of feature vectors , where the feature vector and the feature vector set The mathematical expressions are: Where, Represents the feature vector corresponding to each spatial position (x, y), C is the dimension of the feature channel, is the normalized feature map of high, is the normalized feature map width, is a natural number; Based on the feature vector , the feature vector The feature weight and the eigenvector Last updated timestamp Constructing memory items , and build memory information based on N memory items ,in, , , the feature weight pre-allocation; Introduce a time decay mechanism to update the memory item Feature weight at the current moment , the memory item Feature weight at the current moment The calculation formula is: Where, For memory items The feature weight at the previous moment, is the attenuation coefficient, For the current moment With the last updated timestamp The difference between The feature weights of all memory items at the current moment are compared with the preset weight threshold, and the memory items whose feature weights at the current moment are lower than the weight threshold are removed to improve the memory information. Make updates; Introducing an update mechanism, when a new normal sample image is input, extracting the feature vector of the new normal sample image , and calculate the eigenvector and updated memory information The cosine similarity of all memory items in , the maximum cosine similarity is selected, and the feature vector is calculated based on the maximum cosine similarity The value of novelty , wherein the smaller the value of the maximum cosine similarity is, the greater the feature vector The value of novelty The bigger; In the feature vector The value of novelty When the novelty value is greater than the preset threshold, it is determined whether the number of memory items in the current memory information is full. If not, the feature vector Add to the memory information If so, calculate the elimination scores of all memory items in the current memory information, and based on the feature vector The memory item with the lowest elimination score is replaced and the timestamp is updated to update the memory information. Update, wherein the calculation formula of the elimination score is: Where, Indicates the Elimination score for each memory item.

6. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 1, characterized in that: The total loss is calculated based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, including: The reconstruction loss is calculated based on the reconstruction error between the reconstruction features of the input image and the enhanced fusion features. , calculating edge loss based on the enhanced fusion features of the input image and the dynamic memory library , calculate the contrast loss based on the training sample set , and based on the reconstruction loss , the edge loss and the contrast loss Calculate total loss , where the reconstruction loss , the edge loss , the contrast loss 、the total loss The mathematical expressions are: Where, is the total number of eigenvectors in the memory bank, To enhance the fusion features, The dynamic memory library memory items, is the edge threshold, represents a positive sample set consisting of multiple normal sample images, represents the training sample set, Represents samples in the positive sample set The eigenvector of Indicates the sample in the total sample set The eigenvector of Represents the feature vector of the samples participating in the training, is the first weight, is the second weight, The third weight.

7. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 6, characterized in that: The calculation method of the reconstruction feature and the reconstruction error includes: Calculating query features Cosine similarity with all memory items in the dynamic memory library , wherein the query feature is the enhanced fusion feature of the input image or the enhanced fusion feature of the image to be detected; Cosine similarity from all memory items Take the largest top K items and weight them to get the reconstructed features , the reconstruction feature The mathematical expression is: in, is the weight coefficient, which decreases linearly according to the ranking and can be calculated by the following formula: Calculate the reconstruction feature With the query characteristics The reconstruction error , the reconstruction error The mathematical expression is: .

8. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 1, characterized in that: Calculating a probability anomaly score of a reconstruction error of the image to be detected based on a pre-built probability distribution model includes: The reconstruction error of the image to be detected is brought into the pre-built probability distribution model to obtain the probability anomaly score , ,in, represents a pre-built probability distribution model, wherein the method for building the probability distribution model includes: A mixture model consisting of K Gaussian distributions is constructed, and the mathematical expression of the mixture model is: Where e represents the reconstruction error scalar value, Indicates the The mixing coefficient of the Gaussian distribution, Indicates the The mean of the reconstruction error of a Gaussian distribution, Indicates the The reconstruction error variance of a Gaussian distribution; The reconstruction error of multiple normal sample images combined with the EM algorithm is used to estimate the The mixing coefficient of a Gaussian distribution , No. The mean of the reconstruction error of a Gaussian distribution Hedi The reconstruction error variance of a Gaussian distribution , and obtain the probability distribution model.

9. The method for detecting contact lens printed pattern defects based on semi-supervised anomaly detection according to claim 8, characterized in that: Performing defect location detection based on probability anomaly scores of multiple positions of the image to be detected includes: The probability anomaly score Compare it with the set anomaly score threshold and convert the probability anomaly score Positions with anomaly score greater than the set threshold are located as defect locations.

10. A contact lens printing pattern defect detection system based on semi-supervised anomaly detection, characterized in that: include: An acquisition module is used to acquire multiple normal sample images and images to be tested in the production line; an abnormality simulation module, configured to preprocess and simulate abnormalities on the plurality of normal sample images to obtain simulated abnormal images; a feature extraction module, configured to construct a training sample set based on a plurality of simulated abnormal images and a plurality of normal sample images, select an input image from the training sample set, extract multi-scale features of the input image based on a preset encoder, fuse the multi-scale features of the input image based on a feature pyramid to obtain an enhanced fused feature, generate a reconstructed feature based on the enhanced fused feature and a pre-constructed dynamic memory library, and calculate a reconstruction error between the fused feature and the reconstructed feature, wherein the reconstructed feature is composed of a plurality of memory items in the memory library that have the highest similarity to the fused feature, and the reconstruction error is used to represent the difference between the fused feature and the reconstructed feature; a training module, configured to calculate a total loss based on the training sample set, the enhanced fusion features of the input image, and the dynamic memory library, adjust parameters of the encoder and the feature pyramid based on the total loss, and return to select an input image from the training sample set until the total loss is minimized, and complete the training when the total loss is minimized; wherein the total loss includes reconstruction loss, edge loss, and contrast loss; The defect detection module is used to extract the reconstruction error of the image to be detected based on the trained encoder, the trained feature pyramid and the dynamic memory library, calculate the probability anomaly score of the reconstruction error of the image to be detected based on a pre-built probability distribution model, and perform defect location detection based on the probability anomaly scores of multiple positions of the image to be detected.

Images