Optical element defect detection method and system and readable storage medium

Abstract

The invention discloses an optical element defect detection method and system and a readable storage medium, and the method comprises the steps: obtaining an input image which comprises a multi-channel image; determining defect features of different channels based on the input image, and generating the defect features by using a preset generative adversarial network and a diffusion model; performing cross-modal information fusion based on the defect features to obtain global features, and performing weighted integration on the defect features of different channels to obtain the global features; based on the global features, the global features serve as defect detection results to be output outwards, and the defect detection results comprise detection anomalies. According to the method, through multi-channel heterogeneous abnormal distribution modeling, cross-modal information fusion and an adaptive learning and optimization mechanism, the precision, robustness and adaptability of optical element defect detection are remarkably improved, and the method is particularly suitable for industrial scenes needing high-precision detection.

Description

Technical Field

[0001] The present invention relates to the technical field of optical component detection, and more specifically, to an optical component defect detection method, system and readable storage medium. Background Art

[0002] Anomaly detection technology has important application value in the industrial field and is widely used in product quality control, equipment fault diagnosis, and production process monitoring. With the advancement of industrial automation and intelligence, the accuracy and efficiency requirements of anomaly detection are constantly increasing.

[0003] However, existing anomaly detection technologies still face many challenges when dealing with complex industrial scenarios, including heterogeneous data distribution, difficulty integrating different information channels, and scarcity of anomaly samples. Among them, the shortcomings of existing technologies are as follows:

[0004] 1. Data distribution assumptions are overly simplified. Existing methods often assume uniform data distribution, assuming there are only two types of data: normal data and abnormal data. However, in real industrial applications, data distribution is often more complex, with different types of defects corresponding to different data patterns. Traditional methods fail to effectively account for these complex data distribution characteristics, limiting their accuracy and robustness when dealing with complex defects.

[0005] 2. Insufficient single-channel information. Existing inspection methods mostly rely on image data from a single channel for defect identification, which limits the comprehensive detection of diverse lens defects. Image data from different channels carries their own defect information, which can complement each other and improve overall inspection capabilities. However, existing methods fail to fully integrate these features from multiple channels, resulting in poor results in identifying subtle or hidden defects.

[0006] 3. Abnormal sample processing is difficult. In actual industrial production, it is usually very difficult to obtain abnormal samples. Existing technologies often rely on a large number of abnormal samples for training. Some methods assume that abnormal samples come from the same type of distribution. This simplified assumption limits the model's adaptability when processing different types of abnormal data. In particular, when faced with unseen abnormal types, the model's generalization ability is weak, which is prone to false positives or missed detections. Summary of the Invention

[0007] The purpose of the present invention is to provide an optical element defect detection method, system and readable storage medium for solving the technical problem of how to perform defect detection on multi-channel image data in the optical element lens manufacturing process in the prior art.

[0008] A first aspect of the present invention provides a method for detecting defects in an optical element, comprising the following steps:

[0009] Acquire an input image, wherein the input image includes a multi-channel image;

[0010] Determining defect features of different channels based on the input image, wherein the defect features are generated using a preset generative adversarial network and a diffusion model;

[0011] Performing cross-modal information fusion based on the defect features to obtain global features, wherein the defect features of different channels are weighted and integrated to obtain the global features;

[0012] The global features are used as defect detection results for external output, wherein the defect detection results include detection anomalies.

[0013] In this solution, the acquiring of the input image, wherein the input image includes a multi-channel image, specifically includes:

[0014] Get an input multi-channel image, where the channels include a first channel, a second channel, a third channel, a fourth channel, and a fifth channel, where:

[0015] The first channel includes a coaxial light grayscale image for detecting standard defects;

[0016] The second channel includes a laser projected image for detecting lens molding defects;

[0017] The third channel includes a multi-level laser color image for detecting glass structural defects;

[0018] The fourth channel includes a coaxial light blue image for detecting detail defects;

[0019] The fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects;

[0020] The multi-channel image is preprocessed, wherein the preprocessing method includes denoising and normalization processing.

[0021] In this solution, defects are generated for the input image of each channel based on the preset generative adversarial network and diffusion model to obtain the defect features, and sample classification and clustering are performed on the defect features, and the center point of each class is determined by minimizing the intra-class variance.

[0022] In this solution, the method also includes modeling the heterogeneous abnormal defect distribution and normal data distribution according to the defect characteristics of different channels, and iteratively optimizing and training the generative adversarial network and the diffusion model.

[0023] In this solution, cross-modal information fusion based on the defect features is performed to obtain global features, specifically including:

[0024] Extract defect features of different channels;

[0025] The weighted average algorithm is used to fuse the defect features of different channels to obtain the global feature. The formula is as follows:

[0026]

[0027] Among them, f global is the global feature, w i is the weighting coefficient of each channel defect feature, K is the number of channels, f i Defect characteristics.

[0028] In this solution, the output of the defect detection result based on the global feature specifically includes:

[0029] Based on the defect detection results, normal detection and abnormal detection, as well as corresponding confidence scores, are determined, wherein:

[0030] Output the normal data based on the detection and corresponding confidence score, and store the normal data in the verification sample library;

[0031] Based on the detected anomalies and the corresponding confidence scores, the anomaly data is output externally and stored in the training sample library.

[0032] A second aspect of the present invention further provides an optical element defect detection system, comprising a memory and a processor, wherein the memory comprises an optical element defect detection method program, and when the optical element defect detection method program is executed by the processor, the following steps are implemented:

[0033] Acquire an input image, wherein the input image includes a multi-channel image;

[0034] Determining defect features of different channels based on the input image, wherein the defect features are generated using a preset generative adversarial network and a diffusion model;

[0035] Performing cross-modal information fusion based on the defect features to obtain global features, wherein the defect features of different channels are weighted and integrated to obtain the global features;

[0036] The global features are used as defect detection results for external output, wherein the defect detection results include detection anomalies.

[0037] In this solution, the acquiring of the input image, wherein the input image includes a multi-channel image, specifically includes:

[0038] Get an input multi-channel image, where the channels include a first channel, a second channel, a third channel, a fourth channel, and a fifth channel, where:

[0039] The first channel includes a coaxial light grayscale image for detecting standard defects;

[0040] The second channel includes a laser projected image for detecting lens molding defects;

[0041] The third channel includes a multi-level laser color image for detecting glass structural defects;

[0042] The fourth channel includes a coaxial light blue image for detecting detail defects;

[0043] The fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects;

[0044] The multi-channel image is preprocessed, wherein the preprocessing method includes denoising and normalization processing.

[0045] In this solution, defects are generated for the input image of each channel based on the preset generative adversarial network and diffusion model to obtain the defect features, and sample classification and clustering are performed on the defect features, and the center point of each class is determined by minimizing the intra-class variance.

[0046] In this solution, the method also includes modeling the heterogeneous abnormal defect distribution and normal data distribution according to the defect characteristics of different channels, and iteratively optimizing and training the generative adversarial network and the diffusion model.

[0047] In this solution, cross-modal information fusion based on the defect features is performed to obtain global features, specifically including:

[0048] Extract defect features of different channels;

[0049] The weighted average algorithm is used to fuse the defect features of different channels to obtain the global feature. The formula is as follows:

[0050]

[0051] Among them, f global is the global feature, w i is the weighting coefficient of each channel defect feature, K is the number of channels, f i Defect characteristics.

[0052] In this solution, the output of the defect detection result based on the global feature specifically includes:

[0053] Based on the defect detection results, normal detection and abnormal detection, as well as corresponding confidence scores, are determined, wherein:

[0054] Output the normal data based on the detection and corresponding confidence score, and store the normal data in the verification sample library;

[0055] Based on the detected anomalies and the corresponding confidence scores, the anomaly data is output externally and stored in the training sample library.

[0056] The third aspect of the present invention provides a computer-readable storage medium, which includes a machine optical element defect detection method program. When the optical element defect detection method program is executed by a processor, the steps of an optical element defect detection method as described in any one of the above items are implemented.

[0057] The present invention discloses an optical component defect detection method, system, and readable storage medium. Through multi-channel heterogeneous anomaly distribution modeling, cross-modal information fusion, and adaptive learning and optimization mechanisms, the method significantly improves the accuracy, robustness, and adaptability of optical component defect detection. The method is particularly suitable for industrial scenarios requiring high-precision detection. It is not only applicable to lens defect detection but can also be expanded to other industrial fields such as semiconductor chip defect detection and automotive component defect detection. The specific beneficial effects are as follows:

[0058] 1. Improved detection accuracy and robustness. By introducing multi-channel heterogeneous anomaly distribution modeling and generative adversarial networks (GANs), diffusion models (Diffusion Transformer) and other technologies, the present invention can generate diverse anomaly samples, thereby improving the accuracy and robustness of the model in identifying various complex defects.

[0059] 2. Enhanced the fusion capability of multi-channel information. The present invention adopts cross-modal information fusion technology and intelligently integrates image data from multiple channels through a weighted average algorithm to ensure that the information of each channel is utilized to the maximum extent, thereby improving the accuracy and stability of the overall detection model.

[0060] 3. Adaptive optimization and overfitting control. The adaptive learning and optimization module of the present invention can dynamically adjust the weights of sub-models to avoid instability during multi-model training, and intelligently update through historical training information to effectively prevent overfitting problems and improve the model's generalization ability on unseen abnormal samples.

[0061] 4. Reduced dependence on abnormal samples and adaptable to complex industrial applications. The present invention greatly reduces the demand for a large number of abnormal samples through the technology of modeling heterogeneous abnormal distribution and generating diversified abnormal samples. It is particularly suitable for situations where there is a lack of a large amount of abnormal data in real industrial environments, greatly improving the feasibility and effectiveness of practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flow chart of a method for detecting defects in an optical element according to the present invention is shown;

[0063] Figure 2 A block diagram of an optical element defect detection system according to the present invention is shown. DETAILED DESCRIPTION

[0064] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0065] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0066] Abnormality detection (AD) technology has extremely important application value in the industrial field and is widely used in multiple aspects such as product quality control, equipment fault diagnosis, and production process monitoring. With the continuous advancement of industrial automation and intelligence, the accuracy and efficiency requirements of anomaly detection are constantly increasing. However, existing anomaly detection technologies still face many challenges when dealing with complex industrial scenarios, including heterogeneous data distribution, difficulty in fusing different information channels, and scarcity of abnormal samples. These challenges make it difficult for these technologies to meet the requirements of efficiency and accuracy in practical applications.

[0067] Taking lens manufacturing as an example, due to the complexity of the manufacturing process and the diversity of defect types, traditional anomaly detection methods struggle to effectively identify subtle defects, resulting in inadequate product quality assurance, which in turn impacts production efficiency and a company's market competitiveness. In lens defect detection, accurate and efficient identification of various defects is crucial to ensuring product quality. Existing technologies rely on data from a single image channel for defect detection, which often fails to fully cover the diverse range of lens defects. Traditional methods, in particular, have significant limitations in detection accuracy when detecting complex defects.

[0068] Lens defects exhibit diverse characteristics, and image information from different optical channels often reflects different types of defects. For example, some channels are suitable for detecting surface defects, while others are more suitable for capturing internal structural anomalies. Due to the complexity of these defects, single-channel detection methods cannot fully address the needs of multi-dimensional defects. Therefore, traditional anomaly detection methods have significant shortcomings in identifying small or hidden defects and cannot meet the efficient detection requirements of actual production.

[0069] The heterogeneity and multimodal distribution characteristics of lens defects make the data representation more complex. Different channel images convey different types of defect information, such as surface problems and internal anomalies. The basic task of anomaly detection is to identify data points that deviate from the normal pattern from the overall data. Most traditional anomaly detection methods simplify the detection process to the assumption of homogeneous distribution, that is, they assume that the data consists only of normal distribution and abnormal distribution. However, actual data is often heterogeneous, that is, different defect types exhibit different distribution patterns. For example, certain image channels are mainly used to identify surface defects, while other channels focus on capturing internal defects. Traditional methods fail to effectively consider the heterogeneity of data distribution, which directly affects the accuracy of detection.

[0070] In addition, the distribution of normal data itself is not completely uniform, and there may be heterogeneous features inside. Based on this, the present invention proposes a new idea, that is, not to regard the data distribution as homogeneous, but to regard it as a "heterogeneous abnormal distribution" composed of multiple different distributions.

[0071] To address this challenge, this paper proposes an optical component defect detection method based on modeling the heterogeneous distribution of anomalies in multi-channel data. Specifically, by jointly modeling data from different channels, the method first extracts features from the image information of each channel, and then fuses these features using an attention mechanism to ensure complementarity between the different channel data. Furthermore, the method introduces a weighted aggregation strategy, weighting the data from each channel according to its importance distribution, thereby improving overall detection accuracy and overcoming the shortcomings of traditional methods.

[0072] In industrial applications, it is relatively easy to obtain abnormal samples. Therefore, the core issue of the present invention is not how to collect abnormal samples, but how to perform feature modeling based on heterogeneous defect distribution under limited abnormal sample conditions to improve detection performance.

[0073] Specifically, Figure 1 A flow chart of an optical element defect detection method of the present application is shown.

[0074] like Figure 1 As shown, the present application discloses a method for detecting defects in optical components, comprising the following steps:

[0075] S102, acquiring an input image, where the input image includes a multi-channel image;

[0076] S104, determining defect features of different channels based on the input image, wherein the defect features are generated using a preset generative adversarial network and a diffusion model;

[0077] S106, performing cross-modal information fusion based on the defect features to obtain a global feature, wherein the defect features of different channels are weighted and integrated to obtain the global feature;

[0078] S108: Outputting the global features as defect detection results, wherein the defect detection results include detection anomalies.

[0079] It should be noted that, in this embodiment, the use of generative adversarial networks and diffusion models to generate multi-channel defect features can effectively fuse the information between channels, and the use of multi-channel collaborative modeling can better capture the heterogeneous abnormal characteristics of defects.

[0080] Specifically, an input image is first acquired. Accordingly, the input image includes a multi-channel image, and the specific channel types will be described in detail in the subsequent description. Then, based on the input image, defect features of different channels are determined. The defect features are generated using a preset generative adversarial network and a diffusion model. Each image channel reflects different types of defect features. Traditional methods have limitations in using only single-channel data for training. The present invention utilizes a generative adversarial network (GAN) and a diffusion transformer to generate diverse defect features. Then, based on the defect features, cross-modal information fusion is performed to obtain a global feature. The global feature is obtained by weighted integration of the defect features of different channels. One of the core challenges of lens defect detection is how to fuse image data from multiple channels. Since the defect information captured by different channels has multimodal characteristics, the present invention uses a weighted fusion algorithm to weightedly integrate the features of each channel to form a unified detection model. Finally, the global feature is output as a defect detection result based on the global feature. The defect detection result includes detected anomalies. When the detected anomalies are output, the anomaly data is updated to the training sample library.

[0081] According to an embodiment of the present invention, obtaining an input image, wherein the input image includes a multi-channel image, specifically includes:

[0082] Get an input multi-channel image, where the channels include a first channel, a second channel, a third channel, a fourth channel, and a fifth channel, where:

[0083] The first channel includes a coaxial light grayscale image for detecting standard defects;

[0084] The second channel includes a laser projected image for detecting lens molding defects;

[0085] The third channel includes a multi-level laser color image for detecting glass structural defects;

[0086] The fourth channel includes a coaxial light blue image for detecting detail defects;

[0087] The fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects;

[0088] The multi-channel image is preprocessed, wherein the preprocessing method includes denoising and normalization processing.

[0089] It should be noted that, in this embodiment, each channel in the input multi-channel image represents a different detection task, specifically, the channels include a first channel, a second channel, a third channel, a fourth channel and a fifth channel, wherein the first channel includes a coaxial light grayscale image for detecting standard defects, wherein the standard defects correspond to obvious defects, such as inconsistent grayscale images; the second channel includes a laser projection image for detecting poor lens molding defects; the third channel includes a multi-level laser color image for detecting glass structure defects; the fourth channel includes a coaxial light blue image for detecting detail defects; the fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects; the multi-channel image is preprocessed, wherein the image data of each channel is denoised and normalized to ensure data quality and reduce the interference of noise on subsequent feature extraction.

[0090] According to an embodiment of the present invention, defects are generated for the input image of each channel based on a preset generative adversarial network and diffusion model to obtain the defect features, and sample classification and clustering are performed on the defect features, and the center point of each class is determined by minimizing the intra-class variance.

[0091] It should be noted that, in this embodiment, the detection model is composed of a preset generative adversarial network and a diffusion model. The defect features can be obtained by generating defects for the input image of each channel. The core principle of the generative adversarial network is based on the adversarial training of two networks: the generator and the discriminator. The generator is used to generate pseudo-abnormal samples to simulate the actual defect distribution; the discriminator evaluates them and feeds back to the generator for optimization. The diffusion model generates high-quality abnormal data through a step-by-step denoising process to enhance the generalization ability of the model. Computationally, the goal of the generator G is to pass the following loss function L G To generate the real abnormal data D real Matched pseudo samples:

[0092] L G =E z～p(z) [log(1-D(G(z)))];

[0093] Here, G(z) represents the pseudo sample output by the generator, D(·) is the discriminator, and z~p(z) is the distribution of latent variables. The goal is to make the generated samples G(z) more difficult to distinguish from real samples, thereby enhancing the robustness of the generative model.

[0094] Furthermore, the defect features are classified and clustered, and the center point of each class is determined by minimizing the intra-class variance. The generated abnormal data samples often have different defect types, which requires the samples to be classified and clustered. In this embodiment, the K-means clustering algorithm is used to cluster the defect features generated by multiple channels, and the center point of each class is determined by minimizing the intra-class variance. Assuming that the data set contains n samples, the cluster center is μ k , then the objective function of clustering is:

[0095]

[0096] Among them, x i is the sample point, μ k is the center point of the kth class, i is the sample calculation element, and k is the clustering calculation element. By clustering different types of defects, features can be enhanced according to the differences in defect types, so that the model can handle more types of defects.

[0097] According to an embodiment of the present invention, the method further includes modeling the heterogeneous abnormal defect distribution and the normal data distribution according to the defect characteristics of different channels, and iteratively optimizing and training the generative adversarial network and the diffusion model.

[0098] It should be noted that in this embodiment, when modeling heterogeneous anomalies, the heterogeneous abnormal defect distribution and normal data distribution are modeled based on the defect characteristics of different channels. Traditional methods assume that the distribution of abnormal data and normal data is uniform. However, in actual industrial inspection, abnormal data exhibits heterogeneous distribution characteristics, and the abnormal distribution varies between different channels. Therefore, this embodiment adopts heterogeneous anomaly distribution modeling technology to divide the abnormal data into multiple distribution regions and establish an independent distribution model for each region.

[0099] Specifically, the model learns each data distribution by maximizing the likelihood function of the data. In multiple channels, it is assumed that the defect distribution of each channel can be modeled by a different generative model. The goal is to optimize the model parameters of each channel to maximize the overall anomaly detection accuracy. The image data used for training includes a training set, a validation set, and a test set, and the normal data and abnormal data have also been labeled.

[0100] Furthermore, the generative adversarial network and diffusion model are iteratively optimized and trained. To further improve the stability and efficiency of the model, this embodiment uses an adaptive learning module to dynamically adjust the weights of each sub-model to achieve optimization for different defect types. At the same time, historical training information is used to avoid overfitting and improve the generalization ability of the model.

[0101] In order to ensure the effectiveness of each sub-model during the training process, this embodiment dynamically adjusts the weight of each sub-model according to its performance on the validation set. Assuming that the loss function of the model is L total , then the optimal weight α is solved by minimizing the loss function i :

[0102]

[0103] Among them, L i is the loss of the i-th sub-model, α i is its weight, by optimizing α i , which can make the sub-model with better performance play a greater role in the final detection task. Furthermore, in order to avoid overfitting, this embodiment also introduces historical training information to update the model. By incorporating the loss information of historical training into the weight update process, the model can adaptively adjust the weights and optimize the parameters to avoid overfitting of certain specific abnormal types. Let the current weight be θ t , the historical training loss is L history , the update mechanism can be expressed as:

[0104]

[0105] Where η is the learning rate, is the gradient of the loss function.

[0106] According to an embodiment of the present invention, performing cross-modal information fusion based on the defect features to obtain global features specifically includes:

[0107] Extract defect features of different channels;

[0108] The weighted average algorithm is used to fuse the defect features of different channels to obtain the global feature. The formula is as follows:

[0109]

[0110] Among them, f global is the global feature, w i is the weighting coefficient of each channel defect feature, K is the number of channels, f i Defect characteristics.

[0111] It should be noted that, in this embodiment, first, feature extraction is performed on the image data of each channel, where it is assumed that the image data of the i-th channel is X i , then the feature f of the channel is extracted through the detection model i (X i ), and the final feature representation of the channel is obtained through the activation function:

[0112] f i (X i )=σ(W i ·X i +b i );

[0113] Among them, W i and b i are the corresponding weights and biases, σ is the activation function, where each channel extracts the most relevant defect features through its corresponding network. After the channel features f1, f2, ..., fK are extracted, the weighted average algorithm is used to fuse the information of different channels to obtain the global feature representation f global :

[0114]

[0115] Among them, f global is the global feature, w i is the weighting coefficient of each channel defect feature, K is the number of channels, f i For defect features, weighted fusion is used to ensure information complementarity and improve the accuracy and robustness of the model.

[0116] According to an embodiment of the present invention, outputting the defect detection result based on the global feature specifically includes:

[0117] Based on the defect detection results, normal detection and abnormal detection, as well as corresponding confidence scores, are determined, wherein:

[0118] Output the normal data based on the detection and corresponding confidence score, and store the normal data in the verification sample library;

[0119] Based on the detected anomalies and the corresponding confidence scores, the anomaly data is output externally and stored in the training sample library.

[0120] It should be noted that, in this embodiment, the detected defect detection results need to be output externally, wherein the normal detection and the abnormal detection, as well as the corresponding confidence scores, are determined based on the defect detection results. Specifically, the normal detection and the corresponding confidence scores are output externally, and the normal data is stored in the verification sample library to enrich the verification data set, while the abnormal detection and the corresponding confidence scores are output externally, and the abnormal data is stored in the training sample library to enrich the training data set.

[0121] Figure 2 A block diagram of an optical element defect detection system according to the present invention is shown.

[0122] like Figure 2 As shown, the present invention discloses an optical element defect detection system, including a memory and a processor, wherein the memory includes an optical element defect detection method program, and when the optical element defect detection method program is executed by the processor, the following steps are implemented:

[0123] Acquire an input image, wherein the input image includes a multi-channel image;

[0124] Determining defect features of different channels based on the input image, wherein the defect features are generated using a preset generative adversarial network and a diffusion model;

[0125] Performing cross-modal information fusion based on the defect features to obtain global features, wherein the defect features of different channels are weighted and integrated to obtain the global features;

[0126] The global features are used as defect detection results for external output, wherein the defect detection results include detection anomalies.

[0127] It should be noted that, in this embodiment, the use of generative adversarial networks and diffusion models to generate multi-channel defect features can effectively fuse the information between channels, and the use of multi-channel collaborative modeling can better capture the heterogeneous abnormal characteristics of defects.

[0128] Specifically, an input image is first acquired. Accordingly, the input image includes a multi-channel image, and the specific channel types will be described in detail in the subsequent description. Then, based on the input image, defect features of different channels are determined. The defect features are generated using a preset generative adversarial network and a diffusion model. Each image channel reflects different types of defect features. Traditional methods have limitations in using only single-channel data for training. The present invention utilizes a generative adversarial network (GAN) and a diffusion transformer to generate diverse defect features. Then, based on the defect features, cross-modal information fusion is performed to obtain a global feature. The global feature is obtained by weighted integration of the defect features of different channels. One of the core challenges of lens defect detection is how to fuse image data from multiple channels. Since the defect information captured by different channels has multimodal characteristics, the present invention uses a weighted fusion algorithm to weightedly integrate the features of each channel to form a unified detection model. Finally, the global feature is output as a defect detection result based on the global feature. The defect detection result includes detected anomalies. When the detected anomalies are output, the anomaly data is updated to the training sample library.

[0129] According to an embodiment of the present invention, obtaining an input image, wherein the input image includes a multi-channel image, specifically includes:

[0130] Get an input multi-channel image, where the channels include a first channel, a second channel, a third channel, a fourth channel, and a fifth channel, where:

[0131] The first channel includes a coaxial light grayscale image for detecting standard defects;

[0132] The second channel includes a laser projected image for detecting lens molding defects;

[0133] The third channel includes a multi-level laser color image for detecting glass structural defects;

[0134] The fourth channel includes a coaxial light blue image for detecting detail defects;

[0135] The fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects;

[0136] The multi-channel image is preprocessed, wherein the preprocessing method includes denoising and normalization processing.

[0137] It should be noted that, in this embodiment, each channel in the input multi-channel image represents a different detection task, specifically, the channels include a first channel, a second channel, a third channel, a fourth channel and a fifth channel, wherein the first channel includes a coaxial light grayscale image for detecting standard defects; the second channel includes a laser projection image for detecting poor lens molding defects; the third channel includes a multi-level laser color image for detecting glass structure defects; the fourth channel includes a coaxial light blue image for detecting detail defects; the fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects; the multi-channel image is preprocessed, wherein the image data of each channel is denoised and normalized to ensure data quality and reduce the interference of noise on subsequent feature extraction.

[0138] According to an embodiment of the present invention, defects are generated for the input image of each channel based on a preset generative adversarial network and diffusion model to obtain the defect features, and sample classification and clustering are performed on the defect features, and the center point of each class is determined by minimizing the intra-class variance.

[0139] It should be noted that, in this embodiment, the detection model is composed of a preset generative adversarial network and a diffusion model. The defect features can be obtained by generating defects for the input image of each channel. The core principle of the generative adversarial network is based on the adversarial training of two networks: the generator and the discriminator. The generator is used to generate pseudo-abnormal samples to simulate the actual defect distribution; the discriminator evaluates them and feeds back to the generator for optimization. The diffusion model generates high-quality abnormal data through a step-by-step denoising process to enhance the generalization ability of the model. Computationally, the goal of the generator G is to pass the following loss function L G To generate the real abnormal data D real Matched pseudo samples:

[0140] L G =E z～p(z) [log(1-D(G(z)))];

[0141] Here, G(z) represents the pseudo sample output by the generator, D(·) is the discriminator, and z~p(z) is the distribution of latent variables. The goal is to make the generated samples G(z) more difficult to distinguish from real samples, thereby enhancing the robustness of the generative model.

[0142] Furthermore, the defect features are classified and clustered, and the center point of each class is determined by minimizing the intra-class variance. The generated abnormal data samples often have different defect types, which requires the samples to be classified and clustered. In this embodiment, the K-means clustering algorithm is used to cluster the defect features generated by multiple channels, and the center point of each class is determined by minimizing the intra-class variance. Assuming that the data set contains n samples, the cluster center is μ k , then the objective function of clustering is:

[0143]

[0144] Among them, x i is the sample point, μ k is the center point of the kth class, i is the sample calculation element, and k is the clustering calculation element. By clustering different types of defects, features can be enhanced according to the differences in defect types, so that the model can handle more types of defects.

[0145] According to an embodiment of the present invention, the method further includes modeling the heterogeneous abnormal defect distribution and the normal data distribution according to the defect characteristics of different channels, and iteratively optimizing and training the generative adversarial network and the diffusion model.

[0146] It should be noted that in this embodiment, when modeling heterogeneous anomalies, the heterogeneous abnormal defect distribution and normal data distribution are modeled based on the defect characteristics of different channels. Traditional methods assume that the distribution of abnormal data and normal data is uniform. However, in actual industrial inspection, abnormal data exhibits heterogeneous distribution characteristics, and the abnormal distribution varies between different channels. Therefore, this embodiment adopts heterogeneous anomaly distribution modeling technology to divide the abnormal data into multiple distribution regions and establish an independent distribution model for each region.

[0147] Specifically, the model learns each data distribution by maximizing the likelihood function of the data. In multiple channels, it is assumed that the defect distribution of each channel can be modeled by a different generative model. The goal is to optimize the model parameters of each channel to maximize the overall anomaly detection accuracy. The image data used for training includes a training set, a validation set, and a test set, and the normal data and abnormal data have also been labeled.

[0148] Furthermore, the generative adversarial network and diffusion model are iteratively optimized and trained. To further improve the stability and efficiency of the model, this embodiment uses an adaptive learning module to dynamically adjust the weights of each sub-model to achieve optimization for different defect types. At the same time, historical training information is used to avoid overfitting and improve the generalization ability of the model.

[0149] In order to ensure the effectiveness of each sub-model during the training process, this embodiment dynamically adjusts the weight of each sub-model according to its performance on the validation set. Assuming that the loss function of the model is L total , then the optimal weight α is solved by minimizing the loss function i :

[0150]

[0151] Among them, L i is the loss of the i-th sub-model, α i is its weight, by optimizing α i , which can make the sub-model with better performance play a greater role in the final detection task. Furthermore, in order to avoid overfitting, this embodiment also introduces historical training information to update the model. By incorporating the loss information of historical training into the weight update process, the model can adaptively adjust the weights and optimize the parameters to avoid overfitting of certain specific abnormal types. Let the current weight be θ t , the historical training loss is L history , the update mechanism can be expressed as:

[0152]

[0153] Where η is the learning rate, is the gradient of the loss function.

[0154] According to an embodiment of the present invention, performing cross-modal information fusion based on the defect features to obtain global features specifically includes:

[0155] Extract defect features of different channels;

[0156] The weighted average algorithm is used to fuse the defect features of different channels to obtain the global feature. The formula is as follows:

[0157]

[0158] Among them, f global is the global feature, w i is the weighting coefficient of each channel defect feature, K is the number of channels, f i Defect characteristics.

[0159] It should be noted that, in this embodiment, first, feature extraction is performed on the image data of each channel, where it is assumed that the image data of the i-th channel is X i , then the feature f of the channel is extracted through the detection model i (X i ), and the final feature representation of the channel is obtained through the activation function:

[0160] f i (X i )=σ(W i ·X i +b i );

[0161] Among them, W i and b i are the corresponding weights and biases, σ is the activation function, where each channel extracts the most relevant defect features through its corresponding network. After the channel features f1, f2, ..., fK are extracted, the weighted average algorithm is used to fuse the information of different channels to obtain the global feature representation f global :

[0162]

[0163] Among them, f global is the global feature, w i is the weighting coefficient of each channel defect feature, K is the number of channels, f i For defect features, weighted fusion is used to ensure information complementarity and improve the accuracy and robustness of the model.

[0164] According to an embodiment of the present invention, outputting the defect detection result based on the global feature specifically includes:

[0165] Based on the defect detection results, normal detection and abnormal detection, as well as corresponding confidence scores, are determined, wherein:

[0166] Output the normal data based on the detection and corresponding confidence score, and store the normal data in the verification sample library;

[0167] Based on the detected anomalies and the corresponding confidence scores, the anomaly data is output externally and stored in the training sample library.

[0168] It should be noted that, in this embodiment, the detected defect detection results need to be output externally, wherein the normal detection and the abnormal detection, as well as the corresponding confidence scores, are determined based on the defect detection results. Specifically, the normal detection and the corresponding confidence scores are output externally, and the normal data is stored in the verification sample library to enrich the verification data set, while the abnormal detection and the corresponding confidence scores are output externally, and the abnormal data is stored in the training sample library to enrich the training data set.

[0169] The third aspect of the present invention provides a computer-readable storage medium, which includes an optical element defect detection method program. When the optical element defect detection method program is executed by a processor, it implements the steps of an optical element defect detection method as described in any one of the above items.

[0170] The present invention discloses an optical component defect detection method, system, and readable storage medium. Through multi-channel heterogeneous abnormal distribution modeling, cross-modal information fusion, and adaptive learning and optimization mechanisms, the method significantly improves the accuracy, robustness, and adaptability of optical component defect detection. The method is particularly suitable for industrial scenarios requiring high-precision detection.

[0171] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0172] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0173] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0174] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0175] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A method for detecting defects in an optical element, characterized in that: The following steps are involved: Acquire an input image, wherein the input image includes a multi-channel image; Determining defect features of different channels based on the input image, wherein the defect features are generated using a preset generative adversarial network and a diffusion model; Performing cross-modal information fusion based on the defect features to obtain global features, wherein the defect features of different channels are weighted and integrated to obtain the global features; The global features are used as defect detection results for external output, wherein the defect detection results include detection anomalies.

2. The optical element defect detection method according to claim 1, characterized in that: The acquiring of an input image, wherein the input image includes a multi-channel image, specifically includes: Get an input multi-channel image, where the channels include a first channel, a second channel, a third channel, a fourth channel, and a fifth channel, where: The first channel includes a coaxial light grayscale image for detecting standard defects; The second channel includes a laser projected image for detecting lens molding defects; The third channel includes a multi-level laser color image for detecting glass structural defects; The fourth channel includes a coaxial light blue image for detecting detail defects; The fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects; The multi-channel image is preprocessed, wherein the preprocessing method includes denoising and normalization processing.

3. The optical element defect detection method according to claim 2, characterized in that: Based on the preset generative adversarial network and diffusion model, defects are generated for the input image of each channel to obtain the defect features, and sample classification and clustering are performed on the defect features, and the center point of each class is determined by minimizing the intra-class variance.

4. The optical element defect detection method according to claim 3, characterized in that: The method also includes modeling heterogeneous abnormal defect distribution and normal data distribution according to the defect characteristics of different channels, and iteratively optimizing and training the generative adversarial network and the diffusion model.

5. The optical element defect detection method according to claim 4, characterized in that: The cross-modal information fusion based on the defect features to obtain global features specifically includes: extracting defect features of different channels; The weighted average algorithm is used to fuse the defect features of different channels to obtain the global feature. The formula is as follows: Among them, f global is the global feature, w i is the weighting coefficient of each channel defect feature, K is the number of channels, f i Defect characteristics.

6. The optical element defect detection method according to claim 5, characterized in that: The outputting of the defect detection result based on the global feature specifically includes: Based on the defect detection results, normal detection and abnormal detection, as well as corresponding confidence scores, are determined, wherein: Output the normal data based on the detection and corresponding confidence score, and store the normal data in the verification sample library; Based on the detected anomalies and the corresponding confidence scores, the anomaly data is output externally and stored in the training sample library.

7. An optical element defect detection system, characterized in that: The system comprises a memory and a processor, wherein the memory comprises an optical element defect detection method program, and when the optical element defect detection method program is executed by the processor, the following steps are implemented: Acquire an input image, wherein the input image includes a multi-channel image; Determining defect features of different channels based on the input image, wherein the defect features are generated using a preset generative adversarial network and a diffusion model; Performing cross-modal information fusion based on the defect features to obtain global features, wherein the defect features of different channels are weighted and integrated to obtain the global features; The global features are used as defect detection results for external output, wherein the defect detection results include detection anomalies.

8. The optical element defect detection system according to claim 7, characterized in that: The acquiring of an input image, wherein the input image includes a multi-channel image, specifically includes: Get an input multi-channel image, where the channels include a first channel, a second channel, a third channel, a fourth channel, and a fifth channel, where: The first channel includes a coaxial light grayscale image for detecting standard defects; The second channel includes a laser projected image for detecting lens molding defects; The third channel includes a multi-level laser color image for detecting glass structural defects; The fourth channel includes a coaxial light blue image for detecting detail defects; The fifth channel includes a laser grayscale image for detecting edge chipping defects and burr defects; The multi-channel image is preprocessed, wherein the preprocessing method includes denoising and normalization processing.

9. The optical component defect detection system according to claim 8, characterized in that: Based on the preset generative adversarial network and diffusion model, defects are generated for the input image of each channel to obtain the defect features, and sample classification and clustering are performed on the defect features, and the center point of each class is determined by minimizing the intra-class variance.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes an optical element defect detection method program, and when the optical element defect detection method program is executed by a processor, the steps of the optical element defect detection method according to any one of claims 1 to 6 are implemented.

Images