A method, system, device and medium for multi-dimensional defect detection of an optical lens

Abstract

The application discloses a kind of multi-dimensional defect detection method, system, equipment and medium of optical lens, belong to optical lens intelligent manufacturing field.The method comprises: S1, carry out multimodal data cooperation collection, the geometric dimension, optical dimension and material dimension data of the optical lens to be measured are synchronously acquired;S2, multimodal feature fusion and defect identification, the multimodal data are input to the multimodal feature fusion network of preset using double-tower cross-modal interaction architecture, carry out depth feature fusion and defect identification;S3, dynamic self-adaptive optimization, according to line environment change or new sample, network model is carried out online incremental learning and light weight deployment.The application can carry out high-precision, high-efficiency automatic detection to geometric deformation, surface flaw, material defect by integrating multimodal sensing, hybrid supervised learning and dynamic self-adaptive optimization, especially suitable for the rapid deployment of new model lens and small sample defect scene.

Description

Technical Field

[0001] This invention belongs to the field of optical intelligent manufacturing and automated inspection technology. More specifically, it relates to a method, system, device and medium for full-dimensional defect detection of optical lenses that integrates multimodal sensing, deep learning and dynamic adaptive optimization technology. Background Technology

[0002] As the core component of a precision optical system, the quality of the optical lens directly determines the final performance of the imaging system. In the complex manufacturing process of lenses, various defects are inevitably introduced. These defects can be broadly categorized into three types: geometric deformation, such as chipping or cracking at the lens edge, or minor deviations in the curvature of aspherical surfaces; surface imperfections, such as scratches, pitting, uneven coating on the lens surface, and bubbles or inclusions inside the lens; and material defects, such as uneven refractive index, abnormal stress distribution, or embedded impurities in the glass material itself. The presence of any of these defects can lead to a significant deterioration in the final image quality.

[0003] Currently, defect detection in optical lenses mainly relies on the following technologies, but all of them have significant limitations:

[0004] 1. Manual visual inspection: This is the most traditional method. Inspectors observe the equipment under specific lighting conditions (such as dark or bright illumination) using a microscope or magnifying glass. This method heavily relies on the inspector's experience and physical condition, resulting in low efficiency, high labor intensity, and strong subjectivity. It is prone to missing small, low-contrast defects, and the consistency and reliability of the inspection results are difficult to guarantee. It is no longer suitable for the needs of large-scale, high-speed modern production.

[0005] 2. Single-sensor detection methods: Machine vision-based automated inspection has been widely applied, but it typically employs a single type of sensor. For example, a 2D industrial camera is used to detect surface scratches, or a laser profilometer is used to detect geometric dimensions. The fundamental limitation of this method lies in its limited information dimension. 2D images struggle to acquire accurate three-dimensional geometric information, thus failing to effectively detect edge chipping or curvature anomalies; while 3D scanners are insensitive to minute scratches or internal bubbles on transparent surfaces. This information limitation leads to the system's inability to adequately address complex and diverse defect types, resulting in frequent missed detections and false alarms, hindering comprehensive quality control.

[0006] 3. Limitations of Traditional Machine Learning and Deep Learning Methods: While deep learning has achieved great success in image recognition, its direct application to industrial defect detection still faces challenges. First, it is highly data-dependent. Traditional supervised learning methods require a large number of precisely labeled defect samples for training. However, in actual production, defect samples, especially those of rare defects, are very scarce, making data collection and labeling costly. Second, it has poor adaptability. When the production line switches to a new lens model, changes in size, curvature, and material often cause a sharp drop in the performance of the original model, requiring data recollection and model retraining. This results in long deployment cycles and fails to meet the rapid response requirements of flexible manufacturing.

[0007] 4. Insufficient System Robustness: The production environment is complex and variable. Factors such as light fluctuations, equipment vibrations, and dust can all interfere with the stability of the detection system. Most existing detection systems are static and lack the ability to adapt to environmental changes. Once the environment changes, the detection accuracy will be affected, requiring manual intervention and recalibration, which will affect the continuity of production.

[0008] In summary, existing technologies have shortcomings in terms of comprehensiveness, accuracy, adaptability to small sample scenarios, and robustness in dynamic environments. Therefore, there is an urgent need for a new detection technology solution that can integrate multi-dimensional information, intelligently adapt to new products and defects, and operate stably in dynamic environments. Summary of the Invention

[0009] The main objective of this invention is to overcome the aforementioned deficiencies of existing technologies and provide a method, system, device, and medium for full-dimensional defect detection of optical lenses. This solution constructs a closed-loop system integrating multimodal perception, intelligent fusion recognition, and dynamic adaptive optimization. It aims to achieve high-precision, high-efficiency, and robust automated identification of three major categories of defects in optical lenses: geometric deformation, surface imperfections, and material defects. It is particularly suitable for the rapid deployment of new lens models on production lines and for high-precision detection in small-sample defect scenarios.

[0010] In a first aspect, the present invention provides a multi-dimensional defect detection method for optical lenses, comprising the following steps:

[0011] Multimodal data collaborative acquisition includes: acquiring point cloud data of the optical lens using a 3D laser scanner to obtain a 3D contour model of the lens as geometric dimension data; acquiring optical images of the optical lens using an industrial camera in coaxial bright field illumination mode and lateral dark field illumination mode respectively as visual dimension data; and acquiring the transmission spectrum passing through the optical lens using a spectrometer to evaluate the uniformity of the lens material as material dimension data.

[0012] Multimodal feature fusion and defect recognition are performed. The multimodal feature fusion and defect recognition are based on a dual-tower cross-modal interactive architecture multimodal feature fusion network. The multimodal feature fusion network receives the multimodal data, realizes the interactive fusion of geometric features and visual features through a bidirectional gated attention bridge, and outputs defect detection results.

[0013] Dynamic adaptive optimization is performed, which includes: using an online incremental learning algorithm to update the multimodal feature fusion network model when a new lens model or a new defect type appears; and using a model lightweight pipeline to perform knowledge distillation and quantization compression on the multimodal feature fusion network model for deployment on edge computing devices.

[0014] As an optional implementation of the first aspect of this application, the acquisition of point cloud data of the optical lens by a 3D laser scanner specifically includes: deploying a 3D laser scanner using active fringe projection technology; running an active fringe projection compensation algorithm, wherein the active fringe projection compensation algorithm dynamically adjusts the frequency and phase of the projected fringes according to a preset lens CAD model to improve the sampling density of the lens edge region; and registering and stitching the acquired multi-view point cloud data through an iterative nearest-point algorithm to form a complete 3D contour model.

[0015] As an optional implementation of the first aspect of this application, the acquisition of optical images of the optical lens using an industrial camera in coaxial bright field illumination mode and lateral dark field illumination mode specifically includes: configuring a composite multispectral ring light source, the composite multispectral ring light source including a coaxial bright field channel and a lateral dark field channel; in the coaxial bright field channel, red light with a wavelength of 630nm is used to vertically illuminate the lens surface, and a polarizing filter group is used to suppress specular highlight reflection; in the lateral dark field channel, blue light with a wavelength of 450nm is used to illuminate the lens surface at an adjustable incident angle between 30° and 60°.

[0016] As an optional implementation of the first aspect of this application, the dual-tower cross-modal interactive architecture multimodal feature fusion network specifically includes: a geometric feature tower for processing point cloud data, the geometric feature tower being constructed based on a dynamic graph convolutional network and employing a curvature-sensitive convolutional kernel, wherein the weight coefficients of the curvature-sensitive convolutional kernel are dynamically weighted by the product of the distance between neighboring points and the local curvature gradient; and a visual feature tower for processing optical images, the visual feature tower being constructed based on the MobileNetV3 backbone network, with an SE attention mechanism embedded after the C3 module of the visual feature tower, and a 160×160 pixel shallow feature fusion branch added to the feature pyramid network structure.

[0017] As an optional implementation of the first aspect of this application, the bidirectional gated attention bridge operates by setting a gate function, which dynamically generates weights based on the feature content input from one feature tower, and uses these weights to adjust the degree of fusion of feature information input from another feature tower into the current tower, so as to achieve interactive enhancement of geometric features and visual features.

[0018] As an optional implementation of the first aspect of this application, the step of performing multimodal feature fusion and defect identification adopts a two-stage progressive learning framework in its training process, including: the first stage, performing unsupervised pre-training, training a variational autoencoder using normal samples, and applying Gaussian noise perturbation to the latent vector of the variational autoencoder during training to enhance the model's learning of normal mode boundaries; the second stage, performing small-sample fine-tuning, after obtaining a small number of labeled defect samples, using a contrast-based learning loss function based on prototype networks to fine-tune the model, the goal of which is to increase the similarity of the sample to its similar prototypes while decreasing its similarity to dissimilar prototypes.

[0019] As an optional implementation of the first aspect of this application, the dynamic adaptive optimization specifically includes: the online incremental learning algorithm is an elastic weight solidification algorithm, which dynamically updates the Fisher information matrix of parameters on the old task through a sliding window, and applies solidification constraints only to the top 30% of parameters in terms of importance, while using an adaptive elastic coefficient that dynamically decays with the number of iterations; the quantization compression uses a dynamic range quantization algorithm to convert the model parameters from 32-bit floating-point numbers to 8-bit integers.

[0020] Secondly, embodiments of this application provide a multi-dimensional defect detection system for optical lenses, the system comprising:

[0021] The data acquisition module is used for multimodal data collaborative acquisition, including: acquiring point cloud data of the optical lens through a 3D laser scanner to obtain a 3D contour model of the lens as geometric dimension data; acquiring optical images of the optical lens through an industrial camera in coaxial bright field illumination mode and side dark field illumination mode respectively as visual dimension data; and acquiring the transmission spectrum passing through the optical lens through a spectrometer to evaluate the uniformity of the lens material as material dimension data.

[0022] The defect identification module is used for multimodal feature fusion and defect identification. The multimodal feature fusion and defect identification are based on a dual-tower cross-modal interactive architecture multimodal feature fusion network. The multimodal feature fusion network receives the multimodal data, realizes the interactive fusion of geometric features and visual features through a bidirectional gated attention bridge, and outputs the defect detection result.

[0023] An adaptive optimization module is used to perform dynamic adaptive optimization, which includes: using an online incremental learning algorithm to update the multimodal feature fusion network model when a new lens model or a new defect type appears; and using a model lightweight pipeline to perform knowledge distillation and quantization compression on the multimodal feature fusion network model for deployment on edge computing devices.

[0024] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0025] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] 1. Comprehensiveness and High Precision: By integrating information from three dimensions—geometric, optical, and material science—information complementarity is achieved, overcoming the detection blind spots of single sensors. For example, the geometric tower can detect curvature anomalies that are difficult for the visual tower to detect, while the visual tower can capture internal bubbles that the geometric tower cannot perceive. This synergistic effect significantly reduces the false alarm and missed detection rates for complex defects, resulting in higher detection accuracy.

[0028] 2. Rapid Adaptability and High Efficiency: The innovative hybrid learning framework of "unsupervised pre-training + few-sample fine-tuning" greatly reduces the dependence on expensive labeled data. When introducing new lens models or discovering new types of defects, only a small number of samples are needed to complete the rapid iteration and deployment of the model, greatly shortening the new product introduction cycle and improving the overall efficiency and flexibility of the production line.

[0029] 3. Strong Robustness and Intelligence: The integrated dynamic adaptive optimization mechanism enables the inspection system to "self-evolve." Through online incremental learning, the system can continuously adapt to the dynamic changes in the production line environment and constantly learn new knowledge, avoiding performance degradation. Lightweight model design enables efficient deployment on edge devices. This ensures the long-term stability and reliability of the inspection system in real industrial environments, moving towards true intelligent manufacturing. Attached Figure Description

[0030] Figure 1 This is a flowchart of a multi-dimensional defect detection method for an optical lens according to an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of a multi-dimensional defect detection system for an optical lens provided in an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0034] Example 1

[0035] Please see Figure 1 This embodiment details the implementation process of a multi-dimensional defect detection method for optical lenses. The method can be decomposed into three core parts: a multi-modal collaborative sensing system (corresponding to S1), a deep learning-driven defect recognition engine (corresponding to S2), and a dynamic adaptive detection system (corresponding to S3).

[0036] S1: Perform multimodal data collaborative acquisition, including: acquiring point cloud data of the optical lens through a 3D laser scanner to obtain a 3D contour model of the lens as geometric dimension data; acquiring optical images of the optical lens through an industrial camera in coaxial bright field illumination mode and lateral dark field illumination mode respectively as visual dimension data; and acquiring the transmission spectrum passing through the optical lens through a spectrometer to evaluate the uniformity of the lens material as material dimension data.

[0037] This step aims to create a comprehensive, high-fidelity digital archive of the lens.

[0038] First, geometric dimension data collection.

[0039] We deployed a GOM ATOS Q-type industrial-grade 3D laser scanner. This scanner employs active fringe projection technology, capable of generating 3D point clouds with sub-micron precision. Specifically, to address the measurement noise and data loss issues that are prone to occur on optical mirrors, especially in high-curvature areas, we developed an active fringe projection compensation algorithm. This algorithm can dynamically adjust the frequency and phase of the projected fringes based on a preset lens CAD model, increasing the sampling density in critical areas such as lens edges to 800 points / mm. 2 The above-mentioned multi-view point cloud data is registered with high precision using an optimized Iterative Closest Point (ICP) algorithm, and finally stitched together to form a complete, high-density 3D lens contour model, which serves as the geometric dimension data.

[0040] Secondly, optical dimension data acquisition.

[0041] We configured a Basler Ace 2Pro industrial camera, which features high dynamic range and low noise. To effectively distinguish different types of surface defects, we designed a composite multispectral ring light source. This light source includes two independently controllable channels:

[0042] Coaxial Bright Field Channel: Using red light with a wavelength of 630nm, the light is directed perpendicularly to the lens surface along the camera's optical axis. In this mode, the mirror itself presents a bright background, while scratches, dents, etc., disrupt light reflection and appear as dark features. We also added a polarizing filter group to this optical path to minimize interference from specular highlights, improving scratch contrast by more than 30%.

[0043] Side Dark Field Channel: This channel uses 450nm blue light to illuminate the lens from a low, side angle (the incident angle can be dynamically adjusted between 30° and 60°). In this mode, intact mirror surfaces do not reflect light into the lens, creating a dark background. However, internal bubbles, impurities, or surface defects scatter light, appearing as bright spots. The dynamic adjustability of the incident angle allows it to adapt to the optical path characteristics of lenses with different curvatures, achieving optimal signal-to-noise ratio.

[0044] Subsequently, material dimension data was collected.

[0045] We integrated an Ocean Insight STS miniature spectrometer to evaluate the uniformity of lens materials using transmission spectroscopy. A broadband light source (400-1000nm) passes through the lens under test, and the spectrometer receives and analyzes its transmission spectrum. By comparing it with the baseline spectrum of a standard lens and applying Beer-Lambert's law, the refractive index distribution in local areas can be calculated. A material defect warning is triggered when the system detects that the standard deviation of the refractive index exceeds a preset threshold (e.g., 0.005), or when the transmittance deviates from the baseline by more than ±10% in certain characteristic wavelength bands (e.g., 785nm).

[0046] Finally, the data synchronization strategy.

[0047] To ensure precise alignment of the three types of heterogeneous data in the time dimension, we adopted a dual-insurance strategy of hardware triggering and software time-stamp alignment. An FPGA board serves as the main control unit, generating high-precision synchronization pulse signals to hardware-trigger the exposure cycles of the 3D scanner and industrial camera, controlling the timing deviation at the hardware level to within 0.5ms. Simultaneously, all acquired data (point cloud, image, spectrum) is tagged with high-precision timestamps. At the software level, a timestamp interpolation compensation algorithm is used for final alignment, ensuring that the multimodal data alignment accuracy reaches the 1ms level.

[0048] S2: Perform multimodal feature fusion and defect recognition. The multimodal feature fusion and defect recognition is based on a dual-tower cross-modal interactive architecture multimodal feature fusion network. The multimodal feature fusion network receives the multimodal data, realizes the interactive fusion of geometric features and visual features through a bidirectional gated attention bridge, and outputs defect detection results.

[0049] This step is the core of defect detection, and its key lies in the deep learning-driven defect recognition engine we have built. The core of this engine is a multimodal feature fusion network with a dual-tower cross-modal interaction architecture.

[0050] First, the geometric feature tower is a deeply customized improvement based on the Dynamic Graph Convolutional Network (DGCNN) specifically designed for processing the point cloud data acquired in S1. We designed a curvature-sensitive convolutional kernel. Its mathematical expression is:

[0051]

[0052] Where, p i N(p) represents the i-th point in the point cloud; i ) represents point p i The set of neighborhood points; the key lies in the weight coefficient α. ij It is determined not only by the point spacing, but also by the combination of the point spacing and the local curvature gradient. The product is dynamically weighted, which means that in regions with drastic curvature changes (large gradients) (such as edge collapse), the feature contribution of neighboring points to the center point will be significantly enhanced, thus allowing the network to "focus" on these areas with high incidence of geometric defects; Conv3D represents the three-dimensional convolution operation; Point p i The curvature gradient at the point. This convolutional kernel enhances the extraction of geometric features in edge-damped regions, enabling the model to more accurately identify geometric defects.

[0053] Secondly, to balance accuracy and efficiency, the tower employs the lightweight MobileNetV3 as its backbone network. We embed an SE (Squeeze-and-Excitation) attention mechanism after its critical C3 module. This mechanism learns the importance of each feature channel through a small fully connected network and reweights the original feature map using the learned weights. The formula is as follows:

[0054] SEBlock(F)=σ(W2δ(W1GAP(F)))☉F

[0055] Here, F represents the input feature map; GAP is the global average pooling operation, used to average pool each channel of the feature map to obtain a one-dimensional vector; W1 and W2 are the weight matrices of the fully connected layer; δ is the ReLU activation function; σ is the Sigmoid activation function; and ⊙ represents the element-wise multiplication operation. This allows the network to automatically "increase the volume" of channels containing scratch and bubble information, while "decreases the volume" of background noise channels. Furthermore, we added an additional 160×160 pixel shallow feature fusion branch to the Feature Pyramid Network (FPN) structure to improve the detection sensitivity for small scratches.

[0056] Subsequently, we designed a bidirectional gated attention bridge (Bi-GAB), which does not simply stitch geometric and visual features together. Its formula can be illustrated as follows:

[0057]

[0058] Among them, F g F represents geometric features. v Represents visual features; Gate is a gate function used to dynamically adjust the weights for cross-modal information fusion; Conv represents the convolution operation; LayerNorm is the layer normalization operation; and These are the fused geometric features and visual features, respectively. This means that the geometric feature stream can selectively absorb visual information for enhancement based on the content of the visual features (and vice versa). For example, when the visual tower detects a suspicious texture in a certain area, the gating mechanism will "open," allowing more of the corresponding geometric features (such as curvature) to flow to the visual tower, helping to confirm whether it is a real pit, thereby achieving intelligent information complementarity and avoiding noise propagation.

[0059] In terms of training strategy, a two-stage progressive learning framework is adopted.

[0060] The first stage is unsupervised pre-training. We train a variational autoencoder (VAE) using a large number of readily available normal samples. The VAE learns the distribution of normal lenses in the low-dimensional latent space z through an "encode-decode" process. Its loss function is... for:

[0061]

[0062] Where x represents the input sample; z represents the latent variable; q(z|x) is the posterior distribution of the latent variable output by the encoder network; p(x|z) is the distribution of the reconstructed sample output by the decoder network; E q(z∣x) This represents the expectation of the posterior distribution q(z|x); β is a hyperparameter used to balance reconstruction error and KL divergence; D KL (q(z|x)||p(z) is the KL divergence, which measures the difference between the posterior distribution q(z|x) and the prior distribution p(z).

[0063] Meanwhile, we introduce adversarial perturbation enhancement: during training, a small Gaussian noise perturbation (standard deviation σ = 0.03) is applied to the latent vector z, and the decoder is required to still reconstruct a clear, normal image. This forces the model to learn more robust normal pattern boundaries, thus becoming more sensitive to any small real defects (anomalies).

[0064] The second stage is fine-tuning with a small number of samples. We switch to this stage after obtaining a small number of labeled samples (e.g., only 10-20 per defect class). We employ a contrast-based learning loss based on the prototype network.

[0065]

[0066] Where K represents the sample size; x i + Let x represent a positive sample. i -Let f represent a negative sample; f() is the feature extraction function; c is the class prototype; and margin is a pre-defined boundary value. The goal of this loss function is to maximize the similarity between a sample and its class prototype while minimizing its similarity to its dissimilar prototype, thereby forming clear, compact intra-class, and separated inter-class clusters in the feature space, enabling fast and efficient learning of new defect types.

[0067] S3: Perform dynamic adaptive optimization, which includes: using an online incremental learning algorithm to update the multimodal feature fusion network model when a new lens model or a new defect type appears; and using a model lightweight pipeline to perform knowledge distillation and quantization compression on the multimodal feature fusion network model for deployment on edge computing devices.

[0068] This step ensures the "vitality" of the detection system during long-term operation.

[0069] First, online incremental learning. We adopted an improved version of the Elastic Weight Fixation (EWC) algorithm. When a new lens model is introduced to the production line or a completely new type of defect is discovered, we do not need to train the model from scratch. The EWC algorithm evaluates the importance of parameters by calculating the Fisher information matrix on the old task. We set a sliding window (e.g., the most recent 500 samples) to dynamically update the Fisher information and apply "fixation" constraints only to the top 30% of parameters in terms of importance. This is like ensuring that an experienced craftsman does not forget his core skills when learning a new skill. Simultaneously, we designed an adaptive elasticity coefficient:

[0070]

[0071] Where, λ t λt is the elastic coefficient at the t-th iteration; λ0 is the initial elastic coefficient; γ is the decay coefficient; t is the iteration number; T max This is the maximum number of iterations. This coefficient dynamically decays with the number of iterations t, balancing the retention strength of new and old knowledge.

[0072] Secondly, edge computing optimization. To deploy complex deep learning models to edge computing devices near the production line, we established a lightweight model pipeline. First, knowledge distillation is performed. The large, high-precision multimodal fusion network trained in S2 is used as the "teacher model," and its output (soft label) guides a simpler MobileNetV3 "student model" in its learning. The loss function consists of two parts:

[0073]

[0074] Among them, L KDThis is the knowledge distillation loss; α is a hyperparameter with a value of 0.7, used to balance the cross-entropy loss L. CE And KL divergence loss; T() is the temperature scaling function; f tea This is the output of the teacher model; f stu This is the output of the student model; KL represents the KL divergence. Further, quantization compression is performed. Dynamic Range Quantization (DRQ) is used to convert the model parameters from 32-bit floating-point numbers (FP32) to 8-bit integers (INT8). This process compresses the model size to approximately 35% of its original size, increases inference speed by 2-3 times, and keeps accuracy loss within 1%, perfectly adapting to the deployment requirements of edge devices.

[0075] In summary, in this embodiment, the present invention enhances multimodal complementarity by achieving deep interaction between geometric point clouds and optical images through a bidirectional gated attention mechanism (Bi-GAB). When detecting internal impurities in transparent lenses, this reduces the false negative rate and false positive rate compared to single-modal methods. Regarding rapid adaptation to small sample sizes, the fusion of adversarial enhancement pre-training and contrast fine-tuning requires only a small number of defect samples to complete the transfer of the detection model for new lens models, reducing the sample size required compared to traditional supervised learning. In terms of robustness to dynamic environments, the online incremental learning module supports adaptive adjustment of production line environmental parameters (such as light intensity fluctuations of ±15% and mechanical vibration frequencies ≤200Hz), shortening downtime. This solution, through the integration of innovations from multiple fields, constructs a complete defect detection technology system, meeting the high-precision, high-efficiency, and highly adaptable detection needs in industrial scenarios.

[0076] Example 2

[0077] Please see Figure 2 The diagram shown is a structural schematic of a multi-dimensional defect detection system for optical lenses according to the second embodiment of this application. The system includes the following key modules:

[0078] The data acquisition module 100 is used for multimodal data collaborative acquisition, including: acquiring point cloud data of the optical lens through a 3D laser scanner to obtain a 3D contour model of the lens as geometric dimension data; acquiring optical images of the optical lens through an industrial camera in coaxial bright field illumination mode and side dark field illumination mode respectively as visual dimension data; and acquiring the transmission spectrum passing through the optical lens through a spectrometer to evaluate the uniformity of the lens material as material dimension data.

[0079] The defect identification module 200 is used to perform multimodal feature fusion and defect identification. The multimodal feature fusion and defect identification is based on a dual-tower cross-modal interactive architecture multimodal feature fusion network. The multimodal feature fusion network receives the multimodal data, realizes the interactive fusion of geometric features and visual features through a bidirectional gated attention bridge, and outputs the defect detection result.

[0080] The adaptive optimization module 300 is used to perform dynamic adaptive optimization, which includes: using an online incremental learning algorithm to update the multimodal feature fusion network model when a new lens model or a new defect type appears; and using a model lightweight pipeline to perform knowledge distillation and quantization compression on the multimodal feature fusion network model for deployment on edge computing devices.

[0081] The multi-dimensional defect detection system for optical lenses in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application embodiment does not impose specific limitations.

[0082] The multi-dimensional defect detection system for an optical lens in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0083] This application provides a multi-dimensional defect detection system for optical lenses, which can achieve... Figure 1 The various processes implemented in the method embodiment of a multi-dimensional defect detection method for optical lenses will not be described again here to avoid repetition.

[0084] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of a multi-dimensional defect detection method for an optical lens and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0085] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of a multi-dimensional defect detection method for optical lenses and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0086] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0087] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0089] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for multi-dimensional defect detection of optical lenses, characterized in that, Includes the following steps: Multimodal data collaborative acquisition includes: acquiring point cloud data of the optical lens using a 3D laser scanner to obtain a 3D contour model of the lens as geometric dimension data; acquiring optical images of the optical lens using an industrial camera in coaxial bright field illumination mode and lateral dark field illumination mode respectively as visual dimension data; and acquiring the transmission spectrum passing through the optical lens using a spectrometer to evaluate the uniformity of the lens material as material dimension data. Multimodal feature fusion and defect recognition are performed. The multimodal feature fusion and defect recognition are based on a dual-tower cross-modal interactive architecture multimodal feature fusion network. The multimodal feature fusion network receives the multimodal data, realizes the interactive fusion of geometric features and visual features through a bidirectional gated attention bridge, and outputs defect detection results. The dual-tower cross-modal interactive architecture multimodal feature fusion network specifically includes: a geometric feature tower for processing point cloud data, which is constructed based on a dynamic graph convolutional network and uses a curvature-sensitive convolutional kernel. The weight coefficients of the curvature-sensitive convolutional kernel are dynamically weighted by the product of the distance between neighboring points and the local curvature gradient; and a visual feature tower for processing optical images, which is constructed based on the MobileNetV3 backbone network and embeds an SE attention mechanism after the C3 module of the visual feature tower. At the same time, a shallow feature fusion branch of 160×160 pixels is added to the feature pyramid network structure. The bidirectional gated attention bridge operates by setting a gate function that dynamically generates weights based on the feature content input from one feature tower. These weights are used to adjust the degree to which feature information input from another feature tower is fused to the current tower, thereby achieving interactive enhancement of geometric and visual features. Dynamic adaptive optimization is performed, which includes: using an online incremental learning algorithm to update the multimodal feature fusion network model when a new lens model or a new defect type appears; and using a model lightweight pipeline to perform knowledge distillation and quantization compression on the multimodal feature fusion network model for deployment on edge computing devices.

2. The multi-dimensional defect detection method for optical lenses according to claim 1, characterized in that, The acquisition of point cloud data from the optical lens using a 3D laser scanner specifically includes: Deploy a 3D laser scanner that uses active stripe projection technology; An active fringe projection compensation algorithm is implemented, which dynamically adjusts the frequency and phase of the projected fringes based on a preset lens CAD model in order to improve the sampling density of the lens edge region. The collected multi-view point cloud data is registered and stitched together using an iterative nearest-point algorithm to form a complete 3D contour model.

3. The multi-dimensional defect detection method for an optical lens according to claim 1 or 2, characterized in that, The process of acquiring optical images from the optical lens using an industrial camera in both coaxial bright-field illumination mode and lateral dark-field illumination mode specifically includes: Configure a composite multispectral ring light source, the composite multispectral ring light source including a coaxial bright field channel and a lateral dark field channel; In the coaxial bright field channel, red light with a wavelength of 630nm is used to vertically illuminate the lens surface, and a polarizing filter group is used to suppress specular highlight reflection. In the lateral dark field channel, blue light with a wavelength of 450nm is used to illuminate the lens surface at an adjustable incident angle between 30° and 60°.

4. The multi-dimensional defect detection method for optical lenses according to claim 1, characterized in that, The training process for the steps of multimodal feature fusion and defect identification adopts a two-stage progressive learning framework, including: In the first stage, unsupervised pre-training is performed, using normal samples to train a variational autoencoder, and Gaussian noise perturbation is applied to the latent vector of the variational autoencoder during training to enhance the model's learning of normal pattern boundaries. In the second stage, a small-sample fine-tuning is performed. After obtaining a small number of labeled defective samples, the model is fine-tuned using a contrast-based learning loss function based on the prototype network. The goal of the contrast-based learning loss function is to increase the similarity between the sample and its similar prototypes, while decreasing its similarity with different prototypes.

5. The multi-dimensional defect detection method for optical lenses according to claim 1, characterized in that, The aforementioned dynamic adaptive optimization specifically includes: The online incremental learning algorithm is an elastic weight solidification algorithm, which dynamically updates the Fisher information matrix of parameters on the old task through a sliding window, and applies solidification constraints only to the top 30% of parameters in terms of importance, while using an adaptive elastic coefficient that dynamically decays with the number of iterations. The quantization compression uses a dynamic range quantization algorithm to convert model parameters from 32-bit floating-point numbers to 8-bit integers.

6. A multi-dimensional defect detection system for optical lenses, characterized in that, The system includes: The data acquisition module is used for multimodal data collaborative acquisition, including: acquiring point cloud data of the optical lens through a 3D laser scanner to obtain a 3D contour model of the lens as geometric dimension data; acquiring optical images of the optical lens through an industrial camera in coaxial bright field illumination mode and side dark field illumination mode respectively as visual dimension data; and acquiring the transmission spectrum passing through the optical lens through a spectrometer to evaluate the uniformity of the lens material as material dimension data. A defect detection module is used for multimodal feature fusion and defect detection. This multimodal feature fusion and defect detection is based on a dual-tower cross-modal interactive architecture multimodal feature fusion network. The multimodal feature fusion network receives the multimodal data, achieves interactive fusion of geometric and visual features through a bidirectional gated attention bridge, and outputs defect detection results. Specifically, the dual-tower cross-modal interactive architecture multimodal feature fusion network includes: a geometric feature tower for processing point cloud data, which is constructed based on a dynamic graph convolutional network and employs a curvature-sensitive convolutional kernel. The weight coefficients of the curvature-sensitive convolutional kernel are determined by the inter-point relationships between neighboring points. The distance is dynamically weighted by the product of the distance and the local curvature gradient; a visual feature tower for processing optical images is constructed based on the MobileNetV3 backbone network, and an SE attention mechanism is embedded after the C3 module of the visual feature tower. At the same time, a shallow feature fusion branch of 160×160 pixels is added to the feature pyramid network structure; the bidirectional gated attention bridge works by setting a gating function, which dynamically generates weights based on the feature content input from one feature tower, and uses them to adjust the degree of fusion of feature information input from another feature tower to the current tower, so as to achieve interactive enhancement of geometric features and visual features; An adaptive optimization module is used to perform dynamic adaptive optimization, which includes: using an online incremental learning algorithm to update the multimodal feature fusion network model when a new lens model or a new defect type appears; and using a model lightweight pipeline to perform knowledge distillation and quantization compression on the multimodal feature fusion network model for deployment on edge computing devices.

7. An electronic device, characterized in that, The method includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of a multi-dimensional defect detection method for an optical lens as described in any one of claims 1-5.

8. A readable storage medium, characterized in that, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by a processor, they implement the steps of the multi-dimensional defect detection method for an optical lens as described in any one of claims 1-5.

Images