An optical lens surface defect automatic classification and grading method and system

Abstract

The application provides a kind of optical lens surface defect automatic classification and rating method and system, the method comprises the following steps: step S1, the multi-angle surface image of free-form optical lens under multiple spectrum illuminations is collected;Step S2, according to the surface data of the free-form surface, the original image is carried out geometric inverse transformation development, and the improved Retinex algorithm is used to suppress highlight area, to obtain the preprocessed image;Step S3, the preprocessed image is input to a pre-trained double-branch deep learning neural network FFO-Net, the network simultaneously outputs the pixel-level segmentation mask of scratch and pit and the target detection frame;Step S4, based on the segmentation mask, the pixel size of each defect instance is extracted;The application can solve the problems of curvature distortion, high light interference, large defect scale span (0.01mm~0.5mm) and standard mapping complexity in free-form surface detection, and realize efficient, objective and standardization.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of optical precision testing and artificial intelligence, and specifically relates to a method and system for automatic classification and rating of surface defects of optical lenses. Background Technology

[0002] Freeform surface optical elements are increasingly widely used in cutting-edge fields such as AR / VR headsets, ultra-short-throw projection, automotive HUDs, and high-end microscopy systems due to their superior aberration correction capabilities and design freedom. Their surface quality (especially scratches and pitting) directly determines the system's imaging resolution, stray light level, and overall reliability.

[0003] Currently, most domestic optical companies still rely on manual visual inspection, which presents the following serious problems:

[0004] Highly subjective: Different inspectors may have different judgments on the same defect by more than two levels.

[0005] Low efficiency: Single lens inspection takes 20-60 seconds and is prone to fatigue, leading to missed detections.

[0006] Inconsistent implementation of standards: Optical standards (such as MIL-PRF-13830B, and the 10 / 5 to 80 / 50 of enterprise internal controls) are difficult to implement in a quantitative manner.

[0007] Freeform surfaces present unique challenges: areas with high curvature and steep slopes exhibit severe reflection, making it difficult for traditional automated optical inspection (AOI) to maintain stable focus or extract defects. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide an automatic classification and rating method for optical lens surface defects that can solve the problems of curvature distortion, specular interference, large defect size span (0.01mm~0.5mm) and complex standard mapping in freeform surface inspection, and achieve efficient, objective, standardized and high-precision quality inspection.

[0009] This invention is implemented using the following method:

[0010] Step S1: Multi-angle multispectral image acquisition

[0011] The system is equipped with at least three LED ring light sources at different angles (including 0° coaxial light, 30° low-angle light, and 60° grazing light) and a switchable wavelength filter (center wavelength 455nm blue light, 525nm green light, and 625nm red light).

[0012] The high-resolution industrial camera (global shutter, pixel size ≤2.4μm, resolution ≥20 million pixels) automatically switches the light source mode within 0.2 seconds and acquires 3 to 5 original images under different lighting conditions.

[0013] Advantages: Different wavelengths can enhance the contrast of specific defects (blue light enhances shallow scratches, red light enhances the edges of pits and craters); multiple angles can eliminate local reflective blind spots on freeform surfaces.

[0014] Step S2: Image preprocessing based on freeform surface shape

[0015] S2.1 Geometric Inverse Transformation Expansion

[0016] Using the freeform surface equation z=f(x,y) or discrete surface point cloud known during the design phase, the 3D surface normal and gradient corresponding to each pixel are calculated. A constant arc length mapping method is employed to map the points (x,y,z) on the 3D surface to 2D plane coordinates (u,v), where u is along the principal curvature direction of the surface and v is along the perpendicular direction. The mapping relationship is established through local differential geometry.

[0017] , Bicubic interpolation is used on the mapped non-integer coordinates to eliminate pixel stretching or compression caused by curvature. After processing, long scratches that were originally deformed in the image are restored to straight lines or regular curves, which is convenient for deep learning models to learn.

[0018] S2.2 Highlight Area Suppression

[0019] An improved Retinex algorithm is employed: the image is decomposed into an illuminance component L and a reflectance component R. For the highlight region (grayscale saturation region) of a freeform surface, the illuminance component L is much larger than the surrounding area. Guided filtering is used to estimate L, and then a threshold is set. (8-bit image), for Adaptive gamma correction (γ=1.5) is applied to the affected areas to reduce highlight brightness while preserving shadow details, resulting in an enhanced output image. .

[0020] Step S3: Construct and train the deep learning network (FFO-Net)

[0021] S3.1 Training Dataset Construction Method (1) Sample Collection: Collect 1000 freeform optical lenses of different grades (covering AR freeform prisms, aspherical lenses, freeform mirrors, etc.). Among them, 300 are qualified products and 700 are defective products of various grades (50 for grade 10 / 5, 100 for grade 20 / 10, 200 for grade 40 / 20, 200 for grade 60 / 40, and 150 for grade 80 / 50). Five images are taken for each lens under the illumination scheme in step S1, resulting in a total of 5000 original images.

[0022] (2) Manual annotation: Using professional annotation tools (such as LabelMe, CVAT), three senior engineers with over five years of experience in optical inspection independently annotated the labels. Annotation rules:

[0023] Scratches: Draw the outline of the scratch using polygons or polylines. For thin lines with a width of less than 3 pixels, record the width attribute as the center line.

[0024] Pockmarks: marked with circles or ellipses, with the diameter taken as the average value of the major axis of the circumscribed ellipse.

[0025] After each image is labeled, the quality control supervisor arbitrates any discrepancies, and the final labeling consistency is ≥95%.

[0026] Standard grade label: Based on the marked physical dimensions of the defects (after calibration and conversion), the grade label is automatically assigned to the image with reference to the table below (multi-label classification, scratches and pitting are labeled with independent grades):

[0027]

[0028] (4) Data Augmentation: To improve the network's generalization ability, online augmentation was performed on the original images and annotations:

[0029] Geometric transformations: random rotation (-10°~10°), random scaling (0.8~1.2 times), random horizontal flip.

[0030] Brightness and contrast adjustment: Randomly change the brightness (0.7~1.3) and contrast (0.8~1.2).

[0031] Noise injection: Add Gaussian noise ( =0.01), salt and pepper noise (density 0.002).

[0032] Simulated contamination: random occlusion (small squares simulate oil stains), slight defocus blur (Gaussian kernel size 3-5). The augmented dataset was expanded to 25,000 images and divided into training, validation, and test sets in an 8:1:1 ratio.

[0033] S3.2 Network Structure (FFO-Net) This invention designs a dual-branch multi-task network, with structural parameters as shown in the table below:

[0034] Input layer - 1024×1024×3 - 3 - - Encoder Block 1 Conv+BN+ReLU (twice)+MaxPool 1024×1024×3 3×3 / 1,2×2 / 2 64 ReLU 9.4K Encoder Block 2 Conv+BN+ReLU (twice)+MaxPool 512×512×64 3×3 / 1,2×2 / 2 128 ReLU 73.7K Encoder Block 3 Conv+BN+ReLU (twice)+MaxPool 256×256×128 3×3 / 1,2×2 / 2 256 ReLU 295K Encoder Block 4 Conv+BN+ReLU (triple)+MaxPool 128×128×256 3×3 / 1,2×2 / 2 512 ReLU 1.18M Hollow convolution block 4 parallel dilated convolutions (dilation rates 2, 4, 8, 16) + concatenation 64×64×512 3×3 / 1 512×4 ReLU 2.36M Feature Pyramid 1×1 Conv dimensionality reduction + upsampling fusion 64×64×2048 1×1 / 1 256 ReLU 0.52M Decoder Block 3 Upsampling + skip connections + Conv×2 64×64×256+ encoder 3 3×3 / 1 256 ReLU 590K Decoder Block 2 Upsampling + skip connections + Conv×2 128×128×256+ encoder2 3×3 / 1 128 ReLU 147K Decoder Block 1 Upsampling + skip connections + Conv×2 256×256×128+ encoder 1 3×3 / 1 64 ReLU 36.8K Scratched split head Conv1×1+Sigmoid 512×512×64 1×1 / 1 1 Sigmoid 0.065K Segmentation head (pockmarked) Conv1×1+Sigmoid 512×512×64 1×1 / 1 1 Sigmoid 0.065K Detection head (YOLOv8) Three scales (32×32, 64×64, 128×128) Multiscale features - 5 × (class + 4) - 1.5M Module Name Layer type Input dimensions Kernel / Stride Output Channel Activation function Parameters

[0035] Total parameters: approximately 6.7M, inference speed (NVIDIA RTX 3060) up to 50FPS.

[0036] S3.3 Training Strategy

[0037] Loss function: ,in Using DiceLoss and FocalLoss ( =0.25, γ=2) combination to address the class imbalance problem where defective pixels are far fewer than the background; CIoULoss was used for bounding box regression.

[0038] Optimizer: AdamW, initial learning rate 1×10−4, weight decay 1×10−5.

[0039] Batch size: 8 (due to high image resolution).

[0040] Training epochs: 200 epochs, using cosine annealing with learning rate decay.

[0041] Early stopping: Stop if the validation set loss does not decrease for 20 consecutive epochs.

[0042] Step S4: Defect Instance Segmentation and Physical Scale Calculation

[0043] During inference, the preprocessed image is input into FFO-Net to obtain: scratch probability map. Pockmark probability diagram

[0044] The detection head outputs the bounding box.

[0045] Post-processing: For and Threshold binarization (threshold=0.5) was performed to remove small noise pixels with an area <20 pixels. Connectivity analysis was used to extract the pixel contour of each defect instance. For scratches: the minimum bounding rectangle of the contour was calculated, and the maximum value of the rectangle width was taken as the scratch pixel width; for pits: the equivalent circle diameter of the contour was calculated ( The actual physical dimensions are obtained using the physical scale conversion factor k (mm / pixel) obtained from the calibration plate. , .

[0046] Step S5: Automatic Mapping of Standard Levels

[0047] Based on the general standard for surface quality of optical components (built-in mapping table, configurable), the most severe level of all detected defects is taken as the final level of the lens.

[0048] Step S6: Output the result

[0049] The system displays the original image and a defect overlay mask (red for scratches, blue for pits) on the monitor, and outputs text results, such as: "2 scratches detected, widest 0.025mm; 3 pits detected, maximum diameter 0.15mm; final grade: 40 / 20, qualified." It also automatically generates an inspection report, including a heatmap of defect locations and a statistical histogram.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] Freeform surface adaptive: The surface unfolding algorithm eliminates curvature distortion, and specular highlight suppression ensures that defects in reflective areas are detectable, improving the detection accuracy of freeform surfaces by approximately 40% compared to traditional AOI; High-precision micro-defect detection: The deep learning segmentation network can stably identify scratches with a width of 10μm and pits with a diameter of 0.05mm, with detection accuracy meeting the most stringent standard of 10 / 5; Standardized rating: Automatically mapped to industrial general grades (10 / 5 to 80 / 50), eliminating human differences, with rating consistency ≥98%; High efficiency: End-to-end inspection time for a single lens ≤0.5 seconds, 40 to 120 times faster than manual inspection; Traceability: Outputs defect distribution heatmaps and reports, facilitating process improvement and quality backtracking. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0053] Figure 2 This is a schematic diagram of a freeform surface lens multi-angle multispectral image acquisition system.

[0054] Figure 3 This is the architecture diagram of the FFO-Net deep learning network.

[0055] Figure 4 This is a diagram illustrating the classification of lens grades.

[0056] Figure 5 This is a schematic diagram of the scratch detection effect of S10 (10 / 5 level).

[0057] Figure 6 This is a schematic diagram of scratch grade S20 (20 / 10).

[0058] Figure 7 This is a diagram of scratch grade S40 (40 / 20).

[0059] Figure 8 This is a diagram of scratch grade S60 (60 / 40).

[0060] Figure 9 This is a diagram of scratch grade S80 (80 / 50).

[0061] Figure 10This is a schematic diagram of the detection effect of D5 (10 / 5 grade) pitting.

[0062] Figure 11 This is a schematic diagram of a D10 (20 / 10) pitting pattern.

[0063] Figure 12 This is a schematic diagram of D20 (40 / 20 grade) pitting.

[0064] Figure 13 This is a schematic diagram of a D40 (60 / 40) level of freckles.

[0065] Figure 14 This is a schematic diagram of D50 (80 / 50 grade) pitting.

[0066] Figure 15 This is a flowchart of the training dataset construction process.

[0067] Figure 16 It is a mapping curve from the physical scale of the defect to the standard level. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The following embodiments are for illustrative purposes only and do not constitute a limitation on the scope of protection of the present invention.

[0069] Example 1: Batch sampling inspection of freeform prisms for AR glasses (standard 40 / 20)

[0070] Scenario Description: An AR glasses production line requires full inspection of the first 100 freeform prisms in each batch, with a pass standard of 40 / 20 (i.e., maximum scratch width ≤ 0.04mm and maximum pit diameter ≤ 0.20mm). The system of this invention is used for automated inspection.

[0071] Implementation preparation

[0072] System Calibration: Place the standard calibration plate (including circular pitted samples with known dimensions of 0.05mm, 0.10mm, 0.20mm, and 0.40mm, and scratched samples with widths of 0.01mm, 0.02mm, 0.04mm, and 0.06mm) in the center of the stage and run the calibration program. The camera acquires images of the calibration plate under 0° coaxial light, and the pixel equivalent k is automatically calculated. After 10 repeated measurements and averaging, lk = 0.00248mm / pixel (standard deviation 0.00003) is obtained.

[0073] Preset parameters: In the software interface, select "Pass Grade" as 40 / 20; check "Output Defect Heatmap"; set "Out of Tolerance Marker" to red highlight.

[0074] Import surface shape data: Import the design surface shape equations (Zemax exported .zmx file or discrete point cloud .DAT file) of the batch of freeform prisms to be tested into the preprocessing module.

[0075] Detailed procedure for single lens testing

[0076] Step S1: The image acquisition operator places the prism on the vacuum suction fixture (vacuum pressure -60kPa), and the fixture automatically adjusts the optical surface of the lens to be horizontal. After triggering the "Start Detection" button, the system automatically executes the following sequence:

[0077] 0.00s: Coaxial blue light (455nm, 80% intensity) is turned on, camera exposure is 10ms, and the first image is captured (enhancing the contrast of shallow scratches).

[0078] 0.05s: 30° low-angle green light (525nm, 90% light intensity) is applied, camera exposure is 15ms, and the second image is captured (enhancing the edges of the speckled areas).

[0079] 0.10s: 60° grazing red light (625nm, 100% intensity) is applied, camera exposure is 20ms, and the third image is captured (highlighting roughness defects).

[0080] 0.15s: All light sources are turned on simultaneously (white light mode, color temperature 5000K), camera exposure is 8ms, and the 4th image is captured (for overall appearance).

[0081] 0.20s: Coaxial light + 30° ring light combination, capture the 5th image (remove shadows).

[0082] The five images mentioned above all have a resolution of 5472×3648 (approximately 20 million pixels) and were transmitted to the host computer in 12-bit RAW format.

[0083] Step S2: Preprocessing

[0084] Surface Unfolding: The lens surface point cloud (0.01mm grid spacing) is read. For each pixel in the image, it is back-projected onto a 3D surface according to the camera imaging model, and the corresponding unfolded coordinates (u,v) are calculated using the equal arc length mapping formula. A 1024×1024 pixel unfolded image is generated using bicubic interpolation. This process corrects the originally curved scratches caused by the surface curvature into straight line segments, facilitating recognition by deep learning models.

[0085] Spectrum suppression: Retinex decomposition is performed independently on each channel of the unfolded image. Taking the green channel as an example: the guiding filter radius is 15, the regularization parameter is 0.01, and the illuminance component L is estimated. The L histogram is calculated, and a specular threshold of 250 is set. For regions where L > 250 (approximately 1.2% of the total pixels), gamma correction γ = 1.5 is applied. After correction, the grayscale of the highlight area is reduced from 250~255 to 210~240, while the dark details inside the pits and holes (grayscale 40~80) remain unchanged. Processed image. Store in memory.

[0086] Step S3: Load the pre-trained model weight file fhunet_40_20.pth (optimized for 40 / 20 levels) for FFO-Net inference. Input network, batch size=1. Measured inference time: 187ms. Network output:

[0087] Scratch probability chart Size 1024×1024, each pixel value is the probability (0~1) that the location belongs to a scratch.

[0088] Pockmark probability chart Same size.

[0089] Detection box: Output a list containing bounding boxes with 3 dots (x1, y1, x2, y2, confidence score).

[0090] Step S4: Defect Extraction and Physical Dimension Calculation

[0091] Binarization: Set a threshold of 0.5, then... Convert to a binary mask. Remove isolated noise points with an area <20 pixels to obtain 3 connected components for the scratches. Similarly... We obtain two connected components with pockmarked spots.

[0092] Scratch width measurement: For each connected region of the scratch, calculate the minimum bounding rectangle. For example:

[0093] Scratch A: The circumscribed rectangle is 32 pixels wide and 8 pixels high → Width is 32 × 0.00248 = 0.0794 mm (Note: This width is the pixel span; the actual scratch width should be taken along its shorter side, which needs correction). Correct approach: Calculate the width along the normal direction of the skeleton line to obtain the true maximum width of 0.035 mm.

[0094] The system automatically corrects the error: After internal calibration by the algorithm, the actual output scratch A has a maximum width of 0.035mm and a length of 2.35mm.

[0095] Pockmark diameter measurement: For each connected region of a pockmark, calculate the area A and the equivalent diameter. .

[0096] Spot 1: Area A = 2147 pixels → Pixel → =52.3×0.00248=0.130.

[0097] Spot 2: Area A = 1024 pixels → Pixel → =0.0895.

[0098] Calibration error correction: Due to local magnification variations caused by lens curvature, the system corrects the measured values ​​based on surface gradient information. The correction formula is as follows: ,in These are empirical coefficients. After correction, the diameter of pit 1 becomes 0.132 mm, and the diameter of pit 2 becomes 0.091 mm.

[0099] Step S5: Level Mapping Based on the mapping table:

[0100] Maximum scratch width 0.035mm: between 0.02 and 0.04mm → Scratch grade 40 / 20.

[0101] Maximum diameter of pitting 0.132mm: between 0.10 and 0.20 → Pitting grade 40 / 20.

[0102] The final grade is the stricter of the two: 40 / 20.

[0103] Step S6: The output and recording system displays the following on the monitor:

[0104] Top left corner of the original image (4th white light image).

[0105] Defect overlay diagram (middle): Scratches are marked with red semi-transparent lines, and pits are marked with blue circles. Out-of-tolerance defects (none in this example) are marked with a red flashing border.

[0106] Result text box:

[0107] Test report

[0108] Lens ID: Lens_2025_001

[0109] Detection time: 2025-05-21 10:32:17

[0110] Scratches: 3, widest 0.035mm (40 / 20 grade)

[0111] Pockmarks: 2, maximum diameter 0.132mm (40 / 20 grade)

[0112] Overall rating: 40 / 20 Judgment: Pass

[0113] It also automatically generates a PDF report, which includes a heat map of defect locations (colored contour lines represent defect density) and a histogram of size distribution (distribution of scratch width and distribution of pit diameter).

[0114] Batch testing of 100 prisms took approximately 52 seconds (including loading and unloading), with an average testing time of 0.52 seconds per prism. This is compared to the results of manual re-inspection by three senior quality inspectors.

[0115] The system detected all lenses (92 lenses) that met the 40 / 20 rating, with a false negative rate of 0%.

[0116] The system correctly identified all 8 defective lenses (including 5 with scratches exceeding the tolerance and 3 with pitting exceeding the tolerance) as defective and pointed out the location of the defects.

[0117] Manual re-inspection revealed a 0.21mm diameter pit on one of the lenses (slightly larger than 0.20mm), which was manually deemed unacceptable; the system determined the diameter to be 0.208mm, also classifying it as unacceptable. The results from both methods were consistent.

[0118] The system achieves a 98% consistency rate with manual assessment (98 out of 100 tablets are completely identical in grade, and two tablets that differ by one sub-grade have the same conclusion of being qualified / unqualified).

[0119] Example 2: Rigorous Testing of High-Precision Laser Components (10 / 5 Level)

[0120] Scenario Description: A laser cavity reflector (with a planar substrate coated with a high-reflectivity film, requiring extremely high surface quality) needs to be inspected according to a 10 / 5 standard, meaning that scratch width is allowed to be ≤0.01mm and pit diameter is allowed to be ≤0.05mm. The lens diameter is 20mm, and the system of this invention is used for full inspection.

[0121] System Adjustment

[0122] Optical configuration: The camera lens was changed from a 2× to a 5× telecentric lens, and the working distance was adjusted to 90mm. Recalibration yielded k=0.00099mm / pixel (≈1μm / pixel).

[0123] Light source strategy: Due to the high-reflectivity coating on the lens, overexposure of coaxial light is avoided. A 30° low-angle blue light (455nm) is used as the main light source, with the light intensity set to 70% and the camera gain set to 1.5×. Grazing light is used as an auxiliary light source.

[0124] Preset grade: Select "10 / 5" as the pass grade in the software. The system will automatically adjust the segmentation threshold (increase from 0.5 to 0.6 to reduce artifacts) and enable the "super-resolution subpixel measurement" function (perform local interpolation on detected defects to improve measurement accuracy).

[0125] Testing process

[0126] Image acquisition: Due to the small lens area (20mm in diameter), a single image can cover the entire surface. Five different lighting conditions were used to acquire images, with each image exposure time ranging from 5 to 12 ms.

[0127] Preprocessing: Highlight suppression parameter adjustment: Threshold (Due to strong reflection), gammaγ=1.8. The surface unfolding procedure is simplified (for plane lenses, only radial distortion correction is performed).

[0128] FFO-Net inference: Input 1024×1024, output probability graph.

[0129] Defect extraction:

[0130] A scratch was detected: the connected component skeleton length is 120 pixels, and the normal width is fitted to 10.2 pixels by subpixel fitting → the actual width is 10.2 × 0.00099 = 0.0101 mm.

[0131] One pit was detected: area 12 pixels → equivalent diameter =3.91 pixels → =0.00387mm (approximately 3.9μm).

[0132] Level determination:

[0133] The scratch width is 0.0101mm, slightly larger than 0.01mm, exceeding the 10 / 5 standard → the scratch grade is 20 / 10.

[0134] Pockmark diameter 0.0039mm ≤ 0.05mm → Pockmark grade 10 / 5.

[0135] The final grade is the stricter one: 20 / 10.

[0136] Judgment: Not acceptable (NG), reason for rejection: "scratches out of tolerance".

[0137] Output: The system highlights the scratch in red on the image and displays its width measurement of 0.0101 mm (accurate to 0.0001 mm). A non-conformance report is generated.

[0138] Compared with manual verification, the same lens was manually inspected using a 20x microscope, and the width of the scratch was measured to be 0.0098~0.0105mm (fluctuations may occur depending on the measurer). The average of two measurements was 0.01015mm, which differs from the system's result of 0.0101mm by 0.00005mm, with a relative error of 0.5%. The measurement repeatability of this system (standard deviation of 10 measurements) is ±0.0003mm, which is better than manual verification (±0.0008mm).

[0139] In terms of efficiency, manual inspection of one lens takes approximately 90 seconds (including movement, focusing, and recording), while this system takes only 0.6 seconds per lens (including 4 seconds for loading and unloading, but multiple lenses can be inspected in parallel). For a batch of 100 lenses, manual inspection takes 2.5 hours, while this system takes only 7 minutes (including loading and unloading).

[0140] Example 3: Rapid Transfer Learning of Models (New Freeform Surface Lens)

[0141] The production line has added a new type of freeform surface reflector for automotive HUDs. Its surface shape differs significantly from the original training set samples (radius of curvature changes from 50mm to 35mm, coating changes from MgF2 to TiO2 / SiO2 multilayer film). When directly using the original model for detection, the false positive rate increases to 15% (mainly due to changes in the distribution of highlight areas caused by the new coating). A rapid model update is necessary.

[0142] Transfer learning solutions

[0143] New sample collection: 200 new model lenses were randomly selected from the production line, covering qualified products and various defects (pre-classified by experienced quality inspectors). Among them, 120 lenses were qualified products of grade 40 / 20 and above, and 80 lenses were defective products of various grades.

[0144] Image acquisition and annotation: Images were captured on 200 lenses (5 images per lens) using the acquisition system of this invention, resulting in a total of 1000 raw images. Two quality inspectors annotated the images using the LabelMe tool, and a high-quality annotation set was obtained after arbitration. Annotation took approximately 3 hours.

[0145] Model fine-tuning:

[0146] Load the original pre-trained model (the weight file fhunet_pretrain.pth, which has been trained for 200 epochs on 1000 old models).

[0147] Freezing strategy: Freeze all parameters of encoder blocks 1, 2, and 3 (approximately 0.38M parameters in total), keeping their weights unchanged. Decoder blocks 1, 2, and 3, as well as the segmentation head and detection head (approximately 0.77M parameters) are set to trainable.

[0148] Training parameters: Initial learning rate (One order of magnitude smaller than training from scratch), batch size 4 (due to the small size of the new dataset), training for 50 epochs. The loss function remains the same, but a regularization term is added: , To prevent overfitting.

[0149] Data augmentation: Only slight enhancements are used (random horizontal flip, ±5° rotation, brightness adjustment of 0.9~1.1) to avoid excessive distortion of new features.

[0150] Verification and Testing:

[0151] Twenty pieces (10%) were selected from the 200 pieces as the test set, and the rest were used for training. During training, the results were evaluated on the validation set (10%) every 5 epochs.

[0152] The validation set loss stabilized at the 35th epoch, and early stopping was not triggered.

[0153] Ultimately, on the 20 test sets, the false positive rate decreased from 15% before migration to 3.2%, and the false negative rate decreased from 8% to 2.1%. The consistency with manually labeled levels reached 96%.

[0154] Deployment: The finely tuned model weights fhunet_hud_finetuned.pth were deployed to the online detection system. The entire process (from collecting new samples to deploying the model) took approximately one business day, while training a new model from scratch requires collecting at least 1000 samples, labeling for two weeks, and training for 200 epochs for approximately two days. Transfer learning saved more than 90% of the time and cost.

[0155] Further note: This transfer learning approach is also applicable to the following situations.

[0156] Freeform surfaces with different curvature ranges (e.g., from a sphere to a freeform surface).

[0157] Different coating materials (such as metal films, dielectric films, and antireflective films).

[0158] Different defect level distributions (e.g., switching from 40 / 20 as the main type to 20 / 10 as the main type).

[0159] Example 4: Real-time Defect Size Statistics and Process Feedback

[0160] Scene Description

[0161] An optical lens manufacturer wants to optimize its grinding and polishing process using the inspection data from this invention. During continuous production inspection, the system automatically records the defect size of each lens and generates a statistical trend chart.

[0162] Implementation

[0163] Data logging: After each test, the system writes the following data to the SQLite database:

[0164] Lens ID, batch number, production date, and process number.

[0165] Number of scratches, maximum width, average width, and total length.

[0166] Number of pits, maximum diameter, average diameter, and total area.

[0167] Overall rating, pass / fail marking.

[0168] Statistical Analysis Module: Provides a web interface for querying defect statistics for any time period.

[0169] Histogram: Displays the scratch width distribution of the most recent 1000 lenses (bin=0.01mm). If the peak value shifts to the right, it indicates a deterioration in the manufacturing process.

[0170] Control charts: Plot the mean maximum scratch width (X-bar chart) and range (R chart) for each batch, and set upper and lower control limits (UCL / LCL). When 7 consecutive points exceed the control limits, the system automatically sends an alarm email to the process engineer.

[0171] Defect thermal overlay map: The defect locations of all lenses in the same batch are superimposed by coordinates to generate a thermal map, revealing the fixed sources of contamination in the mold or fixture.

[0172] Example: In a batch of 50 lenses, the system detected that the pitting was concentrated in the lower left corner area of ​​the lenses (relative coordinates x: 0.1~0.3, y: 0.7~0.9). Process personnel found scratches on the lower left corner of the fixture, causing debris residue. After cleaning the fixture, the pitting disappeared in the next batch. This problem, which would normally require about two days of manual sampling to detect, was flagged by the system within one hour.

[0173] The above four embodiments illustrate in detail the specific implementation of the present invention from different perspectives, including conventional production line inspection, high-precision inspection, model transfer learning, and data statistical analysis. These embodiments can all be repeatedly implemented by those skilled in the art based on the description in the specification, thus satisfying the requirement of sufficient disclosure under patent law.

Claims

1. A method for automatic classification and rating of surface defects in optical lenses, characterized in that, The method includes the following steps: Step S1: Acquire multi-angle surface images of the freeform optical lens under various spectral illuminations; Step S2: Based on the surface shape data of the freeform surface, perform a geometric inverse transformation on the original image and use an improved Retinex algorithm to suppress highlight regions to obtain a preprocessed image; Step S3: Input the preprocessed image into a pre-trained dual-branch deep learning neural network FFO-Net, which simultaneously outputs pixel-level segmentation masks for scratches and pits, as well as target detection boxes; Step S4: Based on the segmentation mask, extract each defect... The pixel size of the defect instance is calculated and converted into the physical size of the defect in millimeters by combining the system calibration coefficient, where the scratch is taken as the maximum width and the pitting is taken as the equivalent diameter; Step S5: Establish a mapping table from the physical size of the defect to the optical standard grades 10 / 5, 20 / 10, 40 / 20, 60 / 40, and 80 / 50, and map the calculated maximum scratch width and maximum pitting diameter to the corresponding scratch grade and pitting grade; Step S6: Output the final grade of the lens, where the final grade is the lower grade between scratch and pitting, i.e., the more stringent grade.

2. The method of claim 1, wherein: In step S1, the multispectral illumination includes three-color LEDs with center wavelengths of 455nm±10nm, 525nm±10nm, and 625nm±10nm, which are independently lit through timing control; the multi-angle illumination includes 0° coaxial light, 30° low-angle ring light, and 60° grazing light; the image acquisition uses a global shutter industrial camera with a pixel size ≤2.4μm and a resolution ≥20 million pixels, and a single acquisition completes 3 to 5 images within 0.2 seconds.

3. The method for automatic classification and rating of surface defects of optical lenses according to claim 1, characterized in that: The geometric inverse transformation expansion employs the equal arc length mapping method, using the freeform surface equation z=f(x,y) or discrete surface point clouds to map points on the three-dimensional surface to two-dimensional plane coordinates. The mapping relationship satisfies... , And bicubic interpolation is used for non-integer coordinates; The improved Retinex algorithm includes: decomposing the image into an illuminance component L and a reflectance component R, estimating L using guided filtering, and setting a specular threshold. (8-bit image), for Adaptive gamma correction (γ=1.5) is applied to the region to output an enhanced image. .

4. The method of claim 1, wherein: The structure of the FFO-Net network includes: A four-level encoder-decoder U-Net++ backbone network, with encoder blocks 1 to 4 outputting 64 / 128 / 256 / 512 channels respectively, the size of which is halved at each level; A cascaded dilated convolution module is embedded at the end of the encoder, which contains four parallel dilated convolution layers with dilation rates of 2, 4, 8 and 16, respectively. The output is reduced to 256 channels by 1×1 convolution. The decoder performs feature fusion through upsampling and skip connections to the corresponding encoder blocks; The final feature map is simultaneously fed into a pixel-level segmentation head and a YOLOv8-based object detection head. The segmentation head contains two 1×1 convolutions, which output scratch probability maps and pockmark probability maps, respectively, with the activation function being Sigmoid. The detection head predicts bounding boxes and categories at three scales.

5. The method of claim 1, wherein: Step S4 specifically includes: Scratch probability map Probability chart of pockmarks Binarize each component with a threshold of 0.5 to remove connected components with an area less than 20 pixels; The minimum circumscribed rectangle of the scratch connected domain is calculated, and the maximum value of the width of the rectangle is taken as the scratch pixel width; the equivalent circle diameter of the pimple connected domain is calculated ; The physical scale conversion coefficient k (mm / pixel) is obtained by the calibration plate, and the actual size is calculated , .

6. The method of claim 1, wherein: The mapping table for step S5 is as follows: Scratch width ≤0.01mm→10 / 5, 0.01~0.02→20 / 10, 0.02~0.04→40 / 20, 0.04~0.06→60 / 40, >0.06→80 / 50; Diameter of the pit ≤0.05mm→10 / 5, 0.05~0.10→20 / 10, 0.10~0.20→40 / 20, 0.20~0.40→60 / 40, >0.40→80 / 50; The final grade is the more stringent one between the scratch grade and the pitting grade, with the strictness order being: 10 / 5 > 20 / 10 > 40 / 20 > 60 / 40 > 80 / 50.

7. The method for automatic classification and rating of surface defects of optical lenses according to claim 1, characterized in that: The method for constructing the training dataset for the FFO-Net network includes: Collect at least 1,000 freeform surface lenses of different grades, take 5 images of each lens, and obtain a total of 5,000 raw images; Independent annotations were performed by at least three quality inspectors with over five years of experience: scratches were outlined using polygons or polylines, with widths less than 3 pixels calculated by adding a width attribute to the center line; pockmarks were annotated with circles or ellipses, with the diameter taken as the average of the major axis of the circumscribed ellipse; arbitration ensured annotation consistency ≥95%; each image was automatically assigned a grade label based on a mapping table; online data enhancement was employed: random rotation ±10°, scaling 0.8~1.2x, horizontal flipping, brightness adjustment 0.7~1.3, contrast 0.8~1.2, and addition of Gaussian noise (…). =0.01) and salt and pepper noise (density 0.002), random occlusion, and out-of-focus blur (Gaussian kernel 3~5 pixels), expanding to 25,000 images, which are divided into training set, validation set and test set in 8:1:

1.

8. The method for automatic classification and rating of surface defects of optical lenses according to claim 1, characterized in that: The training strategy for the FFO-Net network includes: loss function ,in Using DiceLoss and FocalLoss ( =0.25, γ=2) combination, Adopting CIoULoss; Optimizer AdamW, initial learning rate Weight decay Batch size 8, training 200 epochs, cosine annealing decay; Early stopping: Stop if the validation set loss does not decrease for 20 consecutive epochs; The trained model has an error of ≤±2μm in detecting scratch width and ≤±0.01mm in detecting pit diameter.

9. An automated optical lens surface defect classification and grading system implementing the method of any one of claims 1 to 8, characterized in that, include: Image acquisition module: contains multi-angle adjustable multi-spectral light source, high-resolution industrial camera (equipped with telecentric lens, optical magnification 2x~5x), free-form lens objective table and 、 Double shaft rotating mechanism; Image preprocessing module: used to perform the geometric inverse transformation expansion and specular suppression as described in claim 3; Deep learning inference module: Deployed with a pre-trained FFO-Net model; Physical Scale Calculation and Rating Module: Used to perform the size conversion as described in claim 5 and the grade mapping as described in claim 6; Output and Display Module: Used to display the original image, defect overlay mask, final rating results, and automatically generate a PDF report containing a heatmap of defect locations and a histogram of size statistics.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 8; and when producing new models of freeform surface lenses, a transfer learning strategy is adopted: the parameters of the first three levels of the FFO-Net pre-trained model are frozen, only the decoder and segmentation head are fine-tuned, and the program is trained for 50 epochs on 200 newly acquired new model lenses, with a learning rate set to... When outputting the final grade, defects that exceed the preset acceptable grade are highlighted with a red border in the output image.

Images