Intelligent detection method and system for defects of optical lens plastic parts

Abstract

This invention discloses an intelligent detection method and system for defects in plastic parts of optical lenses, belonging to the field of machine vision inspection. The method includes: controlling a multimodal imaging module to acquire bright-field, dark-field, and backlight images of the plastic part under test; inputting the preprocessed images into a deep learning fusion detection network; the network extracting deep features of each modality through parallel feature extraction branches, weighted fusion via an attention mechanism, and outputting defect classification, location, and confidence level; finally, controlling a sorting execution mechanism to automatically sort the defects based on a comparison of the detection results with a tolerance threshold. The system includes a motion platform, a multimodal imaging module, an image processing and controller, and a sorting execution mechanism. This invention achieves high-precision detection and automated sorting of various defects such as scratches and weld lines through multimodal imaging and adaptive feature fusion.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, specifically to an intelligent detection method and system for defects in plastic parts of optical lenses. Background Technology

[0002] Plastic components of optical lenses (such as lens barrels, pressure rings, and spacers) are key components of optical imaging systems, and their molding quality directly affects the lens's assembly accuracy and optical performance. During injection molding, plastic parts are highly susceptible to various defects, including scratches, dirt, weld lines, flash, and bubbles, both on the surface and inside. Traditional inspection methods rely primarily on manual visual inspection, which suffers from low efficiency, high subjectivity, inconsistent standards, and susceptibility to missed defects due to fatigue. These methods fail to meet the demands of modern precision manufacturing for high-speed, high-precision, and highly consistent inspection.

[0003] In recent years, automated optical inspection (AOI) technology based on machine vision has been gradually introduced. However, existing vision inspection solutions typically have the following shortcomings: Single imaging modality: Most solutions use only one illumination method (such as bright field illumination), making it difficult to highlight different types of defects simultaneously. For example, scratches are obvious under bright field illumination, but weld lines and flash are clearer under dark field illumination. A single imaging modality can lead to the missed detection of some defects.

[0004] Limited feature extraction capabilities: Traditional image processing algorithms rely on manually designed features, which are not robust to defects with complex shapes and low contrast. Simple deep learning applications fail to effectively utilize complementary information between multimodal images, resulting in insufficient feature fusion and limited detection and classification accuracy.

[0005] Insufficient detection efficiency and flexibility: The defect detection, classification and sorting processes are fragmented, the system integration is low, it is difficult to achieve closed-loop control, and it is impossible to sort products in real time and accurately based on the detection results.

[0006] Therefore, this solution proposes an intelligent detection method and system for defects in plastic parts of optical lenses to solve the above problems. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, the purpose of this invention is to provide an intelligent detection method and system for defects in plastic parts of optical lenses.

[0008] To achieve this objective, the technical solution of this invention is implemented as follows: A method for intelligent detection of defects in plastic parts of optical lenses, comprising the following steps: Step S1: Image Acquisition The multimodal imaging module is controlled to acquire images of the plastic part of the optical lens under test, which is mounted on a motion platform, and to obtain multimodal image data including at least bright field images, dark field images and backlight images.

[0009] Step S2: Image Preprocessing Denoising, enhancement, and region of interest extraction are performed on the multimodal image data to obtain the standardized multimodal input image.

[0010] Step S3: Defect Feature Fusion and Detection The standardized multimodal input images are fed into a pre-built deep learning fusion detection network. The deep features of each modality image are extracted through the parallel feature extraction branch in the network, and multi-scale feature fusion is performed at the feature pyramid layer. Finally, the defect detection results containing defect classification, defect location and defect confidence are output. Among them, the defect classification includes at least scratches, dirt, weld lines, flash and bubbles.

[0011] Step S4: Defect Identification and Sorting The defect detection results are compared with the preset defect tolerance threshold. Based on the comparison results, a sorting control command is generated to control the sorting execution mechanism to classify the plastic parts of the optical lens to be tested into the qualified product area or the unqualified product area.

[0012] As a preferred embodiment of the present invention, the multimodal imaging module in step S1 includes: A bright field light source, configured as a ring light source or a coaxial light source, is used to illuminate the surface of the plastic part of the optical lens under test at a first incident angle to obtain a bright field image for detecting scratches and dirt. A dark field light source, configured as a low-angle ring light source or a line light source, is used to illuminate the surface of the plastic part of the optical lens under test at a second incident angle to obtain a dark field image for detecting weld lines and flash, wherein the second incident angle is smaller than the first incident angle. A backlight, positioned below the light-transmitting area of ​​the motion platform, is used to transmit light through the plastic part of the optical lens under test to obtain a backlit image for detecting bubbles and internal impurities.

[0013] As a preferred embodiment of the present invention, the deep learning fusion detection network in step S3 includes: The input layer is used to receive the standardized multimodal input image; The multimodal feature extraction backbone network contains at least three structurally identical but parameter-independent convolutional neural network branches, which are used to process bright field images, dark field images and backlit images respectively, to extract their respective modality-specific feature maps; The feature fusion module uses an attention mechanism to weightedly fuse modality-specific feature maps to generate a fused feature map; The detection head performs target detection based on the fused feature map and outputs the defect detection results.

[0014] As a preferred embodiment of the present invention, the feature fusion module is specifically used for: Global average pooling and global max pooling are performed on the feature map specific to each modality to obtain the pooled features of each modality; The pooling features of each modality are concatenated and then input into the fully connected layer to generate the attention weights for each modality. The attention weights are multiplied by the corresponding modality-specific feature maps and then summed element by element to obtain the fused feature map.

[0015] As a preferred embodiment of the present invention, the extraction of the region of interest in step S2 specifically includes: Based on the backlight image, the physical contour of the plastic part of the optical lens under test is located by an adaptive threshold segmentation algorithm; Based on the preset test area template, multiple key test areas are divided within the physical contour. The key test areas include at least the outer wall of the lens barrel, the inner wall of the lens barrel, the bearing surface, and the flange surface. The bright-field image, dark-field image and key detection region are registered together, and the bright-field sub-image and dark-field sub-image corresponding to each key detection region are extracted as the standardized multimodal input image.

[0016] The present invention also provides an intelligent detection system for defects in plastic parts of optical lenses, for implementing any of the above methods, the system comprising: A motion platform used to carry and transport the plastic optical lens component under test; A multimodal imaging module is positioned above and / or below the motion platform to acquire multimodal image data of the plastic part of the optical lens under test in response to control commands. The image processing and controller are electrically connected to the multimodal imaging module, the motion platform, and the sorting actuator, respectively. The image processing and controller includes: The image acquisition and control module is used to control the multimodal imaging module and the motion platform to work together. The image preprocessing module is used to preprocess multimodal image data; The defect detection module is equipped with a deep learning fusion detection network, which is used to perform the defect feature fusion and detection steps mentioned above. The sorting control module is used to generate and send sorting control commands; The sorting execution mechanism is used to classify the plastic parts of the optical lenses to be tested into the qualified or unqualified areas according to the sorting control instructions.

[0017] As a preferred embodiment of the system of the present invention, the multimodal imaging module includes: At least one high-resolution industrial camera; Bright field and dark field light sources that can be detachably mounted on high-resolution industrial cameras; And an independently set backlight; The image acquisition and control module uses time-series control to sequentially control the bright field light source, dark field light source, and backlight source to work in coordination with the high-resolution industrial camera at a single detection station, so as to acquire bright field images, dark field images, and backlight images in a time-division manner.

[0018] As a preferred embodiment of the system of the present invention, the image processing and controller further includes: The online learning module receives the defect detection results after manual review and uses the reviewed image data as new samples to incrementally learn the deep learning fusion detection network in order to update the network parameters.

[0019] Furthermore, the present invention also provides: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent detection method for defects in plastic parts of optical lenses as described above.

[0020] An image processing and control device includes at least one processor and at least one memory, the memory storing at least one program, which, when executed by at least one processor, enables at least one processor to implement the intelligent detection method for defects in plastic parts of optical lenses as described above.

[0021] The beneficial effects of this invention are reflected in: Multimodal imaging complementarity: By acquiring bright field, dark field and backlight images at the same workstation in a time-sharing manner, it can fully highlight the characteristics of various defects such as scratches, weld lines, and bubbles, and overcome the problem of easy omission of defects under single illumination.

[0022] Feature adaptive fusion: Construct a multimodal deep learning network based on attention mechanism, extract deep features of each modality in parallel and dynamically weight and fuse them, which significantly improves the accuracy of defect detection and classification.

[0023] Automated closed-loop control: Integrating image acquisition, intelligent detection and automatic sorting, it achieves high-speed continuous online detection, avoids human subjective error, and ensures detection consistency and efficiency.

[0024] The system is self-evolving: the optimal solution supports incremental learning, and the model can be continuously optimized based on manually reviewed data to adapt to new defect forms and enhance the system's robustness and versatility. Attached Figure Description

[0025] In the attached diagram: Figure 1 The flowchart illustrates the overall process of the intelligent detection method for defects in plastic parts of optical lenses provided in this embodiment of the invention.

[0026] Figure 2 This is a schematic diagram of the structure and optical path of the multimodal imaging module provided in an embodiment of the present invention.

[0027] Figure 3 This is an architecture diagram of a deep learning fusion detection network provided in an embodiment of the present invention.

[0028] Figure 4 This is a flowchart illustrating the workflow of the feature fusion module provided in an embodiment of the present invention.

[0029] Figure 5 This is a structural block diagram of an intelligent detection system for defects in plastic parts of optical lenses provided in an embodiment of the present invention. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the invention, and not all of them. Unless otherwise specified, the embodiments and features described in this application can be combined with each other. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0031] Furthermore, "multiple" refers to two or more. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of a person skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the invention. Example

[0032] Please see Figure 1 This embodiment provides an intelligent detection method for defects in plastic parts of optical lenses.

[0033] Step S1: Image Acquisition This step aims to obtain a multimodal image containing rich defect information. For example... Figure 2 As shown, the multimodal imaging module is configured to acquire three types of images in a single detection station through time-division multiplexing via timing control: Brightfield image: A ring light source or coaxial light source is activated to illuminate the surface of the plastic part under test at a large incident angle (e.g., 45°-90°). For smooth plastic surfaces, scratches and dirt will disrupt the specular reflection properties of the surface, forming high-contrast features of dark or light colors in the brightfield image.

[0034] Dark field imaging: A low-angle ring light or line light source is activated, sweeping across the surface of the plastic part at a very small incident angle (e.g., 10°-20°). Small bumps or abrupt changes in texture on the surface, such as weld lines and flash, will scatter light into the camera, appearing as bright features against a dark background, significantly enhancing the visibility of such defects.

[0035] Backlit Image: With the top light source off, activate the backlight source beneath the motion platform. Light passes through the transparent or translucent plastic parts. If air bubbles or impurities are present inside, they will block the light, creating clear dark spots or shadows in the backlit image, thus clearly revealing the outline and location of internal defects.

[0036] Step S2: Image Preprocessing To improve the accuracy and efficiency of subsequent detection, the three types of images were preprocessed respectively.

[0037] Denoising and Enhancement: Medium-range filtering or Gaussian filtering is applied to remove noise during image acquisition. Subsequently, different enhancement algorithms, such as histogram equalization and contrast-limited adaptive histogram equalization (CLAHE), are used according to the image type (bright field, dark field, backlight) to enhance the contrast between defects and the background.

[0038] Region of Interest (ROI) extraction: This is a crucial step. First, based on the backlit image (where the outline is clearest), the physical outline of the plastic part is accurately located using Otsu's method or an adaptive thresholding algorithm. Then, according to a pre-defined detection region template for this type of plastic part (e.g., an annular band for the outer wall of the lens barrel, a circular area for the inner wall), multiple key detection regions are segmented within the physical outline coordinate system. Finally, using affine or perspective transformations, the bright-field and dark-field images are spatially registered with these key detection regions, and a sub-image corresponding to each region is cropped as input for subsequent networks. This region-based processing transforms the global detection problem into a local fine-grained detection problem, effectively reducing background interference.

[0039] Step S3: Defect Feature Fusion and Detection This step is the core component, implemented through a pre-trained deep learning fusion detection network.

[0040] Please see Figure 3 The network structure specifically includes: Input layer: Receives three sets of images (bright field sub-image, dark field sub-image, and backlight image) after processing in step S2.

[0041] Multimodal feature extraction backbone network: Employs three convolutional neural network branches with identical structures (e.g., all ResNet-50 or EfficientNet) but trained independently by parameters. Each branch processes an image of a specific modality, extracting its deep feature maps step-by-step through multiple convolutional and pooling layers. This parameter-independent parallel structure ensures that each branch can learn the optimal feature representation for its respective modality.

[0042] Feature fusion module: The core of this module is the attention mechanism. For example... Figure 4 As shown, its workflow is as follows: For the feature maps F_{bright}, F_{dark}, and F_{back} output from the three branches, global average pooling and global max pooling are performed in parallel to capture global contextual information and salient features.

[0043] The average pooling result and the max pooling result of each mode are concatenated to form the pooling feature vector of that mode.

[0044] The pooled feature vectors of the three modalities are concatenated and then input into one or more fully connected layers (FC layers). The attention weights w1, w2, and w3 of the three modalities are generated by using the softmax or sigmoid activation function.

[0045] The generated weights are multiplied element-wise with their respective modal feature maps, i.e., wi·Fi.

[0046] Finally, the weighted feature maps are summed element by element to obtain a fused feature map F_{fused}=w1·F_{bright}+w2·F_{dark}+w3·F_{back} that incorporates the advantages of each modality.

[0047] The detection head inputs the fused feature map into a Feature Pyramid Network (FPN) structure for multi-scale feature fusion, enhancing the detection capability for defects of different sizes (such as large areas of dirt and minor scratches). Finally, through classification and regression sub-networks, the final detection results are output, including the category of each detected defect (scratches, dirt, weld lines, burrs, bubbles), bounding box coordinates (location), and confidence score.

[0048] Step S4: Defect Identification and Sorting The image processing and controller receives the defect detection results output in step S3. The system internally presets tolerance thresholds (such as maximum defect area, maximum length, and lower confidence limit) for different defect types and detection areas. If all indicators of the detection results are within the threshold range, the product is judged as qualified; otherwise, it is judged as unqualified. Based on the judgment result, the controller generates corresponding sorting control commands (such as activating the air valve, driving the robotic arm, etc.) to drive the sorting actuator to accurately move the product into the qualified or unqualified product area. Example

[0049] Please see Figure 5 This embodiment provides an intelligent detection system for defects in plastic parts of optical lenses, used to implement the method described in Embodiment 1. The system mainly includes: Motion platform: Typically a circular turntable or linear conveyor belt, with a light-transmitting area (such as a glass turntable). It is responsible for sequentially conveying the plastic parts to be tested to the inspection and sorting stations.

[0050] Multimodal imaging module: Located at the inspection station. Its hardware configuration includes: a high-resolution industrial camera (such as a 12-megapixel CMOS camera), with a detachable ring-shaped bright-field light source and a low-angle ring-shaped dark-field light source mounted below the lens via a quick-change interface; and a backlight mounted below the motion platform, directly opposite the camera. Through image processing and timing control by the controller, different light sources are sequentially illuminated within a very short time (e.g., milliseconds), triggering the camera to take pictures, achieving time-division acquisition of three images.

[0051] Image processing and controller: The core processing unit, which can be an embedded computing platform or an industrial control computer. Its internal modules include: Image acquisition and control module: responsible for coordinating the position signal of the motion platform, the triggering of the light source and the triggering of the camera to ensure that the correct image is acquired at the correct position.

[0052] Image preprocessing module: Executes all image processing algorithms described in step S2 of Example 1.

[0053] Defect detection module: Deploys and runs a trained deep learning fusion detection network, performs inference calculations, and outputs detection results.

[0054] Sorting control module: Outputs control signals based on detection results and preset thresholds.

[0055] Online Learning Module: This is a preferred module for continuous system optimization. When detection results become suspicious or are manually reviewed by users, this module can use the verified images (including missed and false positive samples) as new training data to incrementally learn the network. This approach enables the model to continuously adapt to new production conditions and defect patterns, achieving self-evolution and reducing performance degradation after long-term operation. Example

[0056] This embodiment provides a computer-readable storage medium storing a computer program. When the program is executed by a processor, it can implement the intelligent detection method for defects in plastic parts of optical lenses as described in Embodiment 1. The storage medium can be any medium capable of storing program code, such as ROM, RAM, USB flash drive, portable hard drive, or optical disc. Example

[0057] This embodiment provides an image processing and control device, which includes one or more processors and one or more memories. The memories store one or more programs that, when executed by the processors, enable the processors to implement the intelligent defect detection method for plastic parts of optical lenses as described in Embodiment 1. This device can be an industrial computer, an embedded industrial control computer, or a server cluster, used to deploy and implement the technical solution of this invention.

[0058] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent detection of defects in plastic parts of optical lenses, characterized in that, Includes the following steps: Step S1: Image acquisition. Control the multimodal imaging module to acquire images of the plastic part of the optical lens under test mounted on the motion platform, and obtain multimodal image data including at least bright field image, dark field image and backlight image. Step S2: Image preprocessing, denoising, enhancement and region of interest extraction are performed on the multimodal image data to obtain a standardized multimodal input image; Step S3: Defect Feature Fusion and Detection. The standardized multimodal input image is input into a pre-constructed deep learning fusion detection network. The deep features of each modality image are extracted through the parallel feature extraction branch in the network, and multi-scale feature fusion is performed at the feature pyramid layer. Finally, the defect detection result containing defect classification, defect location, and defect confidence is output. The defect classification includes at least scratches, dirt, weld lines, burrs, and bubbles. Step S4: Defect judgment and sorting. The defect detection results are compared with the preset defect tolerance threshold. Based on the comparison results, a sorting control command is generated to control the sorting execution mechanism to classify the plastic parts of the optical lens to be tested into the qualified product area or the unqualified product area.

2. The intelligent detection method and system for defects in plastic parts of optical lenses according to claim 1, characterized in that, The multimodal imaging module mentioned in step S1 includes: A bright field light source, configured as a ring light source or a coaxial light source, is used to illuminate the surface of the plastic part of the optical lens under test at a first incident angle to obtain a bright field image for detecting scratches and dirt. A dark field light source, configured as a low-angle ring light source or a line light source, is used to illuminate the surface of the plastic part of the optical lens under test at a second incident angle to obtain a dark field image for detecting weld lines and flash, wherein the second incident angle is smaller than the first incident angle. A backlight source is positioned below the light-transmitting area of ​​the motion platform to transmit light through the plastic part of the optical lens under test, thereby obtaining a backlit image for detecting bubbles and internal impurities.

3. The intelligent detection method for defects in plastic parts of optical lenses according to claim 1, characterized in that, The deep learning fusion detection network mentioned in step S3 includes: An input layer is used to receive the standardized multimodal input image; The multimodal feature extraction backbone network includes at least three structurally identical but parameter-independent convolutional neural network branches, which are used to process the bright field image, dark field image and backlight image respectively to extract their respective modality-specific feature maps; The feature fusion module uses an attention mechanism to perform weighted fusion of the modality-specific feature maps to generate a fused feature map; The detection head performs target detection based on the fused feature map and outputs the defect detection result.

4. The intelligent detection method for defects in plastic parts of optical lenses according to claim 3, characterized in that, The feature fusion module is specifically used for: Global average pooling and global max pooling are performed on the feature map specific to each modality to obtain the pooled features of each modality; The pooling features of each modality are concatenated and then input into the fully connected layer to generate the attention weights for each modality. The attention weights are multiplied by the corresponding modality-specific feature maps and then added element by element to obtain the fused feature map.

5. The intelligent detection method for defects in plastic parts of optical lenses according to claim 1, characterized in that, The extraction of the region of interest in step S2 specifically includes: Based on the backlight image, the physical contour of the plastic part of the optical lens under test is located using an adaptive threshold segmentation algorithm; According to the preset test area template, multiple key detection areas are divided within the physical contour. The key detection areas include at least the outer wall of the lens barrel, the inner wall of the lens barrel, the bearing surface, and the flange surface. The bright field image and dark field image are registered with the key detection region, and the bright field sub-image and dark field sub-image corresponding to each key detection region are extracted as the standardized multimodal input image.

6. An intelligent detection system for defects in plastic parts of optical lenses, characterized in that, For implementing the method as described in any one of claims 1 to 5, characterized in that it comprises: A motion platform used to carry and transport the plastic optical lens component to be tested; A multimodal imaging module is disposed above and / or below the motion platform to acquire multimodal image data of the plastic part of the optical lens under test in response to control commands. An image processing and controller is electrically connected to the multimodal imaging module, the motion platform, and the sorting actuator, respectively. The image processing and controller includes: The image acquisition and control module is used to control the multimodal imaging module and the motion platform to work together. An image preprocessing module is used to preprocess the multimodal image data; The defect detection module is equipped with a deep learning fusion detection network for performing the defect feature fusion and detection steps as described in any one of claims 1 to 5; The sorting control module is used to generate and send the sorting control commands. The sorting execution mechanism is used to classify the plastic parts of the optical lens to be tested into the qualified product area or the unqualified product area according to the sorting control instructions.

7. The intelligent detection system for defects in plastic parts of optical lenses according to claim 6, characterized in that, The multimodal imaging module includes: At least one high-resolution industrial camera; Bright field light source and dark field light source that can be detachably mounted on the high-resolution industrial camera; And an independently set backlight; The image acquisition control module uses timing control to sequentially control the bright field light source, dark field light source, and backlight source to work in coordination with the high-resolution industrial camera at a single detection station, so as to acquire the bright field image, dark field image, and backlight image in a time-division manner.

8. The intelligent detection system for defects in plastic parts of optical lenses according to claim 6, characterized in that, The image processing and controller also includes: The online learning module is used to receive the defect detection results after manual review, and to use the reviewed image data as new samples to incrementally learn the deep learning fusion detection network in order to update the network parameters.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the intelligent detection method for defects in plastic parts of optical lenses as described in any one of claims 1 to 5.

10. An image processing and control device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the intelligent detection method for defects in plastic parts of optical lenses as described in any one of claims 1 to 5.

Images