Automatic optical detection equipment
By combining a multi-mode optical imaging system with a dual-branch deep learning network, the problems of single imaging dimension and poor algorithm adaptability of AOI equipment are solved, achieving efficient and accurate defect detection and autonomous equipment optimization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-03-27
AI Technical Summary
Existing AOI equipment has a single imaging dimension, making it difficult to identify defects related to three-dimensional shape. Its algorithm has poor adaptability, leading to missed detections and misjudgments. Furthermore, it is complex to change models and debug, making it difficult to adapt to flexible manufacturing of small batches and multiple varieties.
A multi-mode optical imaging system is adopted in conjunction with a dual-branch deep learning network, integrating two-dimensional, three-dimensional and hyperspectral imaging modules, and incremental training is carried out in conjunction with a self-learning module to achieve multi-dimensional information fusion and intelligent decision-making.
It significantly improves the detection rate and identification accuracy of complex defects, reduces the false alarm and missed alarm rates, and enhances the adaptability of equipment and production flexibility.
Smart Images
Figure CN121740753A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of optical detection equipment, in particular to an automatic optical detection equipment. BACKGROUND
[0002] Automated Optical Inspection (AOI) equipment is a key quality control equipment in modern precision manufacturing, especially in electronic assembly (such as Surface Mount Technology SMT), semiconductor packaging and high-precision component production. Its core principle is to obtain the image of the measured object through an optical imaging system, and to automatically identify various defects on the surface of the product, such as tin bridges, virtual welding, offsets, missing parts, scratches, etc., by using image processing algorithms, so as to realize efficient, non-contact full inspection, replacing traditional inefficient and easy-to-fatigue manual visual inspection.
[0003] With the rapid development of electronic products towards miniaturization, high density and high reliability, the detection accuracy, speed and complex defect recognition ability of AOI equipment are almost demanding. However, the current mainstream AOI technology still faces a series of technical bottlenecks and challenges: limitation of single imaging dimension: traditional AOI equipment mainly relies on high-resolution two-dimensional visible light imaging. This single modal detection method has serious shortcomings in defect recognition ability for defects with low contrast to the background, existing in three-dimensional topography level or related to the material itself. For example, for internal cavities (pores) of solder joints, small height differences (coplanarity) of chip mounting, slight collapse of BGA solder balls, and pollution caused by different materials (such as residual flux and normal solder), it is difficult to make reliable judgments based on two-dimensional gray or color information, which may lead to missed detection or misjudgment. Poor algorithm adaptability, complex changeover debugging: the detection algorithm of mainstream AOI equipment mainly relies on pre-set rules based on brightness, contrast, geometric size and other threshold values (Rule-based). When facing new products, new components or new types of defects, this kind of algorithm has weak generalization ability and requires experienced engineers to spend a lot of time to re-set and debug thousands of detection parameters. This not only increases the use threshold and maintenance cost of the equipment, but also makes the production line changeover inefficient, which is difficult to adapt to the trend of flexible manufacturing of small batches and multiple varieties.
[0004] Therefore, an integrated intelligent AOI equipment capable of integrating multi-dimensional information, having autonomous learning and continuous evolution ability, and being able to efficiently and cooperatively complete defect positioning and classification is urgently needed to fundamentally improve the defect detection rate, recognition accuracy and long-term stability of the equipment in complex manufacturing scenarios. The present application is proposed in this background. SUMMARY
[0005] Based on this, it is necessary to provide an automatic optical inspection equipment for the technical problems of single imaging dimension, low precision and low efficiency of existing optical inspection equipment.
[0006] An automatic optical inspection equipment includes: A rack and a motion bearing platform arranged on the rack; A multi-mode optical imaging system including a main imaging module for collecting two-dimensional surface images of a measured object, a three-dimensional scanning module for collecting three-dimensional topographic information of the measured object, and a hyperspectral imaging module for collecting spectral information of the measured object; A computing control unit including an image processing module and a decision module; The image processing module is configured to receive and process multi-modal image data from the multi-mode optical imaging system; The decision module is configured to run a dual-branch deep learning network, the first branch of which is used for defect positioning, and the second branch is used for defect classification, and the processing results of the multi-modal image data are fused to output the final detection results; The computing control unit further includes a self-learning module for incrementally training the dual-branch deep learning network based on suspected defect sample data after artificial review.
[0007] In one embodiment, the main imaging module described above includes a linear array CMOS camera and a switchable illumination unit, which provides at least two of the following illumination modes: ring light, coaxial light, and low-angle polarized light.
[0008] In one embodiment, the multi-mode optical imaging system described above further includes an infrared thermal imaging module for collecting temperature field distribution images of the measured object in a working state or an excited state.
[0009] In one embodiment, in the dual-branch deep learning network described above, the first branch is a YOLO target detection network embedded with an attention mechanism, and the second branch is a Vision Transformer image classification network.
[0010] In one embodiment, the self-learning module described above is configured to automatically start incremental training during an offline period, and uses a knowledge distillation technique during the training process to retain the ability to recognize historical defect patterns while incorporating new knowledge.
[0011] In an embodiment, the automatic optical inspection device described above further comprises a predictive maintenance module for monitoring the illumination decay of light sources provided by switchable illumination units in the multi-modal optical imaging system, the contamination level of lenses, and the vibration spectrum of motion mechanisms, and generating maintenance warnings in advance based on trend analysis of the monitoring data.
[0012] In an embodiment, the motion-carrying platform described above is a precision motion platform based on linear motors, and the rack is provided with an active damping system; the computing control unit further comprises an environmental temperature control module for maintaining the temperature stability of the imaging area inside the device.
[0013] In an embodiment, the automatic optical inspection device described above further comprises a digital twin module for constructing a virtual model of the device and production process, which receives real-time data from the physical device and supports offline program debugging, collision detection, and production simulation.
[0014] A control method for the automatic optical inspection device, comprising the following steps: S1: receiving a task, initializing the product model parameters, and initializing the motion-carrying platform; S2: synchronously driving the multi-modal optical imaging system to obtain two-dimensional surface images, three-dimensional topographic point cloud data, and hyperspectral image data of the measured object through the main imaging module, three-dimensional scanning module, and hyperspectral imaging module; S3: pre-processing and feature extraction of each modality image data through the image processing module, including image denoising, enhancement, point cloud registration, and spectral feature selection; S4: inputting the processed multi-modal data into the double-branch deep learning network of the decision module for defect positioning and defect classification to monitor potential defect areas and identify defect types; S5: based on the D-S evidence theory, decision-level fusion of the defect positioning results of the first branch and the defect classification results of the second branch to generate defect confidence and final defect judgment results; when a defect is found, mark it as a defect sample and jump to step S6, otherwise jump to step S8; S6: defect confidence judgment on the defect sample; when the confidence of the defect sample is in the fuzzy interval, mark it as a suspected defect sample and jump to step S7; otherwise, confirm it as a defect sample and end the single detection process; S7: sending the suspected defect sample to the artificial composite queue and judging whether it is a defect sample after artificial review; if yes, mark the defect of the defect sample as a new defect and add it to the incremental training sample library, and jump to step S9; if not, it is a false alarm, mark it as a good product, and update the false sample library for algorithm optimization; S8: The sample is marked as good, and the data is archived, and the single detection process is completed; S9: Trigger the self-learning module 23 to perform incremental training during the device idle period to update the network model, and after completing the new model verification and deployment, jump to step S1 to complete the closed-loop optimization.
[0015] In one embodiment, the processing of the three-dimensional topographic point cloud data in step S3 includes extracting three-dimensional geometric features through an improved PointNet++ network; and the processing of the hyperspectral image data includes spectral unmixing and feature band extraction.
[0016] In one embodiment, the specific steps of the decision-level fusion in step S5 include: converting the defect category probability vectors output by each branch network into basic probability assignment functions, synthesizing based on the D-S combination rule to obtain a fused joint basic probability assignment, and finally outputting a judgment result according to a decision rule.
[0017] The automatic optical inspection device described above realizes an active and dynamic visual inspection process for the production process by constructing a cognitive inspection system integrating multi-dimensional perception, intelligent decision-making, and continuous learning. Not only does it greatly improve the efficiency of real-time inspection, but also provides a basis for continuous optimization and intelligent upgrading of the manufacturing process through the closed loop of data and algorithms. Specifically, the technical effect of the present scheme lies in the construction of a multi-dimensional perception system. Two-dimensional imaging serves as the foundation, providing high-resolution surface texture and color information. Three-dimensional scanning endows the device with "depth vision", enabling accurate quantification of geometric features such as solder height, component coplanarity, and scratch depth, and detecting deformation, collapse, or warping defects that traditional 2D cannot detect. Hyperspectral imaging endows the device with "component vision", enabling effective identification of unknown foreign components, detection of coating uniformity, and discovery of material variations caused by oxidation or contamination. The three modalities are fused and analyzed by the computing control unit, and the cross-validation of multi-source evidence enables the system to break through the limitations of single physical quantity detection, significantly improving the detection rate and recognition accuracy of complex and composite defects, and greatly reducing false positives and false negatives. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 FIG. 1 is a structural schematic diagram of an automatic optical inspection device according to one embodiment; Figure 2 FIG. 2 is a control flow diagram of an automatic optical inspection device according to one embodiment. Figure 1 FIG. 2 is a control flow diagram of an automatic optical inspection device according to one embodiment. DETAILED DESCRIPTION
[0019] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0020] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0021] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0022] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0023] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0024] It should be noted that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.
[0025] Please see Figures 1 to 2 This invention discloses an automated optical inspection device, comprising: a frame and a motion support platform mounted on the frame; a multi-mode optical imaging system 10, including a main imaging module 11 for acquiring two-dimensional surface images of the object under test, a three-dimensional scanning module 12 for acquiring three-dimensional morphological information of the object under test, and a hyperspectral imaging module 13 for acquiring spectral information of the object under test; and a computational control unit 20, including an image processing module 21 and a decision module 22. The image processing module 21 receives and processes multi-modal image data from the multi-mode optical imaging system 10; the decision module 22 is configured to run a dual-branch deep learning network, where the first branch is used for defect localization and the second branch is used for defect classification, and the processing results of the multi-modal image data are fused to output a final inspection result; wherein, the computational control unit 20 further includes a self-learning module 23 for incrementally training the dual-branch deep learning network based on manually reviewed suspected defect sample data. Based on the above configuration, this solution is a complete technical solution composed of the multi-mode optical imaging system 10 and a specific computational control unit 20. Its protection scope covers the core system architecture that enables high-precision, multi-dimensional detection and adaptive capabilities. The resulting technical effect is to break through the limitations of the traditional AOI single imaging mode. Through multi-source information fusion and intelligent algorithms, it significantly improves the detection rate and recognition accuracy of complex defects. Furthermore, through self-learning capabilities, the device has the ability to continuously optimize, constituting the most basic and extensive protection of this invention.
[0026] Furthermore, the main imaging module 11 includes a linear CMOS camera 111 and a switchable illumination unit 112, which provides at least two of the following illumination modes: ring illumination, coaxial illumination, and low-angle polarized illumination. Thus, through a switchable and targeted illumination scheme, the imaging characteristics of defects of different natures (such as concavity, scratches, and material abnormalities) can be highlighted, significantly enhancing the information richness and contrast of 2D imaging and providing a high-quality raw data foundation for subsequent accurate analysis.
[0027] Furthermore, the multi-mode optical imaging system 10 also includes an infrared thermal imaging module 14, used to acquire images of the temperature field distribution of the object under test in its working or excited state. Thus, the infrared thermal imaging module 14, through this special mode of infrared thermal imaging, enables the device to detect defects related to thermal characteristics, such as cold solder joints (abnormal temperature rise due to different thermal resistance) and uneven thermal diffusion in internal stacked structures. This expands the device's physical quantity detection dimensions, enabling the detection of certain functional defects that are "invisible" to traditional optics.
[0028] Furthermore, in the dual-branch deep learning network, the first branch is a YOLO object detection network with an embedded attention mechanism, and the second branch is a Vision Transformer image classification network. Thus, employing "YOLO + attention mechanism" facilitates the rapid and accurate localization of minute defect regions in complex backgrounds; while using "Vision Transformer" helps model long-range dependencies in images, enabling more accurate classification of subtle textures and global morphology of defects. The combined effect achieves an optimal balance between accuracy and efficiency in localization and classification tasks, providing a key algorithmic guarantee for high detection performance.
[0029] Furthermore, the self-learning module 23 is configured to automatically initiate incremental training during offline periods, and employs knowledge distillation technology during training to retain the ability to recognize historical defect patterns while integrating new knowledge. Thus, timed offline training does not affect online detection efficiency, and the technical effect of using knowledge distillation technology is to effectively alleviate the "catastrophic forgetting" problem commonly seen in neural networks when continuously learning new defects, ensuring that the original detection capabilities do not degrade during continuous device evolution, achieving stable and reliable autonomous performance improvement.
[0030] Furthermore, the automated optical inspection equipment also includes a predictive maintenance module 30. This module monitors the illuminance attenuation of the light source provided by the switchable illumination unit 112 in the multi-mode optical imaging system 10, the contamination level of the lens, and the vibration spectrum of the motion mechanism. Based on trend analysis of the monitoring data, it generates maintenance warnings in advance. Thus, by transforming equipment health management from "reactive maintenance" or "routine maintenance" to "condition-based predictive maintenance," it is possible to detect performance degradation of optical and mechanical components in advance, allowing intervention before a substantial decrease in inspection accuracy occurs. This significantly improves the overall availability and stability of the equipment and reduces the risk of unexpected downtime.
[0031] Furthermore, the motion support platform is a precision motion platform based on a linear motor, and the frame is equipped with an active vibration damping system; the calculation control unit 20 also includes an environmental temperature control module 24 for maintaining temperature stability in the imaging area inside the device. The combination of the precision linear motor platform and the active vibration damping system ensures extremely high positioning accuracy and dynamic stability during imaging, eliminating the interference of motion errors and external vibrations on image quality. Environmental temperature control reduces the impact of thermal expansion and contraction on mechanical precision and ensures stable performance of optical components. These are the fundamental prerequisites for the entire system to achieve ultra-high-definition (e.g., 1μm) and high-speed detection from a hardware perspective.
[0032] Furthermore, the automated optical inspection equipment also includes a digital twin module 40, used to construct a virtual model of the equipment and production process. This virtual model receives real-time data from the physical equipment and supports offline program debugging, collision detection, and production simulation. By adding this intelligent functional module, the digital twin module 40, real-time interaction and mapping between the physical equipment and the virtual space are achieved. Users can safely and efficiently complete the development of inspection programs for new products, process optimization, and prediction of potential problems in a virtual environment without interrupting production, greatly shortening changeover and debugging time and improving the forward-looking and flexible application of the equipment.
[0033] The present invention also discloses a control method for the aforementioned automated optical inspection equipment, comprising the following steps: S1: Receive task, initialize the parameters added to the product model, and the motion carrier platform is set to... S2: Synchronously drive the multi-mode optical imaging system 10 to acquire two-dimensional surface images, three-dimensional topography point cloud data and hyperspectral image data of the object under test through the main imaging module 11, three-dimensional scanning module 12 and hyperspectral shaping module 13. S3: The image processing module 21 performs preprocessing and feature extraction on the image data of each modality, including image denoising, enhancement, point cloud registration, and spectral feature selection. S4: The processed multimodal data is input into the dual-branch deep learning network of the decision module 22 to perform defect localization and defect classification, so as to monitor potential defect areas and identify defect types. S5: Based on the DS evidence theory, the defect location results of the first branch and the defect classification results of the second branch are fused at the decision level to generate the defect confidence score and the final defect judgment result; when a defect is found, it is marked as a defect sample and the process jumps to step S6; otherwise, the process jumps to step S8. S6: Generate an inspection report, including the location and type of defects, and determine the confidence level of the defect sample. If the confidence level of the defect sample is in the fuzzy range, mark it as a suspected defect sample and proceed to step S7; otherwise, confirm it as a defect sample and end the single inspection process. S7: Send the suspected defective sample to the manual composite queue, and determine whether it is a defective sample after manual review; if it is, mark the defect of the defective sample as a new defect and add it to the incremental training sample library, and jump to step S9; otherwise, it is a false alarm, mark it as a good product, and update the false alarm sample library for algorithm optimization. S8: Mark the sample as good and archive the data; the single testing process is now complete. S9: Trigger the self-learning module 23 to perform incremental training during the device's idle period to update the network model. After completing the new model verification and deployment, jump to step S1 to complete the closed-loop optimization.
[0034] Based on automated optical inspection equipment, the above steps outline the specific process for achieving multimodal fusion detection and autonomous evolution, clarifying a unique method chain of "synchronous acquisition - dual-branch processing - decision fusion - self-learning iteration." The resulting technical effect lies in streamlining the multi-dimensional data acquisition, specific neural network processing architecture, and model evolution mechanism, ensuring the controllability, consistency, and continuous optimization of the detection process—a closed loop that protects the core technical solution from a "process" perspective.
[0035] Furthermore, in step S3, the processing of the three-dimensional topographic point cloud data includes: extracting three-dimensional geometric features using an improved PointNet++ network; the processing of the hyperspectral image data includes: performing spectral unmixing and feature band extraction. Thus, step S2 refines the specific technical means for multimodal data processing. Using PointNet++ to process 3D point clouds enables efficient and robust learning of the spatial hierarchy features of the point cloud, accurately identifying three-dimensional defects such as height, coplanarity, and volume. Analyzing and extracting features from hyperspectral data effectively identifies material types, coating thicknesses, and contaminant components. The resulting technical effect is the full exploitation of the deep information contained in 3D and spectral data, providing richer and more accurate input features for subsequent decision fusion.
[0036] Furthermore, the specific steps of decision-level fusion in step S5 include: transforming the defect category probability vectors output by each branch network into basic probability assignment functions, synthesizing them based on the DS combination rule to obtain the fused joint basic probability assignment, and finally outputting the judgment result according to the decision rule. Step S4 uses DS evidence theory for fusion, and its technical advantage lies in its ability to effectively handle the uncertainty and conflict of sensor (here, the algorithm branch) information. Compared with simple weighted averaging or voting methods, this method can more scientifically and robustly integrate uncertain information from multiple sources, especially when the judgments of each branch are inconsistent, it can give a more reliable final decision, thereby significantly reducing the false alarm and false negative rates.
[0037] In summary, the automated optical inspection equipment disclosed in this invention, by constructing a cognitive inspection system integrating multi-dimensional perception, intelligent decision-making, and continuous learning, achieves proactive and dynamic visual inspection of the production process. This not only significantly improves the efficiency of real-time inspection but also provides a foundation for continuous optimization and intelligent upgrading of the manufacturing process through a closed loop of data and algorithms. Specifically, the technical effect of this solution lies in constructing a multi-dimensional perception system. Two-dimensional imaging serves as the foundation, providing high-resolution surface texture and color information. Three-dimensional scanning endows the equipment with "depth vision," enabling precise quantification of geometric features such as solder joint height, component coplanarity, and scratch depth, detecting deformation, collapse, or warping defects that cannot be detected by traditional 2D methods. Hyperspectral imaging endows the equipment with "compositional vision," effectively identifying unknown foreign matter components, detecting coating uniformity, and discovering material variations caused by oxidation or contamination by analyzing the spectra of different materials. These three modes are fused and analyzed through a computational control unit, and cross-validation of multi-source evidence enables the system to overcome the limitations of detecting a single physical quantity. This results in an order-of-magnitude improvement in the detection rate and identification accuracy of complex and composite defects, and greatly reduces false alarms and missed detections.
[0038] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0039] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.
Claims
1. An automated optical inspection device, characterized in that, include: A frame, and a motion-bearing platform mounted on the frame; A multi-mode optical imaging system includes a main imaging module for acquiring two-dimensional surface images of the object under test, a three-dimensional scanning module for acquiring three-dimensional morphological information of the object under test, and a hyperspectral imaging module for acquiring spectral information of the object under test. The computational control unit includes an image processing module and a decision-making module; The image processing module is used to receive and process multimodal image data from the multimodal optical imaging system; The decision module is configured to run a two-branch deep learning network, where the first branch is used for defect localization and the second branch is used for defect classification. The network also fuses the processing results of the multimodal image data to output the final detection result. The computation control unit further includes a self-learning module, which is used to incrementally train the dual-branch deep learning network based on suspected defect sample data after manual review.
2. The automatic optical inspection device according to claim 1, characterized in that, The main imaging module includes a linear CMOS camera and a switchable illumination unit, which provides at least two of the following modes: ring illumination, coaxial illumination, and low-angle polarized illumination.
3. The automatic optical inspection device according to claim 2, characterized in that, The multi-mode optical imaging system also includes an infrared thermal imaging module, used to acquire temperature field distribution images of the object under test in working or excited states.
4. An automatic optical inspection device according to claim 3, characterized in that, In the dual-branch deep learning network, the first branch is a YOLO object detection network with an embedded attention mechanism, and the second branch is a VisionTransformer image classification network.
5. An automatic optical inspection device according to claim 4, characterized in that, The self-learning module is configured to automatically start incremental training during offline periods, and knowledge distillation technology is used during the training process to retain the ability to identify historical defect patterns while integrating new knowledge.
6. An automatic optical inspection device according to claim 5, characterized in that, The automated optical inspection equipment also includes a predictive maintenance module, which monitors the illuminance attenuation of the light source provided by the switchable illumination unit in the multi-mode optical imaging system, the contamination of the lens, and the vibration spectrum of the motion mechanism, and generates maintenance warnings in advance based on trend analysis of the monitoring data.
7. An automatic optical inspection device according to claim 6, characterized in that, The computing control unit also includes an environmental temperature control module for maintaining a stable temperature in the imaging area inside the device.
8. A control method for an automatic optical inspection device according to any one of claims 1 to 7, characterized in that, Includes the following steps: S1: Receive task, initialize the parameters added to the product model, and the motion carrier platform is set to... S2: Synchronously drive the multi-mode optical imaging system to acquire two-dimensional surface images, three-dimensional topography point cloud data and hyperspectral image data of the object under test through the main imaging module, three-dimensional scanning module and hyperspectral shaping module; S3: The image processing module performs preprocessing and feature extraction on image data of each modality, including image denoising, enhancement, point cloud registration, and spectral feature selection. S4: The processed multimodal data is input into the dual-branch deep learning network of the decision module to perform defect localization and defect classification respectively, so as to monitor potential defect areas and identify defect types; S5: Based on the DS evidence theory, the defect location results of the first branch and the defect classification results of the second branch are fused at the decision level to generate the defect confidence score and the final defect judgment result. If a defect is found, mark it as a defect sample and proceed to step S6; otherwise, proceed to step S8. S6: Determine the confidence level of the defect sample. If the confidence level of the defect sample is in the fuzzy range, mark it as a suspected defect sample and proceed to step S7; otherwise, confirm it as a defect sample and end the single detection process. S7: Send the suspected defective sample to the manual composite queue, and determine whether it is a defective sample after manual review; if it is, mark the defect of the defective sample as a new defect and add it to the incremental training sample library, and jump to step S9; otherwise, it is a false alarm, mark it as a good product, and update the false alarm sample library for algorithm optimization. S8: Mark the sample as good and archive the data; the single testing process is now complete. S9: Trigger the self-learning module 23 to perform incremental training during the device's idle period to update the network model. After completing the new model verification and deployment, jump to step S1 to complete the closed-loop optimization.
9. An automatic optical inspection device according to claim 8, characterized in that, In step S3, the processing of the three-dimensional topographic point cloud data includes: extracting three-dimensional geometric features through an improved PointNet++ network; the processing of the hyperspectral image data includes: performing spectral unmixing and feature band extraction.
10. An automatic optical inspection device according to claim 9, characterized in that, The specific steps of decision-level fusion in step S5 include: converting the defect category probability vectors output by each branch network into basic probability allocation functions, synthesizing them based on the DS combination rule to obtain the fused joint basic probability allocation, and finally outputting the judgment result according to the decision rule.