Computer vision based semiconductor leadframe mold defect detection apparatus

Abstract

The application belongs to the technical field of optical detection, and particularly relates to a semiconductor lead frame mold defect detection device based on computer vision, which comprises a rack and a carrier for carrying a workpiece to be detected, the carrier is fixedly installed on the upper end face of the rack, the side wall of the carrier is provided with a limiting plate for clamping the workpiece to be detected, the upper end face of the rack is provided with a double-shaft sliding rail, a sliding table is slidingly installed on the double-shaft sliding rail, the bottom surface of the sliding table is fixedly installed with a detection part and a mounting plate for mounting a nozzle; the detection part is internally provided with an optical imaging module and an image processing unit, which are used for collecting an initial image of the surface of the mold under active illumination, the initial image contains multispectral information and polarization information; a physical disturbance is applied to a local area on the surface of the workpiece through the nozzle, the energy of the physical disturbance is previously calibrated to be lower than the minimum energy required for completely removing typical attachments; and the image processing unit; the problems of high misjudgment rate caused by similar appearance in pure visual detection are solved, and the overkill rate is significantly reduced.

Description

Technical Field

[0001] This invention belongs to the field of optical inspection technology, specifically a semiconductor lead frame mold defect detection device based on computer vision. Background Technology

[0002] Semiconductor leadframe molds are core forming tools in the integrated circuit packaging process. Their surface quality directly affects the dimensional accuracy of the leadframe, the coplanarity of the leads, and the reliability of subsequent wire bonding. Common abnormalities on the mold surface fall into two categories: one is permanent defects, including scratches, pitting, wear, and microcracks. These defects will directly transfer to the product, leading to batch scrap. The other is surface contaminants, including metal shavings, dust, oil stains, and fingerprints. These contaminants can be removed in an appropriate manner and do not indicate that the mold itself is damaged.

[0003] For example, Chinese invention patent CN112014409A discloses a method and system for detecting defects in semiconductor etched lead frame molds. It directly gives a qualified or unqualified judgment by acquiring images of the workpiece surface and using template matching or deep learning algorithms. The core logic of this type of method is one-time imaging and direct judgment.

[0004] However, since surface deposits and permanent defects may appear similar in grayscale images, the system cannot distinguish between the two based on a single image. When there is dust, oil stains or debris on the mold surface, it is easy to misjudge them as scratches or pits, resulting in a high over-cut rate. A large number of qualified molds are wrongly reported as scrap, which seriously interferes with the normal operation of the production line.

[0005] To this end, the present invention provides a semiconductor lead frame mold defect detection device based on computer vision. Summary of the Invention

[0006] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0007] The technical solution adopted by the present invention to solve its technical problem is: the semiconductor lead frame mold defect detection device based on computer vision of the present invention includes a frame and a platform for carrying the workpiece to be inspected. The platform is fixedly installed on the upper end face of the frame, and the side wall of the platform is provided with a limiting plate for clamping the workpiece to be inspected. The upper surface of the frame is equipped with a dual-axis slide rail, on which a slide table is slidably mounted. The bottom surface of the slide table is fixedly equipped with a detection unit and a mounting plate for mounting the nozzle. The detection unit has a built-in optical imaging module and image processing unit, which are used to acquire an initial image of the mold surface under active illumination. The initial image contains multispectral information and polarization information. Physical disturbance is applied to a localized area of ​​the workpiece surface by spraying through a nozzle. The energy of the physical disturbance is pre-calibrated to be below the minimum energy required to completely remove typical deposits. The image processing unit is used to identify suspected abnormal regions in the initial image, extract the spectral and polarization features of each suspected abnormal region, and control the movement of the slide and the operation of the nozzle. The optical imaging module is also used to acquire a second image of the mold surface after the disturbance. The image processing unit is also used to perform subpixel registration between the initial image and the second image, calculate the quantitative difference index of the registered image in the disturbed area, and determine whether the abnormal area belongs to loose attachment or permanent defect based on the comparison result of the quantitative difference index and the preset threshold.

[0008] The beneficial effects of this invention are as follows: 1. This invention actively verifies whether abnormal areas undergo displacement, deformation, or grayscale changes in response to disturbances by setting up low-energy physical perturbations and comparing sub-pixel registration and quantitative difference of images before and after the perturbation. If there is a response, it is determined to be loose attachment; if there is no response, it is determined to be a permanent defect. The two types of abnormalities can be accurately distinguished without completely removing the attachments, which solves the problem of high misjudgment rate caused by similar appearance in pure visual detection and significantly reduces the over-detection rate.

[0009] 2. This invention achieves closed-loop feedback coupling between perturbation and detection by preserving complete image information before and after perturbation and calculating multi-dimensional quantitative indicators such as differential energy, edge displacement, and area change rate. This avoids the defect in existing technologies where the cleaning process erases the original information, making backtracking verification impossible. At the same time, it can adaptively select gas, polar or non-polar solvent nozzles based on spectral polarization characteristics to achieve intelligent medium matching and improve the reliability of verification in mixed contamination scenarios.

[0010] 3. This invention calibrates the disturbance energy to be below the minimum energy required for complete removal, uses non-contact gas or micro-solvent spraying, and utilizes high-brightness narrowband active illumination, short exposure, and background difference to achieve interference-free anti-interference without a light shield. It causes zero mechanical damage to the mold and eliminates the need for a darkroom. At the same time, it directly skips the disturbance step through high-confidence permanent defects, which significantly improves detection efficiency and reduces operating costs. Attached Figure Description

[0011] The invention will now be further described with reference to the accompanying drawings.

[0012] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a schematic diagram of the slide table in this invention; Figure 3 This is a flowchart of the overall testing process of this invention; Figure 4This is a flowchart of the spectral polarization feature extraction and adaptive medium selection in this invention; Figure 5 This is a flowchart of the dynamic determination and online correction of the expansion radius in this invention.

[0013] In the diagram: 1. Frame; 2. Dual-axis slide rail; 3. Slide table; 4. Detection section; 5. Mounting plate; 6. Platform; 7. Limiting plate. Detailed Implementation

[0014] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0015] like Figures 1 to 5 As shown, the semiconductor lead frame mold defect detection device based on computer vision of the present invention includes a frame 1 and a platform 6 for carrying the workpiece to be inspected. The platform 6 is fixedly installed on the upper end face of the frame 1. The side wall of the platform 6 is provided with a limiting plate 7 for clamping the workpiece to be inspected. The frame 1 serves as the structural support component of the device. The platform 6 is used to place the semiconductor lead frame mold workpiece to be inspected. The limiting plate 7 is provided on the side wall of the platform 6 for clamping and positioning the workpiece to be inspected placed on the platform 6. An electric telescopic rod is provided inside the platform 6. The movable end of the electric telescopic rod is connected to the outer wall of the limiting plate 7 for controlling the sliding of the limiting plate 7. The outer wall of the limiting plate 7 is provided with a rubber pad for contacting the mold workpiece. The limiting plate 7 is driven to slide by the electric telescopic rod to clamp and fix the mold workpiece on the platform 6.

[0016] The upper surface of the frame 1 is provided with a dual-axis slide rail 2, which consists of two mutually perpendicular electric linear guide rails. The slide table 3 is mounted on the dual-axis slide rail 2 and can move in a two-dimensional plane along the dual-axis slide rail 2.

[0017] The upper end of the frame 1 is equipped with a vertical lifting slide rail, and the fixed end of the dual-axis slide rail 2 is connected to the mounting platform of the lifting slide rail, so as to facilitate the lifting and adjustment of the dual-axis slide rail 2 and control the slide table 3 to approach the platform 6 (vertical direction).

[0018] The detection unit 4 and the mounting plate 5 for mounting the nozzle are fixedly installed on the bottom surface of the slide table 3, so that the detection module and the agitation nozzle can move synchronously with the slide table 3.

[0019] The inspection unit 4 has a built-in optical imaging module and an image processing unit. The optical imaging module is used to acquire images of the mold surface, and the image processing unit is responsible for receiving and processing image data, and performing calculations for image recognition, feature extraction, registration and defect determination.

[0020] Under active illumination, the optical imaging module acquires an initial image of the mold surface. Active illumination uses an artificial light source to illuminate the detection area to ensure image acquisition quality and stability. The initial image contains multispectral and polarization information.

[0021] Multispectral information refers to image data acquired within multiple specific wavelengths or bands, which can reflect the reflection or absorption characteristics of an object under different spectra. This can be achieved by placing filters of different bands in front of a light source or by using a multispectral camera to acquire images of red, green, and blue light in three bands at different times or simultaneously.

[0022] Polarization information refers to the polarization state of light waves, which can reflect the surface structure, roughness, and material properties of an object. By installing a rotatable polarizer in front of the camera lens, parallel polarized and perpendicular polarized images are acquired in stages, and then calculations are performed.

[0023] Physical disturbance is applied to a localized area of ​​the workpiece surface by spraying gas or liquid. The physical disturbance exerts mechanical or chemical effects on the localized area of ​​the mold surface by spraying gas or liquid. The energy of the physical disturbance is pre-calibrated to be lower than the minimum energy required to completely remove typical deposits.

[0024] The energy calibration was determined experimentally. For gas disturbances, a fixed air pressure and injection time were set, and for liquid disturbances, a fixed injection volume was set. These parameters were adjusted so that only some loose attachments could be moved or deformed, but not completely removed.

[0025] The image processing unit is used to identify suspected abnormal regions in the initial image, extract the spectral and polarization features of each suspected abnormal region, and control the movement of the slide 3 and the operation of the nozzle. Suspected abnormal regions are areas that the image processing unit initially identifies in the initial image that may have defects or attachments. They can be identified using traditional image processing algorithms, such as edge detection, threshold segmentation, and connected component analysis.

[0026] Spectral features are extracted from multispectral information and are used to describe the optical response characteristics of suspected anomalous regions in different bands. Polarization features are extracted from polarization information and are used to describe the polarization characteristics of suspected anomalous regions, such as calculating the average gray value or contrast of polarized images.

[0027] The image processing unit sends instructions to the motion controller to drive the slide table 3 on the dual-axis slide rail 2 to move above the suspected abnormal area, and at the same time triggers the nozzle to spray through the control circuit.

[0028] The optical imaging module acquires a second image of the mold surface after the disturbance. The second image is an image of the mold surface acquired by the optical imaging module at the same position and under the same lighting conditions after the physical disturbance is completed, in order to obtain the state information after the disturbance.

[0029] The image processing unit is also used to perform subpixel registration between the initial image and the second image, calculate the quantitative difference index of the registered image in the disturbed area, and determine whether the abnormal area belongs to loose attachment or permanent defect based on the comparison result of the quantitative difference index and the preset threshold.

[0030] Subpixel registration is an image registration technique that enables precise alignment between images down to one pixel. It employs algorithms based on feature point matching or region correlation to achieve precise alignment between two images.

[0031] The quantitative difference index quantifies the change before and after the disturbance by comparing the difference between the initial image and the second image within the disturbance area. It is calculated by summing the absolute values ​​of the pixel grayscale differences in the suspected abnormal area before and after the disturbance, or by calculating the average value of the pixel changes within the area.

[0032] A preset threshold is used to compare with a quantitative differential index to distinguish between loose attachments and permanent defects. A pre-defined numerical limit is set by statistically analyzing a large amount of differential index data of loose attachments and permanent defects through experiments to set an empirical threshold that distinguishes between the two.

[0033] The calculated quantitative difference index is compared with a preset threshold. If the difference index is greater than the threshold, it indicates a significant change and is judged as loose attachment. If the difference index is less than or equal to the threshold, it indicates no significant change and is judged as a permanent defect.

[0034] Loose deposits refer to foreign objects on the mold surface that can be removed or moved by physical disturbance, such as dust, debris, and oil stains. Permanent defects refer to structural damage to the mold material itself, such as scratches, pits, and wear, which cannot be removed by physical disturbance.

[0035] By combining multispectral and polarization imaging, low-energy physical perturbation, and subpixel registration and quantitative differential analysis of images before and after perturbation, the problem of high false positive rate caused by the similar appearance of loose attachments and permanent defects in semiconductor lead frame mold inspection is effectively solved. The two types of abnormalities are distinguished, avoiding false alarms for qualified molds and reducing the overkill rate, thereby ensuring the normal operation of the production line and improving the accuracy and efficiency of mold inspection.

[0036] The spectral features include average gray values ​​of at least three different bands; the polarization features include indicators reflecting the degree of polarization; the image processing unit calculates the indicators reflecting the degree of polarization based on the gray value relationship between the parallel polarized image and the cross-polarized image, and constructs an auxiliary judgment index based on the relative difference of the average gray values ​​of at least two different bands to help identify the type of attachment.

[0037] The spectral features include at least three bands. , and The average gray value, polarization features including average polarization degree, are calculated by the image processing unit according to the following formula:

[0038] The type of attachment is further determined using the normalized difference index.

[0039] in, For parallel polarization image grayscale, This represents the grayscale value of the cross-polarized image.

[0040] Specifically, the spectral characteristics include at least three bands. , and The average gray value refers to the average gray value of all pixels in a specific area of ​​an image. The gray values ​​of different wavelengths reflect the reflection or transmission characteristics of an object under the illumination of that wavelength. A multispectral camera with multiple built-in filters can be used. This camera can acquire images of different wavelengths simultaneously or sequentially. For example, at least three wavelengths can be selected from the visible light band (such as blue light 450nm, green light 550nm, red light 650nm) and the near-infrared band (such as 850nm, 940nm) for acquisition.

[0041] In each band of the image, the image processing unit sums the gray values ​​of all pixels within the suspected abnormal region and divides the sum by the number of pixels to obtain the average gray value for that band. By quantifying the average gray value of multiple bands, richer information on material composition can be provided, effectively distinguishing different attachments such as metal scraps, oil stains, and dust from the mold substrate.

[0042] Polarization characteristics include the average degree of polarization, which is a physical quantity that measures the degree of polarization of light waves and reflects the proportion of polarization components in light waves. For object surfaces, their polarization characteristics are closely related to surface roughness, material, optical constants, etc.

[0043] A polarization camera is used, which typically integrates a micro-polarizer array (such as 0°, 45°, 90°, 135°) at each pixel, allowing images with different polarization directions to be acquired in one go.

[0044] The image processing unit calculates the grayscale of the parallel polarization image from the polarization image. and cross-polarized image grayscale (For example, calculated using Stokes vectors or obtained directly from orthogonal polarization direction images), then the average degree of polarization is calculated according to the formula above. The introduction of this feature can intuitively reflect the differences in surface structure and material in abnormal areas, and further distinguish between loose attachments and permanent defects.

[0045] The image processing unit calculates the average degree of polarization according to the above formula:

[0046] This formula clarifies the specific calculation method for the average degree of polarization, ensuring the accuracy and repeatability of the quantization results. The image processing unit receives polarization image data from the optical imaging module and performs the aforementioned arithmetic operations.

[0047] In addition, the image processing unit also uses the normalized difference index:

[0048] It helps determine the type of deposits. The normalized difference index introduces an additional quantitative indicator of spectral characteristics. Through normalization, it enhances the sensitivity to differences between different wavelengths and reduces the influence of factors such as uneven illumination.

[0049] Image processing unit acquires in band and Average gray value of suspected abnormal areas in the acquired image and Then perform the above arithmetic operations.

[0050] In this embodiment, the three wavelengths are three specific wavelengths of light pre-selected based on the material properties of the semiconductor lead frame mold (usually copper alloy, iron-nickel alloy, etc.) and the spectral reflectance characteristics of common adhering substances (metal scraps, oil stains, dust), as follows: (e.g., 450nm blue light): It is sensitive to minute scratches, pits and other subtle structural changes on the metal surface, and can enhance the contrast of defect edges. At the same time, metal debris usually exhibits high reflectivity and high brightness characteristics in this band, making it easy to distinguish from dark oil stains.

[0051] (For example, 550nm green light): It is located in the wavelength range that the human eye is most sensitive to, and it has a good response to common pollutants such as dust and fibers. It can be used as a reference wavelength to eliminate common-mode interference caused by uneven illumination.

[0052] (For example, 850nm near-infrared): It is sensitive to oils and organic pollutants. Oil stains will show strong light absorption characteristics (significantly reduced reflectivity) in this band, while metal substrates and metal debris will maintain high reflectivity.

[0053] The normalized difference index amplifies the spectral reflectance differences between different anomalies by normalizing the gray values ​​of specific bands and effectively suppresses external interference from uneven illumination, thereby improving the distinguishability of features.

[0054] To address the problem of fuzzy and inaccurate anomaly type identification through quantitative characterization of spectral and polarization features, this application specifies spectral features as having at least three bands. , and The average gray value, and the average polarization degree is introduced. Its calculation formula and normalized difference index As an auxiliary basis for judgment, it enables precise quantification of the material composition and surface characteristics of suspected abnormal areas.

[0055] Different types of deposits and permanent defects exhibit significant differences in their reflection and polarization characteristics across different wavebands. By quantifying the average grayscale values ​​across multiple wavebands, richer information on material composition can be provided, effectively distinguishing different deposits such as metal scraps, oil stains, and dust from the mold substrate.

[0056] Meanwhile, average polarization degree The introduction of this feature can intuitively reflect the differences in surface structure and material in abnormal areas, further distinguishing between loose attachments and permanent defects, and normalizing the difference index. By normalizing the gray values ​​of specific bands, the spectral reflectance differences between different anomalies are amplified, and external interference from uneven illumination is effectively suppressed, thereby improving the distinguishability of the features.

[0057] By acquiring quantitative features, a clear and comparable decision basis is provided for the subsequent image processing unit to accurately determine the anomaly type and select the appropriate perturbation medium. This solves the problem of misjudgment caused by fuzzy feature representation, improves the accuracy and reliability of semiconductor lead frame mold defect detection, avoids qualified molds being falsely reported as scrap, and ensures the normal operation of the production line.

[0058] Compared to existing technologies that rely solely on single-channel images or indiscriminate cleaning, this solution combines multispectral and polarization information to achieve online prediction of the composition of materials in abnormal areas. This upgrades the detection process from passive observation to intelligent prediction based on physical optical properties, providing a direct and reliable decision-making basis for the subsequent adaptive selection of gas, polar solvent, or non-polar solvent perturbation head.

[0059] The mounting plate is equipped with at least three independently controlled nozzles, used to output clean gas, polar volatile solvents, and non-polar volatile solvents, respectively; the image processing unit selects the corresponding nozzle based on the relationship between spectral and polarization characteristics and their respective preset thresholds. When the index reflecting the degree of polarization is higher than the first polarization threshold and the auxiliary judgment index is higher than the first index threshold, the gas nozzle is selected. When the index reflecting the degree of polarization is lower than the second polarization threshold and the auxiliary judgment index is lower than the second index threshold, a polar solvent nozzle is selected. When none of the above conditions are met and the morphological characteristics of the abnormal area point to a stubborn oil film, a non-polar solvent nozzle should be selected.

[0060] Three independently controlled nozzles are installed on mounting plate 5.

[0061] The image processing unit selects the corresponding nozzle based on spectral and polarization characteristics: like and If so, then select a gas nozzle; like and If so, select a polar solvent nozzle; If none of the above conditions are met and the morphological characteristics indicate a stubborn oil film, then a non-polar solvent nozzle should be selected. in, , , and This is a preset threshold.

[0062] The mounting plate 5 is equipped with multiple spray devices, each of which can be independently turned on and off and spray parameters (such as pressure, flow rate, time, etc.) can be adjusted. The nozzles are controlled by a miniature solenoid valve array.

[0063] The clean gas used is filtered compressed air or inert gas (such as nitrogen). Its main function is to disturb the loose deposits (such as metal shavings and dust particles) on the mold surface through the physical impact force of the airflow, causing them to shift or fall off.

[0064] Polar volatile solvents are selected from organic solvents with high polarity, such as isopropanol (IPA), ethanol or acetone. The solvent can effectively dissolve or soften polar deposits, such as water-soluble stains and fingerprint residues, by using chemical action to assist physical disturbance.

[0065] Non-polar volatile solvents are selected from low-polarity organic solvents, such as n-hexane, toluene, or xylene. These solvents are mainly used to dissolve or disperse non-polar deposits, especially stubborn oil films and grease contaminants, changing their form through chemical action. The above media are all volatile to ensure that the media can evaporate quickly from the mold surface after disturbance, avoiding secondary pollution of the residue for subsequent testing or mold use.

[0066] As the decision center, the image processing unit receives and analyzes the spectral and polarization features in the initial image obtained from the optical imaging module. Based on these features, the image processing unit performs preset logical judgments to intelligently select the most suitable perturbation medium for the current suspected abnormal region.

[0067] The selection process is automated and requires no human intervention, ensuring the objectivity and consistency of the detection. Spectral and polarization characteristics are important parameters for describing the optical properties of an object's surface. Spectral characteristics reflect the reflection, absorption, or scattering properties of an object under different wavelengths of light and can reveal information about the chemical composition of the material. Polarization characteristics describe the vibration direction of light waves and have unique advantages in distinguishing surface roughness, material type, and the physical state of the deposits (such as crystal structure and film thickness). By combining these two characteristics, the type of deposits on the mold surface can be characterized more comprehensively and accurately.

[0068] When the image processing unit determines if and At this time, the condition judgment logic aims to identify abnormal regions with high polarization degree and high normalized difference index. High polarization degree is usually associated with rough surface and strong scattering particles (such as metal debris and dust), while high normalized difference index may further distinguish the specific spectral response of these particles. When the condition is met, the image processing unit determines that the abnormal region is very likely to be loose solid particle attachment. Therefore, a clean gas nozzle is selected to disturb it and use the impact force of the airflow to make it displaced.

[0069] When the image processing unit determines if and At this time, the condition judgment logic aims to identify abnormal regions with low polarization degree and low normalized difference index. Low polarization degree may indicate that the surface is relatively smooth or the adhering material is thin film, while low normalized difference index may correspond to the spectral characteristics of certain polar substances. When the condition is met, the image processing unit determines that the abnormal region may be a polar stain (such as water-soluble residue or fingerprint), and therefore selects a polar volatile solvent nozzle to agitate it, changing its morphology through dissolution.

[0070] When the image processing unit determines that none of the above conditions are met and the morphological features point to a stubborn oil film, this condition serves as a supplement to the two aforementioned cases and is used to handle anomalies that do not meet the gas or polar solvent disturbance conditions.

[0071] When the image processing unit determines that an abnormal region is a stubborn oil film by analyzing its morphological features (e.g., its shape, edge blurring, texture, etc.) and possible spectral polarization features (although it does not meet the aforementioned threshold conditions, it is still indicative), it selects a non-polar volatile solvent nozzle to agitate the region. The non-polar solvent has the solubility for the oil film and can effectively cause a detectable change in its morphology.

[0072] , , and The preset threshold, based on the actual application scenario, mold material, common attachment types, and experimental data, is used as a decision boundary to distinguish different types of spectral and polarization characteristics.

[0073] By adjusting the above thresholds, the system can adapt to different detection standards and environmental conditions, achieving accurate classification of attachment types and optimized selection of disturbance media. It can be configured during system initialization or dynamically optimized through machine learning or expert experience.

[0074] By setting at least three independently controlled nozzles on the mounting plate 5 to provide clean gas, polar volatile solvent and non-polar volatile solvent respectively, targeted disturbance methods are prepared for various types of deposits that may appear on the mold surface. The image processing unit no longer blindly applies a single disturbance, but judges the nature of the suspected abnormal area based on the spectral features and polarization features extracted from the initial image.

[0075] For areas with high polarization and high normalized difference index, the system can identify the possibility of solid particle deposits and accurately select gas nozzles for physical purging. For areas with low polarization and low normalized difference index, the system can determine that they are polar stains and select polar solvents for dissolution and disturbance. For areas that do not meet the above conditions but whose morphological characteristics indicate stubborn oil films, the system can select non-polar solvents for targeted treatment.

[0076] Through an adaptive perturbation medium selection mechanism, each perturbation can produce positional or morphological changes in the target attachment that can be captured by the optical imaging module with the lowest energy and the most suitable medium. This significantly improves the accuracy of distinguishing loose attachments from permanent defects. At the same time, it avoids unnecessary solvent waste and potential contamination of the mold, improves detection efficiency and system robustness, and effectively reduces the misjudgment rate when detecting defects in semiconductor lead frame molds.

[0077] The energy of physical disturbance is defined as follows: the actual applied disturbance energy is equal to a preset scaling factor between 0 and 1 multiplied by the minimum energy required to completely remove the corresponding type of deposit; wherein, for gas disturbance, the disturbance energy is determined by the product of the gas injection pressure and the injection pulse width; for solvent disturbance, the disturbance energy is determined by the volume of a single injection.

[0078] In this regard, this application further proposes that the energy calibration of the physical disturbance satisfies:

[0079] in, The minimum energy required to completely remove the corresponding type of deposit. It is a preset proportional constant and ; For gas disturbances:

[0080] in, To use pressure, The pulse width; For solvent disturbances:

[0081] That is, the volume of a single spray.

[0082] Among them, the energy calibration formula for physical disturbance is:

[0083] Defines the energy of the physical disturbance actually applied. Minimum energy required to remove a specific type of deposit The quantitative relationship between them is determined by introducing a preset proportional constant. This ensures that the actual applied disturbance energy is always lower than the energy required to completely remove the deposits, thus achieving non-destructive disturbance of the deposits.

[0084] in, It refers to the minimum physical disturbance energy required to completely remove different types of deposits (such as metal shavings, dust, oil stains, etc.) from the mold surface without damaging the mold body.

[0085] Minimum energy values ​​usually need to be determined through prior experimental calibration or based on empirical data to ensure their accuracy. For example, for metal shavings of a specific size and material, it may be necessary to continuously spray gas at a specific pressure for a certain period of time to completely remove them. For oil stains of a specific viscosity, it may be necessary to use a specific volume of solvent to completely dissolve and remove them.

[0086] It is a preset proportionality constant between 0 and 1, whose function is to adjust the actual applied perturbation energy so that it is always less than 1. By adjusting The value can control the degree of detectable displacement or deformation of the attached material without completely removing it.

[0087] For gas disturbances, the actual disturbance energy used By the gas operating pressure With pulse width The product is used to represent this, where, This refers to the instantaneous or average pressure when the gas nozzle injects gas. It can be precisely controlled by adjusting the gas source pressure valve or the nozzle drive circuit, and can be set to 0.1MPa, 0.2MPa, etc.

[0088] This refers to the duration of gas injection, which can be precisely adjusted by controlling the opening time of the nozzle valve. It can be set to 10ms, 50ms, 100ms, etc.

[0089] This product form effectively reflects the momentum and energy exerted by the gas on the adherent material, facilitating precise system control based on the material's characteristics. For solvent disturbances, the actual disturbance energy used... Directly from the volume of a single spray To characterize.

[0090] in, This refers to the total amount of solvent released by the nozzle in a single spraying operation. It is achieved by controlling the opening time of the nozzle valve, the nozzle orifice diameter, and the flow rate of the solvent pump. It is set to 1 microliter, 5 microliters, 10 microliters, etc. Using the volume of a single spray as an energy indicator can intuitively reflect the dissolving and rinsing effect of the solvent on the attached objects (especially oil stains), which is convenient for quantitative control of the system.

[0091] By introducing a proportionality constant To ensure the actual applied disturbance energy Always below the minimum energy required to completely remove the corresponding type of deposit. This avoids the situation where excessive energy completely removes loose deposits, making it impossible to obtain images before and after disturbance for judgment. It also avoids unnecessary damage to the mold body. At the same time, it avoids the problem of insufficient energy causing loose deposits to produce detectable displacement or deformation, leading to misjudgment as permanent defects, thus significantly improving the accuracy of detection.

[0092] For gas disturbances, energy Defined as pressure With pulse width The product of these factors conforms to the physical characteristics of the effect of gas disturbance on the attached material, enabling the system to precisely control the intensity of the gas disturbance.

[0093] For solvent perturbation, energy Defined as single spray volume This directly reflects the solvent's dissolving and scouring effect on the adhering substances, making it easy to quantitatively control solvent disturbance.

[0094] After refined energy calibration, the detection unit 4 can acquire an initial image under active illumination and then apply just the right amount of physical disturbance to a local area on the workpiece surface through the nozzle. This can cause identifiable displacement or deformation of loose attachments without completely removing them. This provides reliable basic data for the image processing unit to perform sub-pixel registration between the initial and second images, calculate quantitative difference indices, and ultimately determine whether the abnormal area belongs to loose attachments or permanent defects. This greatly improves the ability and accuracy of the semiconductor lead frame mold defect detection device to distinguish between loose attachments and permanent defects.

[0095] The quantitative difference indices include: the intensity of grayscale difference in the image before and after the perturbation near the abnormal region, the overall positional offset of the abnormal region, and the degree of change in the area of ​​the abnormal region; the intensity of grayscale difference is obtained by calculating the average pixel grayscale difference in the joint region before and after the perturbation, and the joint region is formed by the union of the abnormal regions before and after the perturbation after dilation; the positional offset is the geometric distance between the centers of the abnormal regions before and after the perturbation; the degree of change in area is the ratio of the absolute value of the difference between the areas before and after the perturbation to the area before the perturbation.

[0096] In response, this application further proposes quantitative differential indices including differential energy, edge displacement, and area change rate, which are calculated as follows:

[0097] Differential energy:

[0098] in, , The preset expansion radius; Edge displacement The Euclidean distance between the centroids of the anomalous region before and after the perturbation; Area change rate:

[0099] in, and These represent the areas of the abnormal regions before and after the disturbance.

[0100] Quantitative difference indexes are used to quantify and compare the changes in a specific region of an image at different time points (i.e. before and after the perturbation). They are constructed based on changes in pixel grayscale values, color information, texture features, etc.

[0101] Differential energy It measures the average intensity of the change in pixel grayscale values ​​within a specific region of the image before and after the perturbation. Besides using the mean absolute difference, the mean squared difference can also be used, i.e.:

[0102] Before calculation, the grayscale values ​​can be preprocessed, such as by gamma correction or histogram equalization, to enhance contrast. Then, the differential energy can be calculated, which reflects the degree of impact of the disturbance on the overall visual characteristics of the abnormal area.

[0103] It is a unified region for differential computation, which analyzes the anomalous regions before and after the perturbation. and The result is obtained by performing dilation and taking the union. The dilation operation can use different structural elements, such as circles, squares or rhombuses, and the appropriate size can be selected according to the shape and size of the abnormal region.

[0104] In addition to morphological dilation, smoothing operations such as Gaussian blur can be used to expand the region before performing a union operation. Unifying the region is to ensure that the difference calculation can fully cover all the ranges that the attachment may move or spread before and after the disturbance, thereby avoiding information omissions due to the movement of the attachment.

[0105] The preset expansion radius is a parameter used for morphological expansion operations, which determines the expansion range. This radius can be preset to a fixed value based on empirical values ​​or experimental data, determined according to the size of common attachments on the mold and the expected displacement range, or dynamically adjusted by looking up tables or simple functional relationships based on factors such as image resolution, the initial size or shape of abnormal areas, etc. Its function is to control the coverage of the differential calculation area to balance calculation accuracy and computational load.

[0106] Edge displacement It measures the change in the center position of the abnormal area before and after the disturbance. Euclidean distance can be used, but Manhattan distance or Chebyshev distance can also be used to measure the centroid displacement. Edge displacement can directly reflect the overall movement trend of the abnormal area under the action of disturbance, which is crucial for distinguishing between movable attachments and fixed defects.

[0107] Area change rate It measures the relative degree of change in the area of ​​the anomalous region before and after the disturbance. It can be expressed as the ratio of the absolute difference to the area before the disturbance, or as the ratio of the difference to the average area of ​​the two.

[0108] Area can be calculated based on pixel counts or estimated by fitting geometric shapes (such as ellipses or polygons). The rate of change of area can quantify the change in shape or size of an abnormal region under disturbance. For example, loose attachments may be blown away or dissolved, resulting in a decrease or spread of area.

[0109] and These represent the areas of the abnormal regions before and after the disturbance. In image processing, the abnormal regions are obtained by segmentation or recognition algorithms, and their corresponding physical size is determined by counting pixels in the binarized abnormal regions. This data serves as the basis for calculating the rate of change of area and reflects the actual physical extent of the abnormal regions before and after the disturbance.

[0110] Differential energy expands and merges the abnormal regions before and after the disturbance to form a differential calculation region that fully covers the range of the attachment's movement. Within this region, the average absolute value of the pixel grayscale difference before and after the disturbance is calculated, which can comprehensively and accurately reflect the impact of the disturbance on the overall visual characteristics of the abnormal region. Loose attachments show significant grayscale changes under disturbance, while permanent defects show only slight changes.

[0111] Edge displacement directly quantifies the overall positional shift of the abnormal region by calculating the Euclidean distance between the centroids of the abnormal region before and after the disturbance. Loose attachments are prone to displacement under physical disturbance, and their centroids will move significantly, while permanent defects remain fixed, and their centroid displacement is close to zero.

[0112] The area change rate is calculated by measuring the relative change in the area of ​​the abnormal region before and after the disturbance. It reflects the change in the shape or size of the abnormal region under the influence of the disturbance. Loose attachments (such as dust and oil stains) may be blown away, dissolved or diffused under the disturbance, resulting in a significant change in area, while the area of ​​permanent defects remains basically unchanged.

[0113] By combining quantitative differential indicators, the problem of existing solutions being unable to accurately distinguish between loose attachments and permanent defects by a single indicator is solved. This significantly improves the accuracy of defect judgment and avoids problems such as high over-kill rate and low production line efficiency caused by misjudgment. Compared with the traditional method of cleaning before detection, by always retaining two complete images before and after the disturbance and calculating quantitative indicators such as differential energy, edge displacement and area change rate, the dynamic response data of abnormal areas to disturbance can be directly obtained. This achieves closed-loop coupling between disturbance and detection, providing a multi-dimensional and reliable basis for subsequent classification and judgment.

[0114] The image processing unit selects a preset threshold corresponding to the type of perturbation medium currently in use, and performs the following determination: If any of the following exceeds the threshold corresponding to the current medium: grayscale difference intensity, positional offset, or area change, it is determined to be loose attachment. Otherwise, if the medium currently in use is consistent with the preferred medium inferred from the spectral and polarization characteristics, it is determined to be a permanent defect; If there is a discrepancy, switch to the preferred medium and re-perturb and make subsequent judgments.

[0115] In response, this application further proposes that the image processing unit selects a corresponding preset threshold based on the currently used perturbation medium, and the determination logic is as follows: like or or If the current medium matches the preferred medium predicted by the feature analysis, it is determined to be a permanent defect. If they do not match, the preferred medium is switched and the perturbation and determination are re-executed. Pick , or These correspond to gases, polar solvents, and non-polar solvents, respectively. , and These are the preset thresholds for each medium.

[0116] The image processing unit selects the corresponding preset threshold according to the currently used perturbation medium, and dynamically adjusts the threshold used for defect determination according to the type of medium to which the physical perturbation is actually applied (e.g., clean gas, polar volatile solvent or non-polar volatile solvent).

[0117] Because different media exert different mechanisms and intensities of influence on adhering materials—for example, gas disturbance primarily displaces the adhering material through impact force, while solvent disturbance alters the morphology or adhesion of the adhering material through dissolution or wetting—the differential energy of loosely attached materials after disturbance in different media varies. Edge displacement and area change rate The magnitude of change of equal quantitative difference indicators can vary significantly.

[0118] By presetting a specific threshold for each type of disturbance medium ( , and This ensures that the judgment logic can accurately reflect the actual physical response under the action of the medium, thereby improving the accuracy of the judgment.

[0119] Specifically, the thresholds corresponding to different media can be stored in the memory of the image processing unit, and when making a judgment, the corresponding threshold set can be read from the memory according to the type of perturbation medium currently being used.

[0120] In this decision logic, if or or If the image is subjected to physical disturbances below the energy required for complete removal, it is identified as loose attachment. When loose attachment is subjected to physical disturbances below the energy required for complete removal, its appearance on the image (such as grayscale, position, and shape) will undergo observable changes. After sub-pixel registration of the image after disturbance and the image before disturbance and calculation of quantitative differential indices, if any one of these indices (differential energy, edge displacement, or area change rate) exceeds the preset threshold corresponding to the current disturbance medium, it indicates that the abnormal area has a significant response to the disturbance, which is a typical characteristic of loose attachment. Because permanent defects (such as scratches and pits) usually do not undergo such changes under low-energy disturbances, this "or" logic ensures that as long as any one of the indices shows a sufficient response, it can be classified as loose attachment, thus improving the sensitivity of the determination.

[0121] Otherwise, if the medium currently used is consistent with the preferred medium inferred from the features, it is determined to be a permanent defect. When all quantitative differential indicators do not exceed the preset threshold, if the preferred medium most suitable for clearing the abnormal area inferred by the image processing unit based on the spectral and polarization features of the initial image is the same as the perturbation medium currently used, it is inferred that even if the most suitable low-energy perturbation is used, the abnormal area does not produce an observable physical response, indicating that the abnormal area is not a loose attachment, but a permanent defect of the mold itself. This judgment avoids misjudgment caused by improper medium selection and enhances the reliability of the judgment.

[0122] If there is a discrepancy, the system switches to the preferred medium and re-executes the perturbation and judgment process. In cases where all quantitative differential indicators do not exceed the preset threshold and the currently used perturbation medium is inconsistent with the preferred medium inferred from the features, it cannot be immediately determined to be a permanent defect. This is because the currently used medium may not be suitable for removing this type of loose attachment, resulting in insufficient physical response. To avoid such misjudgment, the image processing unit controls the system to switch to the preferred medium derived from the initial feature analysis, and then re-executes the physical perturbation, image acquisition, and quantitative differential indicator calculation and judgment process. This ensures that the medium most likely to remove the attachment is used for verification at least once before making a final judgment, thereby significantly reducing the risk of misjudging removable loose attachments as permanent defects.

[0123] in, Pick , or These correspond to gases, polar solvents, and non-polar solvents, respectively. , and The preset thresholds for each medium clarify the type of disturbance medium and the naming rules for its corresponding thresholds.

[0124] Represents clean gas. Represents a polar volatile solvent. Representing nonpolar volatile solvents, each medium corresponds to an independent set of threshold values, i.e. , and Used for gas disturbance , and Used for perturbation of polar solvents , and Used for perturbation of nonpolar solvents.

[0125] The preset thresholds are determined through experiments and statistical analysis of the response of different types of loose attachments under low-energy disturbances in different media before the system is put into use. They represent the distinction between loose attachments and permanent defects in terms of quantitative differential index under the action of a specific medium. These thresholds can be stored in the configuration parameters of the image processing unit and can be fine-tuned and optimized according to the actual production environment and mold material characteristics.

[0126] The image processing unit selects the corresponding preset threshold according to the current perturbation medium, which results in different magnitudes of changes in features such as differential energy, edge displacement, and area change rate after the loose attachment is disturbed. By matching its exclusive preset threshold to each medium, the system can adapt to the characteristics of different media, avoid the judgment bias that may be caused by using a uniform threshold, and make the judgment results more accurate.

[0127] In the classification judgment logic, if any quantitative difference index exceeds the preset threshold corresponding to the current medium, it is judged as loose attachment. This captures the characteristic that the gray scale, position or area of ​​loose attachment will change significantly after being subjected to low energy disturbance.

[0128] If any one indicator meets the required change range, the abnormality can be confirmed as a removable deposit, balancing the sensitivity of the judgment with the efficiency of the detection.

[0129] Furthermore, when all quantitative differential indicators do not exceed the corresponding thresholds, the system does not immediately determine it as a permanent defect, but first determines whether the medium currently in use is consistent with the preferred medium previously inferred based on spectral polarization characteristics.

[0130] If they match, it means that the disturbance has been completed using a suitable medium that meets the prediction, and no observable characteristic changes have occurred in the abnormal area. At this point, it can be quickly and stably determined to be a permanent defect in the mold itself.

[0131] If there is a discrepancy, the system will intelligently switch to the preferred medium and re-execute the disturbance and judgment process. This retry mechanism effectively avoids the problem of loose attachments being misjudged as permanent defects due to an improper initial medium selection that fails to fully stimulate their physical response.

[0132] By employing a multi-media, adaptive threshold, and retesting strategy, this application elevates the judgment criteria from simple appearance similarity to verification based on differences in physical behavior, significantly reducing the false alarm rate and ensuring the overall accuracy of defect detection. This prevents a large number of qualified molds from being falsely reported as scrap, thus ensuring the normal operation of the production line.

[0133] When the image processing unit determines that the probability of matching the spectral or morphological features of the abnormal area with the permanent defect is greater than the preset confidence threshold, it directly outputs the permanent defect determination without triggering the nozzle to work.

[0134] When the image processing unit determines that the probability of matching the spectral or morphological features of an abnormal region with a permanent defect is greater than a preset confidence threshold... At this time, the permanent defect judgment is directly output without triggering the nozzle to work.

[0135] The image processing unit is used to determine the matching probability between the spectral or morphological features of the abnormal region and the permanent defect. The spectral features can be derived from the multispectral and polarization information in the initial image acquired by the optical imaging module, and can include the average gray value and average polarization degree of at least three bands.

[0136] Morphological features can refer to the geometry, size, texture, aspect ratio, compactness, etc. of the abnormal region. These features are extracted from the initial image through image processing algorithms. The matching probability is calculated based on a pre-trained machine learning model, such as a classifier like a support vector machine (SVM), neural network, or random forest. By inputting the spectral and morphological features of the abnormal region, the output is the probability value that the region belongs to a permanent defect.

[0137] Preset reliability threshold It is a threshold value used to determine whether the matching probability is high enough to directly identify it as a permanent defect. This threshold balances detection efficiency and the risk of false positives. Through experiments and verification on a large number of known defect and non-defect samples, the threshold is empirically adjusted and optimized according to the requirements of the actual production line for false positive and false negative rates.

[0138] When the image processing unit determines that the probability of matching an abnormal region is greater than a preset confidence threshold When the system directly outputs a permanent defect determination, the detection device will immediately mark the area as a permanent defect and output this result through the display interface, data log, or interface with the Manufacturing Execution System (MES) without the need for subsequent physical disturbance detection processes.

[0139] At the same time, the system will not trigger the nozzle to work, that is, the control system will suppress the working commands of the dual-axis slide rail 2 and the nozzle to avoid gas or solvent spraying disturbance in this area.

[0140] When the image processing unit determines an abnormal region as a permanent defect with high confidence based on its spectral and morphological features, the system can skip time-consuming steps such as slide 3 movement, nozzle spraying disturbance, secondary imaging, and image registration difference, and directly output the determination result, thus shortening the detection time of clear defects and improving the overall detection efficiency.

[0141] Furthermore, by avoiding unnecessary nozzle operation, reducing the consumption of disturbing media, and decreasing nozzle wear, the operating costs and maintenance requirements of the equipment are effectively reduced.

[0142] The repeatability of the dual-axis slide rail 2 and slide stage 3 is less than the pixel resolution of the optical imaging module; the registration accuracy of subpixel registration is better than 0.1 pixels; the active illumination uses a high-brightness narrow-band light source, the camera exposure time is much less than 1 millisecond, and real-time background difference is used to subtract ambient light interference.

[0143] To reduce errors at the source, ensure the accuracy of image difference calculations before and after disturbance, and provide a basis for accurately distinguishing between loose attachments and permanent defects, the repeatability of the dual-axis slide rail 2 and slide table 3 is crucial. Smaller than the pixel resolution of the optical imaging module Registration accuracy of subpixel registration Less than 0.1 pixels; active illumination uses a high-brightness narrow-band light source, and the camera exposure time... satisfy It also employs real-time background difference to subtract ambient light interference.

[0144] Specifically, the repeatability of the dual-axis slide rail 2 and the slide table 3 refers to the positional consistency that the slide table 3 can achieve when returning to the same target position after multiple round trips, while the pixel resolution of the optical imaging module refers to the actual physical size that each pixel can represent on the image sensor.

[0145] Repeat positioning accuracy Controlled to a resolution smaller than the pixel resolution Within this range, it means that the mechanical position deviation of the slide 3 between two imaging sessions will not exceed the physical size of one pixel. To achieve this goal, a high-precision linear motor is used to drive the dual-axis slide rail 2, and a high-resolution grating ruler or magnetic grating ruler is used for position feedback. The movement of the slide 3 is precisely controlled by a closed-loop control system.

[0146] Subpixel registration refers to improving the alignment accuracy of images to less than one pixel during the image registration process. Less than 0.1 pixels means that when aligning two images before and after the perturbation, their relative position error is precisely controlled within one-tenth of a pixel.

[0147] To achieve this high-precision registration, subpixel-level displacement is determined by calculating the phase difference of the Fourier transforms of the two images, for example, by using Fast Fourier Transform (FFT) and upsampling techniques.

[0148] Active lighting employs high-brightness narrowband light sources, which are light sources that emit a specific wavelength range and have high light intensity. Such light sources can enhance the contrast of specific spectral characteristics and provide sufficient illumination to support short exposure times. They employ high-power LED arrays whose emission wavelengths are either filtered or are themselves narrowband spectra, such as using blue or green LED light sources with specific wavelengths.

[0149] Camera exposure time satisfy This refers to the extremely short time that the camera sensor receives light, much less than 1 millisecond. Short exposure time helps to freeze motion and avoid image blurring caused by minor vibrations of the slide 3 or the workpiece. This is achieved by using a high frame rate industrial camera and the aforementioned high-brightness light source to ensure that an image with sufficient brightness can still be obtained within an extremely short exposure time.

[0150] Real-time background subtraction of ambient light interference eliminates the influence of stray ambient light on the grayscale value of the target image by acquiring the ambient background image in real time and subtracting it from the currently acquired image. This is achieved by acquiring a background image without or with the workpiece occluded before each acquisition of the workpiece image, and then subtracting the background image from the workpiece image.

[0151] By controlling the repeatability of the dual-axis slide rail 2 and slide stage 3 to within a range smaller than the pixel resolution of the optical imaging module, excessive mechanical position deviation of slide stage 3 when it moves into place twice is effectively avoided, thereby limiting the initial mechanical error to within a single pixel and greatly reducing the difficulty and computational burden of subsequent image registration.

[0152] Based on this, by achieving a sub-pixel registration accuracy of less than 0.1 pixels, the slight positional deviation remaining after mechanical positioning is further eliminated in a more refined manner, ensuring that the corresponding pixels of the two images before and after the disturbance can be accurately aligned. This avoids false differences introduced by inaccurate registration and provides a solid foundation for the accurate calculation of subsequent quantitative difference indicators.

[0153] In addition, the use of a high-brightness narrow-band light source for active illumination not only improves the contrast of abnormal areas on the mold surface, making its features clearer, but also, in conjunction with a camera exposure time of much less than 1ms, effectively suppresses motion blur that may occur during the movement of the mechanism, ensuring the clarity and detail integrity of the image, and meeting the requirements of high-precision differential calculation.

[0154] Meanwhile, by subtracting ambient light interference through real-time background difference technology, the grayscale values ​​of the acquired images are ensured to truly reflect the optical characteristics of the mold surface. This eliminates the influence of stray ambient light on the accuracy of image data, further ensuring the reliability of the differential index calculation and improving the accuracy of the image difference calculation before and after disturbance. As a result, it is possible to more reliably and accurately distinguish between loose attachments and permanent defects on the mold surface, effectively solving the misjudgment problem caused by insufficient image quality and registration accuracy in existing technologies, reducing the overkill rate, and improving detection efficiency and reliability.

[0155] Expansion radius The displacement is dynamically determined based on the spectral characteristics of the abnormal region and the type of the disturbing medium; the image processing unit pre-establishes a mapping relationship between the type of attachment and the expected displacement. For regions identified as metal debris, the expected displacement is... Due to gas disturbance pressure and particle size estimate The following formula is given:

[0156] in, This is a preset proportionality constant; for areas identified as oil stains, the expected displacement is determined by the solvent spray volume. And oil film viscosity estimate The following formula is given:

[0157] Final expansion radius taken ,in For safety factor and , Minimum expansion radius; expected displacement Also based on the displacement actually measured at the previous moment Perform online corrections:

[0158] in For learning rate, By dynamically determining the expansion radius, the differential calculation region is adapted to the actual displacement range of the attached material.

[0159] The method for optimizing the difference computation region, where the expansion radius... The expansion radius is dynamically determined based on the spectral characteristics of the anomalous region and the type of the perturbed medium. It is a key parameter used to define the size of the difference calculation region. Its dynamic determination means that the radius is not a fixed value, but is adjusted according to the information detected in real time.

[0160] The image processing unit can pre-store the types of attachments (such as metal debris, oil stains, dust, etc.) corresponding to different spectral features (e.g., gray-scale ratios of different bands, degree of polarization, etc.) and the recommended initial expansion radius range.

[0161] Once the spectral characteristics of the abnormal region are identified, a suitable initial value for the expansion radius is inferred from a pre-set lookup table or through a machine learning model, taking into account the type of perturbation medium currently in use (such as gas, polar solvent, or non-polar solvent).

[0162] To achieve dynamic determination of the expansion radius, a mapping relationship between the type of attachment and the expected displacement is established in advance. This mapping relationship is a knowledge base or model used to predict the displacement of different types of attachments under specific disturbance conditions. The mapping relationship is the basis for dynamically determining the expansion radius. It links the physical characteristics of the attachment and the disturbance parameters, thereby providing a reasonable displacement prediction.

[0163] This mapping relationship can be established using experimental data to create a lookup table. For example, under laboratory conditions, gas disturbances of different pressures can be applied to metal scraps of different sizes and materials, and their displacement can be recorded. These data can then be compiled into a table. Similarly, solvent sprays of different volumes can be applied to oil stains of different viscosities and thicknesses, and their diffusion or displacement can be recorded.

[0164] Furthermore, for areas identified as metal debris, the expected displacement is... Due to gas disturbance pressure and particle size estimate The following formula is given:

[0165] in, As a preset scaling constant, after identifying a suspected metal debris area, the image processing unit can estimate the particle size s (e.g., maximum diameter or equivalent diameter) through image analysis (such as edge detection and morphological operations).

[0166] At the same time, gas disturbance pressure These are the known parameters of the system control; by substituting them into the formula, the expected displacement can be calculated.

[0167] proportionality constant The particle size can be obtained through experimental calibration, or, in addition to directly estimating the particle size, it can be combined with information such as the particle's grayscale value and texture features, and a pre-trained machine learning model (such as a regression model) can be used to estimate the particle size more accurately. Alternatively, the displacement can be predicted directly.

[0168] For areas identified as oil stains, the expected displacement is determined by the solvent spray volume. And oil film viscosity estimate The following formula is given:

[0169] After identifying a suspected oil stain area, the image processing unit can estimate the viscosity of the oil film using multispectral information (such as absorption or reflection characteristics of specific wavelength bands) or polarization information (such as degree of polarization). .

[0170] Solvent spray volume These are the known parameters of the system control; by substituting them into the formula, the expected displacement can be calculated.

[0171] proportionality constant Obtained through experimental calibration, or, oil film viscosity. The estimation can also be performed by analyzing the morphological characteristics of the oil stain area (such as the degree of edge diffusion and surface smoothness) combined with its spectral and polarization characteristics, using image processing algorithms or machine learning models. Furthermore, the solvent spray volume... It can be precisely controlled and quantified based on parameters such as nozzle type, spray time, and spray pressure.

[0172] Final expansion radius taken ,in For safety factor and , This is the minimum expansion radius.

[0173] Image processing unit in computation Then, compare it with the preset safety factor. Multiplying these values ​​yields an expansion value based on the predicted displacement. Simultaneously, the system presets a minimum expansion radius. (For example, based on the pixel resolution of the optical imaging module or the minimum detectable defect size).

[0174] Final expansion radius Choose the larger of these two values; the safety factor is... It doesn't have to be a fixed value, but rather dynamically adjusted based on factors such as the type of attachment, the intensity of disturbance, and environmental conditions. For example, for attachment types whose displacement is difficult to predict, It can be set to larger. It can also be adjusted according to the precision requirements of the mold or the minimum size of the defect to ensure that there is a sufficient differential area even if the predicted displacement is very small.

[0175] In addition, the expected displacement Also based on the displacement actually measured at the previous moment Perform online corrections:

[0176] in For learning rate, After each perturbation and difference calculation, the image processing unit obtains a measured edge displacement. ,in Used as feedback, in relation to the current Together, the new exponential smoothing formula is calculated using the above formula. This new prediction will be used to calculate the expansion radius when the same type of attachment is disturbed again.

[0177] Learning rate It is set to a fixed value, for example, between 0.1 and 0.5, to balance the influence of historical data and the latest observations.

[0178] By dynamically determining the expansion radius, the differential calculation region is adapted to the actual displacement range of the attachment. The image processing unit uses this dynamically determined expansion radius when performing differential energy calculations. To build a joint region The accuracy of this region directly affects the differential energy. Edge displacement and area change rate The accuracy of the calculation and the realization of dynamically determining the expansion radius depend on the comprehensive analysis and real-time calculation of the spectral characteristics of the abnormal region, the type of the disturbing medium, the type of the attachment, the disturbance parameters, and historical displacement data.

[0179] By dynamically adjusting the expansion radius, it can accurately adapt to the actual displacement range of different types of attachments under different disturbance intensities, thus overcoming the limitation that a fixed expansion radius cannot adapt to the displacement of diverse attachments. This dynamic adjustment avoids the omission of abnormal area information after disturbance due to an excessively small expansion radius, ensuring the integrity of differential calculation. It also avoids noise interference caused by introducing too many irrelevant pixels due to an excessively large expansion radius, ensuring the accuracy of differential index calculations such as differential energy, edge displacement, and area change rate.

[0180] The accuracy of the differential index directly improves the accuracy of subsequent judgments that distinguish between loose attachments and permanent defects, effectively reducing misjudgments and omissions. In addition, the introduction of a calculation model for expected displacement based on physical laws, such as the positive correlation between metal debris displacement and gas pressure and particle size, and the positive correlation between oil stain displacement and solvent volume and negative correlation with viscosity, makes the prediction results closer to reality.

[0181] The online correction mechanism enables the system to continuously optimize the prediction of expected displacement by learning from historical data, which improves the robustness and adaptability of the model and ensures the stability and accuracy of long-term operation.

[0182] Specifically, initial image acquisition and anomaly region identification: Once a mold is placed on stage 6 and fixed, the optical imaging module acquires an initial image of the mold surface under active illumination. The active illumination uses a high-brightness narrow-band light source, and the camera exposure time is... satisfy The initial image contains multispectral information (e.g., at least three bands) and employs real-time background subtraction to remove ambient light interference. , and (average gray value) and polarization information (parallel polarized image gray value) and cross-polarized image grayscale The image processing unit receives the initial image and identifies a suspected abnormal area.

[0183] Feature extraction and perturbation medium selection: The image processing unit extracts the spectral and polarization features of the suspected anomalous region. The spectral features include bands. , and The average gray value and polarization characteristics are calculated by measuring the average degree of polarization. Simultaneously, the image processing unit obtains the normalized difference index. To aid in judgment, the analysis results are assumed to indicate that the region... and This set of features points to loose particles, such as metal shavings. Therefore, the image processing unit selects the gas nozzle as the preferred disturbance medium and controls the slide 3 to move so that the gas nozzle is aimed at the suspected abnormal area.

[0184] Controlled physical disturbance: The gas nozzle sprays clean gas into the area, applying physical disturbance; the energy of this physical disturbance... Pre-calibrated to be below the minimum energy required to completely remove typical deposits. Specifically satisfy ,in For gas disturbances, energy is generated through the use of pressure. and pulse width Control, i.e. The image processing unit pre-establishes the attachment type and expected displacement. The mapping relationship between them, for areas identified as metal debris, the expected displacement. Due to gas disturbance pressure and particle size estimate pass Provided.

[0185] Image acquisition and differential analysis after perturbation: After the disturbance, the optical imaging module acquires a second image of the mold surface again. If the disturbance medium used this time is a solvent (polar or non-polar), the device first waits for a preset evaporation time (e.g., 0.5 to 2 seconds, the specific value is calibrated according to the solvent volatility and ambient temperature). After the solvent evaporates, the optical imaging module acquires a second image of the mold surface. If gas disturbance is used, the image is acquired directly, and the image processing unit performs sub-pixel registration between the initial image and the second image, with registration accuracy... For pixels smaller than 0.1 pixels, after registration, the image processing unit calculates quantitative differential indices within the perturbation region, including differential energy. Edge displacement and area change rate .

[0186] Differential energy ,in , The expansion radius is dynamically determined.

[0187] Expansion radius The anomaly region is dynamically determined based on its spectral characteristics and the type of perturbed medium, and then... ,in For safety factor and , This is the minimum expansion radius.

[0188] Expected displacement Also based on the displacement actually measured at the previous moment Perform online correction, i.e. ,in For learning rate, By dynamically determining the expansion radius, the differential calculation region is adapted to the actual displacement range of the attached object.

[0189] Edge displacement The Euclidean distance between the centroids of the anomalous region before and after the perturbation, and the rate of change of area. .

[0190] Defect identification: The image processing unit selects the corresponding preset threshold based on the currently used perturbation medium (gas). , and If the calculation result shows or or If so, the abnormal area is determined to be loosely attached material. In this example, assuming the calculation result shows... Significantly greater than This indicates that the area has undergone significant changes under slight gas disturbance, and the image processing unit therefore determines that the abnormal area is loose attachment (e.g., dust or metal debris).

[0191] Unlike existing technologies that rely solely on a single image for direct determination, this method effectively distinguishes between loosely attached materials and permanent defects that appear similar by introducing controlled physical disturbances and differential image analysis before and after the disturbance. If traditional methods are used, this area may be misjudged as scratches or pits due to its initial appearance characteristics, leading to the incorrect scrapping of the mold. However, this solution accurately identifies the material as removable attachments by observing significant changes after the disturbance, thus preventing misjudgment.

[0192] Suspected stubborn oil film detected: Feature extraction and perturbation medium selection: In mold inspection, the image processing unit identifies suspected abnormal areas and extracts their spectral and polarization features. Assuming the analysis results show... and Meanwhile, the morphological features point to a stubborn oil film. According to the preset logic, the image processing unit selects a non-polar solvent nozzle as the preferred disturbance medium and controls the slide 3 to move so that the non-polar solvent nozzle is aligned with the area.

[0193] Controlled physical disturbance: The non-polar solvent nozzle sprays non-polar volatile solvent into this area, and the energy of the solvent disturbance... By single spray volume Control, and For areas identified as oil stains, the expected displacement is... From solvent spray volume And oil film viscosity estimate pass Provided.

[0194] Image acquisition and differential analysis after perturbation: After the disturbance, the optical imaging module acquires a second image, the image processing unit performs sub-pixel registration, and calculates the differential energy. Edge displacement and area change rate .

[0195] Defect identification: The image processing unit selects the corresponding preset threshold based on the currently used perturbation medium (non-polar solvent). , and If all differential indicators do not exceed the threshold, the following judgment process will be initiated: First, it is determined whether the medium currently used is consistent with the preferred medium inferred from the spectral and polarization characteristics. In this example, the current medium (non-polar solvent) is consistent with the preferred medium. However, it cannot be directly determined to be a permanent defect because the feature inference may have errors. Therefore, the system will try other unused perturbation media in sequence (in a preset priority order, for example, if the preferred medium is non-polar solvent, then try gas and polar solvent in sequence) to re-perform low-energy perturbation, secondary imaging and differential analysis on the same abnormal area.

[0196] If, after trying a certain medium, any quantitative difference index exceeds the preset threshold corresponding to that medium, it is determined to be loose attachment and subsequent attempts will be stopped.

[0197] If all available media (gas, polar solvent, non-polar solvent) have been tried and no differential index exceeds its respective threshold, then it is ultimately determined to be a permanent defect.

[0198] If the current medium is inconsistent with the preferred medium predicted by the feature (e.g., the current medium is a gas while the preferred medium is a non-polar solvent), the system will first switch to the preferred medium to re-perturb and determine the medium, instead of immediately starting multi-medium traversal.

[0199] In this example, if all differential indices still do not exceed the threshold after trying gas and polar solvent in turn, the image processing unit will ultimately determine that the area is a permanent defect.

[0200] Special Case: High-Confidence Permanent Defect In some cases, if the image processing unit determines that the probability of a match between the spectral or morphological features of an abnormal region and a permanent defect is greater than a preset confidence threshold, (For example, if a clear deep scratch has spectral and morphological characteristics that are highly consistent with known permanent defects), the device will directly output a permanent defect determination without triggering the nozzle to work, thereby improving detection efficiency.

[0201] By combining multispectral and polarization information for preliminary identification, and then through controlled physical perturbation and precise image difference analysis before and after the perturbation, the surface attachments and permanent defects of semiconductor lead frame molds can be accurately distinguished. This reduces the misjudgment rate caused by appearance similarity in traditional methods, and improves the accuracy of mold inspection and the efficiency of the production line.

[0202] It should be noted that the disturbance energy used in this device is pre-calibrated to be lower than the minimum energy required for the complete removal of typical deposits. For some stubborn deposits that require higher energy to remove (such as sintered particles, cured adhesive residues, and high-temperature oxidation products), they may not produce detectable displacement or deformation under low-energy disturbance, and thus be judged as permanent defects. Users can adjust the energy level according to the actual needs of the production line. The core of this device is to distinguish between loose deposits that respond under low energy and surface anomalies that do not respond. The latter, whether permanent defects or stubborn deposits, require manual re-inspection or special treatment in actual production.

[0203] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A semiconductor lead frame mold defect detection device based on computer vision, comprising a frame and a platform for carrying the workpiece to be inspected, the platform being fixedly mounted on the upper end face of the frame, and a limiting plate for clamping the workpiece to be inspected being provided on the side wall of the platform, characterized in that: The upper surface of the frame is provided with a dual-axis slide rail, a slide table is slidably mounted on the dual-axis slide rail, and a detection part and a mounting plate for mounting the nozzle are fixedly mounted on the bottom surface of the slide table. The mounting plate is equipped with at least three independently controlled nozzles, which are used to output clean gas, polar volatile solvents and non-polar volatile solvents, respectively. The detection unit has a built-in optical imaging module and an image processing unit, which are used to acquire an initial image of the mold surface under active illumination. The initial image contains multispectral information and polarization information. The image processing unit is used to identify suspected abnormal regions in the initial image, extract the spectral and polarization features of each suspected abnormal region, and control the movement of the slide and the operation of the nozzle. The image processing unit selects the corresponding nozzle based on the relationship between the spectral features and polarization features and their respective preset thresholds; Physical disturbances are applied to suspected abnormal areas on the surface of a workpiece by spraying through a nozzle. The energy of these physical disturbances is pre-calibrated to be below the minimum energy required to completely remove typical deposits. The optical imaging module is also used to acquire a second image of the mold surface again after the disturbance; The image processing unit is also used to perform sub-pixel registration of the initial image and the second image, calculate the quantitative difference index of the registered image in the disturbance area, and determine whether the abnormal area belongs to loose attachment or permanent defect based on the comparison result of the quantitative difference index and the preset threshold.

2. The semiconductor lead frame mold defect detection device based on computer vision according to claim 1, characterized in that: The spectral features include average gray values ​​in at least three different spectral bands, and the polarization features include indicators reflecting the degree of polarization. The image processing unit calculates the index reflecting the degree of polarization based on the grayscale relationship between the parallel polarized image and the cross-polarized image, and constructs an auxiliary judgment index based on the relative difference of the average grayscale values ​​of at least two different bands to help identify the type of attachment.

3. The semiconductor lead frame mold defect detection device based on computer vision according to claim 2, characterized in that: When the index reflecting the degree of polarization is higher than the first polarization threshold and the auxiliary judgment index is higher than the first index threshold, the gas nozzle is selected; When the index reflecting the degree of polarization is lower than the second polarization threshold and the auxiliary judgment index is lower than the second index threshold, a polar solvent nozzle is selected. When none of the above conditions are met and the morphological characteristics of the abnormal area point to a stubborn oil film, a non-polar solvent nozzle should be selected.

4. The semiconductor lead frame mold defect detection device based on computer vision according to claim 1, characterized in that: The energy of the physical disturbance is defined as: The actual applied disturbance energy is equal to a preset scaling factor between 0 and 1 multiplied by the minimum energy required to completely remove the corresponding type of deposit; Specifically, for gas disturbances, the disturbance energy is determined by the product of the gas injection pressure and the injection pulse width; for solvent disturbances, the disturbance energy is determined by the volume of a single injection.

5. The semiconductor lead frame mold defect detection device based on computer vision according to claim 1, characterized in that: The quantitative differential indicators include: the intensity of grayscale difference in the image before and after the disturbance near the abnormal region, the overall positional offset of the abnormal region, and the degree of area change of the abnormal region; The grayscale difference intensity is obtained by calculating the average pixel grayscale difference in the joint region before and after the perturbation. The joint region is formed by the union of the abnormal regions before and after the perturbation after dilation. The position offset is the geometric distance between the centers of the abnormal area before and after the disturbance; The degree of area change is the ratio of the absolute value of the difference between the areas before and after the disturbance to the area before the disturbance.

6. The semiconductor lead frame mold defect detection device based on computer vision according to claim 5, characterized in that: The image processing unit selects a preset threshold corresponding to the type of perturbation medium currently in use, and performs the following determination: If any one of the grayscale difference intensity, the positional offset, or the area change exceeds the threshold corresponding to the current medium, it is determined to be loose attachment. Otherwise, if the medium currently in use is consistent with the preferred medium inferred from the spectral and polarization characteristics, it is determined to be a permanent defect; If there is a discrepancy, switch to the preferred medium and re-perturb and make subsequent determinations.

7. The semiconductor lead frame mold defect detection device based on computer vision according to claim 1, characterized in that: When the image processing unit determines that the probability of matching the spectral or morphological features of the abnormal area with the permanent defect is greater than the preset confidence threshold, it directly outputs the permanent defect determination without triggering the nozzle to work.

8. The semiconductor lead frame mold defect detection device based on computer vision according to claim 1, characterized in that: The repeatability of the dual-axis slide rail and slide table is less than the pixel resolution of the optical imaging module, and the registration accuracy of the sub-pixel registration is better than 0.1 pixels. The active illumination uses a high-brightness narrowband light source, the camera exposure time is much less than 1 millisecond, and real-time background difference is used to subtract ambient light interference.

9. The semiconductor lead frame mold defect detection device based on computer vision according to claim 5, characterized in that: Expansion radius The displacement is dynamically determined based on the spectral characteristics of the abnormal region and the type of the disturbing medium; the image processing unit pre-establishes a mapping relationship between the type of attachment and the expected displacement, and for regions identified as metal debris, the expected displacement is... Due to gas disturbance pressure and particle size estimate It is given by the following formula: ； in, This is a preset proportionality constant; for areas identified as oil stains, the expected displacement is determined by the solvent spray volume. And oil film viscosity estimate It is given by the following formula: ； Final expansion radius taken ,in For safety factor and , For the minimum expansion radius, It is a proportionality constant; the expected displacement is... Also based on the displacement actually measured at the previous moment Perform online corrections: ； in For learning rate, By dynamically determining the expansion radius, the differential calculation region is adapted to the actual displacement range of the attached material.

Images