Intelligent detection system for chip appearance defect

Abstract

The present invention belongs to the technical field of chip defect detection. Disclosed is an intelligent detection system for a chip appearance defect. According to the present invention, chip appearance defects are categorized according to similarity of image representations, and on the basis of detection advantages of different detection algorithms, the defects are detected in corresponding categories, so that more than 39 types of common and uncommon defects can be detected, thereby achieving comprehensive detection of the chip appearance defects. In addition, during defect detection, filtering parameters are set to filter NG-type products determined on the basis of recognition confidence, and strict detection parameters are used to ensure that all defects are found. Moreover, user-defined filtering parameters are allowed on a software interface. A user can adjust the filtering parameters on the basis of an enterprise quality standard to control detection quality.

Description

An intelligent detection system for chip appearance defects Technical Field

[0001] The present invention relates to the technical field of chip detection, and in particular to an intelligent detection system for chip appearance defects. Background Art

[0002] Before the finished chip is shipped, the appearance of the product will be inspected to ensure that there is no abnormality in the appearance of the assembled components. The front, back and side components need to be inspected, and quality inspections are also carried out before packaging and shipment.

[0003] Existing chip appearance inspections are mostly performed manually. Due to functional differences, work intensity, fatigue, and other reasons, manual inspections are very likely to lead to missed or false detections of defects. The inspection speed is slow and the efficiency is low. In addition, chip inspections use traditional algorithms without a training platform, deep learning, or intelligent algorithms. Although the inspection speed is superior to manual inspections, it still cannot solve the problems of missed and false detections of defects. At the same time, existing AOI inspections mainly focus on the quality of patch and plug-in welding-related processes, and the detection rate of irregular defects such as scratches, component damage, and dirt is not high.

[0004] Summary of the Invention

[0005] Technical purpose: In response to the above-mentioned shortcomings of the existing chip appearance defect detection, the present invention discloses an intelligent chip appearance defect detection system that can automatically detect chip surface defects comprehensively and improve the detection rate.

[0006] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] An intelligent chip appearance defect detection method classifies defects based on their appearance, grouping defects with similar image representations into one category. Based on the defect category, the method then selects a corresponding detection method from traditional visual algorithm detection, AI visual algorithm detection, and a combination of traditional and AI visual algorithms to identify and detect the corresponding defects.

[0008] During inspection, an image of the chip surface is first acquired. After preprocessing the image, the detection algorithm is used to detect defects and identify the type of defects.

[0009] According to the recognition confidence of the area in the detection image corresponding to the sample image, the recognition results are divided into three categories: OK, NG and suspicious and saved separately. Suspicious results are randomly inspected and reviewed.

[0010] Preferably, the image preprocessing process of the present invention includes: performing reference positioning and mask processing on the image, and cropping the image while performing image brightness equalization to keep the image to be detected corresponding to the sample image.

[0011] Preferably, the recognition confidence of the present invention is the similarity between the detection image and the sample image. If the recognition confidence is greater than 0.85 and less than 0.95, the detection result is set as a suspicious class. For the sampling and review results of suspicious objects, after the sampling and review are completed, the results are fed back to the detection system. The system locally labels and trains, generates a new model, and performs semi-automatic learning.

[0012] Preferably, the present invention uses a multi-scale feature fusion approach to detect small defects in chip appearance defects, extracting features of different scales from different layers of the image detection network for prediction.

[0013] Preferably, the present invention sets filtering parameters during defect detection, and the filtering parameters are upper and lower limit range parameters related to the corresponding defect size, number, shape, and color. The quality of the detected chip products is controlled by using the filtering parameters to divide the NG-type images according to the recognition confidence level, and the original NG-type images are filtered and divided twice, and the images of the detected defects and filtered by the filtering parameters are used as the final NG images.

[0014] Preferably, the present invention reviews the NG-type images generated by filtering. When the review result is inconsistent with the algorithm detection result, the parameters of the reviewed image are compared with the filtering parameters of the corresponding defect detection. When only one detection indicator exceeds the upper and lower limits of the filtering parameters, the corresponding filtering parameter range is adjusted; and after adjusting the filtering parameters, the identified and classified pictures are used for verification.

[0015] Preferably, the process of obtaining the chip surface image of the present invention includes: setting a reference photographing size, and when the chip size is smaller than the reference photographing size, using a single photographing method to obtain images of the bottom, front and side surfaces of the chip; when the chip size is larger than the reference photographing size, dividing the bottom, side and front surfaces of the chip into single photographing areas with a size smaller than the reference photographing size, photographing the divided photographing areas separately, and after the photographing is completed, synthesizing the images of the same surface.

[0016] The present invention also provides an intelligent detection system for chip appearance defects, which uses the above-mentioned intelligent detection method for chip appearance defects, including a feeding mechanism, a loading module, a transport and rotation module, a photographing module and a unloading module arranged on the machine platform, and a cache station for switching the chip clamping state is arranged on the machine platform; the photographing module includes a lower camera, a side camera and an upper camera fixed on the machine platform, the lower camera and the side camera are arranged at the transport and rotation module, and when the transport and rotation module drives the chip to move and rotate, the bottom and side images of the chip are obtained; the upper camera is arranged behind the transport and rotation module, and a translation module for driving the chip to move to obtain the front image is arranged below the upper camera, and the unloading module is located behind the translation module to transport and move the chip to the unloading station.

[0017] Preferably, the feeding mechanism and the translation module of the present invention are provided with a tooling plate for positioning the chip on the corresponding moving end; the transport rotation module includes a servo motor, the servo motor is connected to the moving end of the three-coordinate moving mechanism on the machine, and a clamp for sucking and fixing the chip is provided at the driving end of the servo motor.

[0018] Preferably, the number of the detection systems of the present invention is two, which are symmetrically arranged side by side on the machine platform to perform chip appearance detection simultaneously.

[0019] Beneficial effects: The chip appearance defect intelligent detection system provided by the present invention has the following beneficial effects:

[0020] 1. The present invention classifies chip appearance defects according to the similarity of image representation, and conducts targeted detection of defects in corresponding categories based on the detection strengths of different detection algorithms. It can detect more than 39 major categories of common and uncommon defects, achieving comprehensive detection of chip appearance defects.

[0021] 2. The present invention divides the recognition results into three categories, OK, NG and suspicious, according to the recognition confidence, and saves them separately, and performs manual review on the suspicious results. On the one hand, it ensures the accuracy of the detection results, while reducing the difficulty of manual review and reducing the workload. At the same time, for the defects that the system recognizes incorrectly or misses in the manual review results, they can be re-labeled and trained locally to generate a new model that can identify the corresponding defects, realize semi-automatic learning, and enable the system to be upgraded according to the corresponding production line.

[0022] 3. Aiming at the detection of small and micro defects, the present invention adds feature fusion to the detection network of high-definition large images and adopts a multi-scale feature fusion method without upsampling process. That is, features of different scales are extracted from different layers of the network for prediction. This method does not increase the additional computational workload and improves the detection rate of small and micro defects.

[0023] 4. The present invention detects defects by superimposing the defect-free image onto the sample image generated by the normal product image. For the chips of a certain production line, only sample images corresponding to the number of defects are needed to achieve a detection accuracy of 97%.

[0024] 5. The present invention sets filtering parameters during defect detection, filters NG products classified according to recognition confidence, and uses strict detection parameters to ensure that all defects are detected. At the same time, customized filtering parameters are opened on the software interface. Users can adjust the filtering parameters according to the company's quality standards to control the detection quality.

[0025] 6. For images where manual verification results are inconsistent with test results, the present invention automatically adjusts the corresponding filter parameter and issues a pop-up notification when only one of the test indicators exceeds the upper or lower limit of the filter parameter. This reduces the over-detection (over-kill) rate without missing any detections. By leveraging user spot checks and review functions, the system continuously learns and improves detection accuracy without increasing user workload.

[0026] 7. The present invention acquires chip surface images by dividing the photographing area according to the reference photographing size. For large-sized chips, the images are photographed in segments and then merged. This allows for the detection of chips of different sizes while ensuring the quality of image acquisition.

[0027] 8. The present invention uses a servo motor to drive the chip to rotate to obtain a side image, which can achieve high-precision and high-efficiency side detection. At the same time, the system can simultaneously detect two groups of chips by symmetrically arranging them side by side on the machine, thereby improving detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0029] FIG1 is a front view of the detection system of the present invention;

[0030] FIG2 is a top view of the detection system of the present invention;

[0031] Among them, 1-machine, 2-feeding mechanism, 3-loading module, 4-handling rotation module, 5-unloading module, 6-cache station, 7-lower camera, 8-side camera, 9-upper camera, 10-translation module, 11-unloading station, 12-tooling plate, and 13-servo motor. DETAILED DESCRIPTION

[0032] The present invention will be described more clearly and completely below by way of a preferred embodiment in conjunction with the accompanying drawings, but the present invention is not limited to the scope of the embodiment.

[0033] The present invention discloses an intelligent detection method for chip appearance defects. The method classifies chip appearance defects according to their types, groups defects with similar image representations into one category, and selects corresponding detection methods from traditional visual algorithm detection, AI visual algorithm detection, and traditional visual algorithm and AI visual algorithm combined detection to identify and detect corresponding defects according to the defect categories.

[0034] In an embodiment of the present invention, various chip defects are divided into three categories, and each category corresponds to a visual inspection algorithm for defect detection, as follows:

[0035] Defects detected by traditional visual algorithms: barcode printing position offset, barcode overlap - misalignment, barcode pasted upside down, barcode fuzziness, barcode offset, over-partitioning / under-partitioning of panels, and misalignment.

[0036] AI visual algorithm detects defects such as barcode bubbles, broken barcodes, barcode stains, foreign matter, handwriting, dust, bumps and concave spots, damage to the connector J2 body, lifted pins, deformed springs, dirty pins, J2 bubbles, J7 floating, water stains on the back of the chip, and inductor damage.

[0037] Traditional visual algorithms and AI vision are combined to detect defects such as BGA defects, damaged barcodes, scratches in the socket area, scratches in the circuit area, scratches in non-circuit areas, scratches in other areas, copper leakage on PADs, copper leakage, solder joints, warped and bent legs, foreign matter and dirt, socket gaps, body damage, pin deformation, pin deformation, multiple pieces on the chip surface, and chip edge damage. For copper leakage defects, the color image is converted into a binary image based on the RGB spatial characteristics of the copper color, and then labeled and trained to accurately identify and detect the defects.

[0038] Among them, the AI ​​visual inspection algorithm uses three models to identify defects, with one model used for the front, bottom and side of the chip respectively. It adopts the ResNet-50 network and transfer learning technology, uses the open source chip detection large model, and combines the independently collected and labeled sample images to train the model. It uses knowledge distillation technology to use the knowledge learned by the large model to guide the training of the small model, so that the small model has the same performance as the large model, but the number of parameters is greatly reduced, thereby achieving model compression and acceleration, and speeding up the model training.

[0039] During inspection, an image of the chip surface is first acquired. After preprocessing the image, the detection algorithm is used in sequence to perform defect detection and identify the type of defect. The process of acquiring the chip surface image includes: setting a reference photographing size. When the chip size is smaller than the reference photographing size, a single photographing method is used to acquire images of the bottom, front, and side surfaces of the chip. When the chip size is larger than the reference photographing size, the bottom, side, and front surfaces of the chip are divided into a single photographing area smaller than the reference photographing size. Each divided photographing area is photographed separately. After the photographing is completed, the images of the same surface are synthesized, thereby ensuring the detection accuracy and being compatible with chip specifications of different sizes, and realizing the detection of appearance defects of chips of different sizes.

[0040] The image preprocessing process includes: performing reference positioning and mask processing on the image, and cropping the image while performing image brightness equalization to keep the image to be detected corresponding to the sample image.

[0041] The detection algorithm divides the recognition results into three categories: OK, NG, and suspicious according to the recognition confidence of the area in the detection image corresponding to the sample image, and saves them separately. Suspicious results are randomly inspected and reviewed.

[0042] The recognition confidence of the present invention is the similarity between the detection image and the sample image. If the recognition confidence is greater than 0.85 and less than 0.95, the detection result is set as a suspicious class. For the sampling and review results of suspicious objects, after the sampling and review are completed, the results are fed back to the detection system. The system locally labels and trains, generates a new model, and performs semi-automatic learning.

[0043] The present invention uses a multi-scale feature fusion approach to detect small defects in chip appearance defects, extracting features of different scales from different layers of the image detection network for prediction. This approach does not increase the amount of additional calculations and improves the detection rate of small defects.

[0044] The present invention sets filtering parameters during defect detection. The filtering parameters are upper and lower limit range parameters related to the corresponding defect size, number, shape, and color. The filtering parameters are used to determine whether the detected defects meet the enterprise quality standards and whether the number, size, shape, and color of the defects are within the qualified range, so as to release some defects within the allowable range; the quality of the detected chip products is controlled by the NG-type images divided according to the recognition confidence through the filtering parameters, the original NG-type images are filtered and divided twice, and the images of the detected defects filtered by the filtering parameters are used as the final NG images. The values ​​of the filtering parameters can be adjusted according to the processing quality standards to avoid over-inspection while ensuring product quality, thereby reducing the workload of manual review.

[0045] The present invention manually reviews the NG images generated by filtering. When the review result is inconsistent with the system detection result, the parameters of the review image are compared with the filter parameters of the corresponding defect detection. When only one detection indicator exceeds the upper and lower limits of the filter parameters, the corresponding filter parameter range is adjusted; and after adjusting the filter parameters, the identified and classified images are used for verification to display the result differences in order to decide whether to maintain the parameter changes. The detection method of the present invention can be adjusted according to the feedback from production and manual review, and the over-inspection (over-kill) rate can be reduced under the premise of no missed inspections. The user spot check and review functions are utilized to continuously learn and improve the detection accuracy without increasing the user's workload; the results of manual spot checks or reviews can also be used to record defects that have not been entered, and through deep learning, defects that have never been encountered can be autonomously detected, and defects can be automatically analyzed through intelligent algorithms.

[0046] As shown in Figures 1 and 2, the present invention also provides an intelligent detection system for chip appearance defects, which uses the above-mentioned intelligent detection method for chip appearance defects, including a feeding mechanism 2, a loading module 3, a transport and rotation module 4, a photographing module and a blanking module 5 arranged on a machine 1, and a cache station 6 for switching the chip clamping state is arranged on the machine 1; the photographing module includes a lower camera 7, a side camera 8 and an upper camera 9 fixed on the machine 1, and the lower camera 7 and the side camera 8 are arranged at the transport and rotation module 4, and when the transport and rotation module 4 drives the chip to move and rotate, the bottom and side images of the chip are obtained; the upper camera 9 is arranged behind the transport and rotation module 4, and a translation module 10 for driving the chip to move to obtain the front image is arranged below the upper camera 9, and the blanking module 5 is located behind the translation module 10, and the chip is transported and moved to the blanking station 11.

[0047] The loading module 3 of the present invention includes a lifting motor and a linear module for driving the lifting motor to translate to remove the chip from the feeding mechanism 2 and place it at the cache station 6. The transporting and rotating module 4 clamps the chip from the corresponding cache station and moves it to the lower camera 7 and the side camera 8 to obtain images of the bottom and side surfaces of the chip. The chip is transported to the translation module 10 through the same mechanism as the loading module 3 arranged behind the transporting and rotating module, and moved to the upper camera 9 to obtain the front image of the chip. Finally, the chip is removed from the translation module 10 by the unloading module 5 and placed in the unloading station 11, and removed from the detection system. The detection accuracy is above 0.05mm, the detection speed is within 3s, and the accuracy rate is as high as above 99.7%.

[0048] The feeding mechanism 2 and the translation module 10 of the present invention are both provided with a tooling plate 12 for positioning the chip on the corresponding moving end; the transport rotation module 4 includes a servo motor 13, and the servo motor 13 is connected to the moving end of the three-coordinate moving mechanism on the machine 1. A clamp for sucking and fixing the chip is provided at the driving end of the servo motor 13. In order to improve the detection efficiency, the number of the detection system of the present invention is two groups, which are symmetrically arranged side by side on the machine 1 to perform chip appearance detection simultaneously.

[0049] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A chip appearance defect intelligent detection method, characterized in that: Classify defects based on chip appearance, group defects with similar image representations into one category, and select corresponding detection methods for defect identification and detection in traditional visual algorithm detection, AI visual algorithm detection, and traditional visual algorithm and AI visual algorithm combined detection according to the defect category; During the inspection, the image of the chip surface is first obtained, and after preprocessing the image, the detection algorithm is used in sequence to perform defect detection and identify the type of defect; According to the recognition confidence of the area in the detection image corresponding to the sample image, the recognition results are divided into three categories: OK, NG and suspicious, and saved separately. Suspicious results are sampled and reviewed.

2. The method for intelligently detecting chip appearance defects according to claim 1, characterized in that: The image preprocessing process includes: performing reference positioning and mask processing on the image, and cropping the image while performing image brightness equalization to keep the image to be detected corresponding to the sample image.

3. The method for intelligently detecting chip appearance defects according to claim 1, characterized in that: The recognition confidence is the similarity between the detection image and the sample image. If the recognition confidence is greater than 0.85 and less than 0.95, the detection result is set as a suspicious class. After the sampling and review results of suspicious objects are completed, the results are fed back to the detection system. The system locally labels and trains, generates a new model, and performs semi-automatic learning.

4. The method for intelligently detecting chip appearance defects according to claim 1, characterized in that: For small defects in chip appearance, multi-scale feature fusion is used for detection, and features of different scales are extracted from different layers of the image detection network for prediction.

5. The method for intelligently detecting chip appearance defects according to claim 1, characterized in that: Filter parameters are set during defect detection. The filter parameters are upper and lower limit range parameters related to the corresponding defect size, number, shape, and color. The quality of the detected chip products is controlled by using the filter parameters to classify the NG images according to the recognition confidence level. The original NG images are filtered and divided twice, and the images with detected defects and filtered by the filter parameters are used as the final NG images.

6. The method for intelligently detecting chip appearance defects according to claim 5, characterized in that: The NG images generated by filtering are reviewed. When the review results are inconsistent with the algorithm detection results, the parameters of the reviewed images are compared with the filtering parameters of the corresponding defect detection. When only one detection indicator exceeds the upper and lower limits of the filtering parameters, the corresponding filtering parameter range is adjusted; and after adjusting the filtering parameters, the identified and classified images are used for verification.

7. The method for intelligently detecting chip appearance defects according to claim 1, characterized in that: The process of obtaining the chip surface image includes: setting a reference photographing size, when the chip size is smaller than the reference photographing size, using a single photographing method to obtain images of the bottom, front and side surfaces of the chip, when the chip size is larger than the reference photographing size, dividing the bottom, side and front surfaces of the chip into single photographing areas smaller than the reference photographing size, photographing the divided photographing areas separately, and after the photographing is completed, synthesizing the images of the same surface.

8. A chip appearance defect intelligent detection system, using a chip appearance defect intelligent detection method according to any one of claims 1 to 7, characterized in that: The invention comprises a feeding mechanism (2), a loading module (3), a transporting and rotating module (4), a photographing module and a material unloading module (5) arranged on a machine platform (1); a buffer station (6) for switching the chip clamping state is arranged on the machine platform (1); the photographing module comprises a lower camera (7), a side camera (8) and an upper camera (9) fixed on the machine platform (1); the lower camera (7) and the side camera (8) are arranged at the transporting and rotating module (4); when the transporting and rotating module (4) drives the chip to move and rotate, the bottom and side images of the chip are obtained; the upper camera (9) is arranged behind the transporting and rotating module (4); a translation module (10) for driving the chip to move and obtain the front image is arranged below the upper camera (9); the material unloading module (5) is located behind the translation module (10) and moves the chip to the material unloading station (11).

9. The chip appearance defect intelligent detection system according to claim 8, characterized in that: The feeding mechanism (2) and the translation module (10) are both provided with a tooling plate (12) for positioning the chip on the corresponding moving ends; the transport rotation module (4) includes a servo motor (13), the servo motor (13) is connected to the moving end of the three-coordinate moving mechanism on the machine platform (1), and a clamp for sucking and fixing the chip is provided at the driving end of the servo motor (13).

10. The chip appearance defect intelligent detection system according to claim 8, characterized in that: The number of the detection systems is two, which are symmetrically arranged side by side on the machine platform (1) to perform chip appearance detection simultaneously.

Images