Plastic package chip appearance defect detection method and system, medium and electronic equipment

Abstract

The invention provides a plastic package chip appearance defect detection method and system, a medium and electronic equipment. The method comprises the following steps: acquiring chip image packets of each chip in different light fields; for each chip image packet, segmenting each chip in the chip image packet according to a preset chip contour template to obtain a plurality of single chip images; combining the plurality of single-chip images and the chip appearance defect anomaly detection model to obtain a plurality of mask contours for representing anomaly features; for each mask contour, generating a plurality of defect mask images according to a plurality of preset chip appearance regions of interest; determining an abnormal feature type corresponding to each defect mask image; and for each chip, combining the abnormal features of the same type corresponding to each chip in different light fields and in the same chip appearance region of interest so as to determine the type and the number of all the abnormal features corresponding to each chip. The problems of low manual detection efficiency and poor detection effect in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method, system, medium, and electronic device for detecting appearance defects of plastic encapsulated chips. Background Art

[0002] QFN (Quad Flat No-leads Package) chips are one of the currently widely used surface mount packages.

[0003] In the production of QFN packaged chips, common appearance defects include printing errors, surface scratches, and burrs on electrodes. Currently, it mainly relies on manual inspection of the product appearance through microscopic tools, which not only has low efficiency, but is also greatly affected by manual experience and senses. Visual fatigue is more likely to occur during long-term work, and the detection effect is not good. Summary of the Invention

[0004] Aiming at the deficiencies in the prior art, the present invention provides a method, system, medium, and electronic device for detecting appearance defects of plastic encapsulated chips, which solves the problems of low efficiency and poor detection effect in manual detection in the prior art.

[0005] At least one embodiment of the present invention provides a method for detecting appearance defects of plastic encapsulated chips, including:

[0006] Obtaining chip image packets of each chip arranged in the same area and under different light fields;

[0007] For each of the chip image packets, segment each chip in the chip image packet according to a preset chip contour template to obtain a plurality of single-chip images;

[0008] Combining a plurality of the single-chip images and a chip appearance defect anomaly detection model to obtain a plurality of mask contours representing anomaly features;

[0009] For each of the mask contours, generate a plurality of defect mask images according to a plurality of preset chip appearance regions of interest;

[0010] Determine the types of anomaly features corresponding to each of the defect mask images;

[0011] For each chip, combine the same types of anomaly features corresponding to it under different light fields and the same chip appearance regions of interest to determine the types and quantities of all anomaly features corresponding to each chip.

[0012] The technical solution publicly provided by the present invention has at least the following beneficial effects:

[0013] Through the segmentation of the chip image package and multiple preset regions of interest in the chip appearance, multiple defect mask images are generated, which can detect defects at specific positions in the chip image package, avoid unnecessary detection processing for regions that do not need to be detected, and improve the detection efficiency. At the same time, by acquiring images of each chip under multiple light fields, the optical characteristics of the product surface and various defects are effectively utilized, avoiding the situation where the detection effect of some defects is poor or cannot be detected under a single light field. Finally, by merging the same type of abnormal features in the regions of interest in the appearance of the same chip under different light fields, the types and quantities of all abnormal features corresponding to each chip are quickly determined, realizing precise positioning of the surface flaw parts and accurate classification, saving the time for chip detection.

[0014] In a method for detecting appearance defects of plastic encapsulated chips provided in one embodiment of the present invention, for each of the chip image packages, according to a preset chip contour template, each chip in the chip image package is segmented to obtain multiple single-chip images, including:

[0015] For any one of the chip image packages, according to the preset chip contour template, the segmentation positioning points corresponding to each chip in the chip image package are determined;

[0016] According to the segmentation positioning points, all the chip image packages are segmented to obtain multiple single-chip images.

[0017] The technical solutions publicly provided by the present invention have at least the following beneficial effects:

[0018] Through the above solution, for one chip image package, it is processed using a preset chip contour template, and subsequent chip image packages only need to be segmented according to the segmentation positioning points of the above processing, greatly simplifying the segmentation and acquisition steps of single-chip images under multiple light fields and improving the overall detection efficiency.

[0019] In a method for detecting appearance defects of plastic encapsulated chips provided in one embodiment of the present invention, the steps for constructing a chip appearance defect anomaly detection model include:

[0020] Using the PatchCore algorithm, a positive sample feature library is constructed, and in combination with the positive sample feature library and an industrial anomaly detection model, a chip appearance defect anomaly detection model is constructed.

[0021] The technical solutions publicly provided by the present invention have at least the following beneficial effects:

[0022] Since common supervised models need to know existing defects in advance and collect defects to construct defect features, while the anomaly detection of this chip appearance defect classification model only needs to collect normal product images, and the detection calculates features that are inconsistent with the normal product images from the positive sample library, it is more convenient to use.

[0023] In a method for detecting appearance defects of a plastic-encapsulated chip provided in one embodiment of the present invention, determining the types of abnormal features corresponding to each of the defect mask images includes:

[0024] Combining a classification model based on ResNet and a channel attention mechanism to construct a chip appearance defect classification model, and classifying each of the defect mask images according to the types of abnormal features through the chip appearance defect classification model to determine the types of abnormal features corresponding to each of the defect mask images.

[0025] The technical solution publicly provided by the present invention has at least the following beneficial effects:

[0026] By constructing a chip appearance defect classification model, the defect mask images can be quickly classified.

[0027] In a method for detecting appearance defects of a plastic-encapsulated chip provided in one embodiment of the present invention, determining the types of abnormal features corresponding to each of the defect mask images includes:

[0028] Performing Blob analysis on each of the defect mask images to pre-classify each of the defect mask images using preset features and obtain a plurality of pre-classified images;

[0029] Combining a classification model based on ResNet and a channel attention mechanism to construct a chip appearance defect classification model, and performing secondary classification on each of the pre-classified images according to the types of abnormal features through the chip appearance defect classification model to determine the types of abnormal features corresponding to each of the defect mask images.

[0030] The technical solution publicly provided by the present invention has at least the following beneficial effects:

[0031] By performing pre-classification and secondary classification, the classification accuracy of the chip appearance defect classification model can be improved.

[0032] In a method for detecting appearance defects of a plastic-encapsulated chip provided in one embodiment of the present invention, it further includes:

[0033] Assigning different first weight values to different chip appearance regions of interest and different second weight values to different abnormal features;

[0034] For each chip, sorting the chip according to the first weight values and the second weight values corresponding to all the abnormal features on the chip.

[0035] The technical solution publicly provided by the present invention has at least the following beneficial effects:

[0036] By sorting according to the first weight value and the second weight value of the chip abnormal features, the problems commonly existing in this batch of chips can be clearly known, and then accurately fed back to the previous production process for improvement.

[0037] At least one embodiment of the present invention further provides a plastic-encapsulated chip appearance defect detection system, including:

[0038] An image acquisition module, configured to obtain chip image packets of each chip arranged in the same area and under different light fields controlled by the light source control module;

[0039] An algorithm processing module, the algorithm processing module includes a separation and positioning unit, a defect detection unit, a defect analysis unit and a classification unit. Among them, the separation and positioning unit is used for each of the chip image packets, according to a preset chip contour template, to segment each chip in the chip image packet to obtain a plurality of single-chip images;

[0040] The defect detection unit is used to combine a plurality of the single-chip images and a chip appearance defect abnormal detection model to obtain a plurality of mask contours used to represent abnormal features; for each of the mask contours, according to a plurality of preset chip appearance regions of interest, generate a plurality of defect mask images;

[0041] The defect analysis unit is used to determine the types of abnormal features corresponding to each of the defect mask images;

[0042] The classification unit is used for each chip, to merge the same types of abnormal features corresponding to it under different light fields and the same chip appearance region of interest, so as to determine the types and quantities of all abnormal features corresponding to each chip.

[0043] In a plastic-encapsulated chip appearance defect detection system provided by one embodiment of the present invention, the separation and positioning unit further includes:

[0044] For any one of the chip image packets, according to a preset chip contour template, determine the segmentation and positioning points corresponding to each chip in the chip image packet;

[0045] According to each of the segmentation and positioning points, segment all the chip image packets to obtain a plurality of single-chip images.

[0046] The present invention also provides a computer-readable storage medium, in which instructions are stored. When the instructions run on a terminal device, the terminal device is enabled to execute a plastic-encapsulated chip appearance defect detection method as described above.

[0047] The present invention also provides an electronic device, including a memory, a processor, and a program stored on the memory and running on the processor. When the processor executes the program, it implements a method for detecting appearance defects of plastic-encapsulated chips as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a schematic flowchart of a method for detecting appearance defects of plastic-encapsulated chips according to the present invention;

[0049] Figure 2 is a logic diagram of a system for detecting appearance defects of plastic-encapsulated chips according to the present invention;

[0050] Figure 3 is a schematic structural diagram of the electronic device provided by the present invention.

[0051] In the drawings, the list of components represented by each reference numeral is as follows:

[0052] 10. Electronic device, 11. Processor, 12. Read-only memory (ROM), 13. Random access memory (RAM), 14. Bus, 15. Input / output (I / O) interface, 16. Input unit, 17. Output unit, 18. Storage unit, 19. Communication unit. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0054] The present invention provides a method for detecting appearance defects of plastic-encapsulated chips. Please refer here Figure 1 as shown, including:

[0055] Obtain chip image packets of each chip arranged in the same area and under different light fields;

[0056] For each of the chip image packets, segment each chip in the chip image packet according to a preset chip contour template to obtain a plurality of single-chip images;

[0057] Combine the plurality of single-chip images and a chip appearance defect anomaly detection model to obtain a plurality of mask contours representing anomaly features;

[0058] For each of the mask contours, generate a plurality of defect mask images according to a plurality of preset chip appearance regions of interest;

[0059] Determine the types of anomaly features corresponding to each of the defect mask images;

[0060] For each chip, under different optical fields and for the same region of interest in the chip appearance, the same types of abnormal features are merged to determine the types and quantities of all abnormal features corresponding to each chip.

[0061] Through the segmentation of the chip image package and a plurality of preset regions of interest in the chip appearance, the present invention generates a plurality of defect mask images, which can perform defect detection on specific positions of the chip image package, avoid unnecessary detection processing on regions that do not need to be detected, improve the detection efficiency. At the same time, by acquiring images of each chip under multiple optical fields, the optical characteristics of the product surface and various defects are effectively utilized, avoiding the situation where the detection effect of some defects is poor or cannot be detected under a single optical field. Finally, by merging the same types of abnormal features in the same region of interest in the chip appearance under different optical fields, the types and quantities of all abnormal features corresponding to each chip are quickly determined, realizing precise positioning of the surface defect parts and accurate classification, saving the time for chip detection.

[0062] Further, the specific steps of the present invention include:

[0063] 1. Obtain chip image packages of each chip arranged in the same region and under different optical fields;

[0064] Specifically, each chip is arranged in a two-dimensional array form, and each chip image package contains all the chips arranged in the above two-dimensional array form. By photographing all the above chips under different optical fields, multiple chip image packages under multiple optical fields are obtained as the initial data for algorithm processing;

[0065] 2. For each of the chip image packages, according to a preset chip contour template, each chip in the chip image package is segmented to obtain a plurality of single-chip images; in this embodiment, the above segmentation step is specifically: for any one of the chip image packages, according to the preset chip contour template, the segmentation positioning points corresponding to each chip in the chip image package are determined;

[0066] According to each of the segmentation positioning points, all the chip image packages are segmented to obtain a plurality of single-chip images. By the above scheme, each chip product under different optical fields can be segmented, reducing repeated positioning calculations, and thus improving the overall detection efficiency.

[0067] 3. Construct a region of interest in the appearance of the packaged chip, and assign different first weight values to the region of interest in the chip appearance according to the degree of influence on the chip quality;

[0068] 4. Combine a plurality of the single-chip images and a chip appearance defect abnormal detection model to obtain a plurality of mask contours representing abnormal features;

[0069] The steps for constructing the chip appearance defect anomaly detection model include:

[0070] Using the PatchCore algorithm, construct a positive sample feature library, and combine the positive sample feature library and the Anomalib industrial anomaly detection model to construct a chip appearance defect anomaly detection model.

[0071] Train the above chip appearance defect anomaly detection model with normal product chip images, and then send multiple single chip images into the trained chip appearance defect anomaly detection model to obtain multiple mask contours.

[0072] Since common supervised models need to know existing defects in advance, collect defects to construct defect features, while the anomaly detection of this chip appearance defect classification model only needs to collect normal product images, and the detection calculates features inconsistent with normal product images with the positive sample library, which is more convenient to use.

[0073] 5. For each of the mask contours, generate multiple defect mask images according to a preset plurality of chip appearance regions of interest to isolate invalid regions in the image;

[0074] 6. Determine the types of abnormal features corresponding to each of the defect mask images;

[0075] Specifically, in one embodiment of the present invention, determining the types of abnormal features corresponding to each of the defect mask images may be: combining a classification model based on ResNet and a channel attention mechanism to construct a chip appearance defect classification model, and through the chip appearance defect classification model, classifying each of the defect mask images according to the types of abnormal features to determine the types of abnormal features corresponding to each of the defect mask images.

[0076] To improve the classification accuracy, the present invention also provides another embodiment to determine the types of abnormal features corresponding to each of the defect mask images, which specifically includes:

[0077] Perform Blob analysis on each of the defect mask images to pre-classify each of the defect mask images using preset features (such as geometric features and gray-scale features) to obtain multiple pre-classified images;

[0078] Combine a classification model based on ResNet and a channel attention mechanism, assign weights designed by the ResNet network to different feature channels to highlight important information and suppress unimportant information, so as to construct a chip appearance defect classification model, and through the chip appearance defect classification model, perform secondary classification on each of the pre-classified images according to the types of abnormal features to determine the types of abnormal features corresponding to each of the defect mask images.

[0079] 7. For each chip, merge the same types of abnormal features corresponding to it under different optical fields and in the region of interest of the same chip appearance, that is, exclude multiple detection records of the same defect under different optical fields, and then determine the types and quantities of all abnormal features corresponding to each chip.

[0080] 8. Assign different second weight values to different abnormal features;

[0081] For each chip, according to the first weight value and the second weight value corresponding to all the abnormal features on the chip, and according to the cumulative size of the first weight value and the second weight value, sort the chips to divide the chips into regions with different fault severity levels. By sorting according to the first weight value and the second weight value of the chip abnormal features, the problems commonly existing in this batch of chips can be clearly known, and then accurately fed back to the previous production process for improvement.

[0082] 9. When the number of chips in the preset fault severity level region reaches the preset number, or the number of times of preset types of fault defects reaches the preset value, an alarm can be triggered, and then control the device to prompt or stop for manual inspection and confirmation;

[0083] 10. In different fault severity level regions, chips with the same abnormal features can also be placed in the same Tray according to the types of abnormal features that appear on the chips, for the convenience of subsequent manual inspection.

[0084] In summary, through the plastic-encapsulated chip appearance defect detection method provided by the present invention, the need for global detection of chips can be met. Through the combined layout of multiple light sources and optical fields, the optical characteristics of the product surface and various defects are effectively utilized, and the surface defect parts can be accurately located and accurately classified, saving the time for chip detection. By constructing an abnormal detection model for chip appearance defects, it is more convenient to obtain the mask contour representing abnormal features. At the same time, the secondary classification process also improves the classification accuracy of the chip appearance defect classification model used in the present invention. Finally, through the comprehensive statistics of the detection results, users can obtain multi-faceted chip defect information according to the statistical results, and the system can automatically sort the chips with different defects into the Tray according to the strategy after detection, improving the sorting speed of defective products and normal products.

[0085] The present invention also provides a plastic-encapsulated chip appearance defect detection system. Please refer to Figure 2 as shown, including:

[0086] An image acquisition module for acquiring chip image packets of each chip arranged in the same region and under different optical fields controlled by the light source control module;

[0087] Algorithm processing module, the algorithm processing module includes a separation and positioning unit, a defect detection unit, a defect analysis unit and a classification unit. Among them, the separation and positioning unit is used for each of the chip image packets, according to a preset chip contour template, to segment each chip in the chip image packet to obtain a plurality of single-chip images;

[0088] The defect detection unit is used to combine a plurality of the single-chip images and a chip appearance defect anomaly detection model to obtain a plurality of mask contours representing abnormal features; for each of the mask contours, according to a preset plurality of chip appearance regions of interest, generate a plurality of defect mask images;

[0089] The defect analysis unit is used to determine the types of abnormal features corresponding to each of the defect mask images;

[0090] The classification unit is used for each chip, to merge the same types of abnormal features corresponding to it under different light fields and the same chip appearance regions of interest, so as to determine the types and quantities of all abnormal features corresponding to each chip.

[0091] Further, please combine here Figure 2 As shown, the above-mentioned modules achieve information interaction through a host computer. The host computer module is mainly responsible for UI interaction operations, scheduling other modules and displaying;

[0092] The user selects the established recipe on the interface popped up by the host computer. The host computer, according to whether the system operation mode in the recipe is an offline simulation mode or an online real-time mode, if the set is the offline mode, the system will ignore the initialization operations of the image acquisition module, the light source control module and the sorting module; if the set is the online real-time mode, it will normally initialize all modules, and transfer the light source control parameters, camera parameters and defect detection parameters set in the recipe to the corresponding modules;

[0093] The specific control process of the above-mentioned light source control module is: according to the light source lighting scheme set in the recipe, control the light source controller to light different light sources in turn, and synchronously trigger the camera image acquisition mechanism in the image acquisition module;

[0094] The above-mentioned image acquisition module is specifically used for, according to whether the camera in the recipe is in an offline simulation mode or an online real-time mode; when the selected mode is the online real-time mode, after the camera in the image acquisition module receives the trigger signal from the light source controller, capture the appearance image of the packaged chip; when the selected mode is the offline simulation mode, read the local image named according to the light source lighting method in the recipe according to the pop-up window; the image acquisition module sends the image data to the host computer, and it packs the chip image packets under different light fields and conveys them to the algorithm processing module;

[0095] The segmentation and positioning unit in the above algorithm processing module further includes:

[0096] For any one of the chip image packets, according to a preset chip contour template, determine the segmentation and positioning points corresponding to each chip in the chip image packet;

[0097] According to the respective segmentation and positioning points, segment all the chip image packets to obtain a plurality of single-chip images.

[0098] Furthermore, the steps for constructing the chip appearance defect anomaly detection model in the defect detection unit include:

[0099] Utilize the PatchCore algorithm to construct a positive sample feature library, and combine the positive sample feature library and an industrial anomaly detection model to construct a chip appearance defect anomaly detection model.

[0100] Furthermore, the defect analysis unit specifically includes:

[0101] Combine a classification model based on ResNet and a channel attention mechanism to construct a chip appearance defect classification model. Through the chip appearance defect classification model, classify each of the defect mask images according to the types of abnormal features to determine the types of abnormal features corresponding to each of the defect mask images.

[0102] Furthermore, the defect analysis unit may specifically further include:

[0103] Perform Blob analysis on each of the defect mask images to pre-classify each of the defect mask images using preset features to obtain a plurality of pre-classified images;

[0104] Combine a classification model based on ResNet and a channel attention mechanism to construct a chip appearance defect classification model. Through the chip appearance defect classification model, perform secondary classification on each of the pre-classified images according to the types of abnormal features to determine the types of abnormal features corresponding to each of the defect mask images.

[0105] Furthermore, this system further includes:

[0106] A sorting module that assigns different first weight values to different chip appearance regions of interest and different second weight values to different abnormal features;

[0107] For each chip, sort the chip according to the first weight values and second weight values corresponding to all the abnormal features on the chip, so that after detection, chips with different defects can be automatically sorted into a Tray according to a strategy

[0108] Furthermore, this system further includes:

[0109] An early warning prompt module, which is used to trigger an alarm when the number of chips in the preset fault severity level area reaches the preset number, or when the number of preset types of fault defects reaches the preset value, and then control the device to prompt or stop for manual inspection and confirmation;

[0110] Furthermore, the present system further includes:

[0111] A defect statistics module, which is used to, after receiving the defect information of each product chip, assign a sorted defect level to each defective product according to the defect priority strategy, and summarize and record the defect information of each product for subsequent export of production inspection reports;

[0112] In summary, the specific working steps of the present system include:

[0113] Select the system operation mode on the upper computer interface for encapsulation chip detection.

[0114] Specifically, in the embodiment of the present invention, the system operation mode includes an offline simulation mode and an online real-time operation mode. Among them, when the offline simulation mode is selected, the operator can select to read the images named according to the light source lighting method stored locally in the system and send them to the algorithm module for detection; when the selected mode is the online real-time mode, the upper computer verifies and schedules the light source control module and the image acquisition module according to the logic to obtain the chip image packet;

[0115] Specifically, when the selected system operation mode is the offline simulation mode, the upper computer loads the selected folder storing the chip image packet, and the operator can preview the chip image packet to be detected on the chip detection interface to confirm whether the detected image matches the light source lighting method in the selected recipe;

[0116] Specifically, the offline simulation mode of the system is applicable to the situation of testing and re-inspecting products that have been collected. The operator can send the original chip image packet of the product saved by the system to the algorithm processing module for detection to check and adjust the detection effect of the detection parameters.

[0117] Specifically, when the selected system operation mode is the real-time detection mode, the upper computer verifies the light source control module, the image acquisition module and the sorting module. When normal communication recognition is possible, no operation is required; when an abnormality is recognized, the upper computer obtains the status information of the abnormal module and obtains the cause of the abnormality based on the status information;

[0118] After the upper computer collects the images transmitted by the image acquisition module, it packs the image data in sequence according to the order of light source lighting in the recipe. After being transmitted by the upper computer to the algorithm processing module, the collected image data is denoised and the appearance defects are detected.

[0119] Specifically, in the matching and separating positioning unit of the algorithm processing module, in order to improve the detection efficiency and ensure the detection accuracy of the system, one or more products arranged in a two-dimensional array are included in the field of view, so it is necessary to separate different products. Through template matching, the positions and angle information of all products in a field of view are obtained, and according to the actual image size of the products, each is enlarged by 5% in both the length and width directions, and each light field area of the products is cropped synchronously;

[0120] Specifically, if template path information is passed into the algorithm processing module, the image saved in the path is read as the matching and positioning template image; if no template path information is passed into the algorithm processing module, a template creation window will pop up, and a set of currently captured images can be previewed. A suitable light field image is selected, and an area of interest in the appearance of a product chip is drawn through the window UI interaction, and the image corresponding to the area of interest in the chip appearance is cropped and saved in the system as the template image;

[0121] Further, in the algorithm processing module, a hierarchical region of interest in the chip appearance is constructed. In the encapsulation of chip appearance defect detection, different levels of ROIs, such as Level1, LeveL2,..., Leveln and ROI regions not to be detected, can be constructed according to the inconsistent requirements of customers for different regions;

[0122] Further, in the chip appearance defect anomaly detection model of the algorithm processing module, a detection data set is constructed by pre-screening normal product images, and a chip appearance defect anomaly detection model is trained. When the algorithm processing module receives a product image data packet, each product is positioned and segmented, and the segmented product image is subjected to blur denoising processing and then sent into an industrial anomaly detection network for detection, and a mask wheel with abnormal features detected is output;

[0123] Further, in the Blob analysis and classification of the algorithm processing module, a classification data set is constructed by the images of each type of defect collected in advance, and the similar category images are grouped into one classification data set, and multiple classification models are actually divided according to this. The geometric size, gray level and other information are obtained by calculating the defect mask image output by the anomaly detection to obtain each abnormal area, and the image of each abnormal area is segmented. Blob analysis is performed on these abnormal area images to pre-classify the defect areas; the pre-classified images are sent into a classification model based on the ResNet network for accurate secondary classification;

[0124] Furthermore, since the contrast of defects is inconsistent under different light fields, defect detection is performed for each light field, which may result in defects detected under different light fields being in the same position on the product. It is necessary to merge the defect regions under different light fields, construct a defect mask image through the defects in each light field, and finally perform an AND-OR operation on the corresponding position pixels of all defect mask images to count the number of defects in each product and assign a sorting grade to the product according to the set defect level;

[0125] Specifically, when there are no defects after the product detection passes through the anomaly detection and classification, the host computer controls the sorting module to sort the product into the defect-free Tray; when there are defects, the product will be sorted to the corresponding position in the defect Tray according to whether the sorting strategy is multi-level sorting.

[0126] After all detections are completed, the defect and statistics module will count the detection results of all products according to the recorded detection results of each product and generate a production report.

[0127] Specifically, the defect and statistics module counts the number of all products detected, the number of all defect-free products, and the number of each type of defect in all products according to the recorded detection results of each product, forms a statistical table, and obtains the defect rate corresponding to all chips based on the statistical table. Among them, the formula for the yield rate corresponding to all chip products is: p = m / M, where p is the generated detection yield rate, M is the number of all products detected, and m is the number of defect-free chips among all products. In addition, the proportion of each type of defect of all chips is obtained based on the statistical table to generate a detection report for this batch of detected products.

[0128] The present invention also provides a computer-readable storage medium, in which instructions are stored. When the instructions run on a terminal device, the terminal device is enabled to execute a method for detecting the appearance defects of a plastic-encapsulated chip as described above.

[0129] The present invention also provides an electronic device, including a memory, a processor, and a program stored on the memory and running on the processor. When the processor executes the program, it implements a method for detecting the appearance defects of a plastic-encapsulated chip as described above.

[0130] Figure 3The structural schematic diagram of an electronic device 10 that can be used to implement the embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0131] As Figure 3 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0132] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0133] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for detecting appearance defects of encapsulated chips.

[0134] In some embodiments, a method for detecting appearance defects of a plastic encapsulated chip can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for detecting appearance defects of a plastic encapsulated chip described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute a method for detecting appearance defects of a plastic encapsulated chip by any other suitable means (e.g., by means of firmware).

[0135] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] In order to provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or an LCD (liquid crystal display)); and a keyboard and a pointing device (e.g., a mouse or a trackball), by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein can be implemented in a computing system including a back-end component (e.g., as a data server), or a computing system including a middleware component (e.g., an application server), or a computing system including a front-end component (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of the communication network include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0140] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0141] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0142] In the present invention, unless otherwise clearly specified and defined, terms such as "installed", "connected", "coupled", "fixed", etc. shall be construed in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two components or the interaction relationship between two components, unless otherwise clearly defined. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0143] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0144] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting appearance defects of plastic-encapsulated chips, characterized in that: include: Obtain chip image packages in which the chips are arranged in the same area and under different light fields; For each chip image package, segmenting each chip in the chip image package according to a preset chip outline template to obtain a plurality of single chip images; Combining a plurality of the single chip images and a chip appearance defect anomaly detection model to obtain a plurality of mask contours for representing abnormal features; For each of the mask contours, generating a plurality of defect mask images according to a plurality of preset chip appearance interest regions; Determining the type of abnormal feature corresponding to each of the defect mask images; For each chip, the abnormal features of the same type corresponding to the same chip appearance region of interest in different light fields are merged to determine the types and quantities of all abnormal features corresponding to each chip.

2. A method for detecting appearance defects of plastic-encapsulated chips according to claim 1, characterized in that: For each chip image package, segmenting each chip in the chip image package according to a preset chip outline template to obtain multiple single chip images, including: For any of the chip image packages, determining the segmentation positioning points corresponding to each chip in the chip image package according to a preset chip outline template; All the chip image packages are segmented according to the segmentation positioning points to obtain a plurality of single chip images.

3. The method for detecting appearance defects of plastic-encapsulated chips according to claim 1, characterized in that: The steps to build the chip appearance defect anomaly detection model include: The PatchCore algorithm is used to build a positive sample feature library, and a chip appearance defect anomaly detection model is built by combining the positive sample feature library with the industrial anomaly detection model.

4. The method for detecting appearance defects of plastic-encapsulated chips according to claim 1, characterized in that: Determining the type of abnormal feature corresponding to each of the defect mask images includes: Combining the ResNet-based classification model and the channel attention mechanism, a chip appearance defect classification model is constructed. Through the chip appearance defect classification model, each defect mask image is classified according to the type of abnormal feature to determine the type of abnormal feature corresponding to each defect mask image.

5. The method for detecting appearance defects of plastic-encapsulated chips according to claim 1, characterized in that: Determining the type of abnormal feature corresponding to each of the defect mask images includes: Performing Blob analysis on each of the defect mask images to pre-classify each of the defect mask images using preset features to obtain a plurality of pre-classified images; Combining the ResNet-based classification model and the channel attention mechanism, a chip appearance defect classification model is constructed. Through the chip appearance defect classification model, each of the pre-classified images is secondary classified according to the type of abnormal feature to determine the type of abnormal feature corresponding to each of the defect mask images.

6. The method for detecting appearance defects of plastic-encapsulated chips according to claim 1, characterized in that: Also includes: Assigning different first weight values ​​to different chip appearance regions of interest, and assigning different second weight values ​​to different abnormal features; For each chip, the chip is sorted according to the first weight value and the second weight value corresponding to all abnormal features on the chip.

7. A plastic-encapsulated chip appearance defect detection system, characterized in that: include: An image acquisition module is used to obtain chip image packages in which chips are arranged in the same area and under different light fields controlled by the light source control module; an algorithm processing module, the algorithm processing module comprising a separation and positioning unit, a defect detection unit, a defect analysis unit and a classification unit, wherein the separation and positioning unit is used for segmenting each chip in the chip image package according to a preset chip outline template to obtain a plurality of single chip images for each chip image package; The defect detection unit is used to combine the plurality of single chip images and the chip appearance defect anomaly detection model to obtain a plurality of mask contours used to represent abnormal features; for each of the mask contours, a plurality of defect mask images are generated according to the preset plurality of chip appearance regions of interest; The defect analysis unit is used to determine the type of abnormal feature corresponding to each of the defect mask images; The classification unit is used to merge the abnormal features of the same type corresponding to each chip under different light fields and the same chip appearance interest area, so as to determine the types and quantities of all abnormal features corresponding to each chip.

8. The plastic-encapsulated chip appearance defect detection system according to claim 7, characterized in that: The separation and positioning unit also includes: For any of the chip image packages, determining the segmentation positioning points corresponding to each chip in the chip image package according to a preset chip outline template; All the chip image packages are segmented according to the segmentation positioning points to obtain a plurality of single chip images.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes a method for detecting appearance defects of a plastic-encapsulated chip as claimed in any one of claims 1 to 6.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and running on the processor, characterized in that: When the processor executes the program, a method for detecting appearance defects of a plastic-encapsulated chip as described in any one of claims 1 to 6 is implemented.

Images