Chip detection system and chip detection method

Abstract

The invention relates to the technical field of chip detection, and discloses a chip detection system and a chip detection method. The multi-modal data acquisition module is used for acquiring optical visual data, thermal performance data and electrical performance data of the chip; the fusion feature modeling module is used for carrying out preprocessing, feature extraction and feature fusion modeling on the multi-modal data; the intelligent detection module is used for carrying out defect detection and positioning on the fusion features; the anomaly classification and marking module is used for carrying out anomaly classification and marking based on a defect detection result; and a feedback optimization dynamic adjustment module. A fusion feature modeling module is adopted, a deep learning network is utilized to perform feature extraction on optical, thermal and electrical performance data, feature weight optimization is realized through a cross-modal attention mechanism, a multi-modal feature fusion technology remarkably enhances the expression ability of a system to a complex defect mode, the detection precision and reliability are improved, and the detection precision and reliability are improved. And particularly, the problems of micro cracks, poor heat dissipation, unstable signals and the like can be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of chip detection, and in particular to a chip detection system and a chip detection method. Background Art

[0002] With the rapid development of electronic technology, chips are increasingly used in modern industrial and consumer electronics fields, and their quality reliability directly affects the performance and life of terminal products. In the chip production process, testing is an important link to ensure chip quality.

[0003] Traditional chip inspection methods are usually based on a single mode, and optical inspection is used to identify chip surface defects. This inspection method is often difficult to fully cover the multi-dimensional defects that may exist in the chip, especially when facing multi-modal defect associations or complex defect patterns. It shows obvious shortcomings. Due to the single data mode and limited feature mining capabilities, the existing inspection methods have accuracy limitations when identifying complex problems such as tiny cracks, poor heat dissipation, and electrical performance fluctuations. The existing chip inspection system usually uses fixed acquisition parameters and inspection strategies during operation, and cannot be dynamically adjusted according to the inspection results. Parameter optimization during the inspection process relies on manual operation, which is inefficient and difficult to adapt to complex and changing inspection environments. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a chip detection system and a chip detection method, which solve the problems of single chip detection modality, insufficient detection accuracy, lack of data fusion analysis capability and dynamic optimization adaptability.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a chip detection system, comprising: Multimodal data acquisition module, used to collect optical visual data, thermal performance data and electrical performance data of the chip; Melting feature modeling module, used for preprocessing, feature extraction and feature fusion modeling of multimodal data; Intelligent detection module, used for defect detection and location based on fused features; Anomaly classification and marking module, used to classify and mark anomalies based on defect detection results; Feedback optimization dynamic adjustment module, used to optimize the detection process according to the classification results and dynamically adjust the parameters of the data acquisition module; The data storage module is used to store the detection data.

[0006] Preferably, the multimodal data acquisition module includes: An optical vision acquisition unit, used to acquire global and local image information of the chip surface; Infrared thermal imaging acquisition unit, used to collect dynamic thermal distribution of the chip during operation; The electrical performance signal acquisition unit is used to collect the dynamic current, voltage and frequency signals of the chip.

[0007] Preferably, the fusion feature modeling module includes: A data preprocessing unit, used for denoising, normalizing and frequency domain conversion of the collected multimodal data; A feature extraction unit for extracting optical, infrared and electrical performance features through a deep learning network; The modal feature fusion unit is used to achieve multimodal feature fusion through the cross-modal attention mechanism.

[0008] Preferably, the intelligent detection module includes: An object detection unit, used to detect the type and location of chip defects; The feature extraction unit is used to further analyze the feature information of the defect area.

[0009] Preferably, the abnormal classification and marking module; An abnormality classification unit, used to classify the abnormality type of the chip; The abnormality marking unit is used to generate marking data for abnormal results for subsequent analysis.

[0010] Preferably, the feedback optimization dynamic adjustment module includes A real-time feedback unit, used to adjust the acquisition parameters of the multimodal data acquisition module according to the detection results; Dynamic optimization unit, used to optimize detection strategies based on digital twin models.

[0011] Preferably, the data storage module comprises: A feature data storage unit, used to store the extracted features of each modal data; A detection result storage unit, used to store defect detection and classification results; Data compression and archiving unit, used to compress and archive historical data for long-term storage.

[0012] Preferably, the inter-modal feature fusion unit fuses features using the following formula:

[0013] in, Represents the optical characteristics, Indicates thermal performance characteristics, Indicates electrical performance characteristics.

[0014] Preferably, the dynamic optimization unit optimizes the detection system through an objective function; The formula is:

[0015] in, represents the detection error, represents the energy consumption constraint, Indicates runtime delay.

[0016] A chip detection method comprises the following steps: S1. Collect the optical visual data, thermal performance data and electrical performance signals of the chip simultaneously through the multimodal data acquisition module; S2, preprocessing the collected data through the fusion feature modeling module, including denoising, normalization and feature extraction, and generating a unified fusion feature through multimodal feature fusion; S3. Analyze the fusion features through the intelligent detection module to detect the defect category and defect location of the chip, and then classify the abnormal conditions in the detection results through the abnormal classification and marking module to generate marked data; S4. The dynamic adjustment module is optimized through feedback to adjust the collection parameters and detection strategies according to the classification results, optimize the system operation, and store the collected raw data, processed features and detection results through the data storage module.

[0017] The present invention provides a chip detection system and a chip detection method, which have the following beneficial effects: 1. The present invention adopts a fusion feature modeling module, uses a deep learning network to extract features from optical, thermal and electrical performance data, and optimizes feature weights through a cross-modal attention mechanism. The multimodal feature fusion technology significantly enhances the system's ability to express complex defect patterns and improves the accuracy and reliability of detection, especially the identification of problems such as tiny cracks, poor heat dissipation and unstable signals.

[0018] 2. The present invention integrates optical vision data, infrared thermal performance data and electrical performance signals through a multimodal data acquisition module to form a comprehensive chip information representation. Compared with the traditional single-modal detection method, the system can simultaneously detect the appearance defects, thermal performance anomalies and electrical performance failures of the chip, realize multi-dimensional and all-round monitoring of chip quality, and greatly improve the detection coverage and comprehensiveness.

[0019] 3. The present invention combines the intelligent detection module with the abnormal classification and marking module, which can not only quickly locate the defect position of the chip, but also classify and mark different types of abnormalities. The classification results generate marked data in an intuitive form, including heat maps, defect locations and severity score reports, which facilitates subsequent quality control and problem tracing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a system logic diagram of the present invention; Figure 2 It is a schematic diagram of the system layer distribution of the present invention; Figure 3 It is a schematic diagram of a multimodal data acquisition module of the present invention; Figure 4 It is a schematic diagram of a melting characteristic modeling module of the present invention; Figure 5 It is a schematic diagram of the intelligent detection module of the present invention; Figure 6 It is a schematic diagram of an abnormal classification and marking module of the present invention; Figure 7 This is a schematic diagram of a feedback optimization dynamic adjustment module of the present invention; Figure 8 It is a schematic diagram of a data storage module of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Please refer to the attached Figure 1 -Attached Figure 8 , an embodiment of the present invention provides a chip detection system and a chip detection method, including; Multimodal data acquisition module, used to collect optical visual data, thermal performance data and electrical performance data of the chip; Melting feature modeling module, used for preprocessing, feature extraction and feature fusion modeling of multimodal data; Intelligent detection module, used for defect detection and location based on fused features; Anomaly classification and marking module, used to classify and mark anomalies based on defect detection results; Feedback optimization dynamic adjustment module, used to optimize the detection process according to the classification results and dynamically adjust the parameters of the data acquisition module; A data storage module, used for storing detection data; Specific; In this embodiment, the multimodal data acquisition module comprehensively acquires the appearance image, thermal performance data and electrical performance signal of the chip by integrating the optical vision acquisition unit, the infrared thermal imaging acquisition unit and the electrical performance signal acquisition unit to form multidimensional data, which provides a comprehensive basis for subsequent analysis. The multimodal data acquisition module ensures the synchronous acquisition of optical, thermal imaging and electrical performance signals through a time synchronization controller, generates multimodal data frames, and provides time and space consistent data for subsequent fusion modeling. Through the multimodal data acquisition module, the system can comprehensively acquire multidimensional information of the chip, realize comprehensive support for the detection of defects on the chip surface, abnormal thermal performance and faults in electrical performance, and provide a reliable data basis for subsequent detection; In this embodiment, the fusion feature modeling module performs unified preprocessing on the collected multimodal data, extracts optical, thermal and electrical performance features respectively, and constructs a unified fusion feature through the modal feature fusion mechanism. The modal feature fusion unit realizes feature interaction modeling through the cross-modal attention mechanism. By dynamically adjusting the weights of different modal features, optimizing the collaborative relationship between optical, thermal and electrical performance features, and globally modeling the fusion features through the Transformer network to generate a unified high-dimensional feature representation, the fusion feature modeling module realizes efficient fusion of multimodal data, combines the advantages of optical, thermal and electrical performance features, and provides semantically consistent and highly representative fusion features for subsequent defect detection; In this embodiment, the intelligent detection module detects and locates defects on the fused features through a deep learning target detection network, completes the category determination and spatial position calibration of chip defects, and realizes rapid detection and accurate positioning of chip defects, providing high-quality input data for subsequent abnormal classification and labeling; In this embodiment, the abnormal classification and marking module classifies the abnormal conditions of the chip according to the detection results of the intelligent detection module and generates marking data. The abnormal classification and marking module provides a structured and intuitive abnormal representation, which significantly improves the efficiency and accuracy of abnormal diagnosis. In this embodiment, the feedback optimization dynamic adjustment module adjusts the acquisition parameters and detection strategies through real-time feedback and dynamic optimization mechanisms to improve system performance. The feedback optimization dynamic adjustment module enhances the adaptability and detection efficiency of the system through dynamic adjustment and optimization, and adapts to complex detection scenarios. In this embodiment, the data storage module effectively manages the multimodal data, fusion features and detection results generated during the operation of the system. The data storage module provides comprehensive data support for system optimization and historical tracing, and improves data management capabilities.

[0023] The multimodal data acquisition module comprises: An optical vision acquisition unit, used to acquire global and local image information of the chip surface; Infrared thermal imaging acquisition unit, used to collect dynamic thermal distribution of the chip during operation; Electrical performance signal acquisition unit, used to collect dynamic current, voltage and frequency signals of the chip; Specifically, the optical vision acquisition unit is used to collect global images and local microscopic images of the chip surface to obtain detailed information on chip appearance defects, and the infrared thermal imaging acquisition unit is used to record the heat distribution of the chip in the working state and analyze the chip's heat dissipation performance and internal thermal anomalies; Thermal imaging data acquisition: The collected temperature distribution data is expressed as , reflecting the temperature changes when the chip is running dynamically.

[0024] The normalization formula is:

[0025] in, and are the mean and standard deviation of temperature, respectively.

[0026] Regional division: Based on the temperature difference threshold , identify potential thermal anomaly areas:

[0027] The electrical performance signal acquisition unit obtains the dynamic current, voltage and frequency signals of the chip through automatic test equipment (ATE).

[0028] The fusion feature modeling module includes: A data preprocessing unit, used for denoising, normalizing and frequency domain conversion of the collected multimodal data; A feature extraction unit for extracting optical, infrared and electrical performance features through a deep learning network; Modal feature fusion unit, used to achieve multimodal feature fusion through cross-modal attention mechanism; Specific; The data preprocessing unit is used to normalize and process the noise of multimodal data, laying the foundation for subsequent feature extraction and fusion.

[0029] 1. Optical data preprocessing:

[0030] Input optical image After Gaussian filtering to remove background noise, the formula is:

[0031] in, is the Gaussian kernel, is the standard deviation.

[0032] Infrared thermal imaging data preprocessing: Thermal imaging data After normalization, the influence of ambient temperature is eliminated, and the formula is:

[0033] in, is the mean temperature, is the temperature standard deviation.

[0034] Electrical performance data preprocessing: Electrical performance signal Perform frequency domain conversion and extract frequency domain features:

[0035] Frequency domain data Reflects the high-frequency characteristics of chip operation and is used to analyze transient faults; Feature extraction; The feature extraction unit uses a deep learning network to extract features from optical, infrared and electrical performance data respectively; Optical feature extraction; Extracting spatial features of optical images using convolutional neural networks:

[0036] The extracted optical features mainly include edge information and texture features, which are used to describe the defects on the chip surface; Infrared feature extraction: A thermal sensing network is used to extract the dynamic distribution characteristics of thermal performance data.

[0037]

[0038] Thermal signatures can reflect the chip's cooling status and potential overheating issues.

[0039] Electrical performance feature extraction: Use multi-layer perceptron to analyze the frequency domain data of electrical performance Perform feature extraction:

[0040] The extracted electrical performance characteristics include amplitude change and frequency distribution, which are used to evaluate the electrical stability of the chip; Modal feature fusion The modal feature fusion unit achieves deep integration of multimodal features through cross-modal attention mechanism and Transformer network.

[0041] Cross-modal attention mechanism: In the fusion process, in order to model the interaction between modalities, the attention mechanism is used to calculate the importance weight of modal features.

[0042] in: : query features; , ;Key and value features; ; Feature dimension; Transformer Fusion: Extracted multimodal features , and Input into the Transformer network, and global feature integration is achieved through multi-head attention mechanism and feedforward neural network:

[0043] Output fusion features It is a unified semantic representation that contains the essence of each modal information; The fusion feature modeling module realizes the complete process of multimodal data from preprocessing to feature extraction and then to feature fusion. The joint modeling of optical, infrared and electrical performance data provides a solid foundation for multi-dimensional detection of chips. Target detection unit; Input data: The main function of the target detection unit is to integrate the input features Conduct defect detection and output defect category and location information; Fusion Features It is the output of the fusion feature modeling module, which contains multimodal information of optical, infrared and electrical properties. Its format is:

[0044] in, is the sample size, is the dimension of fusion features.

[0045] Object Detection Model: Use deep learning object detection networks to identify and locate defects; The detection process is carried out through the network model Represented as 20

[0046] in: Indicates the inspection results, including defect categories and the position bounding box ; To detect the training parameters of the model; Output format: The inspection results include defect categories and the position bounding box

[0047] in, is the number of defects detected, the bounding box Defined as:

[0048]

[0049] is the coordinate of the upper left corner of the bounding box, are the width and height of the bounding box respectively; In order to adapt to defects of different sizes, the target detection unit uses a feature pyramid network to detect multi-scale features to ensure that both large-scale defects and tiny cracks can be accurately located; Feature Extraction Unit The feature extraction unit further performs feature analysis on the target detection area and extracts more fine-grained information for evaluating the nature and severity of the defects.

[0050] Detection area feature extraction: The bounding box area output by the object detection unit Crop the corresponding local features from the fused features:

[0051] in, Represents the local features after clipping; Local feature analysis: Local features of the detection area Further analysis using classification models to assess the nature and severity of defects; Classification Model The output is:

[0052] in: represents the probability distribution of defects; are the parameters of the classification model.

[0053] Defect Severity Assessment: Evaluate the severity score of defects based on the statistical properties of local features

[0054] in: is the image intensity of the local area; is the average intensity of normal samples; Results Integration: Severity Scoring Defect category and location Integrate to form a complete test report; Data Output The final output of the intelligent detection module includes; Defect Category ; Defect location bounding box ; Defect Severity Score ; The test results are output in a structured data format to facilitate subsequent classification and analysis; The target detection unit is responsible for detecting the type and location of chip defects, and the feature extraction unit further conducts in-depth analysis of the detection area to generate defect severity assessment results.

[0055] Abnormal taxa; The anomaly classification unit is responsible for fine-grained classification of anomalies in the detection results according to categories and properties for subsequent processing; Input data: Receive detection results from the intelligent detection module, including: Defect Category Defect location bounding box Defect Severity Score

[0056] Using deep learning-based anomaly classification models , classify exceptions into different types

[0057] in: Classification rules: is the abnormal type after classification; The classification model combines the defect categories and severity , improve classification accuracy; The classification model combines predefined rules to perform secondary classification of anomalies: Appearance defects: dominated by optical features; Abnormal thermal performance: dominated by infrared thermal characteristics; Electrical performance failure: dominated by electrical performance characteristics.

[0058] Abnormal marking unit; The abnormality marking unit generates corresponding marking data according to the classification results for subsequent analysis.

[0059] Token generation rules: According to the exception category and location Generate labeled data, including: Heat map generation: Generate abnormal heat maps based on thermal performance data and mark areas with excessively high temperatures; Defect location marking: Mark the defect location with a rectangular frame on the chip image ; Severity Score Report: Based on severity score Generate a scoring report; The generated labeled data is post-processed to ensure that the labeled area is highly consistent with the abnormal area, such as optimizing the edge of the heat map labeled area through morphological processing.

[0060] The anomaly classification unit performs fine-grained analysis on defect data, and the anomaly labeling unit generates labeled data that facilitates intuitive presentation and subsequent analysis.

[0061] The feedback optimization dynamic adjustment module includes A real-time feedback unit, used to adjust the acquisition parameters of the multimodal data acquisition module according to the detection results; A dynamic optimization unit for optimizing detection strategies based on the digital twin model; Specific; In this embodiment, the task of the real-time feedback unit is to adjust the operating parameters of the data acquisition module according to the abnormal classification results to ensure the pertinence and efficiency of the collected data; Data input: Receive data output by the anomaly classification module, including: Exception category ; Defect location ; Severity Rating ; Adjustment of acquisition parameters; Dynamically adjust the working parameters of each acquisition unit according to the severity and characteristics of the classification results; Optical vision acquisition: Camera exposure time Taking the severity of the defect as a reference, the adjustment formula is;

[0062] is the initial exposure time, is the adjustment factor.

[0063] Infrared thermal imaging acquisition: Thermal imaging sensitivity Dynamically adjust according to the detected temperature difference

[0064] in, is the default sensitivity, is the temperature difference range, is the regulating factor; Electrical performance signal acquisition: According to the exception category , adjust the sampling frequency

[0065] in, is the basic sampling frequency, is the classification association coefficient.

[0066] Feedback adjustment mechanism: The real-time feedback unit transmits the updated parameters to the data acquisition module to ensure that the data is re-collected under the new parameter conditions; The real-time feedback unit transmits the updated parameters to the data acquisition module to ensure that the data is re-collected under the new parameter conditions; Dynamic Optimization Unit; In this embodiment, the dynamic optimization unit is responsible for simulating the operating status of the detection system based on the digital twin technology and adjusting the system strategy based on the optimization target; Digital twin model construction: Build a digital twin model that is consistent with the actual detection system operation The model simulates the multimodal acquisition and detection process, with the input being the current parameters and the output being the simulated detection results; Optimization objective function: Define a comprehensive optimization objective function , while balancing detection accuracy, operational efficiency and energy consumption:

[0067] in: : Detection error, used to measure the accuracy of detection; : Runtime delay, reflecting the system detection speed; : Energy consumption index, controlling the energy consumption of the system; , : Weight factor, which determines the optimization priority of each item; Optimization strategy: By simulating different parameter combinations, the optimization objective function is solved to find the optimal parameter configuration:

[0068] in, Indicates acquisition parameters and detection strategies; The dynamic optimization unit transmits the optimized parameters and strategies to the real-time feedback unit to guide the system operation adjustment; The feedback optimization dynamic adjustment module realizes the dynamic adjustment of multimodal data acquisition parameters and the strategy optimization of the detection system. The real-time feedback unit flexibly adjusts the parameters based on the detection results, and the dynamic optimization unit simulates the system performance through digital twin technology and provides optimization suggestions to ensure that the system maintains efficient operation in complex detection environments.

[0069] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A chip detection system, characterized in that: include; Multimodal data acquisition module, used to collect optical visual data, thermal performance data and electrical performance data of the chip; Melting feature modeling module, used for preprocessing, feature extraction and feature fusion modeling of multimodal data; Intelligent detection module, used for defect detection and location based on fused features; Anomaly classification and marking module, used to classify and mark anomalies based on defect detection results; Feedback optimization dynamic adjustment module, used to optimize the detection process according to the classification results and dynamically adjust the parameters of the data acquisition module; The data storage module is used to store the detection data.

2. A chip detection system according to claim 1, characterized in that: The multimodal data acquisition module comprises: An optical vision acquisition unit, used to acquire global and local image information of the chip surface; Infrared thermal imaging acquisition unit, used to collect dynamic thermal distribution of the chip during operation; The electrical performance signal acquisition unit is used to collect the dynamic current, voltage and frequency signals of the chip.

3. A chip detection system according to claim 1, characterized in that: The fusion feature modeling module includes: A data preprocessing unit, used for denoising, normalizing and frequency domain conversion of the collected multimodal data; A feature extraction unit for extracting optical, infrared and electrical performance features through a deep learning network; The modal feature fusion unit is used to achieve multimodal feature fusion through the cross-modal attention mechanism.

4. A chip detection system according to claim 1, characterized in that: The intelligent detection module comprises: An object detection unit, used to detect the type and location of chip defects; The feature extraction unit is used to further analyze the feature information of the defect area.

5. A chip detection system according to claim 1, characterized in that: The abnormal classification and marking module; An abnormality classification unit, used to classify the abnormality type of the chip; The abnormality marking unit is used to generate marking data for abnormal results for subsequent analysis.

6. A chip detection system according to claim 1, characterized in that: The feedback optimization dynamic adjustment module includes A real-time feedback unit, used to adjust the acquisition parameters of the multimodal data acquisition module according to the detection results; Dynamic optimization unit, used to optimize detection strategies based on digital twin models.

7. A chip detection system according to claim 1, characterized in that: The data storage module comprises: A feature data storage unit, used to store the extracted features of each modal data; A detection result storage unit, used to store defect detection and classification results; Data compression and archiving unit, used to compress and archive historical data for long-term storage.

8. A chip detection system according to claim 1, characterized in that: The inter-modal feature fusion unit fuses features using the following formula: in, Represents the optical characteristics, Indicates thermal performance characteristics, Indicates electrical performance characteristics.

9. A chip detection system according to claim 1, characterized in that: The dynamic optimization unit optimizes the detection system through an objective function; The formula is: in, represents the detection error, represents the energy consumption constraint, Indicates runtime delay.

10. A chip detection method, according to a chip detection system according to claims 1-9, characterized in that: The steps include: S1. Collect the optical visual data, thermal performance data and electrical performance signals of the chip simultaneously through the multimodal data acquisition module; S2, preprocessing the collected data through the fusion feature modeling module, including denoising, normalization and feature extraction, and generating a unified fusion feature through multimodal feature fusion; S3. Analyze the fusion features through the intelligent detection module to detect the defect category and defect location of the chip, and then classify the abnormal conditions in the detection results through the abnormal classification and marking module to generate marked data; S4. The dynamic adjustment module is optimized through feedback to adjust the collection parameters and detection strategies according to the classification results, optimize the system operation, and store the collected raw data, processed features and detection results through the data storage module.

Images