Chip defect detection method, device and equipment based on multi-mode optical imaging

Abstract

The invention relates to the technical field of semiconductor defect detection, and discloses a chip defect detection method, device and equipment based on multi-modal optical imaging.The multi-modal imaging data of a chip are obtained through a plurality of optical detection channels, a digital twin is constructed based on the data, a plurality of detection reproduction models are generated, and the detection reproduction models are used for detecting defects of the chip. The method comprises the steps of calculating the information confidence coefficient of each model, converting the information confidence coefficient into a detection feature matrix, calculating the correlation between the models through cross-modal correlation analysis, carrying out abnormal labeling on the feature matrix according to correlation information, carrying out defect perception and generating a detection result. And particularly, the method has high application value in identification of tiny and potential defects, and solves the problem of low detection precision of a single optical imaging technology in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of semiconductor defect detection, and in particular to a chip defect detection method, device and equipment based on multimodal optical imaging. Background Art

[0002] Semiconductor chips play a core role in modern electronic devices, so their quality and precision must be strictly controlled during the manufacturing process. With the development of integrated circuit technology, the complexity and precision requirements of chips are becoming increasingly higher, resulting in traditional defect detection methods facing increasing challenges in identifying tiny defects, especially potential defects. Traditional optical defect detection methods usually rely on a single optical imaging technology, but this method often has problems with low detection accuracy and weak ability to identify complex defects. Summary of the Invention

[0003] The purpose of the present invention is to provide a chip defect detection method, device and equipment based on multimodal optical imaging, aiming to solve the problem of low detection accuracy of single optical imaging technology in the prior art.

[0004] The present invention is implemented as follows: In a first aspect, the present invention provides a chip defect detection method based on multimodal optical imaging, comprising: performing optical inspection on the semiconductor chip through a plurality of optical inspection channels to obtain multimodal optical imaging data of the semiconductor chip; constructing digital twins of semiconductor chips respectively according to the multimodal optical imaging data to generate a plurality of detection replication models; Calculating information confidence for each of the detection and reproduction models to convert the detection and reproduction models into corresponding detection feature matrices; Calculating cross-modal correlations of the detection feature matrices based on the optical detection channels corresponding to the detection reproduction models to generate correlation information between the detection reproduction models; Anomaly marking processing is performed on each of the detection feature matrices according to the correlation information, and defect perception is performed on the semiconductor chip according to each of the detection feature matrices that have been anomaly marked to generate defect detection information.

[0005] In a second aspect, the present invention provides a chip defect detection device based on multimodal optical imaging, which is used to implement the chip defect detection method based on multimodal optical imaging described in any one of the first aspects, including: an optical detection module, configured to perform optical detection on the semiconductor chip through a plurality of optical detection channels to obtain multimodal optical imaging data of the semiconductor chip; A digital reproduction module, configured to construct digital twins of semiconductor chips based on the multimodal optical imaging data to generate a plurality of detection reproduction models; An information conversion module, configured to calculate information confidence for each of the detection and reproduction models, so as to convert the detection and reproduction models into corresponding detection feature matrices; a correlation analysis module, configured to calculate cross-modal correlations of the detection feature matrices based on the optical detection channels corresponding to the detection replication models, so as to generate correlation information between the detection replication models; The defect perception module is used to perform abnormality marking processing on each of the detection feature matrices according to the correlation information, and to perform defect perception on the semiconductor chip according to each of the detection feature matrices that have been abnormally marked to generate defect detection information.

[0006] In a third aspect, the present invention provides a chip defect detection device based on multimodal optical imaging, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements a computer method described in any one of the first aspects.

[0007] The present invention provides a chip defect detection method based on multimodal optical imaging, which has the following beneficial effects: The present invention obtains multimodal imaging data of the chip through multiple optical detection channels, builds a digital twin based on these data, and generates multiple detection reproduction models. The information confidence of each model is calculated and converted into a detection feature matrix. The correlation between the models is calculated through cross-modal correlation analysis, and the feature matrix is ​​annotated according to the correlation information. Defects are perceived and detection results are generated. This method can improve the accuracy and robustness of defect detection, especially in the identification of small and potential defects. It has high application value and solves the problem of low detection accuracy of single optical imaging technology in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 1 is a schematic diagram of the steps of a chip defect detection method based on multimodal optical imaging provided by an embodiment of the present invention; Figure 2 It is a structural schematic diagram of a chip defect detection device based on multimodal optical imaging provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0009] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0010] The implementation of the present invention is described in detail below with reference to specific embodiments.

[0011] Reference Figure 1 、 Figure 2 As shown, a preferred embodiment of the present invention is provided.

[0012] In a first aspect, the present invention provides a chip defect detection method based on multimodal optical imaging, comprising: S1: Optically inspecting a semiconductor chip through a plurality of optical inspection channels to obtain multimodal optical imaging data of the semiconductor chip; S2: constructing digital twins of semiconductor chips respectively according to the multimodal optical imaging data to generate a plurality of detection replication models; S3: Calculating information confidence for each of the detection and reproduction models to convert the detection and reproduction models into corresponding detection feature matrices; S4: calculating cross-modal correlations of the detection feature matrices based on the optical detection channels corresponding to the detection reproduction models to generate correlation information between the detection reproduction models; S5: performing abnormality marking processing on each of the detection feature matrices according to the correlation information, and performing defect perception on the semiconductor chip according to each of the detection feature matrices that have been abnormally marked, so as to generate defect detection information.

[0013] Specifically, in step S1 of the embodiment provided by the present invention, when performing optical inspection of semiconductor chips, it is first necessary to select a suitable optical inspection channel based on the characteristics of the chip, the expected inspection target and the required accuracy. Common optical inspection channels include: bright field illumination: used to capture the visible features of the chip surface and help detect surface defects; dark field illumination: suitable for detecting tiny defects on the surface that are difficult to observe directly through the bright field; polarized light imaging: used to detect changes caused by stress, strain or crystal defects; infrared imaging: used to detect the thermal characteristics of the chip and detect thermal anomalies or electrical failures; fluorescence imaging: using fluorescent markers to detect chemicals or defects in the chip; confocal microscopy: providing high-resolution surface images for fine defect detection.

[0014] More specifically, the chip is imaged simultaneously or sequentially using the above-mentioned different optical detection channels. Each detection method captures different features of the chip through different imaging principles. For example, bright field illumination can display macroscopic defect areas, infrared imaging can help identify heat-related defects, and polarized light may reveal stress distribution. The imaging data acquired through these channels will form a multimodal dataset containing multi-dimensional information of the chip. Each optical imaging data is preprocessed, including denoising, image enhancement, segmentation and other steps. Each imaging data is processed by a feature extraction algorithm to extract features unique to each channel, such as edges, textures, color distribution, thermal imaging data, etc. The data from different detection channels are fused to form a comprehensive optical imaging dataset. This dataset contains detailed information from each channel for subsequent defect analysis.

[0015] It is understandable that multimodal optical imaging technology complements a variety of different detection methods and can perform high-precision inspections on chips at different scales and angles. This means that even tiny surface cracks, structural defects or thermal failures can be accurately captured. Due to the use of multiple optical channels, this method can capture different types of defects at the same time. For example, infrared imaging can detect thermal anomalies, while polarized light can reveal tiny defects caused by stress. In this way, various types of chip defects can be comprehensively and accurately identified. The imaging data of different detection channels can be fused to provide richer information, thereby improving the accuracy of detection. For example, some defects may not be obvious in a single channel, but they can be revealed by the combination of multimodal data, avoiding the risk of missed detection.

[0016] More specifically, because multiple optical detection methods cover different physical properties, they complement each other and can effectively reduce false alarms (areas incorrectly marked as defects) and missed detections (situations where actual defects are not found). This comprehensive detection method significantly improves the accuracy of overall detection. Through the collaboration of multiple detection channels, defects on the chip can be located more accurately. For example, infrared imaging can quickly locate areas of thermal anomalies, while fluorescence imaging can indicate anomalies in certain specific substances or areas, thereby helping to locate potential problems. Multimodal imaging data can generate digital twins of semiconductor chips, which not only provides a basis for defect detection, but also provides model support for subsequent fault analysis and optimization design. The digital twin can simulate the state of the chip under different conditions and predict the occurrence of potential problems.

[0017] Specifically, in step S2 of the embodiment provided by the present invention, imaging data obtained from multiple optical detection channels (such as bright field, dark field, polarized light, infrared, fluorescence, etc.) need to be collected first. Each type of optical imaging data provides different chip surface or internal feature information. Since different detection channels may have slight spatial displacements, data alignment (such as image registration) is necessary, which ensures that the images of all channels are in the same coordinate system for accurate fusion and analysis.

[0018] More specifically, multimodal optical imaging data is used to convert two-dimensional image data into three-dimensional models through image processing algorithms. These models may involve synthesizing different imaging channels (such as polarized light imaging, infrared images, etc.) into a comprehensive three-dimensional digital twin through three-dimensional reconstruction algorithms. Each surface feature of the chip is reconstructed in three dimensions, and possible missing areas are filled through various algorithms to ensure the integrity of the model. Multimodal imaging data is synthesized into a global coordinate system through computer vision technology (such as stereo vision, structured light, etc.), and finally a detailed digital twin is generated to represent the geometric shape of the chip and its various physical properties. In order to ensure the high accuracy of the digital twin, each part of the three-dimensional model needs to be refined, including optimizing the geometric structure, improving the resolution, repairing missing parts, and further optimizing the physical performance of the model through additional information provided by multimodal data (such as heat maps, stress maps). In this process, some machine learning and deep learning methods (for example, image segmentation and enhancement functions of deep neural networks) may also be used to further improve the details of the model.

[0019] More specifically, for the image data of each optical detection channel, an independent detection reproduction model is generated based on its specific imaging method and target. For example, bright field imaging generates a model based on surface features, infrared imaging generates a thermal distribution model, and polarized light generates a stress distribution model. Each model shows the physical characteristics of the chip under specific conditions. These reproduction models can provide a specific perspective for subsequent defect diagnosis. The detection data from multiple optical detection channels are fused to generate a comprehensive detection reproduction model. This model not only shows the surface structure of the chip, but also shows its characteristics under multiple optical conditions (such as thermal characteristics, stress distribution, etc.). Through multi-channel fusion, the defect information revealed by different channels can be integrated into one model, making the detection results more comprehensive and accurate. When generating the detection reproduction model, the physical properties of the semiconductor chip (such as material properties, stress-strain behavior, etc.) can also be combined to establish a physical model to simulate the performance of the chip under different environmental conditions. This physical model helps to understand the causes of certain detection results, such as thermal failure, mechanical stress, material aging and other problems.

[0020] More specifically, verification is performed using experimental data or actual samples of known defects to ensure that the generated digital twin and detection reproduction model match the characteristics and defects of the actual chip. The accuracy of the model is verified by comparing the experimental data with the model's predictions. If the model deviates, it needs to be tuned. Based on the verification results, the model is optimized and parameters are adjusted to enhance the model's adaptability to abnormal situations, improve reproduction accuracy and diagnostic accuracy. This step may include further image processing, data reconstruction, and algorithm optimization to ensure the final model has high precision and robustness.

[0021] It is understandable that by constructing a digital twin, a high-precision three-dimensional model of the chip can be generated, perfectly restoring the chip's appearance, internal structure and surface features. Combined with multimodal optical imaging data, it can provide comprehensive chip information, including geometry, thermal distribution, stress state, etc. The fusion of multi-channel data further improves the accuracy of the model and avoids the limitations of single-channel imaging. Through the generated digital twin and reproduction model, defect detection can be effectively performed. For example, the thermal imaging model can help detect thermal anomaly areas, while the polarization light model can reveal surface stress-related defects. The reproduction model of multimodal data makes the detection results more accurate and can detect chip defects in multiple dimensions, thereby avoiding the risk of missed detection and false alarms.

[0022] More specifically, digital twins can not only help detect defects during the chip production process, but also provide valuable feedback during the chip design stage. By simulating the performance of chips under different environmental conditions, designers can optimize the chip design and avoid potential failure problems. The simulation results can be used to predict performance changes of chips in long-term use, thereby providing manufacturers with a basis for performance optimization. By generating multiple reproduction models, the specific location of defects on the chip can be accurately located, and the nature of the defects can be analyzed. For example, the infrared imaging model can quickly locate the thermal anomaly area, and polarized light imaging can further analyze the stress distribution to help discover potential cracks or material defects. This method has good scalability and can be adaptively adjusted for different types of semiconductor chips or different detection targets. The model of each channel can be customized according to needs to meet specific detection requirements.

[0023] Specifically, in step S3 of the embodiment provided by the present invention, for each detection reproduction model (for example, a reproduction model based on different optical imaging channels, such as bright field, polarized light, infrared, etc.), it is necessary to define an appropriate feature extraction method. These features can be image statistics, geometric shape features, thermal distribution, stress distribution, texture features, etc. Computer vision techniques such as edge detection (Sobel, Canny, etc.), region growing, texture analysis, contour extraction, etc. are used to extract relevant detection features from each reproduction model. In this step, deep learning methods (such as convolutional neural networks) may be required to automatically extract complex features from the reproduction model.

[0024] More specifically, for the features extracted from each detection reproduction model, the feature consistency between the reproduction models of different detection channels is calculated. For example, by comparing the feature values ​​of various detection channels (such as infrared and polarized light) at the same location, their consistency (such as Pearson correlation coefficient, Euclidean distance, etc.) is calculated. The higher the consistency, the more reliable the defect detection results at that location. The significance of each feature is evaluated, that is, the importance of the feature in distinguishing normal and defective areas. Methods such as information gain and chi-square test can be used to calculate significance. In multimodal data, different modalities may give different detection results, so it is necessary to calculate the comprehensive confidence through weighted fusion methods. For example, the confidence of each detection model is combined through weighted averaging, Bayesian inference, etc. to obtain the final detection confidence.

[0025] More specifically, for each eigenvalue of the reproduction model, algorithms such as weighted averaging, Bayesian method or fuzzy logic can be used to calculate the final confidence. For example, if some features have stronger recognition capabilities and other features have weaker recognition effects, the former can be assigned higher weights. According to the credibility weight of the feature, the detection results of each feature are weighted to obtain the final detection confidence. The weight setting can be optimized through model training, expert judgment or based on historical data. Using Bayesian theorem, the feature confidence obtained in different detection reproduction models is used as prior information, and the comprehensive confidence is calculated through posterior probability.

[0026] More specifically, after completing feature extraction and confidence calculation, the features and confidence of each detection and reproduction model need to be integrated into a feature matrix. Each row of this matrix represents the detection result of a chip, and each column corresponds to a feature. The construction method of the feature matrix can be arranged according to the type of extracted features (such as geometric features, thermal imaging features, stress features, etc.). Each element of the matrix represents the confidence value of a feature in a certain detection and reproduction model. For high-dimensional feature data, dimensionality reduction techniques such as principal component analysis (PCA) and linear discriminant analysis (LDA) can be used to reduce the dimension of the feature matrix while retaining as much useful information as possible to facilitate subsequent processing.

[0027] More specifically, in order to improve the quality of the feature matrix, feature selection is required to eliminate redundant or irrelevant features. Algorithms such as L1 regularization (Lasso regression) and random forest feature importance evaluation can be used for feature selection to ensure that each feature in the matrix is ​​highly relevant to the target detection task. Certain features can be enhanced, such as using data augmentation (such as rotation, cropping, flipping, etc.) to increase data diversity and further improve the robustness of the model.

[0028] More specifically, the constructed feature matrix can be passed as input to a classification model (such as support vector machine (SVM), random forest, neural network, etc.) or a regression model (used to predict specific parameters of defects, such as size and shape). After the model is trained, the output will provide a classification result (defect type) or regression result (specific location and size of the defect) for each chip inspection.

[0029] It is understandable that by converting the multimodal detection reproduction model into a feature matrix, key features can be extracted from information obtained from multiple dimensions and multiple detection channels. This high-dimensional feature representation helps improve the accuracy and sensitivity of defect detection. The feature matrix provides rich feature information, enabling subsequent machine learning algorithms to better perform classification or regression, accurately identify and locate defects. By calculating information confidence, the model can more accurately assess the reliability of each detection result, which helps to screen out high-confidence detection results and reduce the risk of false or missed detections. The confidence fusion method effectively balances the advantages and disadvantages of different modal data, further improving overall detection performance.

[0030] More specifically, the feature matrix integrates data from different optical inspection channels and can comprehensively present the inspection characteristics of semiconductor chips. Through dimensionality reduction and feature selection, it avoids feature redundancy and ensures the efficiency and accuracy of the model. The generated feature matrix not only facilitates the automated detection of defects, but also provides explainability for analysis, helping engineers understand the role of different features in the inspection process. The multimodal information contained in the feature matrix provides rich input for the model, making the inspection system more robust to different types of defects and environmental changes, and reducing dependence on single-modal data.

[0031] Specifically, in step S4 of the embodiment provided by the present invention, features are extracted from the detection reproduction models obtained from different optical detection channels (such as infrared, polarized light, X-rays, etc.). The detection features of each modality can generate a feature matrix through feature extraction technology (such as edge detection, texture analysis, thermal distribution analysis, etc.). Each row of the feature matrix corresponds to a sample (for example, an image block of a specific area), and each column corresponds to the extracted feature. Since the features of different modalities may have different dimensions and scales, it is often necessary to standardize or normalize each feature matrix before performing correlation calculation to ensure that they are compared on the same scale. Common methods include Z-score normalization, Min-Max normalization, etc.

[0032] More specifically, when calculating cross-modal correlation, several common statistical measurement methods can be used, such as the Pearson correlation coefficient, the Spearman rank correlation coefficient, and mutual information. The Pearson correlation coefficient is suitable for measuring linear correlation and calculates the linear relationship between two feature matrix columns. The closer the value is to 1 or -1, the stronger the correlation. The Spearman rank correlation coefficient is suitable for measuring nonlinear relationships and evaluates the monotonic relationship between features. Mutual information measures the amount of information shared between two features and is suitable for measuring more complex and nonlinear data relationships.

[0033] More specifically, for each pair of optical detection channels (i.e., different detection and reproduction models), the correlation between their feature matrices is calculated. For example, the correlation between polarized light and infrared imaging results, infrared and bright field, etc., is calculated. By calculating these correlation coefficients, a cross-modal correlation matrix is ​​constructed. Each element in this matrix represents the correlation between two optical detection channels (modalities). If there are multiple modalities, a cross-modal correlation graph can be constructed, where each node represents a detection channel and the edge weight represents the correlation between the two channels. This graph structure can be used to analyze correlation patterns between modalities.

[0034] More specifically, based on the calculated cross-modal correlation, the similarities and differences between different optical channels can be evaluated. Stronger correlation indicates that the information provided by these channels has similar characteristics and may be complementary to defect detection. Different modal combinations can be analyzed to see which combinations of modalities have higher redundancy and which combinations provide more complementary information. High-redundancy combinations may lead to information overfitting, while low-redundancy combinations can provide richer feature information. By calculating cross-modal correlation, the correlation weight between each modality is obtained. Based on these weights, the features of different modalities can be weightedly fused. For example, if the correlation between two channels (such as infrared and polarized light) is high, you can choose to weightedly fuse the features of these two channels to further improve the representativeness of the features. For channels with lower correlation, you can consider independently modeling their features.

[0035] More specifically, if the cross-modal correlation is high, it means that some features are redundant in multiple modalities. Redundant features can be reduced through dimensionality reduction (such as principal component analysis PCA, canonical correlation analysis CCA, etc.). When fusing features, combined with cross-modal correlation, canonical correlation analysis (CCA) can be used to select the feature combination with the maximum correlation.

[0036] It is understandable that through cross-modal correlation calculation, the relevant information between different detection reproduction models can be fully integrated, thereby improving the overall detection accuracy. Cross-modal feature fusion can effectively utilize the advantages of different modalities and increase the detection capability of complex defects. Based on correlation analysis, the features of different modalities can be weighted and fused, and the most representative features can be selected for combination to avoid redundant features and information overfitting. Effective feature fusion not only improves the performance of the model, but also improves the computational efficiency. By understanding the correlation between modalities, we can better cope with the impact of external factors such as environmental changes and lighting conditions on different modalities, and enhance the robustness of the model. The complementarity of different modalities can improve robustness while reducing false detection or missed detection that may be caused by a single modality.

[0037] Specifically, in step S5 of the embodiment provided by the present invention, based on the correlation information between each detection and reproduction model, the data of different modalities can be weighted, fused or features with higher correlation can be selected. According to the feature matrix, an anomaly detection algorithm is used to mark out samples that have significantly different relationships with other modalities or features. These abnormal samples usually indicate areas where defects may exist. Statistical methods such as the mean-standard deviation method calculate the mean and standard deviation of each sample in the feature matrix to mark samples that deviate greatly from the mean. Distance-based methods such as k-nearest neighbor (KNN), isolation forest, etc. calculate the distance or density between feature samples to mark points that are far away from other samples. Model-based methods such as autoencoders, support vector machines (SVM), etc. learn the distribution of normal data by training an unsupervised model, and then mark points that are significantly different from the normal data distribution.

[0038] More specifically, based on the anomaly detection results, the detection feature matrix is ​​labeled to clearly identify which areas are abnormal areas. Usually, these labels will appear as changes in label values, such as marking abnormal values ​​as "1" (defective areas) and normal values ​​as "0" (non-defective areas).

[0039] More specifically, defect perception refers to identifying potential defects in semiconductor chips through a feature matrix that has been annotated with anomalies. At this stage, the feature matrix with anomaly annotations can be directly used to locate and identify defective areas. For each abnormal area, the system will extract relevant features, such as the degree of abnormality, location, shape, size and other information. Based on this information, image processing algorithms (such as edge detection, region segmentation, contour extraction, etc.) are used to further refine the boundaries of the defective area, and combined with cross-modal correlation, the nature of the defect (such as defect type, severity, etc.) is analyzed.

[0040] More specifically, the defect information generated includes the location of the defect, which clearly marks the specific area where the defect occurs, usually expressed in coordinate form; the defect type, which identifies the type of defect (such as cracks, bubbles, overexposure, corrosion, etc.) by analyzing the characteristics of the abnormal area; the defect size and shape, which quantifies the size and shape of the defect area to help further quality control; and the defect severity, which evaluates the severity of the defect by comparing it with a standard or reference model, such as: minor defect, severe defect, etc.

[0041] More specifically, a final defect detection report is generated, which not only contains defect information, but also includes defect correlation analysis, impact assessment, etc. The report content lists in detail the defect status, type, size, etc. of each detection area, and provides decision support based on anomaly annotation. Through automated generation tools, the defect perception results are converted into a structured report to provide data support for subsequent quality control and production adjustments.

[0042] More specifically, based on defect detection information, the production line can take corresponding processing measures, such as automatically rejecting defective products, or focusing on monitoring and correcting areas where problems may occur. For problematic chips, secondary inspections or repairs may be carried out to reduce the flow of unqualified products into the market. According to defect detection information, the production process can be adjusted, such as optimizing exposure, adjusting material quality, improving process control, etc., and the detection model can be regularly fed back and updated to adapt to new production conditions and improved technologies.

[0043] It is understandable that through anomaly labeling processing, the defective areas on the semiconductor chip can be accurately located to avoid missed detection or false detection. The fusion of cross-modal information and anomaly detection labeling provide richer information, which helps to identify complex defect types and details. Through feature analysis of abnormal areas, not only the location of the defect can be identified, but also the type, shape and severity of the defect can be determined, thereby providing a scientific basis for subsequent defect classification, quality assessment and production adjustment. The entire defect detection process from data collection, anomaly labeling to defect perception and report generation can be completed by an automated system, which greatly improves detection efficiency and reduces human intervention. Through machine learning and data analysis technology, the detection system can continuously learn and optimize to adapt to the new production environment.

[0044] The present invention provides a chip defect detection method based on multimodal optical imaging, which has the following beneficial effects: The present invention obtains multimodal imaging data of the chip through multiple optical detection channels, builds a digital twin based on these data, and generates multiple detection reproduction models. The information confidence of each model is calculated and converted into a detection feature matrix. The correlation between the models is calculated through cross-modal correlation analysis, and the feature matrix is ​​annotated according to the correlation information. Defects are perceived and detection results are generated. This method can improve the accuracy and robustness of defect detection, especially in the identification of small and potential defects. It has high application value and solves the problem of low detection accuracy of single optical imaging technology in the existing technology.

[0045] Preferably, the step of optically inspecting the semiconductor chip through a plurality of optical inspection channels to obtain multimodal optical imaging data of the semiconductor chip includes: S11: placing the semiconductor chip to be inspected at a designated inspection position, activating each pre-set inspection function module, and sequentially constructing corresponding optical inspection channels by adjusting the inspection function parameters of each inspection function module; wherein the optical inspection channels include a bright field illumination inspection channel, a dark field illumination inspection channel, a polarized light illumination inspection channel, an infrared imaging inspection channel, a fluorescence imaging inspection channel, a confocal microscopy inspection channel, and an optical coherence tomography inspection channel; S12: performing optical detection on the semiconductor chip at the designated detection position through the constructed optical detection channel to obtain optical imaging data corresponding to the optical detection channel; S13: adjusting key detection parameters of the optical detection channel to be executed according to the optical imaging data to obtain optical imaging data of the optical detection channel to be executed in the next round; S14: combining the optical imaging data of each optical detection channel to obtain multimodal optical imaging data of the semiconductor chip.

[0046] Specifically, the semiconductor chip to be inspected is placed at a designated inspection position (such as a test bench or under a microscope). This position needs to ensure that the chip can be completely covered by multiple optical inspection channels during the subsequent optical inspection process. The positioning of the chip usually relies on a high-precision positioning system to ensure that the chip remains stable during multi-channel inspection and avoid errors caused by position offset. The pre-set optical inspection function modules are activated through the control system. Different inspection modules can adapt to different types of optical imaging requirements by adjusting their parameters. Functional module settings include selecting different lighting modes, adjusting light source intensity, selecting imaging magnification, etc.

[0047] More specifically, brightfield illumination is a commonly used basic inspection method, using uniform illumination to image the chip surface and is suitable for large-scale surface defect detection. Darkfield illumination produces high-contrast reflection images by illuminating light at a low angle, making it suitable for detecting small, low-reflection surface defects (such as cracks and tiny stains). Polarized light detection adjusts the polarization direction of light to detect subtle changes in surface structure or material properties, with particular sensitivity to surface microscopic defects such as stress cracks. Infrared imaging is primarily used to detect the thermal properties of materials and is suitable for detecting internal defects, uneven surface temperatures, and locations of stress concentration in semiconductor chips. Fluorescence imaging utilizes an excitation light source of a specific wavelength, causing certain materials or defects to fluoresce under this light source, helping to detect defects related to materials or thin film coatings. Confocal microscopy utilizes a point light source and a spatial pinhole for high-resolution imaging and is suitable for three-dimensional imaging and in-depth analysis of tiny structures. Optical coherence tomography measures the coherence of light through interference effects and is suitable for detecting surface and near-surface defects, playing a particularly important role in high-precision three-dimensional tomography.

[0048] More specifically, after initial imaging in each optical detection channel, key parameters of the detection channel are adjusted based on the preliminary optical imaging data. Key parameter adjustments may include optimizing image brightness and clarity, adjusting focal length and magnification to increase image resolution for higher-precision imaging data, and adjusting exposure time and scanning speed to optimize image acquisition time and avoid image blur or excessive noise. Through real-time feedback of optical imaging data and parameter adjustments, each detection channel is guaranteed to provide the most optimal image.

[0049] More specifically, the imaging data from different optical detection channels are fused and the advantages of each channel are combined to obtain a comprehensive multimodal optical imaging dataset. Through feature extraction and fusion algorithms, the imaging information under different modalities is effectively integrated, the unique information of each channel is retained, and duplication and redundancy are reduced.

[0050] More specifically, during the data fusion process, images from different channels need to be aligned and registered to ensure spatial consistency. For example, image registration algorithms are used to geometrically align images from different channels for more accurate comparison and analysis. Through the above processing steps, multimodal optical imaging data is ultimately generated. These data integrate information from different optical detection channels and can comprehensively present the defects, material properties, and other key performance characteristics of semiconductor chips.

[0051] It is understandable that through the use of multiple optical detection channels, semiconductor chips can be comprehensively inspected from different angles. Each channel provides a specific type of information and can effectively identify various types of defects (such as surface cracks, tiny stains, material defects, etc.). Different optical detection technologies can complement each other's shortcomings. For example, the polarized light channel can enhance the perception of tiny surface defects, while infrared imaging can identify intrinsic defects that are difficult to capture with traditional imaging methods. Through multimodal imaging, the sensitivity and accuracy of detection are greatly improved.

[0052] More specifically, by combining data from different optical imaging channels, a more comprehensive and detailed defect analysis of semiconductor chips can be performed. For example, high-resolution local structural data can be obtained through confocal microscopy, while optical coherence tomography can provide three-dimensional depth information. The combination of the two can enable a more in-depth analysis of defects. Through automated image processing and parameter adjustment processes, the detection results can be optimized and imaging conditions can be adjusted in real time. The automated feedback mechanism improves detection efficiency and can make timely adjustments to adapt to different samples and detection requirements. Multimodal data fusion provides richer information and provides strong data support for subsequent defect classification, quality assessment and production optimization. By combining data from different optical channels, the accuracy and reliability of defect detection can be improved, supporting more precise quality control.

[0053] Preferably, the step of adjusting key detection parameters of the optical detection channel to be executed according to the optical imaging data to obtain optical imaging data of the optical detection channel to be executed in the next round includes: S131: performing data dimensionality reduction processing on the optical imaging data to obtain a downsampled image, and performing lightweight convolution processing on the downsampled image using a pre-trained CNN model to generate a low-dimensional anomaly heat map; S132: The optical detection channel corresponding to the low-dimensional anomaly heat map is used as the first detection channel, and the optical detection channel to be executed subsequently is used as the second detection channel. A key detection parameter adjustment tendency analysis is performed on the second detection channel relative to the first detection channel based on the low-dimensional anomaly heat map using the optical detection knowledge graph to generate an expected detection mode for the second detection channel. S133: Analyze the synergistic effect of the key detection parameters of the second detection channel relative to the first detection channel, so as to select the key detection parameters with maximized information complementarity in the expected detection mode to configure the optical detection channel to be executed, so as to detect the semiconductor chip and obtain optical imaging data of the corresponding optical detection channel.

[0054] Specifically, from the preliminary optical imaging data, high-dimensional data is converted into low-dimensional representation through dimensionality reduction techniques (such as PCA, t-SNE, U-Net, etc.), which helps to reduce the complexity of the data while retaining key information to facilitate subsequent processing. The image after dimensionality reduction is usually a downsampled image, which can significantly reduce the computational burden and help quickly locate abnormal areas. The downsampled image is processed by convolutional neural network (CNN). The CNN model used here is a pre-trained lightweight network, which usually includes a smaller model architecture (such as MobileNet, EfficientNet, etc.), aiming to reduce computing resource consumption. Through the CNN model, the network extracts abnormal features in the image, such as chip surface defects, hot spots or other abnormal areas, thereby generating a low-dimensional abnormal heat map to show possible problem areas.

[0055] More specifically, the generated low-dimensional anomaly heat map will correspond to the optical detection channel currently in use (the first detection channel). The heat map shows areas where anomalies may exist, providing key clues for subsequent detection. Based on the abnormal areas in the low-dimensional anomaly heat map, the next detection channel to be executed (the second detection channel) is determined. According to the characteristics of optical detection, the second detection channel may select different detection modes (such as bright field, dark field, infrared, etc.) to supplement the detection of the first channel.

[0056] More specifically, the optical detection knowledge graph is used to analyze the relationship between the first detection channel and the second detection channel. The optical detection knowledge graph contains the dependencies and action mechanisms between different detection channels, optical imaging technologies, and various detection modes and parameters. By analyzing the relationships in the graph, the key detection parameter adjustment tendency analysis is performed based on the low-dimensional anomaly heat map. That is to say, according to the current detection results and the known channel parameter adjustment rules, the parameter adjustment trend of the second detection channel relative to the first detection channel is analyzed.

[0057] More specifically, based on the results of the trend analysis, the expected detection mode of the second detection channel is derived, which takes into account the detection results of the first detection channel. The goal is to select the most appropriate detection mode (for example, increasing the magnification, selecting a light source with a specific wavelength, adjusting the scanning speed, etc.) based on the defect area shown in the abnormal thermogram.

[0058] More specifically, the synergistic effect between the first detection channel and the second detection channel is analyzed. The goal of this step is to identify the possible information complementarity between the two channels. For example, certain defects may not be easy to detect in one channel, but may appear more clearly in another channel. Synergistic effect analysis usually considers the combination of parameters, for example, whether to maximize the complementarity of information by adjusting parameters such as light source angle, exposure time, and focal length.

[0059] More specifically, based on the results of the synergy analysis, the most complementary key inspection parameters are selected and applied to the second inspection channel. These parameter adjustments help to further improve the sensitivity and accuracy of subsequent inspections, ensuring that defects that may be missed in the first inspection channel can be captured.

[0060] More specifically, a second detection channel is configured according to the selected key detection parameters. At this time, the second detection channel has been optimized in the previous steps and has a parameter configuration that is more suitable for the current detection target. The optimized second detection channel is used to detect semiconductor chips to obtain new optical imaging data. At this time, the detection data not only takes into account the results of the first detection channel, but also through preliminary analysis, can more accurately detect more types of defects.

[0061] It is understandable that through data dimensionality reduction and lightweight convolution processing, key information can be automatically extracted from large amounts of optical imaging data, and the parameters of subsequent detection channels can be automatically adjusted. By analyzing optical imaging data in real time, the optimization of the detection process is more efficient, avoiding manual intervention. Through low-dimensional abnormal heat maps, abnormal areas in semiconductor chips can be quickly located, providing accurate target areas for subsequent detection. Further, through optical detection knowledge graphs and synergistic effect analysis, the most appropriate detection mode can be selected, thereby improving the sensitivity and accuracy of detection.

[0062] More specifically, with the help of CNN models and knowledge graphs, abnormal areas can be effectively identified and efficiently analyzed, rather than relying solely on traditional manual adjustments and single-channel detection. This multimodal data fusion method can capture chip defects more comprehensively. Through synergistic effect analysis, parameter adjustments between different channels can not only optimize the performance of individual channels, but also maximize the complementarity between channels, ensuring that the optical imaging data of multiple channels can complement each other, thereby covering a wider range of defect types. This method enables the system to automatically adjust subsequent detection parameters based on the previous detection results, forming an adaptive optimization process without manual intervention. The system can flexibly adjust the detection channel parameters according to different defect types and detection requirements.

[0063] Preferably, the steps of constructing digital twins of semiconductor chips respectively according to the multimodal optical imaging data to generate several detection replication models include: S21: Acquire standard structural information of a semiconductor chip, and encode information corresponding to various optical detection channels of the semiconductor chip according to the standard structural information, so as to obtain ideal structural feature information of the semiconductor chip corresponding to various optical detection channels; S22: Decomposing each item of optical imaging data in the multimodal optical imaging data into inherent structural features and suspected defect features based on the ideal structural feature information, so as to decompose each item of optical imaging data in the multimodal optical imaging data into inherent structural features and suspected defect features; S23: Dividing the semiconductor chip into a surface layer, an intermediate layer, and a deep layer based on the ideal structural feature information, and dividing the surface layer into a plurality of detection areas to construct a digital reproduction framework of the semiconductor chip; S24: performing a physical space grid refinement process on the digital reproduction framework to construct a physical space grid system in each level and each area of ​​the digital reproduction framework; S25: Information mapping is performed on the physical space grid system of the digital reproduction framework according to the inherent structural features and the suspected defect features, so as to perform digital twin simulation of the detection data of the digital reproduction framework having the physical space grid system, so as to obtain a detection reproduction model corresponding to the optical detection channel.

[0064] Specifically, standard structural information is obtained from the design documents or drawings of the semiconductor chip, including the chip's geometric shape, hierarchical structure, material information, etc. These standard structural information serve as the basis for subsequent modeling to ensure the accuracy of the digital twin. According to the standard structural information of the semiconductor chip, the response of the semiconductor chip under different optical detection channels is encoded. Each optical detection channel will produce different responses to different structural features. The ideal structural features of each detection channel are extracted through encoding as the benchmark for subsequent modeling.

[0065] More specifically, based on optical imaging data, data decomposition techniques (such as principal component analysis (PCA), independent component analysis (ICA), convolutional neural network (CNN), etc.) are used to decompose the data into two major parts: inherent structural features: these features represent the normal structural information of the chip, including surface structure, size, material distribution, etc.; suspected defect features: these features represent possible defect areas, such as cracks, detachment, contamination, process defects, etc. Through this decomposition, normal areas and suspected defect areas can be clearly distinguished, providing a basis for subsequent defect detection and repair.

[0066] More specifically, based on the ideal structural feature information obtained, the semiconductor chip is divided into multiple levels, usually divided into: surface layer: the top layer of the chip, usually involving the most subtle surface defects, middle layer: between the surface layer and the deep layer, usually the main part of the circuit and functional structure, deep layer: the deepest part of the chip, involving core functions and electrical connections. Furthermore, the surface layer is divided into several detection areas, which are reasonably divided according to the actual detection needs to facilitate more refined detection and modeling.

[0067] More specifically, the layered information is integrated to construct a digital reproduction framework of the semiconductor chip, which includes all structural levels and detection areas of the chip and corresponds to the optical imaging data, providing a structural basis for subsequent digital twin modeling.

[0068] More specifically, the physical space grid of the digital reproduction framework is refined, that is, the entire framework is divided into small grid units, each grid unit represents a specific physical area in the chip. The gridding process enables the detection and reproduction model to accurately simulate the microstructural changes of the actual chip, especially in the defect area, through fine spatial division, providing higher-precision simulation results.

[0069] More specifically, the inherent structural features and suspected defect features obtained from the previous decomposition are mapped to the physical space grid system of the digital reproduction framework. This process corresponds the structural features in the optical imaging data to the grid cells and embeds the defect information into the corresponding grid positions of the digital reproduction framework. Based on the information mapping results, digital twin technology is used to perform digital twin simulation of the detection data of the digital reproduction framework. This step is to simulate the virtual detection of semiconductor chips through mathematical models and algorithms to predict defects or structural problems that may occur in actual detection. Based on the simulation results, a detection reproduction model for the corresponding optical detection channel is generated. This model can reproduce the actual detection process in a virtual environment and can simulate the response of different optical detection channels to the same defect. These reproduction models provide a theoretical basis and optimization solutions for subsequent actual detection.

[0070] It is understandable that by acquiring standard structural information, encoding ideal structural features and decomposing optical imaging data, a digital twin that is highly consistent with the actual semiconductor chip can be generated. The twin not only reflects the real structure of the chip, but can also virtually reproduce the detection effects of different optical detection channels, and decompose inherent structural features from suspected defect features, which helps to accurately identify and predict defect areas in the chip. Through information mapping and digital twin simulation, defects can be discovered in advance in a virtual environment, thereby reducing missed detections and false alarms in actual detection. By layering the chip and combining detection areas at different levels, hierarchical defect detection can be achieved, which is especially important for complex chips and can ensure that potential defects can be accurately detected in each layer and each area.

[0071] More specifically, the refinement of the physical space grid enables the digital reproduction framework to be accurate to the micron scale or even smaller, ensuring the high precision of the simulation results. This enables the digital twin to more realistically reproduce every detail of the chip, improving the credibility of the simulation. By generating multiple detection reproduction models, the performance of different optical detection channels can be analyzed, and the configuration of the optical detection channels can be optimized. Based on the results of the digital twin simulation, the optimal channel configuration scheme can be provided for actual detection, thereby improving detection efficiency and accuracy. The digital twin provides a real-time feedback framework, which can adjust the detection strategy according to the actual detection results to further improve the detection accuracy.

[0072] Preferably, the step of calculating the information confidence of each of the detection and reproduction models to convert the detection and reproduction models into corresponding detection feature matrices includes: S31: Retrieving corresponding optical detection adaptability information from a preliminary database according to the optical detection channel corresponding to the detection replication model, and performing optical detection adaptability analysis on the digital replication framework of the semiconductor chip according to the optical detection adaptability information to obtain an optical detection adaptability feature distribution of the digital replication framework of the semiconductor chip corresponding to the optical detection channel; S32: performing independent confidence calculation of each spatial grid in the physical spatial grid system of the detection and reproduction model based on the optical detection adaptation feature distribution to generate information independent confidence of each spatial grid in the detection and reproduction model; S33: Using the spatial grid corresponding to the suspected defect feature on the detection and reproduction model as a reference analysis point, and delineating a neighboring interval of a specified range of the detection and reproduction model starting from the reference analysis point to obtain a reference information set of the reference analysis point; S34: performing a defect probability analysis on the reference analysis point based on the information independence confidence of the reference information set and the spatial grid simulation content to obtain defect explanation information of the reference analysis point; wherein the defect explanation information includes several possible judgment conclusions and corresponding judgment probabilities; S35: performing structured multidimensional tensor decomposition on the detection and reproduction model according to the information independence confidence of each spatial grid at each location of the detection and reproduction model, so as to convert the simulation content of each spatial grid at each location into a corresponding multidimensional feature vector; S36: performing information supplementation processing on the multi-dimensional feature vector corresponding to the suspected defect feature according to the defect analysis information to obtain a detection feature matrix corresponding to the detection reproduction model.

[0073] Specifically, optical detection adaptability information corresponding to the optical detection channels is retrieved from a preparatory database. This information may include the sensitivity, response characteristics, and interference noise of each optical detection channel, and is used to evaluate the channel's responsiveness to various chip structural features. Based on this retrieved optical detection adaptability information, an optical detection adaptability analysis is performed on the semiconductor chip in the digital reproduction framework. This analysis assesses the responsiveness of each region (including the surface layer, intermediate layer, and deep layer) under different optical detection channels, resulting in a distribution of optical detection adaptability characteristics for the semiconductor chip under the corresponding optical detection channel. This distribution reflects the sensitivity and response characteristics of each region under that detection channel, providing a basis for subsequent confidence calculations.

[0074] More specifically, based on the distribution of optical detection adaptation features, the independent confidence of each grid in the physical space grid system of the digital reproduction framework is calculated. This process is carried out according to the adaptability of each grid under the optical detection channel, ensuring that the detection reliability of the area can be evaluated at each grid, and the information independence confidence of each grid is obtained. This value measures the detection effectiveness and accuracy of each area under the optical detection channel.

[0075] More specifically, a spatial grid corresponding to the suspected defect feature in the detection and reproduction model is selected as a benchmark analysis point, which represents the location of the potential defect. Based on the defect feature of this point, further analysis is performed. Starting from the benchmark analysis point, the adjacent interval within its specified range is circled to form a reference information set. This process is to ensure that the area adjacent to the benchmark analysis point in the spatial grid can also be used as a reference for defect analysis. Based on the information independence confidence of the reference information set where the benchmark analysis point is located and the spatial grid simulation content of the area, a defect probability analysis is performed. Through statistical methods or machine learning algorithms, it is calculated whether the point is a defect point and the probability of the defect occurring. The defect interpretation information is obtained, including several possible conclusions that the area is a defect (such as cracks, holes, foreign matter, etc.) and the corresponding judgment probabilities, providing an important basis for defect identification.

[0076] More specifically, based on the independent confidence of the information of each spatial grid, the detection reproduction model is subjected to structured multidimensional tensor decomposition. This step integrates the simulation content, confidence, defect information, etc. of different spatial grids to form a multidimensional tensor. Through tensor decomposition technology, complex spatial grid information can be converted into feature vectors at multiple levels, and multidimensional feature vectors can be extracted from the tensor decomposition results. The simulation content of each grid (such as structural features, defect type, confidence, etc.) will be converted into a multidimensional vector representation. These vectors provide a high-dimensional spatial representation that can more comprehensively characterize the detection characteristics of semiconductor chips.

[0077] More specifically, based on the defect analysis information, the multidimensional feature vectors corresponding to the suspected defect features are supplemented with information. This processing can fill in possible missing information or enhance its detection effect through data interpolation, feature fusion, deep learning and other technologies. Finally, by integrating and supplementing the multidimensional feature vectors of all spatial grids, a complete detection feature matrix is ​​obtained. This matrix contains the detection features of each spatial grid (such as defect type, occurrence probability, confidence level, etc.), providing a basis for subsequent defect analysis, diagnosis and decision-making.

[0078] It can be understood that by calculating the independent confidence of each spatial grid, the detection effectiveness of each area can be evaluated under different optical inspection channels. This refined confidence calculation ensures that the defect detection of each grid reflects its actual importance and reliability, thereby reducing false positives and missed negatives. Based on the analysis of the benchmark analysis point and its adjacent intervals, a comprehensive defect probability analysis can be performed on suspected defect points. This not only helps to determine whether a certain area is a defect point, but also provides the possibility of different defect types and their probability of occurrence, making defect identification more accurate. Through structured multidimensional tensor decomposition, complex spatial grid information can be converted into high-dimensional feature vectors. These feature vectors reflect the detection information of the chip in multiple dimensions. This method can better capture the detailed features of the chip and improve the detection effect. The resulting detection feature matrix provides a comprehensive and accurate representation for defect detection. These features not only contain structural information, but also include multiple factors such as defect type and occurrence probability, providing strong support for subsequent defect analysis and repair solution optimization. Information supplementation processing can effectively improve the integrity and accuracy of the multidimensional feature vector, making defect detection more comprehensive and reliable, especially in the case of missing data or large noise interference.

[0079] Preferably, the step of calculating cross-modal correlation of each detection feature matrix based on the optical detection channel corresponding to each detection reproduction model to generate correlation information between the detection reproduction models includes: S41: interactively analyzing the optical detection adaptation feature distribution of each optical detection channel to generate a theoretical mapping relationship of detection contents between each optical detection channel; S42: performing cross-modal information mapping relationship analysis on each of the detection feature matrices to obtain actual information mapping relationships of each multidimensional feature vector in each of the detection feature matrices; S43: Based on the theoretical mapping relationship of the detection content, the deviation fluctuation range of the actual information mapping relationship of each multidimensional feature vector in each detection feature matrix is ​​calculated to obtain the cross-modal correlation parameters of the multidimensional feature vectors at each location between each detection feature matrix, which are used together as the correlation information between each detection reproduction model.

[0080] Specifically, an interactive analysis is performed on the optical detection adaptation feature distribution of each optical detection channel. The purpose of this analysis is to identify the interactions between different optical detection channels and their mapping relationships. The adaptation feature distributions of different channels may have overlapping or complementary parts. Through interactive analysis, the theoretical mapping relationship of detection content between these channels can be obtained. This mapping relationship describes how different optical detection channels theoretically correspond to their respective detection contents, that is, the detection capabilities of each channel on the chip surface or inside, and its response characteristics to various features.

[0081] More specifically, we analyze the cross-modal information mapping relationships of the multidimensional feature vectors in each detection feature matrix. In this step, we analyze how feature vectors between different detection and reproduction models are mapped at the information level, exploring their similarities and differences. Each feature matrix contains multiple dimensions (such as spatial location, defect type, and defect probability). The goal of cross-modal mapping is to identify the actual information mapping relationships between each feature vector under different detection channels. This process provides a foundation for subsequent cross-modal correlation calculations.

[0082] More specifically, based on the theoretical mapping relationship of the detection content in step 1, the deviation fluctuation range of the actual information mapping relationship of the multi-dimensional eigenvectors in each detection feature matrix is ​​calculated. The goal of this calculation is to evaluate the influence range of different optical detection channels on each eigenvector and determine their fluctuation range and uncertainty in cross-modal conversion. The calculation of the deviation fluctuation range involves multi-dimensional data analysis, and statistical methods such as standard deviation and deviation are usually used to measure the variability between different detection channels during the conversion process to obtain the stability and accuracy of cross-modal information.

[0083] More specifically, the deviation fluctuation range is combined with the information mapping relationship to calculate the cross-modal correlation parameter. This process actually converts the fluctuation range of the previous step into a numerical parameter, which is used to describe the correlation between different detection and reproduction models. The cross-modal correlation parameter can reflect the similarities and differences between the detection and reproduction models at the level of multidimensional feature vectors, reflecting the performance differences between different models under the same or similar conditions.

[0084] More specifically, based on the aforementioned cross-modal correlation parameters, correlation information is generated between each detection and reproduction model. This information reflects the relationship between each detection and reproduction model in terms of detection content, defect judgment, etc., and provides a quantitative description, making information sharing and comparison between different models more reliable. The correlation information can be used in subsequent analysis, optimization, and repair solutions, providing a basis for consistency verification and cross-detection between models.

[0085] It can be understood that through the analysis of the interactivity between optical detection channels and the analysis of cross-modal information mapping relationships, the integration between different detection reproduction models can be effectively improved. Each detection reproduction model is no longer independent, but its characteristics and detection capabilities are organically linked through cross-modal correlation parameters. The deviation fluctuation range of the cross-modal correlation calculation provides a quantitative basis for the correlation between each detection model. Through this calculation, the consistency of different models in specific detection tasks can be measured, thereby reducing the error between different detection models and improving the overall detection reliability. The generated correlation information can help to more accurately identify and judge defects in chips. Cross-modal correlation analysis makes defect judgment not rely on a single detection model, but comprehensively considers the relationship between multiple models, thereby improving the accuracy of defect identification.

[0086] More specifically, different optical detection channels and reproduction models often focus on different defect types or characteristics. Cross-modal correlation analysis can provide a collaborative working framework for multimodal detection systems by revealing the relationship between them. This collaborative working mechanism can ensure the comprehensiveness and consistency of detection in multi-channel and multi-model scenarios. By calculating the cross-modal correlation of multi-dimensional feature vectors, richer feature inputs can be provided for subsequent data fusion and decision support, which is of great significance for high-precision defect prediction, anomaly detection and performance optimization. Since cross-modal correlation parameters can provide quantitative information about the differences and similarities between different models, they can eliminate inconsistencies between models during the detection process, making the final detection results more stable and reliable.

[0087] Preferably, the steps of performing abnormality marking processing on each of the detection feature matrices according to the correlation information, and performing defect perception on the semiconductor chip according to each of the abnormally marked detection feature matrices to generate defect detection information include: S51: performing an abnormality assessment on the cross-modal correlation parameters of the multidimensional feature vectors in the correlation information according to a preset correlation standard, and performing an abnormality marking process on each of the detection feature matrices based on the abnormality assessment result to generate a correlation abnormality mark on each of the detection feature matrices; S52: performing detection semantic expressions for specific locations of the semiconductor chip according to the multidimensional feature vectors in each detection feature matrix to generate a defect diagnosis semantic space for the semiconductor chip corresponding to each detection feature matrix, and assigning the correlation abnormality mark to the corresponding location in the defect diagnosis semantic space; S53: Overlapping the defect diagnosis semantic spaces to perform weighted information fusion on specific locations of the defect diagnosis semantic spaces to determine first defect information directly displayed on the semiconductor chip through the optical detection channel; S54: Simultaneously, based on the correlation anomaly marks in each of the defect diagnosis semantic spaces, comprehensively evaluate the information of potential defects of the semiconductor chip to determine second defect information on the semiconductor chip that is not directly displayed by the optical detection channel; S55: Combine the first defect information and the second defect information to generate defect detection information.

[0088] Specifically, the cross-modal correlation parameters in the correlation information are evaluated, and the evaluation criteria are usually based on the similarity, difference, volatility, etc. of the multi-dimensional feature vectors. In this evaluation process, any abnormal parameter that deviates from the normal range will be marked as a potential abnormal signal. Specifically, the evaluation process involves calculating the deviation and fluctuation range of each cross-modal parameter, and judging whether there is an abnormality by comparing it with the preset correlation standard. According to the abnormal evaluation results, the multi-dimensional feature vectors in each detection feature matrix are anomaly labeled. In the anomaly labeling process, the system will mark each feature vector with abnormal cross-modal correlation parameters with a correlation anomaly mark. The mark identifies the places where inconsistencies and excessive fluctuations occur in different detection channels or models, indicating that these feature points may be problematic places. Through anomaly evaluation based on cross-modal correlation parameters, abnormal signals in the detection process can be accurately identified, thereby accurately anomaly labeling each detection feature matrix. This process improves the accuracy of defect diagnosis and ensures that abnormal points can be effectively identified and located.

[0089] More specifically, a detection semantic expression is performed on the multi-dimensional feature vectors in each detection feature matrix. The detection semantic expression helps the system understand the specific information of each location on the chip surface or inside by mapping different feature vectors. Specifically, by mapping each feature vector to the actual location of the chip, a detection semantic expression is formed, and each detection feature matrix is ​​combined with its corresponding detection semantic expression to form a defect diagnosis semantic space. This semantic space provides defect perception at each location on the chip surface or inside, ensuring that the system can identify potential defects from information from multiple models and channels. The corresponding correlation anomaly mark will be assigned to the corresponding position in the defect diagnosis semantic space. In this way, the defect diagnosis semantic space not only includes the general information of various locations on the chip, but also combines the anomaly marks with the actual detection results to form a semantic space containing potential defects.

[0090] More specifically, all generated defect diagnosis semantic spaces are overlapped. During this process, the defect diagnosis semantic spaces generated by multiple different detection channels or models will be subjected to information weighted fusion, which means that the defect information of different channels at the same position will be weighted and integrated to fully utilize the advantages of different detection channels. Through weighted fusion, multiple detection results can be combined to determine which defects are directly visible on the chip, that is, the first defect information. The result after weighted fusion can determine the defect information directly displayed on the chip through the optical detection channel. These defect information are usually defects on the surface or inside of the chip that can be directly captured by current optical detection technology. The overlapping processing and information weighted fusion of the defect diagnosis semantic space enable the integration of multi-channel and multi-model detection information, maximize the advantages of different detection channels, and thus improve the comprehensiveness and accuracy of detection. This weighted fusion ensures the accuracy of the first defect information, while also helping to discover potential defect information that is not directly displayed.

[0091] More specifically, at the same time, the correlation anomaly marks in the generated defect diagnosis semantic space are used to conduct a comprehensive assessment of potential defects in the chip. These potential defects may be defect types that the current optical detection channel cannot directly display. The correlation anomaly marks serve as a supplementary information to help discover defects that may not be obvious in visual detection but actually exist. Through comprehensive evaluation, these potential second defect information can be inferred. Through the correlation anomaly marks, a comprehensive assessment of potential defects can be effectively performed, which means that even if some defects are not captured by the direct detection channel, the system can still raise possible problems through the potential risk identification mechanism and improve the depth perception capability of the detection system. The working principle of this potential risk identification mechanism is that the correlation displayed by various optical detection channels of qualified semiconductor chips should be within a certain range. When the coherence exceeds a certain range, there is a potential risk even if the defect is not directly detected.

[0092] More specifically, finally, the first defect information obtained through weighted fusion is combined with the second defect information obtained through comprehensive evaluation to generate the final defect detection information. This combination provides a comprehensive and detailed defect detection result, covering all types of defects on the chip surface and inside.

[0093] Reference Figure 2 As shown, in a second aspect, the present invention provides a chip defect detection device based on multimodal optical imaging, which is used to implement a chip defect detection method based on multimodal optical imaging as described in any one of the first aspects, including: an optical detection module, configured to perform optical detection on the semiconductor chip through a plurality of optical detection channels to obtain multimodal optical imaging data of the semiconductor chip; A digital reproduction module, configured to construct digital twins of semiconductor chips based on the multimodal optical imaging data to generate a plurality of detection reproduction models; An information conversion module, configured to calculate information confidence for each of the detection and reproduction models, so as to convert the detection and reproduction models into corresponding detection feature matrices; a correlation analysis module, configured to calculate cross-modal correlations of the detection feature matrices based on the optical detection channels corresponding to the detection replication models, so as to generate correlation information between the detection replication models; The defect perception module is used to perform abnormality marking processing on each of the detection feature matrices according to the correlation information, and to perform defect perception on the semiconductor chip according to each of the detection feature matrices that have been abnormally marked to generate defect detection information.

[0094] In this embodiment, for the specific implementation of each module in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0095] In a third aspect, the present invention provides a chip defect detection device based on multimodal optical imaging, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements a computer method described in any one of the first aspects.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A chip defect detection method based on multimodal optical imaging, characterized in that: include: performing optical inspection on the semiconductor chip through a plurality of optical inspection channels to obtain multimodal optical imaging data of the semiconductor chip; constructing digital twins of semiconductor chips respectively according to the multimodal optical imaging data to generate a plurality of detection replication models; Calculating information confidence for each of the detection and reproduction models to convert the detection and reproduction models into corresponding detection feature matrices; Calculating cross-modal correlations of the detection feature matrices based on the optical detection channels corresponding to the detection reproduction models to generate correlation information between the detection reproduction models; Anomaly marking processing is performed on each of the detection feature matrices according to the correlation information, and defect perception is performed on the semiconductor chip according to each of the detection feature matrices that have been anomaly marked to generate defect detection information.

2. The chip defect detection method based on multimodal optical imaging according to claim 1, characterized in that: The steps of optically inspecting a semiconductor chip through a plurality of optical inspection channels to obtain multimodal optical imaging data of the semiconductor chip include: The semiconductor chip to be inspected is placed at a designated inspection position, and each pre-set inspection function module is activated. By adjusting the inspection function parameters of each inspection function module, corresponding optical inspection channels are sequentially constructed; wherein the optical inspection channels include a bright field illumination inspection channel, a dark field illumination inspection channel, a polarized light illumination inspection channel, an infrared imaging inspection channel, a fluorescence imaging inspection channel, a confocal microscopy inspection channel, and an optical coherence tomography inspection channel; Performing optical detection on a semiconductor chip at a designated detection position through the constructed optical detection channel to obtain optical imaging data corresponding to the optical detection channel; Adjusting key detection parameters of the optical detection channel to be executed according to the optical imaging data to obtain optical imaging data of the optical detection channel to be executed in a subsequent round; The optical imaging data of each optical detection channel are combined to obtain multimodal optical imaging data of the semiconductor chip.

3. The chip defect detection method based on multimodal optical imaging according to claim 2, characterized in that: The step of adjusting key detection parameters of the optical detection channel to be executed according to the optical imaging data to obtain optical imaging data of the optical detection channel to be executed in the next round includes: Performing data dimensionality reduction processing on the optical imaging data to obtain a downsampled image, and performing lightweight convolution processing on the downsampled image through a pre-trained CNN model to generate a low-dimensional anomaly heat map; The optical detection channel corresponding to the low-dimensional anomaly heat map is used as the first detection channel, and the optical detection channel to be executed subsequently is used as the second detection channel. The optical detection knowledge graph is used to analyze the key detection parameter adjustment tendency of the second detection channel relative to the first detection channel based on the low-dimensional anomaly heat map to generate an expected detection mode for the second detection channel. A synergistic effect analysis is performed on the key detection parameters of the second detection channel relative to the first detection channel, so as to select key detection parameters with maximized information complementarity in the expected detection mode to configure the optical detection channel to be executed, so as to detect the semiconductor chip and obtain optical imaging data of the corresponding optical detection channel.

4. The chip defect detection method based on multimodal optical imaging according to claim 1, characterized in that: The steps of constructing digital twins of semiconductor chips according to the multimodal optical imaging data to generate several detection replication models include: Acquiring standard structural information of a semiconductor chip, and encoding information corresponding to various optical detection channels of the semiconductor chip according to the standard structural information, so as to obtain ideal structural feature information of the semiconductor chip corresponding to various optical detection channels; Decomposing each item of optical imaging data in the multimodal optical imaging data into inherent structural features and suspected defect features based on the ideal structural feature information, so as to decompose each item of optical imaging data in the multimodal optical imaging data into inherent structural features and suspected defect features; Dividing the semiconductor chip into a surface layer, an intermediate layer, and a deep layer based on the ideal structural feature information, and dividing the surface layer into a plurality of detection areas to construct a digital reproduction framework of the semiconductor chip; Performing a physical space grid refinement process on the digital reproduction framework to construct a physical space grid system in each level and each area of ​​the digital reproduction framework; Information mapping is performed on the physical space grid system of the digital reproduction framework according to the inherent structural characteristics and the suspected defect characteristics, so as to perform digital twin simulation of the detection data of the digital reproduction framework having the physical space grid system to obtain a detection reproduction model corresponding to the optical detection channel.

5. The chip defect detection method based on multimodal optical imaging according to claim 4, characterized in that: The step of calculating the information confidence of each detection and reproduction model to convert the detection and reproduction model into a corresponding detection feature matrix includes: Retrieving corresponding optical detection adaptability information from a preliminary database according to the optical detection channel corresponding to the detection replication model, and performing an optical detection adaptability analysis on the digital replication framework of the semiconductor chip according to the optical detection adaptability information to obtain an optical detection adaptability feature distribution of the digital replication framework of the semiconductor chip corresponding to the optical detection channel; Based on the optical detection adaptation feature distribution, independent confidence calculation is performed on each spatial grid of the physical spatial grid system of the detection and reproduction model to generate information independent confidence of each spatial grid of the detection and reproduction model; Taking the spatial grid corresponding to the suspected defect feature on the detection and reproduction model as a reference analysis point, and starting from the reference analysis point, delineating a neighboring interval of a specified range of the detection and reproduction model to obtain a reference information set of the reference analysis point; Performing a defect probability analysis on the benchmark analysis point based on the information independence confidence of the reference information set and the spatial grid simulation content to obtain defect interpretation information for the benchmark analysis point; wherein the defect interpretation information includes several possible determination conclusions and corresponding determination probabilities; Performing structured multidimensional tensor decomposition on the detection and reproduction model according to the information independence confidence of each spatial grid at each location of the detection and reproduction model, so as to convert the simulation content of each spatial grid at each location into a corresponding multidimensional feature vector; The multidimensional feature vector corresponding to the suspected defect feature is supplemented with information according to the defect explanation information to obtain a detection feature matrix corresponding to the detection reproduction model.

6. The chip defect detection method based on multimodal optical imaging according to claim 1, characterized in that: The step of calculating cross-modal correlation of each detection feature matrix based on the optical detection channel corresponding to each detection reproduction model to generate correlation information between each detection reproduction model includes: Interactively analyzing the optical detection adaptation feature distribution of each optical detection channel to generate a theoretical mapping relationship of detection content between each optical detection channel; Performing cross-modal information mapping relationship analysis on each of the detection feature matrices to obtain actual information mapping relationships of each multidimensional feature vector in each of the detection feature matrices; Based on the theoretical mapping relationship of the detection content, the deviation fluctuation range of the actual information mapping relationship of each multidimensional feature vector in each detection feature matrix is ​​calculated to obtain the cross-modal correlation parameters of the multidimensional feature vectors at each location between each detection feature matrix, which are used together as the correlation information between each detection reproduction model.

7. The chip defect detection method based on multimodal optical imaging according to claim 6, characterized in that: The steps of performing abnormality marking processing on each of the detection feature matrices according to the correlation information, and performing defect perception on the semiconductor chip according to each of the abnormally marked detection feature matrices to generate defect detection information include: Performing an abnormality evaluation on the cross-modal correlation parameters of the multidimensional feature vectors at each location in the correlation information according to a preset correlation standard, and performing an abnormality labeling process on each of the detection feature matrices based on the abnormality evaluation result to generate a correlation abnormality mark on each of the detection feature matrices; Performing semantic expressions of detection at specific locations of the semiconductor chip according to the multidimensional feature vectors at each location in each detection feature matrix to generate a defect diagnosis semantic space of the semiconductor chip corresponding to each detection feature matrix, and assigning the correlation anomaly mark to the corresponding location in the defect diagnosis semantic space; Overlapping the defect diagnosis semantic spaces to perform weighted information fusion on specific locations of the defect diagnosis semantic spaces to determine first defect information directly displayed on the semiconductor chip through the optical detection channel; At the same time, based on the correlation anomaly marks possessed by each of the defect diagnosis semantic spaces, a comprehensive evaluation of information on potential defects of the semiconductor chip is performed to determine second defect information on the semiconductor chip that is not directly displayed by the optical detection channel; The first defect information and the second defect information are combined to generate defect detection information.

8. A chip defect detection device based on multimodal optical imaging, characterized in that: A chip defect detection method based on multimodal optical imaging for implementing any one of claims 1 to 7, comprising: an optical detection module, configured to perform optical detection on the semiconductor chip through a plurality of optical detection channels to obtain multimodal optical imaging data of the semiconductor chip; A digital reproduction module, configured to construct digital twins of semiconductor chips based on the multimodal optical imaging data to generate a plurality of detection reproduction models; An information conversion module, configured to calculate information confidence for each of the detection and reproduction models, so as to convert the detection and reproduction models into corresponding detection feature matrices; a correlation analysis module, configured to calculate cross-modal correlations of the detection feature matrices based on the optical detection channels corresponding to the detection replication models, so as to generate correlation information between the detection replication models; The defect perception module is used to perform abnormality marking processing on each of the detection feature matrices according to the correlation information, and to perform defect perception on the semiconductor chip according to each of the detection feature matrices that have been abnormally marked to generate defect detection information.

9. A chip defect detection device based on multimodal optical imaging, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the computer method according to any one of claims 1 to 7 is implemented.

Images