Method and system for automatically detecting defects of semiconductor chip based on hyperspectral imaging

Abstract

The invention provides a semiconductor chip defect automatic detection method and system based on hyperspectral imaging, and relates to the technical field of chip detection, and the method comprises the steps: collecting a chip hyperspectral image, carrying out the image enhancement and illumination non-uniformity correction through an improved neuromorphic adaptive processing network and a dual-path diffusion model, and carrying out the detection of the hyperspectral image. Extracting multi-level features by using a dynamic mixed feature enhancement network based on neural architecture search, fusing the features through a cross-layer attention mechanism, generating an initial defect detection result by using a multi-branch expert decision-making module and an uncertainty dynamic integration strategy, constructing a space-time knowledge inference graph, and determining a defect detection result; and performing reliability evaluation and optimization on the initial defect detection result in combination with the form similarity, the spatial position relationship and the process parameter influence model to obtain a final defect detection result.

Description

Technical Field

[0001] The present invention relates to the field of chip detection technology, and in particular to a method and system for automatically detecting semiconductor chip defects based on hyperspectral imaging. Background Art

[0002] The manufacturing process of semiconductor chips is complex, and various defects are prone to occur during the production process, such as scratches, cracks, and contamination. These defects will seriously affect the performance and reliability of the chip. Traditional defect detection methods mainly rely on manual visual inspection, which is inefficient, costly, and easily affected by subjective factors, resulting in inconsistent detection results.

[0003] With the development of deep learning technology, image recognition and target detection technologies based on deep learning have achieved remarkable results in various fields. Applying deep learning technology to semiconductor chip defect detection can achieve automated, high-precision, and high-efficiency defect detection. However, existing semiconductor chip defect detection methods based on deep learning still have problems such as difficulty in effectively extracting and distinguishing different types of defect features, resulting in low detection accuracy and ignoring the influence of process parameters.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for automatic detection of semiconductor chip defects based on hyperspectral imaging, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for automatically detecting semiconductor chip defects based on hyperspectral imaging, comprising:

[0007] Collecting hyperspectral image data of a semiconductor chip to be inspected, adding the hyperspectral image data to an improved neuromorphic adaptive processing network and calculating a noise distribution density map and a texture complexity map of a local area, constructing an adaptive enhancement matrix based on the noise distribution density map and the texture complexity map, calculating wavelet decomposition coefficients of different scales based on the adaptive enhancement matrix, performing nonlinear mapping on the wavelet decomposition coefficients to obtain enhancement coefficients, and reconstructing the image data by inverse wavelet transform to obtain preliminary enhanced image data; inputting the preliminary enhanced image data into a dual-path diffusion model, extracting high-frequency detail features through a first path, extracting low-frequency structural features through a second path, and adaptively fusing the high-frequency detail features and the low-frequency structural features to obtain a compensation feature map, and performing illumination non-uniformity correction on the preliminary enhanced image data based on the compensation feature map to obtain corrected image data;

[0008] A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, the corrected image data is added to the dynamic hybrid feature enhancement network and a multi-level feature pyramid is generated, a cross-layer attention operation is performed on the feature maps in adjacent layers in the multi-level feature pyramid, inter-layer association weights are generated, and the feature maps of each level are reconstructed and enhanced, a fused feature map is generated and added to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes a plurality of sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and the independent detection results are fused in combination with a dynamic integration strategy based on uncertainty to obtain an initial defect detection result;

[0009] A spatiotemporal knowledge reasoning graph is established based on the initial defect detection results, the initial defect detection results are set as graph nodes, the morphological similarity and spatial position relationship between each initial defect detection result are calculated and the edge connection strength is constructed, the initial defect detection results are hierarchically clustered based on the edge connection strength to generate similar defect groups, a graph attention network operation is performed on the spatiotemporal knowledge reasoning graph, a defect-related feature vector is generated and a time series correlation analysis is performed between the defect-related feature vector and a process parameter sequence, a process parameter influence model is constructed, and the reliability of the initial defect detection results is evaluated and optimized based on the process parameter influence model and the similar defect groups to obtain a final defect detection result.

[0010] In an optional embodiment,

[0011] The hyperspectral image data of the semiconductor chip to be detected is collected, the hyperspectral image data is added to the improved neuromorphic adaptive processing network and the noise distribution density map and the texture complexity map of the local area are calculated, an adaptive enhancement matrix is ​​constructed based on the noise distribution density map and the texture complexity map, wavelet decomposition coefficients of different scales are calculated based on the adaptive enhancement matrix, and the enhancement coefficients obtained by nonlinear mapping of the wavelet decomposition coefficients include:

[0012] Collecting hyperspectral image data of the semiconductor chip to be inspected, wherein the hyperspectral image data includes image data of visible light band, near infrared band and short-wave infrared band;

[0013] Inputting the hyperspectral image data into an improved neuromorphic adaptive processing network, calculating the local statistical features of the hyperspectral image data at different scales, extracting noise distribution feature vectors and texture feature vectors, constructing a multi-layer neural network based on the noise distribution feature vectors and calculating a noise distribution density map, constructing a convolutional neural network based on the texture feature vectors and calculating a texture complexity map, inputting the noise distribution density map and the texture complexity map into an attention mechanism module, and generating an attention weight map;

[0014] According to the attention weight map, the noise distribution density map and the texture complexity map are feature reconstructed, and the reconstructed feature maps are fused to construct an adaptive enhancement matrix. Based on the adaptive enhancement matrix, different bands of the hyperspectral image data are adaptively enhanced respectively, and a multi-scale directional filter group and a wavelet decomposition unit are constructed. The enhanced hyperspectral image data is simultaneously subjected to directional decomposition, scale decomposition and wavelet decomposition, and a plurality of directional sub-bands and scale sub-bands are obtained through the multi-scale directional filter group, and an approximate coefficient sub-band and a detail coefficient sub-band are obtained through the wavelet decomposition unit.

[0015] Adaptive threshold segmentation is performed on the direction subband, the scale subband, the approximate coefficient subband and the detail coefficient subband to extract significant feature coefficients, adaptive weight modulation is performed on the significant feature coefficients and the adaptive enhancement matrix, a nonlinear activation function operation is performed on the modulated coefficients to generate enhancement coefficients.

[0016] In an optional embodiment,

[0017] The image data is initially enhanced by reconstructing through inverse wavelet transform; the image data is input into a dual-path diffusion model, high-frequency detail features are extracted through a first path, low-frequency structural features are extracted through a second path, and the high-frequency detail features and the low-frequency structural features are adaptively fused to obtain a compensation feature map; and the initial enhanced image data is corrected for illumination non-uniformity based on the compensation feature map to obtain corrected image data, including:

[0018] Constructing a deep residual reconstruction network, inputting the enhancement coefficient into the deep residual reconstruction network, and obtaining preliminary enhanced image data through residual learning and skip connection structure reconstruction;

[0019] Constructing a dual-branch attention diffusion network, the dual-branch attention diffusion network includes a spatial attention branch and a channel attention branch, inputting the preliminary enhanced image data into the dual-branch attention diffusion network, extracting a high-frequency detail feature map through the spatial attention branch, and extracting a low-frequency structure feature map through the channel attention branch;

[0020] Constructing a hierarchical feature pyramid network, the hierarchical feature pyramid network includes a feature mapping module, a feature reconstruction module and a feature optimization module, inputting the high-frequency detail feature map and the low-frequency structure feature map into the feature mapping module, generating a feature map group of multiple scales, inputting the feature map group into the feature reconstruction module, calculating the similarity matrix between features of different scales in combination with a self-attention mechanism and reconstructing features based on the similarity matrix, inputting the reconstructed feature map into the feature optimization module, iteratively optimizing the features through a recursive neural network and a feedback mechanism, and obtaining an optimized compensation feature map;

[0021] An adaptive illumination compensation network is constructed, wherein the adaptive illumination compensation network comprises an illumination analysis module, a reflectance analysis module and an illumination balancing module, the compensation feature map is input into the illumination analysis module, an image illumination component and an illumination distribution map are estimated through a multi-scale convolutional neural network, the compensation feature map is input into the reflectance analysis module, an image reflectance component and a texture detail map are restored through a deep separable convolutional network, the illumination distribution map and the texture detail map are input into the illumination balancing module, illumination non-uniformity correction is performed on the initially enhanced image data through an adaptive exposure compensation algorithm and a local contrast enhancement algorithm, and the corrected image data is output.

[0022] In an optional embodiment,

[0023] A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, the corrected image data is added to the dynamic hybrid feature enhancement network and a multi-level feature pyramid is generated, a cross-layer attention operation is performed on the feature maps in adjacent layers in the multi-level feature pyramid, inter-layer association weights are generated, and the feature maps of each level are reconstructed and enhanced, and a fused feature map is generated and added to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes a plurality of sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and the independent detection results are fused in combination with a dynamic integration strategy based on uncertainty, and the initial defect detection results include:

[0024] A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, wherein the neural architecture search method selects and combines a variety of convolutional layers, pooling layers, and activation function components from a predefined network structure space through a search algorithm, generates a candidate network, evaluates the performance of the candidate network, and selects the network architecture with the best performance as the structure of the dynamic hybrid feature enhancement network;

[0025] Inputting the modified image data into the dynamic hybrid feature enhancement network, performing multiple downsampling operations through the encoder module in the dynamic hybrid feature enhancement network, converting the modified image data into low-resolution features, performing multiple upsampling operations through the decoder module in the dynamic hybrid feature enhancement network, restoring the low-resolution features to original resolution features, extracting multiple feature maps of different scales at different levels of the encoder module and the decoder module, and constructing a multi-level feature pyramid;

[0026] Performing a cross-layer attention operation on the feature maps of adjacent layers in the multi-level feature pyramid, mapping the modified feature maps of adjacent layers to the same number of channels through a convolution operation, calculating a query matrix and a key matrix of the modified feature maps in the adjacent layers, generating an attention weight matrix based on the query matrix and the key matrix, reconstructing the modified feature maps based on the attention weight matrix, performing the cross-layer attention operation and feature reconstruction operation on the modified feature maps in each layer of the multi-level feature pyramid, and generating a fused pyramid feature map;

[0027] Constructing a multi-branch expert decision module, the multi-branch expert decision module includes a plurality of independent sub-networks, each of the sub-networks adopts an encoder-decoder structure and is provided with a classifier, the fused pyramid feature map is input into the multi-branch expert decision module, different types of defects are detected respectively through the plurality of sub-networks, and each sub-network outputs a corresponding independent detection result;

[0028] The confidences corresponding to the independent detection results output by the multiple sub-networks are obtained, and based on the confidences, the independent detection results are weighted averaged and fused through a dynamic integration strategy to obtain an initial defect detection result.

[0029] In an optional embodiment,

[0030] The cross-layer attention operation is performed on the feature maps of adjacent layers in the multi-level feature pyramid as shown in the following formula:

[0031] ;

[0032] Among them, A represents the calculated attention weight matrix element, represents the kth element of the query vector at the mth position in the i-th layer feature map, Represents the weight matrix of the query matrix of the i-th layer, transforming the query vector from d h Dimension conversion to d l Dimensions, represents the kth element of the key vector at the nth position in the jth layer feature map, represents the weight matrix of the key matrix of the i-th layer, converting the key vector from dh Dimension conversion to d l Dimension, d h represents the original dimensions of the query vector and key vector, d l represents the dimension of the query vector and key vector after linear transformation, h represents the number of heads in the multi-head attention mechanism, represents the attention weight between the mth position in the i-th layer feature map and the nth position in the j-th layer feature map in the g-th attention head, Represents the weight matrix of the output matrix, O is used to indicate the output.

[0033] In an optional embodiment,

[0034] A spatiotemporal knowledge reasoning graph is established based on the initial defect detection results, the initial defect detection results are set as graph nodes, the morphological similarity and spatial position relationship between each initial defect detection result are calculated and the edge connection strength is constructed, the initial defect detection results are hierarchically clustered based on the edge connection strength to generate similar defect groups, a graph attention network operation is performed on the spatiotemporal knowledge reasoning graph, a defect-related feature vector is generated and a time series correlation analysis is performed between the defect-related feature vector and the process parameter sequence, a process parameter influence model is constructed, and the reliability of the initial defect detection results is evaluated and optimized based on the process parameter influence model and the similar defect groups, and the final defect detection results include:

[0035] Acquire an initial defect detection result of the chip image, wherein the initial defect detection result includes morphological features and spatial position coordinates of each defect, wherein the morphological features include length, width and area, and the spatial position coordinates include the row number and column number of the defect center point;

[0036] Each defect in the initial defect detection result is set as a graph node, and a spatiotemporal knowledge reasoning graph is established, each of the graph nodes contains the morphological features and spatial position coordinates of the corresponding defect. For each pair of graph nodes in the spatiotemporal knowledge reasoning graph, the morphological similarity is obtained by calculating the cosine similarity between the morphological features of different graph nodes, and the spatial position distance is obtained by calculating the Euclidean distance between the spatial position coordinates of different graph nodes. The edge connection strength is constructed based on the morphological similarity and the spatial position distance. If the morphological similarity between the two graph nodes is greater than a preset similarity threshold and the spatial position distance is less than a preset distance threshold, the edge connection strength is calculated according to the spatial position distance;

[0037] Based on the edge connection strength, clustering the graph nodes by a bottom-up hierarchical clustering method, initially setting each graph node as an independent cluster, iteratively merging two clusters with the largest edge connection strength, and repeating the merging until the edge connection strength between all clusters is less than a preset connection strength threshold, thereby generating multiple similar defect groups;

[0038] Constructing a graph attention network, the graph attention network includes an attention layer and a feature propagation layer, for the spatiotemporal knowledge reasoning graph, calculating the attention weight between each graph node and the adjacent node corresponding to the current graph node through the attention layer, the attention weight is determined based on the similarity of the node features, based on the attention weight, updating the features of the graph nodes through the feature propagation layer, so that the features of each graph node aggregate the weighted information of the corresponding adjacent nodes, and generating a node feature matrix;

[0039] Extract the defect-associated feature vector corresponding to each of the similar defect groups in the node feature matrix, obtain the process parameter sequence within the time period corresponding to the similar defect group, the process parameter sequence includes temperature parameters and pressure parameters, calculate the cross-correlation coefficient between the defect-associated feature vector and the process parameter sequence by the Granger causality test method, and at the same time, build a process parameter influence model according to the cross-correlation coefficient, input each defect in the initial defect detection result into the process parameter influence model, determine the corresponding similar defect group based on the morphological features in the initial defect detection result, judge whether the current defect is caused by process abnormality according to the prediction result output by the process parameter influence model, mark the defect determined to be caused by process abnormality as a reliable defect, filter out defects not caused by process abnormality, and obtain the final defect detection result.

[0040] In an optional embodiment,

[0041] Calculating the cross-correlation coefficient between the defect-associated feature vector and the process parameter sequence by the Granger causality test method includes:

[0042] Obtaining a process parameter sequence and a defect-related feature vector in a chip manufacturing process, and dividing the process parameter sequence into a training data set and a test data set in chronological order, wherein the training data set is used to construct a prediction model, and the test data set is used to evaluate the performance of the prediction model;

[0043] A first prediction model is constructed based on the training data set, and the historical data of the process parameter sequence within a preset time window is used to predict the process parameter value at the current moment; a second prediction model is constructed based on the training data set, and the historical data of the process parameter sequence and the defect-associated feature vector within the preset time window are used to predict the process parameter value at the current moment;

[0044] Input the test data set into the first prediction model and the second prediction model, calculate the residual sum of squares of the two prediction models by summing the squares of the differences between the predicted values ​​and the actual values, and construct a Granger causality test statistic, wherein the Granger causality test statistic is calculated based on the residual sum of squares of the first prediction model, the residual sum of squares of the second prediction model, the number of parameters of the first prediction model, the number of parameters of the second prediction model, and the number of samples in the test data set;

[0045] A critical value is determined based on a preset significance level. When the Granger causality test statistic is greater than the critical value, the defect-associated feature vector is determined to be the Granger cause of the process parameter sequence. The cross-correlation coefficients between each feature in the defect-associated feature vector and the process parameter sequence at different time lags are calculated based on the covariance and variance of the defect-associated feature vector and the process parameter sequence.

[0046] A second aspect of an embodiment of the present invention provides a semiconductor chip defect automatic detection system based on hyperspectral imaging, comprising:

[0047] The first unit is used to collect hyperspectral image data of a semiconductor chip to be detected, add the hyperspectral image data to an improved neuromorphic adaptive processing network and calculate a noise distribution density map and a texture complexity map of a local area, construct an adaptive enhancement matrix based on the noise distribution density map and the texture complexity map, calculate wavelet decomposition coefficients of different scales based on the adaptive enhancement matrix, perform nonlinear mapping on the wavelet decomposition coefficients to obtain enhancement coefficients, and obtain preliminary enhanced image data through inverse wavelet transformation reconstruction; input the preliminary enhanced image data into a dual-path diffusion model, extract high-frequency detail features through a first path, extract low-frequency structural features through a second path, and adaptively fuse the high-frequency detail features and the low-frequency structural features to obtain a compensation feature map, perform illumination non-uniformity correction on the preliminary enhanced image data based on the compensation feature map to obtain corrected image data;

[0048] The second unit is used to construct a dynamic hybrid feature enhancement network based on a neural architecture search method, add the corrected image data to the dynamic hybrid feature enhancement network and generate a multi-level feature pyramid, perform cross-layer attention operations on feature maps in adjacent layers in the multi-level feature pyramid, generate inter-layer association weights, reconstruct and enhance feature maps at each level, generate fused feature maps and add them to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes multiple sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and combines the independent detection results with a dynamic integration strategy based on uncertainty to obtain an initial defect detection result;

[0049] The third unit is used to establish a spatiotemporal knowledge reasoning graph based on the initial defect detection results, set the initial defect detection results as graph nodes, calculate the morphological similarity and spatial position relationship between each initial defect detection result and construct the edge connection strength, hierarchically cluster the initial defect detection results based on the edge connection strength to generate similar defect groups, perform graph attention network operations on the spatiotemporal knowledge reasoning graph, generate defect-related feature vectors and perform time series correlation analysis on the defect-related feature vectors and process parameter sequences, construct a process parameter influence model, and perform reliability evaluation and optimization on the initial defect detection results based on the process parameter influence model and the similar defect groups to obtain final defect detection results.

[0050] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including: a processor;

[0051] a memory for storing processor-executable instructions;

[0052] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0053] According to a fourth aspect of the embodiments of the present invention,

[0054] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0055] In the present invention, the wavelet decomposition coefficients are optimized by adaptive enhancement matrix, and the illumination non-uniformity is corrected in combination with the dual-path diffusion model, which effectively enhances the image quality and highlights the defect features, thereby improving the accuracy of defect detection. The dynamic hybrid feature enhancement network constructed based on neural architecture search, combined with the cross-layer attention mechanism and the multi-branch expert decision module, can effectively extract and fuse multi-level features to achieve accurate identification of different types of defects. The spatiotemporal knowledge reasoning graph is used to evaluate and optimize the reliability of the initial detection results. By analyzing the morphological similarity, spatial position relationship and temporal correlation between defects and process parameters, the false detection rate and missed detection rate are effectively reduced, and the detection accuracy is improved. The reliability of the test results, hyperspectral imaging can obtain spatial and spectral information at the same time, each pixel contains a continuous spectral curve, and can provide richer spectral feature information than traditional RGB images. Different types of defects have unique spectral characteristics in specific bands, and the defects can be accurately identified and classified through spectral characteristics. By using the images obtained by hyperspectral imaging, various surface defects such as scratches, cracks, pollution, oxidation, etc. can be identified at the same time. It can identify defects of different depths and sizes and provide quantitative information of defects. Hyperspectral data has high-dimensional characteristics and is insensitive to interference factors such as changes in ambient light and surface reflection. The interference of irrelevant information can be removed through band selection to improve detection reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the process of the automatic detection method of semiconductor chip defects based on hyperspectral imaging according to an embodiment of the present invention;

[0057] Figure 2 It is a schematic diagram of the structure of a semiconductor chip defect automatic detection system based on hyperspectral imaging according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0060] Figure 1 FIG. 1 is a flow chart of a method for automatically detecting semiconductor chip defects based on hyperspectral imaging according to an embodiment of the present invention. Figure 1As shown, the method includes:

[0061] S1. Collecting hyperspectral image data of a semiconductor chip to be inspected, adding the hyperspectral image data to an improved neuromorphic adaptive processing network and calculating a noise distribution density map and a texture complexity map of a local area, constructing an adaptive enhancement matrix based on the noise distribution density map and the texture complexity map, calculating wavelet decomposition coefficients of different scales based on the adaptive enhancement matrix, performing nonlinear mapping on the wavelet decomposition coefficients to obtain enhancement coefficients, and reconstructing the image data by inverse wavelet transform to obtain preliminary enhanced image data; inputting the preliminary enhanced image data into a dual-path diffusion model, extracting high-frequency detail features through a first path, extracting low-frequency structural features through a second path, and adaptively fusing the high-frequency detail features and the low-frequency structural features to obtain a compensation feature map, performing illumination non-uniformity correction on the preliminary enhanced image data based on the compensation feature map to obtain corrected image data;

[0062] The neuromorphic adaptive processing network is a network structure based on a neuromorphic computing model, which aims to imitate the structure and processing mechanism of a biological nervous system, and adapts to different input data characteristics by adaptively adjusting network parameters, thereby achieving effective processing of noise and signals. The noise distribution density map is an image that describes the distribution of noise in a signal or image, and is usually used to represent the noise intensity distribution of different regions or pixels. The texture complexity map is an image that reflects the texture complexity and is generated by analyzing the texture features of different regions in an image. The inverse wavelet transform reconstruction is a signal or image reconstruction method, which restores the data processed in the wavelet domain to the original time domain or space domain by inversely executing the wavelet transform process. The dual-path diffusion model is a mathematical model for image processing or signal transmission, and is widely used in image denoising, diffusion, smoothing, etc. The non-uniformity correction is a technology used in signal processing and image processing, which aims to correct the non-uniformity of signals or images caused by factors such as environment and equipment.

[0063] In an optional embodiment,

[0064] The hyperspectral image data of the semiconductor chip to be detected is collected, the hyperspectral image data is added to the improved neuromorphic adaptive processing network and the noise distribution density map and the texture complexity map of the local area are calculated, an adaptive enhancement matrix is ​​constructed based on the noise distribution density map and the texture complexity map, wavelet decomposition coefficients of different scales are calculated based on the adaptive enhancement matrix, and the enhancement coefficients obtained by nonlinear mapping of the wavelet decomposition coefficients include:

[0065] Collecting hyperspectral image data of the semiconductor chip to be inspected, wherein the hyperspectral image data includes image data of visible light band, near infrared band and short-wave infrared band;

[0066] Inputting the hyperspectral image data into an improved neuromorphic adaptive processing network, calculating the local statistical features of the hyperspectral image data at different scales, extracting noise distribution feature vectors and texture feature vectors, constructing a multi-layer neural network based on the noise distribution feature vectors and calculating a noise distribution density map, constructing a convolutional neural network based on the texture feature vectors and calculating a texture complexity map, inputting the noise distribution density map and the texture complexity map into an attention mechanism module, and generating an attention weight map;

[0067] According to the attention weight map, the noise distribution density map and the texture complexity map are feature reconstructed, and the reconstructed feature maps are fused to construct an adaptive enhancement matrix. Based on the adaptive enhancement matrix, different bands of the hyperspectral image data are adaptively enhanced respectively, and a multi-scale directional filter group and a wavelet decomposition unit are constructed. The enhanced hyperspectral image data is simultaneously subjected to directional decomposition, scale decomposition and wavelet decomposition, and a plurality of directional sub-bands and scale sub-bands are obtained through the multi-scale directional filter group, and an approximate coefficient sub-band and a detail coefficient sub-band are obtained through the wavelet decomposition unit.

[0068] Adaptive threshold segmentation is performed on the direction subband, the scale subband, the approximate coefficient subband and the detail coefficient subband to extract significant feature coefficients, adaptive weight modulation is performed on the significant feature coefficients and the adaptive enhancement matrix, a nonlinear activation function operation is performed on the modulated coefficients to generate enhancement coefficients.

[0069] The multi-scale directional filter group is a group of filters that processes input data at multiple scales and directions. The wavelet decomposition unit is a module for performing wavelet transform on a signal or image. The significant feature coefficient refers to a value or coefficient with significant features extracted from the input data during image processing or signal processing.

[0070] Collect hyperspectral image data of the semiconductor chip to be inspected. Use a hyperspectral camera equipped with a filter wheel to obtain image data of the chip in different bands. These bands cover the visible light band (e.g. 400-700nm, for example, collect red, green and blue images), the near infrared band (e.g. 700-1000nm, for example, collect images of 750nm, 850nm and 950nm), and the short-wave infrared band (e.g. 1000-2500nm, for example, collect images of 1200nm, 1600nm and 2200nm). The image data of each band is saved as a grayscale image, and finally a hyperspectral image data set is formed. For example, for one chip, we collected images of three visible light bands of red, green and blue, three near infrared bands and three short-wave infrared bands, a total of nine bands of image data.

[0071] The hyperspectral image data is input into the improved neuromorphic adaptive processing network. The network structure includes multiple modules for extracting local statistical features at different scales, performing multi-scale Gaussian filtering on the input hyperspectral image data, such as filtering using Gaussian filters with standard deviations of 1, 2, and 4, to obtain smooth images at different scales. At each scale, the statistical features within each pixel neighborhood (such as 3x3, 5x5, and 7x7 neighborhoods) are calculated, such as mean, variance, gradient, etc. These statistical features constitute noise distribution feature vectors and texture feature vectors. A multi-layer neural network is constructed based on the noise distribution feature vector, such as a multi-layer perceptron containing two hidden layers, and the pattern of noise distribution is learned through training, and a noise distribution density map is output. A convolutional neural network is constructed based on the texture feature vector, such as a convolutional neural network containing three convolutional layers and two pooling layers, and the pattern of texture is learned through training, and a texture complexity map is output. The noise distribution density map and the texture complexity map are input into the attention mechanism module, such as using a spatial attention mechanism, to calculate the weight of each position and generate an attention weight map. For example, if the noise density of a pixel is high and the texture is complex, the value of the pixel on the attention weight map will be high.

[0072] The noise distribution density map and the texture complexity map are feature reconstructed according to the attention weight map. The attention weight map is multiplied with the noise distribution density map and the texture complexity map to obtain a weighted feature map, and the weighted noise distribution density map and the texture complexity map are fused, for example, they are added in a certain proportion to construct an adaptive enhancement matrix. For example, the weighted noise distribution density map and the texture complexity map are multiplied by 0.6 and 0.4 respectively and then added to obtain the final adaptive enhancement matrix. Based on the adaptive enhancement matrix, different bands of the hyperspectral image data are adaptively enhanced. The adaptive enhancement matrix is ​​multiplied with the image data of each band to obtain enhanced image data.

[0073] Construct a multi-scale directional filter bank and a wavelet decomposition unit. The multi-scale directional filter bank contains filters of multiple directions and scales, such as Gabor filters, which are used to extract texture information of different directions and scales. The wavelet decomposition unit decomposes the image using wavelet transform to obtain approximate coefficient subbands and detail coefficient subbands. For example, a three-level decomposition is performed using Haar wavelet. Adaptive threshold segmentation is performed on the directional subband, scale subband, approximate coefficient subband, and detail coefficient subband. According to the statistical characteristics of each subband, such as the mean and standard deviation, the threshold is adaptively determined, and the coefficients less than the threshold are set to zero, and the coefficients greater than the threshold are retained, thereby extracting significant feature coefficients. For example, for a certain subband, its mean and standard deviation are calculated, and the threshold is set to the mean plus twice the standard deviation.

[0074] Adaptively weight modulate the significant feature coefficients with the adaptive enhancement matrix. Multiply the significant feature coefficients with the elements at the corresponding positions of the adaptive enhancement matrix to obtain the modulated coefficients. Perform a nonlinear activation function operation on the modulated coefficients, such as a ReLU function or a Sigmoid function, to generate enhancement coefficients. For example, use a ReLU function to perform nonlinear activation on the modulated coefficients, set values ​​less than zero to zero, and keep values ​​greater than zero unchanged.

[0075] In this embodiment, through hyperspectral data fusion and adaptive enhancement, the characteristics of semiconductor chip defects are effectively highlighted, and the accuracy and reliability of defect detection are improved. Through the construction of noise distribution density map and texture complexity map, the interference of background noise and irrelevant texture can be effectively suppressed, and the robustness of defect detection is improved. Through multi-scale directional filtering and wavelet decomposition, rich multi-scale and multi-directional features are extracted, and the expression ability of features is enhanced through nonlinear mapping, which is conducive to more accurate identification and positioning of defects.

[0076] In an optional embodiment,

[0077] The image data is initially enhanced by reconstructing through inverse wavelet transform; the image data is input into a dual-path diffusion model, high-frequency detail features are extracted through a first path, low-frequency structural features are extracted through a second path, and the high-frequency detail features and the low-frequency structural features are adaptively fused to obtain a compensation feature map; and the initial enhanced image data is corrected for illumination non-uniformity based on the compensation feature map to obtain corrected image data, including:

[0078] Constructing a deep residual reconstruction network, inputting the enhancement coefficient into the deep residual reconstruction network, and obtaining preliminary enhanced image data through residual learning and skip connection structure reconstruction;

[0079] Constructing a dual-branch attention diffusion network, the dual-branch attention diffusion network includes a spatial attention branch and a channel attention branch, inputting the preliminary enhanced image data into the dual-branch attention diffusion network, extracting a high-frequency detail feature map through the spatial attention branch, and extracting a low-frequency structure feature map through the channel attention branch;

[0080] Constructing a hierarchical feature pyramid network, the hierarchical feature pyramid network includes a feature mapping module, a feature reconstruction module and a feature optimization module, inputting the high-frequency detail feature map and the low-frequency structure feature map into the feature mapping module, generating a feature map group of multiple scales, inputting the feature map group into the feature reconstruction module, calculating the similarity matrix between features of different scales in combination with a self-attention mechanism and reconstructing features based on the similarity matrix, inputting the reconstructed feature map into the feature optimization module, iteratively optimizing the features through a recursive neural network and a feedback mechanism, and obtaining an optimized compensation feature map;

[0081] An adaptive illumination compensation network is constructed, wherein the adaptive illumination compensation network comprises an illumination analysis module, a reflectance analysis module and an illumination balancing module, the compensation feature map is input into the illumination analysis module, an image illumination component and an illumination distribution map are estimated through a multi-scale convolutional neural network, the compensation feature map is input into the reflectance analysis module, an image reflectance component and a texture detail map are restored through a deep separable convolutional network, the illumination distribution map and the texture detail map are input into the illumination balancing module, illumination non-uniformity correction is performed on the initially enhanced image data through an adaptive exposure compensation algorithm and a local contrast enhancement algorithm, and the corrected image data is output.

[0082] The hierarchical feature pyramid network is a deep learning network structure, which constructs a feature pyramid through multi-level and different-scale feature maps, and gradually extracts and fuses multi-level features in the image. The adaptive exposure compensation algorithm is an image processing algorithm used to automatically adjust the exposure parameters of the image to compensate for underexposure or overexposure problems that may occur during shooting. The local contrast enhancement algorithm is an image enhancement technology that aims to enhance local details and contrast in the image.

[0083] Obtain low-quality image data to be processed, which may have problems such as noise, blur, and uneven illumination. Use inverse wavelet transform to preliminarily enhance the image data. Specifically, perform wavelet decomposition on the low-quality image data to obtain sub-band coefficients of multiple different frequency bands. Perform enhancement processing on each sub-band coefficient according to the image quality requirements. For example, enhance the high-frequency sub-band coefficients to improve image details, enhance the low-frequency sub-band coefficients to improve the overall brightness of the image, and reconstruct the processed sub-band coefficients through inverse wavelet transform to obtain preliminarily enhanced image data. For example, perform three-level wavelet decomposition on a low-quality image with a resolution of 512x512, and use the db4 wavelet basis function to obtain low-frequency sub-band LL3 and high-frequency sub-bands LH3, HL3, and HH3. Multiply the high-frequency sub-band coefficients by 1.2 for enhancement, and multiply the low-frequency sub-band coefficients by 1.1 for enhancement, and then perform inverse wavelet transform reconstruction to obtain preliminarily enhanced image data.

[0084] The preliminarily enhanced image data is input into a dual-path diffusion model, which includes two paths: the first path is used to extract high-frequency detail features of the image, such as edges and textures; the second path is used to extract low-frequency structural features of the image, such as shapes and contours. The first path can be implemented using a deep residual reconstruction network, which effectively extracts high-frequency detail features of the image and suppresses noise interference through residual learning and skip connection structure. The second path can be implemented using a dual-branch attention diffusion network. The network includes a spatial attention branch and a channel attention branch, which are used to focus on the spatial details and channel information of the image, respectively, so as to extract low-frequency structural features. For example, assuming that the size of the preliminarily enhanced image data is 256x256, the deep residual reconstruction network of the first path contains 3 residual blocks, each residual block contains 2 convolutional layers, and the convolution kernel size is 3x3. The dual-branch attention diffusion network of the second path contains a spatial attention module and a channel attention module, which process the input features respectively.

[0085] The extracted high-frequency detail feature map and low-frequency structure feature map are input into a hierarchical feature pyramid network, which includes a feature mapping module, a feature reconstruction module and a feature optimization module. The feature mapping module maps the input feature map to multiple scales to form a feature map group. The feature reconstruction module combines the self-attention mechanism to calculate the similarity matrix between features of different scales, and reconstructs the features based on the matrix to fuse the feature information of different scales. The feature optimization module iteratively optimizes the reconstructed features through a recursive neural network and a feedback mechanism to obtain the final compensation feature map. For example, assuming that the size of the high-frequency detail feature map and the low-frequency structure feature map are both 128x128, they are mapped to three scales of 64x64, 32x32 and 16x16 respectively through the feature mapping module. The feature reconstruction module calculates the similarity matrix between features of different scales and reconstructs the features. The feature optimization module uses a recursive neural network with the number of iterations set to 5 to optimize the features and obtain the final compensation feature map.

[0086] The compensation feature map is input into an adaptive illumination compensation network, which includes an illumination analysis module, a reflectance analysis module and an illumination equalization module. The illumination analysis module estimates the illumination component and illumination distribution map of the image through a multi-scale convolutional neural network. The reflectance analysis module restores the reflectance component and texture detail map of the image through a deep separable convolutional network. The illumination equalization module inputs the illumination distribution map and the texture detail map into the adaptive exposure compensation algorithm and the local contrast enhancement algorithm, corrects the illumination unevenness of the initially enhanced image data, and outputs the final corrected image data. For example, assuming that the size of the compensation feature map is 256x256, the illumination analysis module uses a three-layer convolutional neural network with convolution kernel sizes of 7x7, 5x5 and 3x3 to estimate the illumination component and illumination distribution map of the image. The reflectance analysis module uses two layers of deep separable convolution with a convolution kernel size of 3x3 to restore the image reflectance component and texture detail map. The illumination equalization module corrects the illumination unevenness of the initially enhanced image data based on the illumination distribution map and the texture detail map.

[0087] In this embodiment, the details and structural features of the image are effectively extracted through a deep residual reconstruction network and a dual-branch attention diffusion network, and feature fusion and optimization are performed through a hierarchical feature pyramid network, thereby improving the clarity and detail expression of the image. Through an adaptive illumination compensation network, the illumination component and reflectance component of the image can be accurately estimated, and illumination equalization processing can be performed, thereby effectively correcting the illumination unevenness problem of the image and making the image more natural and beautiful. The image is preliminarily enhanced through wavelet transform, which can effectively suppress noise and blur and improve the robustness of the image. The introduction of an adaptive illumination compensation network also makes the method more adaptable to images under different illumination conditions.

[0088] S2. Construct a dynamic hybrid feature enhancement network based on a neural architecture search method, add the corrected image data to the dynamic hybrid feature enhancement network and generate a multi-level feature pyramid, perform cross-layer attention operations on feature maps in adjacent layers in the multi-level feature pyramid, generate inter-layer association weights, reconstruct and enhance feature maps at each level, generate fused feature maps and add them to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes multiple sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and combines the independent detection results with a dynamic integration strategy based on uncertainty to obtain an initial defect detection result;

[0089] The neural architecture search method is a technology for automatically designing a neural network structure. By exploring different network structure combinations, the optimal architecture parameters are automatically selected to improve the performance and efficiency of the model. The inter-layer association weight refers to the connection weight between different layers in the neural network. The cross-layer attention operation is a mechanism for establishing an attention mechanism between different layers of the neural network to enhance the mutual attention of features at different levels. The multi-branch expert decision module is a neural network module containing multiple independent branches, where each branch acts as an expert and is responsible for processing different types of information or tasks.

[0090] In an optional embodiment,

[0091] A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, the corrected image data is added to the dynamic hybrid feature enhancement network and a multi-level feature pyramid is generated, a cross-layer attention operation is performed on the feature maps in adjacent layers in the multi-level feature pyramid, inter-layer association weights are generated, and the feature maps of each level are reconstructed and enhanced, and a fused feature map is generated and added to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes a plurality of sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and the independent detection results are fused in combination with a dynamic integration strategy based on uncertainty, and the initial defect detection results include:

[0092] A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, wherein the neural architecture search method selects and combines a variety of convolutional layers, pooling layers, and activation function components from a predefined network structure space through a search algorithm, generates a candidate network, evaluates the performance of the candidate network, and selects the network architecture with the best performance as the structure of the dynamic hybrid feature enhancement network;

[0093] Inputting the modified image data into the dynamic hybrid feature enhancement network, performing multiple downsampling operations through the encoder module in the dynamic hybrid feature enhancement network, converting the modified image data into low-resolution features, performing multiple upsampling operations through the decoder module in the dynamic hybrid feature enhancement network, restoring the low-resolution features to original resolution features, extracting multiple feature maps of different scales at different levels of the encoder module and the decoder module, and constructing a multi-level feature pyramid;

[0094] Performing a cross-layer attention operation on the feature maps of adjacent layers in the multi-level feature pyramid, mapping the modified feature maps of adjacent layers to the same number of channels through a convolution operation, calculating a query matrix and a key matrix of the modified feature maps in the adjacent layers, generating an attention weight matrix based on the query matrix and the key matrix, reconstructing the modified feature maps based on the attention weight matrix, performing the cross-layer attention operation and feature reconstruction operation on the modified feature maps in each layer of the multi-level feature pyramid, and generating a fused pyramid feature map;

[0095] Constructing a multi-branch expert decision module, the multi-branch expert decision module includes a plurality of independent sub-networks, each of the sub-networks adopts an encoder-decoder structure and is provided with a classifier, the fused pyramid feature map is input into the multi-branch expert decision module, different types of defects are detected respectively through the plurality of sub-networks, and each sub-network outputs a corresponding independent detection result;

[0096] The confidences corresponding to the independent detection results output by the multiple sub-networks are obtained, and based on the confidences, the independent detection results are weighted averaged and fused through a dynamic integration strategy to obtain an initial defect detection result.

[0097] The activation function component is a function module used to activate neurons in a neural network, and the upsampling operation is an operation commonly used in signal processing or image processing, which aims to increase the resolution or size of data.

[0098] A dynamic hybrid feature enhancement network is constructed using a neural architecture search method. A network structure search space containing a variety of convolutional layers (e.g., different convolution kernel sizes, different convolution strides), pooling layers (e.g., max pooling, average pooling), and activation functions (e.g., ReLU, Sigmoid) is predefined. A search algorithm, such as an evolutionary algorithm or a reinforcement learning algorithm, is used to select and combine these components from the search space to generate multiple candidate network structures. For each candidate network structure, an image dataset with defect annotations is used for training, and the performance on the validation set is evaluated, such as mean average precision (mAP). The network architecture with the best performance is selected as the final dynamic hybrid feature enhancement network. For example, the network structure finally selected by the search algorithm contains three convolutional layers with kernel sizes of 3x3, 5x5, and 7x7, respectively, and strides of 1. Each convolutional layer is followed by a ReLU activation function and a 2x2 max pooling layer.

[0099] The collected image data that may contain defects is preprocessed, such as grayscale, normalization and other operations, and the corrected image data is input into the constructed dynamic hybrid feature enhancement network. The network includes an encoder module and a decoder module. The encoder module converts the input image into a low-resolution feature map through multiple downsampling operations. For example, the input image size is 256x256, and it becomes 16x16 after passing through the encoder. The decoder module restores the low-resolution feature map to the original resolution through multiple upsampling operations. For example, the 16x16 feature map is restored to 256x256. At different levels of the encoder and decoder, multiple feature maps of different scales are extracted to construct a multi-level feature pyramid. For example, the feature maps of the third layer of the encoder and the second layer of the decoder are extracted to form a two-layer feature pyramid.

[0100] Perform cross-layer attention operations on feature maps of adjacent layers in a multi-level feature pyramid. Taking a two-layer feature pyramid as an example, the feature maps from the third layer of the encoder and the second layer of the decoder are mapped to the same number of channels through convolution operations, for example, both are mapped to 64 channels. Calculate the query matrix and key matrix of these two feature maps respectively. Based on the query matrix and the key matrix, calculate the attention weight matrix. Use the attention weight matrix to weightedly reconstruct the feature map to obtain the enhanced feature map. Perform cross-layer attention operations and feature reconstruction operations on each layer in the pyramid, and finally generate a fused pyramid feature map.

[0101] Construct a multi-branch expert decision module, which contains multiple independent sub-networks. Each sub-network adopts an encoder-decoder structure and sets a classifier. Input the fused pyramid feature map into the multi-branch expert decision module. Each sub-network focuses on detecting different types of defects, such as scratches, cracks, stains, etc. Each sub-network processes the input feature map independently and outputs the corresponding independent detection results, such as the location and category of the defect. Assume that there are three sub-networks, one for detecting scratches, one for detecting cracks, and one for detecting stains.

[0102] Get the confidence level corresponding to the independent detection results output by each subnetwork. Based on the confidence level, a dynamic integration strategy, such as weighted average based on uncertainty, is used to fuse the independent detection results. Detection results with high confidence levels are given greater weights. The independent detection results of multiple subnetworks are weighted averaged and fused to obtain the final defect detection result. For example, if the confidence level of scratches detected by subnetwork 1 is 0.9, the confidence level of cracks detected by subnetwork 2 is 0.6, and the confidence level of stains detected by subnetwork 3 is 0.3, the final result is more inclined to scratches.

[0103] In this embodiment, the optimal network structure is found through neural architecture search, and combined with the dynamic hybrid feature enhancement network and cross-layer attention mechanism, feature information of different scales can be effectively extracted and fused to improve the accuracy of defect detection, especially in complex backgrounds and tiny defects. The multi-branch expert decision module and dynamic integration strategy can effectively handle different types of defects and reduce the impact of misjudgment of a single sub-network on the final result, enhance the robustness and generalization ability of the model, and optimize the network structure through neural architecture search, reduce the number of model parameters and computational complexity, improve the efficiency of defect detection, and meet real-time requirements.

[0104] In an optional embodiment,

[0105] The cross-layer attention operation is performed on the feature maps of adjacent layers in the multi-level feature pyramid as shown in the following formula:

[0106] ;

[0107] Among them, A represents the calculated attention weight matrix element, represents the kth element of the query vector at the mth position in the i-th layer feature map, Represents the weight matrix of the query matrix of the i-th layer, transforming the query vector from d h Dimension conversion to d l Dimensions, represents the kth element of the key vector at the nth position in the jth layer feature map, represents the weight matrix of the key matrix of the i-th layer, converting the key vector from d h Dimension conversion to dl Dimension, d h represents the original dimensions of the query vector and key vector, d l represents the dimension of the query vector and key vector after linear transformation, h represents the number of heads in the multi-head attention mechanism, represents the attention weight between the mth position in the i-th layer feature map and the nth position in the j-th layer feature map in the g-th attention head, Represents the weight matrix of the output matrix, O is used to indicate the output.

[0108] In this embodiment, by fusing the features of adjacent layers, the contextual information in the image can be better captured, thereby improving the expressiveness of the features. The attention mechanism can make the model pay more attention to important features and ignore irrelevant features, thereby improving the robustness of the model. The cross-layer attention mechanism can promote the flow of information between features at different levels, so that the detailed information of low-level features can be effectively transmitted to high-level features, and the semantic information of high-level features can also be effectively transmitted to low-level layers.

[0109] S3. Establish a spatiotemporal knowledge reasoning graph based on the initial defect detection results, set the initial defect detection results as graph nodes, calculate the morphological similarity and spatial position relationship between each initial defect detection result and construct the edge connection strength, hierarchically cluster the initial defect detection results based on the edge connection strength to generate similar defect groups, perform graph attention network operations on the spatiotemporal knowledge reasoning graph, generate defect-related feature vectors and perform time series correlation analysis on the defect-related feature vectors and process parameter sequences, construct a process parameter influence model, and perform reliability evaluation and optimization on the initial defect detection results based on the process parameter influence model and the similar defect groups to obtain the final defect detection results.

[0110] The morphological similarity is a measurement method for measuring the similarity between different morphological features, and is usually used in fields such as image analysis and pattern recognition. It classifies, matches or clusters objects by calculating the shape similarity of different objects or graphics. The hierarchical clustering is an unsupervised learning algorithm used to divide data into different hierarchical structures. By calculating the similarity between data points, hierarchical clustering can generate a tree structure (also called a tree diagram) to represent the hierarchical relationship of the data. It is often used in data analysis and pattern recognition tasks. The similar defect group refers to the classification of different defects into the same group based on the similarity of the defects in defect detection or classification tasks. The time series correlation analysis is a technology for analyzing the relationship between different variables in time series data, usually by calculating the correlation coefficient of time series data or using other statistical methods to evaluate the dependency and time series relationship between different data points.

[0111] In an optional embodiment,

[0112] A spatiotemporal knowledge reasoning graph is established based on the initial defect detection results, the initial defect detection results are set as graph nodes, the morphological similarity and spatial position relationship between each initial defect detection result are calculated and the edge connection strength is constructed, the initial defect detection results are hierarchically clustered based on the edge connection strength to generate similar defect groups, a graph attention network operation is performed on the spatiotemporal knowledge reasoning graph, a defect-related feature vector is generated and a time series correlation analysis is performed between the defect-related feature vector and the process parameter sequence, a process parameter influence model is constructed, and the reliability of the initial defect detection results is evaluated and optimized based on the process parameter influence model and the similar defect groups, and the final defect detection results include:

[0113] Acquire an initial defect detection result of the chip image, wherein the initial defect detection result includes morphological features and spatial position coordinates of each defect, wherein the morphological features include length, width and area, and the spatial position coordinates include the row number and column number of the defect center point;

[0114] Each defect in the initial defect detection result is set as a graph node, and a spatiotemporal knowledge reasoning graph is established, each of the graph nodes contains the morphological features and spatial position coordinates of the corresponding defect. For each pair of graph nodes in the spatiotemporal knowledge reasoning graph, the morphological similarity is obtained by calculating the cosine similarity between the morphological features of different graph nodes, and the spatial position distance is obtained by calculating the Euclidean distance between the spatial position coordinates of different graph nodes. The edge connection strength is constructed based on the morphological similarity and the spatial position distance. If the morphological similarity between the two graph nodes is greater than a preset similarity threshold and the spatial position distance is less than a preset distance threshold, the edge connection strength is calculated according to the spatial position distance;

[0115] Based on the edge connection strength, clustering the graph nodes by a bottom-up hierarchical clustering method, initially setting each graph node as an independent cluster, iteratively merging two clusters with the largest edge connection strength, and repeating the merging until the edge connection strength between all clusters is less than a preset connection strength threshold, thereby generating multiple similar defect groups;

[0116] Constructing a graph attention network, the graph attention network includes an attention layer and a feature propagation layer, for the spatiotemporal knowledge reasoning graph, calculating the attention weight between each graph node and the adjacent node corresponding to the current graph node through the attention layer, the attention weight is determined based on the similarity of the node features, based on the attention weight, updating the features of the graph nodes through the feature propagation layer, so that the features of each graph node aggregate the weighted information of the corresponding adjacent nodes, and generating a node feature matrix;

[0117] Extract the defect-associated feature vector corresponding to each of the similar defect groups in the node feature matrix, obtain the process parameter sequence within the time period corresponding to the similar defect group, the process parameter sequence includes temperature parameters and pressure parameters, calculate the cross-correlation coefficient between the defect-associated feature vector and the process parameter sequence by the Granger causality test method, and at the same time, build a process parameter influence model according to the cross-correlation coefficient, input each defect in the initial defect detection result into the process parameter influence model, determine the corresponding similar defect group based on the morphological features in the initial defect detection result, judge whether the current defect is caused by process abnormality according to the prediction result output by the process parameter influence model, mark the defect determined to be caused by process abnormality as a reliable defect, filter out defects not caused by process abnormality, and obtain the final defect detection result.

[0118] The graph node is one of the basic elements in graph theory, representing a data point or entity in the graph. In the graph model, a node usually represents an object, event or other entity, and the nodes are connected by edges. The edge connection strength is an attribute of the edge in the graph, which is used to measure the closeness or strength of the connection between two nodes. The edge connection strength can reflect the relationship strength between nodes and is usually used in tasks such as network analysis and social network analysis. The independent cluster means that there is no direct connection or relationship between nodes or data points in the graph or data set. The Granger causality test method is a statistical method used to determine whether one time series can predict another time series.

[0119] Obtain the initial defect detection results of the chip image, use defect detection equipment, such as an optical microscope or a scanning electron microscope, scan the chip image, identify and extract the defect information in the image. This information includes the morphological characteristics and spatial position coordinates of each defect. The morphological characteristics include the length, width and area of ​​the defect, which can be calculated using image processing techniques, such as edge detection and region segmentation algorithms. The spatial position coordinates record the row and column numbers of the defect center point in the image. For example, three defects are detected in a chip image. Defect 1 has a length of 10 microns, a width of 5 microns, an area of ​​50 square microns, and a center point coordinate of (100, 200); Defect 2 has a length of 8 microns, a width of 6 microns, an area of ​​48 square microns, and a center point coordinate of (150, 250); Defect 3 has a length of 12 microns, a width of 4 microns, an area of ​​48 square microns, and a center point coordinate of (300, 400).

[0120] Establish a spatiotemporal knowledge reasoning graph. Take each detected defect as a graph node to form a spatiotemporal knowledge reasoning graph. Each graph node contains the morphological features (length, width, area) and spatial position coordinates (row number, column number) of the corresponding defect. For example, based on the three detected defects, establish a spatiotemporal knowledge reasoning graph containing three nodes.

[0121] Calculate the edge connection strength. For each pair of nodes in the graph, calculate the morphological similarity and spatial distance between them. Morphological similarity is measured by calculating the cosine similarity between the morphological feature vectors (length, width, area) of the two nodes. Spatial distance is measured by calculating the Euclidean distance between the coordinates of the center points of the two nodes. If the morphological similarity of the two nodes is greater than the preset similarity threshold (for example, 0.9), and the spatial distance is less than the preset distance threshold (for example, 100 microns), the edge connection strength is calculated based on the spatial distance. The smaller the spatial distance, the greater the edge connection strength. For example, the morphological similarity of node 1 and node 2 is 0.95, and the spatial distance is 50 microns. The conditions are met, and the calculated edge connection strength is 0.9; the morphological similarity of node 1 and node 3 is 0.8, and the spatial distance is 200 microns. The conditions are not met, so there is no edge connection.

[0122] Perform hierarchical clustering. Use a bottom-up hierarchical clustering method to cluster the graph nodes based on the calculated edge connection strength. Initially, each node is treated as an independent cluster. Then iteratively merge the two clusters with the largest edge connection strength. Repeat the merging process until the edge connection strength between all clusters is less than the preset connection strength threshold (for example, 0.5). Finally, the nodes in the graph are divided into multiple groups of similar defects. For example, the edge connection strength between node 1 and node 2 is 0.9, which is greater than the threshold 0.5, then node 1 and node 2 are merged into one cluster.

[0123] Perform graph attention network operations. Construct a graph attention network, which includes an attention layer and a feature propagation layer. Input the spatiotemporal knowledge reasoning graph into the graph attention network. The attention layer calculates the attention weights between each node and its neighboring nodes. The attention weights are determined based on the similarity of node features. The feature propagation layer updates the features of each node according to the attention weights, so that the features of each node aggregate the weighted information of its neighboring nodes to generate a node feature matrix.

[0124] Construct a process parameter influence model. Extract the defect-associated feature vector corresponding to each similar defect group in the node feature matrix. Obtain the process parameter sequence in the corresponding time period of the similar defect group, including temperature parameters and pressure parameters. Use the Granger causality test method to calculate the cross-correlation coefficient between the defect-associated feature vector and the process parameter sequence. Construct a process parameter influence model based on the cross-correlation coefficient. For example, assuming that the defect-associated feature vector of a similar defect group is [0.1, 0.2, 0.3], the temperature parameter sequence in the corresponding time period is [200, 210, 220], and the pressure parameter sequence is [1.0, 1.1, 1.2]. The cross-correlation coefficient between the temperature parameter and the defect-associated feature vector is 0.8, and the cross-correlation coefficient between the pressure parameter and the defect-associated feature vector is 0.6 calculated by the Granger causality test method. Then, a process parameter influence model is constructed, which reflects the degree of influence of temperature and pressure on defects.

[0125] Perform defect reliability assessment and optimization. Input each defect in the initial defect detection results into the process parameter impact model. Determine the similar defect group to which the defect belongs based on its morphological characteristics. Determine whether the defect is caused by process abnormality based on the prediction results of the process parameter impact model. Mark the defects determined to be caused by process abnormality as reliable defects, filter out defects not caused by process abnormality, and obtain the final defect detection results. For example, input defect 1 into the process parameter impact model, and the model predicts that the defect is caused by temperature abnormality, then mark defect 1 as a reliable defect.

[0126] In this embodiment, by combining the morphological characteristics, spatial position relationship and process parameters of the defects, defects caused by process abnormalities can be identified more accurately, and false detections caused by non-process abnormalities can be effectively filtered out, thereby improving the accuracy of defect detection. By grouping similar defects through hierarchical clustering, the efficiency of defect classification can be effectively improved, and targeted analysis and processing of different types of defects can be facilitated. By constructing a process parameter influence model, the correlation between process parameters and defects can be analyzed, providing guidance for optimizing the process flow and reducing the occurrence of defects.

[0127] In an optional embodiment,

[0128] Calculating the cross-correlation coefficient between the defect-associated feature vector and the process parameter sequence by the Granger causality test method includes:

[0129] Obtaining a process parameter sequence and a defect-related feature vector in a chip manufacturing process, and dividing the process parameter sequence into a training data set and a test data set in chronological order, wherein the training data set is used to construct a prediction model, and the test data set is used to evaluate the performance of the prediction model;

[0130] A first prediction model is constructed based on the training data set, and the historical data of the process parameter sequence within a preset time window is used to predict the process parameter value at the current moment; a second prediction model is constructed based on the training data set, and the historical data of the process parameter sequence and the defect-associated feature vector within the preset time window are used to predict the process parameter value at the current moment;

[0131] Input the test data set into the first prediction model and the second prediction model, calculate the residual sum of squares of the two prediction models by summing the squares of the differences between the predicted values ​​and the actual values, and construct a Granger causality test statistic, wherein the Granger causality test statistic is calculated based on the residual sum of squares of the first prediction model, the residual sum of squares of the second prediction model, the number of parameters of the first prediction model, the number of parameters of the second prediction model, and the number of samples in the test data set;

[0132] A critical value is determined based on a preset significance level. When the Granger causality test statistic is greater than the critical value, the defect-associated feature vector is determined to be the Granger cause of the process parameter sequence. The cross-correlation coefficients between each feature in the defect-associated feature vector and the process parameter sequence at different time lags are calculated based on the covariance and variance of the defect-associated feature vector and the process parameter sequence.

[0133] The Granger causality test statistic is a statistic that measures the strength of the causal relationship between time series. It is used to decide whether to reject the null hypothesis (i.e., there is no causal relationship) and further quantify the strength of the causal relationship. The Granger cause refers to the variable or time series that can provide causal prediction capabilities in the Granger causality test. The time lag refers to the degree of influence of the value of a variable at a certain moment in a time series on the value of another variable at a subsequent moment. The cross-correlation coefficient is a statistic that measures the similarity between two time series. By calculating the cross-correlation coefficient, the correlation between two time series at different time lags can be analyzed. It is often used in signal processing, system modeling and other fields.

[0134] Collect process parameter data and defect-related feature vector data during the chip manufacturing process. The process parameter data includes, for example, temperature, pressure, deposition rate, etc., while the defect-related feature vector may include information such as defect type, size, and location, which needs to be cleaned and preprocessed, such as removing outliers and missing values.

[0135] The collected process parameter sequences are divided into training data sets and test data sets in chronological order. For example, the first 80% of the data is used as the training data set, and the remaining 20% ​​is used as the test data set. The training data set is used to build the prediction model, and the test data set is used to evaluate the performance of the model.

[0136] Construct two prediction models. The first prediction model uses only the historical data of the process parameter sequence to predict the process parameter value at the current moment. For example, the temperature data at the past 10 time points is used to predict the temperature at the current moment. Methods such as autoregressive models or recurrent neural networks can be used. The second prediction model uses both the process parameter sequence and the historical data of the defect-related feature vector within a preset time window to predict the process parameter value at the current moment. For example, the temperature data at the past 10 time points and the defect size data at the corresponding time points are used to predict the temperature at the current moment. This model can also use a prediction method similar to the first model.

[0137] Input the test data set into the two prediction models, calculate the sum of squares of the difference between the predicted value and the actual value, and get the residual sum of squares of the two prediction models. Assume that the residual sum of squares of the first model on the test data set is 100, and the residual sum of squares of the second model is 80.

[0138] The Granger causality test statistic is constructed using the residual sum of squares of the two models, the number of parameters of the two models, and the number of samples in the test data set. This statistic can reflect the degree to which the defect-associated feature vector improves the prediction accuracy of the process parameter sequence. For example, if the number of parameters of the first model is 5, the number of parameters of the second model is 7, and the number of samples in the test data set is 1000, the Granger causality test statistic can be calculated according to a specific formula.

[0139] According to the preset significance level (for example, 0.05), the critical value is determined, which can be obtained by looking up a table or statistical software. If the calculated Granger causality test statistic is greater than the critical value, it is determined that the defect-associated feature vector is the Granger cause of the process parameter sequence. For example, if the calculated Granger causality test statistic is 10 and the critical value is 5, it can be determined that the defect-associated feature vector has a Granger causal relationship with the process parameter sequence.

[0140] If the defect-associated feature vector is determined to be the Granger cause of the process parameter sequence, the cross-correlation coefficient between each feature in the defect-associated feature vector and the process parameter sequence at different time lags is further calculated. For example, the cross-correlation coefficient between the defect size and temperature at 1 time point, 2 time points, etc. can be calculated by calculating the covariance of the defect size and temperature and their respective variances, and then calculating the correlation coefficient, which helps to understand how the defect-associated feature vector affects the process parameter sequence and the time lag at which the impact occurs.

[0141] In this embodiment, by identifying defect-associated feature vectors that have a Granger causal relationship with the process parameter sequence, the root cause of the defect can be quickly located, thereby improving the efficiency of defect diagnosis. This can help engineers understand which process parameters have the greatest impact on product quality, thereby optimizing the process parameter control strategy in a more targeted manner and reducing the occurrence of defects. Through effective control of key process parameters, the yield of chip products can ultimately be improved and production costs can be reduced.

[0142] Figure 2 FIG. 1 is a schematic diagram of the structure of a semiconductor chip defect automatic detection system based on hyperspectral imaging according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0143] The first unit is used to collect hyperspectral image data of a semiconductor chip to be detected, add the hyperspectral image data to an improved neuromorphic adaptive processing network and calculate a noise distribution density map and a texture complexity map of a local area, construct an adaptive enhancement matrix based on the noise distribution density map and the texture complexity map, calculate wavelet decomposition coefficients of different scales based on the adaptive enhancement matrix, perform nonlinear mapping on the wavelet decomposition coefficients to obtain enhancement coefficients, and obtain preliminary enhanced image data through inverse wavelet transformation reconstruction; input the preliminary enhanced image data into a dual-path diffusion model, extract high-frequency detail features through a first path, extract low-frequency structural features through a second path, and adaptively fuse the high-frequency detail features and the low-frequency structural features to obtain a compensation feature map, perform illumination non-uniformity correction on the preliminary enhanced image data based on the compensation feature map to obtain corrected image data;

[0144] The second unit is used to construct a dynamic hybrid feature enhancement network based on a neural architecture search method, add the corrected image data to the dynamic hybrid feature enhancement network and generate a multi-level feature pyramid, perform cross-layer attention operations on feature maps in adjacent layers in the multi-level feature pyramid, generate inter-layer association weights, reconstruct and enhance feature maps at each level, generate fused feature maps and add them to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes multiple sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and combines the independent detection results with a dynamic integration strategy based on uncertainty to obtain an initial defect detection result;

[0145] The third unit is used to establish a spatiotemporal knowledge reasoning graph based on the initial defect detection results, set the initial defect detection results as graph nodes, calculate the morphological similarity and spatial position relationship between each initial defect detection result and construct the edge connection strength, hierarchically cluster the initial defect detection results based on the edge connection strength to generate similar defect groups, perform graph attention network operations on the spatiotemporal knowledge reasoning graph, generate defect-related feature vectors and perform time series correlation analysis on the defect-related feature vectors and process parameter sequences, construct a process parameter influence model, and perform reliability evaluation and optimization on the initial defect detection results based on the process parameter influence model and the similar defect groups to obtain final defect detection results.

[0146] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including: a processor;

[0147] a memory for storing processor-executable instructions;

[0148] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0149] According to a fourth aspect of the embodiments of the present invention,

[0150] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0151] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic detection of semiconductor chip defects based on hyperspectral imaging, characterized in that: include: Acquire hyperspectral image data of a semiconductor chip to be inspected, add the hyperspectral image data to an improved neuromorphic adaptive processing network and calculate a noise distribution density map and a texture complexity map of a local area, construct an adaptive enhancement matrix based on the noise distribution density map and the texture complexity map, calculate wavelet decomposition coefficients of different scales based on the adaptive enhancement matrix, perform nonlinear mapping on the wavelet decomposition coefficients to obtain enhancement coefficients, and reconstruct preliminary enhanced image data by inverse wavelet transform; Inputting the initially enhanced image data into a dual-path diffusion model, extracting high-frequency detail features through a first path, extracting low-frequency structural features through a second path, adaptively fusing the high-frequency detail features and the low-frequency structural features to obtain a compensation feature map, and performing illumination non-uniformity correction on the initially enhanced image data based on the compensation feature map to obtain corrected image data; A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, the corrected image data is added to the dynamic hybrid feature enhancement network and a multi-level feature pyramid is generated, a cross-layer attention operation is performed on the feature maps in adjacent layers in the multi-level feature pyramid, inter-layer association weights are generated, and the feature maps of each level are reconstructed and enhanced, a fused feature map is generated and added to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes a plurality of sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and the independent detection results are fused in combination with a dynamic integration strategy based on uncertainty to obtain an initial defect detection result; A spatiotemporal knowledge reasoning graph is established based on the initial defect detection results, the initial defect detection results are set as graph nodes, the morphological similarity and spatial position relationship between each initial defect detection result are calculated and the edge connection strength is constructed, the initial defect detection results are hierarchically clustered based on the edge connection strength to generate similar defect groups, a graph attention network operation is performed on the spatiotemporal knowledge reasoning graph, a defect-related feature vector is generated and a time series correlation analysis is performed between the defect-related feature vector and a process parameter sequence, a process parameter influence model is constructed, and the reliability of the initial defect detection results is evaluated and optimized based on the process parameter influence model and the similar defect groups to obtain a final defect detection result.

2. The method according to claim 1, characterized in that The hyperspectral image data of the semiconductor chip to be detected is collected, the hyperspectral image data is added to the improved neuromorphic adaptive processing network and the noise distribution density map and the texture complexity map of the local area are calculated, an adaptive enhancement matrix is ​​constructed based on the noise distribution density map and the texture complexity map, wavelet decomposition coefficients of different scales are calculated based on the adaptive enhancement matrix, and the enhancement coefficients obtained by nonlinear mapping of the wavelet decomposition coefficients include: Collecting hyperspectral image data of the semiconductor chip to be inspected, wherein the hyperspectral image data includes image data of visible light band, near infrared band and short-wave infrared band; Inputting the hyperspectral image data into an improved neuromorphic adaptive processing network, calculating the local statistical features of the hyperspectral image data at different scales, extracting noise distribution feature vectors and texture feature vectors, constructing a multi-layer neural network based on the noise distribution feature vectors and calculating a noise distribution density map, constructing a convolutional neural network based on the texture feature vectors and calculating a texture complexity map, inputting the noise distribution density map and the texture complexity map into an attention mechanism module, and generating an attention weight map; According to the attention weight map, the noise distribution density map and the texture complexity map are feature reconstructed, and the reconstructed feature maps are fused to construct an adaptive enhancement matrix. Based on the adaptive enhancement matrix, different bands of the hyperspectral image data are adaptively enhanced respectively, and a multi-scale directional filter group and a wavelet decomposition unit are constructed. The enhanced hyperspectral image data is simultaneously subjected to directional decomposition, scale decomposition and wavelet decomposition, and a plurality of directional sub-bands and scale sub-bands are obtained through the multi-scale directional filter group, and an approximate coefficient sub-band and a detail coefficient sub-band are obtained through the wavelet decomposition unit. Adaptive threshold segmentation is performed on the direction subband, the scale subband, the approximate coefficient subband and the detail coefficient subband to extract significant feature coefficients, adaptive weight modulation is performed on the significant feature coefficients and the adaptive enhancement matrix, a nonlinear activation function operation is performed on the modulated coefficients to generate enhancement coefficients.

3. The method according to claim 1, characterized in that The image data is initially enhanced by reconstructing through inverse wavelet transform; the image data is input into a dual-path diffusion model, high-frequency detail features are extracted through a first path, low-frequency structural features are extracted through a second path, and the high-frequency detail features and the low-frequency structural features are adaptively fused to obtain a compensation feature map; and the initial enhanced image data is corrected for illumination non-uniformity based on the compensation feature map to obtain corrected image data, including: Constructing a deep residual reconstruction network, inputting the enhancement coefficient into the deep residual reconstruction network, and obtaining preliminary enhanced image data through residual learning and skip connection structure reconstruction; Constructing a dual-branch attention diffusion network, the dual-branch attention diffusion network includes a spatial attention branch and a channel attention branch, inputting the preliminary enhanced image data into the dual-branch attention diffusion network, extracting a high-frequency detail feature map through the spatial attention branch, and extracting a low-frequency structure feature map through the channel attention branch; Constructing a hierarchical feature pyramid network, the hierarchical feature pyramid network includes a feature mapping module, a feature reconstruction module and a feature optimization module, inputting the high-frequency detail feature map and the low-frequency structure feature map into the feature mapping module, generating a feature map group of multiple scales, inputting the feature map group into the feature reconstruction module, calculating the similarity matrix between features of different scales in combination with a self-attention mechanism and reconstructing features based on the similarity matrix, inputting the reconstructed feature map into the feature optimization module, iteratively optimizing the features through a recursive neural network and a feedback mechanism, and obtaining an optimized compensation feature map; An adaptive illumination compensation network is constructed, wherein the adaptive illumination compensation network comprises an illumination analysis module, a reflectance analysis module and an illumination balancing module, the compensation feature map is input into the illumination analysis module, an image illumination component and an illumination distribution map are estimated through a multi-scale convolutional neural network, the compensation feature map is input into the reflectance analysis module, an image reflectance component and a texture detail map are restored through a deep separable convolutional network, the illumination distribution map and the texture detail map are input into the illumination balancing module, illumination non-uniformity correction is performed on the initially enhanced image data through an adaptive exposure compensation algorithm and a local contrast enhancement algorithm, and the corrected image data is output.

4. The method according to claim 1, characterized in that: A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, the corrected image data is added to the dynamic hybrid feature enhancement network and a multi-level feature pyramid is generated, a cross-layer attention operation is performed on the feature maps in adjacent layers in the multi-level feature pyramid, inter-layer association weights are generated, and the feature maps of each level are reconstructed and enhanced, and a fused feature map is generated and added to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes a plurality of sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and the independent detection results are fused in combination with a dynamic integration strategy based on uncertainty, and the initial defect detection results include: A dynamic hybrid feature enhancement network is constructed based on a neural architecture search method, wherein the neural architecture search method selects and combines a variety of convolutional layers, pooling layers, and activation function components from a predefined network structure space through a search algorithm, generates a candidate network, evaluates the performance of the candidate network, and selects the network architecture with the best performance as the structure of the dynamic hybrid feature enhancement network; Inputting the modified image data into the dynamic hybrid feature enhancement network, performing multiple downsampling operations through the encoder module in the dynamic hybrid feature enhancement network, converting the modified image data into low-resolution features, performing multiple upsampling operations through the decoder module in the dynamic hybrid feature enhancement network, restoring the low-resolution features to original resolution features, extracting multiple feature maps of different scales at different levels of the encoder module and the decoder module, and constructing a multi-level feature pyramid; Performing a cross-layer attention operation on the feature maps of adjacent layers in the multi-level feature pyramid, mapping the modified feature maps of adjacent layers to the same number of channels through a convolution operation, calculating a query matrix and a key matrix of the modified feature maps in the adjacent layers, generating an attention weight matrix based on the query matrix and the key matrix, reconstructing the modified feature maps based on the attention weight matrix, performing the cross-layer attention operation and feature reconstruction operation on the modified feature maps in each layer of the multi-level feature pyramid, and generating a fused pyramid feature map; Constructing a multi-branch expert decision module, the multi-branch expert decision module includes a plurality of independent sub-networks, each of the sub-networks adopts an encoder-decoder structure and is provided with a classifier, the fused pyramid feature map is input into the multi-branch expert decision module, different types of defects are detected respectively through the plurality of sub-networks, and each sub-network outputs a corresponding independent detection result; The confidences corresponding to the independent detection results output by the multiple sub-networks are obtained, and based on the confidences, the independent detection results are weighted averaged and fused through a dynamic integration strategy to obtain an initial defect detection result.

5. The method according to claim 4, characterized in that The cross-layer attention operation is performed on the feature maps of adjacent layers in the multi-level feature pyramid as shown in the following formula: ； Among them, A represents the calculated attention weight matrix element, represents the kth element of the query vector at the mth position in the i-th layer feature map, Represents the weight matrix of the query matrix of the i-th layer, transforming the query vector from d h Dimension conversion to d l Dimensions, represents the kth element of the key vector at the nth position in the jth layer feature map, represents the weight matrix of the key matrix of the i-th layer, converting the key vector from d h Dimension conversion to d l Dimension, d h represents the original dimensions of the query vector and key vector, d l represents the dimension of the query vector and key vector after linear transformation, h represents the number of heads in the multi-head attention mechanism, represents the attention weight between the mth position in the i-th layer feature map and the nth position in the j-th layer feature map in the g-th attention head, Represents the weight matrix of the output matrix, and O is used to indicate the output.

6. The method according to claim 1, characterized in that A spatiotemporal knowledge reasoning graph is established based on the initial defect detection results, the initial defect detection results are set as graph nodes, the morphological similarity and spatial position relationship between each initial defect detection result are calculated and the edge connection strength is constructed, the initial defect detection results are hierarchically clustered based on the edge connection strength to generate similar defect groups, a graph attention network operation is performed on the spatiotemporal knowledge reasoning graph, a defect-related feature vector is generated and a time series correlation analysis is performed between the defect-related feature vector and the process parameter sequence, a process parameter influence model is constructed, and the reliability of the initial defect detection results is evaluated and optimized based on the process parameter influence model and the similar defect groups, and the final defect detection results include: Acquire an initial defect detection result of the chip image, wherein the initial defect detection result includes morphological features and spatial position coordinates of each defect, wherein the morphological features include length, width and area, and the spatial position coordinates include the row number and column number of the defect center point; Each defect in the initial defect detection result is set as a graph node, and a spatiotemporal knowledge reasoning graph is established, each of the graph nodes contains the morphological features and spatial position coordinates of the corresponding defect. For each pair of graph nodes in the spatiotemporal knowledge reasoning graph, the morphological similarity is obtained by calculating the cosine similarity between the morphological features of different graph nodes, and the spatial position distance is obtained by calculating the Euclidean distance between the spatial position coordinates of different graph nodes. The edge connection strength is constructed based on the morphological similarity and the spatial position distance. If the morphological similarity between the two graph nodes is greater than a preset similarity threshold and the spatial position distance is less than a preset distance threshold, the edge connection strength is calculated according to the spatial position distance; Based on the edge connection strength, clustering the graph nodes by a bottom-up hierarchical clustering method, initially setting each graph node as an independent cluster, iteratively merging two clusters with the largest edge connection strength, and repeating the merging until the edge connection strength between all clusters is less than a preset connection strength threshold, thereby generating multiple similar defect groups; Constructing a graph attention network, the graph attention network includes an attention layer and a feature propagation layer, for the spatiotemporal knowledge reasoning graph, calculating the attention weight between each graph node and the adjacent node corresponding to the current graph node through the attention layer, the attention weight is determined based on the similarity of the node features, based on the attention weight, updating the features of the graph nodes through the feature propagation layer, so that the features of each graph node aggregate the weighted information of the corresponding adjacent nodes, and generating a node feature matrix; Extract the defect-associated feature vector corresponding to each of the similar defect groups in the node feature matrix, obtain the process parameter sequence within the time period corresponding to the similar defect group, the process parameter sequence includes temperature parameters and pressure parameters, calculate the cross-correlation coefficient between the defect-associated feature vector and the process parameter sequence by the Granger causality test method, and at the same time, build a process parameter influence model according to the cross-correlation coefficient, input each defect in the initial defect detection result into the process parameter influence model, determine the corresponding similar defect group based on the morphological features in the initial defect detection result, judge whether the current defect is caused by process abnormality according to the prediction result output by the process parameter influence model, mark the defect determined to be caused by process abnormality as a reliable defect, filter out defects not caused by process abnormality, and obtain the final defect detection result.

7. The method according to claim 6, characterized in that Calculating the cross-correlation coefficient between the defect-associated feature vector and the process parameter sequence by the Granger causality test method includes: Obtaining a process parameter sequence and a defect-related feature vector in a chip manufacturing process, and dividing the process parameter sequence into a training data set and a test data set in chronological order, wherein the training data set is used to construct a prediction model, and the test data set is used to evaluate the performance of the prediction model; A first prediction model is constructed based on the training data set, and the historical data of the process parameter sequence within a preset time window is used to predict the process parameter value at the current moment; a second prediction model is constructed based on the training data set, and the historical data of the process parameter sequence and the defect-associated feature vector within the preset time window are used to predict the process parameter value at the current moment; Input the test data set into the first prediction model and the second prediction model, calculate the residual sum of squares of the two prediction models by summing the squares of the differences between the predicted values ​​and the actual values, and construct a Granger causality test statistic, wherein the Granger causality test statistic is calculated based on the residual sum of squares of the first prediction model, the residual sum of squares of the second prediction model, the number of parameters of the first prediction model, the number of parameters of the second prediction model, and the number of samples in the test data set; A critical value is determined based on a preset significance level. When the Granger causality test statistic is greater than the critical value, the defect-associated feature vector is determined to be the Granger cause of the process parameter sequence. The cross-correlation coefficients between each feature in the defect-associated feature vector and the process parameter sequence at different time lags are calculated based on the covariance and variance of the defect-associated feature vector and the process parameter sequence.

8. A semiconductor chip defect automatic detection system based on hyperspectral imaging, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect hyperspectral image data of a semiconductor chip to be detected, add the hyperspectral image data to an improved neuromorphic adaptive processing network and calculate a noise distribution density map and a texture complexity map of a local area, construct an adaptive enhancement matrix based on the noise distribution density map and the texture complexity map, calculate wavelet decomposition coefficients of different scales based on the adaptive enhancement matrix, perform nonlinear mapping on the wavelet decomposition coefficients to obtain enhancement coefficients, and reconstruct preliminary enhanced image data through inverse wavelet transform; Inputting the initially enhanced image data into a dual-path diffusion model, extracting high-frequency detail features through a first path, extracting low-frequency structural features through a second path, adaptively fusing the high-frequency detail features and the low-frequency structural features to obtain a compensation feature map, and performing illumination non-uniformity correction on the initially enhanced image data based on the compensation feature map to obtain corrected image data; The second unit is used to construct a dynamic hybrid feature enhancement network based on a neural architecture search method, add the corrected image data to the dynamic hybrid feature enhancement network and generate a multi-level feature pyramid, perform cross-layer attention operations on feature maps in adjacent layers in the multi-level feature pyramid, generate inter-layer association weights, reconstruct and enhance feature maps at each level, generate fused feature maps and add them to a pre-set multi-branch expert decision module, wherein the multi-branch expert decision module includes multiple sub-networks for different types of defects, each sub-network independently generates an independent detection result corresponding to the fused feature map, and combines the independent detection results with a dynamic integration strategy based on uncertainty to obtain an initial defect detection result; The third unit is used to establish a spatiotemporal knowledge reasoning graph based on the initial defect detection results, set the initial defect detection results as graph nodes, calculate the morphological similarity and spatial position relationship between each initial defect detection result and construct the edge connection strength, hierarchically cluster the initial defect detection results based on the edge connection strength to generate similar defect groups, perform graph attention network operations on the spatiotemporal knowledge reasoning graph, generate defect-related feature vectors and perform time series correlation analysis on the defect-related feature vectors and process parameter sequences, construct a process parameter influence model, and perform reliability evaluation and optimization on the initial defect detection results based on the process parameter influence model and the similar defect groups to obtain final defect detection results.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images