A method for integrated circuit defect detection incorporating convolutional neural networks

By integrating the temporal feature extraction and spatial neighborhood attention module of convolutional neural networks, and combining them with defect detection and evolution prediction heads, the problem of capturing the temporal evolution process of defects in integrated circuit defect detection is solved, improving the stability and accuracy of detection and enabling the prediction of defect development trends.

CN123118461APending Publication Date: 2026-11-17HUIZHOU JUZHI MICRO TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611121973.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-27
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing integrated circuit defect detection methods cannot effectively capture the time evolution of defects, leading to the easy omission or misjudgment of intermittent, progressive, and latent defects, and the poor stability of detection results.

Method used

By employing a fusion convolutional neural network approach, a temporal consistency loss function is constructed through a temporal feature extraction module, a spatial neighborhood attention module, and a defect detection head and a defect evolution prediction head. This optimizes the defect detection results and enables the discovery and prediction of the spatiotemporal evolution patterns of defects.

Benefits of technology

It significantly improves the ability to detect latent and progressive defects, enhances the accuracy of defect spatial distribution patterns, provides early warning capabilities for defect development trends, and ensures the continuity and consistency of detection results in the temporal dimension.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure FT_1
    Figure FT_1
  • Figure FT_2
    Figure FT_2
  • Figure FT_3
    Figure FT_3
Patent Text Reader

Abstract

This application provides an integrated circuit defect detection method integrating convolutional neural networks, belonging to the interdisciplinary field of artificial intelligence and circuit inspection. The method includes: receiving multi-frame sequence image data; obtaining temporal feature vectors for each frame through a temporal feature extraction module; enhancing the feature interaction between the defect region and the surrounding background through a spatial neighborhood attention module to obtain a neighborhood-enhanced feature representation; inputting this representation into a defect detection head and a defect evolution prediction head respectively, outputting the defect category of the current frame and the evolution trend of future frames; constructing a temporal consistency loss function based on the two outputs to impose temporal constraints on the detection results and optimize detection stability. This method, by mining temporal evolution patterns and spatial neighborhood relationships, combined with temporal consistency constraints, can improve the detection capability of intermittent and progressive defects, enhance the temporal stability of detection results, and achieve defect trend prediction, making it suitable for scenarios such as integrated circuit manufacturing, packaging testing, and aging monitoring.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the technical field of artificial intelligence combined with circuit defect detection, specifically relating to an integrated circuit defect detection method that integrates convolutional neural networks. Background Technology

[0002] In the integrated circuit manufacturing process, accurate detection of chip surface defects is a key link to ensure product quality and yield. Existing defect detection methods are mainly based on the analysis and judgment of single-frame static images. However, a single-frame image can only capture the instantaneous state at the moment of detection and cannot reflect the dynamic evolution process of defects over time. This fundamental limitation makes intermittent defects, progressive defects, and latent defects caused by process fluctuations very easy to be missed or misjudged under the single-frame detection framework, which seriously restricts the reliability and early warning capability of the detection system.

[0003] Common convolutional neural network-based defect detection methods generally improve the feature extraction capability of single-frame images by increasing network depth, optimizing loss functions, or introducing attention mechanisms. However, these improvements are all based on the paradigm of static image analysis. The lack of temporal correlation makes it difficult for detectors to establish a mapping relationship between the historical state and the current state of defects, and thus cannot form an effective understanding of the evolution trend of defects. At the same time, the detection results lacking multi-frame temporal constraints often show significant fluctuations when facing intermittent defects, and the stability is difficult to guarantee. How to effectively mine the spatiotemporal evolution law of defects has become a key problem that urgently needs to be solved in the field of integrated circuit defect detection. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an integrated circuit defect detection method that integrates convolutional neural networks, which can effectively solve the problems mentioned in the background technology.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] This invention provides a method for detecting defects in integrated circuits by incorporating convolutional neural networks, the method comprising the following steps:

[0007] Receive multi-frame sequence image data, input the multi-frame sequence image data into the temporal feature extraction module, and extract the temporal feature vector corresponding to each frame image in the multi-frame sequence image data through the temporal feature extraction module to obtain the multi-frame temporal feature representation;

[0008] The multi-frame temporal feature representation is input into the spatial neighborhood attention module. The spatial neighborhood attention module calculates the attention weight distribution between the defect region and the surrounding neighborhood background to obtain the neighborhood enhancement feature representation.

[0009] The neighborhood enhancement feature representation is input into the defect detection head and the defect evolution prediction head, respectively. The defect detection head outputs the defect detection result of the current frame, and the defect evolution prediction head outputs the defect evolution trend prediction result of the future frame.

[0010] Based on the current frame defect detection result output by the defect detection head and the future frame defect evolution trend prediction result output by the defect evolution prediction head, a temporal consistency loss function is constructed, and the temporal consistency loss function is used to perform temporal constraint optimization on the detection result of the defect detection head to obtain the temporally constrained defect detection result.

[0011] Output the defect detection results after the time-series constraints and the corresponding defect evolution trend prediction results.

[0012] Furthermore, the temporal feature extraction module extracts the temporal feature vectors corresponding to each frame in the multi-frame sequence image data, specifically including:

[0013] The temporal feature vector is extracted using a 3D convolutional neural network, which includes a first 3D convolutional layer, a first pooling layer, a second 3D convolutional layer, a second pooling layer, a third 3D convolutional layer, and a global average pooling layer connected in sequence.

[0014] The first 3D convolutional layer has a kernel spatial size of 3×3 and a temporal depth of 3, and the first 3D convolutional layer has 64 output channels; the second 3D convolutional layer has 128 output channels; the third 3D convolutional layer has 256 output channels; both the first pooling layer and the second pooling layer use max pooling operation, and the pooling window has a spatial dimension of 2×2 and a temporal dimension of 2;

[0015] The global average pooling layer performs average pooling on the features output by the third 3D convolutional layer in the time dimension, compressing them into a fixed-length temporal feature vector.

[0016] Furthermore, after receiving multi-frame sequence image data, a frame alignment processing step is also included:

[0017] The frame alignment unit uses a built-in timestamp parsing module to identify the timestamps embedded in each frame of image data. Based on these timestamps, frames that are out of order due to transmission delays are reordered, and missing frames are marked.

[0018] When the intervals between adjacent frames are inconsistent during continuous acquisition, the frame alignment unit uses an interpolation resampling algorithm to normalize the time axis and outputs a temporally continuous and evenly spaced image sequence.

[0019] Furthermore, the attention weight distribution between the defective region and the surrounding neighborhood background is calculated through the spatial neighborhood attention module to obtain the neighborhood enhancement feature representation, specifically including:

[0020] The multi-frame temporal feature representation is segmented along the time dimension into independent frame-level spatial feature maps for each frame.

[0021] For each feature point in the spatial feature map of each frame, a square or circular neighborhood is defined centered on the feature point. The square neighborhood has a side length of 5 or 7 pixels, and the circular neighborhood has a radius of 2 or 3 pixels. For feature points in the edge region, a zero-filling strategy is used to expand the boundary of the spatial feature map.

[0022] Calculate the similarity between the feature vector of each central feature point and the feature vectors of each neighboring feature point within the neighborhood range. The similarity is calculated using either dot product or cosine similarity. Generate an initial attention weight distribution based on the similarity calculation results. Perform Softmax normalization on the initial attention weights within the neighborhood range to obtain normalized attention weights.

[0023] The normalized attention weights are used to perform a weighted summation of the feature vectors of all neighboring feature points within the neighborhood range, and the result is used as the neighborhood enhanced feature representation of the central feature point. The influence of neighboring feature points on the central feature point follows the distance decay principle, with the basic influence of neighboring feature points that are closer in space being higher and the basic influence of neighboring feature points that are farther in space being lower.

[0024] Furthermore, the temporal feature extraction module employs a lightweight 3D convolutional network to extract the temporal feature vector:

[0025] The lightweight 3D convolutional network uses depth-separable 3D convolutional operations to replace standard 3D convolutional operations. The depth-separable 3D convolutional operations decompose standard 3D convolution into two cascaded steps: spatial dimension depth convolution and temporal dimension pointwise convolution.

[0026] The spatial dimension depthwise convolution performs spatial convolution operations independently on each input channel to extract the spatial features of each channel; the temporal dimension pointwise convolution integrates the temporal information of each channel through a 1×1×1 convolution kernel to achieve temporal feature fusion between channels.

[0027] The lightweight 3D convolutional network includes a depth-separable 3D convolutional layer group, a downsampling layer group, and a global temporal aggregation layer. The depth-separable 3D convolutional layer group has 32 output channels, and the downsampling layer group uses a 3×3×3 max pooling operation with a pooling stride of 2.

[0028] Furthermore, the defect detection results output by the defect detection head for the current frame include the category label, location information, and confidence score of each defect region in the current frame image;

[0029] The network structure of the defect detection head consists of a fully connected layer and a cascaded Softmax activation function. The input dimension of the fully connected layer is consistent with the flattened dimension of the input feature map, and the output dimension of the fully connected layer is set to the total number of defect categories plus 1.

[0030] The fully connected layer linearly maps the input high-dimensional neighborhood enhancement features into a feature vector with a dimension equal to the total number of categories. The Softmax activation function converts the feature vector into a probability distribution of the current frame image belonging to each category. The defect detection head outputs the defect detection result of the current frame based on the category corresponding to the maximum value in the probability distribution.

[0031] Furthermore, the defect evolution prediction head outputs the defect evolution trend prediction results for future frames, specifically including:

[0032] The defect evolution prediction head receives the enhanced feature map sequence of the current frame and multiple consecutive frames before the current frame from the neighborhood enhanced feature representation. The network structure of the defect evolution prediction head consists of a fully connected layer and a cascaded hyperbolic tangent activation function.

[0033] The input dimension of the fully connected layer is consistent with the total dimension of the input multi-frame feature maps after flattening. The output dimension of the fully connected layer is set to 3. The hyperbolic tangent activation function maps each element in the 3-dimensional feature vector output by the fully connected layer to the interval between -1 and 1, resulting in 3 consecutive prediction confidence values.

[0034] The three output dimensions correspond to three different evolution trends: the first dimension corresponds to the trend of defects shrinking, the second dimension corresponds to the trend of defects remaining unchanged, and the third dimension corresponds to the trend of defects expanding.

[0035] The defect evolution prediction head outputs the prediction result of the defect evolution trend of the current frame to the future frame based on the trend category corresponding to the dimension with the largest value among the three confidence values.

[0036] Furthermore, a temporal consistency loss function is constructed, specifically including:

[0037] Calculate the KL divergence value between the defect detection results of the previous frame and the defect detection results of the current frame on the category probability distribution, and use the KL divergence value as the inter-frame detection consistency loss;

[0038] Inter-frame detection consistency loss Defined as:

[0039]

[0040] in, This refers to the inter-frame detection consistency loss. The total number of categories, For category indexing, The first predicted frame of the previous frame The probability values ​​of each category, The predicted number for the current frame The probability values ​​of each category;

[0041] The inter-frame detection consistency loss Loss prediction with defect evolution The time-series consistency loss function is obtained by performing weighted fusion. :

[0042]

[0043] in, Let be the time-series consistency loss function. Predict the loss for the evolution of the aforementioned defect. These are the weighting coefficients for the inter-frame detection consistency loss. The weighting coefficients for the predicted loss of the defect evolution, and .

[0044] Furthermore, constructing the temporal consistency loss function also includes a consistency constraint step between evolution prediction and detection results:

[0045] The evolution trend prediction result output by the defect evolution prediction head is correlated and verified with the inter-frame detection consistency: when the prediction result is that the defect will expand, it is verified whether the probability of the defect category in the current frame detection result is higher than that in the previous frame; when the prediction result is that the defect will shrink, it is verified whether the probability of the defect category in the current frame detection result is lower than that in the previous frame; when the prediction result is that the defect will remain unchanged, it is verified whether the detection results of the two frames are consistent.

[0046] If the direction of change in the detection result does not match the predicted trend, it is determined that there is a conflict between the detection result of the current frame and the temporal logic.

[0047] Furthermore, the evolution trend category label predicted by the defect evolution prediction head is obtained by labeling the training samples during the training phase:

[0048] Collect integrated circuit surface image sequences containing known defect types as training samples. Each training sample contains a set of consecutive frame image sequences and corresponding fine annotations for each frame image. The annotation information includes the defect category, segmentation mask or bounding box of the defect region.

[0049] For the training of the defect evolution prediction head, the evolution trend labels of defects in the sequence in subsequent real frames are labeled, and the change trend of the defect area in subsequent frames is quantified into three discrete labels: expansion, shrinkage or no change. The three discrete labels correspond to the three evolution trends represented by the three output dimensions of the defect evolution prediction head.

[0050] In summary, this application includes at least one of the following beneficial technical effects:

[0051] 1. The integrated circuit defect detection method fused with convolutional neural networks provided by the present invention firstly receives multi-frame sequence image data and inputs the multi-frame sequence image data into a temporal feature extraction module. The temporal feature extraction module extracts the temporal feature vectors corresponding to each frame of the multi-frame sequence image data to obtain multi-frame temporal feature representations. Then, the multi-frame temporal feature representations are input into a spatial neighborhood attention module. The spatial neighborhood attention module calculates the attention weight distribution between the defect region and the surrounding neighborhood background to obtain a neighborhood enhancement feature representation.

[0052] 2. This invention inputs the neighborhood enhancement feature representation into the defect detection head and the defect evolution prediction head, respectively. The defect detection head outputs the defect detection result of the current frame, and the defect evolution prediction head outputs the defect evolution trend prediction result of the future frame. Then, based on the defect detection result of the current frame output by the defect detection head and the defect evolution trend prediction result of the future frame output by the defect evolution prediction head, a temporal consistency loss function is constructed, and the detection result of the defect detection head is temporally constrained and optimized using the temporal consistency loss function to obtain the temporally constrained defect detection result. Finally, the temporally constrained defect detection result and the corresponding defect evolution trend prediction result are output.

[0053] 3. This invention effectively mines the temporal evolution patterns in multi-frame sequence image data through a temporal feature extraction module, solving the technical problem that traditional single-frame static image detection cannot capture the temporal evolution process of defects, and significantly improving the detection capability of latent and progressive defects; it enhances the feature interaction between the defect region and the surrounding background through a spatial neighborhood attention module, solving the technical problem that traditional methods do not fully consider the spatial neighborhood relationship of defects, and improving the accuracy of defect spatial distribution pattern mining; it achieves the prediction of defect development trends through a defect evolution prediction head, solving the problem that existing technologies can only identify defects in the current frame and cannot predict defect development, providing an effective early warning capability for defects caused by process fluctuations; and it solves the problem of large fluctuations and poor stability of intermittent defect detection results through a temporal consistency constraint module, ensuring the continuity and consistency of detection results in the temporal dimension. Attached Figure Description

[0054] Figure 1A schematic diagram of the overall technical solution for an integrated circuit defect detection method incorporating convolutional neural networks;

[0055] Figure 2 This is a schematic diagram illustrating the core principle of the time-series feature extraction module.

[0056] Figure 3 A schematic diagram illustrating the characteristic interactions of the spatial neighborhood attention mechanism;

[0057] Figure 4 A flowchart illustrating the dual-head output and timing consistency constraints for defect detection and evolution prediction. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 4 Specific embodiments are provided to further illustrate the present invention in detail.

[0059] Example 1

[0060] This invention provides an integrated circuit defect detection method incorporating convolutional neural networks. This method constructs a convolutional neural network architecture enhanced with spatiotemporal correlation features, utilizing multi-frame sequence image data to achieve high-precision detection and evolution trend prediction of integrated circuit defects. The method comprises the following core processing stages: temporal feature extraction, spatial neighborhood attention enhancement, dual-head output, and temporal consistency constraint optimization. These stages are sequentially connected according to data flow, forming an end-to-end defect detection pipeline, as described in the following steps.

[0061] The first step, S1, involves receiving and extracting multi-frame sequence image data. This step receives multi-frame sequence image data, preprocesses it, and aligns it with the frames. The temporal feature extraction module then transforms the original image sequence into a feature vector representation rich in temporal dependencies, providing feature input for subsequent spatial neighborhood attention enhancement and defect detection. This is achieved through the following steps.

[0062] Step S101: Receiving and buffering multi-frame sequence image data. The data receiving module establishes a communication connection with the dynamic scanning device or online monitoring system through a standardized data interface, supporting the parsing of mainstream video streaming protocols and image sequence file formats, thereby acquiring continuously acquired multi-frame sequence image data. The multi-frame sequence image data consists of several frames arranged sequentially along the time axis, with a fixed sampling time interval between adjacent frames.

[0063] The received raw sequence image data is first sent to the frame buffer area built into the data receiving module. The frame buffer area is managed using a circular queue data structure. When there is a momentary mismatch between the image data writing speed and the reading speed of the subsequent processing module, the circular queue can play a data smoothing role, thereby maintaining the integrity and temporal continuity of the data in the entire processing flow.

[0064] Step S102, image sequence preprocessing: The data preprocessing unit performs a standardized preprocessing procedure on the multi-frame image sequence data in the buffer to improve image quality and unify data format. The preprocessing procedure specifically includes the following steps:

[0065] An adaptive filtering algorithm is applied to each frame of the image to suppress noise. The adaptive filtering algorithm can remove random noise and interference introduced by the image sensor or transmission circuit, thereby improving the signal-to-noise ratio of the image.

[0066] The denoised image undergoes contrast enhancement processing using histogram equalization or adaptive contrast stretching techniques. This operation expands the dynamic range of image grayscale distribution, making the grayscale difference between potential defective areas and the normal background more significant.

[0067] Perform size normalization to adjust the resolution of all images to a preset standard size, ensuring that the subsequent network model can receive input tensors with fixed dimensions.

[0068] Step S103: Frame alignment of the image sequence. The frame alignment unit performs temporal alignment calibration on the preprocessed image sequence. The frame alignment unit has a built-in timestamp parsing module that can automatically identify the timestamps embedded in each frame of image data. Based on the timestamps, the frame alignment unit reorders frames that are out of order due to transmission delays and marks missing frames.

[0069] For cases where the intervals between adjacent frames are inconsistent during continuous acquisition, the frame alignment unit uses an interpolation resampling algorithm to normalize the time axis, and finally outputs a set of image sequences that are continuous in time and have uniform intervals.

[0070] Step S104, Temporal Feature Extraction: Based on the first architecture of a 3D convolutional network, the temporal feature extraction module is responsible for extracting feature representations rich in temporal dependencies from the aligned sequence after the aforementioned processing. This embodiment provides two optional network architectures to adapt to different application scenarios. As the first architecture, a 3D convolutional neural network is used to extract temporal features. The 3D convolutional neural network utilizes three-dimensional convolutional kernels to perform convolution operations simultaneously in the spatial and temporal dimensions, thereby capturing the spatial semantic information within the image and the dynamic change patterns between frames in one go. The specific hierarchical structure design of the 3D convolutional neural network is as follows:

[0071] The first layer is a 3D convolutional layer C1, with a kernel spatial size of 3×3, a temporal depth of 3, and a stride of 1 in both the spatial and temporal dimensions. The same padding strategy is used to maintain the feature map size. The output channel number of layer C1 is set to 64. After processing the sequence images through layer C1, a shallow spatiotemporal feature map is obtained. This shallow spatiotemporal feature map encodes basic visual information such as edges and textures, as well as their initial trends of change between adjacent frames.

[0072] The output of layer C1 is then connected to the first pooling layer P1. Layer P1 uses max pooling, with a pooling window of 2×2 in spatial dimension and 2 in temporal dimension, and a stride the same as the window size. Max pooling downsamples the feature map, reducing the spatial and temporal resolution of the data, thereby reducing subsequent computation and enhancing the translation invariance of the features.

[0073] The output of layer P1 is then fed into the second 3D convolutional layer C2. The kernel size, stride, and padding of layer C2 are the same as those of layer C1, but the number of output channels is increased to 128. Through the deeper channel dimension, layer C2 is able to encode mid-level semantic information, such as the motion trajectory or continuous morphological changes of defective regions in the sequence.

[0074] After layer C2, a second pooling layer P2 is connected. The parameter configuration of layer P2 is exactly the same as that of layer P1, and the features are further downsampled.

[0075] The output of layer P2 is then fed into the third 3D convolutional layer C3. The output channel number of layer C3 is set to 256, and the other convolutional parameters are consistent with the first two layers. With its larger receptive field and stronger feature combination ability, layer C3 can extract highly abstract deep spatiotemporal features and capture the global evolution patterns and complex temporal dependencies of defects over a long time span.

[0076] The output of layer C3 is fed into the Global Average Pooling (GAP) layer. GAP performs average pooling on the features along the temporal dimension, aggregating and compressing feature responses spanning multiple frames into a fixed-length temporal feature vector. This temporal feature vector integrates the temporal information of the entire input sequence, encapsulating the historical state and temporal evolution patterns of defects, and serves as the final output of the temporal feature extraction module based on the 3D convolutional network architecture.

[0077] Step S105, Temporal Feature Extraction: Based on the second architecture of the Transformer encoder, this second architecture uses a Transformer-based temporal encoder to extract temporal features. The Transformer temporal encoder is particularly suitable for complex application scenarios that require capturing long-range temporal dependencies. Unlike 3D convolutional networks, the Transformer temporal encoder directly models the long-distance dependency between any two time points in a sequence through a self-attention mechanism, effectively overcoming the problem of limited receptive field in convolutional networks. The specific implementation of the Transformer temporal encoder is as follows:

[0078] The input sequence of image data is processed through a feature projection layer, which maps each frame of the image into a fixed-dimensional feature vector. The feature projection layer is implemented using a two-dimensional convolutional network or a fully connected layer. Assume the input sequence contains... Frame images, each frame is mapped to If a feature vector is given, the entire sequence is transformed into a feature vector of size 1. The characteristic matrix of .

[0079] The positional encoding unit adds positional encoding information to the feature matrix. Since the self-attention mechanism itself lacks the ability to perceive order, an encoding scheme based on sine and cosine functions is used to generate unique positional identifiers for frames at different positions in the sequence. Even-numbered positions in the feature vector are encoded using a sine function, and odd-numbered positions are encoded using a cosine function. The encoding dimension is the same as the feature vector dimension. Maintain consistency. The feature sequence after adding positional encoding contains both image content information and temporal order information.

[0080] The feature sequence with added positional encoding is input into the multi-head self-attention mechanism module. The module has 8 attention heads, each with a dimension of 64, resulting in a final output attention feature dimension of 512. The self-attention calculation process is as follows: The input features are mapped to query vectors, key vectors, and value vectors using three independent linear transformation matrices; the dot product of the query vector and all key vectors is calculated, normalized by a scaling factor, and then the attention weight distribution is obtained using the Softmax function; the attention weight distribution is then used to perform a weighted summation of all value vectors to obtain the output of that attention head. The outputs of all attention heads are concatenated and then subjected to another linear transformation to form the final output of the multi-head self-attention mechanism module.

[0081] The output of the multi-head self-attention mechanism module is fed into a feedforward neural network for nonlinear transformation. The feedforward neural network contains two fully connected layers connected by a ReLU activation function, which enhances the model's ability to model complex nonlinear relationships.

[0082] The structure, which includes multi-head self-attention layers and feedforward network layers, is stacked in multiple layers. Depending on the complexity of the application scenario, the number of encoder layers in the Transformer temporal encoder is set to 6 to 12. Through multi-layer stacking, the model can abstract and integrate temporal features layer by layer, and finally output a high-dimensional feature vector rich in long-range temporal dependencies, which serves as the output of the temporal feature extraction module based on the Transformer temporal encoder architecture.

[0083] Step S106: Training strategies for the two temporal feature extraction architectures. Both of the aforementioned temporal feature extraction network architectures require training to achieve effective feature extraction capabilities. The training process for the two temporal feature extraction network architectures is conducted in conjunction with the entire defect detection network. The specific training strategies are as follows:

[0084] The construction of the training dataset is the first step. Image sequences of integrated circuit surfaces containing known defect types and locations are collected as training samples. Each training sample contains a set of consecutive frame image sequences, along with detailed annotations for each frame. The annotation information includes the defect category, the segmentation mask or bounding box of the defect region. For training the defect evolution prediction head, it is also necessary to annotate the evolution trend labels of defects in subsequent ground truth frames. The change trend of the defect region area over several subsequent frames is quantified into three discrete labels: expansion, shrinkage, or no change.

[0085] The training objective of the entire network is to minimize the joint loss function for multiple tasks. The joint loss function consists of three parts: the classification loss of the defect detection head, the prediction loss of the defect evolution prediction head, and the temporal consistency constraint loss.

[0086] For training the defect detection head, the standard cross-entropy loss function is used for classification loss, which measures the difference between the defect class probability distribution predicted by the defect detection head and the true class label. Assume the number of samples in a batch is... The total number of defect categories is For the first For each sample, the one-hot encoding of the true class label is: The category probability output by the defect detection head is Then classification loss The mathematical form is:

[0087]

[0088] in, This represents the total number of samples involved in this loss calculation; The total number of defect categories; It is the index number of the sample; It is a category index; It is the first Each sample for category The actual label value, which can be 0 or 1; It is the first defect detection head prediction Each sample belongs to category The probability value ranges from 0 to 1.

[0089] For training the defect evolution prediction head, the prediction loss also uses the cross-entropy loss function. Suppose a batch contains... The number of trend categories is 3 for the nth sample. One sample, the true evolution trend label is The trend probability of the defect evolution prediction head output is Then predict the loss. The mathematical form is:

[0090]

[0091] in, This represents the total number of samples involved in this loss calculation; The category index represents the evolution trend, with values ​​of 1, 2, or 3, corresponding to the three trends of the defect shrinking, remaining unchanged, and expanding, respectively; It is the first For the nth sample The true label value for the evolutionary trend, which can be 0 or 1; It is the first prediction of defect evolution. The sample belongs to the first The probability value of this evolutionary trend, ranging from 0 to 1.

[0092] Temporal consistency loss The specific construction and calculation methods are explained in detail in step S4.

[0093] Total loss function of the entire network Defined as a weighted sum of classification loss, prediction loss, and temporal consistency loss:

[0094]

[0095] in, , , These are the weight coefficients for classification loss, prediction loss, and temporal consistency loss, used to balance the contribution of each loss term to network training. The value range is from 0.5 to 1.5. The value range is from 0.2 to 1.0. The value ranges from 0.1 to 0.8, and the specific value is selected through cross-validation. During training, the gradient is calculated based on the total loss function, and the backpropagation algorithm is used to jointly optimize all network parameters, including the temporal feature extraction module, spatial neighborhood attention module, defect detection head, and defect evolution prediction head, end-to-end until the model converges.

[0096] In summary, step S1 completes the entire process of processing multi-frame sequence image data from reception to output of temporal feature vectors. The data receiving module acquires continuously collected multi-frame sequence image data and caches it in the frame buffer area; the data preprocessing unit performs denoising, contrast enhancement, and size normalization on the sequence images; the frame alignment unit aligns and normalizes the preprocessed image sequence in the temporal dimension; the temporal feature extraction module provides two optional architectures: one based on a 3D convolutional network and the other based on a Transformer temporal encoder, used for short-to-medium-term temporal feature capture and long-range dependency modeling, respectively; step S106 provides a joint training strategy for the two network architectures. The temporal feature vectors output after processing are input into the subsequent step S2 for spatial neighborhood attention enhancement processing.

[0097] For step S2, the multi-frame temporal feature representation output from step S1 is received. A spatial neighborhood attention mechanism is used to enhance the spatial feature map of each frame, modeling the correlation between each feature point and its surrounding neighborhood feature points. Neighborhood information is adaptively aggregated based on the degree of correlation, thereby strengthening the feature representation of the defective region and suppressing background noise. Step S2 is specifically implemented through the following sub-steps.

[0098] Step S201: Frame-dimensional splitting of the multi-frame temporal feature representation. The feature map splitting unit divides the multi-frame temporal feature representation into independent frame-level spatial feature maps for each frame along the time dimension. The multi-frame temporal feature representation includes... Frame features, obtained after splitting Each frame-level spatial feature map retains the spatial structure and channel information of the corresponding frame.

[0099] Step S202, Neighborhood Range Determination and Edge Filling: The neighborhood range determination unit sets a preset neighborhood range centered on each feature point in the spatial feature map of each frame. The neighborhood shape supports two configurations: square and circular. The side length of a square neighborhood is typically set to 5 or 7 pixels, and the radius of a circular neighborhood is typically set to 2 or 3 pixels. For feature points in edge regions, a zero-filling strategy is used to expand the boundaries of the spatial feature map, ensuring that edge feature points obtain the same amount of neighborhood information as feature points in the central region.

[0100] Step S203, Attention Weight Calculation and Normalization: For each central feature point, the attention weight calculation unit calculates the similarity between the feature vector of the central feature point and the feature vectors of each neighboring feature point within its neighborhood. The similarity calculation uses either dot product or cosine similarity.

[0101] When using dot product operation to calculate similarity, the feature vectors of the central feature point and the feature vectors of the neighboring feature points are first mapped to a new feature space by a linear transformation matrix. Then, the dot product between the two mapped vectors is calculated as the similarity value.

[0102] The dimension of the linear transformation matrix is ​​set to ,in The dimension of the input feature vector. The dimension of the mapped feature vector. The value of and To maintain consistency, the initial values ​​of the linear transformation matrix are assigned using the Xavier uniform initialization method, meaning that each element in the linear transformation matrix is ​​initialized from the interval [missing value]. It was obtained by independent sampling from a uniform distribution within the area.

[0103] The parameters of the linear transformation matrix serve as learnable parameters for the entire defect detection network, and are updated during the network training process using the backpropagation algorithm and gradient descent optimizer.

[0104] The initial attention weight distribution is generated based on the similarity calculation results. The Softmax normalization unit performs Softmax normalization on the initial attention weights within the neighborhood range, converting each initial weight value into a non-negative probability distribution form with a sum of 1, thus obtaining the normalized attention weights.

[0105] Step S204: Neighborhood Feature Weighted Aggregation and Distance Decay. The feature aggregation unit uses normalized attention weights to perform a weighted summation of the feature vectors of all neighboring feature points within the neighborhood range. The result is used as the neighborhood enhanced feature representation of the central feature point. In the weighted summation operation, the neighboring feature points with larger normalized attention weights contribute more to the aggregation result, while the neighboring feature points with smaller weights contribute less.

[0106] In the spatial neighborhood attention mechanism of step S2, the influence of neighborhood feature points on the central feature point follows the distance decay principle: the closer the spatial distance of a neighborhood point, the higher its basic influence; the farther the spatial distance, the lower its basic influence.

[0107] The distance decay principle is indirectly achieved through the spatial location information carried by the feature vector itself. Since feature points that are spatially adjacent in the shallow feature map are naturally correlated in terms of feature values, this correlation is directly reflected in the similarity calculation results, giving nearby neighboring points an advantage in the initial attention weight distribution.

[0108] Step S2 is performed on the spatial feature map of each frame in the multi-frame temporal feature representation, with all frames sharing the same neighborhood range and attention calculation parameters. After processing, the enhanced feature maps of each frame are recombined in the original frame order to form a multi-frame temporal enhanced feature representation, which is then input into the defect detection head and defect evolution prediction head in step S3.

[0109] In step S3 of this embodiment, the multi-frame temporal enhancement feature representation output from step S2 is received. The defect detection result of the current frame and the defect evolution trend prediction result of future frames are output through two parallel branches: a defect detection head and a defect evolution prediction head. The two branches share input features but have different network structures, output targets, and training tasks. Step S3 is specifically implemented through the following sub-steps.

[0110] Step S301: The defect detection head outputs its detection result. The defect detection head receives the enhanced feature map of the current frame from the multi-frame temporal enhanced feature representation output in step S2, and outputs the defect detection result of the current frame. The network structure of the defect detection head consists of a cascaded fully connected layer and a Softmax activation function.

[0111] The input dimension of the fully connected layer is consistent with the flattened dimension of the input feature map. The output dimension of the fully connected layer is set to the total number of defect categories plus 1, where the added dimension corresponds to the normal category, and the remaining dimensions correspond to each defect category. The fully connected layer linearly maps the input high-dimensional neighborhood enhancement features into a feature vector with a dimension equal to the total number of categories.

[0112] The Softmax activation function receives the feature vector output by the fully connected layer, converts each element in the feature vector into a probability value that is between 0 and 1 and the sum of all elements is equal to 1, and obtains the probability distribution of the current frame image belonging to each category.

[0113] The defect detection head outputs the defect detection result of the current frame based on the category corresponding to the maximum value in the probability distribution. The detection result includes the determination of whether there is a defect in the current frame and the specific defect category when a defect exists.

[0114] Step S302: The defect evolution prediction head outputs its evolution trend prediction. The defect evolution prediction head receives the enhanced feature map sequence of the current frame and several consecutive frames preceding the current frame from the multi-frame temporal enhanced feature representation output in step S2, and outputs the predicted evolution trend of the defect in future frames. The network structure of the defect evolution prediction head consists of a cascade of fully connected layers and hyperbolic tangent activation functions.

[0115] The input dimension of the fully connected layer is consistent with the total dimension of the flattened multi-frame feature maps, and the output dimension of the fully connected layer is set to 3. The hyperbolic tangent activation function maps each element in the 3-dimensional feature vector output by the fully connected layer to the interval between -1 and +1, resulting in 3 consecutive prediction confidence values.

[0116] The three output dimensions correspond to three evolution trends: the first dimension indicates the defect will shrink, the second indicates the defect will remain unchanged, and the third indicates the defect will expand. The defect evolution prediction head outputs the prediction result of the defect evolution trend for future frames based on the trend category corresponding to the dimension with the highest confidence value among the three.

[0117] Step S303: The training strategy of dual-head parallel output. The defect detection head and the defect evolution prediction head serve as the two output branches of the entire defect detection network, and are jointly trained with the temporal feature extraction module in step S1 and the spatial neighborhood attention module in step S2. The training tasks and loss functions of the two branches are designed as follows.

[0118] The training objective of the defect detection head is to minimize the classification loss. The classification loss uses the cross-entropy loss function, the mathematical form of which is given in step S106. During training, the parameters of the fully connected layers of the defect detection head are updated through backpropagation and gradient descent optimizer, so that the predicted class probability distribution gradually approximates the one-hot encoded distribution of the true class label.

[0119] The objective of training the defect evolution prediction head is to minimize the prediction loss. The prediction loss uses the cross-entropy loss function, the mathematical form of which is given in step S106. During training, the parameters of the fully connected layers in the defect evolution prediction head are updated using the backpropagation algorithm and gradient descent optimizer, so that the predicted evolution trend confidence distribution gradually approximates the distribution of the true evolution trend labels.

[0120] The fully connected layer parameters of both the defect detection head and the defect evolution prediction head are initially assigned using the Xavier uniform initialization method. The parameters of both branches are set to the same total loss function. Under the constraints of joint optimization, the total loss function has been defined in step S106.

[0121] The defect detection results output in step S301 and the evolution trend prediction results output in step S302 are jointly input into the temporal consistency constraint module in step S4, which is used to construct the temporal consistency loss function and optimize the defect detection results by temporal constraints.

[0122] In summary, step S3 executes the dual-head output of defect detection and evolution prediction. The defect detection head maps the current frame's augmented feature map to probability distributions for various categories using a fully connected layer and a Softmax activation function, outputting whether a defect exists in the current frame and its specific category. The defect evolution prediction head maps a sequence of consecutive multi-frame augmented feature maps to confidence values ​​for three evolution trends using a fully connected layer and a hyperbolic tangent activation function, outputting predictions of whether the defect will shrink, remain unchanged, or expand in future frames. Step S303 explains the classification loss and prediction loss undertaken by the two branches during joint training with the overall network, and their optimization objectives. The outputs of both branches are input into step S4 to construct the temporal consistency loss function and perform temporal constraint optimization.

[0123] The next step, S4, involves receiving the current frame defect detection result output in step S301 and the evolution trend prediction result output in step S302, constructing a temporal consistency loss function, and using this loss function to perform temporal constraint optimization on the detection results of the defect detection head to suppress fluctuations in the temporal dimension of single-frame detection results. Step S4 is specifically implemented through the following sub-steps.

[0124] Step S401: Calculation of inter-frame detection consistency loss. The inter-frame consistency loss calculation unit uses KL divergence to measure the degree of difference between the current frame defect detection result and the previous frame defect detection result in terms of category probability distribution, and uses the KL divergence value as the inter-frame detection consistency loss. The smaller the KL divergence value, the more consistent the detection results of the two frames are; the larger the KL divergence value, the more significant the fluctuation of the detection results between frames.

[0125] Let the category probability distribution of the defect detection head output in the previous frame be... The probability distribution of the categories output by the defect detection head in the current frame is as follows: The total number of categories is Then the inter-frame detection consistency loss Defined as:

[0126]

[0127] in, For category indexing, The first predicted frame of the previous frame The probability values ​​of each category, The predicted number for the current frame The probability values ​​of each category.

[0128] Step S402 involves consistency constraints between evolution prediction and detection results. The evolution consistency constraint unit correlates and verifies the evolution trend prediction result output in step S302 with the inter-frame detection consistency loss. Specifically, the verification logic is as follows: when the prediction result indicates the defect will expand, the probability of the defect category in the current frame detection result should be higher than in the previous frame; when the prediction result indicates the defect will shrink, the probability of the defect category in the current frame detection result should be lower than in the previous frame; when the prediction result indicates the defect will remain unchanged, the detection results of the two frames should be highly consistent. If the direction of change in the detection result does not match the prediction trend, it is determined that there is a conflict between the current frame detection result and the temporal logic.

[0129] Step S403: Construction of the temporal consistency loss function and the total loss function. The temporal consistency loss function construction unit will detect inter-frame consistency loss. Loss prediction with defect evolution We perform weighted fusion to obtain the time-series consistency loss function. . The mathematical form is:

[0130]

[0131] in, and This is a weighting coefficient, with a value ranging from 0.3 to 0.7. The specific values ​​were selected through cross-validation.

[0132] The total loss function construction unit will include the time consistency loss function. Defect detection head classification loss Loss prediction with defect evolution By performing a weighted summation, the total loss function of the entire network is obtained, which has the same form as defined in step S106. Consistency:

[0133]

[0134] in, These are the classification loss weight coefficients. To predict the loss weighting coefficient, The weighting coefficient for time-series consistency loss. The value range is from 0.5 to 1.5. The value range is from 0.2 to 1.0. The value range is from 0.1 to 0.8.

[0135] The parameter optimization unit is based on the total loss function. The gradient is calculated, and the network parameters of the defect detection head, spatial neighborhood attention module, and temporal feature extraction module are updated end-to-end using the backpropagation algorithm and gradient descent optimizer until the network converges or reaches the preset number of training epochs. After training, only the defect detection head is used to output the detection results during the inference phase, and the temporal consistency constraint module no longer participates in the forward computation.

[0136] After steps S401 to S403 are completed, the defect detection results optimized by time consistency constraints and the corresponding evolution trend prediction results are input into step S5 for standardized output.

[0137] Step S401 calculates the KL divergence of adjacent frame detection results as the inter-frame detection consistency loss. Step S402 correlates the evolution trend prediction with the inter-frame detection consistency to determine whether the direction of change in the detection results is consistent with the predicted trend. Step S403 weightedly fuses the inter-frame detection consistency loss and the defect evolution prediction loss into a temporal consistency loss, then weightedly sums it with the classification loss to obtain the total loss function, and updates the network parameters end-to-end through backpropagation. The optimized defect detection results and evolution trend prediction results are input into step S5 for standardized output.

[0138] Finally, step S5 receives the defect detection results optimized by the time-series consistency constraint and the corresponding defect evolution trend prediction results output by step S4, and outputs them in a standardized data format through the result output module.

[0139] The defect detection results include the category label, location information, and confidence score of each defect region in the current frame image. The defect evolution trend prediction results include estimates of the possible evolution direction and magnitude of the defect over several future frames.

[0140] The results output module supports three output methods: file storage interface, network transmission interface, and real-time display interface, to meet the needs of different application scenarios such as offline analysis, remote transmission, and online monitoring.

[0141] Specific application example 1: Quality monitoring in the chip manufacturing process;

[0142] In the quality monitoring scenario of chip manufacturing, the method of this embodiment is deployed in the back-end analysis system of a wafer inspection equipment. A dynamic scanning device continuously scans the wafer after the etching process, acquiring a sequence of data containing dozens of images. This sequence of image data is transmitted to the defect detection system of this embodiment via a high-speed data interface.

[0143] The system first performs denoising and contrast enhancement preprocessing on the received image sequence. Then, a frame alignment unit performs frame alignment to ensure temporal continuity. The preprocessed sequence data is input into a 3D convolutional network in the temporal feature extraction module, which extracts feature representations containing temporal dependencies. A spatial neighborhood attention module performs neighborhood enhancement processing on the features of each frame to improve the feature representation of defect regions. The enhanced features are simultaneously input into the defect detection head and the defect evolution prediction head, which output the defect category of the current frame and the defect evolution trend of future frames, respectively. A temporal consistency loss function is used to constrain and optimize the detection results to ensure temporal stability. Finally, the result output module outputs the defect detection results and evolution trend prediction results, providing defect early warning information for process engineers.

[0144] Specific application example 2: Defect tracing during the packaging and testing phase;

[0145] In the integrated circuit packaging and testing phase, the method in this embodiment is used to perform defect detection and historical traceability analysis on the packaged chip. The packaging and testing equipment continuously acquires sequential image data of the chip surface during the testing process.

[0146] This embodiment's method captures the temporal evolution patterns of defects during the packaging process using a Transformer temporal encoder architecture within the temporal feature extraction module. The spatial neighborhood attention module enhances the feature interaction between the defect region and surrounding structures such as solder joints and pins, improving defect detection capabilities in complex backgrounds. The defect evolution prediction head predicts the defect development trend in future frames based on the defect state of historical frames, providing data support for packaging process optimization. The temporal consistency constraint module ensures that detection results remain stable across consecutive frames, avoiding misjudgments and missed detections caused by detection fluctuations.

[0147] Specific application example 3: Monitoring of aging tests for electronic products;

[0148] In the aging test monitoring scenario of electronic products, the method of this embodiment monitors the changes in defects on the chip surface in real time during the aging process. The aging test equipment continuously monitors the chip under high temperature and high voltage stress conditions, and synchronously acquires sequential image data of the chip surface.

[0149] The temporal feature extraction module of this embodiment captures subtle change patterns of defects during the aging process using a 3D convolutional network. The spatial neighborhood attention module enhances the perception of minute defects on the chip surface. The defect evolution prediction head predicts the expansion trend of defects based on temporal features. When the prediction results indicate that the defect will continue to expand, the defect detection system automatically issues an early warning signal, triggering the corresponding protection mechanism. The temporal consistency constraint module ensures the consistency and reliability of detection results during long-term aging tests.

[0150] Step S5 outputs the defect detection results and evolution trend prediction results, optimized by temporal consistency constraints, in a standardized format through the result output module, supporting file storage, network transmission, and real-time display interfaces. Specific application examples one to three demonstrate the deployment methods and processing flows of the method in this embodiment in three typical scenarios: wafer manufacturing quality monitoring, packaging test defect tracing, and aging test real-time monitoring. The three examples correspond to the application of 3D convolutional networks in wafer inspection and aging monitoring, and the application of Transformer temporal encoders in the complex context of packaging test, respectively, showcasing the adaptability of the two temporal feature extraction architectures in different scenarios.

[0151] Example 2

[0152] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0153] This invention provides an alternative implementation of an integrated circuit defect detection method based on a 3D convolutional network architecture. This solution emphasizes the use of lightweight 3D convolutional network design to adapt to the application requirements of edge computing devices or resource-constrained deployment environments.

[0154] In the temporal feature extraction module, a lightweight 3D convolutional network employs depthwise separable 3D convolution operations instead of standard 3D convolution operations. Depthwise separable 3D convolution decomposes standard 3D convolution into two cascaded steps: spatial depthwise convolution and temporal pointwise convolution. This reduces the number of parameters and computational complexity of the convolution operation. Spatial depthwise convolution performs independent spatial convolution operations on each input channel, extracting spatial features from each channel. Temporal pointwise convolution integrates the temporal information from each channel using a 1×1×1 convolution kernel, achieving temporal feature fusion between channels.

[0155] The temporal feature extraction module adopts a lightweight network structure with the following layer configuration: a group of depthwise separable 3D convolutional layers, each containing a cascaded combination of spatial depthwise convolutional layers, temporally pointwise convolutional layers, and ReLU activation functions, with 32 output channels; a group of downsampling layers, employing 3×3×3 max pooling with a pooling stride of 2 to reduce the spatial and temporal resolution of the feature maps; a group of feature fusion layers, which concatenates and fuses shallow and deep features through skip connections to enhance the preservation of feature details; and a global temporal aggregation layer, which performs average pooling on the fused multi-scale features in the temporal dimension to output the final temporal feature vector.

[0156] The spatial neighborhood attention module employs a grouped attention mechanism in its lightweight implementation. This mechanism divides the channels of the spatial feature map into multiple independent attention groups, each containing the same number of channels. The number of groups is determined by dividing the total number of channels in the input feature map by the number of channels in each group, with each group containing either one-quarter or one-eighth of the total number of channels in the input feature map. Each attention group performs neighborhood attention calculations within its own group, and the results are fused through a channel shuffling operation to achieve cross-group information exchange. This grouped attention mechanism maintains the attention effect while reducing computational complexity to one-half the number of groups required for standard attention.

[0157] The network structure of the defect detection head and the defect evolution prediction head is consistent with the previous embodiment. The number of fully connected hidden units in the defect detection head and the defect evolution prediction head is adjusted according to the dimension of the feature vector output by the lightweight temporal feature extraction module, and the hidden layer dimension is set to match the dimension of the input feature vector.

[0158] The construction method and optimization strategy of the temporal consistency loss function are the same as those in the previous embodiments, and the lightweight network can also achieve the constraint effect of temporal consistency.

[0159] Specific application example 4: Online detection of defects in embedded chips;

[0160] In the online defect detection scenario of embedded chips, the lightweight implementation of the method in this embodiment is deployed on an embedded processor or a dedicated AI acceleration chip. The detection system acquires real-time video streams from the chip surface using a high-speed camera, sampling at a fixed frame rate to obtain a continuous sequence of video frames. A lightweight 3D convolutional network extracts temporal features from the video frame sequence, achieving real-time processing under limited computing resources. The spatial neighborhood attention module calculates attention weights in a grouped manner, resulting in low latency on embedded devices. The defect detection head outputs the defect detection result for the current frame, and the defect evolution prediction head outputs a short-term change prediction of the defect. Temporal consistency constraints ensure the stability of the detection results for consecutive frames. The detection system outputs the defect detection results and evolution trend prediction results through a display interface or communication interface, providing real-time quality feedback to production line operators.

[0161] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0162] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting defects in integrated circuits by incorporating convolutional neural networks, characterized in that, Includes the following steps: Receive multi-frame sequence image data, input the multi-frame sequence image data into the temporal feature extraction module, and extract the temporal feature vector corresponding to each frame image in the multi-frame sequence image data through the temporal feature extraction module to obtain the multi-frame temporal feature representation; The multi-frame temporal feature representation is input into the spatial neighborhood attention module. The spatial neighborhood attention module calculates the attention weight distribution between the defect region and the surrounding neighborhood background to obtain the neighborhood enhancement feature representation. The neighborhood enhancement feature representation is input into the defect detection head and the defect evolution prediction head, respectively. The defect detection head outputs the defect detection result of the current frame, and the defect evolution prediction head outputs the defect evolution trend prediction result of the future frame. Based on the current frame defect detection result output by the defect detection head and the future frame defect evolution trend prediction result output by the defect evolution prediction head, a temporal consistency loss function is constructed, and the temporal consistency loss function is used to perform temporal constraint optimization on the detection result of the defect detection head to obtain the temporally constrained defect detection result. Output the defect detection results after the time-series constraints and the corresponding defect evolution trend prediction results.

2. The integrated circuit defect detection method incorporating convolutional neural networks according to claim 1, characterized in that, The temporal feature extraction module extracts the temporal feature vectors corresponding to each frame in the multi-frame sequence image data, specifically including: The temporal feature vector is extracted using a 3D convolutional neural network, which includes a first 3D convolutional layer, a first pooling layer, a second 3D convolutional layer, a second pooling layer, a third 3D convolutional layer, and a global average pooling layer connected in sequence. The first 3D convolutional layer has a kernel spatial size of 3×3 and a temporal depth of 3, and the first 3D convolutional layer has 64 output channels; the second 3D convolutional layer has 128 output channels; the third 3D convolutional layer has 256 output channels; both the first pooling layer and the second pooling layer use max pooling operation, and the pooling window has a spatial dimension of 2×2 and a temporal dimension of 2; The global average pooling layer performs average pooling on the features output by the third 3D convolutional layer in the time dimension, compressing them into a fixed-length temporal feature vector.

3. The integrated circuit defect detection method fused with convolutional neural networks according to claim 1, characterized in that, After receiving multi-frame sequence image data, the process also includes a frame alignment step: The frame alignment unit uses a built-in timestamp parsing module to identify the timestamps embedded in each frame of image data. Based on these timestamps, frames that are out of order due to transmission delays are reordered, and missing frames are marked. When the intervals between adjacent frames are inconsistent during continuous acquisition, the frame alignment unit uses an interpolation resampling algorithm to normalize the time axis and outputs a temporally continuous and evenly spaced image sequence.

4. The integrated circuit defect detection method incorporating convolutional neural networks according to claim 1, characterized in that, The spatial neighborhood attention module calculates the attention weight distribution between the defect region and the surrounding neighborhood background to obtain a neighborhood enhancement feature representation, specifically including: The multi-frame temporal feature representation is segmented along the time dimension into independent frame-level spatial feature maps for each frame. For each feature point in the spatial feature map of each frame, a square or circular neighborhood is defined centered on the feature point. The square neighborhood has a side length of 5 or 7 pixels, and the circular neighborhood has a radius of 2 or 3 pixels. For feature points in the edge region, a zero-filling strategy is used to expand the boundary of the spatial feature map. Calculate the similarity between the feature vector of each central feature point and the feature vectors of each neighboring feature point within the neighborhood range. The similarity is calculated using either dot product or cosine similarity. Generate an initial attention weight distribution based on the similarity calculation results. Perform Softmax normalization on the initial attention weights within the neighborhood range to obtain normalized attention weights. The normalized attention weights are used to perform a weighted summation of the feature vectors of all neighboring feature points within the neighborhood range, and the result is used as the neighborhood enhanced feature representation of the central feature point. The influence of neighboring feature points on the central feature point follows the distance decay principle, with the basic influence of neighboring feature points that are closer in space being higher and the basic influence of neighboring feature points that are farther in space being lower.

5. The integrated circuit defect detection method fused with convolutional neural networks according to claim 1, characterized in that, The temporal feature extraction module uses a lightweight 3D convolutional network to extract the temporal feature vector: The lightweight 3D convolutional network uses depth-separable 3D convolutional operations to replace standard 3D convolutional operations. The depth-separable 3D convolutional operations decompose standard 3D convolution into two cascaded steps: spatial dimension depth convolution and temporal dimension pointwise convolution. The spatial dimension depthwise convolution performs spatial convolution operations independently on each input channel to extract the spatial features of each channel; the temporal dimension pointwise convolution integrates the temporal information of each channel through a 1×1×1 convolution kernel to achieve temporal feature fusion between channels. The lightweight 3D convolutional network includes a depth-separable 3D convolutional layer group, a downsampling layer group, and a global temporal aggregation layer. The depth-separable 3D convolutional layer group has 32 output channels, and the downsampling layer group uses a 3×3×3 max pooling operation with a pooling stride of 2.

6. The integrated circuit defect detection method incorporating convolutional neural networks according to claim 1, characterized in that, The defect detection head outputs the current frame defect detection results, including the category label, location information, and confidence score of each defect region in the current frame image; The network structure of the defect detection head consists of a fully connected layer and a cascaded Softmax activation function. The input dimension of the fully connected layer is consistent with the flattened dimension of the input feature map, and the output dimension of the fully connected layer is set to the total number of defect categories plus 1. The fully connected layer linearly maps the input high-dimensional neighborhood enhancement features into a feature vector with a dimension equal to the total number of categories. The Softmax activation function converts the feature vector into a probability distribution of the current frame image belonging to each category. The defect detection head outputs the defect detection result of the current frame based on the category corresponding to the maximum value in the probability distribution.

7. The integrated circuit defect detection method fused with convolutional neural networks according to claim 1, characterized in that, The defect evolution prediction head outputs the predicted defect evolution trend for future frames, specifically including: The defect evolution prediction head receives the enhanced feature map sequence of the current frame and multiple consecutive frames before the current frame from the neighborhood enhanced feature representation. The network structure of the defect evolution prediction head consists of a fully connected layer and a cascaded hyperbolic tangent activation function. The input dimension of the fully connected layer is consistent with the total dimension of the input multi-frame feature maps after flattening. The output dimension of the fully connected layer is set to 3. The hyperbolic tangent activation function maps each element in the 3-dimensional feature vector output by the fully connected layer to the interval between -1 and 1, resulting in 3 consecutive prediction confidence values. The three output dimensions correspond to three different evolution trends: the first dimension corresponds to the trend of defects shrinking, the second dimension corresponds to the trend of defects remaining unchanged, and the third dimension corresponds to the trend of defects expanding. The defect evolution prediction head outputs the prediction result of the defect evolution trend of the current frame to the future frame based on the trend category corresponding to the dimension with the largest value among the three confidence values.

8. The integrated circuit defect detection method fused with convolutional neural networks according to claim 1, characterized in that, Constructing a time-series consistency loss function specifically includes: Calculate the KL divergence value between the defect detection results of the previous frame and the defect detection results of the current frame on the category probability distribution, and use the KL divergence value as the inter-frame detection consistency loss; Inter-frame detection consistency loss Defined as: , in This refers to the inter-frame detection consistency loss. The total number of categories, For category indexing, The first predicted frame of the previous frame The probability values ​​of each category, The predicted number for the current frame The probability values ​​of each category; The inter-frame detection consistency loss Loss prediction with defect evolution The time-series consistency loss function is obtained by performing weighted fusion. : , in Let be the time-series consistency loss function. Predict the loss for the evolution of the aforementioned defect. These are the weighting coefficients for the inter-frame detection consistency loss. The weighting coefficients for the predicted loss of the defect evolution, and .

9. The integrated circuit defect detection method fused with convolutional neural networks according to claim 8, characterized in that, Constructing the temporal consistency loss function also includes the consistency constraint step between evolution prediction and detection results: The evolution trend prediction result output by the defect evolution prediction head is correlated and verified with the inter-frame detection consistency: when the prediction result is that the defect will expand, it is verified whether the probability of the defect category in the current frame detection result is higher than that in the previous frame; when the prediction result is that the defect will shrink, it is verified whether the probability of the defect category in the current frame detection result is lower than that in the previous frame; when the prediction result is that the defect will remain unchanged, it is verified whether the detection results of the two frames are consistent. If the direction of change in the detection result does not match the predicted trend, it is determined that there is a conflict between the detection result of the current frame and the temporal logic.

10. The integrated circuit defect detection method fused with convolutional neural networks according to claim 8, characterized in that, The evolution trend category labels predicted by the defect evolution prediction head are obtained by labeling training samples during the training phase: Collect integrated circuit surface image sequences containing known defect types as training samples. Each training sample contains a set of consecutive frame image sequences and corresponding fine annotations for each frame image. The annotation information includes the defect category, segmentation mask or bounding box of the defect region. For the training of the defect evolution prediction head, the evolution trend labels of defects in the sequence in subsequent real frames are labeled, and the change trend of the defect area in subsequent frames is quantified into three discrete labels: expansion, shrinkage or no change. The three discrete labels correspond to the three evolution trends represented by the three output dimensions of the defect evolution prediction head.