A machine vision-based intelligent detection and analysis method for semiconductor chip manufacturing

Abstract

The application discloses a kind of based on machine vision's semiconductor chip manufacturing intelligent detection analysis method, it is related to intelligent detection field, including, to multimodal data stream is preprocessed, and multimodal feature matrix is generated by space registration, according to multimodal feature matrix, using pre-training deep learning model, through convolution coding extraction feature and decoding generation defect area mask and trench parameter, output defect type probability, based on defect area mask, defect type probability and trench parameter, through long short-term memory network and multi-task learning model parallel processing, predict trench parameter deviation, generate process adjustment instruction, while analyzing defect area mask, defect type probability and multimodal feature extracted from multimodal feature matrix, generate defect traceability report;The application is through multi-task learning model, significantly improve the detection accuracy of complex defect, reduce etching deviation, reduce artificial dependence, improve automation level and production efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection, and in particular to an intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision. Background Technology

[0002] Defect detection and process optimization in semiconductor chip manufacturing are core aspects of integrated circuit production, directly impacting chip yield and performance. As semiconductor process nodes shrink to below 5 nanometers, the complexity of wafer surface microstructures and the diversity of defect types have significantly increased. In recent years, machine vision-based inspection technologies have developed rapidly and are widely used for defect identification and process parameter monitoring in wafer etching processes. Early methods primarily relied on single-modal data, using image processing algorithms to identify defects such as scratches and particles. In recent years, with advancements in deep learning technology, convolutional neural networks have been introduced into semiconductor defect detection, significantly improving the accuracy of feature extraction and classification. For example, models based on the U-Net architecture can achieve pixel-level defect segmentation and generate defect region masks; some studies further combine residual convolutional networks to enhance feature extraction capabilities. Furthermore, to address the need for process parameter optimization, time series analysis methods have begun to be used to predict trench depth and width deviations during the etching process.

[0003] While existing chip inspection technologies have made significant progress, there are still areas for improvement. Existing deep learning models can generate defect region masks and defect type probabilities, but their real-time prediction capabilities for trench parameters are insufficient, particularly lacking a dynamic adjustment mechanism that incorporates time-series analysis. This makes it impossible to generate timely process adjustment instructions to address parameter deviations during etching. Furthermore, current technologies are relatively weak in defect tracing, struggling to accurately identify process anomalies through multimodal feature analysis. This leads to reliance on manual experience for process optimization, resulting in low efficiency. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a machine vision-based intelligent inspection and analysis method for semiconductor chip manufacturing to solve the problem of insufficient real-time prediction capability of trench parameters.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a machine vision-based intelligent inspection and analysis method for semiconductor chip manufacturing, comprising: In the semiconductor wafer etching process, optical images, temperature distribution maps, and spectral curves of the wafer surface are acquired to generate multimodal data streams; The multimodal data stream is preprocessed, and a multimodal feature matrix is ​​generated through spatial registration; Based on the multimodal feature matrix, a pre-trained deep learning model is used to extract features through convolutional encoding and decode them to generate defect region masks and groove parameters, and output the probability of defect type. Based on defect region masks, defect type probabilities, and trench parameters, a parallel processing method using a long short-term memory network and a multi-task learning model is employed to predict trench parameter deviations and generate process adjustment instructions. Simultaneously, the defect region masks, defect type probabilities, and multimodal features extracted from the multimodal feature matrix are analyzed to generate a defect tracing report.

[0007] As a preferred embodiment of the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the generation of multimodal data stream refers to using an optical camera, a thermal imager, and a spectrometer to simultaneously acquire optical image data, temperature distribution map data, and spectral curve data of the wafer surface, adjust the light source to enhance contrast, calibrate the thermal imager to eliminate noise, correct the wavelength accuracy and light intensity normalization of the spectrometer, and generate a multimodal data stream.

[0008] As a preferred embodiment of the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the method utilizes an optical camera, a thermal imager, and a spectrometer to simultaneously acquire optical image data, temperature distribution map data, and spectral curve data of the wafer surface; adjusts the light source to enhance contrast; calibrates the thermal imager to eliminate noise; corrects the wavelength accuracy and light intensity normalization of the spectrometer; and generates a multimodal data stream. Specifically, The microstructure of the wafer surface is captured by an optical camera at a fixed frame rate. The light source is adjusted to enhance the image contrast, and the optical image data is output. By continuously measuring temperature with a thermal imager, calibrating the thermal imager to eliminate ambient temperature noise, and outputting temperature distribution map data; The spectrometer is used to collect reflected light signals, correct wavelength accuracy and normalized light intensity, and output spectral curve data. The time synchronization center alignment device triggers the signal, and combines optical image data, temperature distribution map data and spectral curve data to generate a multimodal data stream.

[0009] As a preferred embodiment of the intelligent detection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the preprocessing refers to receiving multimodal data streams, performing denoising, enhancement and correction on optical image data, upsampling and background subtraction on temperature distribution map data, and performing dimensionality reduction and normalization on spectral curve data.

[0010] As a preferred embodiment of the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the step of generating a multimodal feature matrix through spatial registration specifically involves: Using reference markers on the wafer surface, position matching is performed on the corrected optical image data and the upsampled temperature distribution map data; By minimizing the coordinate deviation, the spatial transformation matrix is ​​calculated, and the upsampled temperature distribution map data, spectral curve data, and corrected optical image data are fused into a feature vector and arranged to generate a multimodal feature matrix.

[0011] As a preferred embodiment of the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the step of extracting features and decoding to generate defect region masks and trench parameters by using a pre-trained deep learning model based on a multimodal feature matrix refers to inputting the multimodal feature matrix into a pre-trained deep learning model, extracting spatial features through stepwise convolution of the encoder to generate a high-level semantic feature map, generating a pixel-level output map through upsampling and feature fusion of the decoder, and outputting defect region masks and trench parameters.

[0012] As a preferred embodiment of the intelligent detection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the step of extracting spatial features through stepwise convolution of the encoder to generate a high-level semantic feature map, and generating a pixel-level output map through upsampling and feature fusion of the decoder, specifically involves: The encoder uses convolutional kernels to perform sliding calculations on the multimodal feature matrix, generating feature maps level by level, and finally forming a high-level semantic feature map; The decoder upsamples the high-level semantic feature map and fuses the feature maps of each level of the encoder through skip connections to generate a pixel-level output map.

[0013] As a preferred embodiment of the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the output defect region mask and trench parameters are specifically: Based on the pixel-level output image, a single-channel binary image is generated through a branch, and a defect region mask is formed after defect thresholding. Another branch generates a trench parameter map, identifies the target trench region from the trench parameter map, and extracts the mean values ​​of the depth and width of the target trench region to obtain the trench parameters. Multimodal features are extracted from the defect region mask, and the probability of the defect type is output through the classification layer.

[0014] As a preferred embodiment of the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the method involves: predicting trench parameter deviations and generating process adjustment instructions through parallel processing of a long short-term memory network and a multi-task learning model; simultaneously analyzing defect region masks, defect type probabilities, and multi-modal features extracted from a multi-modal feature matrix to generate a defect tracing report. Specifically, The trench parameters are input into the Long Short-Term Memory network, processed step by step through the internal memory units, and the predicted results of the trench parameter change trend are output. If the trench parameters in the prediction results exceed the tolerance threshold, the controller is used to calculate the compensation amount based on the prediction results, generate process adjustment instructions, and adjust them to the etching equipment control unit. Optical features, temperature features, and spectral features corresponding to the mask of the defect region are extracted from the multimodal feature matrix to form a defect-related multimodal feature subset; The defect region mask, defect type probability, and defect-related multimodal feature subsets are simultaneously input into the multi-task learning model; The multi-task learning model adopts a dual-branch structure of residual convolutional networks; The first output branch of the multi-task learning model is used to perform consistency verification on the probability of defect types, and the defect classification result is output. By analyzing the abnormal patterns in the multimodal feature subset related to defects through the second output branch of the multi-task learning model, the abnormal process factors that cause defects are identified, and defect tracing results are generated. The defect tracing results are compiled into a structured document, which serves as a defect tracing report.

[0015] As a preferred embodiment of the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision described in this invention, the pre-trained deep learning model specifically comprises: The deep learning model was trained using a semiconductor wafer dataset, which was divided into a training set and a validation set. The deep learning model is initialized using an encoder-decoder architecture. The encoder is based on a residual convolutional network, and the decoder uses an upsampling structure and skip connections. Weights from a general image dataset are loaded. The training set is input into the deep learning model in batches, and the loss functions of defect region mask, defect type probability and trench parameters are calculated. The weights of the deep learning model are updated by the optimizer until the loss value of the validation set is stable, thus completing the pre-training of the deep learning model.

[0016] The beneficial effects of this invention are as follows: By inputting a multimodal feature matrix into a pre-trained deep learning model, defect region masks, defect type probabilities, and trench parameters are generated, achieving pixel-level defect localization and process parameter prediction, thus improving detection resolution and parameter accuracy. Long Short-Term Memory (LSTM) networks are used to process time-series prediction biases of trench parameters, generating real-time process adjustment instructions. Simultaneously, a multi-task learning model analyzes defect region masks, defect type probabilities, and multimodal feature subsets to generate defect source tracing reports, achieving dynamic process optimization and process anomaly tracing. This is applied to real-time adjustment of etching parameters and identification of defect causes, significantly improving the detection accuracy of complex defects, reducing etching deviations, decreasing manual reliance, and increasing automation and production efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a machine vision-based intelligent inspection and analysis method for semiconductor chip manufacturing.

[0019] Figure 2 The flowchart for generating the multimodal feature matrix.

[0020] Figure 3 This is a flowchart of a deep learning detection model.

[0021] Figure 4 This is a flowchart for defect tracing. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a machine vision-based intelligent inspection and analysis method for semiconductor chip manufacturing, comprising the following steps: S1: In the semiconductor wafer etching process, optical images, temperature distribution maps, and spectral curves of the wafer surface are acquired to generate a multimodal data stream; Specifically, the steps include the following: Install a high-resolution optical camera equipped with an adjustable focus lens and a 365nm ultraviolet light source. Illuminate the wafer surface at a 45° angle. The high-resolution optical camera is vertically aligned with the target area on the wafer surface. Start the high-resolution optical camera to continuously shoot at a fixed frame rate to acquire grayscale images of the microstructure of the wafer surface and output optical image data. An infrared thermal imager is installed in the same observation area as the high-resolution optical camera. The infrared thermal imager is activated to continuously measure the temperature of the wafer surface, obtain the temperature distribution of the wafer surface, and output temperature distribution map data. A spectrometer is installed behind the quartz window of the etching cavity. The incident fiber of the spectrometer is aligned with the same wafer area as the high-resolution optical camera and infrared thermal imager. The spectrometer is then activated to collect the reflected light signal from the wafer surface, obtain the spectral response characteristics of the material, and output the spectral curve data. The trigger signals of the optical camera, infrared thermal imager, and spectrometer are connected to the same high-precision time synchronization center. The hardware triggering method enables the three devices to start data acquisition simultaneously under a unified clock reference, ensuring that each frame of optical image, each temperature distribution map, and each spectral curve has a strictly aligned timestamp. The edge computing node receives optical image data from a high-resolution optical camera, temperature distribution map data from an infrared thermal imager, and spectral curve data from a spectrometer. It then combines the optical image, temperature distribution map, and spectral curve acquired at the same time into a set of multimodal data units according to the timestamp. Multiple sets of continuously acquired multimodal data units are arranged in chronological order to form a synchronous data sequence containing optical images, temperature distribution maps, and spectral curves, and the output is a multimodal data stream.

[0026] S2: Preprocess the multimodal data stream and generate a multimodal feature matrix through spatial registration; Specifically, the steps include the following: Receives a multimodal data stream containing optical image data, temperature distribution map data, and spectral curve data. S2.1: Perform Gaussian filtering, adaptive histogram equalization and perspective transformation correction on the optical image data in sequence to remove image noise, enhance grayscale contrast and correct lens distortion, and output the corrected optical image data. S2.2: Perform bilinear interpolation on the temperature distribution map data to expand the resolution so that the spatial size of the expanded temperature distribution map data is consistent with the corrected optical image data. After interpolation, subtract the ambient background thermal radiation value to eliminate external thermal interference and output spatially aligned temperature distribution map data. S2.3: Perform principal component analysis to reduce the dimensionality of the spectral curve data, retain the principal components that represent more than 95% of the variance information of the original spectrum, convert the spectral data into a 50-dimensional spectral feature vector, and normalize each dimension of the feature vector after dimensionality reduction to output structured spectral feature data. S2.4: Using the reference markers on the wafer surface, position matching is performed on the corrected optical image data and the spatially aligned temperature distribution map data. By minimizing the coordinate deviation of the reference points, the spatial transformation matrix is ​​calculated to map the temperature distribution map data into the coordinate system of the corrected optical image data, thereby achieving spatial registration between the optical image and the temperature distribution map. S2.5: The spatially registered temperature distribution map data, structured spectral feature data and corrected optical image data are fused at the pixel level, and the optical gray value, temperature value and 50-dimensional spectral features corresponding to the same pixel position are combined into a 52-dimensional feature vector. The 52-dimensional feature vectors of all pixel locations are arranged according to a 4096×4096 spatial grid to form a two-dimensional data structure containing 52 feature channels at each location. The two-dimensional data structure is then output as a multimodal feature matrix.

[0027] S3: Based on the multimodal feature matrix, a pre-trained deep learning model is used to extract features through convolutional encoding and decode to generate defect region masks and groove parameters, and output the probability of defect type. Specifically, the steps include the following: Receive the multimodal feature matrix output from the previous stage; S3.1: Complete the pre-training of the deep learning model. The process involves using a semiconductor wafer dataset containing defect types such as scratches, particles, and pattern offsets. This dataset ensures it covers multiple wafer surface images, multimodal feature matrices, and annotations, including defect region masks, defect type probabilities, and trench parameters. The semiconductor wafer dataset is expanded using data augmentation techniques, including image rotation, flipping, and brightness adjustment, to improve the generalization ability of the deep learning model. A portion (e.g., 20%) is used as the validation dataset, and the remaining dataset is used as the training set.

[0028] The second step involves initializing the deep learning model using an encoder-decoder architecture. The encoder, built on a residual convolutional network, extracts spatial features step-by-step. The decoder employs an upsampling structure, fusing encoder features through skip connections to restore spatial resolution. Weights are initialized using a random distribution, and weights from a general image dataset are loaded before pre-training to accelerate convergence. A general image dataset, a broader dataset used for model initialization, typically refers to a general computer vision dataset like ImageNet, providing pre-trained weights.

[0029] The third step involves entering the training loop. The training set is fed into the deep learning model in batches. The encoder performs convolution operations to generate high-level semantic feature maps, and the decoder generates defect region masks and groove parameters through upsampling and feature fusion. Loss functions are calculated, including the cross-entropy loss of the defect region mask, the classification loss of the defect type probability, and the mean squared error loss of the groove parameters. The gradient descent optimizer is used to update the weights of the deep learning model. After each iteration, the performance on the validation set is evaluated, and the learning rate is adjusted to avoid overfitting.

[0030] During training, the accuracy and loss value of the deep learning model on the validation dataset are monitored. When the loss value is stable and the accuracy reaches the training threshold, training is stopped, the weights of the deep learning model are saved, and finally, the pre-training of the deep learning model is completed. The training threshold is set according to the task requirements.

[0031] S3.2: Input the multimodal feature matrix into the pre-trained deep learning model. First, use a fixed-size convolutional kernel in the first-level convolutional layer of the encoder to perform sliding calculation on the multimodal feature matrix to extract primary edge and texture features, forming the first-level feature map, the expression of which is: ; in, This indicates the location of the first-level feature map. Pixel value at that location, Indicates the position of the convolution kernel The weight value at that point, This indicates that the input multimodal feature matrix is ​​located at... Pixel value at; The first-level feature map is passed to the second-level convolutional layer, where a fixed-size convolutional kernel is used for sliding computation to extract higher-level shape and pattern features, forming the second-level feature map. This process is then advanced to subsequent convolutional layers, where each layer performs convolution operations based on the previous-level feature map, gradually extracting multi-scale spatial features, and finally generating a layer-by-layer abstract high-level semantic feature map.

[0032] The high-level semantic feature map output by the encoder is passed to the decoder. The decoder first performs upsampling on the high-level semantic feature map in the first-level upsampling layer to expand the feature map size. It then fuses the corresponding level feature maps of the encoder through skip connections to achieve feature fusion and restore local details, forming the first-level decoded feature map. The first-level decoded feature map is passed to the second-level upsampling layer for further upsampling and feature fusion to gradually restore spatial resolution, forming the second-level decoded feature map. This process is continued step by step to subsequent upsampling layers. Each layer performs upsampling and feature fusion based on the previous level decoded feature map, ultimately generating a pixel-level output map consistent with the input of the multimodal feature matrix.

[0033] S3.3: Based on the pixel-level output image, two parallel output branches are set at the end of the decoder. One branch processes the pixel-level output image through a convolutional layer to generate a single-channel binary image. Each pixel value in the generated single-channel binary image represents the probability of a defect existing at the corresponding location, and its expression is: ; in, This represents the probability of a regional defect, a floating-point number ranging from [0,1]. This represents the Sigmoid activation function. This represents the weight matrix of the convolutional layer. This indicates the pixel-level output image generated by the decoder at the position. The feature vector at that location, Indicates the bias term; Defect thresholding is applied to single-channel binary images, marking pixels with a defect probability higher than the defect threshold as defect regions, thus forming a defect region mask. The defect threshold is set according to the quality requirements of the semiconductor chip.

[0034] In another branch at the end of the decoder, a weighted calculation is performed on the pixel-level output image through a convolutional layer to generate a trench parameter map of the same size as the input image of the multimodal feature matrix. Each pixel location in the trench parameter map stores the predicted depth and width values. The target trench region is identified from the trench parameter map, and the mean of the predicted depth and width values ​​of the target trench region is extracted as the final trench parameter, expressed as: ; in, This represents the mean value of the trench parameters. This represents the total number of pixels within the target trench area. Indicates the target trench area. The trench parameter map shows the location. Trench parameters at the location; Based on the defect regions identified in the defect region mask, optical, temperature, and spectral features of the corresponding regions are extracted from the multimodal feature matrix. These optical, temperature, and spectral features are then input into a classification logistic regression layer. The confidence level of each type of defect is calculated using the softmax function, and the probabilities of scratch defects, particle defects, or pattern shift defects are output. The expression is as follows: ; in, This represents a probability vector for defect types. express Activation function This represents the weight matrix of the classification logistic regression layer. This represents the multimodal feature vector extracted from the defect region. This represents the bias vector of the classification layer.

[0035] S3.4: Summarize the defect region mask, defect type probability, and trench parameters. The defect region mask provides spatial distribution information of defects, the defect type probability provides confidence information of defect categories, and the trench parameters provide key geometric dimension information to form a complete detection result. Finally, output the defect region mask, defect type probability, and trench parameters.

[0036] Further explanation reveals that high-precision defect detection and multimodal data fusion significantly improve the quality and efficiency of semiconductor wafer etching processes. Utilizing an improved U-Net model and multimodal feature matrices, pixel-level defect localization and trench parameter prediction are achieved. Combined with accurate defect type classification, this provides a reliable basis for process optimization. Real-time inference and edge computing deployment support online monitoring, reducing downtime and improving production efficiency. Transfer learning and data augmentation reduce training costs and enhance cross-process adaptability and scalability. Multimodal fusion improves accuracy, reduces false positives and false negatives, effectively lowers scrap rates, increases chip yield, and significantly saves operating costs.

[0037] S4: Based on the defect region mask, defect type probability, and trench parameters, the trench parameter deviation is predicted through parallel processing using a long short-term memory network and a multi-task learning model. Process adjustment instructions are generated, and the defect region mask, defect type probability, and multi-modal features extracted from the multi-modal feature matrix are analyzed to generate a defect tracing report.

[0038] Specifically, the steps include the following: Receive defect region mask, defect type probability and trench parameters. The trench parameters here are the final trench parameters. The trench parameters mentioned later also refer to the final trench parameters. The trench parameters are stored in chronological order to form a time series data of trench parameters containing data collected at 0.1-second intervals over the past 20 seconds. S4.1: Training the Long Short-Term Memory (LSTM) network involves collecting historical trench parameter datasets from semiconductor wafer etching processes, covering various process conditions such as SF6 gas flow rate of 80-120 sccm and plasma power of 400-600 W. The average depth and width of the target trench region are recorded at each time point. Each time series is labeled with the actual depth and width values ​​for the next 5 seconds as supervision labels. The historical trench parameter dataset is divided into training and validation sets. The depth and width values ​​of each time series are normalized, mapping the depth values ​​to the 0-1 range, and the width values ​​are normalized similarly, generating normalized training and validation set trench parameter time series data.

[0039] A Long Short-Term Memory (LSTM) network with 128 hidden units is initialized, featuring input gates, forget gates, output gates, and memory units, running on an edge computing node. The input layer receives a 2D vector (normalized depth and width), the hidden layers output a 128D vector, and finally, a fully connected layer outputs a 2D prediction vector (future depth and width). Weights are initialized using a random distribution ranging from -0.1 to 0.1, and biases are initialized to 0. To accelerate convergence, pre-trained weights derived from publicly available datasets for general time series prediction tasks (e.g., industrial sensor data) are loaded. The learning rate is set to 0.001, the optimizer to Adam, and the loss function to mean squared error, used to measure the difference between the predicted depth and width and the actual values.

[0040] The training set trench parameter time series data is input into the Long Short-Term Memory (LSTM) network in batches. The LTM network processes the input vector at each time point, retaining new information through the input gate, discarding irrelevant information through the forget gate, and generating a 128-dimensional hidden state sequence through the output gate. At the last time point, the normalized depth and width values ​​for the next 5 seconds are predicted through a fully connected layer, generating a 50-row, 2-column prediction matrix. Each value in the prediction matrix is ​​denormalized to obtain the actual unit prediction value. The mean squared error loss between the actual unit prediction value and the actual label is calculated, and the weights and biases of the LTM network are updated using the Adam optimizer through backpropagation. After each batch of training is completed, the mean squared error loss and prediction bias are calculated on the validation set. Every 10 iterations, the prediction performance of the trench parameter time series data on the validation set is evaluated. If the mean squared error loss changes by less than 0.001 for 5 consecutive iterations, the learning rate is reduced by 10%, and the LTM network is trained for 100 iterations. Alternatively, training is stopped when the percentage of prediction biases with an absolute value less than 5 nanometers in the trench parameter time series data on the validation set reaches 95%. By comparing and validating the mean squared error loss of the set, the weights corresponding to the rounds with the lowest loss are selected and saved as a Long Short-Term Memory network weight file. The 5-nanometer value is based on the tolerance threshold required by the process specifications.

[0041] S4.2: Obtain real-time trench parameters from S3, calculate the average depth and average width of the target trench area, and collect the average depth and average width values ​​of the target trench area over the past 20 seconds using sensors in the etching equipment control unit to form a real-time data sequence. Record one average depth and average width value at each time point. Normalize the depth values ​​of the real-time data sequence, mapping them to the range of 0 to 1; similarly, map the width values. Combine the normalized average depth and average width values ​​to form the real-time trench parameter time series data.

[0042] S4.3: Load the trained Long Short-Term Memory (LSTM) network weight file and run it on the edge computing node. Input the real-time trench parameter time-series data into the LSM network step-by-step. The LSM network processes the input vector at each time point, generating a 128-dimensional hidden state sequence. The process is as follows: The LSM network processes the 2D input vector at each time point through input gates, forget gates, output gates, and memory units. It combines the 128-dimensional hidden state and memory state from the previous time step to update the current memory state and generate a 128-dimensional hidden state sequence. After processing at the last time point, the network predicts the normalized average depth and average width values ​​for the next 5 seconds through a fully connected layer, generating a prediction matrix. The prediction matrix is ​​then denormalized to obtain the actual unit prediction values, such as a depth of 505 nm and a width of 202 nm. The prediction results are stored as a trench parameter prediction sequence, containing the average depth and average width values.

[0043] S4.4: Receive the predicted trench parameter sequence for the next 5 seconds. For each time point, calculate the difference between the predicted average depth and average width and the target value to obtain the deviation value. For example, if the predicted average depth at a certain time point is 505.5 nm, then the average depth deviation is 5.5 nm. Check the deviations at each of the 50 time points. If the absolute value of the average depth deviation or the absolute value of the average width deviation at any time point exceeds 5 nm, record the deviation value and timestamp of that time point, triggering the process adjustment flow. Store all deviation values ​​that exceed the tolerance, forming a deviation sequence. If no deviation exceeds the tolerance, continue monitoring the next set of real-time trench parameter time series data. Transmit the deviation sequence to the proportional-integral-derivative controller to prepare for generating process adjustment instructions.

[0044] S4.5: The proportional-integral-derivative controller generates process adjustment instructions, specifically: Initialize the proportional-integral-derivative (PID) controller, setting the proportional gain, integral gain, and derivative gain. For example, set the proportional gain to 0.5, the integral gain to 0.1, and the derivative gain to 0.05. For each time point in the deviation sequence that exceeds the tolerance, calculate and process the average depth and average width deviations to obtain the control outputs for the average depth and average width. The expressions are as follows: ; in, Indicates control output. Indicates proportional gain. Indicates the deviation value. Indicates integral gain. Indicates a tiny increment. This represents the deviation value within the integration interval. Represents differential gain. Indicates the current time point, This represents the integral variable and the time point of the cumulative deviation.

[0045] The control output is converted into process adjustment instructions, such as "increase SF6 gas flow rate by 5%". The process adjustment instructions are in JSON format, including a timestamp and adjustment parameters. These JSON-formatted instructions are sent to the etching equipment control unit via Industrial Ethernet. After executing the adjustment, the etching equipment control unit records the adjustment timestamp and the effect, such as a reduction in average depth deviation to 2 nanometers, and stores this information in a log file.

[0046] S4.6: Generate a subset of multimodal features related to defects, specifically: Obtain the defect region mask and multimodal feature matrix from S3. Scan the defect region mask and extract the pixel coordinates with a value of 1 to form a set of defect region coordinates. Use the eight-neighborhood connected component analysis algorithm to divide independent defect regions, for example, into 3 regions. For each defect region, extract the 1D optical grayscale value, 1D temperature value, and 50D spectral features of the corresponding pixel from the multimodal feature matrix to generate a 52D feature vector. Calculate the average feature vector. Normalize the average feature vector, mapping the optical grayscale value and temperature value to the 0-1 interval, and the spectral features to the -1-1 interval. Combine the 52D average feature vectors of all defect regions to form a defect-related multimodal feature subset.

[0047] S4.7: Pre-train the multi-task learning model, specifically: (1) Collect defect data for semiconductor wafer etching processes, including defect region masks, defect type probabilities, and multimodal feature matrices. Each set of defect data is labeled with defect type, such as scratches, and process root causes, such as plasma inhomogeneity. Root cause categories include: plasma inhomogeneity, gas flow imbalance, temperature anomalies, material deposition anomalies, and equipment vibration. The defect data contains 10,000 sets, divided into an 80% training set (8,000 sets) and a 20% validation set (2,000 sets). Defect regions are extracted from the defect region masks, and independent regions are divided using eight-neighbor connected component analysis. A 52-dimensional feature vector is extracted from each independent region, the average feature vector is calculated, normalized to 0 to 1, and training and validation set defect data are generated. The training and validation set defect data are stored in the database of the edge computing nodes.

[0048] (2) Initialize the multi-task learning model, which includes a shared ResNet50 feature extractor and two output branches. Configure the shared ResNet50 feature extractor to receive a 52-dimensional average feature vector and output a 2048-dimensional shared feature vector. Configure the first output branch to receive a 2051-dimensional vector (2048-dimensional shared feature vector concatenated with 3-dimensional defect type probabilities), and output 3-dimensional verified defect type probabilities (scratches, particles, and pattern offsets) through a fully connected layer. Configure the second output branch to receive a 2048-dimensional shared feature vector and output 5-dimensional process root cause probabilities (plasma inhomogeneity, gas flow imbalance, temperature anomaly, material deposition anomaly, and equipment vibration) through a fully connected layer. The weights are initialized using a random distribution, ranging from -0.1 to 0.1, and the bias is initialized to 0. Load the ImageNet pre-trained weights of the shared ResNet50 feature extractor and apply them to the shared ResNet50 feature extractor to accelerate feature extraction convergence. Set the learning rate to 0.001, select the Adam optimizer, set the comprehensive cross-entropy loss function, including defect type classification loss and process root cause classification loss, with a weight of 0.5 for each, and store the initial parameters in the memory of the edge computing node.

[0049] (3) Process the training set defect data in batches. Input the 52-dimensional average feature vector into the shared ResNet50 feature extractor to generate a 2048-dimensional shared feature vector. The first output branch concatenates the 2048-dimensional shared feature vector with the 3-dimensional defect type probability to form a 2051-dimensional input vector. The output is a fully connected layer that outputs the verified defect type probability, for example, scratches at 92%. Calculate the cross-entropy loss between the verified defect type probability and the defect type label. The second output branch inputs the 2048-dimensional shared feature vector and outputs the process root cause probability through a fully connected layer, for example, plasma inhomogeneity at 70%. Calculate the cross-entropy loss between the process root cause probability and the process root cause label. Combine the cross-entropy losses of the two output branches, with each weighted at 0.5, and calculate the total loss. Use the Adam optimizer to update the weights and biases of the multi-task learning model through backpropagation. Every 10 iterations, calculate the total loss, defect classification accuracy, and process root cause classification accuracy on the validation set. If the total loss on the validation set changes by less than 0.001 for five consecutive rounds, the learning rate is reduced by 10%, with a minimum of 0.0001, to avoid overfitting. In each iteration, the total loss, defect classification accuracy, and process root cause classification accuracy of the training and validation sets are recorded. The multi-task learning model is trained for 100 rounds, or training stops when the defect classification accuracy on the validation set reaches 95% and the process root cause classification accuracy reaches 90%. The total loss on the validation set is compared, and the weights corresponding to the rounds with the lowest loss are selected to complete the training of the multi-task learning model. The weights are then saved as a multi-task learning model weight file.

[0050] S4.8: Multi-task learning models handle defect classification and root cause analysis, specifically as follows: The trained multi-task learning model is loaded, and a subset of multimodal features related to defects is obtained from S4.6, while defect region masks and defect type probabilities are obtained from S3. For each defect region's 52-dimensional average feature vector, a shared ResNet50 feature extractor is input to generate a 2048-dimensional shared feature vector. In the first output branch, the shared feature vector is concatenated with the defect type probabilities output from S3 to form a 2051-dimensional input vector. The validated defect type probabilities are output through a fully connected layer, for example, scratches 92%, particles 4%, and pattern offset 4%. The cross-entropy loss between the validated defect type probabilities and the defect type probabilities output from S3 is calculated to confirm the final defect classification result. Cross-entropy loss is a loss function commonly used in classification tasks to measure the difference between two probability distributions. In the second output branch, the shared feature vector is input, and a 5-dimensional process root cause probability is output through a fully connected layer, for example, plasma inhomogeneity 70% and gas flow imbalance 20%. Based on the historical defect database, the process root cause probabilities are mapped to specific process anomalies, such as "a 2-degree Celsius increase in temperature leads to abnormal material deposition." The defect classification results and process root cause probabilities after storage and verification are stored in the memory of the edge computing node to prepare for the generation of defect tracing reports. The historical defect database is established based on the actual production data of the semiconductor wafer etching process.

[0051] Collect and verify the defect classification results, process root cause probabilities, defect region masks, and trench parameters. Extract the bounding box coordinates of each defect region from the defect region mask, including center point coordinates, width, and height. Combine this with the historical defect database to generate improvement suggestions, such as "reducing the cavity temperature by 1 degree Celsius" corresponding to the "temperature anomaly" root cause. Compile all information into a JSON-formatted defect tracing report, including timestamp, defect location, defect type, confidence level, process root cause, root cause confidence level, and improvement suggestions. Store the defect tracing report in the database of the edge computing node and send it to the process control system via industrial Ethernet.

[0052] To further explain, by using a long short-term memory network and a multi-task learning model to process defect region masks, defect type probabilities, and trench parameters in parallel, high-precision trench parameter prediction and real-time adjustment are achieved, improving yield. At the same time, defects are accurately classified and process root causes are analyzed, generating defect traceability reports to guide process improvement, reducing defect rate, and providing an efficient, accurate, and economical quality control and optimization solution for semiconductor wafer etching processes.

[0053] This embodiment also provides a computer device applicable to the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as proposed in the above embodiment.

[0054] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0055] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the intelligent detection and analysis method for semiconductor chip manufacturing based on machine vision as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0056] In summary, this invention achieves pixel-level defect localization and process parameter prediction by inputting a multimodal feature matrix into a pre-trained deep learning model to generate defect region masks, defect type probabilities, and trench parameters, thereby improving detection resolution and parameter accuracy. A long short-term memory network is used to process time-series prediction biases of trench parameters, generating real-time process adjustment instructions. Simultaneously, a multi-task learning model analyzes defect region masks, defect type probabilities, and multimodal feature subsets to generate defect source tracing reports, enabling dynamic process optimization and process anomaly tracing. This is applied to real-time adjustment of etching parameters and identification of defect causes, significantly improving the detection accuracy of complex defects, reducing etching deviations, decreasing manual reliance, and increasing automation and production efficiency.

[0057] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A machine vision-based intelligent inspection and analysis method for semiconductor chip manufacturing, characterized in that: include, In the semiconductor wafer etching process, optical images, temperature distribution maps, and spectral curves of the wafer surface are acquired to generate multimodal data streams; The multimodal data stream is preprocessed, and a multimodal feature matrix is ​​generated through spatial registration; The generation of the multimodal feature matrix through spatial registration is specifically as follows: Using reference markers on the wafer surface, position matching is performed on the corrected optical image data and the upsampled temperature distribution map data; By minimizing the coordinate deviation, the spatial transformation matrix is ​​calculated, and the upsampled temperature distribution map data, spectral curve data, and corrected optical image data are fused into a feature vector and arranged to generate a multimodal feature matrix. Based on the multimodal feature matrix, a pre-trained deep learning model is used to extract features through convolutional encoding and decode them to generate defect region masks and groove parameters, and output the probability of defect type. Based on defect region masks, defect type probabilities, and trench parameters, the system uses a long short-term memory network and a multi-task learning model to process in parallel, predict trench parameter deviations, generate process adjustment instructions, and simultaneously analyze defect region masks, defect type probabilities, and multimodal features extracted from the multimodal feature matrix to generate a defect tracing report. The process of predicting trench parameter deviations and generating process adjustment instructions through parallel processing using a long short-term memory network and a multi-task learning model is as follows: The trench parameters are input into the Long Short-Term Memory network, processed step by step through the internal memory units, and the predicted results of the trench parameter change trend are output. If the trench parameters in the prediction results exceed the tolerance threshold, the controller is used to calculate the compensation amount based on the prediction results, generate process adjustment instructions, and adjust them to the etching equipment control unit. Optical features, temperature features, and spectral features corresponding to the mask of the defect region are extracted from the multimodal feature matrix to form a defect-related multimodal feature subset; The defect region mask, defect type probability, and defect-related multimodal feature subsets are simultaneously input into the multi-task learning model.

2. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 1, characterized in that: The generation of multimodal data stream refers to the simultaneous acquisition of optical image data, temperature distribution map data, and spectral curve data of the wafer surface using an optical camera, thermal imager, and spectrometer, adjusting the light source to enhance contrast, calibrating the thermal imager to eliminate noise, correcting the wavelength accuracy and light intensity normalization of the spectrometer, and generating a multimodal data stream.

3. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 2, characterized in that: The process involves using an optical camera, thermal imager, and spectrometer to simultaneously acquire optical image data, temperature distribution data, and spectral curve data of the wafer surface. The light source is adjusted to enhance contrast, the thermal imager is calibrated to eliminate noise, and the wavelength accuracy and light intensity normalization of the spectrometer are corrected to generate a multimodal data stream. The microstructure of the wafer surface is captured by an optical camera at a fixed frame rate. The light source is adjusted to enhance the image contrast, and the optical image data is output. By continuously measuring temperature with a thermal imager, calibrating the thermal imager to eliminate ambient temperature noise, and outputting temperature distribution map data; The spectrometer is used to collect reflected light signals, correct wavelength accuracy and normalized light intensity, and output spectral curve data. The time synchronization center alignment device triggers the signal, and combines optical image data, temperature distribution map data and spectral curve data to generate a multimodal data stream.

4. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 3, characterized in that: The preprocessing refers to receiving multimodal data streams, performing denoising, enhancement, and correction on optical image data, upsampling and background subtraction on temperature distribution map data, and performing dimensionality reduction and normalization on spectral curve data.

5. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 4, characterized in that: The process of extracting features and generating defect region masks and groove parameters by using a pre-trained deep learning model based on a multimodal feature matrix and then decoding them through convolutional encoding refers to inputting the multimodal feature matrix into a pre-trained deep learning model, extracting spatial features through step-by-step convolution by the encoder to generate a high-level semantic feature map, and generating a pixel-level output map through upsampling and feature fusion by the decoder, and outputting defect region masks and groove parameters.

6. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 5, characterized in that: The process involves extracting spatial features through step-by-step convolution by the encoder to generate a high-level semantic feature map, and then generating a pixel-level output map through upsampling and feature fusion by the decoder. Specifically: The encoder uses convolutional kernels to perform sliding calculations on the multimodal feature matrix, generating feature maps level by level, and finally forming a high-level semantic feature map; The decoder upsamples the high-level semantic feature map and fuses the feature maps of each level of the encoder through skip connections to generate a pixel-level output map.

7. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 6, characterized in that: The output defect region mask and trench parameters are specifically as follows: Based on the pixel-level output image, a single-channel binary image is generated through a branch, and a defect region mask is formed after defect thresholding. Another branch generates a trench parameter map, identifies the target trench region from the trench parameter map, and extracts the mean values ​​of the depth and width of the target trench region to obtain the trench parameters. Multimodal features are extracted from the defect region mask, and the probability of the defect type is output through the classification layer.

8. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 7, characterized in that: The analysis of defect region masks, defect type probabilities, and multimodal features extracted from the multimodal feature matrix generates a defect tracing report. Specifically, The multi-task learning model adopts a dual-branch structure of residual convolutional networks; The first output branch of the multi-task learning model is used to perform consistency verification on the probability of defect types, and the defect classification result is output. By analyzing the abnormal patterns in the multimodal feature subset related to defects through the second output branch of the multi-task learning model, the abnormal process factors that cause defects are identified, and defect tracing results are generated. The defect tracing results are compiled into a structured document, which serves as a defect tracing report.

9. The intelligent inspection and analysis method for semiconductor chip manufacturing based on machine vision as described in claim 8, characterized in that: The pre-trained deep learning model is specifically as follows: The deep learning model was trained using a semiconductor wafer dataset, which was divided into a training set and a validation set. The deep learning model is initialized using an encoder-decoder architecture. The encoder is based on a residual convolutional network, and the decoder uses an upsampling structure and skip connections. Weights from a general image dataset are loaded. The training set is input into the deep learning model in batches, and the loss functions of defect region mask, defect type probability and trench parameters are calculated. The weights of the deep learning model are updated by the optimizer until the loss value of the validation set is stable, thus completing the pre-training of the deep learning model.

Images