Semiconductor device defect detection method and device and electronic equipment

Abstract

The invention provides a semiconductor device defect detection method and device and electronic equipment, and belongs to the technical field of semiconductor device defect detection. The method comprises the steps of obtaining a to-be-detected image of the semiconductor device; inputting the to-be-detected image into a defect detection model to obtain a defect detection result output by the defect detection model; wherein the defect detection model is obtained through training based on a plurality of generated defect images of a sample semiconductor device and a sample defect data set constructed by a defect label corresponding to each generated defect image; the sample defect data set is obtained by utilizing a conditional generative adversarial network to perform data enhancement based on noise, a plurality of conditional vectors and a real defect image of the sample semiconductor device; the condition vector comprises defect condition parameters representing defect types of the sample semiconductor device. The method is used for solving the problem that the accuracy and the reliability are low when an existing method is used for defect detection.

Description

Technical Field

[0001] This invention relates to the field of semiconductor device defect detection technology, and more specifically to a semiconductor device defect detection method, a semiconductor device defect detection device, an electronic device, a machine-readable storage medium, and a computer program product. Background Technology

[0002] With the development of semiconductor technology, electronic devices made of semiconductor devices play an indispensable role in people's daily lives. Currently, defect detection in the manufacturing process of semiconductor devices (such as laterally diffused metal oxide semiconductors, LDMOS) mainly relies on traditional optical inspection and manual inspection methods. These methods are inefficient and inaccurate, failing to meet the stringent requirements of modern high-density integrated circuits for defect detection accuracy and speed.

[0003] Therefore, deep learning models have been developed for defect detection in existing methods. While deep learning models perform well in defect detection, improving efficiency and accuracy to some extent, their performance is highly dependent on large-scale, high-quality training data. However, in semiconductor manufacturing, defect data is often scarce and imbalanced, with extremely limited data for some defect types, making it difficult for models to effectively learn and identify these defects. This data imbalance limits the generalization ability of deep learning models, affecting the accuracy and reliability of defect detection. Summary of the Invention

[0004] The purpose of this invention is to provide a semiconductor device defect detection method, apparatus, and electronic device to solve the problem of low accuracy and reliability of existing methods for defect detection.

[0005] To achieve the above objectives, embodiments of the present invention provide a semiconductor device defect detection method, comprising: Acquire the image of the semiconductor device to be inspected; The image to be detected is input into the defect detection model to obtain the defect detection result output by the defect detection model; The defect detection model is trained on a sample defect dataset constructed from multiple generated defect images of the sample semiconductor device and the defect label corresponding to each generated defect image. The sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device. The conditional vectors include defect conditional parameters that characterize the types of defects in the sample semiconductor device.

[0006] Optionally, the sample defect dataset is obtained through the following steps: Obtain the noise and the true background image of the sample semiconductor device; An initial generative adversarial network is used to perform a first data augmentation based on the noise and the real background image to obtain a sample background dataset; the sample background dataset includes multiple sample background images of sample semiconductor devices; The conditional generative adversarial network is used to perform a second data augmentation based on the multiple sample background images, multiple conditional vectors, and real defect images of the sample semiconductor devices to obtain the sample defect dataset.

[0007] Optionally, the initial generative adversarial network includes a first generator and a first discriminator; the first data augmentation using the initial generative adversarial network based on the noise and the real background image to obtain the sample background dataset includes: The noise is repeatedly input into the first generator until a set number of times is reached, thereby obtaining the multiple sample background images output by the first generator; The first generator is trained based on the noise, the real background image, and the first discriminator; the first discriminator is used to distinguish between the real background image and the fake background image of the sample semiconductor device.

[0008] Optionally, the conditional generative adversarial network includes a second generator and a second discriminator; the second data augmentation using the conditional generative adversarial network based on the multiple sample background images, multiple conditional vectors, and the real defect images of the sample semiconductor devices to obtain the sample defect dataset includes: Repeat the following steps until you obtain the generated defect images corresponding to all condition vectors: The sample background image and the first condition vector are input into the second generator to obtain the generated defect image corresponding to the first condition vector output by the second generator; Wherein, the first condition vector is any one of the plurality of condition vectors; the second generator is trained based on the sample background image, the real defect image, the plurality of condition vectors and the second discriminator; the second discriminator is used to distinguish between the real defect image and the fake defect image corresponding to the condition vector.

[0009] Optionally, after inputting the image to be detected into the defect detection model and obtaining the defect detection result output by the defect detection model, the method further includes: If the frequency of occurrence of the first defect detection result exceeds a set threshold, the first defect detection result is input into the reinforcement learning model to obtain the process parameter adjustment strategy output by the reinforcement learning model. Wherein, the first defect detection result is any one of the defect detection results; the reinforcement learning model is trained based on the state space, action space and reward function corresponding to the sample defect detection result.

[0010] Optionally, the condition vector includes defect morphology and defect location; or the condition vector includes defect morphology, defect location, and defect size parameters.

[0011] On the other hand, embodiments of the present invention also provide a semiconductor device defect detection apparatus, comprising: The acquisition module is used to acquire the image of the semiconductor device to be inspected. The detection module is used to input the image to be detected into the defect detection model and obtain the defect detection result output by the defect detection model; The defect detection model is trained on a sample defect dataset constructed from multiple generated defect images of the sample semiconductor device and the defect label corresponding to each generated defect image. The sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device. The conditional vectors include defect conditional parameters that characterize the types of defects in the sample semiconductor device.

[0012] Optionally, the sample defect dataset is obtained through the following steps: Obtain the noise and the true background image of the sample semiconductor device; An initial generative adversarial network is used to perform a first data augmentation based on the noise and the real background image to obtain a sample background dataset; the sample background dataset includes multiple sample background images of sample semiconductor devices; The conditional generative adversarial network is used to perform a second data augmentation based on the multiple sample background images, multiple conditional vectors, and real defect images of the sample semiconductor devices to obtain the sample defect dataset.

[0013] Optionally, the initial generative adversarial network includes a first generator and a first discriminator; the first data augmentation using the initial generative adversarial network based on the noise and the real background image to obtain the sample background dataset includes: The noise is repeatedly input into the first generator until a set number of times is reached, thereby obtaining the multiple sample background images output by the first generator; The first generator is trained based on the noise, the real background image, and the first discriminator; the first discriminator is used to distinguish between the real background image and the fake background image of the sample semiconductor device.

[0014] Optionally, the conditional generative adversarial network includes a second generator and a second discriminator; the second data augmentation using the conditional generative adversarial network based on the multiple sample background images, multiple conditional vectors, and the real defect images of the sample semiconductor devices to obtain the sample defect dataset includes: Repeat the following steps until you obtain the generated defect images corresponding to all condition vectors: The sample background image and the first condition vector are input into the second generator to obtain the generated defect image corresponding to the first condition vector output by the second generator; Wherein, the first condition vector is any one of the plurality of condition vectors; the second generator is trained based on the sample background image, the real defect image, the plurality of condition vectors and the second discriminator; the second discriminator is used to distinguish between the real defect image and the fake defect image corresponding to the condition vector.

[0015] Optionally, after inputting the image to be detected into the defect detection model and obtaining the defect detection result output by the defect detection model, the method further includes: If the frequency of occurrence of the first defect detection result exceeds a set threshold, the first defect detection result is input into the reinforcement learning model to obtain the process parameter adjustment strategy output by the reinforcement learning model. Wherein, the first defect detection result is any one of the defect detection results; the reinforcement learning model is trained based on the state space, action space and reward function corresponding to the sample defect detection result.

[0016] Optionally, the condition vector includes defect morphology and defect location; or the condition vector includes defect morphology, defect location, and defect size parameters.

[0017] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described semiconductor device defect detection method.

[0018] On the other hand, the present invention also provides a machine-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described semiconductor device defect detection method.

[0019] On the other hand, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described semiconductor device defect detection method.

[0020] Through the above technical solution, the defect detection model of this invention is trained based on a sample defect dataset. The sample defect dataset is obtained by using a conditional generative adversarial network (GAN) to augment data based on noise, multiple conditional vectors, and real defect images of the sample semiconductor devices. This invention uses a GAN to augment data based on multiple conditional vectors, thereby generating generated defect images of specified types, thus expanding the training dataset and solving the problems of data imbalance and scarcity. This allows the defect detection model to learn different types of defects more comprehensively, improving the accuracy and reliability of defect detection.

[0021] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is one of the flowcharts of the semiconductor device defect detection method provided by the present invention; Figure 2 This is the second schematic flowchart of the semiconductor device defect detection method provided by the present invention; Figure 3 This is the third flowchart of the semiconductor device defect detection method provided by the present invention; Figure 4 This is the fourth flowchart of the semiconductor device defect detection method provided by the present invention; Figure 5 This is a schematic diagram of the semiconductor device defect detection device provided by the present invention; Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0024] In existing technologies, defect detection in the semiconductor device manufacturing process mainly relies on traditional optical inspection and manual inspection methods. Regarding manual inspection, operators need to examine each chip individually using microscopes or other optical equipment. This method is not only time-consuming but also susceptible to human factors, such as fatigue and subjective judgment bias, leading to inaccurate results. Furthermore, as chip complexity increases, the efficiency of manual inspection further declines, making it difficult to meet the demands of large-scale production. Therefore, defect detection relying on manual inspection is inefficient and prone to errors. As for manual optical inspection, automated inspection systems that rely on simple optical inspection techniques are typically used. This method cannot effectively distinguish complex defect types. Moreover, these systems lack intelligent analytical capabilities and cannot automatically classify and identify different defects, resulting in insufficient detection accuracy and efficiency. With the increasing complexity and density of integrated circuits, there is an urgent need for an automated system capable of rapid and efficient defect detection and classification.

[0025] In addition, existing methods have also utilized deep learning models for defect detection. While deep learning models perform well in defect detection, improving efficiency and accuracy to some extent, their performance is highly dependent on large-scale, high-quality training data. However, in semiconductor manufacturing, defect data is often scarce and imbalanced, with extremely limited data for some defect types, making it difficult for models to effectively learn and identify these defects. This data imbalance limits the generalization ability of deep learning models, affecting the accuracy and reliability of defect detection.

[0026] Therefore, the purpose of this invention is to provide a semiconductor device defect detection method, apparatus, and electronic device to solve the problem of low accuracy and reliability of existing methods for defect detection.

[0027] Method Implementation Examples Please refer to Figure 1 This invention provides a method for detecting defects in semiconductor devices, comprising: Step 100: Obtain the image of the semiconductor device to be tested.

[0028] Semiconductor devices can be various electronic devices that utilize the unique electrical properties of semiconductor materials to perform specific functions. For example, in one embodiment, the semiconductor device can be a laterally diffused metal-oxide semiconductor (LDMOS). LDMOS is widely used in power management circuits due to its high voltage resistance, high current driving capability, and low power consumption. In one embodiment, the electronic device can use virtual metering technology to collect real-time images and related parameters of the LDMOS device from the production line. For example, advanced image sensors and data acquisition systems can ensure that the acquired images of the LDMOS device under test have high accuracy and high real-time performance.

[0029] Furthermore, embodiments of the present invention can also preprocess the collected images to be detected. The preprocessing operations may include at least one of grayscale conversion, denoising, and normalization. In one embodiment, the preprocessing operations may include grayscale conversion, denoising, and normalization. Grayscale conversion can be implemented using weighted averaging or average averaging methods. Denoising can be implemented using spatial domain filtering (e.g., Gaussian filtering), frequency domain filtering (e.g., Fourier transform), etc. Normalization can be implemented using linear normalization methods, Z-score methods, etc. Embodiments of the present invention improve image quality, remove noise interference, and standardize data input by performing grayscale conversion, denoising, and normalization processing on the image, laying the foundation for subsequent defect detection.

[0030] Step 200: Input the image to be detected into the defect detection model to obtain the defect detection result output by the defect detection model.

[0031] The electronic device inputs the image to be inspected into the defect detection model and obtains the defect detection result output by the model. The defect detection result can be the defect type of the LDMOS device. This defect type can be distinguished by the defect's location, morphology (scratches, stains, etc.), size parameters, and severity. The electronic device classifies the detected defects into different types and generates a detailed defect report. The defect detection results provide a comprehensive quality analysis for the production line. The classification process utilizes a deep learning model to ensure accurate identification and classification of different defect types, which helps in subsequent quality control and improvement measures.

[0032] The defect detection model is trained on a sample defect dataset constructed from multiple generated defect images of sample semiconductor devices and the corresponding defect labels for each generated defect image. It should be noted that the sample defect dataset includes not only generated defect images but also original, real defect images of the sample semiconductor devices. The defect detection model can be built using various deep learning models, such as convolutional neural network (CNN) models (e.g., VGGNet, ResNet). In one embodiment, an electronic device uses a convolutional neural network to detect defects in a preprocessed image to be detected. The convolutional neural network model consists of multiple convolutional layers, pooling layers, and fully connected layers, capable of automatically extracting and learning spatial features from the image to be detected. Using these features, the convolutional neural network model can accurately identify the location and shape of defects, improving detection accuracy and efficiency, and meeting the needs of high-density integrated circuit manufacturing. High-quality defect samples are generated using conditional generative adversarial networks (GANs), addressing the problems of data imbalance and scarcity, thereby improving the detection performance of the defect detection model. This combined approach not only improves the efficiency and accuracy of defect detection but also lays the foundation for automated and intelligent detection systems, meeting the high standards of modern semiconductor manufacturing.

[0033] Please refer to Figure 2 The sample defect dataset (the expanded defect dataset) is obtained by using a conditional generative adversarial network (GAN) to augment data based on noise, multiple conditional vectors, and real defect images of the sample semiconductor devices. The conditional vectors include defect condition parameters characterizing the types of defects in the sample semiconductor devices. The conditional vectors include defect morphology and defect location; or the conditional vectors include defect morphology, defect location, and defect size parameters. By using a GAN to augment data based on multiple conditional vectors, generated defect images of a specified type are generated, thereby expanding the training dataset and solving the problems of data imbalance and scarcity. The GAN includes a generator and a discriminator. In one embodiment, using a GAN to augment data based on noise, multiple conditional vectors, and real defect images of the sample semiconductor devices includes: Repeat the following steps until all generated defect images corresponding to the condition vectors are obtained: input the noise and the first condition vector into the generator to obtain the generated defect images corresponding to the first condition vector output by the generator.

[0034] The first condition vector is any one of multiple condition vectors. The generator is trained based on the noise, the real defect image, and the discriminator; the discriminator is used to distinguish between the real defect image and the fake defect image corresponding to the condition vector.

[0035] The training process for the generator and discriminator is as follows: Step 11: Initialize the generator and discriminator parameters.

[0036] Step 12, Iterative Training: Step 121: Fix the generator and train the discriminator to distinguish between real and fake defect images. Optimize the generator weight parameters through backpropagation.

[0037] Step 122: Fix the discriminator and train the generator to produce more realistic defect images. Optimize the discriminator weight parameters through backpropagation.

[0038] Step 13, Convergence condition: Stop training when the discriminator can no longer distinguish between real defect images and fake defect images (accuracy close to 50%).

[0039] The hyperparameter settings are as follows: Learning rate: Generator set to 1e. -4 The discriminator is set to 1e. -5 To avoid training instability. Batch size: 16-32, balancing computational efficiency and gradient stability.

[0040] During generator training, the input consists of noise (e.g., random noise) and a conditional vector (defect shape, defect location, and defect size parameters). The generator concatenates or embeds the conditional vector with the noise and multiplies it. The generator outputs the generated defect image. The generator employs a U-Net architecture, where the encoder extracts background features, and the decoder combines the conditional information to generate a defect mask, which is then fused with the noise. The defect shape in the conditional vector can be represented by a one-hot vector, and the defect location and size parameters are normalized.

[0041] The discriminator is trained by taking the generated defect image / real defect image plus a conditional vector as input to the generator. The discriminator concatenates the conditional vector with the data along either the channel or feature dimension. The discriminator distinguishes between real and fake defect images corresponding to the conditional vector. The defect label for a real defect image is set to 1, and the defect label for a fake defect image is set to 0.

[0042] In this embodiment of the invention, the conditional vector y and noise z are input into the generator to generate a generated defect image G(z|y) that conforms to the conditional vector; simultaneously, the conditional vector y and the real defect image x / generated defect image G(z|y) are input into the discriminator. The objective function of the conditional generative adversarial network is based on the original generative adversarial network with the addition of a conditional loss, and the objective function formula is expressed as follows: ;Formula (1) in, This represents the loss function of the original generative adversarial network (adversarial loss, ensuring image realism). This represents conditional loss (e.g., cross-entropy loss, ensuring the correct defect type).

[0043] Therefore, this embodiment of the invention utilizes a conditional generative adversarial network to incorporate multiple conditional vectors into the defect image generation process, thereby guiding the generation of different types of defect images corresponding to these multiple conditional vectors. This expands the training dataset and solves the problems of data imbalance and scarcity. Consequently, it enhances the generalization ability of the defect detection model (convolutional neural network) and ensures consistent performance across different defect types.

[0044] The defect detection model in this embodiment of the invention is trained based on a sample defect dataset. This sample defect dataset is obtained by using a conditional generative adversarial network (GAN) to augment data based on noise, multiple conditional vectors, and real defect images of the sample semiconductor devices. This embodiment of the invention uses a GAN to augment data based on multiple conditional vectors, thereby generating generated defect images of specified types, thus expanding the training dataset and solving the problems of data imbalance and scarcity. This allows the defect detection model to learn different types of defects more comprehensively, improving the accuracy and reliability of defect detection. In other words, this embodiment of the invention successfully solves the problems of data imbalance and scarcity by using a GAN for data augmentation. By generating realistic defect samples, the defect detection model can be trained more comprehensively, significantly improving the accuracy and efficiency of defect detection. This innovative method meets the stringent requirements of modern high-density integrated circuit manufacturing, ensuring the precision and reliability of the production process.

[0045] In other aspects of embodiments of the present invention, the sample defect dataset is obtained through the following steps: Step 10: Obtain the noise and the real background image of the sample semiconductor device.

[0046] The electronic device acquires noise (random noise) and a true background image of the sample semiconductor device. The true background image of the sample semiconductor device can be an unlabeled background image readily available in an industrial setting. This embodiment of the invention fully utilizes readily available unlabeled background data in industrial settings, significantly reducing data acquisition costs.

[0047] Step 20: Using an initial generative adversarial network, perform the first data augmentation based on the noise and the real background image to obtain a sample background dataset; the sample background dataset includes multiple sample background images of sample semiconductor devices.

[0048] Please refer to Figure 3 The electronic device first utilizes the unsupervised generation capability of the initial generative adversarial network to synthesize defect-free sample background images on a large scale using the noise and the real background image, thereby obtaining a sample background dataset.

[0049] In one embodiment, the initial generative adversarial network includes a first generator and a first discriminator; the first data augmentation using the initial generative adversarial network based on the noise and the real background image to obtain a sample background dataset (expanded background dataset) includes: repeatedly inputting the noise into the first generator until a set number of times is reached to obtain the plurality of sample background images output by the first generator; The first generator is trained based on the noise, the real background image, and the first discriminator; the first discriminator is used to distinguish between the real background image and the fake background image of the sample semiconductor device.

[0050] The training process for the first generator and the first discriminator is as follows: Step 21: Initialize the parameters of the first generator and the first discriminator.

[0051] Step 22, Iterative Training: Step 221: Fix the first generator and train the first discriminator to distinguish between real and fake background images. Optimize the weight parameters of the first generator through backpropagation.

[0052] Step 222: Fix the discriminator and train the first generator to generate a more realistic background image. Optimize the weight parameters of the first discriminator through backpropagation.

[0053] Step 23, Convergence condition: Stop training when the first discriminator cannot distinguish between real background images and fake background images (accuracy close to 50%).

[0054] It should be noted that the loss function of the initial generative adversarial network can be constructed based on the cross-entropy loss.

[0055] Thus, in the first stage, this embodiment of the invention utilizes the unsupervised generation capability of the initial generative adversarial network to synthesize defect-free sample background images on a large scale, thereby facilitating the full utilization of massive defect-free image resources in the future.

[0056] Step 30: Using the conditional generative adversarial network, perform a second data augmentation based on the multiple sample background images, multiple conditional vectors, and the real defect images of the sample semiconductor devices to obtain the sample defect dataset.

[0057] Based on the generated sample background images, a conditional vector is constructed by combining semantic prior knowledge of defect categories (such as defect morphology, defect location distribution, and physical characteristics (defect size features)). The electronic device then uses the conditional generative adversarial network to perform a second data augmentation based on the multiple sample background images, multiple conditional vectors, and the real defect images of the sample semiconductor devices.

[0058] In one embodiment, the conditional generative adversarial network includes a second generator and a second discriminator; the second data augmentation using the conditional generative adversarial network based on the plurality of sample background images, multiple conditional vectors, and real defect images of the sample semiconductor devices to obtain the sample defect dataset includes: Repeat the following steps until you obtain the generated defect images corresponding to all condition vectors: The sample background image and the first condition vector are input into the second generator to obtain the generated defect image corresponding to the first condition vector output by the second generator; Wherein, the first condition vector is any one of the plurality of condition vectors; the second generator is trained based on the sample background image, the real defect image, the plurality of condition vectors and the second discriminator; the second discriminator is used to distinguish between the real defect image and the fake defect image corresponding to the condition vector.

[0059] The training process for the second generator and the second discriminator is as follows: Step 31: Initialize the parameters of the second generator and the second discriminator.

[0060] Step 32, Iterative Training: Step 321: Fix the second generator and train the second discriminator to distinguish between real defect images and fake defect images. Optimize the weight parameters of the second generator through backpropagation.

[0061] Step 322: Fix the second discriminator and train the second generator to generate more realistic defect images. Optimize the weight parameters of the second discriminator through backpropagation.

[0062] Step 33, Convergence condition: Stop training when the second discriminator cannot distinguish between real defect images and fake defect images (accuracy close to 50%).

[0063] The settings for hyperparameter adjustment can be found in the initial generative adversarial network described above, and will not be repeated here.

[0064] The second generator is trained on the input sample background image plus a conditional vector (defect shape, defect location, and defect size parameters). The second generator concatenates or embeds the conditional vector with the sample background image. The second generator outputs the generated defect image. The second generator employs a U-Net architecture: the encoder extracts background features, and the decoder combines the conditional information to generate a defect mask, which is then fused with the sample background image. Specifically, the defect shape in the conditional vector can be represented by a one-hot vector, and the defect location and size parameters are normalized.

[0065] The second discriminator is trained by taking the generated defect image / real defect image plus a conditional vector as input to the second generator. The second discriminator concatenates the conditional vector with the data along either the channel or feature dimension. The second discriminator distinguishes between real and fake defect images corresponding to the conditional vector. The defect label for a real defect image is set to 1, and the defect label for a fake defect image is set to 0.

[0066] The loss function of the conditional generative adversarial network can be found in formula (1), which will not be elaborated here. Therefore, this embodiment of the invention utilizes a conditional generative adversarial network to add multiple conditional vectors to the defect image generation process, thereby guiding the generation of different types of defect images corresponding to these multiple conditional vectors. This expands the training dataset and solves the problems of data imbalance and scarcity. This enhances the generalization ability of the defect detection model (convolutional neural network) and ensures consistent performance of the defect detection model across different defect types.

[0067] Compared to traditional data augmentation methods, the phased generation strategy of this invention not only fully utilizes massive amounts of defect-free image resources but also enhances the diversity and controllability of generated samples through fine-grained semantic control, effectively alleviating the data scarcity problem in few-sample learning scenarios. Furthermore, this method optimizes generation quality through adversarial training, ensuring that the synthetic defects are highly consistent with the distribution of real data, thereby significantly improving the generalization ability of deep learning models and enabling them to exhibit stronger robustness and adaptability in complex and ever-changing industrial inspection scenarios.

[0068] In other aspects of the embodiments of the present invention, after step 200, inputting the image to be detected into the defect detection model and obtaining the defect detection result output by the defect detection model, the method further includes: if the frequency of occurrence of the first defect detection result exceeds a set threshold, inputting the first defect detection result into the reinforcement learning model to obtain the process parameter adjustment strategy output by the reinforcement learning model.

[0069] Wherein, the first defect detection result is any one of the defect detection results; the reinforcement learning model is trained based on the state space, action space and reward function corresponding to the sample defect detection result.

[0070] The state space corresponding to the sample defect detection result can include the defect's morphology (linear, blocky), location (random distribution or specific region), and size parameters (crack length). The action space corresponding to the sample defect detection result includes lithography parameters, ion implantation parameters, etching parameters, etc. The reward function corresponding to the sample defect detection result can be a multi-objective optimization parameter. In one embodiment, the formula for the reward function is as follows: Rt = α * (1 - defect addition rate) + β * (parameter stability); Here, α and β are weighting coefficients that provide positive incentives when defects decrease and deduct points when process parameters fluctuate frequently.

[0071] The reinforcement learning model is trained using a proximal policy optimization algorithm based on the state space, action space, and reward function corresponding to the sample defect detection results. This results in a well-trained reinforcement learning model, which, according to this embodiment of the invention, dynamically adjusts the key process parameters of semiconductor devices using an adaptive optimization algorithm based on the reinforcement learning model.

[0072] To achieve real-time feedback of chip defect detection results and automatic optimization of manufacturing process parameters, this invention constructs an intelligent closed-loop control system. An image sensor first acquires images of the chip surface in real time, and a lightweight deep learning model deployed on the electronic device performs millisecond-level defect detection. The electronic device then uses multimodal data analysis methods to correlate features such as defect morphology, defect location, and defect size parameters with current process parameters, and dynamically adjusts key process parameters using an adaptive optimization algorithm based on the trained reinforcement learning model. In other embodiments, defect detection results can also be transmitted in real time to a central control platform via an Industrial Internet of Things (IIoT) protocol. The central control platform uses multimodal data analysis methods to correlate features such as defect morphology, defect location, and defect size parameters with current process parameters, and dynamically adjusts key process parameters using an adaptive optimization algorithm based on a reinforcement learning model. Furthermore, this invention can also establish a digital twin model to simulate the impact of process parameter adjustments on product quality, and combine statistical process control (SPC) methods to set safety boundaries for process parameter adjustments. In another embodiment, if the frequency of the first defect detection result exceeds a set threshold, the adaptive optimization algorithm of the reinforcement learning model will be triggered immediately. The optimized process parameters will be sent to the actuator of the production equipment through industrial protocols such as OPCUA. At the same time, the changes in the adjusted defect rate will be continuously monitored, forming a complete closed loop of "detection-analysis-adjustment-verification".

[0073] In other embodiments, to ensure system reliability, an anomaly handling mechanism can be designed to automatically roll back and alert engineers when process parameters exceed safety thresholds or fail to achieve expected results. This intelligent real-time feedback and optimization system can significantly improve production yield and equipment utilization while reducing manual intervention costs. Through real-time monitoring and feedback mechanisms, the production line can quickly respond to detection results, make process adjustments, and improve product quality and output. This closed-loop system ensures the efficiency and flexibility of the production process, continuously improving manufacturing levels.

[0074] In one embodiment, please refer to Figure 4 The specific implementation steps of this invention are as follows: 1. Data collection and preprocessing: (1) Use virtual metrology technology to obtain real-time images of LDMOS devices as images to be tested.

[0075] (2) Perform grayscale, noise reduction and normalization on the image to be detected to improve data quality.

[0076] 2. Step-by-step multimodal data augmentation: (1) Train a step-by-step multimodal two-layer generative adversarial network to generate high-quality generative defect images.

[0077] (2) Add these generated defect images to the sample defect dataset to balance the data distribution.

[0078] 3. Defect detection using convolutional neural networks: (1) Design and train a convolutional neural network (CNN) to process the preprocessed image.

[0079] (2) The CNN outputs the defect detection results (defect shape, defect location and defect size parameters) to ensure the accuracy of the detection.

[0080] 4. Defect Classification and Reporting: (1) Classify the detected defects and identify different types.

[0081] (2) Generate a detailed defect report, recording the shape, location, size parameters and severity of the defect.

[0082] 5. Real-time feedback and process optimization: The defect detection results are fed back to the production line in real time, and the manufacturing process parameters are automatically adjusted through reinforcement learning models to achieve production optimization.

[0083] The core of this invention lies in enhancing the defect detection capabilities and production efficiency of LDMOS devices through innovative technologies. First, virtual metrology technology is used to collect LDMOS device data in real time, ensuring high-precision real-time information during manufacturing. Second, a distributed multimodal generative adversarial network (GAN) is employed for step-by-step defect data augmentation, effectively addressing the problem of imbalanced data and generating more high-quality defect samples for model training. Next, a convolutional neural network (CNN) is designed and trained for accurate defect detection and classification, significantly improving detection accuracy and efficiency. Finally, by implementing real-time feedback of defect detection results, production process parameters can be quickly optimized and adjusted, ensuring continuous improvement in product quality. The combination of these technologies not only enhances automated inspection capabilities but also promotes the intelligent development of the entire production process.

[0084] The semiconductor device defect detection method of the present invention has the following advantages: 1. Highly efficient automated defect detection and classification system: This invention provides a highly efficient automated defect detection and classification system, significantly reducing reliance on manual inspection. Through advanced algorithms and intelligent detection processes, the system can quickly identify and classify various defects, greatly reducing the incidence of human error. Automated inspection not only saves a significant amount of time but also improves production efficiency, making the manufacturing process smoother and more reliable.

[0085] 2. Data augmentation using multimodal distributed generative adversarial networks: Generative adversarial networks (initial generative adversarial network and conditional generative adversarial network) are used for data augmentation in this embodiment of the invention, successfully solving the problems of data imbalance and scarcity. The conditional generative adversarial network can generate realistic defect samples, expanding the training dataset and enabling the model to learn different types of defects more comprehensively. This augmented dataset improves the model's robustness and generalization ability, thereby enhancing the overall performance of defect detection.

[0086] 3. High-precision defect detection and classification using convolutional neural networks (CNN): Convolutional neural networks (CNNs) play a crucial role in high-precision defect detection and classification. Through deep learning techniques, CNNs can automatically extract and analyze complex image features, significantly improving the accuracy of defect detection. This technology not only enhances detection precision but also rapidly adapts to different defect types, providing strong technical support for the production process.

[0087] 4. Real-time feedback of defect detection results and automatic adjustment of production process parameters: This invention provides real-time feedback of defect detection results, enabling rapid and automatic adjustment of production process parameters. This mechanism allows for timely optimization of the production process, ensuring continuous improvement in product quality. The real-time feedback system also enhances the flexibility and responsiveness of the production line, contributing to improved overall product quality and output, and achieving more efficient production management.

[0088] Device Examples Please refer to Figure 5 On the other hand, embodiments of the present invention also provide a semiconductor device defect detection apparatus, comprising: The acquisition module 501 is used to acquire the image of the semiconductor device to be inspected; The detection module 502 is used to input the image to be detected into the defect detection model and obtain the defect detection result output by the defect detection model; The defect detection model is trained on a sample defect dataset constructed from multiple generated defect images of the sample semiconductor device and the defect label corresponding to each generated defect image. The sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device. The conditional vectors include defect conditional parameters that characterize the types of defects in the sample semiconductor device.

[0089] The defect detection model in this embodiment of the invention is trained based on a sample defect dataset. This sample defect dataset is obtained by using a conditional generative adversarial network (GAN) to augment data based on noise, multiple conditional vectors, and real defect images of the sample semiconductor devices. This embodiment of the invention uses a GAN to augment data based on multiple conditional vectors, thereby generating generated defect images of specified types, thus expanding the training dataset and solving the problems of data imbalance and scarcity. This allows the defect detection model to learn different types of defects more comprehensively, improving the accuracy and reliability of defect detection.

[0090] Optionally, the sample defect dataset is obtained through the following steps: Obtain the noise and the true background image of the sample semiconductor device; An initial generative adversarial network is used to perform a first data augmentation based on the noise and the real background image to obtain a sample background dataset; the sample background dataset includes multiple sample background images of sample semiconductor devices; The conditional generative adversarial network is used to perform a second data augmentation based on the multiple sample background images, multiple conditional vectors, and real defect images of the sample semiconductor devices to obtain the sample defect dataset.

[0091] Optionally, the initial generative adversarial network includes a first generator and a first discriminator; the first data augmentation using the initial generative adversarial network based on the noise and the real background image to obtain the sample background dataset includes: The noise is repeatedly input into the first generator until a set number of times is reached, thereby obtaining the multiple sample background images output by the first generator; The first generator is trained based on the noise, the real background image, and the first discriminator; the first discriminator is used to distinguish between the real background image and the fake background image of the sample semiconductor device.

[0092] Optionally, the conditional generative adversarial network includes a second generator and a second discriminator; the second data augmentation using the conditional generative adversarial network based on the multiple sample background images, multiple conditional vectors, and the real defect images of the sample semiconductor devices to obtain the sample defect dataset includes: Repeat the following steps until you obtain the generated defect images corresponding to all condition vectors: The sample background image and the first condition vector are input into the second generator to obtain the generated defect image corresponding to the first condition vector output by the second generator; Wherein, the first condition vector is any one of the plurality of condition vectors; the second generator is trained based on the sample background image, the real defect image, the plurality of condition vectors and the second discriminator; the second discriminator is used to distinguish between the real defect image and the fake defect image corresponding to the condition vector.

[0093] Optionally, after inputting the image to be detected into the defect detection model and obtaining the defect detection result output by the defect detection model, the method further includes: If the frequency of occurrence of the first defect detection result exceeds a set threshold, the first defect detection result is input into the reinforcement learning model to obtain the process parameter adjustment strategy output by the reinforcement learning model. Wherein, the first defect detection result is any one of the defect detection results; the reinforcement learning model is trained based on the state space, action space and reward function corresponding to the sample defect detection result.

[0094] Optionally, the condition vector includes defect morphology and defect location; or the condition vector includes defect morphology, defect location, and defect size parameters.

[0095] The semiconductor device defect detection device includes a processor and a memory. The acquisition module 501 and the detection module 502 are stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0096] A processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured.

[0097] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0098] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a semiconductor device defect detection method. This method includes: acquiring a target image of the semiconductor device; inputting the target image into a defect detection model to obtain a defect detection result output by the defect detection model; wherein the defect detection model is trained based on a sample defect dataset constructed from multiple generated defect images of the sample semiconductor device and defect labels corresponding to each generated defect image; the sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device; the conditional vectors include defect condition parameters characterizing the types of defects in the sample semiconductor device.

[0099] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a machine-readable storage medium. When the computer program is executed by a processor, the computer is able to execute a semiconductor device defect detection method. The method includes: acquiring a semiconductor device image to be detected; inputting the image to be detected into a defect detection model to obtain a defect detection result output by the defect detection model; wherein the defect detection model is trained based on a sample defect dataset constructed from multiple generated defect images of the sample semiconductor device and defect labels corresponding to each generated defect image; the sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device; the conditional vectors include defect condition parameters characterizing the types of defects in the sample semiconductor device.

[0101] In another aspect, the present invention also provides a machine-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform a semiconductor device defect detection method. The method includes: acquiring a semiconductor device image to be detected; inputting the image to be detected into a defect detection model to obtain a defect detection result output by the defect detection model; wherein the defect detection model is trained based on a sample defect dataset constructed from multiple generated defect images of a sample semiconductor device and defect labels corresponding to each generated defect image; the sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device; the conditional vectors include defect condition parameters characterizing the types of defects in the sample semiconductor device.

[0102] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0103] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting defects in semiconductor devices, characterized in that, include: Acquire the image of the semiconductor device to be inspected; The image to be detected is input into the defect detection model to obtain the defect detection result output by the defect detection model; The defect detection model is trained on a sample defect dataset constructed from multiple generated defect images of the sample semiconductor device and the defect label corresponding to each generated defect image. The sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device. The conditional vectors include defect conditional parameters that characterize the types of defects in the sample semiconductor device.

2. The semiconductor device defect detection method according to claim 1, characterized in that, The sample defect dataset was obtained through the following steps: Obtain the noise and the true background image of the sample semiconductor device; An initial generative adversarial network is used to perform a first data augmentation based on the noise and the real background image to obtain a sample background dataset; the sample background dataset includes multiple sample background images of sample semiconductor devices; The conditional generative adversarial network is used to perform a second data augmentation based on the multiple sample background images, multiple conditional vectors, and real defect images of the sample semiconductor devices to obtain the sample defect dataset.

3. The semiconductor device defect detection method according to claim 2, characterized in that, The initial generative adversarial network includes a first generator and a first discriminator; the initial generative adversarial network is used to perform a first data augmentation based on the noise and the real background image to obtain a sample background dataset, including: The noise is repeatedly input into the first generator until a set number of times is reached, thereby obtaining the multiple sample background images output by the first generator; The first generator is trained based on the noise, the real background image, and the first discriminator; the first discriminator is used to distinguish between the real background image and the fake background image of the sample semiconductor device.

4. The semiconductor device defect detection method according to claim 2, characterized in that, The conditional generative adversarial network includes a second generator and a second discriminator; the second data augmentation using the conditional generative adversarial network is based on the multiple sample background images, multiple conditional vectors, and the real defect images of the sample semiconductor devices to obtain the sample defect dataset, including: Repeat the following steps until you obtain the generated defect images corresponding to all condition vectors: The sample background image and the first condition vector are input into the second generator to obtain the generated defect image corresponding to the first condition vector output by the second generator; Wherein, the first condition vector is any one of the plurality of condition vectors; the second generator is trained based on the sample background image, the real defect image, the plurality of condition vectors and the second discriminator; the second discriminator is used to distinguish between the real defect image and the fake defect image corresponding to the condition vector.

5. The semiconductor device defect detection method according to claim 1, characterized in that, After inputting the image to be detected into the defect detection model and obtaining the defect detection result output by the defect detection model, the method further includes: If the frequency of occurrence of the first defect detection result exceeds a set threshold, the first defect detection result is input into the reinforcement learning model to obtain the process parameter adjustment strategy output by the reinforcement learning model. Wherein, the first defect detection result is any one of the defect detection results; the reinforcement learning model is trained based on the state space, action space and reward function corresponding to the sample defect detection result.

6. The semiconductor device defect detection method according to any one of claims 1 to 5, characterized in that, The condition vector includes defect shape and defect location; or the condition vector includes defect shape, defect location, and defect size parameters.

7. A semiconductor device defect detection device, characterized in that, include: The acquisition module is used to acquire the image of the semiconductor device to be inspected. The detection module is used to input the image to be detected into the defect detection model and obtain the defect detection result output by the defect detection model; The defect detection model is trained on a sample defect dataset constructed from multiple generated defect images of the sample semiconductor device and the defect label corresponding to each generated defect image. The sample defect dataset is obtained by using a conditional generative adversarial network to perform data augmentation based on noise, multiple conditional vectors, and real defect images of the sample semiconductor device. The conditional vectors include defect conditional parameters that characterize the types of defects in the sample semiconductor device.

8. The semiconductor device defect detection device according to claim 7, characterized in that, The sample defect dataset was obtained through the following steps: Obtain the noise and the true background image of the sample semiconductor device; An initial generative adversarial network is used to perform a first data augmentation based on the noise and the real background image to obtain a sample background dataset; the sample background dataset includes multiple sample background images of sample semiconductor devices; The conditional generative adversarial network is used to perform a second data augmentation based on the multiple sample background images, multiple conditional vectors, and real defect images of the sample semiconductor devices to obtain the sample defect dataset.

9. The semiconductor device defect detection device according to claim 8, characterized in that, The initial generative adversarial network includes a first generator and a first discriminator; the initial generative adversarial network is used to perform a first data augmentation based on the noise and the real background image to obtain a sample background dataset, including: The noise is repeatedly input into the first generator until a set number of times is reached, thereby obtaining the multiple sample background images output by the first generator; The first generator is trained based on the noise, the real background image, and the first discriminator; the first discriminator is used to distinguish between the real background image and the fake background image of the sample semiconductor device.

10. The semiconductor device defect detection device according to claim 8, characterized in that, The conditional generative adversarial network includes a second generator and a second discriminator; the second data augmentation using the conditional generative adversarial network is based on the multiple sample background images, multiple conditional vectors, and the real defect images of the sample semiconductor devices to obtain the sample defect dataset, including: Repeat the following steps until you obtain the generated defect images corresponding to all condition vectors: The sample background image and the first condition vector are input into the second generator to obtain the generated defect image corresponding to the first condition vector output by the second generator; Wherein, the first condition vector is any one of the plurality of condition vectors; the second generator is trained based on the sample background image, the real defect image, the plurality of condition vectors and the second discriminator; the second discriminator is used to distinguish between the real defect image and the fake defect image corresponding to the condition vector.

11. The semiconductor device defect detection device according to claim 7, characterized in that, After inputting the image to be detected into the defect detection model and obtaining the defect detection result output by the defect detection model, the method further includes: If the frequency of occurrence of the first defect detection result exceeds a set threshold, the first defect detection result is input into the reinforcement learning model to obtain the process parameter adjustment strategy output by the reinforcement learning model. Wherein, the first defect detection result is any one of the defect detection results; the reinforcement learning model is trained based on the state space, action space and reward function corresponding to the sample defect detection result.

12. The semiconductor device defect detection apparatus according to any one of claims 7 to 11, characterized in that, The condition vector includes defect shape and defect location; or the condition vector includes defect shape, defect location, and defect size parameters.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the semiconductor device defect detection method according to any one of claims 1 to 6.

14. A machine-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the semiconductor device defect detection method according to any one of claims 1 to 6.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the semiconductor device defect detection method according to any one of claims 1 to 6.

Images