Image recognition-based photoresist defect detection method, system and platform

Abstract

The application provides a photoresist defect detection method, system and platform based on image recognition, belonging to the field of photoetching technology. The method comprises the following steps: based on the process parameters of the current printing batch and the material parameters of the photoresist, a corresponding defect detection model selection identifier and a detection parameter set are determined through a preset mapping relationship library; the original image data of the photoresist in the printing process of the current printing batch are subjected to feature extraction to obtain image feature data; a target defect feature extraction network model is loaded according to the defect detection model selection identifier, the image feature data are input into the target defect feature extraction network model to obtain an initial defect detection result; and the initial defect detection result is processed based on the detection parameter set to determine a target defect detection result. The application improves the accuracy, efficiency and reliability of defect detection.

Description

Technical Field

[0001] This application relates to the field of photolithography technology, and in particular to a method, system and platform for detecting photoresist defects based on image recognition. Background Technology

[0002] In the field of laser transfer printing, photoresist is a key consumable, and its quality directly determines the precision and yield of the final product. As printing processes develop towards higher precision and multi-batch production, the process parameters and photoresist material parameters of different batches vary significantly, greatly increasing the requirements for defect detection adaptability.

[0003] Existing detection methods mostly employ fixed thresholds and general feature extraction algorithms, which struggle to adapt to image feature differences caused by parameter variations. This results in a high rate of missed detection for low-contrast defects, and defects in critical printing areas are easily misjudged due to insufficient feature weights. Furthermore, manually adjusting detection parameters to match the needs of different production batches is not only inefficient but also fails to meet the real-time detection requirements of high-speed mass production scenarios. Detection delays often lead to the scrapping of entire batches of products, causing significant economic losses.

[0004] Therefore, there is an urgent need for a method, system, and platform for detecting photoresist defects based on image recognition, in order to improve the accuracy and efficiency of defect detection and ensure the stability of laser transfer lithography process and product quality. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a method and system for detecting photoresist defects based on image recognition.

[0006] A first aspect of this application provides a method for detecting photoresist defects based on image recognition, comprising: Based on the process parameters and photoresist material parameters of the current printing batch, a corresponding defect detection model selection identifier and detection parameter set are determined through a pre-set mapping relationship library. The mapping relationship library stores multiple sets of parameter combinations and their mapping relationships with the defect detection model identifier and detection parameter set. The parameter combinations are the process parameters of the printing batch and the photoresist material parameters. The detection parameter set includes a defect classification confidence threshold and a region attention weight map used for feature weighting. Feature extraction is performed on the original image data of the photoresist during the printing process of the current printing batch to obtain image feature data; According to the defect detection model, select the identifier to load the target defect feature extraction network model, input the image feature data into the target defect feature extraction network model, and obtain the initial defect detection result; Based on the set of detection parameters, the initial defect detection results are processed to determine the target defect detection results.

[0007] A second aspect of this application provides a photoresist defect detection system based on image recognition, comprising: The data loading module is used to determine the corresponding defect detection model selection identifier and detection parameter set based on the process parameters and photoresist material parameters of the current printing batch, through a preset mapping relationship library. The mapping relationship library stores the mapping relationships between multiple parameter combinations and the defect detection model identifier and detection parameter set. The parameter combinations are the process parameters of the printing batch and the photoresist material parameters. The detection parameter set includes a defect classification confidence threshold and a region attention weight map for feature weighting. The feature extraction module is used to extract features from the original image data of the photoresist during the printing process of the current printing batch to obtain image feature data; The initial detection module is used to select an identifier according to the defect detection model, load the target defect feature extraction network model, input the image feature data into the target defect feature extraction network model, and obtain the initial defect detection result. The detection processing module is used to process the initial defect detection results based on the detection parameter set to determine the target defect detection results.

[0008] A third aspect of this application provides a photoresist defect detection platform based on image recognition, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the above-described photoresist defect detection method based on image recognition.

[0009] In a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described image recognition-based photoresist defect detection method.

[0010] The beneficial effects of the image recognition-based photoresist defect detection method, system, and platform provided in this application are as follows: This application uses a mapping relationship library to match the detection model and parameter set corresponding to the current batch's process and photoresist material parameters, breaking through the limitations of traditional fixed threshold algorithms and achieving dynamic parameter adaptation, significantly reducing the risk of missed detections and misjudgments due to batch parameter differences. Specifically, the detection parameter set includes a regional attention weight map, which can specifically strengthen the feature weights of key regions. Combined with the defect classification confidence threshold, this effectively improves the detection accuracy of low-contrast defects such as micro-pinholes and shallow scratches, as well as defects in pattern edges and fine texture areas. Secondly, it eliminates the need for manual parameter adjustment, enabling rapid loading of adapted models and parameters to meet the real-time detection requirements of high-speed mass production in laser printing, avoiding the scrapping of entire batches of products due to detection delays, and improving detection efficiency and production stability. Attached Figure Description

[0011] Figure 1 A schematic flowchart of a photoresist defect detection method based on image recognition provided in an embodiment of this application; Figure 2 A structural block diagram of a photoresist defect detection system based on image recognition provided in an embodiment of this application; Figure 3 This is a schematic block diagram of a photoresist defect detection platform based on image recognition, provided in an embodiment of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] To make the purpose, technical solution, and advantages of this application clearer, the following will be described in conjunction with the appendix. Figure 1-3 The following is an explanation using specific examples.

[0014] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting photoresist defects based on image recognition, according to an embodiment of this application. The method includes: S101: Based on the process parameters and photoresist material parameters of the current printing batch, the corresponding defect detection model selection identifier and detection parameter set are determined through a preset mapping relationship library. The mapping relationship library stores the mapping relationship between multiple parameter combinations and the defect detection model identifier and detection parameter set. The parameter combinations are the process parameters of the printing batch and the photoresist material parameters. The detection parameter set includes a defect classification confidence threshold and a region attention weight map used for feature weighting. In this embodiment, the process parameters include the photoresist coating thickness, exposure time, exposure intensity, development time, and development temperature; the material parameters include the photoresist viscosity, solid content, refractive index, and chemical composition. These process and material parameters can be acquired in real-time by sensors on the production equipment or retrieved from the production management system. The mapping relationship library in this embodiment is constructed through historical detection data and machine learning training, storing the correspondence between multiple sets of parameter combinations and defect detection model identifiers and detection parameter sets, specifically including: Each set of parameters corresponds to the optimal defect detection model identifier, including different types of defect feature extraction network models such as convolutional neural networks, real-time object detection networks, and semantic segmentation networks, to adapt to different defect detection needs.

[0015] After inputting the process and material parameters for the current batch, the system queries the mapping database to determine the most suitable defect detection model selection identifier and detection parameter set for that batch. Historical production data includes process parameters, material parameters, and corresponding detection configurations.

[0016] The detection parameter set includes a defect classification confidence threshold and a region attention weight map used for feature weighting. The defect classification confidence threshold is a crucial basis for determining the existence and type of defect; different defect types correspond to different confidence thresholds. For example, the confidence threshold for pinhole defects is set to 0.75, for scratch defects to 0.68, and for bubble defects to 0.72.

[0017] The region attention weight map is generated based on the features of the photoresist pattern. Specifically, different weights are assigned to different regions of the image according to the probability and importance of defects in different areas during the photoresist printing process. In subsequent defect detection, regions with higher weights receive more attention. For example, differentiated weights are set for pattern edges, fine texture areas, and non-critical blank areas to further enhance the feature extraction effect of key regions.

[0018] The defect detection model selection identifier is used to uniquely specify the corresponding defect feature extraction network model. Different identifiers correspond to network models optimized for different defect types or different printing conditions.

[0019] S102: Extract features from the original image data of the photoresist during the printing process of the current printing batch to obtain image feature data.

[0020] In this embodiment, an image acquisition system consisting of a high-resolution industrial camera and a light source system is used to acquire images during the photoresist printing process of the current batch, obtaining raw image data. The raw image data includes the complete field of view of the photoresist coating area. The acquisition frequency is matched to the transmission speed of the printing production line, and environmental parameters such as temperature, humidity, and light intensity must be recorded synchronously during image acquisition. This embodiment also performs preprocessing operations on the raw image data, including removing blurred images caused by camera shake, cropping invalid edge areas, and unifying the image format to retain complete pixel information.

[0021] In this embodiment, image feature data is extracted based on the preprocessed image data; wherein, the image feature data includes: geometric features, grayscale features and texture features; geometric features include: the area, perimeter, aspect ratio, and circularity of the defect; grayscale features include: the grayscale difference, grayscale mean, and grayscale variance between the defect area and the normal area; texture features include: texture information of the defect area used to distinguish different defect types.

[0022] S103: Select the identifier according to the defect detection model and load the target defect feature extraction network model. Input the image feature data into the target defect feature extraction network model to obtain the initial defect detection result.

[0023] In this embodiment, the target defect feature extraction network model corresponding to the defect detection model selection identifier is trained with a large amount of labeled data. Different corresponding models learn the feature patterns of different types of defects. This embodiment utilizes the corresponding defect feature extraction network model to analyze and judge the corresponding image feature data, thereby obtaining the initial defect detection result and further accurately identifying defects in the photoresist. The initial defect detection result includes information such as the location, size, type of defect, and corresponding confidence level of the defect.

[0024] S104: Based on the detection parameter set, process the initial defect detection results to determine the target defect detection results.

[0025] In this embodiment, each candidate region of a defect in the initial defect detection result is screened according to the defect classification confidence threshold in the detection parameter set. Candidate regions with confidence scores lower than the corresponding defect classification confidence threshold are removed, and valid candidate regions are retained.

[0026] Based on the regional attention weight map, the confidence of the effective defect candidate regions is weighted and adjusted to obtain the weighted effective defect candidate regions; this further strengthens the identification priority of defects in key regions; the weighting calculation formula is: weighted confidence = initial confidence × corresponding regional weight value.

[0027] For the weighted and adjusted effective defect candidate regions, a non-maximum suppression algorithm is used to remove duplicate defect labels with an overlap degree higher than a preset overlap degree threshold to avoid duplicate counting; the preset overlap degree is based on historical experience statistics, for example, set to 50%.

[0028] Based on knowledge of photoresist printing process and historical defect data, the defect types of the weighted and adjusted valid defect candidate regions are verified a second time. If the corrected confidence score of the weighted and adjusted valid defect candidate regions is greater than or equal to the correction threshold, they are determined to be valid defects. In this embodiment, candidate regions that are below the corresponding defect classification confidence threshold can be marked as suspected defects for secondary verification; If the corrected confidence score of the effective defect candidate region after weighted adjustment is less than the correction threshold, it is also marked as a suspected defect and a secondary verification process is performed. For suspected defects in the marking, the following secondary verification methods are used: For suspected defect areas, high-resolution image features are re-extracted; Call other backup models from the mapping relationship library to re-detect the suspected defect area. If at least two models output that the area has a defect and the average confidence score is ≥0.6, the defect is confirmed as a valid defect; otherwise, it is judged as a false detection and the result is discarded.

[0029] In this embodiment, the finally determined defect information is integrated to form the target defect detection result, specifically including: Defect statistics: Total number of defects detected in the current batch, number and percentage of each type of defect, and defect distribution heatmap; Single defect details: defect ID, defect location, defect type, defect size, confidence score, and inspection timestamp; Batch quality assessment: Based on preset quality standards, the quality of the current batch of photoresist is rated, such as Grade A, Grade B, Grade C, or unqualified, and a quality assessment report is generated.

[0030] The target defect detection results are output in the form of visual reports and data files, and sent to the production control system. If the number or severity of defects exceeds the preset alarm threshold, a real-time alarm is triggered, prompting staff to make process adjustments or equipment maintenance.

[0031] As can be seen from the above, this application, based on a mapping relationship library, matches the detection model and parameter set corresponding to the current batch's process and photoresist material parameters. This overcomes the limitations of traditional fixed threshold algorithms, achieving dynamic parameter adaptation and significantly reducing the risk of missed detections and misjudgments due to batch parameter differences. Specifically, the detection parameter set includes a regional attention weight map, which can specifically strengthen the feature weights of key regions. Combined with the defect classification confidence threshold, this effectively improves the detection accuracy of low-contrast defects such as micro-pinholes and shallow scratches, as well as defects in pattern edges and fine texture areas. Furthermore, it eliminates the need for manual parameter adjustment, enabling rapid loading of adapted models and parameters to meet the real-time detection requirements of high-speed mass production in laser printing. This avoids the scrapping of entire batches of products due to detection delays, improving detection efficiency and production stability.

[0032] In one embodiment of this application, the method for constructing a mapping relation library includes: Cluster analysis was performed on the photoresist material parameters and process parameters of historical printing batches to generate multiple parameter combination clusters; Train a dedicated defect feature extraction network model for each parameter combination cluster and associate it with a unique defect detection model selection identifier; Based on the statistical distribution of various defects in the defect-annotated image data corresponding to each parameter combination cluster, the defect classification confidence threshold corresponding to that parameter combination cluster is determined. Based on the spatial location statistics of defects in the defect-annotated image data corresponding to each parameter combination cluster, a region attention weight map corresponding to that parameter combination cluster is generated. Establish a mapping relationship between each parameter combination cluster and the defect detection model identifier, defect classification confidence threshold, and regional attention weight map, and store them to form a mapping relationship library.

[0033] In this embodiment, a large number of historical printing batches' photoresist material parameters and process parameters are pre-collected, forming multi-dimensional data. A clustering algorithm, such as K-means clustering, is used to group parameters based on their similarity. During clustering, distances between different parameter combinations (e.g., Euclidean distance and Manhattan distance) are calculated, grouping highly similar parameter combinations into the same cluster, forming multiple parameter combination clusters. Each parameter combination cluster represents a type of printing batch with similar process and material characteristics.

[0034] In this embodiment, for each parameter combination cluster, the corresponding photoresist defect sample images and annotation information are collected as training data. Then, based on a deep learning framework, a dedicated defect feature extraction network model is trained for each cluster. The structure of the defect feature extraction network model can be adjusted and optimized according to the characteristics of defects within the cluster to improve the detection capability of defects under this type of parameter combination. After training, a unique defect detection model selection identifier is assigned to each dedicated model for subsequent model invocation and recognition.

[0035] This embodiment analyzes the distribution of defect samples for each parameter combination cluster, including the type of defect, its frequency of occurrence, and its location in the image. Based on these distribution characteristics, the corresponding detection parameter set for that cluster is determined. For example, for clusters with a high concentration of defect types, a targeted defect classification confidence threshold can be set; for clusters where defects have a high probability of appearing in certain areas, a corresponding regional attention weight map is generated to improve the detection sensitivity for these areas.

[0036] Specifically, based on the statistical distribution of various defects in the defect-annotated image data corresponding to each parameter combination cluster, the detection accuracy and false detection rate of each defect in that cluster are calculated to determine the optimal defect classification confidence threshold. For example, if there are 500 pinhole defect annotation samples in a certain parameter combination cluster, and the pinhole defect detection rate reaches 99% and the false detection rate is only 2% when the confidence threshold is set to 0.75, then the confidence threshold for pinhole defects in that cluster is determined to be 0.75. Similarly, based on the statistical distribution of other defects such as scratches and bubbles, corresponding thresholds are determined to form a set of defect classification confidence thresholds specific to that cluster.

[0037] Based on the spatial location statistics of defects in the defect-annotated image data corresponding to each parameter combination cluster, a heatmap analysis tool is used to statistically analyze the spatial distribution frequency of all defects in the photoresist image under that cluster. For clusters where defects have a higher probability of occurrence in certain areas, a corresponding regional attention weight map is generated. For example, a high weight coefficient is assigned to areas where the defect occurrence frequency is ≥5 times / mm², and a low weight coefficient is assigned to areas where the defect occurrence frequency is ≤1 time / mm², generating a regional attention weight map corresponding to that parameter combination cluster to ensure that the model is more sensitive to defect detection in high-risk areas.

[0038] This embodiment associates each parameter combination cluster with its corresponding defect detection model selection identifier and detection parameter set to generate a mapping relationship library. This mapping relationship library can be stored in the form of a database or dictionary for easy retrieval and retrieval. Furthermore, the mapping relationship library needs to be regularly updated and optimized based on new production data to ensure its accuracy and adaptability. In this embodiment, after obtaining the material and process parameters of the current printing batch, the similarity between these parameters and each parameter combination cluster in the mapping database is calculated. Through similarity comparison, the parameters of the current batch are matched to the most similar parameter combination cluster, i.e., the optimal parameter combination cluster. Then, the defect detection model selection identifier and detection parameter set associated with this optimal cluster are retrieved from the mapping database to provide configuration information for the subsequent defect detection process.

[0039] In summary, this embodiment performs cluster analysis on the parameters of historical printing batches to generate multiple parameter combination clusters, and trains a dedicated defect feature extraction network model for each cluster. This fully utilizes the features of historical data to train more targeted models for different parameter combinations, thereby improving the model's detection performance. Secondly, establishing a mapping relationship library between parameter combination clusters, defect detection model selection identifiers, and detection parameter sets allows for rapid matching of the optimal detection configuration based on the parameters of the current batch, reducing the time spent on model selection and parameter adjustment, and improving detection efficiency and adaptability.

[0040] In one embodiment of this application, the corresponding defect detection model selection identifier and detection parameter set are determined according to a preset mapping relationship library, including: Calculate the similarity between the process parameters and photoresist material parameters of the current printing batch and the center points of each parameter combination cluster in the mapping relation library to obtain a similarity ranking list; Select the mapping relationship corresponding to the parameter combination cluster with the highest similarity in the similarity ranking list; Based on the mapping relationship, the corresponding defect detection model is loaded, and the identifier and detection parameter set are selected.

[0041] In this embodiment, the process parameters and photoresist material parameters of the current printing batch are compared with the center point parameters of each parameter combination cluster in the mapping relation library to calculate similarity. The Euclidean distance algorithm is used to calculate parameter differences. Based on the Euclidean distance calculation results, a similarity value is generated, with the similarity value ranging from 0 to 1. The closer the value is to 1, the higher the similarity. The similarity values ​​are then sorted from high to low to generate a similarity ranking list. The parameter combination cluster with the highest similarity in the similarity ranking list is selected. If the similarity value of this cluster is greater than or equal to a preset similarity threshold, the mapping relationship corresponding to this cluster is directly determined. If the highest similarity value is less than the preset similarity threshold, the top 3 parameter combination clusters with the highest similarity are selected and proceed to the manual review stage. Operators can select suitable clusters based on historical detection experience to avoid matching failures due to parameters deviating from historical ranges. The preset similarity threshold is determined based on historical experience. Finally, based on the mapping relationship corresponding to the selected parameter combination cluster, the defect detection model selection identifier and detection parameter set associated with this cluster are retrieved from the mapping relationship library.

[0042] In summary, this embodiment selects the optimal matching model and parameters by calculating parameter similarity, which solves the model adaptation problem caused by parameter fluctuations in actual production, enhances the robustness of the detection method to parameter changes, and ensures that high detection accuracy can still be maintained when there are slight differences in parameters.

[0043] In one embodiment of this application, a method for generating a region attention weight map from a set of detection parameters includes: Extract the spatial distribution heatmap of historical defect samples in the photoresist image; Based on the defect type and frequency of occurrence, initial weighting coefficients are assigned to different regions in the spatial distribution heatmap. The association rules between weight coefficients and defect detection accuracy are learned based on the attention mechanism, the distribution of weight coefficients is optimized, and a regional attention weight map adapted to the current parameter combination cluster is generated.

[0044] In this embodiment, a large number of historical defect samples related to the current parameter combination cluster are pre-collected. These historical defect samples are mapped onto the corresponding photoresist images, and the frequency or probability of defects occurring at each spatial location is counted. The density of defect occurrence is represented by color intensity, generating a spatial distribution heatmap. Darker areas in the heatmap indicate a higher frequency of defects in the historical data. For different areas in the heatmap, initial weights are determined based on the type and frequency of defects occurring in that area. For example, areas with fatal defects and high frequency are assigned higher initial weights, while areas with minor defects and low frequency are assigned lower initial weights. The initial weight coefficient can be set between 0 and 1. This embodiment uses initial weight coefficients as input and defect detection accuracy as output. It trains on an attention mechanism model and learns the intrinsic correlation rules between weight coefficients and detection accuracy by analyzing the changes in defect detection results under different weight coefficient configurations—that is, which regions' weight adjustments significantly improve detection accuracy. Based on the correlation rules learned by the attention mechanism model, this embodiment iteratively optimizes the initial weight coefficients. During optimization, the weight coefficients of each region are continuously adjusted so that the adjusted weight coefficient distribution maximizes defect detection accuracy. After multiple optimizations, the final weight coefficient distribution is obtained and converted into a region attention weight map consistent with the spatial dimension of the photoresist image. This region attention weight map accurately represents the importance of each region in defect detection under the current parameter combination cluster.

[0045] In one embodiment of this application, feature extraction is performed on the original image data of the photoresist during the printing process of the current printing batch to obtain image feature data, including: The original image data of the photoresist during the printing process of the current printing batch is input into the pre-trained interference suppression generation network to generate foreground image data that suppresses background interference. Calculate the global optical flow field and its average motion amplitude between consecutive frames of the original image data; Based on the global optical flow field and average motion amplitude, image feature data is extracted from the foreground image data to obtain image feature data.

[0046] In this embodiment, the interference suppression generator network is trained by a recurrent adversarial generator network. Its training samples include a set of real image samples with interference and a corresponding set of reference images without interference. The generator learns the mapping from the image with interference to the image without interference, and the discriminator constrains the consistency of the distribution between the generated image and the reference image without interference.

[0047] In this embodiment, the interference suppression generation network is trained through a recurrent adversarial generation network. Its training samples include a set of real images with interference and a corresponding set of reference images without interference. The set of real images with interference includes, for example, images of scratches, bubbles, or dust on the photoresist surface. The set of reference images without interference is obtained by offline acquisition of standard photoresist surfaces using high-precision optical inspection equipment. During the training of the interference suppression generation network, the generator learns the mapping from images containing interference to images without interference. The discriminator constrains the realism of the generated image by minimizing the distribution difference between the generated image and the reference image without interference. Simultaneously, a cycle consistency loss ensures the preservation of image features until the peak signal-to-noise ratio (SNR) of the generated image exceeds a preset SNR threshold.

[0048] Specifically, the set of real-world images containing interference includes various interference scenarios that occur under different printing environments and process conditions, such as image blurring caused by changes in lighting, equipment vibration, and background noise. The corresponding set of reference images without interference consists of standard images that have been carefully processed to remove all interference factors. During training, the generator continuously learns the mapping rules from images containing interference to images without interference. By repeatedly adjusting the network parameters, the generator makes the generated images closely resemble the reference images without interference. The discriminator is responsible for distinguishing between the generated images and the reference images without interference. By continuously optimizing its own discrimination ability, it in turn constrains the generator, prompting the generated images to maintain a high degree of consistency with the reference images without interference in terms of distribution, thereby effectively suppressing various interferences in the original images.

[0049] In this embodiment, the global optical flow field and its average motion amplitude are calculated between consecutive frames of the original image data to obtain the average motion amplitude. Specifically, firstly, optical flow calculation methods, such as the Lucas-Kanade algorithm or the Farneback algorithm, are used to process two consecutive frames of the original image data to obtain the optical flow vector of each pixel, thereby obtaining the global optical flow field. This optical flow vector includes the pixel's motion velocity information in the horizontal and vertical directions. Secondly, the amplitude of the optical flow vectors of all pixels is calculated, and then the average amplitude of the optical flow vectors of all pixels is taken to obtain the average motion amplitude of the global optical flow field. The average motion amplitude characterizes the overall motion intensity of the photoresist in consecutive frames of the image.

[0050] In one embodiment of this application, image feature data is extracted from foreground image data based on the global optical flow field and average motion amplitude to obtain image feature data, including: If the motion amplitude is greater than the motion amplitude threshold, motion compensation is performed on the foreground image data based on the global optical flow field to obtain compensated image data; The compensated image data and the foreground image data are fused to obtain fused image data; Feature extraction is performed on the fused image data to obtain image feature data; If the motion amplitude is less than or equal to the motion amplitude threshold, then image feature extraction is performed directly on the foreground image data to obtain image feature data.

[0051] In this embodiment, the preset motion amplitude threshold is determined based on the statistical average motion amplitude of the photoresist during stable motion in historical printing processes, and can be dynamically adjusted according to the printing speed. The calculated average motion amplitude is compared with this threshold to determine whether motion compensation is needed. If the average motion amplitude is greater than the motion amplitude threshold, it indicates that the current photoresist motion state is unstable, which is prone to motion blur and affects the accuracy of defect detection. Motion compensation of the foreground image data based on the global optical flow field is required. Specifically, the reverse optical flow method is used. Based on the motion vector of each pixel in the global optical flow field, each pixel in the foreground image is mapped to its corresponding position in the reference frame (the previous clear image is selected as the reference frame). The blank pixels generated during the mapping process are supplemented by a bilinear interpolation algorithm to obtain compensated image data that eliminates motion blur. Subsequently, the compensated image data and the foreground image data are weighted and fused to obtain fused image data. For example, the weighting coefficient of the compensated image data is set to 0.7-0.8 to highlight the clear features after compensation; the weighting coefficient of the foreground image data is set to 0.2-0.3 to preserve the details of the original image. After weighted fusion, the fused image data is obtained that balances clarity and detail integrity. If the average motion amplitude is less than or equal to the motion amplitude threshold, it indicates that the current motion state of the photoresist is stable, the foreground image data has no obvious motion blur, no motion compensation is needed, and feature extraction can be performed directly on the foreground image data.

[0052] Based on the above, the feature extraction strategy is dynamically selected according to the motion amplitude. When the motion is intense, motion compensation and image fusion are used to reduce the impact of motion blur, while when the motion is gentle, features are extracted directly to ensure efficiency, thus achieving a balance between detection accuracy and efficiency.

[0053] In one embodiment of this application, feature extraction is performed on the fused image data to obtain image feature data, including: Feature extraction is performed on the fused image data to obtain spatial feature maps and temporal feature vectors; A spatial deformation field is generated based on temporal feature vectors, and an affine transformation is performed on the spatial feature map to generate a calibration feature map. Transform the temporal feature vectors into feature maps that match the spatial dimensions of the calibration feature map; The calibration feature map and the feature map are concatenated by channel to obtain image feature data.

[0054] In this embodiment, feature extraction is performed on the fused image data to obtain image feature data. The specific steps include: A three-dimensional convolutional neural network is used to extract features from fused image data of consecutive frames. The size of the convolutional kernel can be set to 3×3×3. The first two dimensions correspond to the spatial dimension of the image, and the third dimension corresponds to the temporal dimension. Through forward propagation, spatial feature maps and temporal feature vectors are output to capture the spatial detail features and temporal change features of the image, respectively. Based on temporal feature vectors, a fully connected layer maps them into a spatial deformation field including affine transformation parameters, where the deformation field dimension and spatial features are defined. Figure 1 To achieve this, a bilinear sampling algorithm is employed to perform an affine transformation on the spatial feature map based on the spatial deformation field. This corrects for feature offsets caused by photoresist motion, generating a dimension-invariant calibration feature map to ensure spatial alignment accuracy of features across different frames. The temporal feature vector is upsampled through a transposed convolutional layer, transforming it into a feature map with the same spatial dimension as the calibration feature map. This achieves dimensional matching between temporal and spatial features, facilitating subsequent feature fusion.

[0055] By employing channel stitching, the calibration feature map and the transformed temporal feature map are merged along the channel dimension to obtain image feature data. This image feature data includes both spatial details and temporal variation information, thereby improving the completeness and discriminability of defect features.

[0056] In summary, this embodiment effectively solves the interference of image deformation on feature extraction by fusing spatial and temporal features and affine transformation calibration. Secondly, converting temporal vectors into spatial feature maps and performing channel stitching enhances the expressive power of features and improves the accuracy of defect identification.

[0057] In one embodiment of this application, image feature extraction is performed on foreground image data to obtain image feature data, including: Perform convolution operations on the foreground image data to generate an initial feature map; The initial feature map is input into the channel attention module to calculate the weight coefficients of each feature channel; The initial feature map is channel-weighted based on the weighting coefficients to obtain the optimized feature map; The optimized feature map is resampled to generate image feature data.

[0058] In this embodiment, a two-dimensional convolutional neural network is used to perform multiple rounds of convolution operations on a single frame of foreground image data to generate an initial feature map; The initial feature map is input into the channel attention module. First, global average pooling is used to compress each channel of the initial feature map, squeezing out channel information. Then, two fully connected layers are used to incentivize the importance of each channel, resulting in weight coefficients for each feature channel. The weight coefficients range from 0 to 1, with larger values ​​indicating a higher contribution of the corresponding channel to defect detection. Finally, the initial feature map is weighted according to the weight coefficients to obtain an optimized feature map with unchanged dimensions, strengthening effective defect feature channels and suppressing interference from redundant channels. Next, the optimized feature map is resampled, and bilinear interpolation is used to adjust the spatial dimension of the optimized feature map to match the size required by the subsequent defect detection model input. L2 normalization is used to eliminate differences in feature dimensions. Furthermore, to maintain consistency with the feature dimensions of the fused image data, convolutional layers are used to adjust the number of channels in the optimized feature map, ultimately generating image feature data and ensuring compatibility with subsequent model operations.

[0059] In summary, this embodiment utilizes the channel attention module to weight the feature channels, thereby strengthening the weight of key defect features and suppressing the interference of irrelevant features. By optimizing the feature data structure through feature resampling, the processing efficiency and detection accuracy of the subsequent defect detection model are improved.

[0060] In one embodiment of this application, feature resampling is performed on the optimized feature map to generate image feature data, including: The optimized feature map is decomposed into high-frequency components and low-frequency components; A dense sampling strategy is used to sample high-frequency components to obtain the first sampling result, wherein the sampling interval of the dense sampling strategy is the first sampling interval; A sparse sampling strategy is used to sample low-frequency components to obtain a second sampling result; wherein the sampling interval of the sparse sampling strategy is the second sampling interval; and the second sampling interval is greater than the first sampling interval. The first and second sampling results are weighted and fused to generate image feature data.

[0061] In this embodiment, the optimized feature map is decomposed into high-frequency and low-frequency components. The high-frequency components correspond to details such as edges and textures in the feature map, while the low-frequency components correspond to smoothing information such as the overall grayscale distribution and large background areas. During the decomposition process, the optimized feature map is blurred using Gaussian filtering to obtain the low-frequency components. The low-frequency components are then subtracted from the original optimized feature map to obtain the high-frequency components, ensuring that the original optimized feature map can be completely recovered after the two components are superimposed.

[0062] This embodiment employs a dense sampling strategy for high-frequency components because these components contain critical details about defects that need to be preserved at high resolution. For example, the first sampling interval is set to 1-2 pixels, meaning one feature point is sampled every 1-2 pixels. A bilinear interpolation algorithm is used for sampling to ensure that the detail loss rate of the high-frequency components after sampling is ≤5%, thus obtaining the first sampling result.

[0063] A sparse sampling strategy is adopted for low-frequency components because low-frequency components have smooth information and high redundancy, which can reduce the sampling density and improve efficiency. For example, the second sampling interval is set to 4-8 pixels, and the second sampling interval is greater than the first sampling interval. The specific interval is dynamically adjusted according to the optimized feature map size. For example, when the feature map size is 512×512, the second sampling interval is set to 8 pixels. The bilinear interpolation algorithm is also used for sampling to obtain a second sampling result with the same dimension as the first sampling result. This improves sampling efficiency while ensuring feature integrity. In this embodiment, weighting coefficients are set according to the importance of the components. Based on the set weighting coefficients, the first sampling result and the second sampling result are weighted and fused to obtain image feature data. Among them, the weighting coefficient of the high-frequency component corresponding to the first sampling result can be set to 0.6-0.7 to highlight the defect details, and the weighting coefficient of the low-frequency component corresponding to the second sampling result is set to 0.3-0.4 to preserve the overall grayscale features.

[0064] In this embodiment, the process of fusing the first sampling result and the second sampling result can be carried out by data stitching. The first sampling result and the second sampling result are combined together in sequence to generate complete image feature data including high-frequency detail information and low-frequency background information. This image feature data can comprehensively and efficiently represent the characteristics of the photoresist, providing reliable feature support for subsequent defect detection.

[0065] In the above embodiment, dense sampling is used to preserve detailed features for high-frequency components, while sparse sampling is used to reduce redundant information for low-frequency components. This reduces the amount of data while ensuring the integrity of defect features, achieving a balance between high efficiency and accuracy in feature extraction.

[0066] In one embodiment of this application, a target defect feature extraction network model is loaded according to a defect detection model selection identifier, and image feature data is input into the target defect feature extraction network to obtain an initial defect detection result, including: Image feature data is input into the multi-layer convolutional module of the target defect feature extraction network model, and spatial hierarchical features are extracted step by step to obtain the final hierarchical feature map; The channel attention module is used to recalibrate the channel dimension of the final hierarchical feature map to generate a recalibrated feature map. By applying the regional attention weight map from the detection parameter set, the recalibration feature map is spatially weighted to generate a weighted feature map. The weighted feature map is input into the classification and regression subnetwork, which simultaneously outputs the probability distribution of defect categories and the coordinates of bounding boxes to generate initial defect detection results.

[0067] In this embodiment, the multi-layer convolution module consists of multiple convolutional layers, activation functions, and pooling layers. Through layer-by-layer convolution operations, it continuously extracts higher-level and more abstract spatial features from the image feature data. Shallow convolutional layers extract basic features such as edges and textures, while deep convolutional layers integrate these basic features to obtain more discriminative and complex features, ultimately outputting a final hierarchical feature map containing rich defect feature information. This embodiment uses a channel attention module to first perform global average pooling on each channel of the final hierarchical feature map to obtain the global features of each channel. Then, attention weights for each channel are calculated using a fully connected layer and a sigmoid activation function. The attention weights of each channel are then multiplied element-wise with the features of the corresponding channel in the final hierarchical feature map, strengthening the features of important channels and suppressing the features of secondary channels, thus achieving channel-dimensional recalibration and generating a recalibrated feature map. Finally, the region attention weight map is multiplied pixel-wise with the recalibrated feature map, enhancing the features of regions with higher weights in the recalibrated feature map, highlighting defect information in these regions, and generating a weighted feature map that better meets actual detection needs.

[0068] The weighted feature map is input into the classification and regression subnetwork, which simultaneously outputs the probability distribution of defect categories and bounding box coordinates to generate initial defect detection results. The classification and regression subnetwork includes a classification branch and a regression branch. The classification branch outputs the probability distribution of various defects present in the photoresist through fully connected layers and a softmax activation function, showing the probability of each defect type. The regression branch outputs the coordinate information of the defect bounding boxes through fully connected layers, used to locate the defects in the image. This embodiment combines the results from the classification and regression branches to generate initial defect detection results that include defect type probabilities and location information.

[0069] In this embodiment, spatial hierarchical features are extracted step-by-step through multi-layer convolutional modules. This allows for the gradual mining of defect features from low to high levels from image feature data, forming a more discriminative final hierarchical feature map, laying a solid foundation for subsequent defect recognition. Channel dimension recalibration of the final hierarchical feature map using a channel attention module automatically learns and strengthens the weights of feature channels more critical to defect detection, suppressing interference from irrelevant channels and improving the quality of the feature map. Secondly, spatial weighting is performed based on the regional attention weight map within the detection parameter set. This assigns different weights to different regions in the feature map according to the spatial distribution characteristics of defects under specific parameter combinations, focusing on high-incidence defect areas and improving the targeting of defect detection. Finally, a classification and regression sub-network is used to simultaneously output the defect category probability distribution and bounding box coordinates, achieving integrated processing of defect recognition and localization, simplifying the detection process, and improving the efficiency of generating initial defect detection results.

[0070] Corresponding to the image recognition-based photoresist defect detection method in the above embodiments, Figure 2 This is a structural block diagram of a photoresist defect detection system based on image recognition, provided as an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The image recognition-based photoresist defect detection system 20 includes: a data loading module 21, a feature extraction module 22, an initial detection module 23, and a detection processing module 24.

[0071] in, The data loading module 21 is used to determine the corresponding defect detection model selection identifier and detection parameter set based on the process parameters and photoresist material parameters of the current printing batch, through a preset mapping relationship library. The mapping relationship library stores the mapping relationship between multiple sets of parameter combinations and defect detection model identifiers and detection parameter sets. The parameter combinations are the process parameters of the printing batch and the photoresist material parameters. The detection parameter set includes a defect classification confidence threshold and a regional attention weight map for feature weighting. The feature extraction module 22 is used to extract features from the original image data of the photoresist during the printing process of the current printing batch to obtain image feature data; The initial detection module 23 is used to select an identifier according to the defect detection model, load the target defect feature extraction network model, input image feature data into the target defect feature extraction network model, and obtain the initial defect detection result. The detection processing module 24 is used to process the initial defect detection results based on the detection parameter set to determine the target defect detection results.

[0072] See Figure 3 , Figure 3This is a schematic block diagram of a photoresist defect detection platform based on image recognition, provided as an embodiment of this application. Figure 3 The image recognition-based photoresist defect detection platform 300 shown in this embodiment may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the above-described device embodiments, for example... Figure 2 The functions of the data acquisition module 21, data generation module 22, feature extraction module 23, data loading module 24, initial detection module 25, detection processing module 26, and target detection module 27 are shown.

[0073] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0074] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0075] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0076] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in any embodiment of the photoresist defect detection method based on image recognition provided in the embodiments of this application, or they can execute the implementation methods of the photoresist defect detection platform based on image recognition described in the embodiments of this application, which will not be repeated here.

[0077] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0078] The computer-readable storage medium can be an internal storage unit of the image recognition-based photoresist defect detection platform of any of the foregoing embodiments, such as a hard drive or memory of an electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0079] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed image recognition-based photoresist defect detection platform and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0082] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0083] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0084] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting photoresist defects based on image recognition, characterized in that, include: Based on the process parameters and photoresist material parameters of the current printing batch, the corresponding defect detection model selection identifier and detection parameter set are determined through a preset mapping relationship library; the detection parameter set includes a defect classification confidence threshold and a regional attention weight map for feature weighting. Feature extraction is performed on the original image data of the photoresist during the printing process of the current printing batch to obtain image feature data; According to the defect detection model, select the identifier to load the target defect feature extraction network model, input the image feature data into the target defect feature extraction network model, and obtain the initial defect detection result; Based on the detection parameter set, the initial defect detection results are processed to determine the target defect detection results; The method for constructing the mapping relationship library includes: Cluster analysis was performed on the photoresist material parameters and process parameters of historical printing batches to generate multiple parameter combination clusters; Train a dedicated defect feature extraction network model for each parameter combination cluster and associate it with a unique defect detection model selection identifier; Based on the statistical distribution of various defects in the defect-annotated image data corresponding to each parameter combination cluster, the defect classification confidence threshold corresponding to that parameter combination cluster is determined. Based on the spatial location statistics of defects in the defect-annotated image data corresponding to each parameter combination cluster, a region attention weight map corresponding to that parameter combination cluster is generated. Establish a mapping relationship between each parameter combination cluster and the defect detection model selection identifier, defect classification confidence threshold, and regional attention weight map, and store them to form the mapping relationship library; The step of determining the corresponding defect detection model selection identifier and detection parameter set through a preset mapping relationship library includes: Calculate the similarity between the process parameters and photoresist material parameters of the current printing batch and the center point of each parameter combination cluster in the mapping relationship library to obtain a similarity ranking list; Select the mapping relationship corresponding to the parameter combination cluster with the highest similarity in the similarity ranking list; Based on the mapping relationship, the corresponding defect detection model selection identifier and detection parameter set are loaded.

2. The method for detecting photoresist defects based on image recognition according to claim 1, characterized in that, The process of extracting features from the original image data of the photoresist during the printing process of the current printing batch to obtain image feature data includes: The original image data of the photoresist during the printing process of the current printing batch is input into a pre-trained interference suppression generation network to generate foreground image data. Calculate the global optical flow field and its average motion amplitude between consecutive frames of the original image data; Based on the global optical flow field and average motion amplitude, image feature extraction is performed on the foreground image data to obtain image feature data.

3. The method for detecting photoresist defects based on image recognition according to claim 2, characterized in that, Based on the global optical flow field and average motion amplitude, image feature extraction is performed on the foreground image data to obtain image feature data, including: If the average motion amplitude is greater than the motion amplitude threshold, then motion compensation is performed on the foreground image data based on the global optical flow field to obtain compensated image data; The compensated image data and the foreground image data are fused to obtain fused image data; Feature extraction is performed on the fused image data to obtain image feature data; If the average motion amplitude is less than or equal to the motion amplitude threshold, then image feature extraction is performed directly on the foreground image data to obtain image feature data.

4. The method for detecting photoresist defects based on image recognition according to claim 3, characterized in that, The step of extracting features from the fused image data to obtain image feature data includes: Feature extraction is performed on the fused image data to obtain spatial feature maps and temporal feature vectors; A spatial deformation field is generated based on the temporal feature vector, and an affine transformation is performed on the spatial feature map to generate a calibration feature map. The temporal feature vector is converted into a feature map that matches the spatial dimension of the calibration feature map; The calibration feature map and the feature map are concatenated by channels to obtain image feature data.

5. The method for detecting photoresist defects based on image recognition according to claim 4, characterized in that, The step of extracting image features from the foreground image data to obtain image feature data includes: Perform a convolution operation on the foreground image data to generate an initial feature map; The initial feature map is input into the channel attention module to calculate the weight coefficients of each feature channel; The initial feature map is channel-weighted according to the weight coefficients to obtain an optimized feature map; The optimized feature map is resampled to generate the image feature data.

6. The method for detecting photoresist defects based on image recognition according to claim 5, characterized in that, The step of resampling the optimized feature map to generate the image feature data includes: The optimized feature map is decomposed into high-frequency components and low-frequency components; A dense sampling strategy is used to sample high-frequency components to obtain a first sampling result, wherein the sampling interval of the dense sampling strategy is the first sampling interval; A sparse sampling strategy is used to sample low-frequency components to obtain a second sampling result; wherein the sampling interval of the sparse sampling strategy is the second sampling interval; and the second sampling interval is greater than the first sampling interval. The first sampling result and the second sampling result are weighted and fused to generate the image feature data.

7. A photoresist defect detection system based on image recognition, characterized in that, include: The data loading module is used to determine the corresponding defect detection model selection identifier and detection parameter set based on the process parameters and photoresist material parameters of the current printing batch, through a preset mapping relationship library. The mapping relationship library stores the mapping relationships between multiple parameter combinations and the defect detection model selection identifier and detection parameter set. The parameter combinations are the process parameters of the printing batch and the photoresist material parameters. The detection parameter set includes a defect classification confidence threshold and a region attention weight map for feature weighting. The feature extraction module is used to extract features from the raw image data of the photoresist during the printing process of the current printing batch to obtain image feature data; The initial detection module is used to select an identifier according to the defect detection model, load the target defect feature extraction network model, input the image feature data into the target defect feature extraction network model, and obtain the initial defect detection result. The detection processing module is used to process the initial defect detection results based on the detection parameter set to determine the target defect detection results.

8. A photoresist defect detection platform based on image recognition, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

Images