Stacked Image Diagnostics for Interpretable Deep Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The opaque nature of deep learning (DL) models in semiconductor manufacturing makes it difficult for users to understand how they generate results, leading to hesitancy in adopting and deploying these models despite their potential advantages.
Innovation Solution
A system and method for generating information by acquiring, separating, aligning, and stacking images based on attributes assigned by a DL model to highlight features influencing its decisions, providing insights into the model's functioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep learning models are used for semiconductor inspection and metrology, then productivity and measurement precision are improved, but the black box nature of the models reduces ease of operation and user understanding
Solution Approach 1:
The patent introduces stacked images as an intermediary visualization tool that bridges the gap between the deep learning model's internal decision-making process and user understanding. These stacked images aggregate and display the most influential features that the model uses for classification, serving as a mediator that translates complex model operations into interpretable visual information without compromising inspection speed.
2Manufacturing precision
If deep learning models are deployed for defect detection and metrology, then manufacturing precision is improved, but device complexity increases due to the need for additional diagnostic tools
Solution Approach 1:
The patent merges the diagnostic visualization functionality directly into the existing deep learning inspection system. The stacked image generation is integrated as part of the model's output process, combining the high-precision defect detection capabilities with interpretability tools in a unified system, thereby avoiding the need for separate complex diagnostic equipment.
3Ease of operation
If stacked images are generated to visualize model features, then ease of operation is improved through better understanding, but loss of time occurs during the image processing and stacking operation
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing the stacked images during model training or validation phases. The feature importance aggregations and image stackings are prepared in advance, so that during actual inspection operations, the pre-generated stacked images can be displayed immediately alongside the classification results, eliminating processing delays during time-critical inspection tasks.
Data Source
AI summary
Methods and systems for generating information for use in evaluating a deep learning (DL) model are provided. One method includes acquiring results generated by a DL model configured for assigning an attribute to images generated for a specimen responsive to a likelihood that the images are images of interest. The method also includes separating the images into groups based on the attribute such that each of the two or more groups corresponds to different values of the attribute and aligning the images in each of the two or more groups to each other. In addition, the method includes stacking the aligned images within each of the two or more groups thereby highlighting in the stacked images one or more features of the images to which the attribute is responsive and outputting the stacked images for use in evaluating the DL model.