Wafer graph defect semantic reasoning method driven by retrieval enhancement large model
By constructing a multi-source dataset and a structured knowledge graph, and combining a large language model for semantic reasoning of wafer defects, the problem of lack of semantic interpretation and role adaptability in existing technologies is solved. High-confidence recognition and multi-role adaptive interpretation are achieved, which is suitable for wafer defect analysis in multi-role collaborative manufacturing scenarios.
Patent Information
- Application Number
- CN202610278694.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-09
- Publication Date
- 2026-04-03
- Estimated Expiration
- 2046-03-09
AI Technical Summary
Existing wafer image defect detection methods lack semantic interpretation capabilities, making it difficult to utilize process knowledge for causal analysis and failing to meet the differentiated decision-making needs of multiple engineering roles.
We construct a multi-source dataset, integrate text related to wafer image defect detection with wafer images, extract visual embedding features through deformable convolutional neural networks, and combine structured knowledge graphs and large language models to perform retrieval-enhanced semantic reasoning, generating adaptive semantic interpretations for specific roles.
It achieves high-confidence identification of wafer image defects and adaptive interpretation by multiple roles, possesses good interpretability and knowledge generalization ability, and supports root cause analysis and intelligent question answering in multi-role collaborative manufacturing scenarios.
Smart Images

Figure CN121787596A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for retrieving enhanced large model-driven semantic reasoning of wafer image defects, belonging to the field of defect detection and semantic interpretation technology in semiconductor manufacturing processes. Background Technology
[0002] Wafer image defect identification is a crucial means of ensuring wafer yield and process stability. It typically involves modeling the spatial distribution of grain test results on the wafer to identify typical defect patterns within the image. With the miniaturization of process nodes and the increasing complexity of hierarchical structures, wafer image defects exhibit characteristics such as hybridization, structural ambiguity, and semantic ambiguity. Common basic defect types, such as center defects, ring defects, and edge local defects, often exist in multiple overlapping ways. Furthermore, defect causes are closely related to process stages and equipment conditions, and engineers with different roles (such as quality engineers and process engineers) have different focuses regarding defects.
[0003] Current mainstream methods are mainly based on convolutional neural networks or Transformer models. Although they can identify defect categories, their outputs are mostly static labels, lacking interpretability and engineering usability. Specifically, they are: 1) unable to answer semantic questions such as "why the defect occurred" and "which process steps are involved"; 2) difficult to integrate structured knowledge in the manufacturing process; and 3) lack the ability to adapt semantics to differences in user roles.
[0004] Therefore, there is an urgent need for a method that integrates wafer image visual features with structured knowledge graphs, and can use pre-trained language models to enhance semantic reasoning through retrieval, in order to support the generation of explanations and multi-role engineering decisions in complex defect scenarios. Summary of the Invention
[0005] The technical problem this invention aims to solve is that existing wafer image defect detection methods lack semantic interpretation capabilities, struggle to utilize process knowledge for causal analysis, and fail to meet the differentiated decision-making needs of multiple engineering roles.
[0006] To address the aforementioned technical problems, the present invention discloses a method for semantic reasoning of wafer image defects driven by enhanced large model retrieval, comprising the following steps:
[0007] Step 1: Construct a multi-source dataset for semantic reasoning of wafer image defects. This multi-source dataset integrates wafer image defect detection-related text and wafer images. Step 2: Extract computable structured knowledge from the multi-source dataset constructed in Step 1, and build a basic text knowledge graph for semantic reasoning; Step 3: Using a deformable convolutional neural network, discriminative visual embedding features are extracted from the wafer images containing a single defect type in the multi-source dataset constructed in Step 1. The visual embedding features are then used as numerical attributes and associated with the corresponding defect type entity nodes in the basic text knowledge graph constructed in Step 2, so as to bridge the visual features with semantic reasoning. Step 4: For the input wafer pattern Visual feature vectors are extracted using the deformable convolutional network from step 3. Calculate visual feature vectors The similarity between the visual embedding features of each defect type in the basic text knowledge graph and the visual embedding features of each defect type is considered if the similarity exceeds a preset threshold. Then determine the corresponding defect type in the wafer diagram. Activate in the middle, obtain the set of entities of the activation defect type. Only activate the defect type entity set. The entities in the map are sent to the next stage for map retrieval; Step 5: Given a user query The system queries the basic text knowledge graph to obtain candidate subgraphs containing sets of knowledge triples that match the query semantics, and then activates the defect type entity set. A graph screening mechanism based on potential defect types is used to filter candidate subgraphs, retaining only knowledge entities and relationships that have path connections with activated defect types, thus obtaining the filtered subgraphs. ; Step 6: Based on the subgraph The connection strength between knowledge entities and activation defect types is analyzed. Knowledge entities are ranked, and structured prompts are constructed based on the ranking results and related edge relationships. These structured prompts are then used in conjunction with user queries. Together with a large language model, it generates adaptive semantic interpretations tailored to specific roles.
[0008] Preferably, in step 1, the multi-source dataset is constructed based on literature data, wafer image data, and process documentation in the field of wafer image defect detection.
[0009] Preferably, step 2 includes the following steps: Step 201: Extract the original text documents from the multi-source dataset. Divide into a group of overlapping sub-text blocks, each sub-text block containing at most [number of characters]. One slice; Step 202: Employ a prompt-driven large language model for multi-round queries to extract key entities and semantic relationships from each sub-text block, obtaining entity sets and semantic relationship sets. Further, perform unified deduplication and merging operations. This will be based on the prompt word-driven large language model. Sub-text blocks The obtained entity set and semantic relation set are respectively represented as as well as ,but: Entity Collection Each entity Defined as:
[0010] In the formula, For entity name, For entity type, A brief description of the entity; Semantic relation set Each relationship Defined as:
[0011] In the formula, and These are the source and target entities of the relationship, respectively. Semantic tags for relationships. Score the confidence level. A set of keywords that reflect relational semantics; Step 203: Construct a searchable domain knowledge graph based on the key-value structure, and finally output the basic text knowledge graph. Each entity node and relationship edge Each is associated with one or more text-based key-value pairs. , Represents the set of all entity nodes in the graph. Represents the set of all semantic relation edges in the graph, key An index used to represent a semantic tag, the value It then provides a complete structured description text.
[0012] In this invention, structured knowledge extraction is accomplished through a large language model guided by prompt words. These prompt words include entity extraction prompts and relationship recognition prompts, and a multi-round extraction mechanism is employed to improve extraction completeness. The constructed wafer image defect text knowledge graph contains the following six entity types: defect type, detection model, detection challenge, model component, process step, equipment tool, and their semantic relationships and confidence scores.
[0013] Preferably, the slicing formula used in step 201 is:
[0014]
[0015] In the formula: For the first Sub-text blocks, , The total number of sub-text blocks; The maximum number of tokens for each sub-text block; This is a function for token-based block operations.
[0016] Preferably, in step 202, during the extraction process of the large language model: For each sub-text block, a set of candidate entities is identified based on predefined entity types, including defect types, detection models, detection challenges, model components, process steps, and equipment tools. The large language model is guided to extract semantic relationships between entities. These semantic relationships are represented by natural language labels, and each semantic relationship is also assigned a confidence score to measure the reliability of the current semantic relationship. If the confidence score of the current semantic relationship is lower than a preset threshold, it will be removed before the subsequent construction of the basic text knowledge graph.
[0017] Preferably, in step 202, when performing the deduplication and merging operations: For entities with the same name but different types, a majority voting mechanism is used to determine the standard type, that is, the entity type with the highest frequency is assigned to the current entity to ensure the consistency of entity type; For semantic relations that appear repeatedly in different sub-text blocks, they are merged into a single edge, and the confidence scores are averaged.
[0018] In the formula, This indicates the number of repeated semantic relations between the same entity pair. Indicates the first Confidence score of semantic relations.
[0019] Preferably, in step 3, the average of the embedding features of multiple samples under the same defect type is used as the visual embedding feature of the current defect type:
[0020] In the formula, Indicates the first This belongs to the defect type. The visual feature vector extracted from the wafer image sample. Indicates the type of defect The aforementioned visual embedding features.
[0021] In this invention, the extracted visual embedding features are constructed using a deformable convolutional network, which outputs a defect feature vector and generates visual prototype vectors for eight basic defects through a mean operation.
[0022] Preferably, in step 4, the set of entities activating defect types The definition is as follows:
[0023] In the formula, For the first Visual embedding features of various defect types Visual embedding features With visual feature vectors The cosine similarity.
[0024] In this invention, a prototype matching strategy is used to perform visual-semantic alignment on the input wafer image. Activation determination is performed based on the cosine similarity between the feature vector of the input image and the prototype embedding in the image. A threshold is set to control the activation result of the defect type.
[0025] Preferably, in step 5, the set of entities with activation defect types is defined as follows: Represented as The obtained candidate subgraph is represented as The subgraph is obtained based on the following filtering rules. : If candidate subgraph An entity or semantic relation in the set of entities of the activation defect type is not associated with the set of entities of the activation defect type. If any node of the activated defect type has a path connection, then from the candidate subgraph... Remove from the middle to obtain the filtered subgraph :
[0026] in, Represent the head entity, relation, and tail entity of a knowledge triple. Indicates the first One active defect type node, This indicates that there are connections in the graph. and or The path.
[0027] In this invention, the constructed filtering subgraph contains only triples that have direct or indirect graph path connections with the activated defect type entity, which is used to filter irrelevant knowledge and enhance semantic relevance.
[0028] Preferably, in step 6, the subgraph is set as follows: Represented as:
[0029] In the formula, For the subgraph The included first Each semantic knowledge triple represents an extracted knowledge fact; For each knowledge entity, the number of connections between it and the activation defect type is calculated as a ranking score, where the knowledge entity... The semantic connectivity score is defined as follows:
[0030] in, Representing knowledge entities With defect type The semantic relationships between entities are defined as follows: if an entity has multiple associations with the same defect, each association is counted separately. All knowledge entities are sorted in descending order based on semantic connectivity scores, and structured prompt words are constructed based on the sorting results and their related edge relationships. These structured prompt words are represented as follows:
[0031] in, This represents the sorted set of entity information. It represents the set of semantic relations associated with it. This represents a text fragment derived from the original data.
[0032] In this invention, a knowledge ranking mechanism based on semantic connection strength is adopted to count the associated edges between each entity and activation defects, and the score is used as the ranking basis to improve the relevance of the explanation content.
[0033] This invention proposes a retrieval-enhanced, large-model-driven semantic reasoning method for wafer image defects. The aim is to address the shortcomings of existing wafer image defect detection methods, such as difficulty in integrating process knowledge and a lack of role adaptability and interpretability. This method integrates visual feature embedding, structured knowledge graph construction, and a three-stage semantic reasoning mechanism to build a defect understanding system that combines perception, retrieval, and reasoning. By introducing a defect prototype-driven visual-semantic alignment strategy, this invention achieves high-confidence identification of potential defect types in wafer images. Combined with a graph path filtering and semantic association scoring mechanism, this invention achieves accurate filtering of knowledge retrieval results and role-adaptive interpretation generation. The method disclosed in this invention possesses good interpretability, upstream and downstream role adaptability, and knowledge generalization ability, making it suitable for wafer defect root cause analysis and intelligent question-answering system construction in multi-role collaborative manufacturing scenarios. Attached Figure Description
[0034] Figure 1This is a block diagram of a wafer image defect semantic reasoning method driven by a large model for enhanced retrieval, according to the present invention. Figure 2 This is a sample of entities and relationships extracted from text according to the present invention. Detailed Implementation
[0035] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.
[0036] Combination Figure 1 This invention discloses a method for semantic reasoning of wafer image defects driven by a large-scale model for enhanced retrieval. It is applicable to various user roles, including but not limited to quality engineers, process engineers, and algorithm engineers. The generated semantic reasoning results can automatically adjust the interpretation focus based on the user's query content. Specifically, it includes the following steps: Step 1: Construct a multi-source dataset for semantic reasoning of wafer map defects.
[0037] A multi-source dataset integrating wafer image defect detection-related text and wafer images was constructed, including the following three types of information sources: (1) Literature data in the field of wafer defect detection: High-quality literature in the field was retrieved from the Web of Science database using “waferdefect detection” as the keyword. The literature covers wafer defect types, detection models, common problems and root cause analysis, etc., to support the structured extraction and ontology construction of defect knowledge.
[0038] (2) Wafer image data: Wafer image samples containing eight typical defect types were collected, namely Center, Donut, Edge_Loc, Edge_Ring, Loc, Near_Full, Scratch, and Random. This type of data serves as the basis for constructing visual prototype representations of defects, which can enhance the stability and interpretability of the subsequent inference system.
[0039] (3) Process documentation: Introduce the company’s internal wafer process documentation, which covers multiple key steps from photolithography and etching to chemical vapor deposition and their potential defect causes, which helps to establish the relationship path between process and defects in the diagram.
[0040] Step 2: Structured extraction of textual knowledge about defects in wafer images.
[0041] Extract computable structured knowledge (such as...) from the literature and process documents related to wafer defect detection in Step 1. Figure 2 As shown, constructing a basic text knowledge graph for semantic reasoning mainly includes the following sub-steps: (1) Token-based document chunking Original text document Divide into a group of overlapping sub-text blocks, each block containing at most [number of sub-text blocks]. Each token (i.e., a slice) has the following slicing formula:
[0042]
[0043] in: For the first Sub-text blocks, , The total number of sub-text blocks; The maximum number of tokens for each sub-text block is 32768 in this embodiment of the invention. This is a function for token-based block operations.
[0044] (2) Entity and relation extraction guided by prompt words A prompt-driven Large Language Model (LLM) is used for multi-round queries to extract key entities and semantic relationships from each subtext block, as defined below:
[0045] in, This is an entity relation extraction function based on LLM. This represents the set of entities extracted from the current sub-text block. This represents the corresponding set of semantic relations.
[0046] Every entity Defined as:
[0047] in, For entity name, For entity type, A brief description of the entity.
[0048] Every relationship Defined as:
[0049] in, and These are the source and target entities of the relationship, respectively. This is the semantic label for the relationship. Score the confidence level. A set of keywords that reflect relational semantics.
[0050] During the extraction process, for each sub-text block, the LLM first identifies a set of candidate entities based on pre-designed prompts. Predefined entity types include: 1) Defect Types: such as edge rings, scratches, and other basic defect categories commonly found in wafer diagrams; 2) Detection Models: such as defect detection models or network architectures like ResNet-50 and VGG-16; 3) Detection Issues: Common detection challenges such as mixed defects and imbalanced data; 4) Model Components: such as Feature Pyramid Network (FPN), Attention Module, and other sub-modules of the detection model; 5) Process Steps: Manufacturing processes that may cause defects, such as etching and chemical mechanical polishing (CMP). 6) Equipment: such as etching machines and lithography systems, etc.
[0051] Subsequently, the LLM is guided to extract semantic relationships between entities, which are represented using natural language labels such as "causes," "improves," "is part of," and "affects performance." Each relationship is also assigned a confidence score of 1 to 5 to measure its reliability. If the confidence score of a relationship is lower than a preset threshold (set to 2 in this embodiment), it is discarded before graph construction to ensure that only semantic edges with high confidence are used to construct subgraphs, thereby effectively filtering low-quality or noisy knowledge.
[0052] To improve extraction completeness and avoid information loss across rounds, the extraction process for each sub-text block is performed in a maximum of three rounds. After each round, the system checks whether the LLM returns an explicit termination signal (such as "No more entities" or "All relationships extracted"). If a termination signal is detected, the extraction of that sub-text block ends early; otherwise, the next round continues until a maximum of three rounds are reached.
[0053] (3) Deduplication and merging of entities and relations To address redundant entities and duplicate relational information within multiple sub-text blocks, this step introduces post-processing mechanisms at both the entity and relational layers to uniformly deduplicate and merge the extraction results. The specific operation definitions are as follows:
[0054] in, Represents the refined set of entities. Represents the refined set of relations. Functions representing deduplication and merging operations.
[0055] For entities with the same name but different types, a majority voting mechanism is used to determine their standard type, assigning the most frequently occurring entity type to that entity to ensure consistency in entity types. For semantic relationships that appear repeatedly in different sub-text blocks, they are merged into a single edge, and their confidence scores are averaged.
[0056] in, This indicates the number of repeated relations between the same entity pairs. Indicates the first Confidence score of the relationship.
[0057] (4) Graph construction based on key-value index To support efficient knowledge retrieval and reasoning system calls, this step introduces a key-value structure to construct a searchable domain knowledge graph. The final output is a wafer image defect domain knowledge graph. Each entity node and relationship edge Each is associated with one or more text-based key-value pairs. This facilitates structured storage and semantic querying. Among them, Represents the set of all entity nodes in the graph. Represents the set of all semantic relation edges in the graph, key An index used to represent a semantic tag, the value It provides a complete structured description text for downstream semantic reasoning modules to call and generate.
[0058] The formal definition of the graph construction process is as follows:
[0059] in, and These are the key-value pair encoding functions for entity nodes and relation edges, respectively. This represents the overall construction operation of the graph structure.
[0060] Step 3: Embedding visual attributes of wafer pattern defects.
[0061] To achieve the organic integration of visual prior features of wafer defects with the knowledge graph, this step uses a deformable convolutional neural network to extract discriminative visual embedding features from the wafer image containing a single defect type in step 1, and associates them as numerical attributes with the corresponding defect type entity nodes in the knowledge graph in step 2, so as to achieve the bridging of visual features and semantic reasoning.
[0062] To improve the model's robustness to samples and noise, the average embedding features of multiple samples under the same defect type are used as the visual prototype embedding for that category. This process can be represented as:
[0063] in, Indicates the first This belongs to the defect type. The visual feature vector extracted from the wafer image sample. This indicates the final visual prototype embedding of this type.
[0064] The generated prototype embeddings are attached as numerical attributes to the corresponding defect type entity nodes in the knowledge graph, enabling the graph to have dual-modal representation capabilities of visual and semantics.
[0065] Step 4: Input the defect visual semantic alignment of the wafer image.
[0066] Design a visual-semantic alignment mechanism for wafer image defects, which can directly identify potential defect types from the input wafer image and achieve a close connection between wafer image features and semantic prototypes in a knowledge graph. Let the input wafer image be... First, the deformable convolutional network from step 3 is used to extract its visual feature vector. This visual encoder can transform a wafer image into a representation vector, capturing its spatial distribution and semantic defect characteristics.
[0067] in, This represents the visual encoding function.
[0068] The knowledge graph already stores the semantic prototype embeddings of the eight basic defects from step 1. These prototype vectors serve as alignment references.
[0069] For each defect type prototype Calculate the cosine similarity between the image and the input wafer image using the following formula:
[0070] If the similarity score between a prototype and the input wafer image exceeds a preset threshold... If a defect type is found to be active (i.e., present) in the wafer diagram, then that defect type is determined to be active. The final set of active defect type entities is defined as follows:
[0071] Only the defect type entities that are activated (i.e. Entities in the dataset are sent to the next stage for graph retrieval, effectively narrowing the semantic retrieval space and focusing only on the category most relevant to the current defect. Furthermore, when the similarity of all defect prototypes does not exceed a threshold, the system will output a message stating "Defect type cannot be identified with sufficient confidence," thus preventing the propagation of erroneous reasoning at its source.
[0072] Step 5: Defect-driven graph knowledge retrieval and filtering.
[0073] Design a graph screening mechanism based on potential defect types, retaining only knowledge entities and relationships that have path connections with activated defect types, thereby ensuring that the retrieved knowledge is semantically relevant and consistent with the wafer graph observation results, thus achieving more accurate and defect-specific knowledge reasoning.
[0074] Let the set of activation defect types identified in step 4 be... In a given user query In this case, we first use LLM to extract keywords from the query, and then use these keywords to query the wafer image defect domain knowledge graph constructed in steps 2 and 3. This yields a candidate subgraph containing a set of knowledge triples that match the query semantics. However, the entities and relations obtained from the initial retrieval may not be associated with the defects in the input wafer graph. To ensure consistency between semantic and visual information, the following filtering rules are designed: If an entity or relation does not have a path connection (direct or indirect) to any node of the activation defect type, then remove it from the candidate subgraph. Subgraphs are removed from the middle and formally defined after filtering. for:
[0075] in, Represent the head entity, relation, and tail entity of a knowledge triple. Indicates the first One active defect type node, This indicates that there are connections in the graph. and or The path. If any entity in a triplet is related to the activation defect type. If there is no path connection between them, the triple is excluded.
[0076] The final subgraph Includes a set of user queries Semantic knowledge triples that are relevant and traceable to the defect type in the input wafer diagram are used as input for subsequent semantic reasoning and interpretation generation. Furthermore, when no valid subgraph connected to the defect type can be found, the system will automatically terminate the reasoning process and output a message stating "Insufficient knowledge confidence, unable to provide a valid explanation," to prevent the propagation of erroneous knowledge reasoning.
[0077] Step 6: Knowledge Ranking and Role-Adaptive Semantic Interpretation Generation. A potential defect semantic relevance scoring mechanism is designed, ranking knowledge entities based on the connection strength between them and the activated defect types. This ensures that the generated semantic interpretation not only closely fits the contextual semantics but also adaptively meets the user's implicit role requirements.
[0078] Define the subgraph filtered in step 5 as:
[0079] Each triple represents an extracted knowledge fact.
[0080] To prioritize knowledge elements most relevant to observed wafer defects, the number of connections between each knowledge entity and the active defect type is calculated as a ranking score. Specifically, for a given entity... Its semantic connectivity score is defined as follows:
[0081] in, Representing entities With defect type The semantic relationships between entities can be bidirectional. If an entity has multiple associations with the same defect, each association is counted separately. This scoring strategy reflects the cumulative semantic connectivity between entities and activated defect types.
[0082] Subsequently, all entities are sorted in descending order based on their scores, and structured cue words are constructed based on the sorting results and their associated edge relationships to guide the LLM in generating semantic interpretations. These structured cue words are represented as follows:
[0083] in, This represents the sorted set of entity information. It represents the set of semantic relations associated with it. This represents a text fragment from the original data, used to supplement detailed descriptions.
[0084] Ultimately, the system will process the user's query. Inputting the structured prompts constructed above into the LLM generates a role-specific adaptive semantic interpretation:
[0085] The interpretation results are semantically closely aligned with the defect types identified in the current wafer diagram, while also taking into account the role preferences implied by the user query (such as quality engineer, process engineer, or algorithm engineer), thus achieving flexible and interpretable reasoning support required in cross-position and upstream-downstream collaborative manufacturing scenarios.
Claims
1. A method for semantic reasoning of wafer image defects driven by a large model for enhanced retrieval, characterized in that, Includes the following steps: Step 1: Construct a multi-source dataset for semantic reasoning of wafer image defects. This multi-source dataset integrates wafer image defect detection-related text and wafer images. Step 2: Extract computable structured knowledge from the multi-source dataset constructed in Step 1, and build a basic text knowledge graph for semantic reasoning; Step 3: Using a deformable convolutional neural network, discriminative visual embedding features are extracted from the wafer images containing a single defect type in the multi-source dataset constructed in Step 1. The visual embedding features are then used as numerical attributes and associated with the corresponding defect type entity nodes in the basic text knowledge graph constructed in Step 2, so as to bridge the visual features with semantic reasoning. Step 4: For the input wafer pattern Visual feature vectors are extracted using the deformable convolutional network from step 3. Calculate visual feature vectors The similarity between the visual embedding features of each defect type in the basic text knowledge graph and the visual embedding features of each defect type is considered if the similarity exceeds a preset threshold. Then determine the corresponding defect type in the wafer image. Activate in the middle, obtain the set of entities of the activation defect type. Only activate the defect type entity set. The entities in the map are sent to the next stage for map retrieval; Step 5: Given a user query The system queries the basic text knowledge graph to obtain candidate subgraphs containing sets of knowledge triples that match the query semantics, and then activates the defect type entity set. A graph screening mechanism based on potential defect types is used to filter candidate subgraphs, retaining only knowledge entities and relationships that have path connections with activated defect types, thus obtaining the filtered subgraphs. ; Step 6: Based on the subgraph The connection strength between knowledge entities and activation defect types is analyzed. Knowledge entities are ranked, and structured prompts are constructed based on the ranking results and related edge relationships. These structured prompts are then used in conjunction with user queries. Together with a large language model, it generates adaptive semantic interpretations tailored to specific roles.
2. The method for semantic reasoning of wafer image defects driven by enhanced large model as described in claim 1, characterized in that, In step 1, the multi-source dataset is constructed based on literature data, wafer image data, and process documentation in the field of wafer image defect detection.
3. The method for semantic reasoning of wafer image defects driven by enhanced large model as described in claim 1, characterized in that, Step 2 includes the following steps: Step 201: Extract the original text documents from the multi-source dataset. Divide into a group of overlapping sub-text blocks, each sub-text block containing at most [number of characters]. One slice; Step 202: Employ a prompt-driven large language model for multi-round queries to extract key entities and semantic relationships from each sub-text block, obtaining entity sets and semantic relationship sets. Further, perform unified deduplication and merging operations. This will be based on the prompt word-driven large language model. Sub-text blocks The obtained entity set and semantic relation set are respectively represented as as well as ,but: Entity Collection Each entity Defined as: In the formula, For entity name, For entity type, A brief description of the entity; Semantic relation set Each relationship Defined as: In the formula, and These are the source and target entities of the relationship, respectively. Semantic tags for relationships. Score the confidence level. A set of keywords that reflect relational semantics; Step 203: Construct a searchable domain knowledge graph based on the key-value pair structure, and finally output the basic text knowledge graph. Each entity node and relationship edge Each is associated with one or more text-based key-value pairs. , Represents the set of all entity nodes in the graph. Represents the set of all semantic relation edges in the graph, key An index used to represent a semantic tag, the value It then provides a complete structured description text.
4. The method for semantic reasoning of wafer map defects driven by enhanced large model as described in claim 3, characterized in that, The slicing formula used in step 201 is: In the formula: For the first Sub-text blocks, , The total number of sub-text blocks; The maximum number of tokens for each sub-text block; This is a function for token-based block operations.
5. The method for semantic reasoning of wafer map defects driven by a large model for enhanced retrieval as described in claim 3, characterized in that, In step 202, during the extraction process of the large language model: For each sub-text block, a set of candidate entities is identified based on predefined entity types, including defect types, detection models, detection challenges, model components, process steps, and equipment tools. The large language model is guided to extract semantic relationships between entities. These semantic relationships are represented by natural language labels, and each semantic relationship is also assigned a confidence score to measure the reliability of the current semantic relationship. If the confidence score of the current semantic relationship is lower than a preset threshold, it will be removed before the subsequent construction of the basic text knowledge graph.
6. The method for semantic reasoning of wafer image defects driven by enhanced large model as described in claim 3, characterized in that, In step 202, during the deduplication and merging operations: For entities with the same name but different types, a majority voting mechanism is used to determine the standard type, that is, the entity type with the highest frequency is assigned to the current entity to ensure the consistency of entity type; For semantic relations that appear repeatedly in different sub-text blocks, they are merged into a single edge, and the confidence scores are averaged. In the formula, This indicates the number of repeated semantic relations between the same entity pair. Indicates the first Confidence score of semantic relations.
7. The method for semantic reasoning of wafer image defects driven by enhanced large model as described in claim 1, characterized in that, In step 3, the mean of the embedding features of multiple samples under the same defect type is used as the visual embedding feature of the current defect type: In the formula, Indicates the first This belongs to the defect type. The visual feature vector extracted from the wafer image sample. Indicates the type of defect The aforementioned visual embedding features.
8. The method for semantic reasoning of wafer image defects driven by enhanced large model as described in claim 1, characterized in that, In step 4, the set of entities with activated defect types The definition is as follows: In the formula, For the first Visual embedding features of various defect types Visual embedding features With visual feature vectors The cosine similarity.
9. The method for semantic reasoning of wafer image defects driven by enhanced large model as described in claim 1, characterized in that, In step 5, let the set of activation defect type entities be... Represented as The obtained candidate subgraph is represented as The subgraph is obtained based on the following filtering rules. : If candidate subgraph An entity or semantic relation in the set of entities of the activation defect type is not associated with the set of entities of the activation defect type. If any node of the activated defect type has a path connection, then from the candidate subgraph... Remove from the middle to obtain the filtered subgraph : in, Represent the head entity, relation, and tail entity of a knowledge triple. Indicates the first One active defect type node, This indicates that there are connections in the graph. and or The path.
10. The method for semantic reasoning of wafer map defects driven by enhanced large model as described in claim 1, characterized in that, In step 6, let the subgraph be... Represented as: In the formula, For the subgraph The included first Each semantic knowledge triple represents an extracted knowledge fact; For each knowledge entity, the number of connections between it and the activation defect type is calculated as a ranking score, where the knowledge entity... The semantic connectivity score is defined as follows: in, Representing knowledge entities With defect type The semantic relationships between entities are defined as follows: if an entity has multiple associations with the same defect, each association is counted separately. All knowledge entities are sorted in descending order based on semantic connectivity scores, and structured prompt words are constructed based on the sorting results and their related edge relationships. These structured prompt words are represented as follows: in, This represents the sorted set of entity information. It represents the set of semantic relations associated with it. This represents a text fragment derived from the original data.
Citation Information
Patent Citations
Wafer defect detection method, device, equipment and medium
CN120031883A
Knowledge graph-based traffic engineering large model intelligent question-answering system and method
CN120407752A
Fault question and answer method based on large language model and knowledge graph
CN120653733A
Knowledge reasoning method and device for integrated circuit wafer process and medium
CN120744140A
Enterprise data dynamic integrated management system based on lightweight
CN121301619A
