Semiconductor aoi vision quality inspection intelligent agent platform

Abstract

The present application relates to the technical field of artificial intelligence large model, and further relates to a semiconductor AOI visual quality inspection intelligent agent platform. A scheduling module constructs an aligned layout reference image and a layout prior area image with pixel-by-pixel record layer identification according to design layout data; a cascaded detection module extracts candidate abnormal areas according to the difference between adjacent die images and the optical image to be detected, and executes 3-way diversion according to the corresponding release threshold and the judgment threshold selected according to the area attribute; the scheduling module assembles evidence packages for gray area abnormal areas and organizes them into multi-modal prompt input visual language large models to obtain a structured judgment report including a judgment conclusion and an evidence chain; an evidence chain verification module performs machine verification on the structured judgment report, adopts the judgment conclusion when the time is adopted, and pushes to the artificial review end when in doubt. The present application constrains the missed detection rate and the overkill rate within the preset risk level, compresses the reasoning amount of the visual language large model, and makes the reviewable and traceable review conclusion.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence large model technology, specifically involving a semiconductor AOI visual quality inspection intelligent agent platform. Background Technology

[0002] Automated Optical Inspection (AOI) is a critical inspection step in semiconductor manufacturing and advanced packaging production lines to ensure yield. Current AOI quality inspection generally uses inter-die comparison, comparing the image of the die to be inspected with adjacent die images and then segmenting it according to a threshold to extract suspected defects. Some solutions further incorporate convolutional neural networks to classify and identify suspected defects. The judgment thresholds of these methods rely on repeated adjustments based on engineer experience, and the trade-off between missed detections and over-detection lacks statistical basis: tightening the threshold increases the over-detection rate, leading to a large influx of false defects requiring manual re-inspection, resulting in high labor costs; relaxing the threshold leads to uncontrolled missed detection risks, potentially causing batch scrap. Simultaneously, the detection sensitivity is set uniformly across the entire image, ignoring the differences in the functional importance of device graphics and redundant graphics such as fill graphics and test graphics. Minor process variations on redundant graphics are treated the same as fatal defects on functional graphics, further exacerbating over-detection.

[0003] In recent years, large-scale visual language models have begun to be introduced into industrial quality inspection for defect re-judgment. However, the output of large-scale visual language models is susceptible to illusion, and the re-judgment conclusions lack verifiable evidence, potentially introducing new sources of misjudgment. Furthermore, performing large-scale visual language model inference on every detected region individually is computationally expensive and time-consuming, making it difficult to match production line pace. Therefore, how to reliably integrate large-scale visual language models into the AOI quality inspection process while controlling the risks of missed detections and over-detection is a pressing issue in this field. Summary of the Invention

[0004] The main objective of this invention is to provide a semiconductor AOI visual quality inspection intelligent agent platform to solve the problems in existing AOI quality inspection, such as the judgment threshold relying on manual experience adjustment, the lack of statistical constraints on the risks of missed detection and over-detection, the neglect of differences in the importance of layout functions in detection sensitivity, the risk of illusion in the re-judgment of large visual language models, the lack of verifiable evidence for re-judgment conclusions, and the difficulty in matching the inference overhead with the production line pace.

[0005] To solve the above problems, the technical solution of the present invention is implemented as follows: The semiconductor AOI visual quality inspection intelligent agent platform includes AOI imaging equipment, cascade detection module, scheduling module, evidence chain verification module and visual language big model. The scheduling module drives the AOI imaging equipment, cascade detection module, visual language big model and evidence chain verification module to work together in a tool call manner. AOI imaging equipment is used to acquire optical images of the die to be inspected on a wafer and images of adjacent dies; The scheduling module is used to construct an aligned layout reference image that corresponds to the optical image under inspection pixel by pixel and records the layer identifier pixel by pixel, based on the design layout data corresponding to the die to be inspected, and to construct a layout prior area map that marks key graphic areas or redundant graphic areas pixel by pixel. The cascaded detection module is used to extract candidate abnormal regions based on the differences between adjacent grain images and the optical image to be inspected. It determines the regional attributes of the candidate abnormal regions based on the layout prior region map, determines the abnormal score of the candidate abnormal regions, and selects the corresponding release threshold and rejection threshold based on the regional attributes to perform three-way splitting: candidate abnormal regions with abnormal scores lower than or equal to the release threshold are judged as good products, candidate abnormal regions with abnormal scores higher than or equal to the rejection threshold are judged as defects, and candidate abnormal regions with abnormal scores higher than the release threshold and lower than the rejection threshold are recorded as gray area abnormal regions and placed into the re-judgment queue. The scheduling module is also used to assemble evidence packages for gray area anomalies in the review queue and organize the evidence packages into a large multimodal prompt input visual language model; The visual language big model is used to output a structured judgment report that includes a judgment conclusion and a chain of evidence. The judgment conclusion is either defective or good. The evidence chain verification module is used to perform machine verification on the structured judgment report. The verification result of the machine verification is either pass or questionable. When the verification result is pass, the evidence chain verification module outputs the judgment conclusion in the structured judgment report as the final judgment result of the gray area anomaly. When the verification result is questionable, the scheduling module pushes the gray area anomaly to the manual review end, and the judgment given by the manual review end is used as the final judgment result of the gray area anomaly.

[0006] Furthermore, the process of constructing the aligned layout reference image by the scheduling module includes: rasterizing the graphics in the design layout data corresponding to the field of view of the optical image under inspection according to the resolution of the optical image under inspection; recording the layer identifier of the graphics covered by each pixel during rasterization; and performing Gaussian smoothing on the rasterization result to make the gray-scale transition shape of the graphic edge close to that of the graphic edge in the optical image under inspection, thus obtaining the layout reference image; initially aligning the layout reference image with the optical image under inspection based on the stage coordinates recorded by the AOI imaging device; and extracting the graphics under inspection from the optical image under inspection. The graphic edge contour of the optical image and the graphic edge contour of the initial aligned layout reference image are used to select the nearest corner point within a preset neighborhood range from the corner points of the graphic edge contour of the initial aligned layout reference image for each corner point in the graphic edge contour of the optical image to be inspected, and pair them as corresponding control points. Based on the corresponding control points, thin plate spline interpolation is used to solve the smooth deformation field. Based on the smooth deformation field, the same alignment transformation is performed on the layout reference image and its pixel-by-pixel recorded layer identifiers to obtain the aligned layout reference image that corresponds to the optical image to be inspected pixel by pixel.

[0007] Furthermore, the scheduling module, based on the layer identifiers recorded for each pixel in the aligned layout reference image, assigns at least one pixel whose recorded layer identifier corresponds to the device functional graphic to the critical graphic region, and assigns the remaining pixels to the redundant graphic region, thus obtaining the layout prior region map; the cascade detection module records the region category to which the pixels in the candidate abnormal region belong as the region attribute of the candidate abnormal region, and the value of the region attribute is either critical graphic region or redundant graphic region.

[0008] Furthermore, adjacent dies are dies on the same wafer with the same layout content as the die to be inspected. The adjacent die images are images acquired by AOI imaging equipment under the same field of view as the optical image to be inspected. The cascaded detection module aligns the adjacent die images with the optical image to be inspected and then performs pixel-by-pixel difference to obtain a difference image. Based on the grayscale distribution of the difference image, it adaptively determines the binarization segmentation value, performs binarization and connected component extraction on the difference image, and records each extracted connected component as a candidate abnormal region. Local optical patches are extracted from the optical image to be inspected with the candidate abnormal region as the center, and the local optical patches are input into a pre-trained feature extraction network to obtain patch features. In the pre-built reference feature library, the reference feature closest to the patch feature is retrieved within the reference feature range corresponding to the layout position of the candidate abnormal region. The feature distance between the patch feature and the retrieved reference feature is used as the abnormal score of the candidate abnormal region. The reference feature library is pre-built by dividing the sample images of good dies of the same model into patches according to the layout position, extracting features through the feature extraction network, and recording and storing them together with the layout position.

[0009] Furthermore, the critical graphic region and the redundant graphic region are each configured with an approval threshold and a rejection threshold, and the preset missed detection risk level configured for the critical graphic region is lower than that configured for the redundant graphic region. The approval threshold and rejection threshold are obtained by calibrating the critical graphic region and the redundant graphic region respectively on the pre-constructed calibration image set: the calibration sample images in the calibration image set are processed in the same way as the optical images to be inspected to obtain the anomaly score and region attribute of each calibration candidate anomaly region. Each calibration candidate anomaly region is manually labeled with a defect conclusion or a good product conclusion as the labeling conclusion. For the same region attribute, the calibration candidate anomaly regions labeled with a defect conclusion are sorted in ascending order of anomaly score. The system sorts out the candidate anomaly regions that meet the omission detection constraint and selects the maximum anomaly score as the release threshold. The omission detection constraint is that the proportion of the number of candidate anomaly regions for defect conclusions with anomaly scores lower than or equal to the release threshold to the total number of candidate anomaly regions for all defect conclusions under the same region attribute is within the preset omission detection risk level. The candidate anomaly regions marked as good products are sorted in descending order of anomaly scores, and the minimum anomaly score that meets the overkill constraint is selected as the rejection threshold. The overkill constraint is that the proportion of the number of candidate anomaly regions for good product conclusions with anomaly scores higher than or equal to the rejection threshold to the total number of candidate anomaly regions for all good product conclusions under the same region attribute is within the preset overkill risk level.

[0010] Furthermore, when the release threshold obtained under the same regional attribute is higher than or equal to the rejection threshold, the maximum value of the abnormal scores of each candidate abnormal region that is lower than the rejection threshold is re-determined as the release threshold. The re-determined release threshold is lower than the rejection threshold, and the missed detection constraint remains valid.

[0011] Furthermore, the evidence package includes: a local optical patch of the gray area anomaly extracted from the optical image under inspection; a layout reference patch extracted from the same position in the aligned layout reference image; a layout category annotation patch obtained by overlaying text annotations of the corresponding design graphic category names at the positions of each graphic in the layout reference patch; the regional attributes of the gray area anomaly; and all defect category entries read from the defect category knowledge base; the design graphic category name in the text annotation is determined according to the layer identifier of the graphic based on the pre-established mapping relationship between the layer identifier and the design graphic category name; the defect category entries have an entry number and record the category name, morphological feature text, and judgment criterion text.

[0012] Furthermore, the multimodal prompting convention specifies that the visual language large model outputs a structured judgment report in a predefined key-value field format, and stipulates the requirements for the chain of evidence: when the region attribute is a key graphic region, the chain of evidence consists of at least 3 evidence items, of which at least 1 evidence item's evidence source identifier is a map reference block; when the region attribute is a redundant graphic region, the chain of evidence consists of at least 2 evidence items; when the judgment conclusion is a defect, the structured judgment report records the number of the matched item; the evidence item includes an evidence source identifier, a location description, and a comparison description. The value of the evidence source identifier can be a local optical block, a map reference block, or a defect category item. When the evidence source identifier is a defect category item, the evidence item... The entry number cited is recorded; the location description uses directional terms and pixel distance with the center of the gray area anomaly region as the origin. The directional terms are taken from a predefined set of directional terms, and each directional term in the set corresponds to a specific direction in the image coordinate system; when the evidence source identifier is a local optical patch, the description records the morphology observed in the local optical patch; when the evidence source identifier is a layout reference patch, the description records the comparison between the observed morphology and the design graphic category name marked in the same position in the layout category label patch; when the evidence source identifier is a defect category entry, the description records the correspondence between the observed morphology and the judgment criterion text under the cited entry number.

[0013] Furthermore, the evidence chain verification module performs machine verification as follows: It checks the completeness of the key-value fields and the structure of the evidence chain in the structured judgment report. If the key-value fields are complete and the evidence chain structure meets the requirements for the evidence chain structure corresponding to the regional attributes of the gray area anomaly region, it verifies each evidence item. In other cases, the verification result is recorded as questionable. Item-by-item verification includes: when the evidence source identifier is a defect category item, it verifies that the recorded item number exists in the defect category knowledge base, and verifies that the judgment criterion text referenced by the evidence item is consistent word-for-word with the judgment criterion text stored under the same item number in the defect category knowledge base; when the evidence source identifier is a map reference block, it converts the position description into an alignment with the map reference image based on the correspondence between directional words and directions in the directional word set. The image coordinates are used to determine the layer identifier recorded by the pixels at the image coordinates. The design graphic category name is then retrieved according to the mapping relationship between the layer identifier and the design graphic category name. The retrieved design graphic category name is checked to see if there is a name that matches the design graphic category name recorded in the comparison description. For each piece of evidence, the image coordinates converted from the location description are checked to see if they fall within the bounding rectangle of the gray area anomaly region. When the judgment conclusion is a defect, the item number of the hit item is checked to see if it exists in the defect category knowledge base. When all the verification items pass, the verification result is passed; otherwise, the verification result is questionable. When the verification result is questionable, the scheduling module pushes the gray area anomaly region along with the evidence package, the structured judgment report, and the verification record to the manual review end.

[0014] Furthermore, the cascaded detection module records candidate abnormal areas that are determined to be good products as good product release results, and records candidate abnormal areas that are determined to be defects as defect determination results. The intelligent agent platform summarizes the good product release results, defect determination results, and the final determination results of each gray area abnormal area, together with the evidence chain of each gray area abnormal area and the verification records generated by machine verification, to generate a wafer-level quality inspection report.

[0015] The semiconductor AOI visual quality inspection intelligent agent platform of the present invention has the following beneficial effects: The beneficial effects of this invention are as follows: the release threshold and rejection threshold are calibrated on the calibration image set according to the counting ratio. The missed detection ratio corresponding to the good product release result and the over-detection ratio corresponding to the defect judgment result are constrained within the preset missed detection risk level and preset over-detection risk level, respectively. The reliability of the judgment is changed from repeated parameter tuning based on experience to verifiable statistical constraints. The layout prior area map transmits the importance of the layout function to the tightness of the risk constraint and the requirements for the formation of the evidence chain. Key graphic areas obtain stricter missed detection control, and false defect alarms in redundant graphic areas are suppressed. The over-detection and manual re-judgment burden are reduced accordingly. The reasoning scope of the visual language large model is... The area is compressed into the gray zone anomaly region, ensuring the determinism of the main detection path's beat. The computing power of the visual language big model is concentrated on samples that are difficult to judge directly. Machine verification performs a deterministic check on the cited text, location identification, and layer affiliation in the structured judgment report item by item. Only evidence items with accurate location identification and consistent name citations are accepted. Cases that fail the verification are transferred to manual review. Illusory outputs are intercepted before reaching the final judgment result, and the review process is traceable. The aligned map reference image simultaneously supports regional attribute division and coordinate-level layer verification. All aspects of the platform form a closed-loop collaboration, and the overall quality inspection efficiency and judgment consistency are significantly improved. Attached Figure Description

[0016] Figure 1 A schematic diagram illustrating the grayscale transition morphology matching between the edge of the layout reference image and the edge of the optical image to be inspected, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of edge contour corner point neighborhood pairing in an embodiment of the present invention. Figure 3 This is a schematic diagram of the smooth deformation field of a thin plate spline provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the principle of prior region map partitioning provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the anomaly score distribution and dual threshold diversion of the candidate anomaly region for key graphic region identification provided in this embodiment of the invention; Figure 6A schematic diagram of the calibration curves for the release threshold and rejection threshold provided in an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the principle of location description conversion and domain verification provided in an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the verification principle of map category labeling blocks and coordinate-level layers provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The specific embodiments described herein are used to explain the implementation of this invention; processes not detailed in the embodiments can be performed in accordance with conventional methods in the art.

[0018] The semiconductor AOI visual quality inspection intelligent agent platform includes AOI imaging equipment, cascade detection module, scheduling module, evidence chain verification module and visual language big model. The scheduling module drives the AOI imaging equipment, cascade detection module, visual language big model and evidence chain verification module to work together in a tool call manner. AOI imaging equipment is used to acquire optical images of the die to be inspected on a wafer and images of adjacent dies; The scheduling module is used to construct an aligned layout reference image that corresponds to the optical image under inspection pixel by pixel and records the layer identifier pixel by pixel, based on the design layout data corresponding to the die to be inspected, and to construct a layout prior area map that marks key graphic areas or redundant graphic areas pixel by pixel. The cascaded detection module is used to extract candidate abnormal regions based on the differences between adjacent grain images and the optical image to be inspected. It determines the regional attributes of the candidate abnormal regions based on the layout prior region map, determines the abnormal score of the candidate abnormal regions, and selects the corresponding release threshold and rejection threshold based on the regional attributes to perform three-way splitting: candidate abnormal regions with abnormal scores lower than or equal to the release threshold are judged as good products, candidate abnormal regions with abnormal scores higher than or equal to the rejection threshold are judged as defects, and candidate abnormal regions with abnormal scores higher than the release threshold and lower than the rejection threshold are recorded as gray area abnormal regions and placed into the re-judgment queue. The scheduling module is also used to assemble evidence packages for gray area anomalies in the review queue and organize the evidence packages into a large multimodal prompt input visual language model; The visual language big model is used to output a structured judgment report that includes a judgment conclusion and a chain of evidence. The judgment conclusion is either defective or good. The evidence chain verification module is used to perform machine verification on the structured judgment report. The verification result of the machine verification is either pass or questionable. When the verification result is pass, the evidence chain verification module outputs the judgment conclusion in the structured judgment report as the final judgment result of the gray area anomaly. When the verification result is questionable, the scheduling module pushes the gray area anomaly to the manual review end, and the judgment given by the manual review end is used as the final judgment result of the gray area anomaly.

[0019] This embodiment uses the appearance inspection of the redistribution layer in wafer-level packaging as an application scenario. In this scenario, the minimum linewidth of the redistribution metal lines is approximately 5 micrometers, and the graphic feature size is above the optical resolution level, which can meet the prerequisite of pixel-by-pixel correspondence between the layout reference image and the optical image to be inspected. Bumps, pads, and metal wiring patterns from mature processes also meet this prerequisite and can all be used as optional application objects. The AOI imaging equipment uses bright-field incident illumination, with an objective lens magnification of 10x, a 3.45-micrometer pixel size for the area array camera, and a calculated object-side pixel size of approximately 0.35 micrometers. A single field of view outputs a 2048-pixel × 2048-pixel image, covering an area of ​​approximately 0.72 mm × 0.72 mm on the wafer surface. The stage's repeatability deviation is within 1 micrometer, and the stage coordinates at the center of the field of view are recorded during each acquisition. During the inspection, the AOI imaging device acquires the optical image of the die to be inspected on the wafer, and acquires the images of adjacent dies with the same layout content as the die to be inspected on the same wafer under the same field of view; when there are no adjacent dies with the same layout content at the edge of the wafer under inspection under the same field of view, as an optional implementation, the image of the adjacent die is taken from the die at the same layout position on another wafer in the same batch.

[0020] The design layout data is read using GDSII format, and each graphic in the data has a layer identifier. In this embodiment, the mapping relationship between layer identifiers and design graphic category names is as follows: layer identifier 31 corresponds to redistribution metal lines, layer identifier 32 corresponds to vias, layer identifier 33 corresponds to pads, layer identifier 41 corresponds to fill metal blocks, and layer identifier 42 corresponds to test graphics. Redistribution metal lines, vias, and pads carry electrical connection functions and belong to device functional graphics; fill metal blocks are placed to ensure the uniformity of electroplating and grinding processes, and test graphics are used for process monitoring; neither carries device electrical functions. The mapping relationship is pre-established in the form of a configuration table, managed according to product model versions, and used for subsequent text annotation generation and machine verification queries. Both stages reference the same configuration table to ensure the uniqueness of the design graphic category name source.

[0021] Rasterization is performed according to the resolution of the optical image under inspection. That is, the layout coordinates are converted to image coordinates at a rate of 1 pixel per 0.35 micrometers, and the center of each pixel is checked to see if it falls within any graphic. During rasterization, the layer identifier of the graphic covered by each pixel is recorded pixel by pixel. When the same pixel is covered by multiple graphics, all corresponding layer identifiers are recorded. The layer identifiers recorded for each pixel constitute the layer identifier set for that pixel. The layer identifier set for background pixels not covered by any graphics is empty. When rendering grayscale, a single layer is selected from the layer identifier set to determine the pixel grayscale according to the visibility priority rule: pads take priority over vias, vias take priority over redistribution metal lines, and redistribution metal lines take priority over fill metal blocks and test graphics. Pixels of metal material graphics are assigned a grayscale value of 200, and background pixels are assigned a grayscale value of 80. The reason for retaining the complete layer identifier set in the recording stage but only taking a single layer in the rendering stage is that subsequent area division and coordinate-level layer verification require complete coverage relationships, while rendering only serves to achieve visual approximation.

[0022] Direct rasterization produces an ideal step-like edge shape. However, actual optical imaging is affected by diffraction and slight defocus, resulting in a gradual transition of several pixels in width at the edge. Directly comparing these two edge shapes can cause systematic deviations in the contour and corner positions. Therefore, Gaussian smoothing is applied to the rasterization result to make the gray-scale transition shape of the image edge closely resemble that of the image edge in the optical image under test. The Gaussian kernel parameters are calibrated based on actual measurements: images are acquired at the edge of the calibration plate, and the edge gray-scale profile is extracted. In this embodiment, the measured transition width of the edge gray-scale from 10% to 90% is approximately 3 pixels. Based on this, the standard deviation of the Gaussian kernel is set to 1.2 pixels, and the kernel size is 7 pixels × 7 pixels. Figure 1 As shown, the horizontal axis represents the pixel position along the direction perpendicular to the edge of the graphic, and the vertical axis represents the normalized grayscale. The rasterized edge is represented by a step curve. The edge transition curve of the layout reference image after Gaussian smoothing basically coincides with the edge transition curve of the measured optical image under test, and the deviation of their half-height positions is within 0.5 pixels. The layout reference image obtained after the above processing has an edge morphology that meets the conditions for direct comparison with the optical image under test.

[0023] Initial alignment is performed based on the stage coordinates recorded by the AOI imaging equipment. The conversion relationship between the layout coordinate system and the wafer coordinate system is pre-established using at least two alignment marks on the wafer: images are acquired at the alignment marks, and the mark center is located. The translation and rotation parameters are solved by simultaneously solving the image position of the mark center and the designed position of the mark graphic in the layout. During inspection, based on the stage coordinates at the center of the field of view and this conversion relationship, the layout reference image is translated to a position that approximately coincides with the optical image to be inspected. Due to the influence of stage repetitive positioning deviation, wafer clamping rotation, and optical distortion, there is still a residual deviation within a few pixels after the initial alignment, which needs to be further eliminated by subsequent non-rigid registration.

[0024] Edge contour extraction employs the Canny operator applied to both the optical image under inspection and the initial aligned layout reference image. The high threshold is set to the 90th quantile of the image gradient magnitude distribution, while the low threshold is set to 40% of the high threshold, allowing contour extraction to adapt to the contrast of different products. For each extracted graphic edge contour, a Douglas-Peucker polygon approximation is performed with a tolerance of 1.5 pixels. Inflection points in the approximation result are used as corner points. Semiconductor layouts primarily consist of rectangular and right-angled polygonal shapes, and the inflection point positions recur stably in both images, making them suitable as registration control points. Alternatively, corner points can be obtained using Harris corner detection.

[0025] Corner point pairing is performed within a preset neighborhood. The preset neighborhood has a radius of 8 pixels. This value is, on the one hand, lower than 0.5 times the minimum repeating period of the graphic in the layout of this embodiment (approximately 20 pixels, corresponding to approximately 7 micrometers), avoiding mispairing with similar corner points in adjacent periods; on the other hand, it is higher than the upper limit of the initial alignment residual deviation, ensuring that the true corresponding corner points fall within the neighborhood. For each corner point in the graphic edge contour of the optical image to be inspected, the nearest corner point within the preset neighborhood is selected from the corner points of the graphic edge contour of the layout reference image after initial alignment, and paired as the corresponding control point. For example... Figure 2 As shown, circular and square markers represent the corner points of the graphic edge contours of two images, respectively. A dashed circle centered on a corner point represents a preset neighborhood range. Corner points within this neighborhood range that are closest to each other are paired as corresponding control points. Similar corner points in adjacent periods are excluded because they are outside the preset neighborhood range. To further improve pairing reliability, this embodiment also applies a nearest neighbor constraint: pairing is only retained when two corner points are the closest to each other in the other's candidate set. Pairings with a distance residual exceeding three times the median of all pairing residuals are discarded as gross errors. Both of these processes are optional implementations.

[0026] The smooth deformation field is solved using thin plate spline interpolation based on the corresponding control points. Thin plate spline interpolation, constrained by control point displacement and aiming to minimize bending energy, yields a deformation field that smoothly transitions between control points. This aligns with the physical characteristics of gradually varying geometric deviations such as optical distortion, wafer warpage, and clamping rotation, making it suitable for describing the non-rigid geometric relationship between the layout reference image and the optical image under inspection. Regularization is introduced during the solution process to suppress overfitting to individual noise pairs. The regularization strength is selected through a leave-out verification: 20% of the corresponding control points are randomly left out of the solution and used only to evaluate the registration residuals. The regularization strength that minimizes the median of the left-out corner point residuals is selected. In this embodiment, the median residual of the left-out corner points after registration is 0.6 pixels; a median residual below 1 pixel is considered a successful registration; if it exceeds 1 pixel, the regularization strength is increased and the solution is recalculated. If it still fails to meet the requirements, the field of view is re-acquired. Figure 3 As shown, the dashed lines represent the regular grid before deformation, and the solid lines represent the grid after the smooth deformation field. The arrows indicate the displacement of each grid node. The displacement is magnified 3 times in the figure. The deformation changes continuously and gradually within the field of view without any abrupt changes. The layout reference image is resampled based on the smooth deformation field, and bilinear interpolation is used for the grayscale channels. The layer identifier set recorded pixel by pixel belongs to discrete category labels. Linear interpolation will produce non-existent intermediate values. Therefore, the resampling of the layer identifier set uses nearest neighbor interpolation, that is, each output pixel takes the layer identifier set of the nearest pixel at the deformation field mapping position. This yields an aligned layout reference image that corresponds pixel by pixel to the optical image under inspection.

[0027] Region segmentation is performed based on the layer identifier set recorded for each pixel in the aligned layout reference image: pixels with at least one layer identifier corresponding to the device functional graphic are assigned to the critical graphic region, and the remaining pixels are assigned to the redundant graphic region. Pixels with an empty layer identifier set for the dielectric background are also assigned to the redundant graphic region. When the device functional graphic and the fill metal block overlap on the same pixel, that pixel is assigned to the critical graphic region. This segmentation is conservative to avoid weakening the detection sensitivity at the functional graphic boundaries. The segmentation result is a priori layout region map with pixel-by-pixel labeled regions. Figure 4 As shown, the pixels covered by the diagonally filled redistribution metal lines, vias, and pads belong to the critical graphic area, while the dotted filled metal blocks, test graphics, and blank media background belong to the redundant graphic area.

[0028] During cascaded inspection, adjacent grain images are first aligned with the optical image to be inspected: the phase correlation method is used to calculate the integer pixel translation, and then parabolic fitting is performed on the neighborhood of the correlation peak to obtain the sub-pixel translation; since the patterns of adjacent grains on the same wafer are consistent, the translation model is sufficient to describe the geometric relationship between them. After alignment, pixel-by-pixel difference is performed and the absolute value is taken to obtain the difference image. The binarized segmentation value is adaptively determined based on the gray-level distribution of the difference image: in this embodiment, the maximum inter-class variance method is used, and the gray-level value that maximizes the inter-class variance between the foreground class and the background class is selected as the binarized segmentation value on the gray-level histogram of the difference image; when the proportion of foreground pixels exceeds 5% after binarization according to this segmentation value, it is judged as overall brightness fluctuation or noise in the segmentation value, and the median gray level of the difference image plus 3 times the absolute deviation of the gray-level median is taken as the binarized segmentation value. Adaptively determining the segmentation value based on the gray-level distribution instead of a fixed segmentation value is to adapt to the changes in contrast and noise levels between different products and different batches. Connected components are extracted from the binarization result by 8 neighborhoods, and each extracted connected component is recorded as a candidate abnormal region; as an optional implementation, connected components with an area of ​​less than 4 pixels are filtered out as isolated noise.

[0029] The attribution of each pixel within a candidate anomaly region to the prior region map is statistically analyzed, and the region category with the majority of pixels is recorded as the region attribute of the candidate anomaly region. A 64-pixel × 64-pixel local optical patch is cropped from the optical image to be inspected, centered on the center of the candidate anomaly region. If the center is less than 32 pixels from the image boundary, the cropping window is shifted inwards. The feature extraction network employs an 18-layer residual network (ResNet-18) pre-trained on a public image dataset. The output feature map from its third stage is then subjected to global average pooling to obtain 256-dimensional patch features. The low- and mid-layer features of the pre-trained network characterize edges, textures, and local structures, exhibiting cross-domain versatility. It can be used to measure appearance differences between patches without defect annotation, making it suitable for semiconductor quality inspection scenarios where defect samples are scarce. As an optional implementation, the feature extraction network can be replaced with an encoder pre-trained in a self-supervised manner on good product patches, which can further align with the appearance distribution of specific products. A reference feature library is pre-constructed: 50 defect-free, high-quality wafers of the same model are selected. For each wafer sample image, a 64-pixel × 64-pixel patch is extracted with a 32-pixel step size. Features are extracted one by one using a feature extraction network, and the layout position of the patch center is recorded and stored. As an optional implementation, greedy farthest-point sampling is performed on reference features near the same layout position to retain a representative subset of features, thus reducing storage and retrieval overhead. During online retrieval, the range of reference features is limited to those within 16 pixels of the center of the candidate anomaly region from the layout position. The reason for limiting comparisons to the same position is that the wafer pattern changes with the layout position, and cross-position retrieval will cause the feature distance to be mixed with pattern differences, failing to reflect appearance anomalies. The Euclidean distance between the patch feature and the nearest retrieved reference feature is used as the anomaly score of the candidate anomaly region. The anomaly score is dimensionless, and in this embodiment, it is retained to 3 decimal places. The larger the value, the farther the appearance deviates from the appearance of the high-quality wafer.

[0030] The release threshold and rejection threshold are obtained by calibrating key graphic regions and redundant graphic regions respectively on a pre-constructed calibration image set. The calibration image set is selected from images of three batches of wafers of the same model to cover process variations between batches. The calibration sample images in the calibration image set are processed in the same way as the optical images to be inspected, resulting in 12,000 calibration candidate anomalous regions, of which 7,150 are key graphic regions and 4,850 are redundant graphic regions. Each calibration candidate anomalous region is manually labeled with a defect conclusion or a good product conclusion as the labeling conclusion. There are 850 defect conclusions and 6,300 good product conclusions in the key graphic regions, and 320 defect conclusions and 4,530 good product conclusions in the redundant graphic regions. In this embodiment, the preset missed detection risk level for the key graphic regions is 1%, and the preset over-detection risk level is 10%; the preset missed detection risk level for the redundant graphic regions is 5%, and the preset over-detection risk level is 5%. The key graphic area carries the device function, and is configured with a lower level of missed detection risk to prioritize the suppression of missed detection, while allowing for relatively lenient over-detection; the redundant graphic area has a large number of false defect alarms, and is configured with a relatively strict level of over-detection risk to suppress over-detection and reduce the burden of re-judgment.

[0031] The calibration process is illustrated using a key graphic region as an example. 850 candidate abnormal regions for defect conclusions are sorted in ascending order of their abnormality scores. A preset miss detection risk level of 1% corresponds to an allowable count of 8 regions, which is calculated by multiplying 850 by 1% and rounding down. The maximum abnormality score that satisfies the miss detection constraint is selected as the release threshold. In this embodiment, the release threshold is 0.284. There are exactly 8 candidate abnormal regions for defect conclusions with an abnormality score lower than or equal to 0.284, accounting for approximately 0.94%, which is within the preset miss detection risk level. Any abnormality score higher would cause this percentage to exceed 1%, thus failing to meet the miss detection constraint. The 6300 candidate abnormal regions for good product conclusions were sorted in descending order of their abnormality scores. A preset over-detection risk level of 10% corresponds to an allowable count of 630 regions. The minimum abnormality score satisfying the over-detection constraint was selected as the rejection threshold. In this embodiment, the rejection threshold was 0.388. There were 627 candidate abnormal regions for good product conclusions with an abnormality score higher than or equal to 0.388, accounting for approximately 9.95%, which is within the preset over-detection risk level. Redundant graphic regions were labeled in the same way, with allowable counts of 16 and 226 respectively. The resulting release threshold was 0.355 and the rejection threshold was 0.374, corresponding to a missed detection rate of 5.00% and an over-detection rate of approximately 4.99%. The threshold interval between the two redundant graphic regions was significantly narrower than that of the critical graphic regions, resulting in fewer gray-area abnormal regions and a lighter re-judgment burden. Figure 5As shown, the horizontal axis represents the anomaly score, and the vertical axis represents the number of candidate anomaly regions. The score distributions of good and defective conclusions overlap in the key graphical region. The release threshold of 0.284 and the rejection threshold of 0.388 divide the score axis into three segments. The light-colored interval between the two dashed lines represents the score interval corresponding to the gray-area anomaly region. The shaded area covered by the defect conclusion distribution curve below the release threshold represents the missed detection count area, and the mesh-like shading covered by the good conclusion distribution curve above the rejection threshold represents the over-detection count area. The proportion of samples in the two count areas represents the constrained missed detection ratio and over-detection ratio. Figure 6 As shown, the horizontal axis represents the abnormal score used as the threshold value, and the vertical axis represents the proportion. As the value increases, the cumulative missed detection proportion remains monotonically constant, while the cumulative over-detection proportion remains monotonically constant. The release threshold is the maximum value where the cumulative missed detection proportion does not exceed 1% of the preset missed detection risk level, and the rejection threshold is the minimum value where the cumulative over-detection proportion does not exceed 10% of the preset over-detection risk level. The monotonicity of the two proportion curves ensures that the threshold selected according to the counting proportion is uniquely determined.

[0032] The essence of the above calibration is to perform count-based risk control on the calibrated image set. Its effectiveness depends on the calibrated candidate abnormal areas and the actual samples on the production line coming from similar distributions. Therefore, the calibrated image set should cover representative batches and operating conditions, and be recalibrated after significant process adjustments. The number of defect conclusion calibration samples determines the resolution capability of the proportional constraint. The number should reach the order of magnitude of the reciprocal of the preset missed detection risk level. For example, when the preset missed detection risk level is 1%, the defect conclusion calibration samples should reach more than 100. If the number of defect conclusion calibration samples under a certain regional attribute is insufficient, or if they are completely missing in the early stage of new product introduction, the release threshold is conservatively determined by default to the quintile of the abnormality scores of all good product conclusion calibration candidate abnormal areas under that regional attribute. This ensures that direct release only covers candidate abnormal areas with appearances highly consistent with good products, while the rejection threshold is still calibrated normally according to the good product conclusion calibration samples. When the release threshold obtained under the same regional attribute is higher than or equal to the rejection threshold, which is a situation that may occur when the score distribution is seriously overlapping and the preset risk level is relatively lenient, the maximum value of the abnormal score of each candidate abnormal region below the rejection threshold will be re-determined as the release threshold. After the release threshold is reduced, the candidate abnormal regions with abnormal scores lower than or equal to the release threshold constitute a subset of the original set, the proportion does not increase, and the missed detection constraint remains valid. The cost of re-determination is that the score range between the two thresholds is expanded and the number of gray area abnormal regions increases, which is an adjustment towards safety.

[0033] During online inspection, a three-way split is performed based on the regional attributes of candidate abnormal regions, selecting corresponding release and rejection thresholds: candidate abnormal regions with anomaly scores lower than or equal to the release threshold are judged as good products and recorded as good product release results; candidate abnormal regions with anomaly scores higher than or equal to the rejection threshold are judged as defects and recorded as defect judgment results; candidate abnormal regions with anomaly scores higher than the release threshold but lower than the rejection threshold are recorded as gray-zone abnormal regions and placed in the re-judgment queue. The re-judgment queue is consumed asynchronously, and re-judgment is performed as a bypass of the main inspection path. The main inspection path runs at a fixed pace and is not hindered by fluctuations in re-judgment time. Statistical results on the verification wafer show that approximately 88% of all candidate abnormal regions are directly judged through the three-way split, and gray-zone abnormal regions account for approximately 12%. The inference time of the large visual language model is correspondingly reduced to approximately 1 / 8 of that when performing inference on all candidate abnormal regions.

[0034] Evidence packages are assembled for each gray-area anomaly region in the review queue. A 64-pixel × 64-pixel local optical patch centered on the gray-area anomaly region is taken, and is considered the same patch as the local optical patch extracted during feature extraction. During assembly, bicubic interpolation is used to enlarge and resample the patch to 448 pixels × 448 pixels to match the image input size of the large visual language model. The layout reference patch is extracted from the aligned layout reference image at the same position and size, and the same enlargement and resampling are performed. The layout category label patch is generated based on the layout reference patch: for each continuous graphic within the patch, the design graphic category name is determined according to its layer identifier and the mapping relationship between the layer identifier and the design graphic category name. The corresponding name is superimposed at the centroid of the graphic, using black text on a white background, with a text height of 1 / 16 of the patch width. If the graphic area is insufficient to accommodate the text, as an optional implementation, the text label is placed in the legend bar at the edge of the patch and pointed to the graphic with a leader line. The regional attributes of the gray-area anomaly region are written into the evidence package as text fields. The defect category knowledge base is structured and contains 18 defect category entries, numbered D01 to D18. Each entry records the category name, morphological feature text, and judgment criterion text. When assembling the evidence package, all 18 defect category entries are read. For example, the category name of entry D01 is "foreign object," and its morphological feature text describes the presence of isolated clumps on the surface of a graphic or medium that have a significant grayscale contrast with the surrounding material. Its judgment criterion text stipulates that an isolated clump covering a redistribution metal line, via, or pad with a maximum span of more than 0.5 times the line width of the graphic is considered a defect. The category name of entry D03 is "metal bridging," and its morphological feature text describes the presence of metal residue connecting adjacent redistribution metal lines. Its judgment criterion text stipulates that a metal residue connecting adjacent redistribution metal lines is considered a defect.

[0035] The multimodal prompt consists of task instructions, evidence package content, and output format conventions. The output format convention specifies that the visual language large model outputs a structured judgment report using a predefined key-value field format. In this embodiment, the key-value field format is JSON (JavaScript Object Notation). The fields include: a judgment conclusion field, with a value of either defect or good product; a hit item number field, filled in when the judgment conclusion is defect; and an evidence chain field, a list of evidence items, including an evidence source identifier, location description, and comparison description. When the evidence source identifier is a defect category item, it also records the referenced item number. The multimodal prompt also specifies the requirements for the evidence chain: when the region attribute is a key graphic region, the evidence chain consists of at least 3 evidence items, of which at least 1 evidence item's evidence source identifier is a layout reference block; when the region attribute is a redundant graphic region, the evidence chain consists of at least 2 evidence items. The requirement that key graphic regions must include evidence items with evidence source identifiers of layout reference blocks is intended to ensure that the re-judgment conclusion on the functional graphics necessarily compares with the design layout data. This comparison undergoes coordinate-level verification during machine validation. The location description uses directional terms and pixel distances with the center of the gray area anomaly region as the origin. The set of directional terms is predefined as eight terms: top, bottom, left, right, top left, top right, bottom left, and bottom right. The image coordinate system is defined as follows: row coordinates increase downwards, column coordinates increase to the right, top corresponds to the direction of decreasing row coordinates, right corresponds to the direction of increasing column coordinates, top left corresponds to the diagonal direction where both row and column coordinates decrease simultaneously, and so on for the remaining directional terms. The location description is converted to image coordinates as follows: along the top and bottom directions, only the row coordinate is increased or decreased by pixel distance; along the left and right directions, only the column coordinate is increased or decreased by pixel distance; along the diagonal direction, the increase or decrease of pixel distance for both row and column coordinates is multiplied by 0.707 and rounded to the nearest integer. For example, if the center of the gray area anomaly is located at row 1024 and column 1024 of the aligned layout reference image, and its position is described as upper right and 12 pixels, then multiplying 12 by 0.707 and rounding to 8 gives the calculated image coordinates as row 1016 and column 1032. To improve the stability of the output format compliance, two demonstration judgment reports are included in the multimodal prompts as few-sample examples. As an optional implementation, supervised fine-tuning of the visual language large model is performed using hundreds of manually written demonstration judgment reports. The visual language large model in this embodiment uses an open-source multimodal large model with hundreds of billions of parameters, deployed on an inference server equipped with a graphics processor; a closed-source multimodal large model service with private deployment that meets production line data security requirements is also feasible.

[0036] Taking an abnormal gray area located at the edge of a redistribution metal line as an example, the structured judgment report output by the visual language large model concludes that it is a defect, with the hit item number D01. The evidence chain contains three evidence items: The evidence source of the first evidence item is identified as a local optical patch, with the location described as being 5 pixels to the right. The description states that a nearly circular dark blob with clear boundaries was observed at this location. The evidence source of the second evidence item is identified as a layout reference patch, with the location described as being 5 pixels to the right. The description states that the design graphic category name marked at the same location in the layout category label patch is "redistribution metal line," and the blob falls on the redistribution metal line. The evidence source of the third evidence item is identified as a defect category item, referencing item number D01, with the location described as being 5 pixels to the right. The description states that the maximum span of the blob is approximately 0.7 times the width of the redistribution metal line, meeting the condition of more than 0.5 times as specified in the judgment criterion text for this item.

[0037] Machine verification is performed on the structured judgment report. First, the key-value fields are parsed and their completeness is checked. Then, the evidence chain is checked to see if it meets the evidence chain requirements corresponding to the regional attributes of the gray area anomaly region. If any requirement is not met, the verification result is directly marked as questionable. If it meets the requirement, each evidence item is verified. For evidence items whose source is identified as defect category entries, the recorded entry number is checked to see if it exists in the defect category knowledge base. The judgment criterion text referenced by the evidence item is compared with the judgment criterion text stored under the same entry number in the defect category knowledge base using string precision comparison. Before comparison, only leading and trailing whitespace characters are removed. If the comparison passes, it is considered to be consistent word by word. Word-by-word consistency is used instead of semantic similarity as the comparison method because semantic similarity judgment itself depends on the model and has uncertainty. Word-by-word consistency verification is a deterministic verification, and the result is reproducible and auditable. Evidence items whose source is identified as map reference blocks are converted into image coordinates in the aligned map reference image based on the correspondence between directional terms and directions in the directional term set. The layer identifier set of the pixel records at the image coordinates is read, and the design graphic category name is retrieved according to the mapping relationship between the layer identifier and the design graphic category name. When the pixel is covered by multiple graphics, multiple names are retrieved. It is verified that there is a name in the retrieved design graphic category name that matches the design graphic category name recorded in the comparison description. If the layer identifier set of the pixel at the image coordinates is empty, that is, when the identified position falls on the medium background, the query result is empty, and the verification will definitely fail. For each evidence item, it is verified that the image coordinates converted from the position description fall within the bounding rectangle of the gray area anomaly region. The bounding rectangle is the smallest bounding rectangle of the connected component of the gray area anomaly region in the row and column directions. The bounding rectangle is used instead of the pixel-by-pixel set as the region criterion to provide a deterministic tolerance for deviations of several pixels in the position description of the large model of visual language. When the judgment conclusion is a defect, it is verified that the hit item number exists in the defect category knowledge base. The verification result is "passed" if all checked items pass; otherwise, the verification result is "questionable." Figure 7 As shown, a correspondence between eight directional terms and image coordinate directions was established with the center of the gray area anomaly as the origin. The evidence item with a location description of upper right and 9 pixels was converted to row 26, column 38. The converted coordinates fell within the bounding rectangle, thus passing the location check. The evidence item with a location description of right and 28 pixels was converted to row 32, column 60. The converted coordinates fell outside the bounding rectangle, failing the location check. Therefore, the entire structured judgment report was subsequently changed to questionable. Figure 8As shown, each graphic in the map category label block has a white background text label with the corresponding design graphic category name superimposed on it. The evidence item with the coordinates located at row 21, column 56, has a design graphic category name of "Rewiring Metal Line" obtained through layer identifier lookup, which matches the name recorded in the comparison description, and the verification passes. The evidence item with the coordinates located at row 33, column 30, falls on the media background, the pixel layer identifier set is empty, the query result is empty, which is inconsistent with the recorded "Rewiring Metal Line", and the verification fails. The machine verification re-executed the query and generated the text label based on the same mapping relationship, so the focus of the verification is on the spatial accuracy of the location identification: even if the evidence item copied the text label in the map block, as long as the identification location is incorrect, the name obtained by the query will not match the recorded name, and the corresponding evidence item will be intercepted. The illusion of location fabrication is identified definitively before reaching the final judgment result.

[0038] When the verification result is "pass," the judgment conclusion in the structured judgment report is adopted as the final judgment result for the gray area anomaly. When the verification result is "questionable," the gray area anomaly, along with the evidence package, the structured judgment report, and the item-by-item verification record, is pushed to the manual review end. The interface of the manual review end displays local optical blocks, layout reference blocks, and layout category label blocks side by side, marking the failed verification items. The manual reviewer then makes a judgment, which serves as the final judgment result for that gray area anomaly. As an optional implementation, the judgment given by the manual reviewer, along with the corresponding blocks, is fed back to supplement the calibration image set, periodically triggering the recalibration of the release threshold and rejection threshold to ensure that the thresholds remain effective as the process state evolves. After all field-of-view processing is completed, the good product release results, defect judgment results, and the final judgment results for each gray area anomaly are summarized, along with the evidence chain for each gray area anomaly and the verification records generated by machine verification, to generate a wafer-level quality inspection report. The report is used for yield statistics, defect spatial distribution analysis, and traceability auditing of the judgment process. The operational data on the verification wafer in this embodiment are as follows: the machine verification pass rate is about 82%, and about 18% of the gray area abnormal areas are transferred to manual review. When converted to all candidate abnormal areas, the manual review rate is about 2.2%. The measured missed detection rate of key graphic areas is 0.8% and the over-detection rate is 9.1%. The measured missed detection rate of redundant graphic areas is 4.2% and the over-detection rate is 4.6%, all of which are within the corresponding preset risk levels.

[0039] In terms of deployment, the AOI imaging equipment and cascaded detection processing are deployed on the edge server of the production line, the visual language large model is deployed on the inference server in the computer room, and the review queue is transmitted via the internal network of the production line; all components can also be centrally deployed on the same server, and neither deployment mode affects the above processing. The above optional implementation methods and the optional implementation methods and this embodiment can be combined with each other.

[0040] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semiconductor AOI visual quality inspection intelligent agent platform, characterized in that, It includes AOI imaging equipment, cascaded detection module, scheduling module, evidence chain verification module and visual language big model. The scheduling module drives the AOI imaging equipment, cascaded detection module, visual language big model and evidence chain verification module to work together in a tool call manner. AOI imaging equipment is used to acquire optical images of the die to be inspected on a wafer and images of adjacent dies; The scheduling module is used to construct an aligned layout reference image that corresponds to the optical image under inspection pixel by pixel and records the layer identifier pixel by pixel, based on the design layout data corresponding to the die to be inspected, and to construct a layout prior area map that marks key graphic areas or redundant graphic areas pixel by pixel. The cascaded detection module is used to extract candidate abnormal regions based on the differences between adjacent grain images and the optical image to be inspected. It determines the regional attributes of the candidate abnormal regions based on the layout prior region map, determines the abnormal score of the candidate abnormal regions, and selects the corresponding release threshold and rejection threshold based on the regional attributes to perform three-way splitting: candidate abnormal regions with abnormal scores lower than or equal to the release threshold are judged as good products, candidate abnormal regions with abnormal scores higher than or equal to the rejection threshold are judged as defects, and candidate abnormal regions with abnormal scores higher than the release threshold and lower than the rejection threshold are recorded as gray area abnormal regions and placed into the re-judgment queue. The scheduling module is also used to assemble evidence packages for gray area anomalies in the review queue and organize the evidence packages into a large multimodal prompt input visual language model; The visual language big model is used to output a structured judgment report that includes a judgment conclusion and a chain of evidence. The judgment conclusion is either defective or good. The evidence chain verification module is used to perform machine verification on the structured judgment report. The verification result of the machine verification is either pass or questionable. When the verification result is pass, the evidence chain verification module outputs the judgment conclusion in the structured judgment report as the final judgment result of the gray area anomaly. When the verification result is questionable, the scheduling module pushes the gray area anomaly to the manual review end, and the judgment given by the manual review end is used as the final judgment result of the gray area anomaly. Based on the layer identifiers recorded for each pixel in the aligned layout reference image, the scheduling module assigns at least one pixel whose recorded layer identifier corresponds to the device functional graphic to the key graphic region, and assigns the remaining pixels to the redundant graphic region, thus obtaining the layout prior region map. The cascaded detection module records the region category to which the pixels in the candidate anomaly region belong as the region attribute of the candidate anomaly region. The value of the region attribute is either a key graphic region or a redundant graphic region. A local optical patch is extracted from the optical image to be inspected, centered on the candidate anomaly region. The local optical patch is then input into a pre-trained feature extraction network to obtain patch features. In a pre-built reference feature library, the reference feature closest to the patch feature is retrieved within the reference feature range corresponding to the layout position of the candidate anomaly region. The feature distance between the patch feature and the retrieved reference feature is used as the anomaly score of the candidate anomaly region. The reference feature library is pre-built by dividing the images of good-quality crystal samples of the same model into patches according to the layout position, extracting features through the feature extraction network, and recording and storing them together with the layout position.

2. The platform according to claim 1, characterized in that, The process of constructing the aligned layout reference image by the scheduling module includes: rasterizing the graphics in the design layout data corresponding to the field of view of the optical image under test according to the resolution of the optical image under test; during rasterization, recording the layer identifier of the graphics covered by each pixel; and performing Gaussian smoothing on the rasterization result to make the gray-scale transition shape of the graphic edge close to that of the graphic edge in the optical image under test, thus obtaining the layout reference image; initially aligning the layout reference image with the optical image under test according to the stage coordinates recorded by the AOI imaging device; and extracting the optical image under test... The graphic edge contour of the image is compared with the graphic edge contour of the initial aligned layout reference image. For each corner point in the graphic edge contour of the optical image to be inspected, the nearest corner point within a preset neighborhood range is selected from the corner points of the graphic edge contour of the initial aligned layout reference image and paired as corresponding control points. Based on the corresponding control points, thin plate spline interpolation is used to solve the smooth deformation field. Based on the smooth deformation field, the same alignment transformation is performed on the layout reference image and its pixel-by-pixel recorded layer identifiers to obtain the aligned layout reference image corresponding to the pixel-by-pixel optical image to be inspected.

3. The platform according to claim 1, characterized in that, Adjacent grains are grains on the same wafer with the same layout as the grain to be inspected. The adjacent grain images are images acquired by AOI imaging equipment under the same field of view as the optical image to be inspected. The cascaded detection module aligns the adjacent grain images with the optical image to be inspected and then performs pixel-by-pixel difference to obtain a difference image. Based on the grayscale distribution of the difference image, it adaptively determines the binarization segmentation value, performs binarization and connected component extraction on the difference image, and records each extracted connected component as a candidate abnormal region.

4. The platform according to claim 1, characterized in that, The critical graphic region and the redundant graphic region are each configured with an approval threshold and a rejection threshold, and the preset missed detection risk level configured for the critical graphic region is lower than that configured for the redundant graphic region. The approval threshold and rejection threshold are obtained by calibrating the critical graphic region and the redundant graphic region respectively on a pre-constructed calibration image set. The calibration sample images in the calibration image set are processed in the same way as the optical images to be inspected to obtain the anomaly score and region attribute of each calibration candidate anomaly region. Each calibration candidate anomaly region is manually labeled with a defect conclusion or a good product conclusion as the labeling conclusion. For the same region attribute, the calibration candidate anomaly regions labeled with a defect conclusion are arranged in ascending order of anomaly score. The maximum anomaly score that meets the missed detection constraint is selected as the release threshold. The missed detection constraint is that the proportion of the number of candidate anomaly regions for defect conclusions with anomaly scores lower than or equal to the release threshold to the total number of candidate anomaly regions for all defect conclusions under the same regional attribute is within the preset missed detection risk level. The candidate anomaly regions marked as good product conclusions are sorted in descending order of anomaly scores, and the minimum anomaly score that meets the overkill constraint is selected as the rejection threshold. The overkill constraint is that the proportion of the number of candidate anomaly regions for good product conclusions with anomaly scores higher than or equal to the rejection threshold to the total number of candidate anomaly regions for all good product conclusions under the same regional attribute is within the preset overkill risk level.

5. The platform according to claim 4, characterized in that, When the release threshold obtained under the same regional attribute is higher than or equal to the rejection threshold, the maximum value of the abnormality scores of each candidate abnormal region that is lower than the rejection threshold is re-determined as the release threshold. The re-determined release threshold is lower than the rejection threshold, and the missed detection constraint remains valid.

6. The platform according to claim 1, characterized in that, The evidence package includes: a local optical patch of the gray area anomaly extracted from the optical image under inspection; a layout reference patch extracted from the same position in the aligned layout reference image; a layout category annotation patch obtained by overlaying text labels corresponding to the design graphic category names at the positions of each graphic in the layout reference patch; the regional attributes of the gray area anomaly; and all defect category entries read from the defect category knowledge base; the design graphic category name in the text label is determined according to the layer identifier of the graphic based on the pre-established mapping relationship between the layer identifier and the design graphic category name; the defect category entries have an entry number and record the category name, morphological feature text, and judgment criterion text.

7. The platform according to claim 6, characterized in that, The multimodal prompting convention specifies that the visual language large model outputs a structured judgment report in a predefined key-value field format, and stipulates the following requirements for the chain of evidence: when the region attribute is a key graphic region, the chain of evidence consists of at least 3 evidence items, of which at least 1 evidence item's evidence source identifier is a map reference block; when the region attribute is a redundant graphic region, the chain of evidence consists of at least 2 evidence items; when the judgment conclusion is a defect, the structured judgment report records the number of the matched item; each evidence item includes an evidence source identifier, a location description, and a comparison description. The value of the evidence source identifier can be a local optical block, a map reference block, or a defect category item. When the evidence source identifier is a defect category item, the evidence item records the reference... The entry number used; the location description uses directional words and pixel distance with the center of the gray area anomaly region as the origin. The directional words are taken from a predefined set of directional words, and each directional word in the set corresponds to a specific direction in the image coordinate system; when the evidence source identifier is a local optical patch, the description records the morphology observed in the local optical patch; when the evidence source identifier is a layout reference patch, the description records the comparison between the observed morphology and the design graphic category name marked in the same position in the layout category label patch; when the evidence source identifier is a defect category entry, the description records the correspondence between the observed morphology and the judgment criterion text under the referenced entry number.

8. The platform according to claim 7, characterized in that, The evidence chain verification module performs machine verification as follows: It checks the completeness of the key-value fields and the structure of the evidence chain in the structured judgment report. If the key-value fields are complete and the evidence chain structure meets the requirements for the evidence chain structure corresponding to the regional attributes of the gray area anomaly region, it verifies each evidence item. Otherwise, the verification result is recorded as questionable. Item-by-item verification includes: when the evidence source identifier is a defect category item, it verifies that the recorded item number exists in the defect category knowledge base and verifies that the judgment criterion text referenced by the evidence item is consistent word-for-word with the judgment criterion text stored under the same item number in the defect category knowledge base; when the evidence source identifier is a map reference block, it converts the location description into an image aligned with the map reference block based on the correspondence between directional words and directions in the directional word set. For example, based on the layer identifier recorded by the pixels at the image coordinates, the design graphic category name is retrieved according to the mapping relationship between the layer identifier and the design graphic category name. The retrieved design graphic category name is checked to see if there is a name that matches the design graphic category name recorded in the comparison description. For each piece of evidence, the image coordinates converted from the position description are checked to see if they fall within the bounding rectangle of the gray area anomaly region. When the judgment conclusion is a defect, the item number of the hit item is checked to see if it exists in the defect category knowledge base. When all the verification items pass, the verification result is passed; otherwise, the verification result is questionable. When the verification result is questionable, the scheduling module pushes the gray area anomaly region along with the evidence package, the structured judgment report, and the verification record to the manual review end.

9. The platform according to any one of claims 1 to 8, characterized in that, The cascaded detection module records candidate abnormal areas that are determined to be good products as good product release results, and records candidate abnormal areas that are determined to be defects as defect determination results. The intelligent agent platform summarizes the good product release results, defect judgment results, and final judgment results of each gray area and abnormal area, together with the evidence chain of each gray area and abnormal area and the verification record generated by machine verification, to generate a wafer-level quality inspection report.

Images