Multimodal fusion of integrated circuit design data and adversarial generative visual inspection methods

By establishing physical spatial alignment between scanning electron microscope images and layout geometric topology files in integrated circuit visual inspection, an accurate symbolic distance field discrete matrix is ​​generated, solving the feature illusion problem in the multimodal fusion process, realizing end-to-end physical compliance constraint training, and improving the effectiveness of model training and computational efficiency.

CN123072016APending Publication Date: 2026-11-10贯文信息技术(苏州)有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611292870.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-25
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In the current integrated circuit visual inspection, feature illusions are easily generated during the multimodal fusion process, resulting in false defect samples that appear reasonable but violate design rules, leading to reduced effectiveness of model training benchmarks and wasted computing power.

Method used

By acquiring scanning electron microscope images and integrating integrated circuit layout geometric topology files, a physical spatial alignment relationship is established to generate an accurate symbolic distance field discrete matrix. Combined with the geometric constraint bias matrix, a cross-modal attention fusion feature matrix is ​​calculated. Differentiable neural agent models are used to perform differentiable discrimination of hallucination features. Based on the dynamic uncertain boundary mechanism, a joint calibration loss is output. Combined with perceptual similarity loss, iterative feature correction is performed to output compliant visual detection training samples.

Benefits of technology

The feature illusion problem has been solved, and end-to-end joint backpropagation training of multimodal fusion and adversarial generation of integrated circuit design data has been achieved. This improves the effectiveness of the model training benchmark and avoids wasting computing power and impairing the diversity of model generalization samples.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123072016A_ABST
    Figure CN123072016A_ABST
Patent Text Reader

Abstract

This invention relates to the field of image recognition technology and discloses a visual detection method for multimodal fusion and adversarial generation of integrated circuit design data. The method acquires scanning electron microscope images and establishes a physical spatial alignment relationship with the corresponding integrated circuit layout, generates a precise symbolic distance field discrete matrix and transforms it into a hard constraint bias matrix, and calculates and generates a cross-modal attention fusion feature matrix. This matrix is ​​then input into a pre-trained differentiable neural proxy model and a joint verification classification network to perform differentiable discrimination of hallucination features and output a joint calibration loss. Micro-step iterative feature correction is performed by combining the hallucination probability error gradient and perceptual similarity loss, and high-fidelity visual detection training samples are output through closed-loop adaptive repair. This invention can solve the problem of feature space misalignment, suppress feature hallucinations that violate design rules in multimodal fusion, achieve end-to-end training of physical compliance constraints, and enable generated samples to possess both circuit topology compliance and visual realism of defects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically a visual detection method for multimodal fusion and adversarial generation of integrated circuit design data. Background Technology

[0002] As integrated circuit manufacturing processes advance towards nanometer-level nodes, visual inspection based on scanning electron microscope (SEM) images has become a crucial step in ensuring yield. To improve the generalization ability of visual inspection models, the use of generative adversarial networks (GANs) to synthesize training samples containing defects has become a mainstream approach. Multimodal fusion in integrated circuit visual inspection refers to combining SEM images with integrated circuit layout geometric topology files. By establishing cross-modal feature mapping, the goal is to generate defect images that possess both physically realistic texture features and conform to the underlying circuit topology geometry.

[0003] Current technologies in this field mostly employ feature-level stitching and conventional attention mechanisms to achieve cross-modal fusion of image and layout data. One mainstream approach converts the layout topology file into a two-dimensional pseudo-image, performs channel stitching with scanning electron microscope images, and directly inputs it into a generative adversarial network for feature mapping and reconstruction. Another approach utilizes commercial discrete rule verification tools to post-screen the samples output by the generative network. By independently executing Boolean logic, it determines whether the generated defective samples violate physical design rules such as minimum line width and minimum line spacing, thereby selecting compliant samples for inclusion in the visual inspection model training set.

[0004] However, existing technologies still have shortcomings in multimodal fusion and physical compliance verification. Specifically, existing integrated circuit visual inspection and adversarial generative networks are prone to feature illusions during multimodal fusion, outputting seemingly reasonable but actually violating integrated circuit design rules and containing false defect samples. This problem arises because there is a misalignment between the feature space of the image pixel coordinate system and the vector topological coordinate system. Furthermore, existing post-verification methods use discrete rule verification, whose Boolean logic is non-differentiable, preventing the integration of physical rule constraints into the neural network for end-to-end joint backpropagation training. Mixing such illusory violation samples into the visual inspection training set significantly reduces the effectiveness of the model training benchmark, leading to an increased false alarm rate in real manufacturing production lines. Simply discarding data that fails discrete verification or high-value marginal data also directly wastes computational resources and impairs the diversity of model generalization samples. Therefore, there is an urgent need for multimodal fusion and adversarial generative visual inspection methods for integrated circuit design data to address these issues. Summary of the Invention

[0005] To address the problems in related technologies, this invention provides a visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data, thereby overcoming the aforementioned technical problems in existing related technologies.

[0006] To solve the aforementioned technical problem, the present invention is achieved through the following technical solution:

[0007] This invention provides a visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data, specifically including:

[0008] A physical spatial alignment relationship is established between the acquired scanning electron microscope image data sequence containing real defects and the corresponding integrated circuit layout geometric topology file, and a fast traversal algorithm is used to generate an accurate symbol distance field discrete matrix;

[0009] Visual feature vectors and topological feature vectors are extracted, the precise symbolic distance field discrete matrix is ​​transformed into a geometric constraint bias matrix, and the cross-modal attention fusion feature matrix is ​​calculated by combining the geometric constraint bias matrix.

[0010] The cross-modal attention fusion feature matrix is ​​input into a pre-trained differentiable neural agent model and a joint verification classification network to perform differentiable discrimination of hallucination features and output a joint calibration loss based on a dynamic uncertainty boundary mechanism.

[0011] The gradient of the hallucination probability error is extracted and combined with the perceptual similarity loss to perform micro-step iterative feature correction. The compliant visual detection training samples are output through closed-loop adaptive repair and the forced posterior mechanism of true rules.

[0012] As a preferred embodiment of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data described in this invention, the generation of the precise symbolic distance field discrete matrix includes:

[0013] The node coordinates of the integrated circuit layout geometric topology file are corrected by using an affine transformation matrix. The topology nodes of the integrated circuit layout are projected onto the pixel coordinate system of the scanning electron microscope image to establish a physical alignment relationship.

[0014] It receives physical space alignment coordinate parameters, parses polygon geometric coordinates in the integrated circuit layout geometric topology file, performs deterministic pixel-level geometric calculations using a fast traversal algorithm, and generates a pixel-by-pixel precise symbolic distance field discrete matrix.

[0015] As a preferred embodiment of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data described in this invention, the method includes correcting the node coordinates of the integrated circuit layout geometric topology file using an affine transformation matrix, comprising:

[0016] Read the integrated circuit layout geometry topology file, parse and obtain the topology nodes of the polygon boundaries of the layout, and establish a vector coordinate system in a two-dimensional continuous space;

[0017] Acquire scanning electron microscope images containing the target observation area and establish an image pixel coordinate system based on a discrete pixel grid;

[0018] Select common source anchor points to construct a set of matching coordinate points for parameter solving;

[0019] The affine transformation matrix is ​​obtained by solving the affine transformation model using the least squares method combined with the matching coordinate point set;

[0020] The vector coordinates of the topological nodes are transformed into homogeneous coordinate vectors and substituted into the affine transformation model to calculate the output projected pixel coordinates.

[0021] As a preferred embodiment of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data described in this invention, a fast traversal algorithm is used to perform deterministic pixel-level geometric calculations to generate a pixel-by-pixel precise symbolic distance field discrete matrix, including:

[0022] Establish a two-dimensional discrete grid coordinate system, extract the physical boundary of the integrated circuit layout geometric topology file, define the grid pixels that intersect with the physical boundary as boundary nodes, and set the set of boundary nodes as the zero-bias level set.

[0023] The spatial polarity of non-boundary nodes is determined by the ray crossing method, and known nodes, trial nodes, and unknown nodes are identified and stored in a minimum binary heap.

[0024] The fast-moving iterative calculation is performed repeatedly, and the process function equation is discretized using the upwind finite difference scheme to calculate and update the precise geometric distance between adjacent test nodes;

[0025] After the iteration stops, the sign is assigned by spatial polarity to form a two-dimensional symbolic distance field, which is then expanded in row-major order to generate an exact symbolic distance field discrete matrix.

[0026] As a preferred embodiment of the visual detection method for multimodal fusion and adversarial generation of integrated circuit design data according to the present invention, the generation of the cross-modal attention fusion feature matrix includes:

[0027] Visual feature vectors from scanning electron microscope images are extracted using a feature pyramid visual coding network with group normalization operators, and topological feature vectors from integrated circuit layout geometric topology files are extracted using a graph convolutional network with node coordinate normalization.

[0028] The exact symbolic distance field discrete matrix is ​​subjected to two-dimensional minimum pooling and then substituted into a composite activation function of logarithmic Maclaurin series expansion form. This nonlinear activation transforms the matrix into a one-dimensional space mask. Using the matrix Kronecker product broadcasting mechanism, the geometric constraint bias matrix is ​​generated by copying along the feature dimension of the topological nodes to achieve hard constraints of physical compliance rules in the manifold mapping.

[0029] In the fusion layer of the multimodal generative adversarial network, the cross-modal attention fusion feature matrix is ​​calculated and generated by combining the geometric constraint bias matrix.

[0030] As a preferred embodiment of the visual detection method for multimodal fusion and adversarial generation of integrated circuit design data according to the present invention, the differentiability discrimination of the illusion feature includes:

[0031] The cross-modal attention fusion feature matrix is ​​input into a pre-trained differentiable neural agent model with a multi-task residual architecture. Through the implicit manifold mapping of physical rule decoupling, the high-dimensional features are projected onto the orthogonal subspace, and a continuous penalty feature vector representing the severity of violations of each independent design rule is output. The continuous penalty feature vector uses an adaptive scaling constant to eliminate the scale difference when splicing multi-task features.

[0032] The cross-modal attention fusion features and the continuous penalty feature vectors are input in parallel into a joint verification classification network that adopts a step-by-step independent training mode to calculate the confidence of the hybrid verification hallucination judgment and realize the differentiability of hallucination features.

[0033] As a preferred embodiment of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data described in this invention, the step of outputting joint calibration loss based on a dynamic uncertain boundary mechanism includes:

[0034] A dynamic fault-tolerant uncertainty boundary and a branch decision sampling triggering mechanism based on adaptive adjustment of classification verification error are set to obtain the illusion decision confidence of the joint verification classification network output;

[0035] The generated multimodal data samples that satisfy the dynamic fault-tolerant uncertainty boundary are pushed into the sampling buffer queue;

[0036] When the number of backlogged samples in the queue reaches the trigger threshold, a non-blocking asynchronous dual-thread parallel fine-tuning mechanism is activated, and the real commercial design rule checking tool for integrated circuits is called to perform batch verification of the reconstructed layout image samples in the sampling buffer queue and obtain the real judgment results.

[0037] The true judgment result is transformed into a true binary hard label and a true continuous violation degree vector. Combined with the prediction output of the joint verification classification network and the prediction output of the pre-trained differentiable neural agent model, a joint calibration loss function is constructed to calculate the joint calibration loss value.

[0038] Online fine-tuning of network weights is achieved by performing gradient backpropagation using the joint calibration loss value.

[0039] As a preferred embodiment of the visual detection method for multimodal fusion and adversarial generation of integrated circuit design data described in this invention, the step of extracting the illusion probability error gradient and performing micro-step iterative feature correction in conjunction with perceptual similarity loss includes:

[0040] Extract the first-order partial derivative of the joint calibration loss function with respect to the input feature tensor to determine the gradient of the hallucination probability error;

[0041] The generated visual feature matrix is ​​reconstructed in reverse and mapped back to the intermediate pseudo-image pixel space with physical spatial attributes through a partial decoding layer of the generator.

[0042] A pre-trained visual perception evaluation model is used to extract multi-layer feature maps. The channel activation difference of the intermediate pseudo-image tensor between the current iteration feature and the initial feature in the multi-layer feature map is calculated to determine the perceptual similarity loss.

[0043] By combining the gradient of the hallucination probability error with the gradient of the perceptual similarity loss penalty, micro-step iterative feature correction is performed on visual features at the boundary of hallucination judgment.

[0044] As a preferred embodiment of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data described in this invention, the step of outputting compliant visual inspection training samples through closed-loop adaptive repair and a real rule-forced posterior mechanism includes:

[0045] Set a maximum iteration step threshold and convergence accuracy tolerance, and monitor the single-step displacement of the feature manifold and the confidence level of the illusion judgment in real time to determine the convergence status.

[0046] If the convergence condition is met, the repaired compliant visual feature matrix will be mapped back to the reconstructed layout image, and the real commercial design rule checking tool will be called to perform the real rule forced posterior test.

[0047] If it is determined that it fully complies with all integrated circuit design rules, it will be confirmed as a compliant visual inspection training sample.

[0048] If the convergence condition is not met, the updated feature matrix will be used as the input for the next iteration, and the computing power overload protection judgment will be monitored and executed.

[0049] As a preferred embodiment of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data described in this invention, the execution of computing power overload protection judgment and forced post-testing of true rules includes the following mechanisms:

[0050] If the maximum number of iterations is reached but the convergence tolerance is not met, the iteration loop is forcibly truncated and repaired, and the corresponding generated multimodal data samples are removed.

[0051] Set a global retry counter. If the global retry counter exceeds the global retry threshold, generate multimodal data samples that are marked as unrepairable and remove them from the training batch queue.

[0052] If the true rule forces a post-hoc judgment that violates the rules, discard the pseudo-compliant sample and trigger a regeneration operation. At the same time, extract the features that triggered the misjudgment and push them into the sampling buffer queue along with the true judgment result to trigger online fine-tuning.

[0053] The present invention has the following beneficial effects:

[0054] 1. This invention establishes a physical spatial alignment relationship between scanning electron microscope image data sequences and corresponding integrated circuit layout geometric topology files. It uses a fast traversal algorithm to generate an accurate symbolic distance field discrete matrix, then transforms the accurate symbolic distance field discrete matrix into a geometric constraint bias matrix, and combines the geometric constraint bias matrix to calculate and generate a cross-modal attention fusion feature matrix. This can solve the problem of feature space misalignment between the image pixel coordinate system and the vector topology coordinate system. The geometric constraint bias matrix is ​​used to suppress feature responses within illegal physical distances, block the combination of illegal pixel regions and topological features, and suppress feature illusions that violate integrated circuit design rules during multimodal fusion.

[0055] 2. This invention inputs the cross-modal attention fusion feature matrix into a pre-trained differentiable neural agent model and a joint verification classification network to perform differentiable discrimination of hallucination features. Based on the dynamic uncertain boundary mechanism, it outputs a joint calibration loss and extracts the corresponding hallucination probability error gradient. This can solve the problem that the post-discrete rule verification tool in the prior art cannot be integrated into the neural network due to the non-differentiability of Boolean logic. It transforms discrete physical geometric rule constraints into continuous and differentiable error penalty parameters, realizing end-to-end joint backpropagation training of physical compliance constraints in multimodal adversarial generative networks.

[0056] 3. This invention combines the hallucination probability error gradient with perceptual similarity loss to perform micro-step iterative feature space backtracking operations on visual features at the boundary of hallucination judgment, and supplements them with a real rule-forced posterior mechanism to output compliant visual detection training samples. This can solve the problems of wasted computing power and loss of model generalization sample diversity caused by simply discarding high-value edge data that failed discrete verification. It uses multi-objective joint gradient guidance to generate features, and backtracks to the physical compliant manifold while preserving the original visual defect structure semantics. This makes the generated samples after closed-loop adaptive repair have both the topological compliance of the integrated circuit and the visual authenticity of physical defects.

[0057] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0059] Figure 1 The flowchart of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data provided by the present invention is shown.

[0060] Figure 2 This is a schematic diagram of the S3 process provided by the present invention.

[0061] Figure 3 This is a schematic diagram of the S4 process provided by the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Example 1

[0064] Existing adversarial generative networks (GANs) for integrated circuit visual inspection are prone to generating illusions during multimodal fusion, outputting seemingly reasonable but actually violating integrated circuit design rules as false defect samples. Including such samples in the visual inspection training set reduces the effectiveness of the model's training benchmark. Current technologies often employ discrete rule-based verification methods, which suffer from issues such as misaligned feature spaces, non-differentiable Boolean logic preventing end-to-end training, and wasted computational resources due to discarding high-value edge data.

[0065] To solve the above technical problems, such as Figure 1As shown, Embodiment 1 of the present invention provides a visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data. Specifically, Embodiment 1 takes the multimodal fusion scenario of scanning electron microscope images and integrated circuit layout geometric topology files in integrated circuit visual inspection as an example. In the specific implementation process, a physical spatial alignment relationship is established between the collected scanning electron microscope image data sequence containing real defects and the corresponding integrated circuit layout geometric topology file. A fast traversal algorithm is used to generate an accurate symbolic distance field discrete matrix. Visual feature vectors and topological feature vectors are extracted, and the accurate symbolic distance field discrete matrix is ​​transformed into a geometric constraint bias matrix. A cross-modal attention fusion feature matrix is ​​calculated by combining the geometric constraint bias matrix. The cross-modal attention fusion feature matrix is ​​input into a pre-trained differentiable neural proxy model and a joint verification classification network to perform differentiable discrimination of hallucination features and output a joint calibration loss based on a dynamic uncertain boundary mechanism. The hallucination probability error gradient is extracted and combined with the perceptual similarity loss to perform micro-step iterative feature space backtracking operations. Through closed-loop adaptive repair and a forced posterior mechanism based on real rules, compliant visual inspection training samples are output.

[0066] Furthermore, to better illustrate the technical solution of Embodiment 1 of the present invention, a detailed description is provided of the visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data, specifically including the following:

[0067] S1. Acquire scanning electron microscope image data sequences containing real defects and establish physical spatial alignment with the corresponding integrated circuit layout geometric topology file. Use a fast traversal algorithm to generate an accurate symbolic distance field discrete matrix. This includes the following sub-steps:

[0068] S11. Acquire scanning electron microscope (SEM) image data sequences containing real defects and the corresponding integrated circuit (IC) layout geometric topology file. Use an affine transformation matrix to correct the node coordinates of the layout geometric topology file, projecting the IC layout topology nodes onto the pixel coordinate system of the SEM image to establish a physical alignment relationship. Specifically, this includes the following steps:

[0069] S111. Read the integrated circuit layout geometric topology file (such as GDSII format), parse and obtain the topology nodes of the polygon boundary of the layout, and establish a vector coordinate system in a two-dimensional continuous space; simultaneously acquire scanning electron microscope images containing the target observation area, and establish an image pixel coordinate system based on discrete pixel grid.

[0070] S112. In the scanning electron microscope image and integrated circuit layout geometric topology file, select... A number of homologous anchor points with significant geometric structural features, such as characteristic corners or alignment markers, wherein Furthermore, the anchor points originating from the same source are not collinear in space. Record the [number]th [anchor point]. Vector coordinates of three homogeneous anchor points in the topological space of the layout and the pixel coordinates in the corresponding scanning electron microscope image space. Construct a set of matching coordinate points for parameter solving.

[0071] S113. Establish a two-dimensional homogeneous affine transformation model that includes scaling, spatial rotation, and translation parameters. Solve for the affine transformation matrix using a matched coordinate point set and the least squares method. The formula for calculating the affine transformation matrix is:

[0072] ;

[0073] In the formula, and The pixel coordinates of the anchor point; and These are the vector coordinates of the corresponding anchor points; , , , The matrix parameters characterize scaling and spatial rotation, and are dimensionless quantities. , Let be the matrix parameters characterizing translation in two-dimensional space. When At that time, an overdetermined system of equations is constructed and solved using a generalized inverse matrix to minimize the projection mean square error.

[0074] S114. Extract all topological node vector coordinates from the integrated circuit layout geometric topology file, transform them into homogeneous coordinate vectors, and substitute them as input variables into the solved affine transformation model. Through matrix multiplication, calculate and output the projected pixel coordinates corresponding to each topological node, complete the global mapping of integrated circuit layout features to the scanning electron microscope image coordinate system, and establish the precise alignment relationship between the two modal data in physical space.

[0075] S12. Receive the physical space alignment coordinate parameters output in step S11, directly parse the polygonal geometric coordinates in the integrated circuit layout file, and perform deterministic pixel-level geometric calculations using a fast traversal algorithm to generate a pixel-by-pixel precise symbolic distance field discrete matrix. This includes the following sub-steps:

[0076] S121. Based on the pixel resolution of the scanning electron microscope image, establish a two-dimensional discrete grid coordinate system of equal size. Map the aligned polygon topology nodes of the integrated circuit layout to the two-dimensional discrete grid coordinate system. Extract the physical boundaries of the layout polygons, define the grid pixels intersecting the physical boundaries as boundary nodes, and set the set of boundary nodes as the zero-bias level set. Assign an initial distance value of 0 to all boundary nodes within the zero-bias level set, and mark their state as known nodes.

[0077] S122. Traverse the non-boundary nodes within the 2D discrete grid coordinate system. Use the ray-crossing method to determine the spatial polarity of the non-boundary nodes, marking the polarity of nodes located inside the polygon as negative and those located outside as positive. Mark the non-boundary nodes directly adjacent to known nodes as trial nodes, calculate their Euclidean distances to adjacent known nodes as initial distance values, and store all trial nodes in a minimum binary heap according to their distance values. Mark the remaining unprocessed non-boundary nodes as unknown nodes and assign them infinite initial distance values.

[0078] S123. Perform a rapid forward iteration calculation repeatedly. Extract the trial node with the smallest current distance value from the minimum binary heap and update its state to that of a known node. Extract all neighboring trial nodes and unknown nodes around this node, update the state of the unknown nodes to that of trial nodes, and push them onto the minimum binary heap. Discretize the equations using an upwind finite difference scheme to calculate and update the precise geometric distance between neighboring trial nodes. The numerical discretization formula for updating the distance between neighboring trial nodes is:

[0079] ;

[0080] in, For the current probe node that needs to update its geometric distance, its grid coordinates are... Scalar distance at point, with the dimension of length ; This is the minimum distance between this node and its adjacent known nodes along the horizontal X-axis, i.e. The dimension is length. ; This is the minimum distance between this node and its adjacent known nodes in the direction perpendicular to the Y-axis, i.e. The dimension is length. ; This refers to the actual physical size calibration value of a single pixel in a discrete raster coordinate system, with the dimension of length. .

[0081] This is due to the fact that the boundaries of the grid coordinate system or adjacent nodes are still in an unknown state. or For the degradation case approaching infinity, establish the windward determination condition and perform a piecewise analytical solution. Sort the known distance parameters and let... and Calculate a one-dimensional trial solution along a single gradient direction: ; Evaluate the distance between the one-dimensional trial solution and the larger known node. The numerical relationship. When satisfying When the condition is met, it indicates that the propagation of the distance gradient occurs only along a single coordinate axis. With the constant term set to 0, the discretized equation degenerates into a one-dimensional form, and the calculation output is as follows: When adjacent nodes are still in an unknown state, causing When, satisfy The windward determination condition is used to automatically calculate the path in a one-dimensional form and output a finite real distance, avoiding abnormal overflow in numerical calculations. When the above condition is not met... When this occurs, it indicates that distance propagation happens simultaneously along both the horizontal and vertical directions. In the discretized equation, both maximum terms are greater than 0, and the original equation is transformed into a standard two-dimensional quadratic equation. Substituting the values ​​into the quadratic formula and extracting the largest real root that satisfies physical causality, the output result is: The analytical solution logic accurately reflects the physical distance attenuation law during the extrapolation of the vertical boundary of the map and the divergence propagation at the corner points, ensuring the numerical stability of multimodal fusion spatial scale calculation under various extreme conditions.

[0082] S124. Iteration stops when the minimum binary heap is empty and all grid nodes have been transformed into known nodes. Combining the spatial polarity symbols extracted in step S122, each distance scalar in the discrete grid coordinate system is assigned a positive or negative sign, forming a two-dimensional symbolic distance field containing relative positional information within and outside the topology. This two-dimensional symbolic distance field is expanded in row-major order, generating a dimension of... The exact symbolic distance field discrete matrix ,in This represents the total number of pixels in a scanning electron microscope image.

[0083] In step S1 of this embodiment, an affine transformation relationship is established between the integrated circuit layout geometric topology file and the scanning electron microscope image to achieve physical coordinate mapping. Combining the spatial polarity determination of the discrete grid and the windward finite difference solution of the equation, a precise symbolic distance field discrete matrix containing local geometric constraint information is constructed. Specifically, for example, the actual physical size calibration value of a single pixel in the discrete grid coordinate system is set. Targeting the probing nodes If it has only two known adjacent nodes in the horizontal direction and the distance between them is minimal. The vertically adjacent nodes are in an unknown state, which makes Defined at this time and .satisfy The windward determination condition is used to solve the path problem in a one-dimensional form, and the finite real distance is calculated and output. Regarding another probing node If there are adjacent known nodes in both its horizontal and vertical directions and the minimum values ​​are respectively and ,definition and Not satisfied The windward condition for determining distance propagation occurs simultaneously in both the horizontal and vertical directions. Substituting these conditions into the two-dimensional analytical equation yields the scalar distance. Based on the spatial polarity determination, if the aforementioned nodes are located outside and inside the polygon of the layout, respectively, the corresponding precise symbolic distance field scalar values ​​are respectively... and The symbolic distance values ​​of all nodes are expanded in row-major order, resulting in a final output dimension of... The precise symbolic distance field discrete matrix ensures that distance calculations are continuous and dimensionally consistent under both limit and normal geometric propagation conditions.

[0084] S2. Extract visual feature vectors and topological feature vectors, and then discrete the precise symbolic distance field matrix output in step S1. The process involves converting the matrix into a geometric constraint bias matrix, and then using this matrix to calculate the cross-modal attention fusion feature matrix. This process includes the following sub-steps:

[0085] S21. Visual feature vectors from scanning electron microscope images are extracted using a feature pyramid visual coding network with group normalization operators. Topological feature vectors from integrated circuit layouts are extracted using a graph convolutional network that includes node coordinate normalization. The precise symbolic distance field is then discretized. The matrix is ​​transformed into a one-dimensional spatial mask through two-dimensional minimum pooling and nonlinear activation. Then, using a matrix Kronecker product broadcasting mechanism, it is copied along the feature dimension of the topological nodes to generate a geometric constraint bias matrix with the same dimension as the cross-modal attention scoring matrix. Specifically, the steps include:

[0086] S211. To address the uneven distribution of localized charge effect noise in scanning electron microscope images caused by electron beam scanning, a residual network that replaces batch normalization with group normalization is adopted as the backbone architecture of the visual coding network. The spatial resolution is... A scanning electron microscope image with 3 channels is input into this backbone architecture and sequentially passed through multiple downsampling convolutional layers. During the calculation of extracting local texture features using a two-dimensional convolution operator, the following method is employed. The convolutional kernels are used for sliding window feature extraction. The downsampling factor is then obtained. The top-level feature map has a spatial dimension of ,in , Multi-scale features are fused using a top-down path through a Feature Pyramid Network (FPN), resulting in an output channel dimension of [missing information]. Fusion feature map tensor The fused feature map tensor is flattened and serialized along the spatial height and width dimensions, and then spliced ​​to form a sequence containing... Visual feature matrix of spatial blocks ,in , The final dimension is .

[0087] S212. Read the integrated circuit layout geometric topology file and extract the geometric center coordinates of the polygonal elements in the layout. , bounding box width Boundary box height And polygon level type identifiers. The center coordinates, width, and height are divided by the global field of view of the layout for dimensionless normalization, and the level type identifier is converted into a one-hot encoded vector. The normalized geometric parameters and the one-hot encoded vector are then concatenated to construct a vector of length [length missing]. The initial node feature vector is derived from The nodes constitute the initial node feature matrix. Undirected edges are established based on the spatial physical connectivity between polygonal elements and the design rule tolerance distance, generating a symmetric graph adjacency matrix. The initial node feature matrix and the graph adjacency matrix are input into the graph convolutional network layer for high-dimensional node feature aggregation. The mathematical expression for graph convolutional node feature aggregation is:

[0088] ;

[0089] in, , is the dimensionless adjacency matrix with self-loops added; for The degree matrix is ​​a dimensionless diagonal matrix; is the dimensionless initial node characteristic matrix; The weight matrix of the learnable graph convolutional layer has dimensions of . During the calculation, the matrix dimensions are passed sequentially as follows: The final output topological feature matrix Dimensions It satisfies the matrix multiplication law and has a dimensionless feature expression. When there is an isolated test pole polygon in the layout, causing the corresponding node to have no connected edges, the node is in In this matrix, only the diagonal elements are 1, and the corresponding elements in the degree matrix are 1. Calculate... When the node's behavior is a one-hot vector, feature aggregation degenerates into a linear mapping of itself. This prevents isolated nodes from having their features collapse to zero during graph convolution propagation.

[0090] S213, To solve the visual pixel feature space With graph topological feature space The semantic dimension mismatch prevents direct computation of the attention inner product, so a cross-modal feature alignment mapping layer is constructed. A multilayer perceptron (MLP) and layer normalization operator are used to map the visual feature matrix... With topological feature matrix Unified projection to the feature dimension is and The shared implicit space. The formula for mapping and calculating the visual query feature matrix and the topological key-value feature matrix is:

[0091] ;

[0092] ;

[0093] ;

[0094] in, The visual query projection weight matrix has dimensions of . ; and These are the projected weight matrices for the topological keys and topological values, respectively, with dimensions of... and ; , , This represents the bias vector for the corresponding dimension. Visual feature matrix. (dimension) )and The product output dimension is Visual query feature matrix Topological feature matrix (dimension) The output dimensions of the product of the product of the corresponding weights are respectively Topological bond feature matrix and dimension are Topological characteristic matrix The entire computation process is performed in a dimensionless, normalized latent space, with dimensionality transformation and matching. When the input visual image is a completely textureless, flat background... The row vectors tend to converge. After linear projection and layer normalization, The internal characteristic variances are forcibly stretched to a standard normal distribution, and a very small positive real number (such as 10) is appended to the variance term during normalization. -8 This prevents division by zero. This operation avoids outputting zero vector queries in flat background regions, ensuring that the subsequent cross-modal attention dot product scoring matrix can produce a uniform attention distribution, rather than computational singularities.

[0095] S214. Obtain the spatial resolution output in step S12. The exact symbolic distance field discrete matrix Set the size of the 2D pooling sliding window to [value]. (This size is consistent with the spatial downsampling rate of the visual encoding network), the step size is... Perform a two-dimensional minimum pooling downsampling operation, calculated using the following formula:

[0096] ;

[0097] in, The grid coordinates after pooling The geometric distance value at; , These are the local offset indices within the sliding window. The min-pooling operation only filters for the minimum values ​​within the local neighborhood, without changing the original physical distance dimensions. This operation ensures that during downsampling, the extreme values ​​closest to the physical violation boundary within each visual block region are preserved, preventing subtle violation risk features from being masked by average downsampling. The downsampled range field matrix... Flattened to the dimension one-dimensional column vector ( ).Will Substitute the geometric mask nonlinear activation function The output dimension is also... A one-dimensional spatial geometric mask matrix; where, Dimensions The dimension is length unit ; This represents the physical threshold of the safety design rules specified in the integrated circuit manufacturing process, with the dimension being length units. ; This represents the scaling factor, with dimensions of . ; This represents the Sigmoid activation function. The nonlinear activation function is chosen as the Sigmoid composite operator in the form of a logarithmic Maclaurin series expansion. It possesses clear mathematical asymptotic characteristics and physical constraint mechanisms. When the feature region fully satisfies the design rules... hour, Activate output Within this asymptotic range, the geometric constraint bias matrix does not attenuate the cross-modal attention score, ensuring the free fusion of visual and topological features within the physically safe region. However, when the feature region severely violates the design rules... At this point, by utilizing the exponential decay property of the Sigmoid function, its mathematical expression asymptotically degenerates into a linear penalty form.

[0098] The activation output tends towards negative infinity. This negative infinity bias term, in the subsequent Softmax operation of the cross-modal attention normalization layer, can forcibly truncate the attention weights of the corresponding violation pixel grid. This involves implementing hard constraints on physical compliance rules in soft manifold mappings of neural networks. From the perspectives of numerical computation stability and gradient flow propagation, [the following is discussed]. Transformation into The explicit analytical form of operator-level fusion solution can avoid the problem caused by the Sigmoid function being close to the threshold value in the early stages of multimodal training. This leads to a logarithmic zero overflow anomaly, while ensuring that in the boundary uncertain region... It can provide continuous, monotonic and unsaturated first-order error partial derivatives to guide multimodal generative networks to perform deterministic manifold boundary corrections.

[0099] Define dimension as A dimensionless matrix of all elements equal to 1. The Kronecker product broadcast expansion is achieved using matrix multiplication, and the calculation formula is as follows: One-dimensional matrix (dimension) ) and row vector Multiplication, according to the rules of linear algebra, generates a dimension of... Two-dimensional geometric constraint bias matrix .

[0100] S22. In the fusion layer of the multimodal generative adversarial network, the cross-modal attention fusion features are calculated by combining the geometric constraint bias matrix. The calculation formula is as follows:

[0101] ;

[0102] In the formula, denoted as the modulated cross-modal attention fusion feature matrix, which is a dimensionless quantity; This represents the query matrix generated by linear projection of visual feature vectors, with dimension 1. , is a dimensionless quantity; , Let these represent the key matrix and value matrix generated by linear projection of topological eigenvectors, respectively, with dimensions of... and , is a dimensionless quantity; This represents the feature channel scaling factor to prevent gradient vanishing in the dot product operation; it is a dimensionless constant. This is used when a visual query location severely violates the design rules. hour, The function output approaches 0, the natural logarithm Approaching negative infinity, the attention bias term corresponding to this row becomes negative infinity after broadcasting. After Softmax operation, the attention weight of this violation visual region is forcibly truncated to 0.

[0103] In step S2 of this embodiment, the physical geometric boundary distance is converted into a bias penalty term for cross-modal attention through a composite calculation of feature projection and spatial masking, thereby achieving forced truncation of illegal feature fusion. Specifically, for example, setting a physical threshold for the security design rules specified by the integrated circuit manufacturing process. Scaling factor Feature channel scaling factor Extract the first visual block within the safe physical region and its corresponding precise symbolic distance. And the precise symbolic distance between the second visual block in the severely violated area and its corresponding symbol. Substituting the geometric mask into the nonlinear activation function for computation, the distance difference for the first visual block is... Calculate and obtain the dimensionless activation input value. Nonlinear activation output spatial geometric mask This zero-bias term does not attenuate the original inner product score in the matrix addition operation of cross-modal attention fusion, ensuring the free fusion of visual and topological features within the physically safe region. For the second visual block, its distance difference... Calculate and obtain the dimensionless activation input value. Utilizing the exponential decay property to output a spatial geometric mask Assuming the original cross-modal feature dot product score is set to 2.0, the fused score drops to -14.0 after adding this bias term. After Softmax exponentiation, the attention weight corresponding to this local grid decreases to... The weight values ​​are close to 0. This numerical calculation process verifies that the geometric constraint bias matrix can accurately identify and suppress feature responses within illegal physical distances, block the combination of illegal pixel regions and topological features, and realize hard constraints on physical compliance rules in manifold mapping.

[0104] S3. Utilize the cross-modal attention fusion feature matrix output from step S2. The input is fed into a pre-trained differentiable neural agent model and a joint validation classification network to perform differentiable discrimination of hallucination features and output a joint calibration loss based on a dynamic uncertainty boundary mechanism. For example... Figure 2 As shown, the specific steps include the following:

[0105] S31. Receive the cross-modal attention fusion feature matrix output from step S2. This is then fed into a pre-trained differentiable neural agent model employing a multi-task residual architecture. Through a latent manifold mapping decoupled from physical rules, high-dimensional features are projected onto an orthogonal subspace to independently represent each design rule, outputting a continuous penalty feature vector representing the severity of violations for each independent design rule. Specifically, this includes the following sub-steps:

[0106] S311. Construct a differentiable neural agent model architecture including an input normalization layer, a multi-layer residual perceptron module, and a multi-task linear output head. In offline mode, extract a large amount of historical generated features for reconstructing the map image. Use a commercial design rule checking tool to perform Boolean geometric verification, calculate violation severity hard labels, and construct a pre-training dataset. Supervised learning pre-training is performed using the mean squared error loss function and an adaptive optimization algorithm. After freezing the weights, it is used for online inference. Specific implementation details are as follows:

[0107] S311-1. Construct a differentiable neural agent model, whose forward propagation path sequentially connects to an input normalization layer, a multi-layer residual perceptron module, and a multi-task linear output head. The input normalization layer uses a layer normalization operator to standardize the high-dimensional visual fusion features of the input with zero mean and unit variance. The multi-layer residual perceptron module includes... A series of cascaded residual blocks, set Each residual block contains two fully connected layers stacked sequentially, and the number of hidden units in each fully connected layer is set to [value missing]. A Gaussian error linear unit activation function is inserted between two fully connected layers, introducing a retention probability of... The random deactivation operator suppresses overfitting in dense connections, and the residual blocks are element-wise summed matrix operations via skip connections at the beginning and end. The multi-task linear output header contains... A parallel fully connected branch, setting the total number of independent design rules. The four branches correspond to four independent design rules: minimum line width, minimum line spacing, minimum end-to-end spacing, and via enclosure, and each has its own output. A continuous rule violation prediction vector is generated. By initializing with orthogonal weights and backpropagating with independent losses for each branch, the cross covariance between output branches is limited, thus achieving an implicit manifold mapping that decouples the physical rules.

[0108] S311-2. Construct a supervised pre-training dataset offline. Randomly extract feature data from the historical iteration cache of the generative adversarial network as the input sample set, with the total number of samples set to the order of magnitude of [missing information]. The sample extraction employs a stratified sampling strategy to ensure a uniform distribution of fully compliant samples and samples with varying degrees of violation within the dataset. The visual feature matrix is ​​input into the pre-trained generator decoding layer to reconstruct a reconstructed layout image with geometric and physical property features. A commercial design rule checking tool is used to perform Boolean geometric verification on the reconstructed layout image to extract actual physical measurements. A continuous mapping function is designed to transform the measurements into hard labels representing the degree of violation of design rules. The continuous mapping calculation formula is as follows:

[0109] ;

[0110] In the formula, Indicates the first The sample is for the first The degree of violation of this design rule is hard-labeled and is a dimensionless quantity; Indicates the first The safety physical limit threshold specified by the design rules is measured in units of length. ; The first measurement of the commercial design rule checking tool The actual minimum geometric dimension of the corresponding rule in each sample, in units of length. The input sample set is combined with continuous violation severity hard labels to form a dataset of size [number missing]. The paired offline pre-trained dataset was divided into a training set and a validation set in an 8:2 ratio.

[0111] S311-3. Supervised learning pre-training of the differentiable neural agent model is performed using mini-batch gradient descent and the mean squared error loss function. The formula for calculating the mean squared error loss function is:

[0112] ;

[0113] In the formula, This represents the mean squared error loss value, which is a dimensionless quantity. The parameter representing the number of samples in a single training batch is a dimensionless constant. This represents the total number of independent integrated circuit design rules; The vector component representing the degree of violation predicted by the multi-task linear output head of the surrogate model is a dimensionless quantity.

[0114] An adaptive moment estimation optimization algorithm is used to minimize the mean squared error loss function. The initial learning rate is set to... The weight decay coefficient is The maximum number of training rounds is At the end of each training epoch, the mean squared error loss on the validation set is calculated. Set convergence criteria: If the validation set loss is continuous... Relative rate of change within each round or the number of iterations reaches This completes the pre-training phase, enabling the model's hidden layers to map high-dimensional visual features to a discrete physical rule space.

[0115] S312. Obtain the cross-modal attention fusion feature matrix output from step S2. To address the issue of inconsistent spatial dimensions of feature matrices caused by scanning electron microscope images with different field-of-view sizes, a global adaptive average pooling operation is employed to compress the spatial dimensions and extract feature vectors that contain a combination of global structure and local texture attributes. , For feature channel dimensions. Input to a multilayer residual perceptron for nonlinear manifold mapping, the first... The formula for calculating the hidden feature transformation of a layer is:

[0116] ;

[0117] in, For the first The input feature vector of the layer has a dimension of Dimensionless quantity; Initialize to ; For the first The weight matrix of the layer freeze has a dimension of ; For the corresponding bias vector. Linear projection operation. Satisfying the matrix multiplication dimension matching rule, the output result, after Gaussian error linear unit activation and layer normalization, retains its dimension and scale unchanged, satisfying the residual connectivity ( The algebraic addition requirement ensures the stability of gradient propagation in deep networks.

[0118] S313. The final latent feature vector output after deep manifold mapping. Input a multi-task linear output header. Use linear mapping to convert the channel dimensions... Dimensional reduction , This represents the total number of predefined integrated circuit design rules. To ensure that the output penalty feature is monotonically nonnegative and globally smooth and differentiable, the Softplus activation function is used instead of the traditional step function or ReLU function. The formula for calculating the continuous penalty feature vector is:

[0119] ;

[0120] in, The output layer projection weight matrix has dimensions of . ; For dimension The output bias vector; Output dimension is The continuous eigenvectors, each component of which corresponds to the implicit violation response value of a specific design rule; The function performs element-wise operations on the input vector without changing its dimensions. To match the sign of the dimensionless scaling constant of the joint validation classifier magnitude, The value is adaptively selected based on the ratio between the maximum amplitude of the penalty feature output by the pre-trained differentiable neural agent model and the variance of the elements of the multimodal fusion visual feature matrix to be validated. This eliminates scale differences during multi-task feature stitching and ensures smooth propagation of the discrimination gradient. Overall formula output. For dimension A dimensionless continuous eigenvector, with the range of each component as follows: Satisfying the visual feature matrix Dimensional requirements for tensor splicing.

[0121] When the input visual features When the corresponding circuit topology is extremely standardized, far removed from any physical threshold boundary of manufacturing processes, the internal mapping value of the output layer is obtained after forward propagation through the network. After substituting into the Softplus activation function, the exponential term... Approaching natural logarithm Approaching .at this time A vector approaching zero indicates that the feature sample has not been penalized by any physical rules, conforming to the physical logic that no violation means no penalty. When the input visual feature contains severe defects such as serious topological short circuits or line spacing approaching zero, the pre-trained surrogate model will produce a strong hidden layer response to the structural feature, making... At this point, the Softplus function enters the linear growth region and outputs a large non-negative penalty feature value, providing a well-directed error backpropagation gradient for the closed-loop adaptive repair in the subsequent step S4, forcing the violation feature to quickly escape the high-penalty dangerous manifold region.

[0122] S32. Parallel input of cross-modal attention fusion features and continuous penalty feature vectors to the joint validation classification network, performing differentiable discriminant calculation of hallucination features, the calculation formula is as follows:

[0123] ;

[0124] In the formula, This represents the confidence level of the mixed-verification hallucination determination, with a value ranging from 0 to 1, and is a dimensionless quantity; The input multimodal fusion visual feature matrix to be verified is a normalized dimensionless quantity with dimension 1. ; This represents the continuous penalty feature vector for rule violations output by the differentiable neural agent model. It is a dimensionless quantity with dimension 1. ; This represents a tensor concatenation operation, where the concatenated combined feature vector has a dimension of . ; The weight matrix of the joint validation classifier is a dimensionless matrix with dimension 1. ; This indicates a scalar bias term, which is a dimensionless quantity. This represents the Sigmoid activation function. When a generative adversarial network generates novel out-of-distribution violation patterns, it leads to... When prediction fails and outputs a low penalty value, if visual features The structural anomaly itself triggered a high-frequency response in the hidden layer of the classifier, which was sampled and calibrated using a real tool. After mapping, the overall bias term is greater than 0, and the activation function still outputs a high confidence level, thus achieving effective fault tolerance and interception of misjudgments by the proxy model.

[0125] In this embodiment, the joint verification classification network adopts a step-by-step independent training mode. After the supervised pre-training phase of the differentiable neural surrogate model is completed, the surrogate model weights are fixed, and then the classification network's parameter updates are activated separately. The classification network establishes a two-layer fully connected mapping architecture in the forward propagation, including a layer with dimension of The combined feature vectors are projected onto a nonlinear transformation layer in the hidden layer space, and the number of neurons in the hidden layer is set to... Furthermore, a batch normalization layer and a LeakyReLU activation function are configured after the hidden layer, and finally, the result is obtained by weight matrix... and bias The output layer is mapped to a one-dimensional probability space. The true labels of the training samples are obtained using a Boolean logic mapping mechanism based on the verification results of a real-world commercial design rule checking tool. The classification network shares a scale of [missing information]. The offline pre-trained dataset. For the first... From a sample, hard labels for the degree of violation of each independent design rule were extracted from the output of the commercial design rule checking tool. If the sample satisfies If the condition indicates that the reconstructed map image violates at least one manufacturing process rule and exhibits feature illusion, then a true binary hard label will be assigned. If the sample satisfies If the condition indicates that the reconstructed map image fully conforms to all physical design rules and has no feature illusions, then a true binary hard label is assigned. Weight matrix Numerical initialization is performed using the Xavier normal distribution initialization method, with scalar bias. The initial value is set to 0. The loss function used for pre-training the classification network is the binary cross-entropy loss function, whose expression is:

[0126] ;

[0127] In the formula, This represents the cross-entropy loss value in binary classification, and is a dimensionless quantity. This represents the number of samples in a single training batch and is a dimensionless constant. Indicates the first The sample is a dimensionless input feature vector. The pre-training process uses an adaptive moment estimation optimization algorithm to minimize the loss function. Configure the initial learning rate as The weight decay coefficient is At the end of each training epoch, calculate the cross-entropy loss on the validation set. The convergence criterion for the classification network is set as follows: when the validation set loss continuously decreases... The relative rate of change within each round is less than Alternatively, when the number of iterations reaches the set maximum number of pre-training rounds, the pre-training of the classification network is terminated, and the underlying weights of the joint validation classification network are locked.

[0128] S33. To address the issue of declining prediction confidence in differentiable neural surrogate models due to out-of-distribution data generated during iteration in generative adversarial networks (GANs), a dynamic fault-tolerant uncertainty boundary and queue triggering mechanism are established. Generated samples at the discrimination boundary are extracted and input into a real commercial design rule checking tool for mandatory verification. The true judgment results are transformed into hard labels and continuous penalty target values. A joint calibration loss function is constructed, and gradient backpropagation is performed to achieve online fine-tuning of the differentiable neural surrogate model and the weights of the joint validation classifier. Specifically, this includes the following sub-steps:

[0129] S331. Set dynamic fault-tolerant uncertainty boundary and branch decision sampling trigger mechanism to obtain the illusion decision confidence of the joint validation classification network output. The dynamic tolerance uncertainty boundary interval of the confidence level is defined as follows: To adapt to varying integrated circuit design rule complexities and model convergence states, this boundary interval employs an adaptive dynamic adjustment mechanism based on classification verification error. Compliance judgment upper limit threshold. With the lower limit threshold for determining violations The adaptive calculation formula is:

[0130] , ;

[0131] In the formula, The fundamental discrimination center value is a dimensionless constant, for example... ; and These represent the lower and upper limits of the physical limit of the contraction of the uncertain boundary, respectively, and are dimensionless constants, for example... , ; This represents the boundary adjustment factor, a dimensionless constant, with a value range of [value range missing]. Preferred ; This represents the rolling root mean square error of the joint validation classification network on the validation set samples during the current training epoch; it is a dimensionless quantity. Through this adaptive calculation, the classification network error in the early stages of training... When the value is large, the boundary interval automatically expands, increasing the sampling verification range to capture more misaligned defect distributions; as the network continues to converge and Decrease, the boundary interval gradually moves towards the center value Shrink and reduce the frequency of calls to real-world commercial tools.

[0132] The construction depth is First-in-first-out sampling buffer queue, and the trigger threshold of the sampling buffer queue. The value range is set to Integers, preferably This value matches the number of multi-threaded parallel processing cores in commercial design rule checking tools, maximizing computational efficiency for a single batch. (This is based on the characteristics of the input.) Based on its corresponding confidence level Execute branch operation:

[0133] First branch: When satisfied This indicates that the decision result of the differentiable neural agent model is in an uncertain state, and the system pushes the generated multimodal data sample corresponding to this feature into the sampling buffer queue. When the number of samples backed up in the queue reaches the trigger threshold... To eliminate the significant drop in training efficiency caused by traditional blocking pause mechanisms, a non-blocking asynchronous dual-thread parallel fine-tuning mechanism is adopted. The main forward generation thread locks the deterministic sample branch with saturated confidence and continues the current adversarial generation and batch training process; simultaneously, an independent background asynchronous verification thread is started, calling the integrated circuit real-world commercial design rule checking tool to check the samples in the sampling buffer queue. Each reconstructed map image sample undergoes batch high-concurrency verification. After the background verification thread obtains the actual judgment result and completes the single-step parameter update, it asynchronously synchronizes the fine-tuned weight parameters to the main forward generation thread through an inter-thread communication mechanism, achieving a balance between training throughput and agent network calibration accuracy.

[0134] Second branch: When not satisfied At that time, the system classifies it as a deterministic sample with saturated confidence, skips the sampling verification and online fine-tuning process using real commercial tools in the background, and performs the following independent operations based on polarity. If This indicates that the probability of the generated multimodal data sample violating the design rules is extremely low. The system classifies it as a high-fidelity sample with absolute physical compliance, directly skipping the subsequent closed-loop adaptive repair stage, outputting it, and incorporating it as valid data into the training set of the visual detection model. This indicates that the generated multimodal data sample has a very high probability of violating the design rules. The system judges it as a deterministic hallucination violation sample and directly imports the generated visual feature matrix into the closed-loop adaptive repair module in step S4. The system then directly uses the continuous penalty feature vector currently output by the surrogate model and the corresponding error gradient to perform feature space backoff operation.

[0135] S332. Extract the reconstructed layout image samples corresponding to the generated feature matrices from the sampling buffer queue. Call the integrated circuit real commercial design rule checking tool to perform Boolean geometric-physical verification on this batch of reconstructed layout image samples. Extract the output logs of the commercial design rule checking tool, parse and generate two sets of real supervision signals: the first set is a real binary hard label for whether hallucination violations have occurred in the overall samples. 0 represents full compliance, 1 represents a violation involving hallucinations; the second group is for... True continuous violation degree vector of independent design rules Its extraction and continuous mapping methods are consistent with those in the offline pre-training stage.

[0136] S333, Apply true binary hard labels Vector of true continuous violations As an online supervisory signal, it is combined with the prediction output of the joint validation classifier. and the predicted output of differentiable neural agent models A joint calibration loss function for online fine-tuning is constructed. The formula for calculating the joint calibration loss function is as follows:

[0137] ;

[0138] in, For online fine-tuning of the joint calibration loss value, a dimensionless quantity. These are true binary hard labels, with values ​​of 0 or 1, and are dimensionless. The confidence score for the hallucination determination output by the joint validation classifier is given, with a value ranging from [value range missing]. , dimensionless quantity. The weighting adjustment coefficient, a dimensionless constant, is used to balance the classification loss and regression loss. The total number of independently designed rules is a dimensionless constant. and The surrogate model predicted the first The degree of consecutive violations for each rule and the corresponding degree of consecutive violations output by the real tool are both dimensionless quantities. When the proxy model and classifier exhibit severe out-of-distribution drift, a perfectly compliant sample is judged as having an extremely high probability of being a hallucinatory violation. and Larger, but the actual tool verification output is larger. and After substituting into the formula, the cross-entropy part... Approaching positive infinity, the mean squared error incurs a significant penalty. This large loss value generates a strong backpropagation gradient. When the classifier determines a violation probability to be extremely low... However, the verification of the actual tools was found to be in serious violation. The cross-entropy part of the formula Similarly, it approaches positive infinity, forcing the generation of a penalized gradient. Extract the joint calibration loss function. The partial derivatives of the classifier weight matrix and the weights of the hidden layers within the differentiable neural agent model are calculated using a minimum online learning rate. The adaptive moment estimator optimizer performs single-step parameter updates. The fine-tuning update formula is:

[0139] ;

[0140] In the formula, and These represent the network weight parameter tensors before and after the update, respectively. For online learning rate, This is the gradient tensor of the loss function with respect to the weights. After the update is complete, the sampling buffer queue is cleared, the paused state of the generative adversarial network is lifted, and the forward generation and loop closure repair process of multimodal fusion features is resumed, enabling the network to have adaptive immunity to novel out-of-distribution hallucination patterns.

[0141] In step S3 of this embodiment, a dual-discrimination architecture comprising a differentiable neural agent model and a joint verification classification network is constructed, combined with a dynamic uncertain boundary mechanism, to achieve continuous differentiable discrimination and online automatic calibration of illegal hallucinations in multimodal fusion features. Specifically, for example, a basic discrimination center value is set. Boundary adjustment factor The physical limits of the contraction of the uncertain boundary are respectively: and Assume that the rolling root mean square error of the joint validation classification network on the validation set during the current training epoch is... Substituting into the adaptive calculation formula, the upper limit threshold for compliance judgment is obtained. The lower limit threshold for determining violations .

[0142] After receiving a cross-modal attention fusion feature matrix as input, the joint validation classification network outputs the hybrid validation hallucination determination confidence score. Because this confidence level satisfies Based on the given criteria, the system determines that the sample is in an uncertain state and pushes it into the sampling buffer queue. After triggering verification by the real commercial design rule checking tool, if the actual judgment result is that there is a phantom violation, the real binary hard label is extracted. Furthermore, it extracts the true degree of continuous violations for the first minimum linewidth design rule. At this point, the differentiable neural agent model outputs the predicted violation for this rule dimension. Set weight adjustment coefficient .

[0143] Substituting the values ​​into the joint calibration loss function, the online fine-tuning joint calibration loss value is calculated. .

[0144] The computational process verifies that the system can accurately capture and intercept out-of-distribution abnormal data at the edge of network discrimination, transforming the non-differentiable discrete physical judgment logic into a continuous real-valued spatial loss value with an accurate penalty gradient, and forcibly driving the classification boundary to align with the real geometric and physical rules.

[0145] S4. Extract the hallucination probability error gradient generated in step S3. It combines perceptual similarity loss to perform micro-step iterative feature space backtracking operations, and outputs compliant visual detection training samples through closed-loop adaptive repair and a forced posterior mechanism based on true rules. Figure 3 As shown, the specific steps include the following:

[0146] S41. Extract the illusion probability error gradient output by the joint verification classifier; inversely reconstruct the one-dimensional serialized visual feature matrix and map it back to the intermediate pseudo-image pixel space with physical spatial attributes through a partial decoding layer of the generator; extract multi-layer feature maps using a pre-trained visual perception network and calculate the perceptual similarity loss between the current repaired features and the initial features. Specifically, this includes the following sub-steps:

[0147] S411. Extract the joint calibration loss function defined in the online fine-tuning stage of step S3. Regarding the current situation at the [number]th [period]... The visual feature matrix is ​​generated in the next iteration. By keeping the weights of the joint validation classifier and the differentiable neural agent model frozen, and using an automatic differentiation mechanism to calculate the first-order partial derivative of the loss function with respect to the input feature tensor, the gradient of the hallucination probability error is extracted. This error gradient implies the a priori direction of the physical design rules for integrated circuits, and its opposite direction points to the shortest path that makes the local topology compliant and escapes the high-penalty illusion manifold.

[0148] S412. Obtain the visual feature matrix of the current iteration. and the original visual feature matrix of the initial output of the generative adversarial network. Both dimensions are ,in The number of space blocks, For the channel dimension. Perform a spatial tensor reconstruction operation, folding the two-dimensional feature matrix into a three-dimensional feature tensor, restoring its dimension to the channel dimension. ,satisfy The 3D feature tensor is input into a portion of the decoding layer embedded within the generator network. The middle part of the decoding layer contains multiple deconvolution operators, instance normalization layers, and nonlinear activation layers. Through upsampling and channel dimensionality reduction mapping, the abstract feature tensor is mapped back to the three-channel intermediate pseudo-image pixel space. The mapping calculation expression is:

[0149] , ;

[0150] In the formula, This represents the intermediate pseudo-image tensor corresponding to the current iteration feature. This represents the intermediate pseudo-image tensor corresponding to the initial features; both have a dimension of 1. The pixel values ​​were normalized to The interval is a dimensionless quantity.

[0151] S413. A VGG network pre-trained on an image classification task is used as the visual perception evaluation model, and all its network parameters are frozen. The intermediate pseudo-image tensor... and The input is fed into the pre-trained VGG network, and the first digit is extracted respectively. The activation feature maps of layers, such as ReLU1_2, ReLU2_2, ReLU3_2, ReLU4_2, and ReLU5_2 layers, are used. The extracted feature maps are then normalized along the channel dimension. The perceptual similarity loss between the current pseudo-image and the initial pseudo-image in the multi-layer deep network space is calculated using the following formula:

[0152] ;

[0153] in, The loss value for perceived similarity is a dimensionless quantity. The total number of layers for extracting feature maps is a dimensionless constant. The first The number of channels, height, and width of the layer feature map are dimensionless quantities. This indicates that after channel normalization, at the th Channels of layer feature maps Spatial coordinates The activation feature scalar at the location is a dimensionless quantity. For the first Layer The learnable scaling weights for each channel are used to adjust the relative importance of different deep features in human visual perception; these are dimensionless constants. The formula calculates the weighted sum of squared activation differences for each channel, takes the spatial average, and finally sums them to obtain the scalar loss. When gradient descent is solely used to comply with the compliance criteria of a proxy model, it leads to an imbalance in the visual feature matrix. Loss of original texture details, decoded output When the color becomes an overly smoothed patch, although the mean square error of pixels in the lower layers (such as ReLU1_2) may not change much, the higher layers (such as ReLU4_2, ReLU5_2) are responsible for extracting activation feature maps of complex edges and topological textures. It will approach zero or undergo drastic changes. At this point, compared to the original characteristics... The differences will be amplified layer by layer and summed by squares, resulting in a loss of perceptual similarity. It increases exponentially. This penalty mechanism will generate a strong reverse pull gradient, which, from an architectural logic perspective, forces the feature repair process to retain the semantic information of the original visual defects (such as real band breaks and particles), preventing it from falling into an adversarial forgery manifold.

[0154] S42. Combining the gradient of the hallucination probability error and the gradient of the perceptual similarity loss penalty, perform micro-step iterative feature space backtracking operation on visual features at the hallucination judgment boundary. The specific calculation and parameter scheduling mechanism is as follows:

[0155] Scaling weights involved in perceptual similarity loss calculation This parameter is derived from a converged perceptual image patch similarity metric network pre-trained on a large-scale human visual perception dataset. During the micro-step iteration phase of performing closed-loop feature inpainting, [the parameter is used]. Setting it as a constant and freezing its parameters prevents it from participating in the backpropagation update of the current network, thus providing a constant and objective structural semantic evaluation benchmark. To prevent gradient oscillations and numerical explosion anomalies caused by abrupt changes in the extreme values ​​of the loss function at the illusion decision boundary, gradient merging and feature updates are performed using... The norm threshold truncation algorithm performs numerical stabilization on the illusion probability error gradient output by the joint validation classification network. The formula for calculating the truncated compliance-guided gradient is:

[0156] ;

[0157] In the formula, This represents the compliant guided gradient tensor after norm truncation, which is a dimensionless quantity; This represents the first-order partial derivative of the original extraction; This represents the globally preset maximum gradient norm cutoff threshold scalar, which is a dimensionless constant.

[0158] Extract the compliance guidance gradient and perceptual similarity penalty gradient after pruning, and use an adaptive dynamic parameter scheduling strategy to determine the repair step size coefficient for the current iteration. With semantic fidelity regularization weight coefficient Repair step size coefficient It is generated using a cosine annealing decay function, whose value varies with the number of iterations. The weights decrease non-linearly and monotonically as the number of iterations increases, ensuring rapid escape from the illegal manifold region in the early stages of iteration and smooth convergence without exceeding the bounds in the later stages. Semantic fidelity regularization weight coefficients Dynamic equilibrium calculations are performed based on the relative magnitudes of the two gradients under the current spatial state. The calculation formula is as follows:

[0159] ;

[0160] In the formula, This represents the baseline regularization coefficient, which is a dimensionless constant. To prevent the introduction of extremely small positive real numbers due to division by zero truncation anomalies; This represents the first-order partial derivative of the perceptual similarity loss function with respect to the input feature tensor. This dynamic balancing allocation mechanism ensures that, during the projection of features onto the physical compliance boundary, the visual semantic fidelity constraint components always adaptively match the physical compliance traction, offsetting the unrealistic visual pixel distortion caused by optimization in a single extreme direction. Substituting the dynamically scheduled coefficient quantization values ​​and the corresponding error inverse gradient matrix into the closed-loop feature repair formula, a single microstep feature space backtracking operation is performed:

[0161] ;

[0162] In the formula, Indicates the first The generated visual feature matrix in the next iteration of repair is a dimensionless quantity; Indicates the first The generated visual feature matrix after the next iteration is a dimensionless quantity. This represents the original visual feature matrix of the initial output of the generative adversarial network, which is a dimensionless quantity. This represents a partial decoding mapping function embedded within the generator. This operation, guided by multi-objective orthogonal gradients, forces the job features to perform state transitions along geodesic paths that are optimal in both physical compliance and semantic fidelity within the multidimensional manifold space.

[0163] S43. Set the maximum iteration step threshold and convergence accuracy tolerance. During the micro-step iteration process of feature loop adaptive repair, monitor the displacement of the feature manifold in real time to determine the convergence status; for samples that have not converged or have been repaired, implement computational overload protection and enforce posterior verification based on true rules. If the posterior verification still reports a violation, trigger the circuit breaker mechanism to directly discard the sample and restart generation. Specifically, this includes the following sub-steps:

[0164] S431. Set the minimum positive real number precision tolerance for iterative convergence. (For example In each gradient descent iteration (the... Next to After (the process is repeated) the updated visual feature matrix is ​​calculated. With the preceding visual feature matrix The Euclidean distance between them quantifies the single-step displacement of the feature manifold under multi-objective joint gradient driving:

[0165] ;

[0166] Simultaneously monitor the confidence level of the hallucination determination output by the joint validation classifier. For the convergence state of the current iteration, execute the following branch operation:

[0167] Convergence is satisfied when the single-step displacement condition is met. Furthermore, the confidence level for hallucination determination has fallen below the upper limit threshold for compliance determination (i.e.) When the multi-objective iterative repair of the sample is successfully converged, stop the iteration loop of the current sample, output the repaired compliant visual feature matrix, and execute step S433 to perform the final mandatory sampling inspection.

[0168] Convergence not met: If the above double convergence condition is not met (i.e., or This indicates that the generated visual features have not yet reached a stable state in both physical rules and visual semantics. At this point, the system will update the feature matrix... As the starting point for the next iteration (i.e., assignment update) The process then proceeds to step S432 to determine if the computing power overload protection is triggered. If step S432 determines that the computing power overload truncation is not triggered, the control flow returns to steps S41 and S42, the joint error gradient at that location is re-extracted, and the feature space backoff operation of the next microstep is executed. This iterative process continues until the convergence condition is met or the truncation and circuit breaker mechanism of step S432 is triggered.

[0169] S432. To prevent features from getting trapped in adversarial local optima and causing infinite iteration loops, as well as retry storms caused by multiple consecutive failures, a maximum iteration step threshold is set. With global retry threshold Set a microstep iteration counter for a single feature. With the global retry counter for this sample channel The initial values ​​are all 0. After each gradient descent iteration... Add 1. At the end of each iteration, perform a computing power overload check: if... And characteristic single-step displacement The convergence tolerance has not yet been reached or the confidence level for the hallucination determination is still higher than [previous level]. The system triggered its computing power overload protection mechanism, forcibly truncating the repair iteration loop for this sample. The system determined that the original violation defect of this sample was abnormal and could not be repaired back to a physically compliant manifold under limited semantic fidelity constraints. Therefore, the sample was directly discarded as a scrap piece, and the global retry counter was reset. Add 1. The system then... The following global retry operation is performed on the value:

[0170] when At this time, the system sends a command to the generative adversarial network to initiate a supplementary generation operation for the missing data channel. After obtaining the newly generated visual feature matrix, it will... Reset it to 0 and re-import it into the physical rule prior gradient extraction and perceptual similarity calculation process.

[0171] when When this occurs, the system triggers a global anti-starvation circuit breaker mechanism. The samples in that data channel are marked as irreparable, and no further generation instructions are sent. The invalid samples are directly removed from the current training batch queue. This reduces the actual data volume of the current batch, preventing the risk of training process starvation caused by infinite retries and ensuring the continuous progress of the entire multimodal vision detection model training task.

[0172] S433. For the repaired features output that meet the convergence condition of S431, the system maps them back to the reconstructed layout image through the decoding layer and initiates the Real Commercial Design Rule Check (DRC) tool to perform a final mandatory sampling inspection. If the DRC tool determines that the repaired sample fully complies with all integrated circuit design rules, it is officially confirmed as a compliant visual inspection training sample and included in the final model training set. If the DRC tool determines that the sample still has a geometric structure that violates any design rule, this indicates that during the repair process, the differentiable neural proxy model generated an unknown out-of-distribution misjudgment, leading the features to fall into the false proxy model compliance trap. At this time, the system triggers the final circuit breaker mechanism: not only immediately discarding the pseudo-compliant sample and starting the regeneration operation; but also forcibly extracting the features that triggered the misjudgment and the real DRC hard label of the sample, unconditionally pushing them to the front of the sampling buffer queue in step S33, immediately triggering an online closed-loop fine-tuning of the differentiable neural proxy model, using high-value negative samples to repair the classification boundary loopholes of the proxy network.

[0173] In step S4 of this embodiment, the norm truncation algorithm is used to stabilize the compliance-guided gradient. The fidelity weights are dynamically adjusted based on the relative magnitudes of the compliance gradient and the perceptual loss gradient. Through the alternating driving of multi-objective orthogonal gradients, the features are encouraged to revert to a feature manifold that optimizes both physical compliance and semantic fidelity. Specifically, for example, a globally preset maximum gradient norm truncation threshold scalar is set. The baseline regularization coefficient Minimal positive real number Precision tolerance of iterative convergence In the first In the next microstep iteration, assume the original extracted first-order partial derivative of norm is ,because The norm cutoff factor is calculated as follows: The norm-truncated compliant guided gradient tensor is obtained. Its norm is forcibly limited to . Assume the first-order partial derivative of the perceptual similarity loss function with respect to the input feature tensor in the current state. of norm is Substituting these values ​​into the dynamic balance calculation formula yields the current semantic fidelity regularization weight coefficients. Set the repair step size coefficient generated by the cosine annealing decay function. Substitute the above scheduling values ​​into the closed-loop feature repair formula to update the features, and calculate the updated feature matrix. With the preceding characteristic matrix The Euclidean distance between them is used to derive the single-step displacement of the characteristic manifold under the joint gradient driven by multiple objectives. .because The dual convergence condition was not met, so the system performed an assignment update and proceeded to the next round of microstep feature space backoff calculation. As the iteration continued to advance to the [number missing]th iteration... Step by step, the displacement per step is calculated. At this point, it is less than the accuracy tolerance. Furthermore, the confidence level of the hallucination determination output by the joint verification classifier under real-time monitoring It is less than the upper limit threshold for compliance determination given in the aforementioned steps. corresponding The system determines that the multi-objective iterative repair has successfully converged and terminates the iteration loop. The output compliant visual feature matrix after repair is mapped back to the reconstructed layout image through a partial decoding layer. The real commercial design rule inspection tool is launched to perform the final mandatory sampling inspection. The verification log shows that the number of violations of physical indicators such as minimum line width and minimum line spacing of the circuit topology is 0, which fully complies with all integrated circuit design rules. It is officially confirmed as a compliant visual inspection training sample.

[0174] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data, characterized in that, include: A physical spatial alignment relationship is established between the acquired scanning electron microscope image data sequence containing real defects and the corresponding integrated circuit layout geometric topology file, and a fast traversal algorithm is used to generate an accurate symbol distance field discrete matrix; Visual feature vectors and topological feature vectors are extracted, the precise symbolic distance field discrete matrix is ​​transformed into a geometric constraint bias matrix, and the cross-modal attention fusion feature matrix is ​​calculated by combining the geometric constraint bias matrix. The cross-modal attention fusion feature matrix is ​​input into a pre-trained differentiable neural agent model and a joint verification classification network to perform differentiable discrimination of hallucination features and output a joint calibration loss based on a dynamic uncertainty boundary mechanism. The gradient of the hallucination probability error is extracted and combined with the perceptual similarity loss to perform micro-step iterative feature correction. The compliant visual detection training samples are output through closed-loop adaptive repair and the forced posterior mechanism of true rules.

2. The visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 1, wherein generating the precise symbolic distance field discrete matrix includes: The node coordinates of the integrated circuit layout geometric topology file are corrected by using an affine transformation matrix. The topology nodes of the integrated circuit layout are projected onto the pixel coordinate system of the scanning electron microscope image to establish a physical alignment relationship. It receives physical space alignment coordinate parameters, parses polygon geometric coordinates in the integrated circuit layout geometric topology file, performs deterministic pixel-level geometric calculations using a fast traversal algorithm, and generates a pixel-by-pixel precise symbolic distance field discrete matrix.

3. The visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 2, wherein the node coordinates of the integrated circuit layout geometric topology file are corrected using an affine transformation matrix, including: Read the integrated circuit layout geometry topology file, parse and obtain the topology nodes of the polygon boundaries of the layout, and establish a vector coordinate system in a two-dimensional continuous space; Acquire scanning electron microscope images containing the target observation area and establish an image pixel coordinate system based on a discrete pixel grid; Select common source anchor points to construct a set of matching coordinate points for parameter solving; The affine transformation matrix is ​​obtained by solving the affine transformation model using the least squares method combined with the matching coordinate point set; The vector coordinates of the topological nodes are transformed into homogeneous coordinate vectors and substituted into the affine transformation model to calculate the output projected pixel coordinates.

4. The visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 2, employing a fast traversal algorithm for deterministic pixel-level geometric calculations to generate a pixel-by-pixel precise symbolic distance field discrete matrix, comprising: Establish a two-dimensional discrete grid coordinate system, extract the physical boundary of the integrated circuit layout geometric topology file, define the grid pixels that intersect with the physical boundary as boundary nodes, and set the set of boundary nodes as the zero-bias level set. The spatial polarity of non-boundary nodes is determined by the ray crossing method, and known nodes, trial nodes, and unknown nodes are identified and stored in a minimum binary heap. The fast-moving iterative calculation is performed repeatedly, and the process function equation is discretized using the upwind finite difference scheme to calculate and update the precise geometric distance between adjacent test nodes; After the iteration stops, the sign is assigned by spatial polarity to form a two-dimensional symbolic distance field, which is then expanded in row-major order to generate an exact symbolic distance field discrete matrix.

5. The visual detection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 1, wherein generating a cross-modal attention fusion feature matrix includes: Visual feature vectors from scanning electron microscope images are extracted using a feature pyramid visual coding network with group normalization operators, and topological feature vectors from integrated circuit layout geometric topology files are extracted using a graph convolutional network with node coordinate normalization. The exact symbolic distance field discrete matrix is ​​subjected to two-dimensional minimum pooling and then substituted into a composite activation function of logarithmic Maclaurin series expansion form. This nonlinear activation transforms the matrix into a one-dimensional space mask. Using the matrix Kronecker product broadcasting mechanism, the geometric constraint bias matrix is ​​generated by copying along the feature dimension of the topological nodes to achieve hard constraints of physical compliance rules in the manifold mapping. In the fusion layer of the multimodal generative adversarial network, the cross-modal attention fusion feature matrix is ​​calculated and generated by combining the geometric constraint bias matrix.

6. The visual detection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 1, wherein the differentiability discrimination of the illusion feature includes: The cross-modal attention fusion feature matrix is ​​input into a pre-trained differentiable neural agent model with a multi-task residual architecture. Through the implicit manifold mapping of physical rule decoupling, the high-dimensional features are projected onto the orthogonal subspace, and a continuous penalty feature vector representing the severity of violations of each independent design rule is output. The continuous penalty feature vector uses an adaptive scaling constant to eliminate the scale difference when splicing multi-task features. The cross-modal attention fusion features and the continuous penalty feature vectors are input in parallel into a joint verification classification network that adopts a step-by-step independent training mode to calculate the confidence of the hybrid verification hallucination judgment and realize the differentiability of hallucination features.

7. The visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 1, wherein the output of joint calibration loss based on dynamic uncertain boundary mechanism includes: A dynamic fault-tolerant uncertainty boundary and a branch decision sampling triggering mechanism based on adaptive adjustment of classification verification error are set to obtain the illusion decision confidence of the joint verification classification network output; The generated multimodal data samples that satisfy the dynamic fault-tolerant uncertainty boundary are pushed into the sampling buffer queue; When the number of backlogged samples in the queue reaches the trigger threshold, a non-blocking asynchronous dual-thread parallel fine-tuning mechanism is activated, and the real commercial design rule checking tool for integrated circuits is called to perform batch verification of the reconstructed layout image samples in the sampling buffer queue and obtain the real judgment results. The true judgment result is transformed into a true binary hard label and a true continuous violation degree vector. Combined with the prediction output of the joint verification classification network and the prediction output of the pre-trained differentiable neural agent model, a joint calibration loss function is constructed to calculate the joint calibration loss value. Online fine-tuning of network weights is achieved by performing gradient backpropagation using the joint calibration loss value.

8. The visual detection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 1, wherein the step of extracting the hallucination probability error gradient and performing micro-step iterative feature correction in conjunction with perceptual similarity loss includes: Extract the first-order partial derivative of the joint calibration loss function with respect to the input feature tensor to determine the gradient of the hallucination probability error; The generated visual feature matrix is ​​reconstructed in reverse and mapped back to the intermediate pseudo-image pixel space with physical spatial attributes through a partial decoding layer of the generator. A pre-trained visual perception evaluation model is used to extract multi-layer feature maps. The channel activation difference of the intermediate pseudo-image tensor between the current iteration feature and the initial feature in the multi-layer feature map is calculated to determine the perceptual similarity loss. By combining the gradient of the hallucination probability error with the gradient of the perceptual similarity loss penalty, micro-step iterative feature correction is performed on visual features at the boundary of hallucination judgment.

9. The visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 8, wherein the step of outputting compliant visual inspection training samples through closed-loop adaptive repair and real rule-forced posterior mechanism includes: Set a maximum iteration step threshold and convergence accuracy tolerance, and monitor the single-step displacement of the feature manifold and the confidence level of the illusion judgment in real time to determine the convergence status. If the convergence condition is met, the repaired compliant visual feature matrix will be mapped back to the reconstructed layout image, and the real commercial design rule checking tool will be called to perform the real rule forced posterior test. If it is determined that it fully complies with all integrated circuit design rules, it will be confirmed as a compliant visual inspection training sample. If the convergence condition is not met, the updated feature matrix will be used as the input for the next iteration, and the computing power overload protection judgment will be monitored and executed.

10. The visual inspection method for multimodal fusion and adversarial generation of integrated circuit design data according to claim 9, wherein the execution of computing power overload protection judgment and forced posterior verification of true rules includes the following mechanisms: If the maximum number of iterations is reached but the convergence tolerance is not met, the iteration loop is forcibly truncated and repaired, and the corresponding generated multimodal data samples are removed. Set a global retry counter. If the global retry counter exceeds the global retry threshold, generate multimodal data samples that are marked as unrepairable and remove them from the training batch queue. If the true rule forces a post-hoc judgment that violates the rules, discard the pseudo-compliant sample and trigger a regeneration operation. At the same time, extract the features that triggered the misjudgment and push them into the sampling buffer queue along with the true judgment result to trigger online fine-tuning.