Heterogeneous transformer wafer three-dimensional detection system and method fusing physical prior

Abstract

The application relates to the field of three-dimensional optical detection and intelligent measurement, and specifically discloses a heterogeneous Transform wafer three-dimensional detection system and method fusing physical priori, which comprises a data acquisition module configured to acquire multi-modal data of a wafer surface to be measured, wherein the multi-modal data at least comprises structured light data and photometric stereo data; a data processing module connected to the data acquisition module and configured to: construct a heterogeneous Transform model fusing physical priori; fuse and register the multi-modal data by taking the physical priori as a constraint to generate a high-precision three-dimensional model of the wafer surface. By effectively embedding physical priori knowledge into the Transform model, the fusion precision of the heterogeneous data is significantly improved; compared with traditional methods, the three-dimensional reconstruction precision can still be kept high under complex reflection and shielding conditions.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional optical inspection and intelligent measurement technology, and in particular to a heterogeneous Transformer wafer three-dimensional inspection system and method that integrates physical priors. Background Technology

[0002] With the continuous improvement of semiconductor manufacturing processes, the accuracy requirements for wafer surface inspection are becoming increasingly stringent. Among existing three-dimensional optical inspection technologies, structured light and photometric stereolithography are widely used in wafer surface three-dimensional reconstruction and defect detection. However, these technologies often exhibit decreased accuracy and data loss when faced with conditions such as high reflectivity, complex curvature, and occlusion. This is particularly true for surfaces with strong reflection and minute structures, where traditional techniques struggle to acquire high-quality three-dimensional data. In particular, traditional registration algorithms often rely on simple geometric models, neglecting the physical consistency between multimodal data, leading to inaccurate measurement results and failed defect identification.

[0003] To address these issues, existing research attempts to incorporate machine learning and deep learning methods, particularly the Transformer model, to improve registration accuracy and enhance data fusion capabilities. However, existing Transformer methods still suffer from low processing efficiency and inaccurate data fusion when handling heterogeneous data, such as structured light and photometric data; the lack of full utilization of physical laws results in significant limitations in data matching and model training processes under complex environments. Summary of the Invention

[0004] This invention aims to at least partially address one of the technical problems in related technologies. Therefore, the objective of this invention is to propose a heterogeneous Transformer wafer 3D inspection system and method that integrates physical priors to improve inspection accuracy.

[0005] To achieve the above objectives, a first aspect of the present invention proposes a heterogeneous Transformer wafer 3D inspection system that integrates physical priors, comprising:

[0006] The data acquisition module is configured to acquire multimodal data of the surface of the wafer under test, wherein the multimodal data includes at least structured light data and photometric stereo data;

[0007] The data processing module is connected to the data acquisition module and configured as follows:

[0008] Construct a heterogeneous Transformer model that incorporates physical priors;

[0009] Using the physical priors as constraints, the multimodal data are fused and registered to generate a high-precision three-dimensional model of the wafer surface.

[0010] To achieve the above objectives, a second aspect of the present invention proposes a method for three-dimensional inspection of heterogeneous Transformer wafers that integrates physical priors, comprising the following steps:

[0011] Data acquisition steps: Use a data acquisition module to acquire multimodal data of the surface of the wafer under test, wherein the multimodal data includes at least structured light data and photometric stereo data;

[0012] Data processing steps: Execute using the data processing module:

[0013] (a) Construct a heterogeneous Transformer model that integrates physical priors, wherein the physical priors include at least position compatibility priors, phase-depth residual priors, normal consistency priors, or reflection similarity priors;

[0014] (b) A phase-guided sparse attention mechanism is adopted, and the phase consistency residual calculated using the phase data is used to select Top-k candidate point pairs to sparsify the attention matrix;

[0015] (c) Using the physical prior as a constraint, the multimodal data is fused and registered to generate a high-precision three-dimensional model of the wafer surface.

[0016] To achieve the above objectives, a third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the above-described heterogeneous Transformer wafer 3D inspection method that integrates physical priors.

[0017] The heterogeneous Transformer wafer 3D inspection system and method of the present invention, which integrates physical prior knowledge, significantly improves the fusion accuracy of heterogeneous data by effectively embedding physical prior knowledge into the Transformer model; compared with traditional methods, it can still maintain high 3D reconstruction accuracy under complex reflection and occlusion conditions.

[0018] Through the phase-guided sparse attention mechanism, the system can optimize the data registration process according to physical laws, reduce noise interference and improve the accuracy of defect detection. In addition, the two-stage registration strategy of this invention effectively combines coarse registration and fine registration steps, further improving the robustness and accuracy of the overall system, thus providing an innovative solution for high-precision three-dimensional reconstruction and defect identification of wafer surfaces. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the heterogeneous Transformer wafer 3D inspection system that integrates physical priors provided by the present invention;

[0020] Figure 2 This is a flowchart illustrating the heterogeneous Transformer wafer 3D inspection method that integrates physical priors provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0023] The following description, with reference to the accompanying drawings, describes a heterogeneous Transformer wafer 3D inspection system, method, and electronic device that incorporates physical priors.

[0024] Example 1:

[0025] Figure 1 This is a schematic diagram of the structure of a heterogeneous Transformer wafer 3D inspection system that integrates physical priors according to an embodiment of the present invention. The system is designed to perform high-precision 3D morphology inspection on the surface of semiconductor wafers, especially wafers with complex micro-nano structures, high reflectivity and multilayer transparent media.

[0026] The system in this embodiment is built upon two core modules: a data acquisition module and a data processing module.

[0027] The data acquisition module further includes a structured light projection module, a photometric acquisition module, an imaging module, and a control and calibration module. The structured light projection module projects coded structured light patterns onto the wafer surface to obtain phase and depth information; the photometric acquisition module acquires surface reflection and texture information through multi-angle, multi-wavelength controllable illumination to estimate surface normal and reflection characteristics; the imaging module consists of multiple industrial cameras for synchronously acquiring structured light and photometric data; and the control and calibration module is responsible for the timing synchronization, geometric calibration, and adaptive control between the components to ensure that multi-source data are fused in a unified coordinate system.

[0028] The data processing module comprises a preprocessing unit and a physics-prior-driven fusion module. The preprocessing unit performs geometric correction, brightness equalization, and coordinate unification on the original images, providing a consistent foundation for subsequent fusion. The physics-prior-driven fusion module is the core of the system. Based on a heterogeneous Transformer network structure, it introduces a phase-guided sparse attention mechanism and embeds physical prior constraints such as position compatibility, phase-depth residuals, normal consistency, and reflection similarity to achieve high-precision fusion and 3D topography reconstruction of multi-view heterogeneous data. Furthermore, the data processing module employs a two-stage registration strategy, combining ICP coarse registration and Transformer residual correction to progressively optimize the reconstruction results, ensuring micron-level precision in topography restoration and defect identification even under complex reflection and occlusion conditions.

[0029] Optionally, the data acquisition module includes the following components:

[0030] 1.1 Structured Light Projection Module

[0031] The structured light projection module is used to project coded structured light patterns onto the wafer surface to obtain raw image data containing phase and depth information. The module includes one or more projection mechanisms, each consisting of a projection device and an industrial camera.

[0032] The projection device can employ piezoelectric ceramic-driven analog phase-shift projection technology, or digital optical projection technologies such as digital micromirrors, liquid crystal light valves, and silicon-based liquid crystals, to achieve highly stable projection of periodic stripes or phase-shift patterns. Analog phase-shift projection generates multi-frame phase-shift images by precisely driving the grating displacement with piezoelectric ceramics, making it suitable for high-precision phase extraction. Digital projection technology, on the other hand, achieves flexible pattern projection through programmable control of pixel-level light intensity modulation, making it suitable for measuring complex surface topography.

[0033] The structured light projection module can be installed at multiple angles depending on the system configuration to reduce shadowing caused by the microstructure on the wafer surface; it can also be installed in a coaxial or near-coaxial manner to achieve consistency between the projection and imaging optical paths, thereby improving measurement coaxiality and signal quality.

[0034] In multi-angle installation mode, multiple projection mechanisms project structured light from different directions to cover various areas of the wafer surface, avoiding data loss due to obstruction; in coaxial mode, the projection light path and imaging light path are coupled through a beam splitter to ensure that the incident light and reflected light are coaxial, which is suitable for stable measurement of highly reflective surfaces.

[0035] The projection device projects patterns including sinusoidal fringes, phase-shifted gratings, or binary coded patterns, while the imaging module simultaneously acquires images of the deformed fringes modulated by the wafer surface. Absolute phase information is extracted from the deformed fringes using phase-calculation algorithms, such as phase-shifting or Fourier transform, and then converted into depth data in conjunction with system calibration parameters.

[0036] Structured light data is therefore defined as including phase data and depth data, where phase data characterizes the continuous distribution of surface deformation and depth data provides three-dimensional geometric information.

[0037] 1.2 Photometric Acquisition Module

[0038] The photometric acquisition module is used to generate multi-angle, multi-wavelength controllable illumination during the unstructured light projection stage to acquire two-dimensional reflection, texture, and photometric information of the wafer surface. This module includes multiple light sources in different orientations, whose illumination sequence, intensity, and wavelength can be programmably controlled to accommodate photometric stereoscopic vision illumination modes. The light source array typically consists of LED or laser light sources, arranged at different angles around the wafer, illuminating the wafer surface from multiple directions through a time-division multiplexing illumination method.

[0039] The imaging module simultaneously acquires surface images under each illumination condition, obtaining a series of reflection images at different illumination angles. For example, using a photometric stereo vision algorithm, surface normal data and surface reflectivity data are estimated from these images. Surface normal data characterizes the orientation of points on the wafer surface, while reflectivity data reflects surface material properties, such as the optical response of metal, oxide, or contaminated areas. Multi-wavelength illumination further expands the material characterization capabilities; by acquiring reflection images at different wavelengths, defects such as surface oxidation, contamination, or corrosion can be distinguished, as these defects exhibit different reflectivity characteristics at different wavelengths.

[0040] The light source selection and exposure sequence of the photometric acquisition module can be programmed and set through the main control system to adapt to different measurement needs. For example, in high-reflectivity areas, the light source intensity can be reduced or the exposure time adjusted to avoid saturation; in low-reflectivity areas, the illumination intensity can be increased to improve the signal-to-noise ratio.

[0041] This module provides important supplementary information for subsequent multimodal fusion. In particular, when structured light data fails due to reflection or occlusion, photometric data can serve as a reliable substitute, ensuring the robustness of the measurement.

[0042] 1.3 Imaging Module

[0043] The imaging module consists of multiple industrial cameras used to simultaneously acquire structured light and photometric data under multi-view conditions. Each camera can be configured with lenses of different focal lengths, resolutions, or depths of field to achieve complementarity between spatial resolution and field of view. For example, some cameras use high-resolution, narrow-field-of-view lenses to capture micrometer-level details, while other cameras use low-resolution, wide-field-of-view lenses to cover macroscopic scenes, ensuring that the system achieves a balance between micrometer-level resolution and macroscopic coverage.

[0044] The camera arrangement of the imaging module considers multi-view coverage to reduce occlusion and improve data diversity. Cameras achieve microsecond-level synchronization through hardware triggering or clock synchronization mechanisms, ensuring simultaneous exposure at the moment of structured light projection or photometric illumination, avoiding data mismatch due to timing errors. Images acquired by each camera include deformable fringe images under structured light and reflectance images under photometric illumination. These images are used in subsequent processing to extract phase, depth, normal, and reflectance information, respectively.

[0045] The heterogeneity of the imaging module is reflected in the differences in camera parameters. Different cameras may have different imaging characteristics, such as distortion, color response, or noise distribution. Through calibration and correction, these differences are uniformly processed to ensure the geometric and optical consistency of multi-view data.

[0046] Specifically, the imaging module includes a first imaging module and a second imaging module, wherein: the first imaging module is used to acquire deformed stripe images modulated on the wafer surface to calculate structured light data; the second imaging module is used to simultaneously acquire wafer surface images under different illumination angles to calculate photometric stereo data.

[0047] 1.4 Control and Calibration Module

[0048] The control and calibration module is used to achieve timing synchronization, geometric calibration, and adaptive control of the components in the data acquisition module. This module includes a main controller and a calibration subsystem. The main controller is responsible for coordinating the triggering timing of the light source, camera, and projection equipment, using hardware triggering or a high-precision clock synchronization mechanism to ensure microsecond-level accuracy in exposure and light source switching.

[0049] For example, in the structured light measurement stage, the controller sequentially triggers the projection device to project a phase-shift pattern and simultaneously triggers the camera to acquire images; in the photometric acquisition stage, the controller illuminates each light source in a time-division manner and triggers the camera to acquire images under the corresponding illumination.

[0050] The calibration subsystem performs extrinsic parameter calibration and distortion correction for multiple cameras and multiple projection mechanisms. Through calibration, it obtains the internal parameters of each camera, such as focal length, principal point, distortion coefficient, and external parameters, such as rotation matrix and translation vector, and then establishes the camera-projection geometric model.

[0051] The calibration process typically uses a standard target, such as a checkerboard or dot array, and acquires target images from multiple perspectives to calculate the transformation relationship between the camera and the projector.

[0052] The calibration results are used to map multi-view data to a unified coordinate system, ensuring that structured light data and photometric stereo data are spatially consistent.

[0053] The control and calibration module also features adaptive control, dynamically adjusting the light source intensity and exposure parameters based on the real-time reflection signal quality.

[0054] For example, when image saturation is detected, the light source intensity is automatically reduced or the exposure time is shortened; when the signal is too weak, the illumination intensity is increased or the exposure time is extended. This adaptive mechanism ensures that the measurement process can obtain stable and reliable data under different surface conditions.

[0055] Optionally, the data processing module includes the following specific components:

[0056] 2.1 Preprocessing Unit

[0057] The preprocessing unit is used to perform preliminary processing on the multi-view structured light and photometric data acquired by the data acquisition module, including geometric correction, brightness equalization and distortion compensation, and to establish a unified coordinate system and camera-projection geometric model.

[0058] Geometric correction, based on calibration parameters, maps images acquired by each camera to a unified world coordinate system, eliminating perspective distortion caused by differences in camera position and attitude.

[0059] Distortion compensation corrects the effects of lens distortion on the image by applying distortion correction models, such as the Brown-Conrad model.

[0060] Brightness equalization processing is applied to photometric data. Since image brightness may vary under different illumination angles and wavelengths, normalization or histogram matching techniques are used to ensure that the brightness distribution of each image is consistent, facilitating subsequent photometric stereoscopic calculations. For structured light data, the preprocessing unit performs phase calculation, extracts the absolute phase map from the deformed fringe image, and generates a preliminary depth map by combining it with calibration parameters.

[0061] The output of the preprocessing unit includes the corrected phase map, depth map, reflection map, and normal map. These data serve as the input to the physics-driven fusion module, providing a spatial and optical consistency basis for multimodal fusion.

[0062] 2.2 Physics Prior-Driven Fusion Module

[0063] The physics-prior-driven fusion module is the core algorithm module of this system. Based on a heterogeneous Transformer network structure, it achieves self-attention fusion and physical consistency constraints for multi-view heterogeneous data. This module introduces a phase-guided sparse attention mechanism and combines it with various physical prior biases, including position compatibility constraints, phase-depth residual constraints, normal consistency constraints, and reflection similarity constraints, to ensure that the fusion result conforms to physical laws at the geometric, phase, and reflection levels. Specifically, it includes the following:

[0064] 2.2.1 Heterogeneous Transformer Model Structure

[0065] The heterogeneous Transformer model employs an encoder-decoder architecture. The encoder extracts feature representations from multimodal data, while the decoder uses an attention mechanism to achieve feature fusion and 3D shape inversion. Input data includes phase maps, depth maps, normal maps, and reflection maps from multiple perspectives. These data are processed by a convolutional neural network to extract local and global features, forming a feature sequence.

[0066] The core of Transformer is the self-attention mechanism, which calculates the similarity between queries, keys, and values, assigns attention weights, and achieves weighted fusion of features. In Heterogeneous Transformer, the self-attention mechanism is extended to multimodal attention, simultaneously processing structured light and photometric data, and introducing physical priors as bias terms to guide the calculation of attention weights.

[0067] 2.2.2 Phase-guided sparse attention mechanism

[0068] Phase-guided sparse attention mechanisms filter high-confidence matching regions through phase consistency, reducing false matches and improving local physical confidence. The specific process is as follows:

[0069] First, based on the phase map acquired from multiple perspectives, absolute phase and position feature information are extracted, and query points corresponding to the main perspective are established;

[0070] Subsequently, based on the geometric calibration relationship, corresponding candidate regions are searched from other perspectives to obtain a preliminary matching candidate set.

[0071] During the candidate set selection phase, the system calculates the phase residual and depth difference between pixels from different viewpoints to construct a sparse candidate set. The phase residual reflects phase consistency, while the depth difference characterizes geometric consistency.

[0072] For example, by using Top-k sorting, only the top k candidate points with the highest phase compatibility are retained, where k is a preset integer, typically set based on a trade-off between measurement accuracy and computational efficiency. This process significantly reduces the attention computation range, performing attention weight calculation only within high-confidence candidate regions, thereby reducing the false match rate.

[0073] 2.2.3 Embedding of Physical Prior Constraints

[0074] Physical priors are defined as a set of physical rules constraining the fusion process of heterogeneous Transformer models, including position compatibility priors, phase-depth residual priors, normal consistency priors, and reflection similarity priors. These priors are embedded as physical bias terms in the calculation of self-attention weights, ensuring that the fusion result simultaneously satisfies data-driven similarity and physical consistency. Specifically:

[0075] (1) Position compatibility constraints are based on multi-view calibration parameters, which map points in different camera coordinate systems to a unified space to ensure the consistency of geometric correspondence. In attention calculation, positional bias is encoded as a bias term, which penalizes feature pairs that do not match in space.

[0076] (2) The phase-depth residual constraint uses phase error and depth difference to form a physical residual bias, which guides the attention allocation. Point pairs with high phase consistency receive higher attention weights, ensuring that the fusion process prioritizes physically reliable regions.

[0077] (3) Normal consistency constraint combines the normal calculations of photometric stereo vision with the structured light geometry results to ensure the consistency between the photometric model and the geometric model at the spatial and physical levels. In the attention mechanism, normal differences are quantified as bias terms to promote feature fusion with normal consistency.

[0078] (4) Reflection similarity constraint ensures that the surface reflectance of the same surface point observed from multiple viewpoints remains consistent by modeling the reflection intensity of multiple wavelengths. Reflection similarity is encoded as an attention bias to identify defects such as oxidation, contamination, and corrosion, as these defects can disrupt reflection consistency.

[0079] 2.2.4 Two-stage registration strategy

[0080] The data processing module employs a two-stage strategy of ICP coarse registration and Transformer residual correction during the topography inversion process:

[0081] The first stage performs coarse registration by iteratively using the nearest neighbor algorithm or its variants to obtain the initial transformation matrix between multimodal data, achieving global alignment of point clouds from multiple perspectives. The ICP algorithm, based on nearest neighbor search and least squares optimization, progressively optimizes the transformation parameters to minimize the distance between point clouds.

[0082] The second stage performs fine registration, inputting the initial transformation matrix and multimodal data into a heterogeneous Transformer model that incorporates physical priors, and outputting a high-precision residual transformation matrix iteratively. Based on the aforementioned physical prior biases, the Transformer residual correction unit adaptively optimizes local details at the deep learning level, correcting residual errors in coarse registration and achieving accurate reconstruction of 3D morphology and defect features.

[0083] 2.2.5 Closed-loop inversion and optimization

[0084] The fusion module supports a closed-loop iterative process of inversion and optimization. In each iteration, the system updates the depth, normal, and reflection features based on the fusion output, recalculates phase consistency and physical bias, and dynamically adjusts the attention weights. This progressive optimization mechanism achieves 3D reconstruction from local matching to global consistency, gradually improving topographic accuracy and defect identification reliability.

[0085] The system in this embodiment achieves significant technological advancements in 3D wafer surface inspection through a heterogeneous Transformer mechanism that incorporates physical priors. First, the introduction of physical priors ensures that the multimodal data fusion process conforms to the physical laws of imaging, avoiding errors caused by neglecting physical consistency in traditional methods. Second, the phase-guided sparse attention mechanism effectively reduces false matches and improves local fusion accuracy by selecting highly reliable candidate points. The two-stage registration strategy combines the global robustness of ICP with the local adaptability of the Transformer, maintaining high-precision reconstruction even under complex reflection and occlusion conditions.

[0086] In summary, this system acquires multimodal optical data through the data acquisition module and uses the data processing module to achieve high-precision 3D reconstruction and defect identification based on a heterogeneous Transformer model with physical prior constraints. The core of the system lies in the phase-guided sparse attention mechanism, the embedding of multiple physical priors, and the application of a two-stage registration strategy, which ensures robustness and accuracy in complex measurement environments.

[0087] Example 2:

[0088] like Figure 2 As shown, corresponding to the above system embodiment, the present invention also proposes a method for three-dimensional inspection of heterogeneous Transformer wafers that integrates physical priors, including the following steps:

[0089] The first step is data acquisition. The data acquisition module acquires multimodal data of the wafer surface under test. Specifically, this includes: projecting an coded structured light pattern onto the wafer surface using a structured light projection module, and acquiring deformed fringe images using a first imaging module to calculate structured light data, including phase and depth data; illuminating the wafer surface from different angles using a photometric acquisition module by illuminating multiple light sources at different times using a second imaging module, and simultaneously acquiring surface images at different illumination angles to calculate photometric stereo data, including surface normal data and surface reflectivity data. The control and calibration module synchronizes the acquisition timing of each component and stores the internal parameters of the imaging module and the external transformation relationships between the multimodal data.

[0090] The second step is preprocessing. The preprocessing unit performs geometric correction, brightness equalization, and distortion compensation on the acquired multimodal data, establishing a unified coordinate system to obtain the corrected phase map, depth map, normal map, and reflection map.

[0091] The third step is the physics-prior-driven fusion process. This involves using a physics-prior-driven fusion module to perform the fusion and registration of multimodal data. Specifically, this includes: constructing a heterogeneous Transformer model with fused physical priors; employing a phase-guided sparse attention mechanism to calculate phase consistency residuals based on phase data, selecting Top-k candidate point pairs to sparsify the attention matrix; embedding physical priors such as position compatibility, phase-depth residuals, normal consistency, and reflection similarity as physical bias terms into the attention weight calculation; performing two-stage registration: first obtaining the initial transformation matrix through ICP coarse registration, then outputting a high-precision residual transformation matrix through Transformer residual correction; and dynamically adjusting the attention bias through closed-loop inversion and optimization to gradually optimize the 3D topography results.

[0092] The fourth step is the output and recognition process. A high-precision 3D model of the wafer surface is generated, and surface defects, such as oxidation, contamination, corrosion, and deformation defects, are identified based on reflection similarity constraints and normal consistency constraints. The output includes a 3D topographic point cloud, a defect distribution map, and quantization parameters.

[0093] This method significantly improves the fusion accuracy of heterogeneous data by effectively embedding prior physical knowledge into the Transformer model. Compared with traditional methods, it can maintain high 3D reconstruction accuracy even under complex reflection and occlusion conditions. This embodiment utilizes a phase-guided sparse attention mechanism, where the system optimizes the data registration process according to physical laws, reducing noise interference and improving the accuracy of defect detection. Furthermore, the two-stage registration strategy of this invention effectively combines coarse and fine registration steps, further improving the robustness and accuracy of the overall system, thus providing an innovative solution for high-precision 3D reconstruction and defect identification of wafer surfaces.

[0094] Example 3:

[0095] Corresponding to the above embodiments, the present invention also proposes an electronic device.

[0096] like Figure 3 The diagram shows a structural schematic of an electronic device according to the present invention. The electronic device 100 includes a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, for example, via a bus 102. Optionally, the electronic device 100 may further include a transceiver 104. It should be noted that in practical applications, the transceiver 104 is not limited to one unit, and the structure of this electronic device 100 does not constitute a limitation on the embodiments of the present invention.

[0097] Processor 101 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 101 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0098] Bus 102 may include a pathway for transmitting information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. Bus 102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0099] The memory 103 stores a computer program corresponding to the heterogeneous Transformer wafer 3D inspection method with integrated physical priors described in the above embodiments of the present invention. This computer program is executed under the control of the processor 101. The processor 101 executes the computer program stored in the memory 103 to implement the content shown in the aforementioned method embodiments.

[0100] Among them, electronic devices 100 include, but are not limited to: mobile terminals such as laptops and PADs (tablet computers) and fixed terminals such as desktop computers. Figure 3 The electronic device 100 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0101] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A heterogeneous Transformer wafer 3D inspection system integrating physical priors, characterized in that, include: The data acquisition module is configured to acquire multimodal data of the surface of the wafer under test, wherein the multimodal data includes at least structured light data and photometric stereo data; The data processing module is connected to the data acquisition module and configured as follows: Construct a heterogeneous Transformer model that incorporates physical priors; Using the physical priors as constraints, the multimodal data are fused and registered to generate a three-dimensional model of the wafer surface.

2. The system according to claim 1, characterized in that, The data acquisition module includes: A structured light projection module is used to project an encoded structured light pattern onto the wafer surface; The first imaging module is used to acquire the deformed stripe image modulated by the wafer surface in order to calculate the structured light data; The structured light data is defined as including phase data and / or depth data.

3. The system according to claim 1 or 2, characterized in that, The data acquisition module includes: The photometric acquisition module includes multiple light sources in different directions, which are used to illuminate the wafer surface from different angles in a time-division manner. The second imaging module is used to simultaneously acquire images of the wafer surface under different illumination angles in order to calculate the photometric three-dimensional data. The photometric three-dimensional data is defined as including surface normal data and / or surface reflectance data.

4. The system according to claim 1, characterized in that, The data acquisition module also includes: The control and calibration module is used to synchronize the acquisition timing of each component in the data acquisition module, and to store the internal parameters of the first imaging module and the second imaging module, as well as the external transformation relationship between the structured light data and the photometric stereo data.

5. The system according to claim 1, characterized in that, The physical priors include at least one or more of the following: Location compatibility prior: characterizing the geometric consistency that the structured light data and the photometric stereo data should have after spatial transformation; Phase-depth residual prior: Characterizes the physical mapping relationship between phase information and depth information in the structured light data; Normal consistency prior: characterizing the consistency between the normal calculated from the structured light data and the normal obtained from the photometric stereo data; Reflection similarity prior: Characterizes that the surface reflectance of the same surface point observed from multiple viewpoints should remain consistent.

6. The system according to claim 1, characterized in that, The core attention mechanism of the heterogeneous Transformer model is a phase-guided sparse attention mechanism. The data processing module is specifically configured as follows: The phase consistency residuals between point pairs are calculated using phase data from the structured light data. The attention matrix in the Transformer model is sparsified based on the phase consistency residual, and only the top K candidate point pairs with the highest phase consistency are selected to participate in the subsequent attention weight calculation, where K is a preset integer.

7. The system according to claim 6, characterized in that, The data processing module is also configured to: The physical prior is embedded as a physical bias term into the weight calculation of the phase-guided sparse attention mechanism; This ensures that the output of the heterogeneous Transformer model satisfies both the data-driven similarity matching and the physical prior constraints.

8. The system according to claim 1, characterized in that, The data processing module adopts a two-stage registration strategy, specifically configured as follows: The first stage of coarse registration is performed by obtaining the initial transformation matrix between the multimodal data through the iterative nearest point algorithm ICP or its variants. The second stage of fine registration is performed by inputting the initial transformation matrix and the multimodal data into the heterogeneous Transformer model that integrates physical priors, and outputting a high-precision residual transformation matrix in an iterative manner to realize the three-dimensional reconstruction of the wafer surface.

9. A method for three-dimensional inspection of heterogeneous Transformer wafers that integrates physical priors, characterized in that, Includes the following steps: Data acquisition steps: Use a data acquisition module to acquire multimodal data of the surface of the wafer under test, wherein the multimodal data includes at least structured light data and photometric stereo data; Data processing steps: Execute using the data processing module: (a) Construct a heterogeneous Transformer model that integrates physical priors, wherein the physical priors include at least position compatibility priors, phase-depth residual priors, normal consistency priors, or reflection similarity priors; (b) A phase-guided sparse attention mechanism is adopted, and the phase consistency residual calculated using the phase data is used to select Top-k candidate point pairs to sparsify the attention matrix; (c) Using the physical prior as a constraint, the multimodal data is fused and registered to generate a high-precision three-dimensional model of the wafer surface.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, which, when executed by the processor, implements the heterogeneous Transformer wafer 3D inspection method with fused physical priors as described in claim 9.

Images