A differential interferometric true-vision 2.5D imager and its imaging method

Abstract

This invention discloses a differential interferometric true-vision 2.5D imager and its imaging method, comprising: an optical imaging module, a main control task management module, a 2.5D video real-time fusion calculation module, a 2.5D video display module, a human-computer interaction module, and a data storage module. Through the structural design of the optical imaging module, the imager achieves a dual-optical-path imaging acquisition structure, enabling simultaneous acquisition of visible light texture images and differential interferometric gradient images of the sample under test. Furthermore, combined with the 2.5D video real-time fusion calculation module, it performs real-time correction, mapping, and fusion calculations of the two images to generate 2.5D video frames with texture information and morphological representation capabilities, which are then output in real time. This imager's design allows it to more intuitively present minute defects such as scratches, dirt, and microcracks in the 2.5D video frames, while simultaneously enabling the rapid output of 2.5D images that accurately represent both surface texture and morphological variations.

Description

Technical Field

[0001] This invention relates to the field of detection imaging technology, and specifically discloses a differential interferometric true vision 2.5D imager and its imaging method. Background Technology

[0002] In the manufacturing process of laser chips and other microstructured optoelectronic devices, high precision and efficiency are required for detecting surface scratches, dirt, particles, microcracks, local depressions, step structures, and other micro-defects. Existing microscopic inspection methods mainly include bright-field microscopy, scanning electron microscopy, atomic force microscopy, and optical profilometers.

[0003] Bright-field microscopy offers advantages such as fast imaging speed, low cost, and ease of industrial integration. However, it primarily provides two-dimensional brightness and texture information, failing to directly reflect the height variations of the sample surface and thus struggling to meet the needs for morphological identification and quantitative assessment of minute defects. Scanning electron microscopy (SEM) and atomic force microscopy (AFM), while possessing high spatial resolution and morphological detection capabilities, suffer from high equipment costs, slow detection speeds, and stringent environmental requirements, making them unsuitable for real-time detection scenarios in industrial production lines. Optical profilometers can provide some three-dimensional surface information, but under conditions of high reflectivity, low texture, and complex microstructure samples, they still suffer from insufficient real-time performance, complex system integration, and limited adaptability to moving samples.

[0004] Differential Interference Contrast (DIC) microscopy can convert optical path differences caused by the sample surface into variations in brightness and darkness, providing strong contrast visualization of local gradient changes on transparent or reflective sample surfaces, thus offering certain advantages in microstructural observation. However, images output by traditional DIC microscopes primarily reflect surface height gradients or optical path differences, typically producing qualitative relief-like images, making it difficult to directly output 2.5D images suitable for real-time observation and interpretation in industrial settings.

[0005] Existing 2.5D or 3D reconstruction schemes typically rely on complex software post-processing, such as deep learning-based depth estimation, multi-step diffusion generation, or iterative optimization reconstruction. While these schemes can improve surface topography restoration to some extent, they involve large-scale neural network inference, multiple rounds of iterative solving, and complex image generation and processing, resulting in high single-frame processing latency. If depth information is inferred from a single frame image solely through software algorithms, even with high accuracy, it is difficult to achieve continuous video-level output in industrial production lines. Especially in scenarios involving continuous transport, online inspection, or rapid sampling, if the 2.5D image output frame rate cannot reach video level, it is often difficult to directly apply it in practice.

[0006] Therefore, there is an urgent need to propose a new imaging device and its implementation method, which can retain the advantage of differential interferometric contrast imaging in its sensitivity to surface undulations, while also being able to quickly output 2.5D images that have the ability to express both real surface texture and morphological undulations, so as to meet the needs of industrial sites for online, non-contact, and low-latency detection. Summary of the Invention

[0007] (a) Technical problems to be solved To address the aforementioned problems, this invention provides a differential interferometric true-vision 2.5D imager and its imaging method. This imager simultaneously acquires visible light texture images and differential interferometric gradient images of the sample under test through a dual-optical-path synchronous acquisition method. Through deterministic real-time correction, mapping, and fusion calculations, it generates true-vision 2.5D video frames with texture information and morphological variation representation capabilities, thereby achieving real-time output of 2.5D video streams to meet the demands of modern industry for online, non-contact, and low-latency detection.

[0008] (II) Technical Solution To address the aforementioned technical problems, this invention proposes a differential interferometric true-vision 2.5D imager. This imager is used for real-time imaging output of sample surface texture and morphology information, and includes: an optical imaging module, a main control task management module, a 2.5D video real-time fusion calculation module, a 2.5D video display module, a human-computer interaction module, and a data storage module. The optical imaging module is used to perform visible light texture imaging and differential interference contrast imaging on the sample under test, so as to simultaneously acquire visible light texture images and DIC images of the same area under test. The main control task management module is connected to the optical imaging module to synchronously trigger and control the optical imaging module, and to schedule the image acquisition, image processing and video output processes. The 2.5D video real-time fusion computing module is connected to the main control task management module and is used to perform real-time registration and correction, DIC intensity-depth gradient mapping, depth map reconstruction and 2.5D image synthesis on the visible light texture image and the DIC image to generate continuous 2.5D video frames. The 2.5D video frames include a texture channel provided by the visible light texture image and a height channel obtained by mapping and reconstructing the DIC image. The 2.5D video display module is connected to the 2.5D video real-time fusion computing module and is used to receive and display continuous 2.5D video frames. The continuous 2.5D video frames are output as a 2.5D video stream in chronological order. The human-computer interaction module is used to receive parameter setting instructions and output device status information; The data storage module is used to store image data, calibration parameters, processing parameters, intermediate results, and output results.

[0009] Preferably, the optical imaging module includes a light source, a collimating lens group, a polarizer, a reflecting mirror group, a first beam splitter, a microscope objective, a second beam splitter, a first tube lens, a Nomarski prism, an analyzer, a second tube lens, a first image sensor, a second image sensor, and a sample stage. The sample stage is used to hold the sample to be tested. The unpolarized light emitted from the light source is collimated by the collimating lens group and enters the polarizer, where it is converted into linearly polarized light. The linearly polarized light enters the microscope objective through the mirror group and the first beam splitter and illuminates the surface of the sample to be tested. The light beam reflected or scattered back by the sample to be tested returns through the microscope objective and is then split into a first optical path and a second optical path by the second beam splitter. The light beam in the first optical path is captured by the first image sensor after passing through the first tube lens, forming a visible light texture image; The light beam in the second optical path is split into two beams with slight shear displacement and orthogonal polarization states after passing through the Nomarski prism. The two beams interfere in the polarization direction after passing through the analyzer, and are then acquired by the second image sensor after passing through the second tube lens to form a DIC image.

[0010] Preferably, the 2.5D video real-time fusion computing module includes an image registration and correction unit, a DIC intensity-depth gradient mapping unit, a depth map reconstruction unit, and a 2.5D image synthesis unit; The image registration and correction unit is used to perform geometric registration, distortion correction and scale unification on the visible light texture image and the DIC image. The DIC intensity-depth gradient mapping unit is used to establish the mapping relationship between the pixel intensity of the DIC image and the depth gradient of the surface of the sample under test along the shear direction, and to convert the pixel intensity in the DIC image into the depth gradient value of the surface of the sample under test along the shear direction. The depth map reconstruction unit is used to reconstruct the relative height distribution of the surface of the sample under test based on the depth gradient value; The 2.5D image synthesis unit is used to fuse the visible light texture image with the relative height distribution of the sample under test reconstructed by the depth map reconstruction unit to generate the 2.5D video frame.

[0011] Preferably, the depth map reconstruction unit includes at least one of the following methods to reconstruct the relative height distribution of the sample surface: discrete integral reconstruction, row and column direction cumulative integration, fast Poisson reconstruction, lookup table mapping, or lightweight optimization solution.

[0012] Preferably, the 2.5D image synthesis unit uses the visible light texture image as the texture channel and the relative height distribution of the surface of the sample under test as the height channel, and generates the 2.5D video frame through at least one or more of pseudo-color mapping, embossing enhancement, lighting simulation, and transparency overlay.

[0013] Preferably, the 2.5D video real-time fusion computing module is executed by an image processor, which includes one or more of a GPU, FPGA, ASIC, DSP, or edge computing board; the image processor is used to perform image registration correction, DIC intensity-depth gradient mapping, depth restoration, and 2.5D image synthesis in a pipelined parallel manner.

[0014] This invention also provides a real-time imaging method for a differential interferometric true-view 2.5D imager. The real-time imaging method is implemented based on the aforementioned differential interferometric true-view 2.5D imager, and includes: S1. Visible light texture imaging and differential interference contrast imaging: The sample to be tested is illuminated by an optical imaging module, and the reflected imaging beam of the sample to be tested is obtained by microscopic imaging. The second beam splitter in the optical imaging module divides the reflected imaging beam into a first optical path and a second optical path. The first optical path is used to acquire visible light texture images, and the second optical path is used to acquire DIC images after differential interference processing. S2. The first and second optical paths are synchronously triggered for exposure and synchronous reading through the main control task management module to obtain the visible light texture image and DIC image with time correspondence within the same trigger period. S3. The visible light texture image and DIC image are registered and corrected in real time using the 2.5D video real-time fusion computing module; S4. Map the DIC image to the depth gradient information of the sample surface under test, and recover the relative height distribution of the sample surface under test based on the depth gradient information. Then, fuse the texture information of the visible light texture image with the relative height distribution of the sample surface under test to generate a true-view 2.5D video frame. S5. The continuously generated true-to-life 2.5D video frames are output as a 2.5D video stream in chronological order through the 2.5D video display module.

[0015] Preferably, in step S4, the 2.5D video real-time fusion computing module converts the DIC image into depth gradient information through a pre-calibrated lookup table, linear mapping relationship, or piecewise mapping relationship, and recovers the relative height distribution of the surface of the sample under test by at least one of integral reconstruction, optimization solution, or lookup table mapping.

[0016] Preferably, when fusing the texture information and height information in step S4, at least one of pseudo-color mapping, transparency overlay, emboss rendering, and shadow enhancement is used to generate the 2.5D video frame.

[0017] Preferably, the visible light texture image and the DIC image are unified to a preset resolution before entering the fusion process, and 2.5D video frames are continuously generated using a pipelined image processing method so that the output frame rate of the 2.5D video stream at the preset resolution is not less than 30 frames / second.

[0018] (III) Beneficial Effects Compared with existing technologies, the differential interferometric true-vision 2.5D imager of the present invention has the following advantages: 1. This invention constructs a dual-optical-path imaging acquisition structure. Through the structural setting of the optical imaging module, this invention divides the returned imaging beam into a visible light texture imaging optical path and a differential interference contrast imaging optical path, enabling the simultaneous acquisition of texture information of the sample surface and gradient information generated by differential interference in the same device, providing a data foundation for real-time 2.5D imaging.

[0019] 2. This invention acquires some surface morphology-sensitive information in the front end through differential interferometric contrast imaging, and adopts deterministic real-time correction, mapping, and fusion calculation methods in the back end. Compared with 2.5D reconstruction schemes that rely on complex deep learning inference, multi-step diffusion generation, or iterative optimization, this invention can effectively reduce single-frame processing latency and is more suitable for industrial online inspection scenarios. 3. This invention innovatively combines DIC images and visible light texture images, enabling the generated 2.5D video frames to quickly output 2.5D images that express both realistic surface textures and morphological undulations, and to intuitively present minute defects such as scratches, dirt, particles, microcracks, local depressions, and step structures.

[0020] In summary, the differential interferometric true-vision 2.5D imager provided by this invention achieves a dual-optical-path imaging acquisition structure through the structural design of the optical imaging module. This allows it to simultaneously acquire visible light texture images and differential interferometric gradient images of the sample under test. Combined with the 2.5D video real-time fusion calculation module, it realizes real-time correction, mapping, and fusion calculation of the two images to generate true-vision 2.5D video frames with texture information and topographic relief capabilities. This enables real-time output of 2.5D video streams, allowing the imager to more intuitively present fine defects such as scratches, dirt, particles, microcracks, local depressions, and step structures in 2.5D video frames. At the same time, it improves the 2.5D image output speed, enabling the imager to quickly output 2.5D images with realistic surface texture and topographic relief capabilities, and further realize 2.5D video stream output to meet the needs of industrial sites for online, non-contact, and low-latency detection. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the overall system framework of the differential interferometric true vision 2.5D imager of the present invention; Figure 2 This is a schematic diagram of the optical imaging module of the differential interferometry true vision 2.5D imager of the present invention; Figure 3 This is a schematic diagram of the optical imaging module of the differential interferometry true vision 2.5D imager of the present invention; Figure 4 This diagram illustrates the data interaction between the main control task management module, the 2.5D video real-time fusion calculation module, the image processor, the 2.5D video display module, and the data storage module of the differential interferometric true vision 2.5D imager of the present invention. Figure 5 This is a flowchart illustrating the workflow of the 2.5D video real-time fusion calculation module in the differential interferometric true vision 2.5D imager of the present invention. Figure 6 This is a schematic diagram of the overall structure of the differential interferometry true vision 2.5D imager of the present invention; Figure 7 This is a schematic diagram of the internal structure of the differential interferometric true vision 2.5D imager of the present invention.

[0023] Explanation of reference numerals in the attached figures: S1, Optical Imaging Module; S2, First Optical Path; S3, Second Optical Path; S4, Chassis; S5, Internal Installation Space; S6, Power Supply Module; S7, Main Control Task Management Module; S8, 2.5D Video Real-time Fusion Computing Module; S9, 2.5D Video Display Module; S10, Human-Computer Interaction Module; S11, Data Storage Module; S12, Sample Stage; S13, Sample to be Tested; P1, Light source; P2, Collimating lens group; P3, Polarizer; P4, Reflecting mirror group; P5, First beam splitter; P6, Microscope objective; P7, Oil immersion medium; P8, Second beam splitter; P9, First tube lens; P10, Nomarski prism; P11, Analyzer; P12, Second tube lens. T1, first image sensor; T2, second image sensor; C1, Main Controller; U1, Synchronous Trigger Reading Control Unit; U2, Image Registration and Correction Unit; U3, DIC Intensity-Depth Gradient Mapping Unit; U4, Depth Map Reconstruction Unit; U5, 2.5D Image Synthesis Unit; M1, Image Processor; M2, Storage Medium; H1, HMI touchscreen; D1, display screen; D2, power interface; D3, sample loading port; L1, bus; L2, data transmission link; B1, Unpolarized light; B2, Linearly polarized light; B3, First beam splitter; B4, Second beam splitter; B5, Interference beam after polarization analysis. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0025] The following is in conjunction with the appendix Figure 1-7 The differential interferometric true vision 2.5D imager of the present invention will be further described.

[0026] Please refer to this carefully. Figure 1-2This invention discloses a differential interferometric true-vision 2.5D imager, which is used for real-time imaging output of sample surface texture and morphology information. The differential interferometric true-vision 2.5D imager includes: an optical imaging module S1, a main control task management module S7, a 2.5D video real-time fusion calculation module S8, a 2.5D video display module S9, a human-computer interaction module S10, and a data storage module S11. The optical imaging module S1 is used to perform visible light texture imaging and differential interferometric contrast imaging on the sample under test S13, so as to simultaneously acquire the visible light texture image and DIC image of the same area under test. The main control task management module S7 is connected to the optical imaging module S1 to synchronously trigger and control the optical imaging module S1, and to schedule the image acquisition, image processing and video output processes. The 2.5D video real-time fusion computing module S8 is connected to the main control task management module and is used to perform real-time registration and correction, DIC intensity-depth gradient mapping, depth map reconstruction and 2.5D image synthesis on visible light texture images and DIC images to generate continuous 2.5D video frames. The 2.5D video frames include texture channels provided by visible light texture images and height channels obtained by mapping and reconstructing DIC images. The 2.5D video display module S9 is connected to the 2.5D video real-time fusion computing module S8, and is used to receive and display continuous 2.5D video frames. The continuous 2.5D video frames are output as a 2.5D video stream in chronological order. The human-computer interaction module S10 is used to receive parameter setting instructions and output device status information. The data storage module S11 is used to store image data, calibration parameters, processing parameters, intermediate results and output results.

[0027] Among them, the 2.5D video real-time fusion computing module S8 is configured to generate 2.5D video frames in a streaming mode, so that the output frame rate of the 2.5D video stream at the preset resolution is not less than 30 frames / second.

[0028] In this embodiment, the differential interferometric true-vision 2.5D imager, through the structural configuration of the optical imaging module S1, achieves a dual-optical-path imaging acquisition structure. This allows it to simultaneously acquire the visible light texture image and the differential interferometric gradient image of the sample under test S13. The 2.5D video real-time fusion calculation module S8 performs real-time correction, mapping, and fusion calculations of the two images, generating a true-vision 2.5D video frame with texture information and morphological variation expression capabilities, thus achieving real-time output of the 2.5D video stream. Furthermore, by combining the advantage of differential interferometric contrast imaging's sensitivity to minute surface height gradient changes with the intuitiveness of visible light texture imaging, this imager improves the 2.5D image output speed while sacrificing some high-precision offline reconstruction capabilities. This makes it suitable for online observation, rapid screening, and defect-aided identification in the manufacturing process of laser chips and other microstructured optoelectronic devices.

[0029] In a preferred embodiment, the real-time fusion calculation in its 2.5D video real-time fusion calculation module S8 is preferably a non-iterative or finite-step deterministic calculation, specifically including at least one of lookup table mapping, linear or piecewise linear mapping, discrete integration, fast filtering and image rendering, which does not rely on multi-step diffusion generation networks or large-scale deep learning inference processes, thereby reducing single-frame processing latency.

[0030] Furthermore, its main control task management module includes a main controller C1 and a synchronous trigger reading control unit U1; the main controller C1 is used to schedule image acquisition, image processing and video output tasks; the synchronous trigger reading control unit U1 is used to synchronously trigger exposure and synchronously read the first image sensor T1 and the second image sensor T2.

[0031] like Figure 2 and Figure 3 As shown, the optical imaging module S1 includes a light source P1, a collimating lens group P2, a polarizer P3, a reflecting mirror group P4, a first beam splitter P5, a microscope objective P6, a second beam splitter P8, a first tube lens P9, a Nomarski prism P10, an analyzer P11, a second tube lens P12, a first image sensor T1, a second image sensor T2, and a sample stage S12. The sample stage S12 is used to hold the sample S13 to be tested. The unpolarized light B1 emitted from the light source P1 is collimated by the collimating lens group P2 and enters the polarizer P3, where it is converted into linearly polarized light B2. The linearly polarized light B2 enters the microscope objective P6 through the reflecting mirror group P4 and the first beam splitter P5 and illuminates the surface of the sample S13 to be tested. The light beam reflected or scattered back from the sample S13 to be tested returns through the microscope objective P6 and is then split into a first optical path S2 and a second optical path S3 by the second beam splitter P8. The light beam in the first optical path S2 is acquired by the first image sensor T1 after passing through the first tube lens P9, forming a visible light texture image; the light beam in the second optical path S3 is split into two beams with small shear displacement and orthogonal polarization states after passing through the Nomarski prism P10 - the first beam B3 and the second beam B4. The two beams interfere in the polarization direction after passing through the analyzer P11, retaining only the polarization component that matches the polarization direction. The polarized interference beam B5 is then acquired by the second image sensor T2 after passing through the second tube lens P12, forming a DIC image.

[0032] In a preferred embodiment, an oil-impregnated medium P7 is disposed between the microscope objective P6 and the sample S13 to improve the numerical aperture and spatial resolution during microscopic observation, thereby enhancing the imaging capability for fine defects such as scratches, dirt, particles, microcracks, local depressions, and step structures on the surface of laser chips and other microstructured optoelectronic devices. Furthermore, the second optical path S3 has a phase offset adjustment function, which is achieved by adjusting at least one of the relative positions of the polarizer P3, analyzer P11, phase compensator, waveplate, Nomarski prism P10, or electrically controlled phase delayer, so that the intensity response of the DIC image is within a preset near-linear operating range.

[0033] In this embodiment, the Nomarski prism P10 splits the beam in the second optical path S3 into orthogonally polarized beams with minute shear displacements, converting minute height changes or optical path difference changes on the surface of the sample S13 into changes in interference intensity. The analyzer P11 projects the two orthogonally polarized beams onto the same polarization direction, causing interference. The polarization angles shown in the illustrations are only for illustrating the polarization conversion relationship and do not limit specific angle values; in one embodiment, a 135° polarization component is preferred. The first image sensor T1 and the second image sensor T2 are CMOS image sensors, preferably global shutter CMOS image sensors, and a synchronous trigger signal is output by the main control task management module to achieve synchronous exposure and synchronous reading, thereby reducing image misalignment caused by movement of the sample S13, platform vibration, or differences in acquisition timing.

[0034] like Figure 4 As shown, the image data collected by the first image sensor T1 and the second image sensor T2 are sent to the main control task management module S7 and the 2.5D video real-time fusion calculation module S8 via the bus L1 and the data transmission link L2. Under the task scheduling of the main controller C1, the image processor M1 performs real-time fusion calculation, and the calculation result is output to the display screen D1 and can be written to the storage medium M2.

[0035] The optical imaging module S1, through its dual-path structure, divides the acquired return imaging beam into a visible light texture imaging path and a differential interference contrast imaging path. This enables the imager to simultaneously acquire the texture information of the sample surface and the gradient information generated by differential interference in the same device, providing a data foundation for real-time 2.5D imaging.

[0036] See Figure 4 and Figure 5 The 2.5D video real-time fusion computing module S8 includes an image registration and correction unit U2, a DIC intensity-depth gradient mapping unit U3, a depth map reconstruction unit U4, and a 2.5D image synthesis unit U5. The image registration and correction unit U2 is used to perform geometric registration, distortion correction, and scale unification on the visible light texture image and the DIC image. The DIC intensity-depth gradient mapping unit U3 is used to establish the mapping relationship between the pixel intensity of the DIC image and the depth gradient along the shear direction of the surface of the sample S13 under test, and convert the pixel intensity in the DIC image into the depth gradient value along the shear direction of the surface of the sample S13 under test. The depth map reconstruction unit U4 is used to reconstruct the relative height distribution of the surface of the sample S13 under test based on the depth gradient value. The 2.5D image synthesis unit U5 is used to fuse the visible light texture image and the relative height distribution of the sample S13 under test reconstructed by the depth map reconstruction unit U4 to generate a 2.5D video frame.

[0037] The depth map reconstruction unit U4 reconstructs the relative height distribution of the surface of the sample S13 using at least one of the following methods: discrete integral reconstruction, row and column direction cumulative integral, fast Poisson reconstruction, lookup table mapping, or lightweight optimization solution. The 2.5D image synthesis unit U5 uses the visible light texture image as the texture channel and the relative height distribution of the sample S13 surface as the height channel. It generates 2.5D video frames through at least one or more of the following methods: pseudo-color mapping, embossing enhancement, lighting simulation, and transparency overlay. These true-view 2.5D video frames simultaneously represent the real texture information of the sample surface and the morphological undulation information derived from the DIC image.

[0038] In a preferred embodiment, the DIC intensity-depth gradient mapping unit U3 converts the DIC image using a pre-calibrated intensity-gradient mapping relationship. This intensity-gradient mapping relationship is established using standard step samples or samples with known height differences. Furthermore, the mapping relationship is implemented using a lookup table, linear mapping relationship, piecewise mapping relationship, or other real-time callable mapping models. Through this mapping relationship, the DIC image can be quickly converted into corresponding depth gradient information, reducing the computational load of real-time processing. Specific mapping functions include: Let the registered DIC pixel intensity be The corresponding depth gradient value is denoted as . It satisfies: ;in, This is the intensity-gradient mapping function obtained from calibration.

[0039] In another embodiment, the mapping function can be represented by a linear mapping relationship or a piecewise linear mapping relationship, for example: ;in, and These are calibration parameters.

[0040] The depth map reconstruction unit U4 is used to recover the relative height distribution of the sample surface based on the depth gradient. Let the depth gradient function be... Then the relative height of the sample surface can be expressed as: ;in, This represents the integration path along the shear direction. This is a correction term for the reference plane or boundary conditions.

[0041] To reduce the integral accumulation error, it is preferable to perform reference plane correction, local smoothing filtering, and abnormal gradient suppression on the results after the depth map reconstruction unit U4 completes the integral reconstruction, so as to reduce the integral accumulation error and obtain a 2.5D image suitable for real-time display.

[0042] In this embodiment, the imager acquires some surface morphology-sensitive information via differential interferometry contrast imaging through the optical imaging module S1. The backend employs a 2.5D video real-time fusion computing module S8, which uses deterministic real-time correction, mapping, and fusion computing. Compared to 2.5D reconstruction schemes that rely on complex deep learning inference, multi-step diffusion generation, or iterative optimization, this reduces single-frame processing latency and is more suitable for industrial online inspection scenarios. The 2.5D video real-time fusion computing module S8 is highly sensitive to local height changes, optical path difference changes, and edge abrupt changes on the sample surface through DIC images. Combined with visible light texture images, it can more intuitively present minute defects such as scratches, dirt, particles, microcracks, local depressions, and step structures in the 2.5D video frames output by the 2.5D video display module S9.

[0043] In one specific implementation, the 2.5D video real-time fusion computing module S8 is executed by the image processor M1, which includes one or more of GPU, FPGA, ASIC, DSP or edge computing board; the image processor M1 is used to perform image registration correction, DIC intensity-depth gradient mapping, depth restoration and 2.5D image synthesis in a pipelined parallel manner. While performing 2.5D fusion computing in the current frame, the optical imaging module S1 acquires the visible light texture image and DIC image of the next frame.

[0044] The human-machine interface module S10 includes one or more of an HMI touchscreen H1, function buttons, and status indicator units. The sample stage S12 is preferably one of a fixed sample stage S12, a two-dimensional moving platform, a vacuum adsorption platform, or an online detection platform connected to an industrial conveying mechanism. The data storage module S11 includes one or more storage media M2 of a hard disk, SSD, and flash memory. The imager also includes a power module and a heat dissipation component. The power module supplies power to the imager, and the heat dissipation component cools the image processor M1, the main controller, and the power module.

[0045] This invention also provides a real-time imaging method for a differential interferometric true-view 2.5D imager. The real-time imaging method is implemented based on the aforementioned differential interferometric true-view 2.5D imager and includes: S1. Visible light texture imaging and differential interference contrast imaging: The optical imaging module S1 illuminates the sample S13 under test, and the reflected imaging beam of the sample S13 under test is obtained by microscopic imaging. The second beam splitter P8 in the optical imaging module S1 divides the reflected imaging beam into the first optical path S2 and the second optical path S3. The first optical path S2 is used to acquire visible light texture images, and the second optical path S3 is used to acquire DIC images after differential interference processing. S2. The main control task management module S7 synchronously triggers exposure and synchronously reads the first optical path S2 and the second optical path S3 to obtain visible light texture images and DIC images with time correspondence within the same trigger cycle. S3. The visible light texture image and DIC image are registered and corrected in real time through the 2.5D video real-time fusion computing module S8. S4. Map the DIC image to the depth gradient information of the surface of the sample S13 under test, and recover the relative height distribution of the surface of the sample S13 under test based on the depth gradient information. Then, fuse the texture information of the visible light texture image with the relative height distribution of the surface of the sample S13 under test to generate a true-view 2.5D video frame. S5. The continuously generated true-to-life 2.5D video frames are output as a 2.5D video stream in chronological order through the 2.5D video display module S9.

[0046] In this embodiment, the imager employs a dual-optical-path structure during imaging, splitting the returned imaging beam into a visible light texture imaging path and a differential interference contrast imaging path. This allows for the simultaneous acquisition of texture information and gradient information generated by differential interference on the sample surface within the same device, providing a data foundation for real-time 2.5D imaging. Furthermore, it acquires some surface morphology-sensitive information upfront via differential interference contrast imaging and some information via visible light texture information. The backend uses deterministic real-time correction, mapping, and fusion calculations to fuse the visible light texture image and the DIC image from differential interference contrast imaging. This imaging method not only more intuitively presents fine defects such as scratches, dirt, particles, microcracks, local depressions, and step structures in 2.5D video frames but also improves the 2.5D image output speed. This enables the imager to quickly output 2.5D images that accurately represent both realistic surface texture and morphological variations, and further realizes 2.5D video stream output to meet the demands of industrial sites for online, non-contact, and low-latency inspection.

[0047] Furthermore, the imager adopts an optical non-contact imaging method, which does not require damaging the sample S13 under test; at the same time, it integrates the image sensor, main control task management module, image processor M1, human-computer interaction module S10 and data storage module S11.

[0048] In one embodiment, the 2.5D video real-time fusion calculation module S8 in step S4 converts the DIC image into depth gradient information using a pre-calibrated lookup table, linear mapping relationship, or piecewise mapping relationship, and recovers the relative height distribution of the surface of the sample S13 under test using at least one of integral reconstruction, optimization solution, or lookup table mapping. Furthermore, when fusing texture information and height information in step S4, at least one of pseudo-color mapping, transparency overlay, embossing rendering, and shadow enhancement is used to generate 2.5D video frames.

[0049] In one specific implementation, the visible light texture image and the DIC image are unified to a preset resolution before entering the fusion process, and 2.5D video frames are continuously generated using a pipelined image processing method so that the output frame rate of the 2.5D video stream at the preset resolution is not less than 30 frames / second. The preset resolution is set according to the detection accuracy requirements, preferably not less than 640×480.

[0050] like Figure 6As shown, the imager's exterior includes a chassis S4, an HMI touchscreen H1, a display screen D1, a power interface D2, and a sample loading port D3. The HMI touchscreen H1 is used for parameter setting, status display, and function control. The display screen D1 is used to display real-time 2.5D video streams and detection results. The power interface D2 is used for external power supply. The sample loading port D3 allows the sample stage S12 to enter the imaging area inside the chassis S4, or it can be used to connect to an industrial conveyor mechanism.

[0051] like Figure 7 As shown, the internal structure of the imager includes a chassis S4, an internal mounting space S5, a power module S6, a main controller C1, an image processor M1, a storage medium M2, and a sample stage S12. A display screen D1, a sample loading port D3, and an HMI touchscreen H1 are located on the front panel of the chassis S4. Figure 7 The internal structure is not shown in detail to highlight its importance. The internal installation space S5 is used to accommodate and fix various functional components, the power module S6 is used to supply power to the whole machine, the main controller C1 is used for system scheduling, the image processor M1 is used for real-time image calculation, and the storage medium M2 is used for storing images and parameters.

[0052] In one embodiment, a heat dissipation component may also be provided inside the chassis S4 to dissipate heat from the image processor M1, the main controller C1, and the power module S6, so as to ensure the stability of the device under continuous real-time operation conditions.

[0053] In another embodiment, the wavelength of the light source P1, the magnification of the objective lens, the magnification of the collimating lens group P2, the shearing amount of the Nomarski prism P10, the CMOS exposure time, and the image processing speed can be adjusted according to different application scenarios to achieve a balance between 2.5D accuracy and video output frame rate. For real-time inspection scenarios in industrial production lines, short exposure, high frame rate acquisition, and lightweight fusion algorithms are preferred to meet online inspection requirements.

[0054] In this invention, "True Vision 2.5D" refers to the generated 2.5D video frame that simultaneously contains real surface texture information provided by visible light texture images acquired within the same triggering period, as well as surface relative height or morphological undulation information obtained by mapping and reconstructing DIC images acquired within the same triggering period. The 2.5D video frame is used to express the texture and morphological undulation of the sample surface in real time in a video format, and does not require the absolute metrological accuracy of offline three-dimensional reconstruction or contact contour measurement.

[0055] Furthermore, the imager of the present invention is provided with a computer-readable storage medium M2, on which a computer program is stored. When the computer program is executed by the processor, it is used to control the aforementioned differential interferometric true vision 2.5D imager to perform at least one step in the imaging method, namely synchronous trigger reading control, image registration correction, DIC intensity-depth gradient mapping, depth map reconstruction, 2.5D image synthesis, and 2.5D video stream output.

[0056] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components; and they can also refer to a "transmission connection," that is, a power connection through various suitable methods such as belt drive, gear drive, or sprocket drive. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

Claims

1. A differential interferometric true-vision 2.5D imager, wherein the differential interferometric true-vision 2.5D imager is used for real-time imaging output of chip surface texture and morphology information, characterized in that, The differential interferometric true vision 2.5D imager includes: an optical imaging module, a main control task management module, a 2.5D video real-time fusion computing module, a 2.5D video display module, a human-computer interaction module, and a data storage module; The optical imaging module is used to perform visible light texture imaging and differential interference contrast imaging on the sample under test, so as to simultaneously acquire visible light texture images and DIC images of the same area under test. The main control task management module is connected to the optical imaging module to synchronously trigger and control the optical imaging module, and to schedule the image acquisition, image processing and video output processes. The 2.5D video real-time fusion computing module is connected to the main control task management module and is used to perform real-time registration and correction, DIC intensity-depth gradient mapping, depth map reconstruction and 2.5D image synthesis on the visible light texture image and the DIC image to generate continuous 2.5D video frames. The 2.5D video frames include a texture channel provided by the visible light texture image and a height channel obtained by mapping and reconstructing the DIC image. The 2.5D video display module is connected to the 2.5D video real-time fusion computing module and is used to receive and display continuous 2.5D video frames. The continuous 2.5D video frames are output as a 2.5D video stream in chronological order. The human-computer interaction module is used to receive parameter setting instructions and output device status information; The data storage module is used to store image data, calibration parameters, processing parameters, intermediate results, and output results.

2. The differential interferometric true-vision 2.5D imager according to claim 1, characterized in that, The optical imaging module includes a light source, a collimating lens group, a polarizer, a reflecting mirror group, a first beam splitter, a microscope objective, a second beam splitter, a first tube lens, a Nomarski prism, an analyzer, a second tube lens, a first image sensor, a second image sensor, and a sample stage. The sample stage is used to hold the sample to be tested. The unpolarized light emitted from the light source is collimated by the collimating lens group and enters the polarizer, where it is converted into linearly polarized light. The linearly polarized light enters the microscope objective through the mirror group and the first beam splitter and illuminates the surface of the sample to be tested. The light beam reflected or scattered back by the sample to be tested returns through the microscope objective and is then split into a first optical path and a second optical path by the second beam splitter. The light beam in the first optical path is captured by the first image sensor after passing through the first tube lens, forming a visible light texture image; The light beam in the second optical path is split into two beams with slight shear displacement and orthogonal polarization states after passing through the Nomarski prism. The two beams interfere in the polarization direction after passing through the analyzer, and are then acquired by the second image sensor after passing through the second tube lens to form a DIC image.

3. The differential interferometric true-vision 2.5D imager according to claim 2, characterized in that, The 2.5D video real-time fusion computing module includes an image registration and correction unit, a DIC intensity-depth gradient mapping unit, a depth map reconstruction unit, and a 2.5D image synthesis unit. The image registration and correction unit is used to perform geometric registration, distortion correction and scale unification on the visible light texture image and the DIC image. The DIC intensity-depth gradient mapping unit is used to establish the mapping relationship between the pixel intensity of the DIC image and the depth gradient of the surface of the sample under test along the shear direction, and to convert the pixel intensity in the DIC image into the depth gradient value of the surface of the sample under test along the shear direction. The depth map reconstruction unit is used to reconstruct the relative height distribution of the surface of the sample under test based on the depth gradient value; The 2.5D image synthesis unit is used to fuse the visible light texture image with the relative height distribution of the sample under test reconstructed by the depth map reconstruction unit to generate the 2.5D video frame.

4. The differential interferometric true-vision 2.5D imager according to claim 3, characterized in that, The depth map reconstruction unit includes at least one of the following methods to reconstruct the relative height distribution of the sample surface: discrete integral reconstruction, cumulative integral in row and column directions, fast Poisson reconstruction, lookup table mapping, or lightweight optimization solution.

5. The differential interferometric true-vision 2.5D imager according to claim 4, characterized in that, The 2.5D image synthesis unit uses the visible light texture image as the texture channel and the relative height distribution of the surface of the sample under test as the height channel to generate the 2.5D video frame through at least one or more of pseudo-color mapping, embossing enhancement, lighting simulation, and transparency overlay.

6. The differential interferometric true-vision 2.5D imager according to claim 5, characterized in that, The 2.5D video real-time fusion computing module is executed by an image processor, which includes one or more of a GPU, FPGA, ASIC, DSP, or edge computing board; the image processor is used to perform image registration correction, DIC intensity-depth gradient mapping, depth restoration, and 2.5D image synthesis in a pipelined parallel manner.

7. A real-time imaging method for a differential interferometric true-vision 2.5D imager, characterized in that, The real-time imaging method is implemented using the differential interferometric true-vision 2.5D imager according to any one of claims 1-6, and the real-time imaging method includes: S1. Visible light texture imaging and differential interference contrast imaging: The sample to be tested is illuminated by an optical imaging module, and the reflected imaging beam of the sample to be tested is obtained by microscopic imaging. The second beam splitter in the optical imaging module divides the reflected imaging beam into a first optical path and a second optical path. The first optical path is used to acquire visible light texture images, and the second optical path is used to acquire DIC images after differential interference processing. S2. The first and second optical paths are synchronously triggered for exposure and synchronous reading through the main control task management module to obtain the visible light texture image and DIC image with time correspondence within the same trigger period. S3. The visible light texture image and DIC image are registered and corrected in real time using the 2.5D video real-time fusion computing module; S4. Map the DIC image to the depth gradient information of the sample surface under test, and recover the relative height distribution of the sample surface under test based on the depth gradient information. Then, fuse the texture information of the visible light texture image with the relative height distribution of the sample surface under test to generate a true-view 2.5D video frame. S5. The continuously generated true-to-life 2.5D video frames are output as a 2.5D video stream in chronological order through the 2.5D video display module.

8. The real-time imaging method according to claim 7, characterized in that, In step S4, the 2.5D video real-time fusion computing module converts the DIC image into depth gradient information through a pre-calibrated lookup table, linear mapping relationship, or piecewise mapping relationship, and recovers the relative height distribution of the surface of the sample under test by at least one of integral reconstruction, optimization solution, or lookup table mapping.

9. The real-time imaging method according to claim 8, characterized in that, In step S4, when fusing the texture information and height information, at least one of the following methods is used to generate the 2.5D video frame: pseudo-color mapping, transparency overlay, emboss rendering, and shadow enhancement.

10. The real-time imaging method according to claim 9, characterized in that, Before entering the fusion process, the visible light texture image and the DIC image are unified to a preset resolution, and 2.5D video frames are continuously generated using a pipelined image processing method so that the output frame rate of the 2.5D video stream at the preset resolution is not less than 30 frames / second.

Images