Super-depth-of-field imaging system and imaging method based on liquid lens
Through the combination of liquid lenses and automatic focus sensors, high-speed focus and panoramic depth imaging of micro-optical imaging systems are achieved, solving the problems of mechanical delay and depth of field limitations, and are suitable for industrial detection, medical imaging and smartphones.
Patent Information
- Application Number
- CN202510807164.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-09-02
AI Technical Summary
There are problems in micro-optical imaging systems where high-speed focus needs are difficult to meet due to mechanical delay, imaging time-consuming due to fixed depth of field limitation, and complex mechanical structures are difficult to miniaturize.
A liquid lens combined with an automatic focus sensor is used to drive the liquid lens through dynamic electrical signals to achieve millisecond-level focal length switching, and a panoramic depth image is obtained by combining an image synthesis algorithm.
High-speed focus and panoramic depth imaging are achieved, avoiding the delay and complexity caused by mechanical movement, and are suitable for industrial detection, medical imaging and smartphones.
Smart Images
Figure CN120577949A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of microscopic optical imaging technology, and in particular to a liquid lens-based ultra-depth-of-field imaging system and imaging method. Background Art
[0002] At present, in microscopic optical imaging technology and systems, clearly focused, high-quality images can not only intuitively display the sample morphology, but also serve as the basis and guarantee for many analyses such as sample identification and classification based on morphological characteristics. However, the depth of field of high-magnification microscopic optical systems is small (about 1-10μm), and it is easy for the image to be out of focus due to mechanical micro-vibration, environmental conditions, etc. With the continuous improvement of the degree of automation and the demand for clear imaging efficiency, higher requirements are placed on the system's automatic focusing. However, there are still many problems with microscopic optical imaging systems, specifically:
[0003] Traditional mechanical focus system: relies on a motor to drive the lens movement, has mechanical delay problems (such as typical response time > 50ms), and cannot meet high-speed focusing requirements (such as sports scenes, etc.).
[0004] Fixed depth of field limitation: A single image can only capture a limited depth of field range, requiring multiple shots and software synthesis (taking > 1 second), making real-time application difficult.
[0005] Size and cost: The mechanical structure is complex and difficult to miniaturize (such as mobile phone cameras).
[0006] Therefore, how to improve the dynamic autofocus performance of the microscopic optical imaging system and capture clear panoramic depth images of the sample under the lens is a technical problem that needs to be urgently solved in microscopic optical imaging. Summary of the Invention
[0007] Based on the above-mentioned defects of the existing technology, the present application provides an ultra-depth-of-field imaging system and imaging method based on a liquid lens, which can generate a dynamic electrical signal according to the feedback data of the autofocus sensor, drive the liquid lens to achieve millisecond-level focal length switching, realize high-speed focusing and capture a clear full-depth image of the sample under the objective lens.
[0008] In a first aspect, an embodiment of the present application provides an ultra-depth-of-field imaging system based on a liquid lens, comprising:
[0009] A light splitting module, used to split the light reflected by the sample into a main light path and a light splitting path;
[0010] An imaging module, used to obtain a full-depth image corresponding to the main light path;
[0011] An autofocus module, comprising a liquid lens and an autofocus sensor, wherein the liquid lens is coaxially arranged on the main optical path between the imaging module and the light splitting module, and the autofocus sensor is used to acquire light on the light splitting path and calculate the defocus amount of the sample;
[0012] The control module is used to store the calibration mapping relationship between the driving parameters of the liquid lens and the defocus amount of the autofocus sensor, and is also used to call the mapping relationship table to adjust the driving parameters of the liquid lens according to the defocus amount of the sample at the current moment, so as to achieve high-speed focusing and full-depth imaging of the imaging module.
[0013] As a preferred solution, the calibration mapping relationship is established by the following steps:
[0014] Adjust the driving parameters of the liquid lens to return the defocus of the calibration sample to zero, record the driving parameters at the current moment, adjust the driving parameters according to the preset defocus range, and obtain a calibration mapping relationship between discrete or continuous driving parameters and defocus.
[0015] As a preferred solution, the autofocus sensor is active and includes a laser ranging element.
[0016] As a preferred solution, the layout of the auto-focus sensor includes coaxial or non-coaxial.
[0017] As a preferred solution, the liquid lens is an electrowetting effect liquid lens, and the driving mode is voltage control.
[0018] As a preferred solution, the liquid lens is a gradient refractive index liquid lens, and the driving mode is current control or temperature regulation refractive index distribution control.
[0019] As a preferred solution, the correspondence between the autofocus sensor and the liquid lens is achieved through pre-calibration, the driving signal of the liquid lens is generated by a high-frequency control circuit, and the focusing response time is less than 5ms.
[0020] As a preferred solution, the light splitting module includes a coaxially arranged objective lens and a beam splitter; or a coaxially arranged objective lens, a beam splitter and a lens barrel.
[0021] In a second aspect, the present application further provides a liquid lens-based super-depth-of-field imaging method, which uses the super-depth-of-field imaging system provided in the first aspect, including:
[0022] Placing the calibration sample in the imaging light path to establish a calibration mapping relationship between the driving parameters of the liquid lens and the defocus amount of the autofocus sensor;
[0023] After the sample to be tested is replaced, the autofocus sensor detects the defocus amount of the sample to be tested in real time;
[0024] The control module queries the calibration mapping relationship according to the detected defocus amount and outputs corresponding driving parameters to the liquid lens;
[0025] The liquid lens adjusts the focal length according to the driving parameters so that the sample is imaged on the focal plane of the imaging module;
[0026] After the sample is focused, the control module continuously adjusts the driving parameters of the liquid lens according to the preset defocus range, and the imaging module collects multiple images under different defocus states;
[0027] The control module combines the multiple images into a panoramic depth image.
[0028] As a preferred solution, image synthesis adopts an image superposition algorithm or a multi-focus image fusion algorithm.
[0029] In summary, the present application provides a liquid lens-based super-depth-of-field imaging system, which includes a spectroscopic module, an imaging module, and an autofocus module. The spectroscopic module is used to split the light reflected by the sample into a main light path and a spectroscopic path. The imaging module is used to obtain a full-depth image corresponding to the main light path. The autofocus module includes a liquid lens and an autofocus sensor. The liquid lens is located between the imaging module and the spectroscopic module and is coaxial. The autofocus sensor is used to obtain light on the spectroscopic path and calculate the defocus amount of the sample. The control module is used to store the calibration mapping relationship between the driving parameters of the liquid lens and the defocus amount of the autofocus sensor, and is also used to call the mapping relationship table according to the defocus amount of the sample at the current moment to adjust the driving parameters of the liquid lens, thereby realizing high-speed focusing and full-depth imaging of the imaging module. The present application can generate a dynamic electrical signal based on the feedback data of the autofocus sensor, drive the liquid lens to achieve millisecond-level focal length switching, achieve high-speed focusing, and capture a clear full-depth image of the sample. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A schematic diagram of a module of a liquid lens-based ultra-depth-of-field imaging system provided in this application;
[0031] Figure 2 A schematic structural diagram of a liquid lens-based ultra-depth-of-field imaging system provided in this application;
[0032] Figure 3 A schematic structural diagram of another liquid lens-based ultra-depth-of-field imaging system provided in this application;
[0033] Figure 4 A schematic structural diagram of another liquid lens-based ultra-depth-of-field imaging system provided in this application;
[0034] Figure 5 A schematic diagram of a liquid lens-based ultra-depth-of-field imaging method provided in this application;
[0035] Figure 6 This is a flow chart of the calibration process for the relationship between the liquid lens voltage and the defocus amount of the focus sensor in this application;
[0036] In the picture:
[0037] 10. Spectral module; 20. Imaging module; 30. Autofocus module; 40. Control module; 11. Objective lens; 12. Spectral mirror; 13. Lens barrel; 31. Liquid lens; 32. Autofocus sensor. DETAILED DESCRIPTION
[0038] The present application will be described in further detail below in conjunction with the accompanying drawings and examples. It will be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only portions, rather than all, of the structures relevant to the present application are shown in the accompanying drawings. Various modifications and variations can be made in the present application without departing from the spirit or scope of the present application, which will be apparent to those skilled in the art. Therefore, the present application is intended to cover modifications and variations of the present application that fall within the scope of the corresponding claims (technical solutions claimed for protection) and their equivalents. It should be noted that the embodiments provided in the present application examples can be combined with each other without contradiction.
[0039] Figure 1 This is a schematic diagram of a module of a liquid lens-based ultra-depth-of-field imaging system provided in this application. Figure 2 This is a schematic diagram of the structure of a liquid lens-based ultra-depth-of-field imaging system provided in this application. Figure 3 This is a schematic diagram of the structure of another liquid lens-based ultra-depth of field imaging system provided in this application. Figure 4 This is a schematic diagram of another liquid lens-based super-depth imaging system provided in this application, refer to Figure 1-Figure 4 The liquid lens-based super-depth of field imaging system provided in the embodiment of the present application includes a spectroscopic module 10, an imaging module 20, an autofocus module 30 and a control module 40. The spectroscopic module 10 is used to split the light reflected by the sample into a main light path and a sub-light path. The imaging module 20 is used to obtain a full-depth image corresponding to the main light path. The autofocus module 30 includes a liquid lens 31 and an autofocus sensor 32. The liquid lens 31 is located on the main light path between the imaging module 20 and the spectroscopic module 10 and is coaxially arranged. The autofocus sensor 32 is used to obtain light on the sub-light path and calculate the defocus amount of the sample. The control module 40 is used to store the calibration mapping relationship between the driving parameters of the liquid lens 31 and the defocus amount of the autofocus sensor 32, and is also used to call the mapping relationship table according to the defocus amount of the sample at the current moment to adjust the driving parameters of the liquid lens 31, so as to achieve high-speed focusing and full-depth imaging of the imaging module 20.
[0040] Specifically, refer to Figure 1 The control module 40 may be implemented as an integrated circuit (IC), a processor, or a microprocessor, such as a central processing unit (CPU). It may also be implemented as another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The control module 40 stores a control program for controlling the imaging module 20 and the autofocus module 30.
[0041] Combine Figure 2-Figure 3 As shown, the spectroscopic module 10 is arranged on the imaging optical path of the sample M, and the spectroscopic module 10 may include a coaxially arranged objective lens 11 and a spectroscope 12; Figure 4 The spectroscopic module 10 includes an objective lens 11 , a spectroscope 12 and a lens barrel 13 which are coaxially arranged.
[0042] The spectrometer 12 may be: Figure 2 and Figure 4 The cube beamsplitter shown can also be Figure 3 The coated beam splitter shown transmits part of the light reflected from the sample M to form a main light path, and reflects part of the light to form a split light path. The lens barrel 13 may be an optical lens, an optical element with focusing and collimating functions.
[0043] The imaging module 20 is installed on the main light path of the sample M and is connected to the control module 40 to record the image of the sample M. The imaging module 20 can be a high-resolution image sensor, such as a CMOS color camera, a CCD color camera, a standard RGB color camera, etc.
[0044] The autofocus module 30 includes a liquid lens 31 and an autofocus sensor 32. The liquid lens 31 is arranged on the main optical path between the imaging module 20 and the spectroscopic module 10 and is coaxially arranged. The layout of the autofocus sensor 32 includes coaxial or non-coaxial arrangements, for example Figure 2-Figure 4 In the embodiment, the autofocus sensor 32 is coaxially arranged on the light splitting path to obtain the light on the light splitting path and calculate the defocus amount of the sample M.
[0045] Among them, liquid lens is a dynamic focusing technology based on liquid optical principles. It achieves rapid zoom and focus by changing the shape or refractive index of the liquid. It has the advantages of fast response speed, compact structure and low power consumption.
[0046] In the embodiment of the present application, the liquid lens 31 may be an electrowetting effect liquid lens 31, driven by voltage control. Specifically, the liquid lens provided herein utilizes the electrowetting effect. A high-frequency circuit in the control module 40 applies a voltage to the liquid lens 31 to change the contact angle between the liquid and the solid surface, thereby adjusting the curvature of the liquid droplet and achieving a change in focal length.
[0047] In the embodiment of the present application, the liquid lens 31 may be a gradient refractive index liquid lens 31, driven by current control or temperature-regulated refractive index distribution control. Specifically, thermal gradient control may be employed, whereby temperature changes induce changes in the liquid's refractive index to achieve focal length variation. Concentration gradient control may also be employed, whereby the mixing ratio of two liquids with different refractive indices is adjusted to achieve a gradual refractive index change, thereby achieving focal length variation.
[0048] In other embodiments, the liquid lens provided in the present application adopts hydraulic / mechanical drive, and uses external pressure (such as piezoelectric drive or microfluidic control) to change the curvature radius of the liquid lens and adjust the optical focal length.
[0049] Among them, the auto-focus sensor 32 (Auto-Focus Sensor, AFS) is a key component in the optical imaging system of the present application, which is used to detect the defocus amount (Defocus Amount) of the target sample in real time and feed it back to the control module 40 to adjust the focal length of the liquid lens 31, thereby achieving fast and accurate focusing.
[0050] refer to Figure 2-Figure 4 The autofocus sensor 32 can be an active type, including a laser ranging element, with a wavelength range of visible light, infrared light, or ultrasonic light. Specifically, the autofocus sensor 32 transmits visible light, infrared light, or ultrasonic light toward the sample M in the reverse direction of the split beam path. By receiving the reflected signal from the sample M, the distance is calculated and the defocus value of the sample M is determined. This configuration is not affected by ambient light and is suitable for low-contrast scenarios, such as industrial inspection and medical endoscopy.
[0051] refer to Figure 2-Figure 4 The autofocus sensor 32 can also be a passive type. The autofocus sensor 32 obtains the detection light from the split light path through the spectroscope 12, calculates the distance by receiving the reflection signal of the target sample M, and determines the defocus amount of the sample M.
[0052] In the ultra-depth-of-field imaging system of the present invention, the core functions of the autofocus sensor are:
[0053] 1. Detect defocus in real time and measure the deviation between the target sample and the ideal focal plane.
[0054] 2. The liquid lens is controlled by a feedback drive signal. According to the defocus amount, the control circuit in the control module can quickly adjust the drive voltage of the liquid lens to achieve millisecond-level focusing (<5ms).
[0055] 3. Supports full-depth imaging. By fine-tuning the driving voltage of the liquid lens, the imaging module collects images at different focal planes and ultimately synthesizes a full-definition image.
[0056] This application combines an autofocus sensor with a liquid lens to achieve millisecond-level dynamic focus + panoramic depth imaging, and is suitable for industrial inspection, medical imaging, smartphones and other fields.
[0057] Based on the same inventive concept, an embodiment of the present invention provides a method for ultra-depth-of-field imaging based on a liquid lens. Figure 5 This is a schematic diagram of a liquid lens-based super-depth imaging method provided in this application, Figure 2-Figure 5 , the ultra-depth-of-field imaging method based on liquid lens includes the following steps:
[0058] S101: Place a calibration sample in the imaging optical path and establish a calibration mapping relationship between the liquid lens drive parameters and the defocus value of the autofocus sensor. The liquid lens drive parameters include voltage or current. Specifically, using voltage control as an example, when establishing the defocus value-voltage calibration mapping relationship, a dense sampling interval is used near zero defocus value to improve the number of calibrations and focus accuracy.
[0059] In this embodiment of the present application, the correspondence between the autofocus sensor 32 and the liquid lens 31 is achieved through pre-calibration. The drive signal for the liquid lens 31 is generated by a high-frequency control circuit, and the focus response time is less than 5ms. The response time for adjusting the drive voltage of the liquid lens is less than 5ms, achieving millisecond-level dynamic focus.
[0060] Figure 6 For the calibration process flow chart of the relationship between the liquid lens voltage and the defocus amount of the focus sensor in this application, refer to Figure 6 In the embodiment of the present application, the relationship between the liquid lens voltage and the defocus amount of the focus sensor is calibrated and mapped by the following steps:
[0061] Adjust the driving parameters of the liquid lens to return the defocus of the calibration sample to zero, record the driving parameters at the current moment, adjust the driving parameters according to the preset defocus range, and obtain a calibration mapping relationship between discrete or continuous driving parameters and defocus.
[0062] Specifically, refer to Figure 4 The driving mode of the liquid lens 31 is voltage control. The specific calibration steps are:
[0063] Step S1: Place the calibration sample in the imaging light path. The imaging light path includes the objective lens 11, the beam splitter 12, the tube lens 13 and the liquid lens 31 in sequence.
[0064] Step S2: The auto-focus sensor 32 detects the defocus amount of the calibration sample M.
[0065] Step S3: Determine whether the image is out of focus.
[0066] Step S4: If yes, adjust the liquid lens voltage;
[0067] Step S5: If no, record the corresponding relationship between the voltage and the defocus amount.
[0068] When the defocus amount is 0, it indicates that the lens is in focus, and the voltage value of the liquid lens 21 at this time is recorded.
[0069] When the defocus amount is not zero, the control module 40 adjusts the driving voltage of the liquid lens 31 until the defocus amount is zero, and records the voltage value of the liquid lens 31 at this time.
[0070] Repeat multiple times to record the driving voltage values of the liquid lens 31 corresponding to different defocus amounts, establish a calibration mapping relationship between defocus amount and voltage, and store it in the control circuit of the control module 40.
[0071] S102: After the sample to be tested is replaced, the autofocus sensor detects the defocus amount of the sample in real time.
[0072] Specifically, refer to Figure 2-Figure 4 , the sample M to be tested is placed in the imaging light path, and the autofocus sensor 32 detects the defocus amount of the sample M in real time and feeds back to the control module 40.
[0073] S103 : The control module queries the calibration mapping relationship according to the detected defocus amount, and outputs corresponding driving parameters to the liquid lens.
[0074] S104 , the liquid lens adjusts the focal length according to the driving parameters so that the sample is imaged on the focal plane of the imaging module.
[0075] Specifically, refer to Figure 2-Figure 4 The control module 40 queries the calibration mapping relationship based on the detected defocus amount, and finds the driving voltage corresponding to the defocus amount. The high-frequency circuit in the control module 40 applies voltage to the liquid lens 31 to change the contact angle between the liquid and the solid surface, thereby adjusting the curvature of the droplet, realizing the focal length change, and adjusting the defocus amount of the sample to be tested to zero, that is, located in the focal plane.
[0076] S105 , after the sample is focused, the control module continuously adjusts the driving parameters of the liquid lens according to a preset defocus range, and the imaging module collects multiple images in different defocus states.
[0077] The preset defocus range includes positive defocus and negative defocus.
[0078] S106: The control module combines the multiple images into a full-depth image.
[0079] In the embodiment of the present application, the image synthesis in step S106 adopts an image superposition algorithm or a multi-focus image fusion algorithm.
[0080] Image synthesis refers to combining multiple images (of different focal lengths) into a single high-quality image. The image synthesis solution in this application includes two implementation methods: a synthesis algorithm based on direct superposition and a multi-focus fusion algorithm based on regional clarity analysis.
[0081] Image overlay algorithms combine multiple images according to certain weights (such as transparency and exposure) to generate a fused image. Common methods include alpha blending, weighted averaging, and Laplacian pyramid blending.
[0082] Multi-focus image fusion algorithm: Due to optical focal length limitations, a single image may not be able to make all areas clear. The multi-focus fusion algorithm analyzes multiple images with different focal points, selects the clear parts of each image, and synthesizes them to obtain a clear full-depth image of the sample.
[0083] For example, if the measurement sample is a silicon wafer, a circular thin slice made of high-purity single crystal silicon (or polycrystalline silicon), is an important device for semiconductor devices. Common diameters are 6 inches (150mm), 8 inches (200mm), 12 inches (300mm), etc., and the thickness is usually several hundred microns, for example, a 12-inch silicon wafer is about 775μm. At present, due to the limitations of the polishing process and the characteristics of the silicon wafer itself, its surface thickness may be uneven or there may be local fluctuations. The present application can quickly focus to a certain position of the silicon wafer by controlling the autofocus sensor 32 and the liquid lens 31. The imaging module 20 collects the silicon wafer morphology at the focal length, and then dynamically combines the upper and lower defocusing by the autofocus sensor 32 and the liquid lens 31 to obtain multiple defocused slices of the position, and then synthesizes them into a 3D image through an image superposition algorithm or a multi-focus image fusion algorithm, thereby quickly acquiring the silicon wafer morphology.
[0084] In summary, this application uses an electrically controlled liquid lens and a calibration mapping relationship, achieving a focus time of <5ms. This optical system, with no mechanical motion, avoids motor wear and improves system life. It also offers strong compatibility, supporting coaxial / non-coaxial, active / passive autofocus sensors, adapting to various application scenarios, ultimately achieving clear, full-depth images of the sample.
[0085] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and the features of the various embodiments of the present invention may be partially or completely coupled or combined with each other, and may cooperate with each other in various ways and be technically driven. It is possible for those skilled in the art to make various obvious changes, readjustments, combinations and substitutions without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A liquid lens-based ultra-depth-of-field imaging system, characterized in that: include: A light splitting module, used to split the light reflected by the sample into a main light path and a light splitting path; An imaging module, used to obtain a full-depth image corresponding to the main light path; An autofocus module, comprising a liquid lens and an autofocus sensor, wherein the liquid lens is coaxially arranged on the main optical path between the imaging module and the light splitting module, and the autofocus sensor is used to acquire light on the light splitting path and calculate the defocus amount of the sample; The control module is used to store the calibration mapping relationship between the driving parameters of the liquid lens and the defocus amount of the autofocus sensor, and is also used to call the mapping relationship table to adjust the driving parameters of the liquid lens according to the defocus amount of the sample at the current moment, so as to achieve high-speed focusing and full-depth imaging of the imaging module.
2. The system according to claim 1, wherein: The calibration mapping relationship is established by the following steps: Adjust the driving parameters of the liquid lens to return the defocus of the calibration sample to zero, record the driving parameters at the current moment, adjust the driving parameters according to the preset defocus range, and obtain a calibration mapping relationship between discrete or continuous driving parameters and defocus.
3. The system according to claim 1, wherein: The autofocus sensor is active and includes a laser ranging element.
4. The system according to claim 1, wherein: The layout of the auto-focus sensor includes coaxial or non-coaxial.
5. The system according to claim 1, wherein: The liquid lens is an electrowetting effect liquid lens, and the driving mode is voltage control.
6. The system according to claim 1, wherein: The liquid lens is a gradient refractive index liquid lens, and the driving mode is current control or temperature regulation refractive index distribution control.
7. The system according to claim 1, wherein: The correspondence between the autofocus sensor and the liquid lens is achieved through pre-calibration. The driving signal of the liquid lens is generated by a high-frequency control circuit, and the focusing response time is less than 5ms.
8. The system according to claim 1, wherein: The light splitting module includes a coaxially arranged objective lens and a beam splitter; or a coaxially arranged objective lens, a beam splitter and a lens barrel.
9. A liquid lens-based super-depth-of-field imaging method, using the super-depth-of-field imaging system according to any one of claims 1 to 8, characterized in that: include: Placing the calibration sample in the imaging light path to establish a calibration mapping relationship between the driving parameters of the liquid lens and the defocus amount of the autofocus sensor; After the sample to be tested is replaced, the autofocus sensor detects the defocus amount of the sample to be tested in real time; The control module queries the calibration mapping relationship according to the detected defocus amount and outputs corresponding driving parameters to the liquid lens; The liquid lens adjusts the focal length according to the driving parameters so that the sample is imaged on the focal plane of the imaging module; After the sample is focused, the control module continuously adjusts the driving parameters of the liquid lens according to the preset defocus range, and the imaging module collects multiple images under different defocus states; The control module combines the multiple images into a full-depth image.
10. The super depth of field imaging method according to claim 9, characterized in that: Image synthesis uses an image superposition algorithm or a multi-focus image fusion algorithm.