Liquid lens-based auto focus zoom lens system and method

Abstract

The application discloses an autofocus zoom lens system and method based on a liquid lens, and belongs to the technical field of optics, and comprises: an image acquisition module, which is used for acquiring continuous image frames of a target; a deep learning target trajectory prediction module, which is in communication connection with the image acquisition module and is used for target identification, tracking and prediction of the position of the target at a future time, and output of an ideal focal length; an electrowetting liquid lens, which is internally filled with two immiscible liquids of electric conduction and insulation, and has no mechanical moving parts; and an electrowetting liquid lens control system, which is in communication connection with the deep learning target trajectory prediction module and is used for output of a control voltage according to the ideal focal length, and realization of electric control adjustment of the focal length of the electrowetting liquid lens by changing the liquid-liquid interface contact angle and the curvature radius based on the electrowetting effect.The application has the advantages of compact structure, rapid response, high system robustness and the like.

Description

Technical Field

[0001] This invention relates to the field of optical technology, and more specifically to an autofocus zoom lens system and method based on liquid lenses. Background Technology

[0002] With the rapid development of industrial automation, intelligent vision systems, drones, and intelligent security, increasingly higher demands are being placed on the autofocus and zoom capabilities of lens systems. Traditional optical focusing systems typically rely on the movement of mechanical components (such as moving lens groups) to achieve focusing or zooming. While this method has achieved relatively mature applications in most scenarios, it suffers from slow response speed, high energy consumption, and complex structure, exhibiting significant lag, especially in applications requiring high-speed response (such as industrial production lines and dynamic target recognition in drones). Furthermore, the lifespan of traditional systems is often limited by mechanical wear, and their reliability and size restrict their deployment in certain demanding scenarios.

[0003] Liquid lenses are an emerging type of optical element whose optical focal length can be adjusted by controlling the surface curvature of the liquid. Electrowetting liquid lenses are miniature liquid lens systems based on the electrowetting on dielectric (EVO) effect. Their core principle lies in adjusting the contact angle of the liquid interface by applying an external voltage, thereby changing the lens curvature in real time and achieving rapid adjustment of the optical focal length. These lenses require no moving mechanical parts and possess microsecond to millisecond-level response speeds, making them particularly suitable for space-constrained applications with extremely high response speed requirements.

[0004] Based on this, the present invention designs an autofocus zoom lens system and method based on a liquid lens to solve the above problems. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides an autofocus zoom lens system and method based on liquid lens.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The autofocus zoom lens system based on liquid lenses includes:

[0008] Image acquisition module: used to acquire continuous image frames of the target;

[0009] Deep learning target trajectory prediction module: It communicates with the image acquisition module and is used to identify, track and predict the future position of the target from the acquired continuous image frames, and output the ideal focal length.

[0010] Electrowetting liquid lens: The interior is filled with two immiscible liquids, one conductive and one insulating, and there are no moving mechanical parts;

[0011] Electrowetting liquid lens control system: It communicates with the deep learning target trajectory prediction module and is used to output control voltage according to the ideal focal length. Based on the electrowetting effect, it realizes the electronic control adjustment of the focal length of the electrowetting liquid lens by changing the contact angle and radius of curvature of the liquid-liquid interface.

[0012] Feedback adjustment module: It is connected to the image acquisition module and the electrowetting liquid lens control system respectively. It is used to obtain the image blur index by machine vision algorithm on the image frames acquired by the image acquisition module, and then perform closed-loop optimization adjustment of the control voltage based on the image blur index.

[0013] A method of using a liquid lens-based autofocus zoom lens system includes the following steps:

[0014] Step 1: The image acquisition module acquires continuous image frames of the target;

[0015] Step 2: The deep learning target trajectory prediction module calls the target detection network to preprocess, identify and track the acquired continuous image frames to obtain the continuous trajectory information of the target. At the same time, it extracts the spatial feature vector of the image frame through the convolutional neural network (CNN), fuses the continuous trajectory information of the target with the spatial feature vector, and then organizes it into a historical fusion feature time sequence according to the time frame.

[0016] Step 3: Input the continuous image frame time sequence into the Long Short-Term Memory (LSTM) network to capture the dynamic change trend of the target's motion over time and predict the target's position at future times;

[0017] Step 4: Calculate the ideal focal length based on the predicted future position of the target using a preset depth-of-field function, and transmit the ideal focal length data to the electrowetting liquid lens control system.

[0018] Step 5: The electrowetting liquid lens control system is based on the built-in voltage-focal length third-order polynomial mapping model. It calculates and outputs the appropriate control voltage according to the ideal focal length. Based on the electrowetting effect, it changes the contact angle of the liquid-liquid interface in the electrowetting liquid lens, which is converted into a change in the radius of curvature of the interface, thereby realizing the electronic control adjustment of the focal length of the electrowetting liquid lens.

[0019] Step 6: The machine vision algorithm obtains the image blur index of the image frame acquired by the image acquisition module. The image blur index participates in the closed-loop regulation of the control voltage to further optimize the voltage control.

[0020] Furthermore, step 2 is performed as follows:

[0021] Step 21: Receive continuous image frames transmitted by the image acquisition module in real time, and serialize and number the image frames according to their timestamps to obtain the image frame sequence. ;

[0022] Step 22: Image Frame Sequence The image frames within the range are sequentially subjected to size normalization, pixel normalization, and lightweight noise reduction preprocessing;

[0023] Step 23: Input the preprocessed image frame into the trained YOLOv8 lightweight detection network. The network's convolutional layers, pooling layers, and decoupling head complete feature extraction and target prediction, and output the target position of a single frame.

[0024] Step 24: Use Kalman filtering to predict the target position, match the target position detected by YOLOv8 with the target position predicted by Kalman filtering, calculate the matching cost through the Hungarian algorithm, accurately match targets across frames, realize cross-frame association of the same target, and update the continuous trajectory information of the target.

[0025] Step 25: Perform multi-scale spatial feature extraction on the preprocessed image frames using a convolutional neural network (CNN) to obtain spatial feature vectors;

[0026] Step 26: The target continuous trajectory information and spatial feature vector are fused, and the fused features are organized into a historical fused feature time sequence according to time frames.

[0027] Furthermore, step 3 is performed as follows:

[0028] Step 31: Sequence of consecutive image frames The corresponding historical fusion feature sequences are used to construct the LSTM-normalized time-series input sequence;

[0029] Step 32: Forward inference using the LSTM network to capture the temporal dynamic features of the LSTM-normalized temporal input sequence;

[0030] Step 33: Establish a mathematical model of target motion based on temporal dynamic characteristics, according to the preset prediction time step. Calculate the target's pixel coordinates / physical coordinates at future time points, remove outlier drift values, and predict the target's future position. .

[0031] Furthermore, predicting the target's position at future moments. Calculation formula:

[0032]

[0033] in, This is a combined deep trajectory prediction model consisting of a convolutional neural network (CNN) and a long short-term memory network (LSTM). This refers to the optimal set of weight parameters obtained after training the deep network model. Counting backwards from the current moment Standardized image frames This is the standardized image frame at the current moment.

[0034] Furthermore, the future moment Ideal focal length The specific calculations are as follows:

[0035]

[0036] in, For the target depth model, It is a pre-calibrated depth-of-field function used to establish the mapping relationship between the target position and the ideal focal length.

[0037] Furthermore, the specific steps in step 5 are as follows:

[0038] Step 51: Based on the ideal focal length output in Step 4 The target control voltage is solved using a voltage-focal length third-order polynomial mapping model. ;

[0039] Step 52: Based on the electrowetting effect, change the contact angle of the liquid-liquid interface inside the electrowetting liquid lens, which is converted into a change in the radius of curvature of the interface, to realize the electronically controlled adjustment of the focal length of the electrowetting liquid lens.

[0040] Furthermore, the target control voltage ,satisfy:

[0041] .

[0042] Furthermore, step 6 is performed as follows:

[0043] Step 61: Image frames transmitted by the image acquisition module Perform grayscale conversion to transform a color image into a grayscale image. ;

[0044] Step 62: Apply a 3×3 Laplacian convolution kernel to the grayscale image. Perform convolution filtering to extract high-frequency features from the image. ;

[0045] Step 63: Process the high-frequency feature map The variance of all pixel values ​​is calculated and used as an image blur index for the current frame. ;

[0046] Step 64: Quantize the fuzzyness index The data is transmitted to the electrowetting liquid lens control system. The control system combines the ideal focal length deviation predicted by deep learning with the image blur feedback to construct a dual-adjustment closed-loop control law and update the control voltage in real time.

[0047] Furthermore, the specific operations of step 64 are as follows:

[0048] Step 641: Using ambiguity index The feedback is given to the initial solution of the ideal focal length to the control voltage in step 5;

[0049] Step 642: Based on the control voltage of the previous moment Current frame blur index Ideal focal length Compared to the actual focal length The control voltage for the next time step is updated using the following closed-loop control formula. ;

[0050] Step 643: Construct a closed-loop control law with dual adjustment terms and adopt a dual-term collaborative mechanism of "forward-looking pre-correction + real-time feedback correction" for correction processing.

[0051] Beneficial Effects: This invention precisely controls the voltage to adjust the interface curvature of the two liquids inside the lens, thereby changing the focal length of the optical system in real time without any mechanical moving structure, achieving millisecond-level focusing. It employs a convolutional neural network (CNN) combined with a long short-term memory network (LSTM) to predict the target's motion trajectory and adjust the liquid lens voltage in real time to pre-focus, improving the image clarity of high-speed moving targets. Integrating the liquid lens, traditional optical lens, and control circuitry into a miniature optical platform offers advantages such as compact structure, rapid response, and high system robustness. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0053] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0055] The present invention will be further described below with reference to embodiments.

[0056] Example 1: An autofocus zoom lens system based on a liquid lens, comprising:

[0057] Image acquisition module: used to acquire continuous image frames of the target;

[0058] Deep learning target trajectory prediction module: It communicates with the image acquisition module and is used to identify, track and predict the future position of the target from the acquired continuous image frames, and output the ideal focal length.

[0059] Electrowetting liquid lens: The interior is filled with two immiscible liquids, one conductive and one insulating, and there are no moving mechanical parts;

[0060] Electrowetting liquid lens control system: It communicates with the deep learning target trajectory prediction module and is used to output control voltage according to the ideal focal length. Based on the electrowetting effect, it realizes the electronic control adjustment of the focal length of the electrowetting liquid lens by changing the contact angle and radius of curvature of the liquid-liquid interface.

[0061] Feedback adjustment module: It is connected to the image acquisition module and the electrowetting liquid lens control system respectively. It is used to obtain the image blur index by machine vision algorithm on the image frames acquired by the image acquisition module, and then perform closed-loop optimization adjustment of the control voltage based on the image blur index.

[0062] This invention achieves millisecond-level focusing by precisely controlling the voltage to adjust the interface curvature of two liquids inside the lens, thereby changing the focal length of the optical system in real time without any mechanical moving structures. It employs a convolutional neural network (CNN) combined with a long short-term memory network (LSTM) to predict the target's trajectory and adjust the liquid lens voltage in real time to pre-focus, improving the image clarity of high-speed moving targets. Integrating the liquid lens, traditional optical lens, and control circuitry into a miniature optical platform offers advantages such as compact structure, rapid response, and high system robustness.

[0063] Example 2, based on Example 1, please refer to... Figure 1 A method of using a liquid lens-based autofocus zoom lens system includes the following steps:

[0064] Step 1: The image acquisition module acquires continuous image frames of the target;

[0065] Step 2: The deep learning target trajectory prediction module calls the target detection network to preprocess, identify and track the acquired continuous image frames to obtain the continuous trajectory information of the target. At the same time, it extracts the spatial feature vector of the image frame through the convolutional neural network (CNN), fuses the continuous trajectory information of the target with the spatial feature vector, and then organizes it into a historical fusion feature time sequence according to the time frame.

[0066] Step 3: Input the continuous image frame time sequence into the Long Short-Term Memory (LSTM) network to capture the dynamic change trend of the target's motion over time and predict the target's position at future times;

[0067] Step 4: Calculate the ideal focal length based on the predicted future position of the target using a preset depth-of-field function, and transmit the ideal focal length data to the electrowetting liquid lens control system.

[0068] Step 5: The electrowetting liquid lens control system is based on the built-in voltage-focal length third-order polynomial mapping model. It calculates and outputs the appropriate control voltage according to the ideal focal length. Based on the electrowetting effect, it changes the contact angle of the liquid-liquid interface in the electrowetting liquid lens, which is converted into a change in the radius of curvature of the interface, thereby realizing the electronic control adjustment of the focal length of the electrowetting liquid lens.

[0069] Step 6: The machine vision algorithm obtains the image blur index of the image frame acquired by the image acquisition module. The image blur index participates in the closed-loop regulation of the control voltage to further optimize the voltage control.

[0070] The deep learning target trajectory prediction module first preprocesses consecutive image frames, performs YOLOv8 target detection and Kalman filtering for cross-frame tracking, and then extracts multi-scale spatial features through CNN. The trajectory information and spatial features are fused and input into an LSTM network to accurately capture the dynamic trend of target motion and predict its future position. Based on the predicted position, the ideal focal length is calculated in advance and focusing is initiated, eliminating the lag defect of the traditional "blurred imaging first, passive adjustment later" approach. Even for high-speed moving or changing-direction moving targets, clear imaging can be achieved throughout the entire process through proactive pre-focusing.

[0071] Step 2 is performed as follows:

[0072] Step 21: Receive continuous image frames transmitted by the image acquisition module in real time, and serialize and number the image frames according to their timestamps to obtain the image frame sequence. ;

[0073]

[0074] For the first Image frames at any given moment;

[0075] Image frames are serialized and numbered according to their timestamps to obtain an image frame sequence. To ensure the temporal order of image frames;

[0076] Step 22: Image Frame Sequence The image frames within the range are sequentially subjected to size normalization, pixel normalization, and lightweight noise reduction preprocessing;

[0077] Step 23: Input the preprocessed image frame into the trained YOLOv8 lightweight detection network. The network's convolutional layers, pooling layers, and decoupling head complete feature extraction and target prediction, and output the target position of a single frame.

[0078] Step 24: Use Kalman filtering to predict the target position, match the target position detected by YOLOv8 with the target position predicted by Kalman filtering, calculate the matching cost through the Hungarian algorithm, accurately match targets across frames, realize cross-frame association of the same target, and update the continuous trajectory information of the target.

[0079] Step 25: Perform multi-scale spatial feature extraction on the preprocessed image frames using a convolutional neural network (CNN) to obtain spatial feature vectors;

[0080] Multi-scale convolutional feature extraction: CNN networks employ lightweight backbone networks (such as MobileNet and ResNet18, adapted for embedded / FPGA hardware deployment) to extract multi-scale features from the input image through the stacking of multiple sets of "convolutional layers + batch normalization layers + ReLU activation layers + max pooling layers".

[0081] Shallow convolutions (such as layers 1-2): extract basic low-dimensional features of the target (such as edges, textures, grayscale distribution, and contour shapes);

[0082] Deep convolutions (such as layers 3-5): extract high-level, high-dimensional features of the target (such as pin features of devices, contour features of pedestrians, and vehicle model features).

[0083] Feature map flattening and dimensionality reduction: The two-dimensional feature map (e.g., 7×7×512 dimensions) output by the last layer of the CNN network is converted into a one-dimensional feature vector (e.g., 512 dimensions) through global average pooling (GAP), and then the feature is reduced in dimensionality through a fully connected layer to obtain a low-dimensional, high-recognition target space feature vector (e.g., 128 dimensions), reducing the computational cost of the subsequent LSTM network.

[0084] Step 26: The target continuous trajectory information and spatial feature vector are fused, and the fused features are organized into a historical fused feature time sequence according to time frames.

[0085] Step 3 is performed as follows:

[0086] Step 31: Sequence of consecutive image frames The corresponding historical fusion feature sequences are used to construct the LSTM-normalized time-series input sequence;

[0087] In the time sequence of consecutive image frames Extract a fixed-length window of historical data from the middle;

[0088] Normalize the fused features to eliminate differences in numerical magnitude and improve the prediction accuracy of LSTM.

[0089] Arranged chronologically, forming a three-dimensional input format that meets the requirements of an LSTM network:

[0090] [Batch size, time step, feature dimension].

[0091] Step 32: Forward inference using the LSTM network to capture the temporal dynamic features of the LSTM-normalized temporal input sequence;

[0092] LSTM networks handle the temporal dependencies of consecutive frames through three core gating structures: forget gate, input gate, and output gate.

[0093] Forget Gate: Automatically filters out invalid historical trajectory noise while retaining key motion information;

[0094] Input gate: Extracts the target location and spatial features of the latest frame to update the network memory units;

[0095] Output gate: Integrates long-term memory and short-term changes to output temporal features.

[0096] Step 33: Establish a mathematical model of target motion based on temporal dynamic characteristics, according to the preset prediction time step. Calculate the target's pixel coordinates / physical coordinates at future time points, remove outlier drift values, and predict the target's future position. .

[0097] Predict the target's position at future times. Calculation formula:

[0098]

[0099] in, This is a combined deep trajectory prediction model consisting of a convolutional neural network (CNN) and a long short-term memory network (LSTM). This refers to the optimal set of weight parameters obtained after training the deep network model. Counting backwards from the current moment Standardized image frames The standardized image frame for the current moment;

[0100] Deep trajectory prediction combined model structure: Image input layer → Target detection and localization layer → CNN spatial feature extraction layer → Feature concatenation layer → LSTM temporal learning layer → Trajectory prediction output layer;

[0101] The image input layer corresponds to steps 21 and 22;

[0102] The target detection and localization layer corresponds to steps 23 and 24;

[0103] Step 25 corresponds to the CNN spatial feature extraction layer;

[0104] The feature splicing layer corresponds to step 26;

[0105] Steps 31 and 32 correspond to the LSTM temporal learning layer;

[0106] The trajectory prediction output layer corresponds to step 33;

[0107] Future Moments Ideal focal length The specific calculations are as follows:

[0108]

[0109] in, For the target depth model, A pre-calibrated depth-of-field function is used to establish the mapping relationship between the target position and the ideal focal length;

[0110] The voltage-focal length third-order polynomial of the voltage-focal length third-order polynomial mapping model is as follows:

[0111]

[0112] : Control voltage is The actual focal length corresponding to the lens;

[0113] : The control voltage applied to the lens;

[0114] , , , Fitting parameters (fixed after calibration to adapt to the characteristics of a single lens).

[0115] The specific steps for step 5 are as follows:

[0116] Step 51: Based on the ideal focal length output in Step 4 The target control voltage is solved using a voltage-focal length third-order polynomial mapping model. ;

[0117] Target control voltage ,satisfy:

[0118]

[0119] Step 52: Based on the electrowetting effect, change the contact angle of the liquid-liquid interface in the electrowetting liquid lens, which is converted into a change in the radius of curvature of the interface, to realize the electronically controlled adjustment of the focal length of the electrowetting liquid lens.

[0120] The change in contact angle is described by the Young-Lippmann equation as follows:

[0121]

[0122] To apply voltage Contact angle at time; Initial contact angle (no voltage); It is the vacuum permittivity; The relative permittivity of the dielectric layer; Liquid-liquid interfacial tension; The dielectric layer thickness; To apply voltage.

[0123] The radius of curvature of the liquid-liquid interface directly determines the focal length of the liquid lens. The basic formula for the focal length of a spherical lens is:

[0124]

[0125] The relative refractive index of the lens medium relative to the surrounding medium; The radius of curvature of the first surface (the surface where the incident light first reaches); Let be the radius of curvature of the second surface (the surface from which the light rays depart);

[0126] Since liquid lenses typically have one side with variable curvature and the other side with a flat or constant curvature, the basic spherical lens focal length formula is simplified to:

[0127]

[0128] The geometric constraint relationship between the radius of curvature and the contact angle of the liquid-liquid interface is as follows:

[0129]

[0130] To fix the cavity aperture; Let be the radius curvature of the interface.

[0131] This function chain clearly shows that by controlling the voltage... It can precisely control the focal length. This allows for rapid focusing.

[0132] The machine vision algorithm uses the Laplacian operator;

[0133] Step 6 is performed as follows:

[0134] Step 61: Image frames transmitted by the image acquisition module Perform grayscale conversion to transform a color image into a grayscale image. ;

[0135]

[0136] R / G / B are the pixel coordinates of the image, and R / G / B are the three-channel pixel values ​​of the original image.

[0137] Step 62: Apply a 3×3 Laplacian convolution kernel to the grayscale image. Perform convolution filtering to extract high-frequency features from the image. ;

[0138] The 3×3 Laplace convolution kernel is defined as:

[0139]

[0140] The result of convolution operation is a high-frequency feature map. The formula is:

[0141]

[0142] This is the convolution operator.

[0143] Step 63: Process the high-frequency feature map The variance of all pixel values ​​is calculated and used as an image blur index for the current frame. ;

[0144]

[0145]

[0146] The resolution of the image frame (e.g., 1920×1080). High-frequency feature map The average pixel value;

[0147] Step 64: Quantize the fuzzyness index The data is transmitted to the electrowetting liquid lens control system. The control system combines the ideal focal length deviation predicted by deep learning with the image blur feedback to construct a dual-adjustment closed-loop control law and update the control voltage in real time.

[0148] The specific steps for step 64 are as follows:

[0149] Step 641: Using ambiguity index The feedback is given to the initial solution of the ideal focal length to the control voltage in step 5;

[0150] It helps to eliminate the following three types of errors:

[0151] Calibration error of the voltage-focal length third-order polynomial model;

[0152] Physical property drift of electrowetting liquid lenses (e.g., temperature changes causing changes in liquid refractive index / interfacial tension).

[0153] Imaging deviations caused by environmental interference (such as changes in lighting or target occlusion).

[0154] Step 642: Based on the control voltage of the previous moment Current frame blur index Ideal focal length Compared to the actual focal length The control voltage for the next time step is updated using the following closed-loop control formula. ;

[0155] The closed-loop control formula is as follows:

[0156]

[0157] : To control the weighting parameters; Indicates ambiguity feedback adjustment; Adjustment for predicted focal length difference.

[0158] Determined through offline calibration + online fine-tuning:

[0159] Offline phase: Testing different focal lengths and motion speeds. / To optimize the focusing accuracy and stability of the combination, the optimal parameter range is selected.

[0160] Online phase: Based on the system's operating status (such as the target's motion speed and the rate of change of image blur), adaptively fine-tune parameters to ensure full scene adaptation.

[0161] Step 643: Construct a closed-loop control law with dual adjustment terms and adopt a dual-term collaborative mechanism of "forward-looking pre-correction + real-time feedback correction" for correction processing;

[0162] Predictive focal length difference adjustment: Based on the future position predicted by deep learning and the ideal focal length, the control voltage is quickly corrected to achieve forward pre-focusing of moving targets and avoid focusing lag caused by high-speed target movement;

[0163] Blur feedback adjustment: The control voltage is finely adjusted in real time according to the actual imaging blur, and the model / hardware error is accurately corrected to ensure that the image can maintain optimal sharpness even under environmental interference and lens drift.

[0164] For example, when the target moves at a constant speed, the predicted focal length difference adjustment dominates the voltage correction to achieve rapid pre-focusing; when the ambient light changes abruptly, causing local blurring of the image, the blur feedback adjustment dominates the fine-tuning to quickly restore image clarity.

[0165] Specific examples are as follows:

[0166] Application Scenario 1: High-speed production line inspection

[0167] In industrial production lines, objects move at speeds of up to 5–10 m / s, making it difficult for traditional mechanical focusing mechanisms to complete focusing within 50 ms.

[0168] This invention is deployed on an electronic device identification production line. By acquiring video frames of the target object moving at high speed on a conveyor belt, the system can predict in real time the time point when the target reaches the center of the camera's field of view and adjust the lens curvature in advance.

[0169] Evaluation results:

[0170] Response time: <5ms;

[0171] Image blur index An increase of approximately 48%;

[0172] Imaging resolution (based on template matching success rate): increased from 73.2% in traditional systems to 95.6%.

[0173] Application Scenario 2: Drone Dynamic Tracking Camera System

[0174] During drone flight, target objects (such as people or vehicles) exhibit complex trajectories and rapid displacement changes. Traditional servo motor focusing systems suffer from significant tracking lag.

[0175] This invention is integrated into a quadcopter drone for real-time tracking of moving targets. The prediction module predicts the future position of the target based on a sequence of aerial images and presets the focal length of the liquid lens.

[0176] Evaluation results:

[0177] The maximum focus delay has been reduced from 120ms to 12ms;

[0178] The average focus tracking error for moving targets is reduced by approximately 62%;

[0179] Power consumption is reduced by 37% compared to traditional servo systems.

[0180] Example 3, based on Example 2:

[0181] Expand the original single-channel liquid lens autofocus system to Channel liquid lens array system, in which The total number of liquid lens channels in the system ( ≥2). Each channel is configured with an independent sub-module link to achieve independent image acquisition, independent target trajectory prediction, independent focus electronic control adjustment, and independent visual feedback closed loop. The specific extended architecture is as follows:

[0182] Each channel is configured with an independent image acquisition submodule, a deep learning target trajectory prediction submodule, an electrowetting liquid lens submodule, an electrowetting lens control submodule, and a feedback adjustment submodule;

[0183] There is no signal interference between channels, each channel can perform focusing tasks independently, and supports parallel processing of multiple targets and multiple focal lengths.

[0184] against For a channel array system, the system-level control parameters and channel-level focal length parameters are defined as follows:

[0185] Control voltage vector: Represents the drive control voltage of each channel, and its vector form is as follows:

[0186]

[0187] Focal length vector: Represents the actual focal length of each channel, and its vector form is as follows:

[0188]

[0189] To achieve multi-channel parallel focusing, an independent predictive focusing strategy is implemented for each channel. The focusing logic for each channel is consistent with the single-channel logic in Example 2, except that the inputs and outputs are channel-specific. The specific strategy expression is as follows:

[0190]

[0191] : No. A channel in the future The ideal focal length;

[0192] : No. A combined model for dedicated depth trajectory prediction for each channel. For a dedicated set of optimal weight parameters;

[0193] : No. A dedicated, continuous, standardized image frame time sequence acquired from each channel;

[0194] : No. Each channel has its own dedicated depth function, along with the optical parameters and target depth model for that channel. Binding;

[0195] : No. The target depth model corresponding to each channel represents the depth mapping relationship of the imaging scene in that channel.

[0196] Through a multi-channel independent predictive focusing strategy, this extended system can achieve simultaneous focusing on targets from multiple viewpoints, parallel imaging at multiple focal points, and collaborative detection across multiple focal lengths. Specific applicable scenarios include:

[0197] Industrial production line inspection scenario: Simultaneously perform feature recognition and focusing on multiple targets of different sizes / types (such as electronic components and mechanical parts) in the production line to improve production line inspection efficiency;

[0198] Multi-point measurement system scenario: Simultaneously focus on multiple measurement points at different distances in space to achieve high-precision synchronous acquisition of multi-position parameters;

[0199] Medical imaging scenarios: Multi-focal parallel image acquisition of tissue structures at different depths within the human body (such as the skin surface, muscle layer, and organ interior) to meet multi-dimensional medical diagnostic needs.

[0200] Electrowetting liquid lenses manufactured by different companies have individual differences in their core manufacturing parameters, such as liquid-liquid interfacial tension. Dielectric layer thickness Relative permittivity of dielectric layer The differences in the above parameters will lead to the same control voltage. Different contact angles correspond to different lenses. radius of curvature and focal length If a universal voltage-focal length model is used directly, it will result in poor lens compatibility and insufficient system versatility.

[0201] To address this issue, this invention designs an adaptive voltage curvature matching mechanism to achieve personalized modeling of different individual lenses, ensuring that the system is plug-and-play.

[0202] Before deploying the multi-channel liquid lens array system, an initial personalized calibration is performed on the liquid lens of each channel. The specific steps are as follows:

[0203] To the Each liquid lens channel receives a set of gradient control voltage sequences. (Covering the lens's rated operating voltage range, such as 0-15V);

[0204] Acquire the actual focal length sequence of this channel under each set of control voltages. Data is collected using high-precision focal length measurement equipment (such as an optical focus meter) to ensure measurement accuracy better than ±1μm;

[0205] Based on the collected voltage-focal length data, a personalized voltage-focal length third-order polynomial model of the lens is fitted using the least squares method.

[0206] The fitting objective is to solve for the optimal polynomial parameters. , , , To minimize the sum of squared errors between the actual focal length and the focal length calculated by the model, the specific formula is as follows:

[0207]

[0208] In the formula:

[0209] : No. The lens at the first Group control voltage The actual measured focal length;

[0210] : No. The lens at the first Group control voltage The model calculates the focal length using a general third-order polynomial. Calculated;

[0211] , , , ,: No. The individual fitting parameters for each lens are calibrated and then stored in the control system of that channel.

[0212] After initial calibration, the personalized voltage-focal length model for each channel is loaded into the corresponding channel's electrowetting liquid lens control system. During system deployment, only the physical installation of the lens and the loading of model parameters are required to directly execute predictive focusing and closed-loop feedback tasks. No secondary debugging for individual lenses is needed, achieving plug-and-play functionality across all scenarios and significantly improving the system's versatility and deployment efficiency.

[0213] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An autofocus zoom lens system based on a liquid lens, characterized in that: include: Image acquisition module: used to acquire continuous image frames of the target; Deep learning target trajectory prediction module: It communicates with the image acquisition module and is used to identify, track and predict the future position of the target from the acquired continuous image frames, and output the ideal focal length. Electrowetting liquid lens: The interior is filled with two immiscible liquids, one conductive and one insulating, and there are no moving mechanical parts; Electrowetting liquid lens control system: It communicates with the deep learning target trajectory prediction module and is used to output control voltage according to the ideal focal length. Based on the electrowetting effect, it realizes the electronic control adjustment of the focal length of the electrowetting liquid lens by changing the contact angle and radius of curvature of the liquid-liquid interface. Feedback adjustment module: It is connected to the image acquisition module and the electrowetting liquid lens control system respectively. It is used to obtain the image blur index by machine vision algorithm on the image frames acquired by the image acquisition module, and then perform closed-loop optimization adjustment of the control voltage based on the image blur index.

2. A method of use, applying the liquid lens-based autofocus zoom lens system according to claim 1, characterized in that, Includes the following steps: Step 1: The image acquisition module acquires continuous image frames of the target; Step 2: The deep learning target trajectory prediction module calls the target detection network to preprocess, identify and track the acquired continuous image frames to obtain the continuous trajectory information of the target. At the same time, it extracts the spatial feature vector of the image frame through the convolutional neural network (CNN), fuses the continuous trajectory information of the target with the spatial feature vector, and then organizes it into a historical fusion feature time sequence according to the time frame. Step 3: Input the continuous image frame time sequence into the Long Short-Term Memory (LSTM) network to capture the dynamic change trend of the target's motion over time and predict the target's position at future times; Step 4: Calculate the ideal focal length based on the predicted future position of the target using a preset depth-of-field function, and transmit the ideal focal length data to the electrowetting liquid lens control system. Step 5: The electrowetting liquid lens control system is based on the built-in voltage-focal length third-order polynomial mapping model. It calculates and outputs the appropriate control voltage according to the ideal focal length. Based on the electrowetting effect, it changes the contact angle of the liquid-liquid interface in the electrowetting liquid lens, which is converted into a change in the radius of curvature of the interface, thereby realizing the electronic control adjustment of the focal length of the electrowetting liquid lens. Step 6: The machine vision algorithm obtains the image blur index of the image frame acquired by the image acquisition module. The image blur index participates in the closed-loop regulation of the control voltage to further optimize the voltage control.

3. The method of use according to claim 2, characterized in that, Step 2 is performed as follows: Step 21: Receive continuous image frames transmitted by the image acquisition module in real time, and serialize and number the image frames according to their timestamps to obtain the image frame sequence. ; Step 22: Image Frame Sequence The image frames within the range are sequentially subjected to size normalization, pixel normalization, and lightweight noise reduction preprocessing; Step 23: Input the preprocessed image frame into the trained YOLOv8 lightweight detection network. The network's convolutional layers, pooling layers, and decoupling head complete feature extraction and target prediction, and output the target position of a single frame. Step 24: Use Kalman filtering to predict the target position, match the target position detected by YOLOv8 with the target position predicted by Kalman filtering, calculate the matching cost through the Hungarian algorithm, accurately match targets across frames, realize cross-frame association of the same target, and update the continuous trajectory information of the target. Step 25: Perform multi-scale spatial feature extraction on the preprocessed image frames using a convolutional neural network (CNN) to obtain spatial feature vectors; Step 26: The target continuous trajectory information and spatial feature vector are fused, and the fused features are organized into a historical fused feature time sequence according to time frames.

4. The method of use according to claim 3, characterized in that, Step 3 is performed as follows: Step 31: Sequence of consecutive image frames The corresponding historical fusion feature sequences are used to construct the LSTM-normalized time-series input sequence; Step 32: Forward inference using the LSTM network to capture the temporal dynamic features of the LSTM-normalized temporal input sequence; Step 33: Establish a mathematical model of target motion based on temporal dynamic characteristics, according to the preset prediction time step. Calculate the target's pixel coordinates / physical coordinates at future time points, remove outlier drift values, and predict the target's future position. .

5. The method of use according to claim 4, characterized in that, Predict the target's position at future times. Calculation formula: ； in, This is a combined deep trajectory prediction model consisting of a convolutional neural network (CNN) and a long short-term memory network (LSTM). This refers to the optimal set of weight parameters obtained after training the deep network model. Counting backwards from the current moment Standardized image frames This is the standardized image frame at the current moment.

6. The method of use according to claim 5, characterized in that, Future Moments Ideal focal length The specific calculations are as follows: ； in, For the target depth model, It is a pre-calibrated depth-of-field function used to establish the mapping relationship between the target position and the ideal focal length.

7. The method of use according to claim 6, characterized in that, The specific steps for step 5 are as follows: Step 51: Based on the ideal focal length output in Step 4 The target control voltage is solved using a voltage-focal length third-order polynomial mapping model. ; Step 52: Based on the electrowetting effect, change the contact angle of the liquid-liquid interface inside the electrowetting liquid lens, which is converted into a change in the radius of curvature of the interface, to realize the electronically controlled adjustment of the focal length of the electrowetting liquid lens.

8. The method of use according to claim 7, characterized in that, Target control voltage ,satisfy: 。 9. The method of use according to claim 8, characterized in that, Step 6 is performed as follows: Step 61: Image frames transmitted by the image acquisition module Perform grayscale conversion to transform a color image into a grayscale image. ; Step 62: Apply a 3×3 Laplacian convolution kernel to the grayscale image. Perform convolution filtering to extract high-frequency features from the image. ; Step 63: Process the high-frequency feature map The variance of all pixel values ​​is calculated and used as an image blur index for the current frame. ; Step 64: Quantize the fuzzyness index The data is transmitted to the electrowetting liquid lens control system. The control system combines the ideal focal length deviation predicted by deep learning with the image blur feedback to construct a dual-adjustment closed-loop control law and update the control voltage in real time.

10. The method of use according to claim 9, characterized in that, The specific steps for step 64 are as follows: Step 641: Using ambiguity index The feedback is given to the initial solution of the ideal focal length to the control voltage in step 5; Step 642: Based on the control voltage of the previous moment Current frame blur index Ideal focal length Compared to the actual focal length The control voltage for the next time step is updated using the following closed-loop control formula. ; Step 643: Construct a closed-loop control law with dual adjustment terms and adopt a dual-term collaborative mechanism of "forward-looking pre-correction + real-time feedback correction" for correction processing.

Images