Liquid lens and control method and control system thereof

Abstract

The application discloses a liquid lens and a control method and system thereof. The method comprises the following steps: firstly, obtaining the real-time working temperature, real-time posture and target focal length of each liquid lens and generating the real-time state parameters of each liquid lens; then, a control parameter prediction model predicts the driving voltage compensation value of the liquid lens according to the real-time state parameters of each liquid lens; finally, the actual driving voltage of the liquid lens is generated by combining the theoretical driving voltage of the liquid lens and is applied to the liquid lens. The application can realize millisecond-level feedforward compensation, does not need external ranging devices, can significantly improve the imaging quality and environmental adaptability of the liquid lens imaging device, and is suitable for scenes such as unmanned aerial vehicles, machine vision and medical endoscopes.

Description

Technical Field

[0001] This invention belongs to the field of liquid lenses, and particularly relates to a liquid lens and its control method and control system. Background Technology

[0002] As a type of miniature optical device, the main principle of a liquid lens is to use an external electric field to change the contact angle between a conductive liquid and an insulating liquid, thereby controlling the curvature of the interface between the two immiscible fluids and achieving continuous adjustment of the focal length without mechanical parts.

[0003] However, in practical applications, the performance of existing liquid lenses is significantly affected by environmental factors (such as temperature and lens orientation), mainly in the following two aspects: First, fluctuations in ambient temperature directly alter the physical properties of the two-phase fluid inside the liquid lens, such as the dynamic viscosity and surface tension of the liquid. Increased temperature leads to decreased viscosity and reduced surface tension, resulting in a larger change in contact angle under the same driving voltage, causing temperature drift; low temperatures, on the other hand, significantly slow down the response time. Second, when a device equipped with a liquid lens tilts, the liquid undergoes asymmetric deformation under the influence of gravity, causing the liquid surface to deviate from the ideal spherical shape. This results in significant aberrations under different orientations such as pitch and roll, severely affecting image quality.

[0004] To overcome the aforementioned problems, the industry has undertaken some exploration. Currently, mainstream electrowetting liquid lens products (such as the Corning Varioptic series and Optotune EL series) typically integrate temperature sensors on the lens module housing or drive circuit board, and compensate for the driving voltage using temperature-voltage lookup tables or polynomial fitting. For example, CN115903103A discloses a compensation method based on temperature-voltage lookup tables at different working distances; this type of solution can achieve relatively reliable temperature drift suppression under static conditions of horizontal placement. However, in terms of attitude sensing, existing electrowetting liquid lens products and related patents do not integrate attitude sensors onto the liquid lens body. In other words, existing electrowetting liquid lenses generally exhibit a situation of "temperature sensing but no attitude sensing," lacking real-time, local sensing capabilities for asymmetric deformation of the liquid surface under gravitational fields, and unable to actively close the loop to suppress coma and astigmatism caused by attitude changes.

[0005] In terms of drive control strategy, the existing electrowetting liquid lens solution also has obvious shortcomings: (1) Due to the lack of real-time attitude sensing for the lens optical axis, off-axis aberrations such as coma and astigmatism caused by attitude tilt can only be indirectly compensated by pre-calibration offline or external attitude signals, making it difficult to achieve active, closed-loop real-time surface reshaping; (2) Even if the temperature sensor has been integrated into the lens module, the control strategy based on one-dimensional lookup table or simple PID feedback can only handle the single variable of temperature and cannot cope with multiple variables such as temperature, attitude tilt angle, and target focal length. The coupled nonlinear deformation problem has a large calibration workload, a large parameter space, and poor generalization ability; (3) The pure closed-loop compensation scheme based on imaging feedback is affected by the liquid inertia and the response is delayed, which makes it difficult to meet the millisecond-level real-time compensation requirements in dynamic scenarios such as drone aerial photography, vehicle vision, handheld imaging, and medical endoscopes; (4) Most existing schemes adopt a single ring or ring-shaped driving electrode structure, which can only change the liquid surface contact angle as a whole and cannot apply asymmetric voltage to different directions of the liquid surface. Therefore, even if attitude information is provided, there is a lack of execution means to actively reshape the non-spherical liquid surface. Summary of the Invention

[0006] To overcome the shortcomings of existing electrowetting liquid lenses under temperature variations and tilting conditions, such as focal length drift, independent sensing and actuators, degraded image quality, limited compensation methods, and insufficient real-time response, this invention provides a liquid lens and its control method and control system. This invention integrates a temperature sensor, an attitude sensor, and a multi-zone driving electrode, and implements adaptive feedforward compensation based on a deep neural network model. This invention introduces a deep neural network model into fluid surface shape prediction and voltage compensation, using simulation and measured data for offline training, thereby outputting the optimal driving voltage in real time within milliseconds.

[0007] The technical solution of the present invention is as follows:

[0008] In a first aspect, the present invention proposes a method for controlling a liquid lens, the method comprising the following steps:

[0009] The system acquires the real-time operating temperature, real-time attitude, and target focal length of each liquid lens and generates real-time state parameters for each liquid lens. The control parameter prediction model predicts the driving voltage compensation value of the liquid lens based on the real-time state parameters of each liquid lens. Then, combined with the theoretical driving voltage of the liquid lens, the system generates the actual driving voltage of the liquid lens and applies it to the liquid lens.

[0010] Furthermore, the control parameter prediction model is a pre-trained deep neural network model.

[0011] Furthermore, the deep neural network model is a physical information neural network model, and its loss function during pre-training includes data fitting loss and physical constraint residual terms. The physical constraint residual terms are constructed from the Young–Lippmann equation and / or the two-phase interface force balance equation characterizing the relationship between the electrowetting contact angle and voltage, so that the output of the physical information neural network model satisfies the physical constraints.

[0012] Furthermore, the real-time attitude includes pitch angle and roll angle.

[0013] Furthermore, the liquid lens includes a liquid lens body, a temperature sensor, and an attitude sensor. The temperature sensor is used to collect the operating temperature of the liquid lens body, and the attitude sensor is used to collect the attitude of the liquid lens body.

[0014] Furthermore, the driving electrode at the liquid lens body is a plurality of electrodes, with different electrodes acting on different sections of the liquid lens body.

[0015] Furthermore, the driving voltage compensation value is the time-varying voltage trajectory corresponding to each partition electrode. This voltage trajectory includes a voltage pre-shaping segment and a voltage steady-state segment. The voltage pre-shaping segment is used to shorten the establishment time of the liquid surface of the liquid lens to the target surface shape and suppress oscillations. The pre-shaping segment is determined based on the inertia and settling time of the two liquids in the liquid lens. The voltage pre-shaping segment applies a transient overshoot voltage during the process of the liquid surface approaching the target surface shape and then converges to the steady-state compensation value.

[0016] Secondly, the present invention proposes a control system for a liquid lens, the system comprising:

[0017] The state data acquisition unit is used to acquire the real-time operating temperature, real-time attitude, and target focal length of each liquid lens and generate the state parameters of each liquid lens.

[0018] The compensation data prediction unit is used to predict the driving voltage compensation value of each liquid lens based on the state parameters of each liquid lens using a control parameter prediction model.

[0019] The driving voltage generation unit is used to generate the actual driving voltage of each liquid lens based on the driving voltage compensation value and the theoretical driving voltage of each liquid lens.

[0020] The drive control unit is used to control the corresponding liquid lens based on the actual drive voltage of each liquid lens.

[0021] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the control method described in the first aspect.

[0022] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the control method described in the first aspect.

[0023] Fifthly, the present invention provides a computer program product comprising a computer program / instruction which, when executed by a processor, implements the steps of the control method described in the first aspect.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. Comprehensive compensation for temperature and gravity disturbances: This invention adopts a control parameter prediction model with temperature and attitude angle as inputs and multi-zone structured voltage vector as output. Combined with multiple independently controllable zone electrodes, it realizes comprehensive prediction and compensation for changes in liquid viscosity / surface tension caused by temperature and non-spherical deformation of the liquid surface caused by gravity. It overcomes the limitations of existing technologies that separate temperature compensation and attitude compensation and can only be adjusted as a whole. It has higher compensation accuracy and stronger generalization ability.

[0026] 2. High-precision proxy model: This invention uses a multiphysics simulation platform as the data source to generate high-quality training data covering a wide temperature range and a wide attitude range, avoiding the problems of high data acquisition cost and incomplete coverage in the pure experimental calibration scheme; the joint training of simulation data and experimental data further improves the practical accuracy of the model; on this basis, physical information constraints are further introduced, and the extrapolation accuracy of the model under untrained conditions can be significantly improved.

[0027] 3. Fast response from feedforward prediction and dynamic trajectory shaping: After the neural network model is deployed, the inference latency is extremely low (millisecond level). It can output compensation voltage immediately after the sensor collects environmental parameters to achieve feedforward prediction compensation. Compared with the pure closed-loop solution that relies on imaging feedback, it has a faster response speed. Combined with the pre-shaping of dynamic voltage trajectory, the liquid surface establishment time can be further shortened, making it more suitable for dynamic changing scenarios, especially for scenarios such as drones, machine vision, and medical endoscopes.

[0028] 4. No external ranging device required: This invention does not require an additional ranging module. The target focal length is directly sent from the host system, and environmental parameters are provided by temperature sensors and inertial measurement units (IMUs). The system has a high degree of integration, avoiding errors caused by insufficient ranging accuracy of external ranging modules, and reducing system cost and complexity. Attached Figure Description

[0029] Figure 1 This is a flowchart of the method of the present invention.

[0030] Figure 2 This is a block diagram of the overall structure of the system of the present invention.

[0031] Figure 3 This is a top view schematic diagram of a multi-zone driving electrode in a liquid lens.

[0032] Figure 4 This is a longitudinal cross-sectional view of the liquid lens of the present invention.

[0033] Figure 5 This is a schematic diagram of the deep neural network model structure of the present invention.

[0034] In the figure: 2—Inertial Measurement Unit; 3—Support Ring; 4—First Electrode; 5—Second Electrode; 6—Rigid Printed Circuit Board; 7—Embedded Processor; 8—Liquid Lens Body; 9—Temperature Sensor. Detailed Implementation

[0035] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0036] It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of the present invention. Although the illustrations only show components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation, the shape, quantity and proportion of each component in the actual implementation can be arbitrarily changed, and the layout of the components may also be more complex.

[0037] like Figure 2 As shown, the control system for a liquid lens proposed in this invention includes:

[0038] The state data acquisition unit is used to acquire the real-time operating temperature, real-time attitude, and target focal length of each liquid lens and generate the state parameters of each liquid lens.

[0039] The compensation data prediction unit is used to predict the driving voltage compensation value of each liquid lens based on the state parameters of each liquid lens using a control parameter prediction model.

[0040] The driving voltage generation unit is used to generate the actual driving voltage of each liquid lens based on the driving voltage compensation value and the theoretical driving voltage of each liquid lens.

[0041] The drive control unit is used to control the corresponding liquid lens based on the actual drive voltage of each liquid lens, so that the focal length of the liquid lens is the target focal length.

[0042] In one feasible implementation, the liquid lens includes a temperature sensor 9 and an attitude sensor installed in the liquid lens body 8. The temperature sensor 9 is used to collect the operating temperature of the liquid lens body 8, and the attitude sensor is used to collect the attitude of the liquid lens body 8.

[0043] In one feasible embodiment, the liquid lens body includes a cavity, two immiscible liquids placed inside the cavity, and a driving electrode disposed around the cavity. The interface morphology between the two liquids can be changed under the action of a voltage applied to the driving electrode, thereby achieving continuous adjustment of the focal length.

[0044] One feasible implementation method is, for example Figure 3 As shown, the driving electrode at the liquid lens body consists of multiple electrodes, with different electrodes acting on different sections of the liquid lens body (i.e., Figure 3 (The fan-shaped region in the middle). The electrode can be configured as a 4-zone or 8-zone array (denoted as the second electrode 5), and together with the common electrode (denoted as the first electrode 4), it can apply asymmetric control to the contact angle in various directions of the liquid surface, actively reshaping the surface shape to compensate for gravitational sagging. Optionally, the electrode is a flexible electrode.

[0045] In one feasible implementation, the temperature sensor 9 is integrated onto the liquid lens body. Specifically, the sensitive surface of the temperature sensor 9 is in close contact with the outer surface of the side wall of the cavity or on the flexible electrode, and is used to collect the operating temperature information of the liquid lens body in real time.

[0046] In one feasible implementation, the attitude sensor is an inertial measurement unit (IMU) 2, such as a Bosch BMI270 or ICM-42688. Specifically, the IMU 2 is surface-mounted on a rigid printed circuit board (PCB) 6, and the IMU 2 and the rigid PCB 6 are soldered together. The rigid PCB 6 and the electrowetting liquid lens body are encapsulated in the same housing and positioned using a support ring structure. The sensitive axis of the IMU 2 is coaxial with or parallel to the optical axis of the liquid lens body, used to acquire the attitude angle information of the liquid lens body relative to the direction of gravity in real time.

[0047] In one feasible implementation, the inertial measurement unit 2 includes a three-axis accelerometer and a three-axis gyroscope, capable of outputting three-axis acceleration signals and three-axis angular velocity signals. The inertial measurement unit also includes an attitude calculation unit, which receives the output three-axis acceleration and angular velocity signals, and then fuses and calculates them using a Kalman filter algorithm to obtain the pitch angle (range ±90°), roll angle (range ±180°), and yaw angle (range ±180°) of the liquid lens body's optical axis relative to the direction of gravity. Finally, the pitch and roll angles are used as attitude information. The angle calculation accuracy of the attitude calculation unit is no less than ±0.5°, and the data update rate is no less than 100Hz.

[0048] In one feasible implementation, the temperature sensor is an NTC thermistor or a PT100 platinum resistance thermometer, with a measurement accuracy of not less than ±0.5℃ and a sampling rate of not less than 10Hz. The temperature sensor is placed on the flexible electrode of the liquid lens body or attached to the outer wall of the cavity of the liquid lens body to reflect the actual temperature state of the liquid inside the cavity.

[0049] In one feasible implementation, the rigid printed circuit board 6 is square or circular, with dimensions slightly larger than the outer diameter of the liquid lens body. It is made of high-frequency FR-4 or an aluminum substrate, exhibiting good dimensional stability. A light-transmitting hole is located at the center of the rigid printed circuit board 6, with a diameter not smaller than the effective optical aperture of the liquid lens body, allowing the imaging light path of the liquid lens body to pass through the hole without being obstructed by the circuit board.

[0050] One feasible implementation method is, for example Figure 4 As shown, the liquid lens body is positioned and mounted on the upper surface of the rigid printed circuit board 6 via a support ring 3. The inner diameter of the support ring 3 is interference-fitted with the outer shell of the liquid lens body, and its lower end is fixed to the rigid printed circuit board 6 by screws or adhesive. The parallelism between the end face of the support ring 3 and the surface of the rigid printed circuit board 6 is better than 0.05 mm, thereby ensuring that the optical axis of the liquid lens body is highly parallel to the normal direction of the rigid printed circuit board 6, that is, coincident with or parallel to the Z-axis direction of the inertial measurement unit 2. Based on this, the attitude angle output by the inertial measurement unit 2 can directly represent the attitude angle of the optical axis of the liquid lens body relative to the direction of gravity, without the need for additional calibration rotation matrix.

[0051] In one feasible implementation, the rigid printed circuit board 6 integrates not only the inertial measurement unit 2, but also an embedded processor 7 (such as an ARM Cortex-A55 or NVIDIA Jetson Nano computing module) with a compensation data prediction unit, as well as a voltage drive circuit for the drive control unit. The outputs of the temperature sensor 9 and the inertial measurement unit 2 are directly fed into the embedded processor 7 through traces on the rigid printed circuit board 6. The embedded processor 7 runs a trained neural network model and outputs an N-dimensional compensation vector to the voltage drive circuit, which amplifies it to the operating voltage required for the multi-zone electrodes (the operating voltage typically ranges from 24 to 70V).

[0052] Through the aforementioned rigid co-encapsulation structure, the liquid lens body, temperature sensor, attitude sensor, compensation data prediction unit, driving voltage generation unit, and driving control unit are highly integrated into a compact intelligent liquid lens module with an overall thickness that can be controlled within 8mm, making it easy to embed into space-constrained applications such as machine vision cameras and medical endoscopes.

[0053] This embodiment describes the specific workflow of the control system.

[0054] After the system is powered on, the temperature sensor and attitude sensor continuously provide the compensation data prediction unit with the current temperature T and pitch angle θ of the liquid lens at an update rate of no less than 100Hz. pitch Roll angle θ roll and target focal length f target The compensation data prediction unit receives the above input, performs neural network forward propagation, and outputs a driving voltage compensation vector ΔV. The driving voltage compensation vector ΔV contains driving voltage compensation values ​​corresponding one-to-one with each partition electrode. The delay of the compensation data prediction unit is less than 5ms to ensure the timeliness of the feedforward compensation. The drive control unit calculates the actual output voltage and applies it to the multi-partition electrodes through the high-voltage drive circuit.

[0055] Preferably, the driving voltage compensation vector output by the compensation data prediction unit is a voltage trajectory that varies with time. This voltage trajectory includes a voltage pre-shaping segment and a voltage steady-state segment. The voltage pre-shaping segment is used to shorten the establishment time of the liquid surface of the liquid lens to the target surface shape and suppress oscillations. The pre-shaping segment is determined based on the inertia and settling time of the two liquids in the liquid lens. The voltage pre-shaping segment applies a transient overshoot voltage during the process of the liquid surface approaching the target surface shape and then converges to the steady-state compensation value.

[0056] The dynamic response of the meniscus of each section of the liquid lens to changes in driving voltage is equivalent to a first-order inertial element, with a time constant of: Time constant Proportional to the dynamic viscosity of the two-phase liquid With cavity geometry factors, and dynamic viscosity The time constant varies with temperature T. The temperature T measured by the temperature sensor is converted online using the following formula:

[0057]

[0058] in, , These are the time constant and viscosity obtained at standard temperatures, respectively. A step voltage can be applied to a liquid lens at a standard temperature, its step response curve can be measured, and the result can be obtained by fitting the step response curve.

[0059] When the target focal length or attitude changes, the steady-state driving voltage of the i-th partition needs to be changed from the current value. Transform into target value At this time, the driving voltage generation unit does not directly apply a step voltage, but instead applies a two-segment voltage trajectory that includes a pre-shaping segment, specifically including:

[0060] Apply an overshoot voltage in the pre-shaping phase (0≤t<t1) , , represents the power exponent. That is, an overshoot of β times the voltage variation is added on the basis of the target voltage, which means additional driving force is injected to accelerate the convergence to the target surface profile.

[0061] In the steady-state phase (t ≥ t1), the voltage is switched and maintained at the steady-state voltage.

[0062] In a feasible implementation, for liquid lenses with low damping and underdamped oscillation in step response, the above first-order model can also be replaced with a second-order underdamped model (natural angular frequency ω n , damping ratio ζ), and zero-oscillation input shaping is adopted.

[0063] In the present invention, the multi-segment electrode is composed of N independently controllable electrode units (N≥2), and the compensation data prediction unit can output an N-dimensional compensation vector [ΔV1, ΔV2, ……, ΔV N , applying different compensation voltages to each segmented electrode respectively to correct local liquid surface distortion caused by electrode consistency errors or liquid inhomogeneity, further improving the accuracy of the liquid lens. Optionally, in a simplified solution that only considers overall temperature compensation, the compensation amounts of the N electrodes can be approximated to the same value, that is, ΔV1=ΔV2=……=ΔV N , so as to reduce the output dimension and training complexity of the neural network model.

[0064] As a preferred option, the control system further includes an online self-calibration unit. The online self-calibration unit statistically analyzes the change trend of the imaging sharpness evaluation index over time at a low update frequency; when a systematic decrease in the sharpness index under the same working condition is detected, it is determined that long-term changes such as two-phase liquid evaporation, device aging or electrode characteristic drift have occurred, and accordingly, the correction term superimposed on the output of the control parameter prediction model is updated online, or the output layer parameters of the control parameter prediction model are slightly fine-tuned, so as to maintain long-term stability of the compensation accuracy throughout the full life cycle of the device.

[0065] As shown in Figure 1 , the control method for a liquid lens proposed by the present invention includes the following steps:

[0066] Acquire the real-time working temperature, real-time attitude and target focal length of each liquid lens to generate real-time state parameters for each liquid lens. The control parameter prediction model predicts the driving voltage compensation value of the liquid lens according to the real-time state parameters of each liquid lens, and then combines it with the theoretical driving voltage of the liquid lens to generate the actual driving voltage of the liquid lens and apply it to the liquid lens, so that the focal length of the liquid lens reaches the target focal length.

[0067] In one feasible implementation, the control parameter prediction model is a pre-trained deep neural network (DNN) model.

[0068] In one feasible implementation, each liquid lens has multiple driving voltage compensation values, the number of which is the same as the number of driving electrodes for each liquid lens. Each driving voltage compensation value is used to adjust the driving voltage of the corresponding electrode, thereby dynamically adjusting the interface morphology between the two liquids in the liquid lens cavity.

[0069] In one feasible implementation, the pre-training process of the deep neural network model is as follows:

[0070] First, a fluid-electric field-heat transfer coupled physical model of the electrowetting liquid lens is established using a multiphysics simulation platform. Then, multiple sets of environmental parameter combinations are generated in the parameter space consisting of temperature (covering the operating temperature range), pitch angle (covering 0° to 90°), roll angle (covering 0° to 90°), and target focal length through Latin hypercube sampling. For each set of parameter combinations, coupled simulation is run and combined with a genetic algorithm for iterative back-solution to obtain the optimal driving voltage for each partition that makes the meniscus achieve the surface shape corresponding to the target focal length and minimizes imaging aberrations. The difference between the optimal driving voltage and the basic driving voltage corresponding to the standard temperature and horizontal attitude is used as the driving voltage compensation value corresponding to the set of parameters, thereby obtaining the driving voltage compensation vector.

[0071] Compared to directly measuring the compensation voltage, this invention synthesizes the optimal control truth value within the simulation loop through "simulation + evolutionary optimization," which can provide high-quality and comprehensive training labels for azimuth-asymmetric, underdetermined multi-zone execution problems.

[0072] Then, the simulation dataset is expanded and corrected using the measured calibration data to form a joint training dataset; finally, the joint training dataset is normalized and divided into training set, validation set and test set in an 8:1:1 ratio.

[0073] like Figure 5As shown, the deep neural network model employs a fully connected feedforward network structure, which includes one input layer, three hidden layers, and one output layer. The number of neurons in the input layer is the same as the feature dimension of the real-time state parameters. The number of hidden layers can be adjusted according to actual needs, generally no less than three layers, with no less than 64 neurons per layer. The activation function for the hidden layers is the Rectified Linear Activation Function (ReLU). The number of neurons in the output layer is the same as the number of partitioned electrodes N in the liquid lens, and the activation function is a linear activation function, suitable for regression tasks. The output layer is used to output an N-dimensional driving voltage compensation vector. In a simplified scheme considering only overall temperature compensation, the output layer can also be a single neuron, outputting the driving voltage compensation value shared by all partitioned electrodes. Furthermore, each hidden layer is followed by a Dropout regularization layer and a batch normalization layer, where the Dropout regularization layer is used to prevent overfitting, and the batch normalization layer is used to accelerate convergence.

[0074] It should be noted that, Figure 5 This diagram illustrates three hidden layers. The circular nodes in each hidden layer are for illustrative purposes only and are not used to limit the actual number of neurons. The Dropout regularization layer and batch normalization layer are placed after each hidden layer. Figure 5 It is not shown separately in the text.

[0075] Specifically:

[0076] The input layer receives the real-time operating temperature T and pitch angle θ from the liquid lens. pitch Roll angle θ roll and target focal length f target The four-dimensional state parameters (i.e., real-time state parameters) are constructed and preprocessed according to the same normalization rules as in the model training phase. Adjacent network layers are fully connected. Each hidden layer performs a weighted summation and bias operation on the output of the previous layer, and then extracts the coupling nonlinear features between temperature, attitude, and target focal length using the ReLU activation function. The output layer outputs a driving voltage compensation vector ΔV, ΔV=[ΔV1, ΔV2, ΔV3, ΔV4], where ΔV1, ΔV2, ΔV3, and ΔV4 are the driving voltage compensation amounts corresponding to the first, second, third, and fourth partition electrodes, respectively. The actual driving voltage of each partition is obtained by superimposing the theoretical driving voltage corresponding to the target focal length with the corresponding driving voltage compensation amount.

[0077] Preferably, the deep neural network model adopts a Physical Information Neural Network (PINN) structure. Specifically, it superimposes a physical constraint residual term on the mean square error loss (i.e., data fitting loss) so that the network output satisfies both data fitting and physical laws. The physical constraint residual term is constructed from the Young-Lippmann equation and / or the two-phase interface force balance equation characterizing the relationship between electrowetting contact angle and voltage, so that the output of the physical information neural network model satisfies physical constraints.

[0078] During the model training phase, for each training sample consisting of temperature, pitch angle, roll angle, and target focal length, the curvature distribution of the target interface and the target contact angle at each zone electrode boundary are first determined based on the target interface morphology corresponding to the target focal length and under the gravity direction determined by the pitch angle and roll angle, using the static force balance relationship of the two-phase interface. Taking the interface normal from liquid phase 1 to liquid phase 2 as positive, the static force balance relationship is:

[0079]

[0080] in, The pressure difference between liquid phase 1 and liquid phase 2 at the reference position; The density difference between the two liquid phases at temperature T; This is the gravitational acceleration vector converted to the liquid lens coordinate system based on the pitch and roll angles; Let k be the discrete point on the target interface. For reference position; The interfacial tension between two liquid phases; For the target interface in The signed curvature at that point.

[0081] The static force balance relationship is solved by combining the two-phase liquid volume conservation condition and the target surface shape constraint corresponding to the target focal length, and the target contact angle is determined by the intersection line between the target interface and the cavity wall corresponding to the i-th partition electrode. If the opposite interface normal is used, the signs of the pressure difference and curvature will be reversed simultaneously.

[0082] In one feasible implementation, within the operating voltage range that does not reach the electrowetting contact angle saturation, for the i-th partition electrode, the network-predicted drive voltage compensation amount is... With base drive voltage Superimpose and construct the following physical constraint residuals based on the Yang-Lippman relation. :

[0083]

[0084] in, The initial contact angle at temperature T when no driving voltage is applied; It is the vacuum permittivity; denoted as the relative permittivity of the dielectric layer at temperature T; d represents the thickness of the dielectric layer. Predict the actual driving voltage for the network of the i-th partition electrode; This refers to the interfacial tension between liquid and liquid. , and This is a temperature function obtained by using measured values ​​or fitting measured data.

[0085] In one feasible implementation, for a mini-batch containing M training samples and each liquid lens containing N partitioned electrodes, the physical constraint residual loss is constructed using the normalized mean squared physical residual:

[0086]

[0087] in, The normalization scale for the physical residuals can be the root mean square of the physical residuals in the training set or a preset feature scale.

[0088] In one feasible implementation, the data fitting loss and the physical constraint residual loss are summed according to their weights to obtain the total loss. ,in, and These are non-negative weighting coefficients, which can be determined based on the amount of data, the dimensions of the physical residuals, and the validation set error. The data fitting loss is the mean squared error (MSE).

[0089] The above model was implemented using the TensorFlow / Keras framework in Python, employing the Adam optimizer. When using a physical information neural network structure, the total loss was used as the optimization objective. The batch size was set to 32, and the epochs were 500. An early stopping mechanism was also implemented to monitor the validation set loss in real time. If the validation set loss no longer improved after 30 consecutive epochs, training was stopped to prevent overfitting. The learning rate decay strategy used the ReduceLROnPlateau function, which automatically halves the learning rate when the validation loss stops improving.

[0090] During training, backpropagation is performed based on the total loss to iteratively update the weights and biases of each fully connected layer until the total loss converges or a preset stopping condition is met. Figure 5 The example uses four partitioned electrodes and their compensation voltages ΔV1 to ΔV4. When the liquid lens has N independently driveable partitioned electrodes, the output layer has N output neurons. After the model training is completed and deployed, the real-time control phase only performs forward propagation from the input layer to the output layer, without performing loss calculation or backpropagation.

[0091] Calculate the coefficient of determination R on the independent test set. 2 Indicators such as mean square error (MSE) and mean absolute error (MAE). When R... 2 When the prediction error (MAE) of the driving voltage compensation is ≥0.97 and ≤0.5V, the model is deemed to meet the deployment accuracy requirements. After conversion using TensorFlow Lite or ONNX format, it is deployed to an embedded processor (such as ARM Cortex-A55 or NVIDIA Jetson Nano). The embedded processor continuously receives the outputs from the temperature and attitude sensors at a preset sampling period and outputs the driving voltage compensation value to the drive control unit in real time via a feedforward method. Preferably, verification simulations are performed at independent simulation points not involved in the training to confirm the error between the neural network prediction results and the COMSOL simulation results, thereby comprehensively verifying the reliability of the surrogate model.

[0092] In terms of the compensation mechanism, when the optical axis of the liquid lens deviates from the direction of gravity by an angle of θ, the two-phase liquid in the cavity accumulates along the tilted orientation under the influence of the tangential component of gravity g·sinθ. The offset of the interface relative to the ideal sphere increases approximately with sinθ and is modulated by the density difference Δρ between the two phases, the interfacial tension γ, and the temperature. This results in an asymmetrical distribution in orientation, effectively introducing off-axis aberrations dominated by coma and astigmatism. Correspondingly, the required compensation voltage is also not uniform in orientation. Specifically, the partition electrode facing the direction of gravity sinking needs to apply a compensation voltage of different amplitude to its opposite partition to locally lower or raise the liquid surface in the corresponding orientation, counteract the gravity accumulation, and restore the liquid surface to near the ideal sphere. Therefore, the compensation amount is not a single scalar bias, but a partition-wise structured voltage vector that corresponds one-to-one with the attitude orientation. This nonlinear orientation-voltage mapping, which varies with temperature and attitude coupling, is difficult to describe precisely using analytical expressions or lookup tables. This is the fundamental reason why this embodiment uses multi-partition driving electrodes and fits this high-dimensional mapping with a deep neural network model.

[0093] In addition, the output layer can also adopt a physical decoupling structure, first outputting the defocus term and the Zernike aberration mode coefficients corresponding to coma and astigmatism, and then converting them into a partition-by-partition compensation vector through inverse mapping by the actuator.

[0094] Structurally, this invention integrates an attitude sensor, an existing temperature sensor, and a multi-zone driving electrode into a liquid lens module, enabling the liquid lens to simultaneously possess temperature and attitude sensing capabilities. The multi-zone driving electrode provides an execution means for asymmetric liquid surface reshaping. In terms of control, it employs an intelligent algorithm capable of simultaneously handling multi-variable coupled nonlinear mappings such as temperature, attitude, and target focal length to achieve millisecond-level feedforward compensation, thereby comprehensively solving problems such as lack of attitude sensing, single control strategy, and lag response.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for controlling a liquid lens, characterized in that, Includes the following steps: The system acquires the real-time operating temperature, real-time attitude, and target focal length of each liquid lens and generates real-time state parameters for each liquid lens. The control parameter prediction model predicts the driving voltage compensation value of the liquid lens based on the real-time state parameters of each liquid lens. Then, combined with the theoretical driving voltage of the liquid lens, the system generates the actual driving voltage of the liquid lens and applies it to the liquid lens.

2. The method for controlling a liquid lens according to claim 1, characterized in that, The control parameter prediction model is a pre-trained deep neural network model.

3. The method for controlling a liquid lens according to claim 2, characterized in that, The deep neural network model is a physical information neural network model, and its loss function during the pre-training process includes data fitting loss and physical constraint residual terms.

4. The method for controlling a liquid lens according to claim 1, characterized in that, The liquid lens includes a liquid lens body, a temperature sensor, and an attitude sensor. The temperature sensor is used to collect the operating temperature of the liquid lens body, and the attitude sensor is used to collect the attitude of the liquid lens body.

5. The method for controlling a liquid lens according to claim 4, characterized in that, The driving electrode at the liquid lens body consists of multiple electrodes, with different electrodes acting on different sections of the liquid lens body.

6. The method for controlling a liquid lens according to claim 1, characterized in that, The driving voltage compensation value is a voltage trajectory that varies with time. This voltage trajectory includes a voltage pre-shaping segment and a voltage steady-state segment. The voltage pre-shaping segment is used to shorten the time for the liquid surface of the liquid lens to establish the target surface shape and suppress oscillations.

7. A control system for a liquid lens, characterized in that, include: The state data acquisition unit is used to acquire the real-time operating temperature, real-time attitude, and target focal length of each liquid lens and generate the state parameters of each liquid lens. The compensation data prediction unit is used to predict the driving voltage compensation value of each liquid lens based on the state parameters of each liquid lens using a control parameter prediction model. The driving voltage generation unit is used to generate the actual driving voltage of each liquid lens based on the driving voltage compensation value and the theoretical driving voltage of each liquid lens. The drive control unit is used to control the corresponding liquid lens based on the actual drive voltage of each liquid lens.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the control method for a liquid lens as described in claim 1.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the control method for a liquid lens as described in claim 1.

10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the control method for a liquid lens as described in claim 1.

Images