Adaptive optical anti-shake control method for camera module

Abstract

The invention discloses an adaptive optical anti-shake control method for a camera module. The adaptive optical anti-shake control method comprises the following steps: S1, collecting real-time vibration data of the camera module in a working process; s2, preprocessing the real-time vibration data to obtain pose change information; s3, constructing a global search model, and defining an anti-shake compensation parameter search space; s4, performing multiple rounds of global iteration by using an improved seagull optimization algorithm, and determining a candidate parameter solution and a candidate region; s5, performing local sampling and iterative tuning by adopting a Bayesian optimization algorithm to obtain an optimal anti-shake compensation parameter; s6, the optimal anti-shake compensation parameters are issued to a camera module control unit, and an optical anti-shake assembly is driven to adjust the position or angle of a lens group; and S7, carrying out quality evaluation on the compensated image, and triggering whether to carry out global search or local optimization again or not. According to the invention, by constructing an adaptive optimization control mechanism, high-precision and low-delay real-time optical anti-shake compensation of the camera module in a dynamic environment is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and control, and particularly to an adaptive optical image stabilization control method for a camera module. Background Art

[0002] With the continuous development of application fields such as mobile devices, drones, and vehicle-mounted monitoring, the requirements for image quality and shooting stability are increasing day by day. Traditional camera modules are prone to problems such as image blurring and defocusing under high-speed movement or vibration conditions. Therefore, optical image stabilization technology has been widely used in the prior art to improve the imaging effect. Optical image stabilization technology usually relies on the movement or adjustment of the lens group in the mechanical structure, and cooperates with a certain control algorithm to compensate the camera module, so that the displacement caused by hand shake or movement can be corrected to a certain extent. However, the existing optical image stabilization control methods have defects such as insufficient response speed, low adjustment accuracy, and limited real-time adaptability when dealing with vibration problems in complex dynamic environments. Especially when facing vibrations with high nonlinearity and uncertainty, traditional methods usually adopt fixed parameters or simple feedback adjustment algorithms, which are difficult to simultaneously consider global search and local optimization, resulting in large fluctuations in the anti-shake compensation effect in different vibration environments and unable to effectively improve the stability and clarity of images.

[0003] Most of the control methods in the prior art rely on preset compensation parameters and perform fine adjustment of the position or angle of the lens group through a mechanical actuator. However, this method often lacks an adaptive adjustment mechanism and is difficult to cope with environmental changes and variable vibration modes. For example, some technical solutions adopt fixed PID control algorithms or simple fuzzy control strategies. Although anti-shake compensation can be achieved to a certain extent, due to the fact that parameter adjustment does not fully consider the pose change of the camera module in a dynamic scene and the actual feedback of the image stability index, the image anti-shake effect cannot be stably maintained in the best state in different scenes. In addition, some methods only rely on single-sensor data for compensation control and fail to achieve the fusion of multi-sensor data, resulting in problems such as slow response and insufficient compensation accuracy when dealing with complex vibration signals.

[0004] In the current technology, although some documents have proposed methods to optimize anti-shake compensation based on adaptive control algorithms, neural networks, and deep reinforcement learning, etc., these methods have defects such as high computational complexity, demanding hardware requirements, and poor real-time performance. Especially when the camera module is applied to scenarios with high requirements for volume, power consumption, and response speed, such as smartphones or drones, the limitations of traditional methods are more obvious. Existing technologies often have difficulty meeting the real-time requirements while ensuring high-precision compensation, resulting in a large gap between the compensation effect and the actual image quality. Furthermore, since image anti-shake involves the intersection of multiple disciplines such as optics, mechanics, electronics, and algorithms, traditional methods often cannot fully consider the synergistic effects between various modules during the implementation process and lack a comprehensive design scheme for global optimization.

[0005] In addition, with the continuous expansion of the application scenarios of camera modules, higher requirements are put forward for image stability. For example, in aspects such as dynamic compensation in complex environments, real-time closed-loop feedback, and image quality evaluation based on multiple metrics, the deficiencies of existing technologies are more prominent. Although some technical solutions attempt to introduce advanced optimization algorithms and adaptive control strategies, due to the lack of targeted design of the algorithms themselves and the failure to fully utilize the real-time feedback information during the image acquisition process of the camera module, it is difficult to achieve a balance between global search and local optimization. Especially in a high-speed dynamic environment, the anti-shake compensation parameters need to be updated in real time. The parameter search space definition and update strategy of existing methods are relatively fixed, lacking a dynamic adaptive adjustment mechanism, resulting in obvious blurring and jitter phenomena in the image during the compensation process.

[0006] Therefore, how to provide an adaptive optical anti-shake control method for camera modules is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0007] An object of the present invention is to propose an adaptive optical anti-shake control method for camera modules. The present invention makes full use of multi-sensor data fusion, extended Kalman filtering, an improved seagull optimization algorithm, and a Bayesian optimization algorithm, and details how to implement closed-loop adaptive optical anti-shake control based on real-time image feedback, global search, and local fine-tuning. This method can not only quickly respond to environmental vibrations but also adjust anti-shake compensation parameters in real time, thereby significantly improving the stability and clarity of images and having the advantages of fast response speed, high compensation accuracy, and strong adaptability.

[0008] An adaptive optical anti-shake control method for a camera module according to an embodiment of the present invention includes the following steps:

[0009] S1. Collect real-time vibration data during the operation of the camera module. The real-time vibration data consists of triaxial angular velocity and acceleration information output by a gyroscope and an accelerometer;

[0010] S2. Preprocess the collected real-time vibration data using the extended Kalman filter algorithm to eliminate noise and obtain pose change information;

[0011] S3. Based on the obtained pose change information and a preset image stability index, construct a global search model and define the search space for anti-shake compensation parameters in the global search model;

[0012] S4. Use the improved seagull optimization algorithm to perform multiple rounds of global iteration on the search space, simulate the hovering and diving predation behaviors of seagulls, quickly screen out candidate parameter solutions, and determine the candidate area;

[0013] S5. In the candidate area, use the Bayesian optimization algorithm for local sampling and iterative tuning to obtain the optimal anti-shake compensation parameters;

[0014] S6. Send the output optimal anti-shake compensation parameters to the camera module control unit to drive the optical anti-shake component to adjust the position or angle of the lens group in real time and compensate the image;

[0015] S7. Perform real-time quality evaluation on the compensated image, extract sharpness, edge sharpness, and motion blur metrics, compare them with preset thresholds, and automatically trigger re-global search or local tuning according to the evaluation results to form a closed-loop adaptive control.

[0016] Optionally, the S2 specifically includes:

[0017] S21. Set the current time as k, input the three-axis angular velocity and acceleration data obtained by the gyroscope and accelerometer, and construct a system state variable, where the system state variable includes the linear acceleration, angular velocity, displacement, and attitude angle information of the camera module;

[0018] S22. According to the state prediction mechanism of the extended Kalman filter algorithm, combine the state estimate value of the previous moment and the control input to predict the state of the current moment and obtain the state prediction value and the corresponding state prediction error covariance matrix P k|k-1 :

[0019]

[0020] where F k-1 is the Jacobian matrix of the state transition function f with respect to the state variable, P k-1|k-1 is the error covariance matrix of the previous moment, and Q k-1 is the process noise covariance matrix;

[0021] S23. Use the measurement function to map the state prediction value to the measurement space to obtain the predicted measurement value of the current moment and compare it with the actual measurement value zk Compare to form the observation residual ∈ k ;

[0022] S24. Generate the Kalman gain K based on the state prediction error covariance matrix and the preset measurement noise covariance matrix k :

[0023]

[0024] where, H k is the Jacobian matrix of the measurement function with respect to the state variable, and R k is the measurement noise covariance matrix;

[0025] S25. Correct the state prediction value according to the Kalman gain, and combine the observation residual ∈ k Update the state estimate to obtain the optimal state estimate value at the current moment

[0026]

[0027] S26. Output the displacement and attitude angle information included in the finally obtained optimal state estimate value as the pose change information of the camera module

[0028] Optionally, the S3 specifically includes:

[0029] S31. Use the pose change information output by the extended Kalman filter algorithm as the state input, including the displacement, rotation angle, acceleration, and angular velocity of the camera module within a continuous time window

[0030] S32. Set the image stability index as the basis of the objective function. The image stability index includes image clarity, motion blur degree, and edge sharpness

[0031] S33. Combine the pose change data and the image stability index to construct a global search model and establish a mapping relationship between pose perturbation and image stability

[0032] S34. Set the search dimension of the anti-shake compensation parameter in the global search model. The anti-shake compensation parameter includes the displacement amplitude of the lens group, the adjustment angle, the compensation response time, and the control step

[0033] S35. Define the value range and boundary conditions of each anti-shake compensation parameter, and combine the mechanism limitations, control accuracy, and response time of the camera module to form the search space of the anti-shake compensation parameter

[0034] S36. Initialize the population distribution strategy in the search space of the anti-shake compensation parameter, and determine the initial value set and fitness evaluation structure of each parameter dimension

[0035] Optionally, S4 specifically includes:

[0036] S41. Initialize a population in the search space of the anti-shake compensation parameters, where each individual is represented as an anti-shake compensation parameter vector:

[0037]

[0038] where d represents the parameter dimension, represents the anti-shake compensation parameter vector of the i-th candidate solution in the population at the initial moment, represents the initial value of the d-th component or dimension of the i-th candidate solution parameter vector;

[0039] S42. Construct a fitness function based on a preset image stability index, and perform fitness evaluation on each individual. The fitness function f(X i ) is based on image sharpness, edge sharpness, and motion blur metrics:

[0040] f(X i ) = γ1·C(X i ) + γ2·E(X i ) - γ3·B(X i );

[0041] where C(X i ) represents the sharpness metric of the image after applying the anti-shake compensation parameter X i , E(X i ) represents the edge sharpness metric of the image, B(X i ) represents the motion blur metric of the image, and γ1, γ2, γ3 are preset weight coefficients;

[0042] S43. In each iteration, perform an improved seagull hovering behavior, and its improvement includes:

[0043] Generate local gradient information based on the position information and local fitness change trend between the current individual and the global optimal individual

[0044] Perform optical flow analysis on the real-time image sequence output by the camera, extract image motion perturbation information, and generate real-time image optical flow feedback

[0045] S44. According to the improved seagull hovering behavior, fuse the global guidance direction, local gradient information, and real-time image optical flow feedback to calculate an intermediate update solution

[0046]

[0047] Among them, α is the global search step size coefficient, and w1, w2, and w3 are adaptive weight factors. represents the individual with the best fitness in the t-th round of iteration. represents the current solution of the i-th candidate solution at the t-th round of iteration. w1 is the weight factor of the global guiding direction, w2 is the weight factor of the local gradient information, and w3 is the weight factor of the real-time image optical flow feedback.

[0048] S45. Update the solution in the middle Based on this, simulate the seagull's diving and predation behavior to perform local search and further correct the position of the candidate solution:

[0049]

[0050] Among them, δ is the initial diving step size coefficient, λ is the step size attenuation factor, and t is the current number of iterations. represents the final updated position of the i-th candidate solution after the (t + 1)-th round of iteration, and exp is the exponential function.

[0051] S46. Re-evaluate the fitness of each updated individual according to the fitness function f(X i ) to screen out the candidate parameter solutions with higher fitness and determine the region where they are located as the candidate region.

[0052] S47. Repeat the iterative process of S43 to S46 until the preset termination condition is met, and finally output the selected candidate parameter solutions and their candidate regions.

[0053] Optionally, the S5 specifically includes:

[0054] S51. Initialize the Bayesian optimization algorithm within the determined candidate region and set the candidate anti-shake compensation parameter vector x as the variable to be optimized.

[0055] S52. Construct a Gaussian process regression model and use the Gaussian process regression method to model the anti-shake compensation parameters within the candidate region:

[0056] Select the sample set within the candidate region where f(x i ) represents the actual evaluation value based on the image stability index, x i represents the i-th candidate anti-shake compensation parameter vector, and n is the number of sample points.

[0057] Use the sample set within the candidate region to train the Gaussian process regression model to obtain the predicted mean μ(x) and the predicted standard deviation σ(x).

[0058] S53. Define the standard expected acquisition function EI(x) to measure the improvement potential of the current candidate anti-shake compensation parameter in the Gaussian process regression model.

[0059]

[0060] Among them, f(x + ) represents the optimal image stability index value, Φ() is the standard normal cumulative distribution function, and φ() is the standard normal probability density function;

[0061] S54. Combine the standard expected acquisition function, local gradient information, and the image stability improvement value based on real-time image optical flow feedback to construct the comprehensive acquisition function A(x):

[0062]

[0063] Among them, η and η2 are adjustment coefficients representing the local gradient and the image optical flow feedback weight respectively, is the gradient of the prediction mean function, and Θ(x) represents the image stability improvement value calculated based on real-time image optical flow feedback;

[0064] S55. Select the parameter vector x that maximizes the comprehensive acquisition function A(x) within the candidate region next ;

[0065] S56. Obtain the actual image stability index value f(x next ) at the selected parameter vector point of the maximum value, and use the new sample (x next , f(x next )) to update the Gaussian process regression model and correct the prediction mean μ(x) and the prediction standard deviation σ(x);

[0066] S57. Repeat the sampling and Gaussian process regression model update processes of S53 to S56 until the preset number of iterations or convergence conditions are reached, and finally output the optimal anti-shake compensation parameter x with the highest image stability index * .

[0067] Optionally, the specific content of S6 includes:

[0068] S61. Receive the optimal anti-shake compensation parameter output by the Bayesian optimization algorithm. The optimal anti-shake compensation parameter includes lens group displacement, adjustment angle, response delay, and execution step control variables;

[0069] S62. Convert the optimal anti-shake compensation parameter into the standard control instruction format and transmit it to the camera module control unit through the communication interface;

[0070] S63. The camera module control unit analyzes the received optimal anti-shake compensation parameter and generates the corresponding drive control signal;

[0071] S64. Drive the actuator in the optical image stabilization component with a drive control signal to adjust the position and angle of the lens group in real time to match the current jitter direction and amplitude;

[0072] S65. After the lens group completes the physical position or angle adjustment, perform a real-time compensation operation on the current image of the camera;

[0073] S66. Record the adjustment results and the current image state, and perform dynamic tracking and feedback on the anti-shake compensation effect.

[0074] The beneficial effects of the present invention are as follows:

[0075] By combining multi-sensor data fusion, extended Kalman filter, improved seagull optimization algorithm and Bayesian optimization algorithm, the present invention realizes the global search and local fine tuning of the adaptive optical image stabilization control of the camera module, significantly improving the response speed and regulation accuracy of the image anti-shake system. The extended Kalman filter algorithm is used to preprocess the pose change information of the camera module in a dynamic environment in real time and accurately, providing a reliable input data basis for subsequent parameter optimization. In the global search stage, the improved seagull optimization algorithm effectively overcomes the problems of fixed parameter update and insufficient adaptability in traditional methods by fusing the global guiding direction, local gradient information and real-time image optical flow feedback, enabling the anti-shake compensation parameters to quickly lock in the best candidate area; in the local optimization stage, the Bayesian optimization algorithm uses the surrogate model constructed by Gaussian process regression, combined with the acquisition function and local gradient information, to achieve efficient sampling and fine tuning of the candidate parameters, thus ensuring the continuous improvement of the image stability index.

[0076] This method has significant advantages in terms of real-time performance and accuracy, and can automatically adjust the optical image stabilization parameters under complex dynamic vibration conditions to compensate for the image jitter and blur caused by the movement of the camera module in real time. Through the closed-loop adaptive control combining global and local optimization, the present invention not only solves the problems of slow response and insufficient compensation of traditional anti-shake systems in variable environments, but also greatly improves the image clarity and stability, thus meeting the stringent requirements for high-quality image acquisition in fields such as smart phones, drones, vehicle-mounted cameras and security monitoring.

[0077] In summary, the beneficial effects of the present invention lie in that through an advanced algorithm combination, it makes full use of real-time sensor data and image feedback to establish an optical image stabilization control system with high adaptability and precise regulation ability, realizing a significant improvement in the image anti-shake compensation effect of the camera module in complex dynamic environments, and providing a stable, reliable and efficient solution for practical applications. Brief Description of the Drawings

[0078] The accompanying drawings are used to provide a further understanding of the present invention and form a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the accompanying drawings:

[0079] Figure 1 is a flowchart of an adaptive optical image stabilization control method for a camera module proposed by the present invention;

[0080] Figure 2 is a schematic diagram of an improved seagull optimization algorithm for an adaptive optical image stabilization control method of a camera module proposed by the present invention. Specific embodiments

[0081] Now, the present invention will be further described in detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0082] Refer to Figure 1 and Figure 2 , an adaptive optical image stabilization control method for a camera module, comprising the following steps:

[0083] S1. Collect the real-time vibration data of the camera module during operation. The real-time vibration data consists of the three-axis angular velocity and acceleration information output by the gyroscope and accelerometer;

[0084] S2. Use the extended Kalman filter algorithm to preprocess the collected real-time vibration data, eliminate noise and obtain the pose change information;

[0085] S3. Based on the obtained pose change information and the preset image stability index, construct a global search model and define the search space for the anti-shake compensation parameters in the global search model;

[0086] S4. Use the improved seagull optimization algorithm to perform multiple rounds of global iteration on the search space, simulate the hovering and diving predation behaviors of seagulls, quickly screen out the candidate parameter solutions and determine the candidate areas;

[0087] S5. Use the Bayesian optimization algorithm to perform local sampling and iterative optimization within the candidate areas to obtain the optimal anti-shake compensation parameters;

[0088] S6. Send the output optimal anti-shake compensation parameters to the camera module control unit to drive the optical image stabilization component to adjust the position or angle of the lens group in real time and compensate the image;

[0089] S7. Perform real-time quality evaluation on the compensated image, extract the clarity, edge sharpness and motion blur indicators, compare them with the preset thresholds, and automatically trigger re-global search or local optimization according to the evaluation results to form a closed-loop adaptive control.

[0090] In this embodiment, S2 specifically includes:

[0091] S21. Set the current time as k, input the three-axis angular velocity and acceleration data obtained by the gyroscope and accelerometer, and construct the system state variables, where the system state variables include the linear acceleration, angular velocity, displacement, and attitude angle information of the camera module;

[0092] S22. According to the state prediction mechanism of the extended Kalman filter algorithm, combine the state estimate value of the previous moment and the control input to predict the state of the current moment, and obtain the state prediction value and the corresponding state prediction error covariance matrix P k|k-1 :

[0093]

[0094] where F k-1 is the Jacobian matrix of the state transition function f with respect to the state variables, P k-1|k-1 is the error covariance matrix of the previous moment, and Q k-1 is the process noise covariance matrix;

[0095] S23. Use the measurement function to map the state prediction value to the measurement space to obtain the predicted measurement value of the current moment and compare it with the actual measurement value z k to form the observation residual ∈ k ;

[0096] S24. Generate the Kalman gain K based on the state prediction error covariance matrix and the pre-set measurement noise covariance matrix k :

[0097]

[0098] where H k is the Jacobian matrix of the measurement function with respect to the state variables, and R k is the measurement noise covariance matrix;

[0099] S25. Correct the state prediction value according to the Kalman gain, and combine the observation residual ∈ k to update the state estimate and obtain the optimal state estimate value of the current moment

[0100]

[0101] S26. Output the displacement and attitude angle information included in the finally obtained optimal state estimate value as the pose change information of the camera module.

[0102] In this embodiment, S3 specifically includes:

[0103] S31. Take the pose change information output by the extended Kalman filter algorithm as the state input, including the displacement, rotation angle, acceleration, and angular velocity of the camera module within a continuous time window;

[0104] S32. Set the image stability index as the basis for the objective function. The image stability index includes image sharpness, motion blur degree, and edge sharpness;

[0105] S33. Combine the pose change data and the image stability index to construct a global search model and establish a mapping relationship between pose perturbation and image stability;

[0106] S34. Set the search dimension of the anti-shake compensation parameter in the global search model. The anti-shake compensation parameter includes the displacement amplitude of the lens group, adjustment angle, compensation response time, and control step;

[0107] S35. Define the value range and boundary conditions of each anti-shake compensation parameter, and combine the mechanical limitations, control accuracy, and response time of the camera module to form the search space of the anti-shake compensation parameter;

[0108] S36. Initialize the population distribution strategy in the search space of the anti-shake compensation parameter, and determine the initial value set and fitness evaluation structure of each parameter dimension.

[0109] In this embodiment, the specific content of S4 includes:

[0110] S41. Initialize the population in the search space of the anti-shake compensation parameter. Each individual is represented as an anti-shake compensation parameter vector:

[0111]

[0112] where d represents the parameter dimension, represents the anti-shake compensation parameter vector of the i-th candidate solution in the population at the initial moment, represents the initial value of the d-th component or dimension of the i-th candidate solution parameter vector;

[0113] S42. Construct a fitness function based on the preset image stability index, and perform fitness evaluation on each individual. The fitness function f(X i ) is based on the image sharpness, edge sharpness, and motion blur index:

[0114] f(X i ) = γ1·C(X i ) + γ2·E(X i ) - γ3·B(X i );

[0115] where C(Xi ) indicates the application of the anti-shake compensation parameter X i The clarity index of the post-image, E(X i ) represents the edge sharpness index of the image, B(X i ) represents the motion blur index of the image, and γ1, γ2, γ3 are preset weight coefficients;

[0116] S43. In each iteration, perform the improved seagull hovering behavior, and its improvement includes:

[0117] Generate local gradient information according to the position information and local fitness change trend between the current individual and the global optimal individual

[0118] Perform optical flow analysis on the real-time image sequence output by the camera, extract the image motion disturbance information, and generate the real-time image optical flow feedback

[0119] S44. According to the improved seagull hovering behavior, fuse the global guiding direction, local gradient information, and real-time image optical flow feedback to calculate the intermediate update solution

[0120]

[0121] Among them, α is the global search step coefficient, and w1, w2, w3 are adaptive weight factors, represents the individual with the optimal fitness in the t-th iteration, represents the current solution of the i-th candidate solution in the t-th iteration, w1 is the weight factor of the global guiding direction, w2 is the weight factor of the local gradient information, and w3 is the weight factor of the real-time image optical flow feedback;

[0122] S45. Based on the intermediate update solution Perform local search by simulating the seagull diving and predation behavior to further correct the candidate solution position:

[0123]

[0124] Among them, δ is the initial diving step coefficient, λ is the step decay factor, and t is the current iteration number, represents the final updated position of the i-th candidate solution after the (t + 1)-th iteration, and exp is the exponential function;

[0125] S46. Re-evaluate the fitness of each updated individual according to the fitness function f(X i ) to screen out the candidate parameter solutions with higher fitness, and determine the region where they are located as the candidate region;

[0126] S47. Repeatedly execute the iterative process from S43 to S46 until a preset termination condition is met, and finally output the selected candidate parameter solutions and their candidate regions.

[0127] In this embodiment, S5 specifically includes:

[0128] S51. Initialize the Bayesian optimization algorithm within the determined candidate region, and set the candidate anti-shake compensation parameter vector x as the variable to be optimized;

[0129] S52. Construct a Gaussian process regression model, and use the Gaussian process regression method to model the anti-shake compensation parameters within the candidate region:

[0130] Select a sample set within the candidate region where f(x i ) represents the actual evaluation value based on the image stability index, x i represents the i-th candidate anti-shake compensation parameter vector, and n is the number of sample points;

[0131] Use the sample set within the candidate region to train the Gaussian process regression model to obtain the predicted mean μ(x) and the predicted standard deviation σ(x);

[0132] S53. Define the standard expected acquisition function EI(x) to measure the improvement potential of the current candidate anti-shake compensation parameter in the Gaussian process regression model:

[0133]

[0134] where, f(x + ) represents the best image stability index value, Φ() is the standard normal cumulative distribution function, and φ() is the standard normal probability density function;

[0135] S54. Combine the standard expected acquisition function, local gradient information, and the image stability improvement value based on real-time image optical flow feedback to construct the comprehensive acquisition function A(x):

[0136]

[0137] where, η and η2 are adjustment coefficients representing the local gradient and the image optical flow feedback weight respectively, is the gradient of the predicted mean function, and Θ(x) represents the image stability improvement value calculated based on real-time image optical flow feedback;

[0138] S55. Select the parameter vector x next ;

[0139] in the candidate region that maximizes the comprehensive acquisition function A(x) next ;

[0139] S56. Obtain the actual image stability index value f(xnext ), and update the Gaussian process regression model using the new sample (x next , f(x next )) to correct the predicted mean μ(x) and the predicted standard deviation σ(x);

[0140] S57. Repeat the sampling and Gaussian process regression model update processes of S53 to S56 until the preset number of iterations or convergence condition is reached, and finally output the optimal anti-shake compensation parameter x with the highest image stability index * .

[0141] In this embodiment, the S6 specifically includes:

[0142] S61. Receive the optimal anti-shake compensation parameter output by the Bayesian optimization algorithm. The optimal anti-shake compensation parameter includes lens group displacement, adjustment angle, response delay, and execution step control variables;

[0143] S62. Convert the optimal anti-shake compensation parameter into the standard control instruction format and transmit it to the camera module control unit through the communication interface;

[0144] S63. The camera module control unit analyzes the received optimal anti-shake compensation parameter and generates the corresponding drive control signal;

[0145] S64. The drive control signal drives the actuator in the optical image stabilization component to adjust the position and angle of the lens group in real time to match the current jitter direction and amplitude;

[0146] S65. After the lens group completes the physical position or angle adjustment, perform real-time compensation operation on the current image of the camera;

[0147] S66. Record the adjustment result and the current image state, and perform dynamic tracking and feedback on the anti-shake compensation effect.

[0148] Example 1:

[0149] To verify the feasibility of the present invention in implementation, the present invention is applied to a certain smart phone, and the smart phone is used to photograph moving vehicles and pedestrians. Due to the dual interference of human body vibration and vehicle driving bumps in the hand-held state of the smart phone, the traditional optical image stabilization technology often cannot accurately correct the position of the lens group in a short time, resulting in obvious image blurring. To solve this problem, the present invention uses an adaptive optical image stabilization control method combining multi-sensor data fusion, extended Kalman filter preprocessing, improved seagull optimization algorithm and Bayesian optimization algorithm to achieve real-time image feedback and closed-loop adaptive control.

[0150] First, the gyroscope and accelerometer built into the smartphone simultaneously collect the triaxial angular velocity and acceleration data of the device during movement. After being preprocessed by the extended Kalman filter algorithm, these data generate the pose change information of the camera module, including displacement, rotation angle, acceleration, and angular velocity, etc. The system uses these preprocessed data as input to evaluate the image stability metrics (such as image sharpness, edge sharpness, and motion blur metrics) in real time. After evaluation, the system determines that the current image stability is poor and enters the global search stage.

[0151] In the global search stage, using the improved seagull optimization algorithm, the system performs multiple rounds of global iteration on the search space of the anti-shake compensation parameters. This algorithm not only searches based on the relative position of the current candidate solution and the global optimal solution but also incorporates local gradient information and the image motion perturbation information obtained from the real-time image optical flow feedback to adjust the search direction of the candidate solution, making the parameter update more in line with the requirements of the actual dynamic environment. After several rounds of iteration, the system can quickly screen out a set of candidate parameter solutions and determine the candidate region, laying the foundation for subsequent local optimization.

[0152] Subsequently, the system enters the local optimization stage, and in the candidate region, it uses the Bayesian optimization algorithm for local sampling and iterative tuning. By constructing a Gaussian process regression surrogate model, the system predicts the image stability metrics using the initial sample data in the candidate region, and combines the standard expected improvement function, local gradient information, and real-time image optical flow feedback to construct a comprehensive acquisition function, selects the best sampling points for further optimization. After multiple samplings and model updates, finally, the system outputs the optimal anti-shake compensation parameters with the highest image stability metrics.

[0153] Finally, the system sends the optimal anti-shake compensation parameters to the camera module control unit, and by controlling the optical anti-shake component, it adjusts the position and angle of the lens group in real time to achieve effective compensation for the image. Through on-site testing, this method can reduce the image jitter amplitude by about 70% during dynamic shooting, improve the image sharpness by about 65%, and shorten the response time from 200 milliseconds of the traditional method to within 80 milliseconds, effectively meeting the shooting requirements in complex dynamic environments.

[0154] In actual tests, we conducted continuous tests at five different time periods (morning rush hour, noon, afternoon, evening, and night) in a bustling area of Chaoyang District, Beijing. For each period, we collected no less than 500 groups of data and compared the anti-shake effects of traditional anti-shake technologies and the method of the present invention. In terms of instantaneous response, the average response delay of the traditional method is 210 milliseconds, while the response delay of the method of the present invention is stably between 80 - 90 milliseconds, greatly improving the real-time compensation effect. In addition, during continuous shooting, the method of the present invention maintained a relatively low volatility of the image stability index, and the standard deviation of the test results was reduced by approximately 40% compared to the traditional method, indicating that both its adaptive control ability and robustness have been significantly improved.

[0155] Table 1 Statistical Table of Test Data for Image Stability and Response Time

[0156]

[0157] Table 1 shows the performance of the method of the present invention and the traditional anti-shake method under five typical time periods, including two core dimensions: the image clarity index and the response delay. It can be clearly seen from the data that the method of the present invention is superior to the traditional method in each time period, with significant image anti-shake compensation effects and fast response capabilities.

[0158] First, from the perspective of the image clarity index, the index range of the traditional method in the five time periods is between 0.61 and 0.68, which is generally low and is greatly affected by the vibration amplitude, showing obvious performance fluctuations. While the image clarity of the method of the present invention always remains above 0.89, reaching the highest value of 0.93 at noon and stabilizing at 0.94 at night, significantly superior to the traditional method. This indicates that in complex environments with weak light or strong vibration intensity, the present invention can still maintain a high level of image quality, demonstrating good stability and robustness.

[0159] Secondly, from the analysis of the response delay, the response time of the traditional method is between 200 and 220 milliseconds, with an average of about 210 milliseconds, which is relatively high and not conducive to rapid compensation. While the response delay of the method of the present invention is lower than 90 milliseconds in all time periods, reaching 80 milliseconds at the fastest during noon and night, and the slowest at 88 milliseconds, with the overall delay significantly shortened. This means that the camera module can complete parameter adjustment and lens group compensation actions in a shorter time, thereby achieving more timely image stable output.

[0160] It is worth noting that the vibration amplitudes are relatively large during the morning rush hour and evening, but the present invention still maintains the image clarity above 0.89 and controls the response delay within the range of 85 - 88 milliseconds, further verifying the high-performance performance of this method under high-dynamic interference conditions.

[0161] In summary, the data in Table 1 fully demonstrate that the proposed adaptive optical image stabilization control method for camera modules of the present invention is superior to traditional technologies in terms of both image sharpness improvement and response speed control, especially performing more excellently in high-vibration environments and having higher practicality and engineering application value.

[0162] As described above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes should be covered within the protection scope of the present invention.

Claims

1. An adaptive optical image stabilization control method for a camera module, characterized in that, It includes the following steps: S1. Collect the real-time vibration data of the camera module during operation. The real-time vibration data consists of the three-axis angular velocity and acceleration information output by the gyroscope and accelerometer; S2. Use the extended Kalman filter algorithm to preprocess the collected real-time vibration data, eliminate noise, and obtain pose change information; S3. Based on the obtained pose change information and the preset image stability index, construct a global search model, and define the search space of the anti-shake compensation parameters in the global search model; S4. Use the improved seagull optimization algorithm to perform multiple rounds of global iteration on the search space, simulate the seagull's hovering and diving predation behaviors, quickly screen out candidate parameter solutions, and determine the candidate area; S5. Use the Bayesian optimization algorithm to perform local sampling and iterative tuning within the candidate area to obtain the optimal anti-shake compensation parameters; S6. Send the output optimal anti-shake compensation parameters to the camera module control unit to drive the optical anti-shake component to adjust the position or angle of the lens group in real time to compensate for the image; S7. Perform real-time quality evaluation on the compensated image, extract the clarity, edge sharpness, and motion blur metrics, compare them with the preset thresholds, and automatically trigger re-global search or local tuning according to the evaluation results to form a closed-loop adaptive control.

2. The adaptive optical image stabilization control method for a camera module according to claim 1, wherein The specific content of S2 includes: S21. Set the current time as k, input the three-axis angular velocity and acceleration data obtained by the gyroscope and accelerometer, and construct a system state variable. The system state variable includes the linear acceleration, angular velocity, displacement, and attitude angle information of the camera module; S22. According to the state prediction mechanism of the extended Kalman filter algorithm, combine the state estimate value at the previous moment and the control input to predict the state at the current moment and obtain the state prediction value and the corresponding state prediction error covariance matrix P k|k-1 : where, F k-1 is the Jacobian matrix of the state transition function f with respect to the state variables, P k-1|k-1 is the error covariance matrix at the previous moment, Q k-1 is the process noise covariance matrix; S23. Map the state prediction value to the measurement space using a measurement function to obtain the predicted measurement value at the current moment and compare it with the actual measurement value z k to form an observation residual ∈ k ; S24. Generate the Kalman gain K based on the state prediction error covariance matrix and the preset measurement noise covariance matrix k : where, H k is the Jacobian matrix of the measurement function with respect to the state variables, and R k is the measurement noise covariance matrix; S25. Correct the state prediction value according to the Kalman gain, and combine the observation residual ∈ k Update the state estimate to obtain the optimal state estimate value at the current moment S26. Output the displacement and attitude angle information included in the finally obtained optimal state estimate value as the pose change information of the camera module.

3. The adaptive optical image stabilization control method for a camera module according to claim 1, wherein, The specific content of S3 includes: S31. Use the pose change information output by the extended Kalman filter algorithm as the state input, including the displacement, rotation angle, acceleration, and angular velocity of the camera module within a continuous time window; S32. Set the image stability index as the basis for the objective function. The image stability index includes image clarity, motion blur degree, and edge sharpness; S33. Combine the pose change data and the image stability index to construct a global search model and establish a mapping relationship between pose perturbation and image stability; S34. Set the search dimension of the anti-shake compensation parameters in the global search model. The anti-shake compensation parameters include the lens group displacement amplitude, adjustment angle, compensation response time, and control step; S35. Define the value range and boundary conditions of each anti-shake compensation parameter, and combine the mechanical limitations, control accuracy, and response time of the camera module to form the search space of the anti-shake compensation parameters; S36. Initialize the population distribution strategy in the search space of the anti-shake compensation parameters, and determine the initial value set and fitness evaluation structure for each parameter dimension.

4. A method for adaptively controlling optical image stabilization of a camera module according to claim 1, characterized in that, The specific content of S4 includes: S41. Initialize the population in the search space of the anti-shake compensation parameters. Each individual is represented as an anti-shake compensation parameter vector: where d represents the parameter dimension, represents the anti-shake compensation parameter vector of the i-th candidate solution in the population at the initial moment, represents the initial value of the d-th component or dimension of the parameter vector of the i-th candidate solution; S42. Construct a fitness function based on a preset image stability index to evaluate the fitness of each individual. The fitness function f(X i ) is based on image sharpness, edge sharpness, and motion blur metrics: f(X i ) = γ1·C(X i ) + γ2·E(X i ) - γ3·B(X i ); Among them, C(X i ) represents the clarity index of the image after applying the anti-shake compensation parameter X i , E(X i ) represents the edge sharpness index of the image, B(X i ) represents the motion blur index of the image, and γ1, γ2, γ3 are preset weight coefficients; S43. In each round of iteration, perform the improved seagull hovering behavior. The improvement includes: Generate local gradient information based on the position information and local fitness change trend between the current individual and the global optimal individual Perform optical flow analysis on the real-time image sequence output by the camera, extract image motion perturbation information, and generate real-time image optical flow feedback S44. According to the improved seagull hovering behavior, fuse the global guiding direction, local gradient information, and real-time image optical flow feedback to calculate the intermediate updated solution where α is the global search step coefficient, and w1, w2, and w3 are adaptive weight factors. represents the individual with the optimal fitness in the t-th iteration. represents the current solution of the i-th candidate solution in the t-th iteration. w1 is the weight factor of the global guiding direction, w2 is the weight factor of the local gradient information, and w3 is the weight factor of the real-time image optical flow feedback. S45. Update the solution in the middle On this basis, simulate the seagull's diving and predation behavior to perform local search and further correct the position of the candidate solution: where δ is the initial dive step coefficient, λ is the step decay factor, and t is the current iteration number, represents the final updated position of the i-th candidate solution after the (t + 1)-th round of iteration, and exp is the exponential function; S46. Re-evaluate the fitness of each updated individual according to the fitness function f(X i ), screen out candidate parameter solutions with higher fitness, and determine the region where they are located as the candidate region; S47. Repeat the iterative process from S43 to S46 until a preset termination condition is met, and finally output the selected candidate parameter solutions and their candidate regions.

5. A method for adaptively optically stabilizing a camera module according to claim 1, characterized in that, The specific steps of S5 are as follows: S51. Initialize the Bayesian optimization algorithm within the determined candidate region, and set the candidate anti-shake compensation parameter vector x as the variable to be optimized. S52. Construct a Gaussian process regression model, and use the Gaussian process regression method to model the anti-shake compensation parameters within the candidate region: Select a sample set within the candidate region where f(x i ) represents the actual evaluation value based on the image stability index, x i represents the i-th candidate anti-shake compensation parameter vector, and n is the number of sample points; Train the Gaussian process regression model using the sample set within the candidate region to obtain the predicted mean μ(x) and the predicted standard deviation σ(x). S53. Define the standard expected improvement function EI(x) to measure the improvement potential of the current candidate anti-shake compensation parameters in the Gaussian process regression model. where f(x + ) represents the optimal image stability index value, Φ() is the standard normal cumulative distribution function, and φ() is the standard normal probability density function; S54. Combine the standard expected improvement function, local gradient information, and the image stability improvement value based on real-time image optical flow feedback to construct the comprehensive acquisition function A(x). where η and η2 are adjustment coefficients respectively representing the local gradient and the image optical flow feedback weight, is the gradient of the predicted mean function, and Θ(x) represents the improved value of image stability calculated based on the real-time image optical flow feedback; S55. Select the parameter vector x that maximizes the comprehensive acquisition function A(x) within the candidate region next ; S56. Obtain the actual image stability index value f(x next ) at the parameter vector point of the selected maximum value, and update the Gaussian process regression model using the new sample (x next , f(x next )) to correct the predicted mean μ(x) and the predicted standard deviation σ(x); Repeat the sampling and Gaussian process regression model update processes of S53 to S56 until a preset number of iterations or convergence conditions are reached, and finally output the optimal anti-shake compensation parameter x with the highest image stability index * .

6. A method for adaptively optically stabilizing a camera module according to claim 1, characterized in that, The specific steps of S6 are as follows: S61. Receive the optimal anti-shake compensation parameters output by the Bayesian optimization algorithm, where the optimal anti-shake compensation parameters include lens group displacement, adjustment angle, response delay, and execution step control variables. S62. Convert the optimal anti-shake compensation parameters into the standard control instruction format and transmit them to the camera module control unit through the communication interface. S63. The camera module control unit analyzes the received optimal anti-shake compensation parameters and generates corresponding drive control signals. S64. The drive control signals drive the actuators in the optical image stabilization component to adjust the position and angle of the lens group in real time to match the current jitter direction and amplitude. S65. After the lens group completes the physical position or angle adjustment, perform real-time compensation operations on the current image of the camera. S66. Record the adjustment results and the current image state, and perform dynamic tracking and feedback on the anti-shake compensation effect.

Images