Image optical anti-shake method for endoscope camera system

Abstract

The invention provides an image optical anti-shake method for an endoscope camera system. The method comprises the following steps: acquiring motion data of an endoscope camera in real time; based on the collected motion data, using an image processing algorithm to analyze and identify the mode and degree of image shake, and calculating the compensation amount required by optical and electronic anti-shake; a high-precision optical anti-shake assembly is integrated, and according to the calculated compensation amount, an optical path or the position of a lens is rapidly adjusted through the optical anti-shake assembly so as to compensate for image offset caused by movement of the camera; an electronic anti-shake assembly is integrated, reading data of an image sensor or parameters of an image processing chip are adjusted through an electronic control system, and residual shake is further eliminated; sensor data and image quality are monitored in real time, optical anti-shake parameters are dynamically adjusted, and continuous image stability and clearness are ensured. Real-time, stable and clear output of images is achieved through the optical technology, and a better auxiliary diagnosis means can be provided for doctors.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscope camera systems, and in particular to an image optical anti-shake method for endoscope camera systems. Background Art

[0002] Endoscopic camera systems play a vital role in modern medical diagnosis and surgery. The stability and clarity of their images directly affect the accuracy of diagnosis and the safety of surgery. However, traditional endoscope camera systems are easily affected by factors such as hand tremors and equipment movement during operation, resulting in image blur and jitter, which affects the accuracy of diagnosis and the safety of surgery. Therefore, an effective image anti-shake method and system is needed to improve the stability and quality of endoscopic images.

[0003] Disadvantages of existing technologies: Existing endoscope camera systems mainly rely on electronic image stabilization technology, but in high-precision medical operations, electronic image stabilization has limitations and cannot completely eliminate image jitter and blur. Although traditional optical image stabilization technology can alleviate the jitter problem to a certain extent, its application effect and implementation method still need to be improved and enhanced. Summary of the invention

[0004] In order to solve the problem of limitations of the electronic anti-shake technology of endoscopic camera systems in the prior art, the purpose of the present invention is to provide an image optical anti-shake method for an endoscopic camera system, which achieves real-time stabilization and clear output of images through optical technology, and can provide doctors with better auxiliary diagnostic means.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] An image optical anti-shake method for an endoscope camera system comprises the following steps:

[0007] Collecting the motion data of the endoscope camera in real time, the motion data including the three-dimensional position change and angular velocity change information of the endoscope camera;

[0008] Based on the collected motion data, image processing algorithms are used to analyze and identify the pattern and degree of image jitter, and calculate the compensation required for optical and electronic image stabilization;

[0009] Integrated high-precision optical image stabilization components, which can quickly adjust the optical path or lens position to compensate for image deviation caused by camera movement based on the calculated compensation amount;

[0010] Integrated electronic anti-shake components, through the electronic control system, adjust the image sensor readout data or image processing chip parameters to further eliminate residual jitter;

[0011] Monitor sensor data and image quality in real time, and dynamically adjust optical image stabilization parameters to ensure continuous image stability and clarity.

[0012] According to an optical image stabilization method for an endoscope camera system provided by the present invention, when collecting motion data of an endoscope camera, the acceleration change of the camera is detected, and then the displacement data is obtained, including:

[0013] The acceleration sensor collects acceleration data of the camera at a certain frequency, wherein the data is collected in the form of prohibited Euler angles;

[0014] By integrating the collected acceleration data, we can get the velocity data. Assuming there is three-axis acceleration data a x ,a y ,a z , then the speed V is calculated as follows:

[0015] V = ∫adt

[0016] Further integrate the velocity data to obtain displacement data. Assuming there is three-axis velocity data V x ,V y ,V z , then the displacement S is calculated as follows:

[0017] S = ∫vdt

[0018] Among them, due to the accumulation of errors in discrete sampling and integral calculation, the displacement data drifts over time. Therefore, the displacement data needs to be corrected and compensated to improve the measurement accuracy and stability.

[0019] According to an image optical image stabilization method for an endoscope camera system provided by the present invention, when collecting motion data of an endoscope camera, a gyroscope is used to detect the angular velocity change of the camera, thereby obtaining rotation data, including:

[0020] Reading data from a gyro sensor to obtain data on angular velocity output by the gyro sensor;

[0021] The read angular velocity data is processed and converted to obtain the rotation angle change of the camera, wherein the angular velocity data is converted into angle data by integration, that is, the angular velocity data is integrated over time to obtain the angle change;

[0022] Estimate the camera’s attitude through the processed angle data, including pitch, roll, and yaw angles;

[0023] The camera's motion control and adjustment are achieved based on the angular velocity changes and attitude estimation results detected by the gyroscope.

[0024] According to an optical image stabilization method for an endoscope camera system provided by the present invention, the image processing algorithm includes:

[0025] The Kalman filter algorithm is used to process motion data including acceleration and angular velocity and generate an estimate of the camera motion state, including:

[0026] Establish a state space model: define the state vector and state transfer matrix of the system. The position, velocity and acceleration of the camera are used as the state vector, and the state transfer matrix is ​​used to describe how the system state evolves over time.

[0027] Initialize the Kalman filter: initialize parameters such as the state vector, state covariance matrix, process noise covariance matrix, and observation noise covariance matrix;

[0028] Prediction: Use the state transfer matrix and control input motion data to predict the state and state uncertainty of the system, and then update the state estimate and state covariance matrix;

[0029] Update: Based on the motion data, update the system state estimate and state uncertainty, calculate the Kalman gain, and use it to fuse the prediction results and measurement results;

[0030] Generate estimation results: Based on the state estimation obtained by the Kalman filter algorithm, an estimation result of the camera motion state can be generated;

[0031] Real-time update: Continuously repeat prediction and update, and update the system state estimation in real time to obtain the optimal motion state estimation result.

[0032] According to an optical image stabilization method for an endoscope camera system provided by the present invention, the calculation of the compensation amount required for optical and electronic image stabilization includes:

[0033] According to the output results of the Kalman filter algorithm, the compensation amount that needs to be adjusted for the image is calculated, including displacement compensation and rotation compensation, specifically including:

[0034] The estimated output of the camera motion state includes the following information:

[0035] The current displacement of the camera P k ;

[0036] The current speed of the camera V k ;

[0037] The current angle of the camera θ k ;

[0038] The covariance matrix P of the motion state k ;

[0039] These output information are used for subsequent image compensation calculations.

[0040] According to an optical image stabilization method for an endoscope camera system provided by the present invention, the displacement compensation amount is calculated as follows:

[0041] ΔP k =P k -P k-1

[0042] Among them, ΔP k Indicates the displacement that needs to be compensated at the current moment, P k is the current displacement, P k-1 is the displacement at the previous moment;

[0043] The speed compensation calculation is expressed as the following formula:

[0044] ΔV k =V k -V k-1

[0045] Where, ΔV k Indicates the speed change that needs to be compensated at the current moment, V k is the current speed, V k-1 is the speed at the previous moment.

[0046] The calculation of the rotation compensation is expressed as the following formula:

[0047] Δθ k =θ k -θ k-1

[0048] Among them, Δθ k Indicates the amount of rotation that needs to be compensated at the current moment, θ k is the current angle, θ k-1 It is the angle at the previous moment.

[0049] According to an optical image stabilization method for an endoscope camera system provided by the present invention, the optical image stabilization component comprises:

[0050] Optical stabilizer: The optical stabilizer is built into the lens module. According to the calculated compensation amount, the lens position is adjusted in real time through a micro motor to compensate for the displacement and rotation changes of the camera.

[0051] Zoom lens: This lens is equipped with multiple independently movable lens groups. The movement of the lens groups is driven by an electronic control system based on the calculated compensation amount, and the optical path is adjusted in real time to achieve an anti-shake effect.

[0052] According to an optical image stabilization method for an endoscope camera system provided by the present invention, the electronic image stabilization component is used to perform:

[0053] Sensor data correction: adjust the image sensor readout parameters based on the sensor data to reduce the impact of jitter;

[0054] Image post-processing module: Apply digital filtering technology to post-process the captured images to reduce noise and jitter.

[0055] According to an image optical image stabilization method for an endoscope camera system provided by the present invention, according to the calculated compensation amount, the following parameters need to be dynamically adjusted during image post-processing to compensate for the movement of the camera, including:

[0056] Image position adjustment: Use digital image processing technology to translate the image position on the image sensor accordingly. Assuming that the coordinates of the current image are (X, Y), the adjusted coordinates are as follows:

[0057] X′=X+ΔP k

[0058] Y′=Y+ΔP k

[0059] Image rotation adjustment: The image angle is adjusted through the image rotation algorithm, where the adjusted pixel coordinates (X′, Y′) can be calculated through the rotation matrix, expressed as the following formula:

[0060]

[0061] Speed ​​compensation adjustment: Adjust the speed change between image frames through a smoothing algorithm to make the image transition smoother and reduce the sense of jitter.

[0062] According to an image optical image stabilization method for an endoscope camera system provided by the present invention, a feedback control module is used to monitor sensor data and image quality in real time, and the feedback control module includes:

[0063] Closed-loop feedback system: real-time monitoring of sensor data and image quality, dynamic adjustment of optical image stabilization and electronic image stabilization parameters;

[0064] Automatic adjustment function: According to feedback information, the working status of the anti-shake component is automatically adjusted to adapt to different operating environments.

[0065] It can be seen that compared with the prior art, the design method provided by the present invention has the following beneficial effects:

[0066] 1. High precision and low latency: The present invention integrates a high-precision optical image stabilization component and an electronic image stabilization component. The two work together to compensate for image jitter through physical adjustment and electronic processing respectively, which significantly improves the anti-shake effect, enables the system to respond to motion changes in a very short time, and provides high-precision image stabilization.

[0067] 2. Improve image quality: The present invention can effectively eliminate image jitter and blur through optical image stabilization technology, and can significantly improve the clarity and detail of endoscopic images.

[0068] 3. Enhanced diagnostic accuracy: Based on the collected motion data, the present invention uses image processing algorithms to analyze and identify the pattern and degree of image jitter, thereby calculating the precise amount of anti-shake compensation. This precise calculation ensures the accuracy and effectiveness of anti-shake, and can help doctors obtain more stable and accurate image information during surgery and diagnosis, thereby improving the effectiveness of diagnosis and treatment.

[0069] 4. Reduce blur: The present invention can significantly reduce image blur caused by camera movement by quickly adjusting the optical path or lens position, and adjusting the readout data of the image sensor or the parameters of the image processing chip.

[0070] 5. Improve clarity: In dark or low-light conditions, the effective application of anti-shake technology can extend the exposure time, thereby increasing the amount of light entering the lens and improving the brightness and clarity of the image.

[0071] 6. Real-time compensation: The present invention collects the motion data of the endoscope camera in real time, including three-dimensional position change and angular velocity change information. This method can quickly respond to the movement of the camera and achieve an instant anti-shake effect.

[0072] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 The present invention is a flowchart of an embodiment of an optical image stabilization method for an endoscope camera system.

[0074] Figure 2 The figure is a schematic diagram of the principle of an embodiment of an image optical anti-shake method for an endoscope camera system of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0077] See also Figure 1 and Figure 2 The present invention provides an image optical anti-shake method for an endoscope camera system, the method comprising the following steps:

[0078] Step S1, collecting motion data of the endoscope camera in real time, where the motion data includes information on three-dimensional position change and angular velocity change of the endoscope camera.

[0079] Step S2: Based on the collected motion data, an image processing algorithm is used to analyze and identify the pattern and degree of image jitter, and the compensation amount required for optical and electronic image stabilization is calculated.

[0080] Step S3, integrating a high-precision optical image stabilization component, and according to the calculated compensation amount, quickly adjusting the optical path or the position of the lens through the optical image stabilization component to compensate for the image offset caused by the camera movement, such as the displacement and rotation change of the camera.

[0081] Step S4, integrating an electronic anti-shake component, and adjusting the readout data of the image sensor or the parameters of the image processing chip through an electronic control system to further eliminate residual jitter.

[0082] Step S5, real-time monitoring of sensor data and image quality, and dynamic adjustment of optical image stabilization parameters to ensure continuous image stability and clarity.

[0083] In the above step S1, when collecting the motion data of the endoscope camera, the acceleration change of the camera is detected, and then the displacement data is obtained, including:

[0084] The acceleration sensor collects acceleration data of the camera at a certain frequency, wherein the data is collected in the form of forbidden Euler angles (eg, accelerations on the x, y, and z axes).

[0085] By integrating the collected acceleration data, we can get the velocity data. Assuming there is three-axis acceleration data a x ,a y ,a z , then the speed V is calculated as follows:

[0086] V = ∫adt

[0087] Further integrate the velocity data to obtain displacement data. Assuming there is three-axis velocity data V x ,V y ,V z , then the displacement S is calculated as follows:

[0088] S = ∫vdt

[0089] Among them, due to the accumulation of errors in discrete sampling and integral calculation, the displacement data drifts over time. Therefore, it is necessary to combine other sensors (such as gyroscopes) or filtering algorithms (Kalman filtering algorithms) to correct and compensate the displacement data to improve measurement accuracy and stability.

[0090] Specifically, the acquisition frequency (also called sampling rate) of the accelerometer refers to the number of times the acceleration data is collected per second. This frequency can be adjusted according to the application requirements. For example, for applications that require precise control or monitoring of rapidly changing movements, a higher sampling rate (such as 100Hz or higher) may be required. For some applications that do not require high real-time performance (such as simple motion monitoring), a lower sampling rate (such as 10Hz) is sufficient.

[0091] For the three-axis acceleration data a x ,a y ,a z , the acceleration on each axis needs to be integrated independently to obtain the velocity component on the corresponding axis. Similarly, for the three-axis velocity data V x ,V y ,V z , also need to be integrated independently to obtain the displacement components on the corresponding axes.

[0092] However, in actual digital signal processing, discrete data points are usually processed rather than continuous functions. Therefore, the integration operation is usually approximated by numerical integration methods (such as trapezoidal rule, Simpson's rule, etc.). In discrete-time systems, this is usually simplified to an accumulation or summation operation.

[0093] For the conversion of acceleration to velocity, the following discrete-time approximation can be used: Vx[n]≈Vx[n-1]+ax

[0094] [n]Δt

[0095] Vy[n]≈Vy[n-1]+ay[n]Δt

[0096] Vz[n]≈Vz[n-1]+az[n]Δt

[0097] Among them, Vx[n], Vy[n], Vz[n] are the velocity components of the current time step, Vx[n-1], Vy[n-1], Vz[n-1] are the velocity components of the previous time step, ax[n], ay[n], az[n] are the acceleration components of the current time step, and Δt is the time step (i.e., the sampling interval).

[0098] Similarly, for the conversion of velocity to displacement, the following discrete-time approximation can be used:

[0099] Sx[n]≈Sx[n-1]+Vx[n]Δt

[0100] Sy[n]≈Sy[n-1]+Vy[n]Δt

[0101] Sz[n]≈Sz[n-1]+Vz[n]Δt

[0102] Among them, Sx[n], Sy[n], Sz[n] are the displacement components of the current time step, Sx[n-1], Sy[n-1], Sz[n-1] are the displacement components of the previous time step, and Vx[n], Vy[n], Vz[n] are the velocity components of the current time step.

[0103] In this embodiment, when collecting motion data of the endoscope camera, a gyroscope is used to detect the angular velocity change of the camera, thereby obtaining rotation data, including:

[0104] The data of the gyroscope sensor is read through the corresponding interface (IIC) and the sensor driver, so as to obtain the data about the angular velocity output by the gyroscope sensor. These data are usually the angular velocity values ​​around three axes (usually x, y, z axes), and the unit is radian / second (rad / s).

[0105] The read raw angular velocity data is processed and converted to obtain the rotation angle change of the camera, wherein the angular velocity data can be converted into angle data by integration, that is, the angular velocity data is integrated over time to obtain the angle change.

[0106] The processed angle data is used to estimate the attitude of the camera, including pitch, roll and yaw angles. Among them, attitude estimation can help understand the specific direction and rotation state of the camera. Specifically, this embodiment involves converting the angle data into a rotation matrix or quaternion to more accurately represent and calculate the rotation in three-dimensional space.

[0107] The camera motion control and adjustment are realized based on the angular velocity change and attitude estimation results detected by the gyroscope. For example, if the camera is detected to have deviated from the predetermined direction, a control signal can be sent to adjust the camera attitude to return it to the correct direction, that is, to communicate with the camera control system (such as the motor driver) and send the corresponding control command.

[0108] In the above step S2, the image processing algorithm includes:

[0109] The Kalman filter algorithm is used to process motion data including acceleration and angular velocity and generate an estimate of the camera motion state, including:

[0110] Establish a state space model: define the state vector and state transfer matrix of the system, where the position, velocity, and acceleration of the camera are used as the state vector, and the state transfer matrix is ​​used to describe how the system state evolves over time.

[0111] Initialize the Kalman filter: Initialize parameters such as the state vector, state covariance matrix, process noise covariance matrix, and observation noise covariance matrix.

[0112] Prediction: Use the state transfer matrix and control input motion data to predict the state and state uncertainty of the system, and then update the state estimate and state covariance matrix.

[0113] Update: Based on the motion data (acceleration and angular velocity data), update the system's state estimate and state uncertainty, calculate the Kalman gain, and use it to fuse the prediction results and measurement results.

[0114] Generate estimation results: Based on the state estimation obtained by the Kalman filter algorithm, an estimation result of the camera motion state (such as position, speed, acceleration) can be generated.

[0115] Real-time update: Continuously repeat prediction and update, and update the system state estimation in real time to obtain the optimal motion state estimation result.

[0116] In the above step S2, the compensation amount required for optical and electronic image stabilization is calculated, including:

[0117] According to the output results of the Kalman filter algorithm, the compensation amount that needs to be adjusted for the image is calculated, including displacement compensation and rotation compensation, specifically including:

[0118] The estimated output of the camera motion state includes the following information:

[0119] The current displacement of the camera P k ;

[0120] The current speed of the camera V k ;

[0121] The current angle of the camera θ k ;

[0122] The covariance matrix P of the motion state k ;

[0123] These output information are used for subsequent image compensation calculations.

[0124] Furthermore, the displacement compensation amount is calculated as follows:

[0125] ΔP k =P k -P k-1

[0126] Among them, ΔP k Indicates the displacement that needs to be compensated at the current moment, P k is the current displacement, P k-1 is the displacement at the previous moment.

[0127] Furthermore, the speed compensation amount is calculated as follows:

[0128] ΔV k =V k -V k-1

[0129] Where, ΔV k Indicates the speed change that needs to be compensated at the current moment, V k is the current speed, V k-1 is the speed at the previous moment.

[0130] Furthermore, the calculation of the rotation compensation amount is expressed as the following formula:

[0131] Δθ k =θ k -θ k-1

[0132] Among them, Δθ k Indicates the amount of rotation that needs to be compensated at the current moment, θ k is the current angle, θ k-1 It is the angle at the previous moment.

[0133] In the above step S3, the optical image stabilization component includes:

[0134] Optical Stabilizer (OIS): The optical stabilizer is built into the lens module and the lens position is adjusted in real time by a micro motor based on the calculated compensation amount to compensate for the displacement and rotation changes of the camera.

[0135] Zoom lens: This lens is equipped with multiple independently movable lens groups. The movement of the lens groups is driven by an electronic control system based on the calculated compensation amount, and the optical path is adjusted in real time to achieve an anti-shake effect.

[0136] In the above step S4, the electronic anti-shake component is used to perform:

[0137] Sensor data correction: Adjust the readout parameters of the image sensor based on the sensor data to reduce the impact of jitter.

[0138] Image post-processing module: Apply digital filtering technology to post-process the captured images to reduce noise and jitter.

[0139] In this embodiment, according to the calculated compensation amount, the following parameters need to be dynamically adjusted during image post-processing to compensate for the movement of the camera, including:

[0140] Image position adjustment: Use digital image processing technology to translate the image position on the image sensor accordingly. Assuming that the coordinates of the current image are (X, Y), the adjusted coordinates are as follows:

[0141] X′=X+ΔP k

[0142] Y′=Y+ΔP k

[0143] Image rotation adjustment: The image angle is adjusted through the image rotation algorithm, where the adjusted pixel coordinates (X′, Y′) can be calculated through the rotation matrix, expressed as the following formula:

[0144]

[0145] Speed ​​compensation adjustment: Adjust the speed change between image frames through smoothing algorithms (such as motion compensation filters) to make image transitions smoother and reduce jitter.

[0146] In the above step S5, a feedback control module is used to monitor sensor data and image quality in real time, and the feedback control module includes:

[0147] Closed-loop feedback system: monitors sensor data and image quality in real time, and dynamically adjusts optical and electronic image stabilization parameters.

[0148] Automatic adjustment function: According to feedback information, the working status of the anti-shake component is automatically adjusted to adapt to different operating environments.

[0149] Specifically, this embodiment constructs a system including a feedback control module, especially for real-time monitoring of sensor data and image quality, and dynamically adjusting optical image stabilization (OIS) and electronic image stabilization (EIS) parameters accordingly, which is responsible for receiving real-time data from sensors (such as acceleration sensors, gyroscopes, etc.) and image quality evaluation results (such as blur, contrast, etc.), and then optimizes the anti-shake performance through a closed-loop feedback system based on these data, including:

[0150] Sensor data: The motion status of the device, including angular velocity, acceleration, etc., is obtained in real time through sensors such as accelerometers and gyroscopes.

[0151] Image quality data: The currently captured image is analyzed through image processing algorithms to evaluate its quality indicators such as blur, contrast, brightness, etc.

[0152] Data fusion: Fusion of sensor data and image quality data to provide a more comprehensive understanding of the device’s motion status and image quality.

[0153] Error calculation: Based on the difference between the ideal state (such as static state) and the current state, the error amount that the anti-shake system needs to compensate is calculated.

[0154] Parameter adjustment: Dynamically adjust the parameters of optical image stabilization and electronic image stabilization, such as lens movement, image compensation algorithm parameters, etc., based on the error calculation results.

[0155] This embodiment uses advanced control algorithms (such as PID control, fuzzy control, etc.) to optimize the adjustment process, ensure a quick and accurate response to device movement, identify the current operating environment (such as lighting conditions, motion intensity, etc.) through sensor data, and automatically adjust the working parameters of the anti-shake component according to the environmental perception results to adapt to different operating environments.

[0156] Furthermore, the present embodiment continuously monitors the anti-shake effect to ensure that the image stability reaches a preset standard.

[0157] Furthermore, this embodiment optimizes the performance of the anti-shake system, such as reducing power consumption and improving response speed, while ensuring stability.

[0158] Therefore, by constructing a feedback control module including a closed-loop feedback system and an automatic adjustment function, the present embodiment can effectively monitor sensor data and image quality in real time, and dynamically adjust optical image stabilization and electronic image stabilization parameters accordingly, which not only improves the stability and quality of the image, but also enhances the adaptability and performance of the device in different operating environments.

[0159] In summary, the present invention integrates a high-precision optical image stabilization component and an electronic image stabilization component, which work together to compensate for image jitter through physical adjustment and electronic processing respectively, thereby significantly improving the image stabilization effect, enabling the system to respond to motion changes in a very short time, and providing high-precision image stabilization.

[0160] The present invention can effectively eliminate image jitter and blur through optical image stabilization technology, and can significantly improve the clarity and detail of endoscopic images.

[0161] Based on the collected motion data, the present invention uses image processing algorithms to analyze and identify the pattern and degree of image jitter, thereby calculating the precise anti-shake compensation amount. This precise calculation ensures the accuracy and effectiveness of anti-shake, and can help doctors obtain more stable and accurate image information during surgery and diagnosis, thereby improving the effectiveness of diagnosis and treatment.

[0162] The present invention can significantly reduce image blur caused by camera movement by quickly adjusting the optical path or lens position, and adjusting the readout data of the image sensor or the parameters of the image processing chip.

[0163] In the present invention, under dark or low light conditions, the effective application of the anti-shake technology can prolong the exposure time, thereby increasing the amount of light entering the lens and improving the brightness and clarity of the image.

[0164] The present invention collects the motion data of the endoscope camera in real time, including three-dimensional position change and angular velocity change information. The method can quickly respond to the movement of the camera and achieve an immediate anti-shake effect.

[0165] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. An image optical image stabilization method for an endoscope camera system, characterized in that: The following steps are involved: Collecting the motion data of the endoscope camera in real time, the motion data including the three-dimensional position change and angular velocity change information of the endoscope camera; Based on the collected motion data, image processing algorithms are used to analyze and identify the pattern and degree of image jitter, and calculate the compensation required for optical and electronic image stabilization; Integrated high-precision optical image stabilization components, which can quickly adjust the optical path or lens position to compensate for image deviation caused by camera movement based on the calculated compensation amount; Integrated electronic anti-shake components, through the electronic control system, adjust the image sensor readout data or image processing chip parameters to further eliminate residual jitter; Monitor sensor data and image quality in real time, and dynamically adjust optical image stabilization parameters to ensure continuous image stability and clarity.

2. The method according to claim 1, characterized in that: When collecting motion data of the endoscope camera, the acceleration change of the camera is detected and the displacement data is obtained, including: The acceleration sensor collects acceleration data of the camera at a certain frequency, wherein the data is collected in the form of prohibited Euler angles; By integrating the collected acceleration data, we can get the velocity data. Assuming there is three-axis acceleration data a x ,a y ,a z , then the speed V is calculated as follows: V = ∫adt Further integrate the velocity data to obtain displacement data. Assuming there is three-axis velocity data V x ,V y ,V z , then the displacement S is calculated as follows: S = ∫vdt Among them, due to the accumulation of errors in discrete sampling and integral calculation, the displacement data drifts over time. Therefore, the displacement data needs to be corrected and compensated to improve the measurement accuracy and stability.

3. The method according to claim 1, characterized in that: When collecting motion data of the endoscope camera, a gyroscope is used to detect the angular velocity change of the camera to obtain rotation data, including: Reading data from a gyro sensor to obtain data on angular velocity output by the gyro sensor; The read angular velocity data is processed and converted to obtain the rotation angle change of the camera, wherein the angular velocity data is converted into angle data by integration, that is, the angular velocity data is integrated over time to obtain the angle change; Estimate the camera's attitude through the processed angle data, including pitch, roll, and yaw angles; The camera's motion control and adjustment are achieved based on the angular velocity changes and attitude estimation results detected by the gyroscope.

4. The method according to claim 1, characterized in that: The image processing algorithm includes: The Kalman filter algorithm is used to process motion data containing acceleration and angular velocity and generate an estimate of the camera's motion state, including: Establish a state space model: define the state vector and state transfer matrix of the system. The position, velocity and acceleration of the camera are used as the state vector, and the state transfer matrix is ​​used to describe how the system state evolves over time. Initialize the Kalman filter: initialize parameters such as the state vector, state covariance matrix, process noise covariance matrix, and observation noise covariance matrix; Prediction: Use the state transfer matrix and control input motion data to predict the state and state uncertainty of the system, and then update the state estimate and state covariance matrix; Update: Based on the motion data, update the system state estimate and state uncertainty, calculate the Kalman gain, and use it to fuse the prediction results and measurement results; Generate estimation results: Based on the state estimation obtained by the Kalman filter algorithm, an estimation result of the camera motion state can be generated; Real-time update: Continuously repeat prediction and update, and update the system state estimation in real time to obtain the optimal motion state estimation result.

5. The method according to claim 4, characterized in that The calculation of the compensation amount required for optical and electronic image stabilization includes: According to the output results of the Kalman filter algorithm, the compensation amount that needs to be adjusted for the image is calculated, including displacement compensation and rotation compensation, specifically including: The estimated output of the camera motion state includes the following information: The current displacement of the camera P k ; The current speed of the camera V k ; The current angle of the camera θ k ; The covariance matrix P of the motion state k ; These output information are used for subsequent image compensation calculations.

6. The method according to claim 5, characterized in that: The displacement compensation calculation is expressed as the following formula: ΔP k =P k -P k-1 Among them, ΔP k Indicates the displacement that needs to be compensated at the current moment, P k is the current displacement, P k-1 is the displacement at the previous moment; The speed compensation calculation is expressed as the following formula: ΔV k =V k -V k-1 Where, ΔV k Indicates the speed change that needs to be compensated at the current moment, V k is the current speed, V k-1 is the speed at the previous moment; The calculation of the rotation compensation is expressed as the following formula: Dth k =θ k -θ k-1 Among them, Δθ k Indicates the amount of rotation that needs to be compensated at the current moment, θ k is the current angle, θ k-1 It is the angle at the previous moment.

7. The method according to claim 1, characterized in that: The optical image stabilization component comprises: Optical stabilizer: The optical stabilizer is built into the lens module. According to the calculated compensation amount, the lens position is adjusted in real time through a micro motor to compensate for the displacement and rotation changes of the camera. Zoom lens: This lens is equipped with multiple independently movable lens groups. The movement of the lens groups is driven by an electronic control system based on the calculated compensation amount, and the optical path is adjusted in real time to achieve an anti-shake effect.

8. The method according to claim 1, characterized in that: The electronic image stabilization component is used to perform: Sensor data correction: adjust the image sensor readout parameters based on the sensor data to reduce the impact of jitter; Image post-processing module: Apply digital filtering technology to post-process the captured images to reduce noise and jitter.

9. The method according to claim 8, characterized in that: Based on the calculated compensation amount, the following parameters need to be dynamically adjusted during image post-processing to compensate for the camera motion, including: Image position adjustment: Use digital image processing technology to translate the image position on the image sensor accordingly. Assuming that the coordinates of the current image are (X, Y), the adjusted coordinates are as follows: X′=X+ΔP k Y′=Y+ΔP k Image rotation adjustment: The image angle is adjusted through the image rotation algorithm, where the adjusted pixel coordinates (X′, Y′) can be calculated through the rotation matrix, expressed as the following formula: Speed ​​compensation adjustment: Adjust the speed change between image frames through a smoothing algorithm to make the image transition smoother and reduce the sense of jitter.

10. The method according to any one of claims 1 to 9, characterized in that: A feedback control module is used to monitor sensor data and image quality in real time, and the feedback control module includes: Closed-loop feedback system: real-time monitoring of sensor data and image quality, dynamic adjustment of optical image stabilization and electronic image stabilization parameters; Automatic adjustment function: According to feedback information, the working status of the anti-shake component is automatically adjusted to adapt to different operating environments.

Images