Optical anti-shake system and method for controlling optical anti-shake

Abstract

The invention provides an optical anti-shake system and control thereof, which can improve compensation capability in a translation-pitch state. The optical anti-shake system comprises a detection unit used for detecting motion of a compensation target object in each frame of a moving image, and the compensation target object is an optical system, a camera module or an image sensor; and a driver for performing blur compensation of an image by moving the compensation target object to offset the detected motion of the compensation target object. The period corresponding to each frame includes a first period in which exposure is performed and a second period in which the exposure is suspended. The driver initiates the compensation so as to perform the compensation at least in a portion of the first period and suspends the compensation so as not to perform the compensation at least in a portion of the second period.

Description

Technical Field

[0001] This invention relates to an optical image stabilization system and a method for controlling optical image stabilization, and more specifically, to an optical image stabilization system and a method for controlling optical image stabilization, an imaging device and a camera module for moving a lens, an image sensor, etc. to counteract the relative movement of a subject within the screen caused by camera shake. Background Technology

[0002] Optical image stabilization (OIS) is widely used in camera modules of portable devices such as smartphones and tablets, as well as other cameras. OIS is a function that prevents camera shake during video recording and preview display. In OIS, a trajectory of camera shake is generated based on angular velocity signals from a gyroscope sensor, acceleration signals from an accelerometer sensor, etc. Compensation is then performed to suppress image blur, thereby counteracting movement along the trajectory of camera shake. In image compensation, there is an OIS method that moves or rotates the lens, image sensor, or camera module itself during exposure in a direction that counteracts the camera shake trajectory. In OIS, the method of moving the lens can be called lens-shift stabilization. Furthermore, the method of moving the image sensor can be called sensor-shift stabilization. Additionally, the method of moving the camera module itself can be called module tilt stabilization.

[0003] All of the aforementioned OIS systems have mechanical limitations. For example, lens-shift and sensor-shift OIS systems have limitations on the amount of movement. Furthermore, module-tilt OIS systems have a maximum rotation angle. These mechanical limitations restrict the practical performance of OIS. For instance, in continuous camera use, such as capturing moving images, the camera's orientation can easily exceed the mechanical limitations of OIS. To handle such situations, when the amount of movement of the lens or image sensor, or the rotation angle of the camera module, approaches the mechanical limits, the imaging device transitions from a normal state to a special state (often called "pan-tilt") and performs control according to this state. For example, when transitioning from the normal state to the pan-tilt state, OIS continues by performing operations to avoid mechanical limitations. For example, the imaging device forcibly changes the position of the lens, image sensor, etc., in a direction close to a reference position. This method is widely used and is currently in practical and commercial applications. However, compared to the normal state, the ability to perform blur compensation during exposure in this pan-tilt state is significantly reduced. Furthermore, this method requires a process for detecting the pan-tilt state.

[0004] Therefore, in traditional OIS methods (methods for detecting and controlling translation-pitch states), the ability to compensate for fuzziness decreases significantly when continuous OIS operations are performed under mechanical constraints. Furthermore, the increased workload associated with detecting translation-pitch states further exacerbates the problem. Summary of the Invention

[0005] This invention provides an optical image stabilization system and a method for controlling optical image stabilization, which can improve the compensation capability under translation-pitch conditions.

[0006] According to the first aspect, an optical image stabilization system is provided, comprising:

[0007] A detection unit is used to detect the motion of a compensation target object in each frame of a motion image, wherein the compensation target object is an optical system, a camera module, or an image sensor;

[0008] A driver is configured to perform image blur compensation by moving the target object to counteract the detected motion of the target object.

[0009] The period corresponding to each frame includes a first period for performing exposure and a second period for pausing the exposure.

[0010] The driver initiates the compensation to perform the compensation at least for a portion of the first cycle, and suspends the compensation so that it is not performed at least for a portion of the second cycle.

[0011] According to the first aspect, the OIS control initiates the compensation to perform the compensation at least for a portion of the first cycle, and pauses the compensation so that it is not performed at least for a portion of the second cycle. Therefore, the quality of the moving image can be improved by suppressing image blur in the translation-pitch state.

[0012] In one possible implementation of the first aspect, the driver is further configured to move the compensation target object to a predetermined reference position during the second period.

[0013] According to this implementation, the power consumption required to move the compensation target object can be reduced by removing the DC component from the segmented target value.

[0014] In one possible implementation of the first aspect, the reference position is the center of the mechanism for moving the compensation target object.

[0015] According to this implementation, by using the center of the mechanism used to move the compensation target object as the reference position, the power consumption required to move the camera module can be reduced.

[0016] In one possible implementation of the first aspect, the reference position is the optical center of the compensation target object.

[0017] According to this implementation, by using the optical center as the reference position, the power consumption required to move the optical system can be reduced.

[0018] In one possible implementation of the first aspect, the driver is further configured to move the compensation target object based on the motion of the compensation target object detected in the previous first cycle, so as to maximize the operability range of the compensation target object in the second cycle.

[0019] According to this implementation, optical image stabilization capability can be improved by moving the compensation target object to maximize the operable range of the compensation target object.

[0020] In one possible implementation of the first aspect, the driver is further configured to shorten the exposure period in a subsequent frame when the motion of the compensation target object detected in the preceding first period exceeds the operable range of the compensation target object.

[0021] This implementation shortens the exposure time in the next frame when the movement of the compensation target exceeds its operable range. Therefore, it reduces the time period during which the compensation target reaches the limit of the actuator's operable range and the image stabilization effect is compromised.

[0022] In one possible implementation of the first aspect, the driver is further configured to move the compensation target object to the reference position at a constant speed during the period from the end of the compensation to the start of the next compensation.

[0023] According to this implementation, during the period from the end of the compensation to the start of the next compensation, the power consumption required to move the compensation target object to the reference position can be suppressed at a constant speed.

[0024] In one possible implementation of the first aspect, the driver is further configured to move the compensation target object such that the time period during which the compensation target object stops moving at the reference position occurs within the period from the end of the compensation to the start of the next compensation.

[0025] According to this implementation, even if the blank period is short, the compensation target object can be quickly moved to the reference position to ensure that the compensation target object is at the reference position before the OIS control of the next frame begins.

[0026] In one possible implementation of the first aspect, the start time of the compensation is different from the start time of the exposure cycle.

[0027] According to this implementation, the start time of the compensation and the start time of the exposure cycle are different, so even when the response is allocated in the feedback control based on the movement of the camera module, the quality of moving images can be improved by suppressing image blur in the translation-pitch state.

[0028] In one possible implementation of the first aspect, the end time of the compensation is different from the end time of the exposure cycle.

[0029] According to this implementation, the end time of the compensation and the end time of the exposure cycle are different, so even when the response is allocated in the feedback control based on the movement of the camera module, the quality of the moving image can be improved by suppressing image blur in the translation-pitch state.

[0030] According to the second aspect, an optical image stabilization method is provided, comprising:

[0031] The motion of a target object to be compensated is detected in each frame of a moving image, wherein the target object to be compensated is an optical system, a camera module, or an image sensor;

[0032] Image blur compensation is performed by moving the target object to counteract the detected motion of the target object.

[0033] The period corresponding to each frame includes a first period for performing exposure and a second period for pausing the exposure.

[0034] In the execution of the compensation, the compensation begins at least during a portion of the first cycle, and the compensation is paused so that it is not executed at least during a portion of the second cycle.

[0035] According to the second aspect, the OIS control initiates the compensation to perform the compensation at least for a portion of the first cycle, and pauses the compensation to avoid performing the compensation at least for a portion of the second cycle. Therefore, by suppressing the image blur in the translation-pitch state, the quality of the moving image is improved.

[0036] In one possible implementation of the second aspect, the performance compensation includes:

[0037] In the second cycle, the compensation target object is moved to a predetermined reference position.

[0038] In one possible implementation of the second aspect, the reference position is the center of the mechanism for moving the compensation target object.

[0039] In one possible implementation of the second aspect, the reference position is the optical center of the compensation target object.

[0040] In one possible implementation of the second aspect, performing the compensation includes: moving the compensation target object based on the motion of the compensation target object detected in the previous first cycle to maximize the operability range of the compensation target object in the second cycle.

[0041] In one possible implementation of the second aspect, the method further includes: shortening the exposure period in the next frame when the motion of the compensation target object detected in the previous first period exceeds the operable range of the compensation target object.

[0042] In one possible implementation of the second aspect, moving the compensation target object to the predetermined reference position includes: moving the compensation target object to the reference position at a constant speed during the period from the end of the compensation to the start of the next compensation.

[0043] In one possible implementation of the second aspect, moving the compensation target object to a predetermined reference position includes moving the compensation target object such that the time period during which the compensation target object stops moving at the reference position occurs within the period from the end of the compensation to the start of the next compensation.

[0044] In one possible implementation of the second aspect, the start time of the compensation is different from the start time of the exposure cycle.

[0045] In one possible implementation of the second aspect, the end time of the compensation is different from the end time of the exposure cycle.

[0046] According to a third aspect, a camera module including the aforementioned optical image stabilization system is provided.

[0047] According to the fourth aspect, an imaging device including the above-described optical image stabilization system is provided. Attached Figure Description

[0048] To more clearly describe the technical solutions in the embodiments, the accompanying drawings used in this embodiment description will be briefly introduced below. Obviously, the drawings described below only depict some possible embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0049] Figure 1 The basic configuration of the optical image stabilization system provided by an embodiment of the present invention is shown.

[0050] Figure 2 A schematic diagram showing the movement of a smartphone and camera module is provided. Figure 2 Image (a) shows the movement of the camera module when the smartphone body is in the reference position. Figure 2 (b) shows the angular displacement θ of the smartphone body moving from the reference position. P The movement of the camera module;

[0051] Figure 3 The figure shows the obtained angular displacement θ. P Example of a graph showing the value;

[0052] Figure 4 The figure shows the obtained angular displacement θ. P and the calculated angle θ IS Example of a graph showing the value;

[0053] Figure 5 The figure shows the angle θ. P –θ IS A curve of the value, Figure 5 (a) in the text represents θ P –θ IS A curve of the value, Figure 5 (b) is Figure 5 Enlarged view of the circled portion in (a) of the image;

[0054] Figure 6 An exemplary functional block diagram related to the conventional OIS method is shown;

[0055] Figure 7 An exemplary functional block diagram related to OIS is shown in one embodiment;

[0056] Figure 8 The detailed communication between the signal processing unit and the feedback control unit is shown;

[0057] Figure 9 A flowchart of the method for controlling optical image stabilization provided in the embodiment is shown;

[0058] Figure 10 A graph showing the motion of the camera module is displayed;

[0059] Figure 11 An example of segmented target values ​​is shown;

[0060] Figure 12 An example of the new target value for the segment is shown;

[0061] Figure 13 A diagram illustrating the method for generating new target values ​​provided in the embodiment is shown;

[0062] Figure 14 The simulation results for calculating the target value provided in the embodiment are shown; Figure 14 (a) shows the angular displacement and motion of the image indicated by the motion signal; Figure 14 (b) shows the calculated new target value and the movement of the lens;

[0063] Figure 15 It shows Figure 14 Simulation results of OIS control in the elliptical closed portion of (a) in the figure; Figure 15 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 15 (b) shows the calculated new target value and the movement of the lens;

[0064] Figure 16 The simulation results of calculating the target value based on the comparative example are shown; Figure 16 (a) shows the angular displacement and motion of the image indicated by the motion signal; Figure 16 (b) shows the calculated target value and the camera movement;

[0065] Figure 17 It shows Figure 16 Simulation results of OIS control in the elliptical closed portion of (a) in the figure; Figure 17 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 17 (b) shows the calculated target value and the camera movement;

[0066] Figure 18 The simulation results for calculating the target value provided in the embodiment are shown; Figure 18 (a) in the figure shows the angular displacement and motion of the image indicated by the motion signal. Figure 18 (b) shows the calculated new target value and the movement of the lens;

[0067] Figure 19 It shows Figure 18 The result of OIS control in the elliptical closed portion of (a) in the figure; Figure 19 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 19 (b) shows the calculated new target value and the movement of the lens;

[0068] Figure 20 The simulation results of calculating the target value based on the comparative example are shown; Figure 20(a) shows the angular displacement and motion of the image indicated by the motion signal; Figure 20 (b) shows the calculated target value and the camera movement;

[0069] Figure 21 It shows Figure 20 Simulation results of OIS control in the elliptical closed portion of (a) in the figure; Figure 21 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 21 (b) shows the calculated target value and the camera movement;

[0070] Figure 22 Simulation results of power consumption in OIS control according to an embodiment are shown;

[0071] Figure 23 Simulation results of power consumption for OIS control based on a comparative example are shown;

[0072] Figure 24 This demonstrates a method for generating new target values ​​when the exposure cycle and OIS control cycle do not match;

[0073] Figure 25 A diagram is shown illustrating a method for controlling optical image stabilization according to an embodiment;

[0074] Figure 26 A diagram is shown illustrating a method for controlling optical image stabilization according to an embodiment;

[0075] Figure 27 A diagram is shown illustrating a method for controlling optical image stabilization according to an embodiment;

[0076] Figure 28 A diagram illustrating a method for generating new target values ​​according to an embodiment is shown;

[0077] Figure 29 Simulation results of power consumption in OIS control according to an embodiment are shown;

[0078] Figure 30 A diagram illustrating a method for generating new target values ​​according to an embodiment is shown. Detailed Implementation

[0079] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0080] Figure 1 This diagram illustrates the basic configuration of an optical image stabilization system provided in an embodiment of the present invention. While this embodiment describes optical image stabilization for a moving or rotating camera module, the present invention can also be applied to lens displacement methods for moving lenses and sensor displacement methods for moving image sensors.

[0081] A camera module 100 employing an optical image stabilization system is mounted on an imaging device, including portable devices such as smartphones and tablets, or other cameras. Although the camera module 100 may have blocks associated with other functions such as autofocus, Figure 1 Only the blocks relevant to this embodiment are shown to facilitate understanding of the invention.

[0082] The camera module 100 is movably supported relative to the body of the imaging device. The camera module 100 can move independently in the X-axis direction (perpendicular to the paper) and the Y-axis direction (the longitudinal direction of the drawing). It should be noted that the direction of movement of the camera module 100 in this embodiment is merely an example. For instance, it can also move independently in a rotational (rolling) direction around the optical center or the center of the imaging plane.

[0083] The camera module 100 includes an imaging lens 102, an image sensor 110, an actuator 104, a controller 106, a position detection unit 108, an analog front end (AFE) 112, a motion sensor 120, and a display 118.

[0084] The imaging lens 102 is positioned in the incident light path of the image sensor 110 to guide the subject light to the image sensor 110.

[0085] Image sensor 110 is configured to perform photoelectric conversion on the light of the subject and output a signal related to the captured image. For example, image sensor 110 may be a complementary metal-oxide-semiconductor (CMOS) sensor. AFE 112 is used to convert the analog signal output by image sensor 110 into a digital signal and output the digital signal to controller 106. Actuator 104 is used to move camera module 100. As described above, camera module 100 can move independently in the X-axis and Y-axis directions, therefore actuator 104 independently controls the movement in the X-axis and Y-axis directions.

[0086] Motion sensor 120 is used to detect the movement of camera module 100. For example, motion sensor 120 may be a gyroscope sensor. Motion sensor 120 is used to output a motion signal indicating the detected movement of camera module 100.

[0087] The position detection unit 108 is used to position the camera module 100. For example, the position detection unit 108 may employ a magnetic detection element such as a Hall sensor. In this case, the magnetic detection element outputs a position detection signal indicating the displacement of the camera module 100.

[0088] The controller 106 includes an actuator driver 114 and an image processing unit 116. The actuator driver 114 receives motion signals from the motion sensor 120 and controls the actuator 104 to counteract the movement of the camera module 100. The actuator driver 114 controls the movement of the camera module 100 based on the motion signals from the motion sensor 120. For example, when a gyroscope sensor is used as the motion sensor 120, the motion signals indicate the angular velocity in the yaw direction (perpendicular to the plane of the paper) and the angular velocity in the pitch direction (the longitudinal direction of the paper). The actuator driver 114 calculates the angular displacement by combining the motion signals in the yaw and pitch directions, respectively. The actuator driver 114 then moves the camera module 100 in response to the desired angular displacement. In this way, the actuator driver 114 outputs a drive signal and performs feedback control, causing the position of the camera module 100, which indicates a position detection signal, to move to a target position in response to the movement of the camera module 100. Actuator 104 performs drive control of camera module 100 based on drive signals from actuator driver 114. It should be noted that the actuation direction of actuator driver 114 in this embodiment is just an example. For example, it can also be driven independently in the rotation (rolling) direction around the optical center or the center of the imaging plane.

[0089] The image processing unit 116 performs predetermined processing on the signal output from the AFE 112 and outputs an image signal. The display 118 displays an image based on the image signal received from the image processing unit 116. For example, the display 118 may be a liquid crystal display (LCD).

[0090] Although in this embodiment, such as Figure 1As shown, an optical image stabilization system is applied to a camera module 100, but the camera module 100 can be adaptively designed according to the compensation target object. For example, if the optical image stabilization system of the present invention is applied to a lens shift method, the imaging lens 102 is supported in a movable manner in the yaw and pitch directions. The position detection unit 108 detects the position of the imaging lens 102, and the motion sensor 120 detects the movement of the imaging lens 102. Furthermore, when the optical image stabilization system of the present invention is applied to a sensor shift method, the image sensor 110 is supported in a movable manner in the yaw and pitch directions. The position detection unit 108 detects the position of the image sensor 110, and the motion sensor 120 detects the movement of the image sensor 110.

[0091] Now for reference Figures 2 to 5 This paper describes a general method for optical image stabilization using tilt-type OIS in a module. While the following description focuses on optical image stabilization in the pitch direction for smartphones and camera modules, optical image stabilization in the yaw direction can be performed in the same manner.

[0092] Figure 2 The movement of the smartphone 202 and camera module 204 is illustrated by example. Figure 2 In the reference position, smartphone 202 is horizontally oriented along axis L1, which is perpendicular to one surface of the smartphone 202's body. θ P θ indicates the angular displacement of the smartphone 202 in space relative to a reference position on the device body. IS The angle of inclination of the camera module 204 relative to the body of the smartphone 202 is indicated, specifically the angle of inclination of the optical axis L2 of the imaging lens 102. When the camera module 204 is in the reference position, the optical axis L2 is parallel to the optical axis L1. θ ISmax Indicator θ IS The maximum value of θ. ISmax This is determined by the mechanical limitations of the camera module 204. For example, the angle between an upward-pointing arrow and the reference position can be 20 degrees, and the angle of a downward-pointing arrow can be -20 degrees.

[0093] exist Figure 2 In the example shown in (a), the angular displacement θ of the smartphone 202's body P The angle is 0°. Camera module 204 can operate at ±θ. ISmax Rotate within the range of θ. If θ ISmax= 20°, then the smartphone 202 can start from 0° and move between approximately +20° and -20°, vibrating in small increments due to camera shake. Triangle 206 indicates the movable range of the camera module 204. As the camera module 204 moves in the pitch direction, the movable range also moves accordingly. The smartphone 202 acquires signals such as those from the motion sensor 120, etc. Figure 2 The angular displacement θ shown in (b) is P As a signal for a stable trajectory, the movable range of the camera module 204 can vary, as shown by triangle 206. Figure 3 This shows the acquired angular displacement θ. P An example of a graph showing the values, with the vertical axis representing angular displacement (degrees) and the horizontal axis representing time (seconds).

[0094] In optical image stabilization, the angle θ of the camera module 204 is calculated using algorithms, such as those based on the output value of the gyroscope sensor. IS To eliminate angular displacement θ P The small changes included, such as Figure 4 As shown. Because the movement of the camera module 204 is mechanically limited, therefore, within ±θ... ISmax Calculate angle θ within the range IS . Figure 5 For θ P – θ IS A graph showing the value of θ. Smartphone 202 can be based on θ. IS The value of the moving camera module 204. According to the traditional ideal OIS algorithm, the result is a smooth image trajectory in space, such as... Figure 5 θ in (a) P – θ IS As shown.

[0095] Figure 6 An example of a functional block involving a conventional OIS method is shown. The actuator driver 614 includes a signal processing unit 604 and a feedback control unit 602. The motion sensor 120 detects the motion of the camera module 100 and outputs a motion signal. This motion signal can be a signal indicating angular velocity or acceleration. The signal processing unit 604 calculates the angle θ of the camera module 204. IS This allows the angular displacement θ to be eliminated based on the value of the motion signal within mechanical constraints. PThe target value is a target value provided from the signal processing unit 604 to the feedback control unit 602. The feedback control unit 602 sends a drive signal to the actuator 104 based on the received target value. The actuator 104 controls the position of the camera module 100 based on the received drive signal. The position of the camera module 100 is detected by the position detection unit 108. The new position information is used in the feedback control unit 602 to generate a new drive signal. The target value indicates a target amount of operation for the actuator 104 to maintain in the feedback control unit 602. The signal processing unit 604 generates the target value based on the movement of the camera module 204. In any configuration of OIS, the signal processing unit 604 can be positioned before the signal indicating the target value, and the feedback control unit 602 can be positioned after the target value, such as... Figure 6 As shown. However, the signal processing unit 604 and the feedback control unit 602 are functional configurations of the actuator driver 114 and may not necessarily match the physical configuration.

[0096] The signal processing unit 604 can also be used to determine the translation-pitch state based on the position detection signal received from the position detection unit 108. Furthermore, the signal processing unit 604 can also output other signals, such as the determination result of the translation-pitch state.

[0097] In traditional OIS, there are two problems:

[0098] First, because traditional OIS uses feedback control, the stabilization process must be performed in real time. In other words, it is not possible to store imaging data and information from the motion sensor in memory and then derive the optimal angle θ. IS The method. In this case, it is necessary to calculate no more than θ. ISmax And there is no delay in the angle θ IS Because the future cannot be accurately predicted, even angle θ IS θ was not reached ISmax It is also necessary to restrict the operation of OIS from θ ISmax Let's start a little earlier. Therefore, even if OIS is within ±θ... ISmax Within its operating range, the performance of OIS cannot be fully proven.

[0099] Second, assuming it can be exported as follows Figure 4 The ideal angle θ shown IS And can be achieved through, for example Figure 5The OIS shown in (a) enables smooth spatial displacement of the image. In this case, OIS can suppress image blurring from 0 to 1 second, 3 to 5 seconds, and 9 to 11 seconds. However, the imaging device is in a translation-pitch state from 1 to 3 seconds, from 5 to 9 seconds, and from 11 to 13 seconds. If the exposure cycle is extended in the offset tilt state, the image blurring within that exposure cycle cannot be suppressed. Figure 5 (b) is Figure 5 The enlarged view of the circled area in (a) shows the angle (degrees) change from 11 seconds to 13 seconds. If the exposure cycle is included within this cycle, image stabilization fails, resulting in a blurry image. In other words, as long as OIS uses conventional algorithms, it is impossible to suppress image blur in pan-pitch mode. This means that when a photographer performs a pan operation while shooting moving images and the OIS system is in pan-pitch mode, image quality will degrade.

[0100] (First embodiment)

[0101] In embodiments of the present invention, image blurring can be effectively suppressed and video quality improved in the pan-tilt state, where image blurring is virtually impossible to stop in conventional OIS with continuous drive compensation of the target object in the pan-tilt state.

[0102] Now for reference Figure 7 An example of a functional block diagram related to OIS according to this embodiment will be described. The actuator driver 114 includes a signal processing unit 804 and a feedback control unit 602. The motion sensor 120 detects the motion of the camera module 204 and outputs a motion signal. The signal processing unit 804 calculates the angle θ of the camera module 204. IS This allows for the elimination of values ​​based on motion signals, including those related to angular displacement θ. P The signal processing unit 802 provides a target value to the feedback control unit 602 based on a minute change in the target value. The feedback control unit 602 then sends a drive signal to the actuator 104 based on the received target value. The actuator 104 controls the position of the camera module 100 based on the received drive signal. The position of the camera module 100 is detected by the position detection unit 108. The new position information is used in the feedback control unit 602 to generate a new drive signal.

[0103] The signal processing unit 804 can communicate with the image sensor 110. Specifically, the signal processing unit 804 sends signals to the image sensor 110 to request a shorter exposure cycle. The image sensor 110 also sends synchronization signals to the signal processing unit 804 to synchronize the timing of each frame to the signal processing unit 804.

[0104] Figure 8A more detailed description of the communication between the signal processing unit 804 and the feedback control unit 602 is shown. Here, the new target value refers to a new target value for the workload of the actuator 104 maintained in the feedback control unit 602. The signal processing unit 604 generates the new target value based on the movement of the camera module. Figure 6 and Figure 8 As can be understood from the comparison, the functions related to OIS according to this embodiment can be implemented by adding new functions to the signal processing unit, and conventional components can be used for other components of the camera module 100. Therefore, a detailed description of other components will be omitted.

[0105] Next, we will refer to Figure 9 The flowchart in this embodiment describes a method for deriving target values.

[0106] In step S1, the motion of the compensation target object is detected in each frame of the moving image. In this embodiment, the compensation target object is a camera module. The compensation target object can be an optical system such as a lens, camera module, or image sensor.

[0107] In step S2, the compensation target object is moved to counteract the detected motion of the compensation target object, and image blur is compensated. Here, the period corresponding to each frame includes an exposure period for performing exposure and a blank period for pausing exposure. In image blur compensation, compensation is started so that compensation is performed in at least a portion of the exposure period, and compensation is paused so that compensation is not performed in at least a portion of the blank period.

[0108] Figure 10 To show the displacement curve of the camera module, the vertical axis indicates the angular displacement of the camera module, and the horizontal axis indicates time. For example... Figure 10 As shown, the period of a frame when capturing a moving image consists of an exposure period and a blanking period. The duration of the frame is synchronized by a sync signal. Here, the sync signal is a signal provided from the image sensor at the beginning of the frame. The white portion in the figure indicates the exposure period, and the gray portion indicates the blanking period. The sync signal does not necessarily need to match the start of the exposure period. If the timing of the exposure period can be derived based on the sync signal output from the image sensor, such timing can be used for optical image stabilization in this embodiment.

[0109] In the algorithm of this embodiment, the signal processing unit 804 can receive, for example... Figure 10 The continuous motion signal shown is similar to that of conventional OIS. In this embodiment, a new target value is generated specifically to suppress image blur during the exposure cycle. During the blanking period, the feedback control system, including the feedback control unit 602, and the actuator 104 do not need to follow the motion signal. That is, as Figure 11 As shown, only a portion of the motion signal within the exposure cycle is used for OIS control. Figure 11 The segmented target values ​​shown include an offset (Direct Current (DC) component) from a reference position at the start of the exposure cycle. Due to the nature of OIS, it is not necessary to align the position of the compensation target object indicated by the target value with the position of the compensation target object operated by actuator 104. OIS is usable if the displacement velocity of the compensation target object indicated by the target value can be matched with the velocity of the compensation target object operated by actuator 104. Accordingly, the DC component can be removed from the target value, leaving only the velocity information of the compensation target object, and this component moves parallel to the origin, such as... Figure 12 As shown. Thus, the target value for parallel movement is used as the new target value. During the blank period, since the object is not exposed, the compensation target object can be moved to a convenient position at the start of the next frame within the blank period. In this way, based on the motion of the compensation target object detected in the previous OIS control cycle, the compensation target object is moved to maximize its operable range during the blank period. This is a summary of the algorithm for calculating the target value according to this embodiment.

[0110] Next, we will refer to Figure 13 This describes a method for generating new target values ​​according to this embodiment.

[0111] Figure 13 The relationship between the exposure cycle, blank cycle, frame synchronization signal, OIS control cycle, and return cycle is shown. The return cycle is the period during which the camera returns to the reference position for each frame while capturing moving images. Figure 13 The diagram shows the Nth frame and a portion of the frames before and after it. The vertical axis represents the displacement of the camera module 100, and the horizontal axis represents time. Each frame includes an exposure period and a blank period without exposure. The start time of each frame coincides with the timing of the frame synchronization signal received by the signal processing unit 802.

[0112] The OIS control cycle is calculated based on the synchronization signal from the image sensor, the image sensor's specifications, or exposure information.

[0113] The exposure cycle is initiated by the signal processing unit 802, with a slight delay upon receiving the frame synchronization signal. The position of the camera module 100 is a reference position (zero position) at the start of the exposure cycle and gradually increases over time. The signal processing unit 802 generates a target value for moving the camera module 100 to compensate for the displacement of the camera module 100.

[0114] When the exposure cycle ends, the OIS control cycle ends. The signal processing unit 802 generates a target value for moving the camera module 100 to a reference position between the beginning and end of the blank cycle. The solid line 1301 represents the displacement of the camera module 100 when it moves to the reference position at a constant speed between the beginning and end of the blank cycle. The dashed line 1303 indicates the displacement of the camera module 100 when it moves rapidly at the beginning of the blank cycle. The dashed line 1302 indicates the displacement of the camera module 100 when it moves at a constant speed, causing it to return to the reference position within the blank cycle.

[0115] A new target value for the return-to-reference-position period can be determined based on the time length and power consumption of the return-to-reference-position period.

[0116] With the frame length remaining constant, the exposure period becomes longer, and the OIS control period also becomes longer. Because the blank period is shorter, the time it takes for the camera module 100 to return to the reference position is reduced. In this case, it is preferable to quickly return the camera module 100 to the reference position and set the target value, causing the camera module 100 to move as shown by the dashed line 1303.

[0117] If the exposure period is short and the time it takes for the camera module 100 to return to the reference position is long, a target value can be set to make the camera module 100 move slowly. In this case, power consumption can be reduced.

[0118] Furthermore, if the camera module 100 balances the speed and power consumption of returning to the reference position, a target value can be generated so that the camera module 100 moves as shown by the dashed line 1302.

[0119] Figures 14 to 23 Simulation results for the target values ​​according to this embodiment and the comparative example are shown. The same motion signal curves are used in all these figures.

[0120] Figure 14 The simulation results of calculating the target value according to this embodiment are shown. Figure 14 (a) in the figure shows the angular displacement and motion of the image indicated by the motion signal. Figure 14 (b) shows the calculated new target value and the lens motion as the camera module moves, such as Figure 14 As shown in (a) in the diagram. The portion enclosed by the ellipse indicates the state before transitioning to a translation-pitch state. Figure 15 It shows in Figure 14 Simulation results of OIS control in the elliptical closed portion of (a) in the figure. Figure 15The solid curve in (a) shows the value of the camera shake signal detected from the original motion of the camera module, and the dashed curve shows the motion of the image after optical image stabilization according to this embodiment. Figure 15 In (b) of the diagram, the solid curve indicates that the lens moves with the camera module. Figure 15 The dashed curve in (b) shows the target value calculated within the corresponding period. Figure 15 In the dashed curve of (a) in the figure, reference numeral 1501 indicates the exposure period. Because OIS control is effectively implemented, the image does not shift within the exposure period. Therefore, when the image generated within the exposure period is displayed on the monitor within the period of each frame, a stable and unblurred image can be displayed.

[0121] Figure 16 Simulation results are shown as a comparative example, demonstrating the calculation of the target value using a traditional method for calculating the target value. The target value is calculated using, for example... Figure 10 It is calculated using the continuous motion signal shown. Figure 16 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 16 (b) shows the calculated target value and the motion of the lens. Figure 17 It shows Figure 16 Simulation results for OIS control in the portion enclosed by the ellipse in (a) of the diagram. Within the region enclosed by the ellipse, the actuator moves within mechanical constraints. Therefore, as... Figure 17 As shown in (a), the image is essentially static, and as Figure 17 As shown in (b), the movement of the lens is effectively controlled.

[0122] Figure 18 The simulation results of calculating the target value according to this embodiment are shown. Figure 18 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 18 (b) shows the calculated target value and the lens motion. In the portion enclosed by the ellipse, the imaging device switches to translation-pitch mode. Figure 19 It shows Figure 18 The simulation results for the OIS-controlled portion of the ellipse enclosed in (a) are shown. Figure 19 In (a), the solid curve indicates the camera shake signal detected from the original motion of the camera module, and the dashed curve indicates the motion of the image after optical image stabilization according to this embodiment. Figure 19 In (b) of the diagram, the solid curve indicates the movement of the lens as the camera module moves, while the dashed curve indicates the target value calculated within the corresponding period. Figure 19In the dashed curve of (a) in the figure, reference numeral 1901 indicates the exposure period. Because OIS control is effectively performed during exposure, the image does not move. Therefore, a stable and unblurred image can be displayed in pan-tilt mode.

[0123] Figure 20 Simulation results of target value calculation using the traditional method for calculating target values ​​are shown as a comparative example. Figure 20 (a) in the figure shows the angular displacement and motion of the image indicated by the motion signal. Figure 20 (b) shows the calculated target value and the movement of the lens. Figure 21 It shows Figure 20 Simulation results for OIS control in the elliptical region (a) are shown. Within the elliptical region, the actuator's motion control exceeds mechanical limits. Figure 21 In (a) of the figure, within the exposure period indicated by reference numeral 2101, the image continues to move, and the lens movement almost stops, as shown. Figure 21 As shown in (b) of the diagram.

[0124] Figure 22 Simulation results of power consumption controlled by OIS according to this embodiment are shown. Figure 22 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 22 (b) shows the calculated target value and the camera movement. Figure 22 (c) shows the simulation results of power consumption. In the OIS control of this simulation, as... Figure 13 As indicated by reference numeral 1302 in the accompanying drawings, the camera module 100 is moved at a constant speed using control, so that the camera module 100 returns to the reference position during a blank period. Additionally, Figure 23 Simulation results of power consumption in OIS control based on a comparative example are shown. Figure 23 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 23 (b) shows the calculated target value and the camera movement. Figure 23 (c) shows the simulated power consumption results. The effective current value of OIS control according to this embodiment is 9.36 mARMS, while the effective current value of OIS control according to the comparative example is 14.9 mARMS. Therefore, it can be understood that this embodiment can reduce power consumption.

[0125] Figure 24 This illustrates a situation where the exposure cycle and OIS control cycle are mismatched. Depending on the application, all areas of the image sensor may not require OIS control. In this case, OIS control can be performed only in the necessary time domain. Figure 24In the example shown, the OIS control cycle is set as the effective cycle of the exposure cycle. Solid line 2401 indicates the displacement of camera module 100 when it moves at a constant speed to the reference position between the end of the effective cycle and the start of the next effective cycle. Dashed line 2403 indicates the displacement of camera module 100 when it moves rapidly. Dashed line 2402 indicates the displacement of camera module 100 when it moves at a constant speed, causing it to return to the reference position within a blank cycle.

[0126] Furthermore, the response of the feedback control unit 602 may be disturbed immediately after the start or end of the OIS control cycle. Accordingly, the start of the exposure cycle and the start of the OIS control cycle may be different. Furthermore, the end of the exposure cycle and the end of the OIS control cycle may be different. Therefore, optical image stabilization can be used to start so that optical image stabilization is performed for at least a portion of the exposure cycle (first cycle) and to stop so that optical image stabilization is not performed for at least a portion of the blank cycle (second cycle). Figures 25 to 27 Some examples of methods for controlling this optical image stabilization are shown.

[0127] exist Figure 26 In the example shown, the OIS control cycle begins between the start time of the Nth frame and the start time of the exposure cycle, and the OIS control cycle ends immediately after the exposure cycle ends. The remainder of the frame cycle can be set as a return cycle for moving the camera module 100 to the reference position.

[0128] exist Figure 26 In the example shown, the OIS control cycle begins between the start time of the Nth frame and the start time of the exposure cycle, and the OIS control cycle ends in the middle of the exposure cycle.

[0129] exist Figure 27 In the example shown, the OIS control cycle begins after the start of the exposure cycle and ends immediately after the end of the exposure cycle.

[0130] exist Figures 25 to 27 In any of the examples shown, providing a period during the exposure period in each frame to perform OIS control and a period during the blank period to not perform OIS control enables the suppression of image blur in the pan-pitch state and the improvement of the quality of moving images.

[0131] (Second Embodiment)

[0132] Next, a second embodiment of the invention will be described, in which the reference position of the camera module 100 is adaptively set. Typically, the reference position is set to the position (zero position) where the mechanical or optical center of the actuator 104 is located at the center of the image sensor. Here, the mechanical center of the actuator 104 is an example of the center of the mechanism used to move the compensation target object. Furthermore, the optical center of the actuator 104 is an example of the optical center of the compensation target object. Figure 13 and Figure 24 In this configuration, the reference position is set according to this convention. Based on the method used to set the reference position, the DC component of the camera module's displacement has been removed; therefore, a significant reduction in power consumption can be expected compared to traditional OIS methods. To further reduce power consumption, the reference position can be set as follows: Figure 28 As shown. In Figure 28 In the example shown, the reference position is set such that the amplitude of the camera module 100 crosses the zero position, and the amplitude is approximately equal in both the positive and negative directions. A method for calculating the target value according to this embodiment will be described. The displacement within an OIS control cycle in a given frame is taken as A. The reference position within the same frame can be calculated as B = –A / 2, taking the displacement of the original reference position (zero position) as B. The signal processing unit 802 generates a target value to move the position of the camera module 100 to the reference position B determined by this formula. By setting the target value in this way, the target value moves from curve 2801 to curve 2802. In this way, during a blank cycle, based on the movement of the compensation target object detected in the previous OIS control cycle, processing is performed to move the compensation target object to maximize the operability range of the compensation target object. Furthermore, since the amplitude starting from the zero position is reduced by approximately half, power consumption is further reduced.

[0133] Figure 29 Simulation results of power consumption controlled by OIS according to this embodiment are shown. Figure 29 (a) shows the angular displacement and motion of the image indicated by the motion signal. Figure 29 (b) shows the calculated new target value and the camera movement. Figure 29 (c) shows the simulation results of power consumption. In the OIS control of this simulation, as... Figure 28 As indicated by reference numeral 2802 in the accompanying drawings, the camera module 100 is moved at a constant speed by control, causing the camera module 100 to return to the reference position within a blank period. The effective current value of the OIS control according to this embodiment is 6.53 mARMS. Therefore, it can be understood that this embodiment can further reduce power consumption.

[0134] (Third embodiment)

[0135] Next, we will refer to Figure 30The third embodiment of the present invention is described below.

[0136] If the target object reaches the limit of the actuator's operable range within the OIS control cycle, the movement of the camera module 100 is paused. As indicated by solid line 3001, the camera module 100 reaches the maximum value Y1 of the actuator's operating range at time T1, and movement stops until the end of the OIS control cycle at time T2.

[0137] At this time, the signal processing unit 802 sends a request signal to the feedback control unit 602 to shorten the exposure cycle starting from the next frame. The feedback control unit 602 shortens the exposure cycle after the next frame based on the received request signal. The exposure cycle of the (N+1)th frame is shorter than the exposure cycle of the Nth frame. By controlling the OIS control cycle in this way, the limitation of compensating the target object to reach the actuator's operable range can be avoided, or the time when OIS control becomes impossible can be shortened. Therefore, it is possible to shorten the cycle where the stabilization effect is compromised.

[0138] This embodiment can be combined with the second embodiment described above. In this case, the signal processing unit 804 sets a reference position such that the amplitude of the camera module 100 crosses zero and the amplitude is approximately equal in both positive and negative directions. In this case, the prediction range for compensating the target object is within the operable range of the actuator 104.

[0139] The algorithm for deriving target values ​​according to the present invention is applicable to optical image stabilization systems of any product, regardless of the physical embodiment.

[0140] According to the above embodiments, the target value of the compensation target object can be repeatedly set in sync with the frame rate of the moving image. That is, the operation of setting a new target value and moving the compensation target object is a 60 Hz loop operation for a 30 fps video, and a 60 fps loop operation for either a 30 Hz or 60 fps video. Therefore, it is desirable to... Figure 7 The transfer function of the target value and the displacement output of the actuator has a flat frequency response, which is sufficient to handle the required frame rate of the motion image.

[0141] The above embodiments can be applied to all optical devices capable of using optical image stabilization. For example, the above embodiments can be applied to built-in cameras in smartphones, tablets, mobile cameras, interchangeable-lens cameras, surveillance cameras, vehicle cameras, and aircraft cameras.

[0142] The above description is merely a specific implementation of the present invention and is not intended to limit the scope of protection of the present invention. Any variations or substitutions that can be readily conceived by those skilled in the art within the scope of the disclosed technology should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An optical image stabilization system, characterized by comprising: comprising: detecting motion of a compensation target object in each frame of a motion image, wherein the compensation target object is an optical system, a camera module or an image sensor; driving to perform blur compensation of the image by moving the compensation target object to counteract the detected motion of the compensation target object, wherein a period corresponding to each frame comprises a first period in which exposure is performed and a second period in which the exposure is suspended, wherein the driving initiates the compensation so as to perform the compensation at least for a portion of the first period and suspends the compensation so as to not perform the compensation at least for a portion of the second period.

2. The optical image stabilization system according to claim 1, wherein The driving further moves the compensation target object to a predetermined reference position within the second period.

3. The optical image stabilization system of claim 2, wherein The reference position is a center of a mechanism for moving the compensation target object.

4. The optical image stabilization system of claim 2, wherein The reference position is an optical center of the compensation target object.

5. The optical image stabilization system of claim 2, wherein The driving further moves the compensation target object based on the motion of the compensation target object detected in a previous first period to maximize an operable range of the compensation target object in the second period.

6. The optical image stabilization system of claim 5, wherein The driving further shortens an exposure period in a subsequent frame when the motion of the compensation target object detected in the previous first period exceeds the operable range of the compensation target object.

7. The optical image stabilization system of claim 2, wherein The driving further moves the compensation target object to the reference position at a constant speed within a period from the end of the compensation to the start of a next compensation.

8. The optical image stabilization system of claim 2, wherein The driving further moves the compensation target object so that a period in which the compensation target object stops moving at the reference position occurs within a period from the end of the compensation to the start of a next compensation.

9. The optical image stabilization system of claim 1, wherein, A start time of the compensation is different from a start time of an exposure period.

10. The optical image stabilization system of claim 1, wherein An end time of the compensation is different from an end time of an exposure period.

11. An optical image stabilization method, characterized by, comprising: detecting motion of a compensation target object in each frame of a motion image, wherein the compensation target object is an optical system, a camera module or an image sensor; driving to perform blur compensation of the image by moving the compensation target object to counteract the detected motion of the compensation target object, wherein a period corresponding to each frame comprises a first period in which exposure is performed and a second period in which the exposure is suspended, wherein in the performing compensation, the compensation is initiated at least for a portion of the first period and the compensation is suspended so as to not perform the compensation at least for a portion of the second period.

12. The method of claim 11, wherein, The performing compensation comprises: moving the compensation target object to a predetermined reference position in the second period.

13. The method of claim 12, wherein, The reference position is a center of a mechanism for moving the compensation target object.

14. The method of claim 12, wherein, The reference position is an optical center of the compensation target object.

15. The method of claim 12, wherein, The performing compensation comprises moving the compensation target object based on the motion of the compensation target object detected in a previous first period to maximize an operable range of the compensation target object in the second period.

16. The method of claim 15, wherein, further comprising: shorten an exposure period in a later frame when the motion of the compensation target object detected in the previous first period exceeds the operable range of the compensation target object.

17. The method of claim 12, wherein, The moving the compensation target object to a predetermined reference position includes moving the compensation target object to the reference position at a constant speed in a period from the end of the compensation to the start of the next compensation.

18. The method of claim 12, wherein, The moving the compensation target object to a predetermined reference position includes moving the compensation target object such that a time period in which the compensation target object stops moving at the reference position occurs in a period from the end of the compensation to the start of the next compensation.

19. The method of claim 11, wherein, A start time of the compensation is different from a start time of an exposure period.

20. The method of claim 11, wherein, An end time of the compensation is different from an end time of the exposure period.

21. A camera module, comprising: An optical image stabilization system according to any one of claims 1 to 10.

22. An imaging device, characterized by An optical image stabilization system according to any one of claims 1 to 10.

Images