Optical anti-shake device and optical anti-shake method

Abstract

The embodiment of the invention discloses an optical anti-shake device and an optical anti-shake method. The optical anti-shake device can comprise a motor driver and an image sensor physically connected with the motor driver. The image sensor is used for reporting the working state of the image sensor to the motor driver; the working state comprises one or more of an exposure state, a reading state and an inter-frame idle state; the motor driver is used for monitoring the working state of the image sensor and adjusting the driving state of the motor driver to a target driving state based on the working state; the driving state comprises a driving mode and a driving frequency. The imaging quality can be improved, and power consumption can be reduced.

Description

Technical Field

[0001] The present application relates to the field of image imaging technology, and in particular to an optical image stabilization device and an optical image stabilization method. Background Art

[0002] During handheld photography, due to some factors, the phone cannot be completely stable. In order to improve the camera effect, anti-shake processing is required. Common anti-shake methods include optical image stabilization (OIS) and electronic image stabilization (EIS). Among them, electronic image stabilization (EIS) technology is an algorithm based on software post-processing; optical image stabilization (OIS) is a method of correcting shaking by causing the optical lens (Lens) or image sensor (Sensor) to have a relative displacement. Through the built-in detection device of the mobile phone, the shaking information of the human hand is obtained, and the displacement required to offset the anti-shake is calculated by the algorithm, and the motor is controlled to push the lens or image sensor to have a relative displacement to complete the anti-shake.

[0003] In optical image stabilization (OIS), the lens or image sensor is displaced by a motor, and a relatively large current is required in the process of pushing the lens or image sensor. That is, the average current flowing through the motor over a period of time determines the position of the motor. The current is usually provided by the mobile phone's power management unit (PMU), and the motor driver controls the current intensity, mode and driving frequency.

[0004] In the prior art, see Figure 1A , Figure 1A The current path inside the driver. The motor driver will control the current intensity of the motor according to the anti-shake requirements. There is usually a device such as a MOS tube or a triode on the current path. When supplying current to the motor, please refer to Figure 1B , Figure 1B It is the equivalent current path inside the driver. At this time, the device on the current path can be equivalent to an on-resistance Ron, and power consumption will be wasted on Ron. In addition, the motor driver driving the motor is an automated closed-loop system. The higher the frequency of the motor driving, the better the closed-loop system performance and the better the effect of responding to OIS displacement, but the power consumption will also increase accordingly. Summary of the invention

[0005] The embodiments of the present application provide an optical image stabilization device and an optical image stabilization method, so as to reduce the power consumption of the optical image stabilization device while ensuring the imaging quality.

[0006] In a first aspect, an embodiment of the present application provides an optical image stabilization device, which may include a motor driver and an image sensor physically connected to the motor driver; the image sensor is used to report its own working status to the motor driver; the working status includes one or more of an exposure state, a readout state, and an inter-frame idle state; the motor driver is used to monitor the working status of the image sensor and adjust its own driving state to a target driving state based on the working status; the driving state includes a driving mode and a driving frequency.

[0007] The embodiment of the present application provides an optical image stabilization device, which mainly includes the design of a structure and drive adjustment strategy of an optical device, wherein the structure includes a motor driver and an image sensor that are physically connected, wherein the image sensor directly reports its different states during operation, such as exposure, readout, or idle state between frames to the motor driver through a physical connection, and the motor driver can adjust its own drive state based on the different working states of the image sensor, specifically including different drive modes and different drive frequencies, so as to better adjust the motor, greatly reduce power consumption and improve motor performance while ensuring imaging quality. In summary, the embodiment of the present application provides a new low-power management solution for the motor driver, which adopts an image sensor (Sensor) component to be linked with a motor driver (Driver), and the motor driver accurately obtains the working state of the image sensor, and accurately controls the drive mode and drive frequency of the motor according to the different working states of the image sensor, thereby reducing power consumption while ensuring shooting quality.

[0008] In a possible implementation, the driving mode includes one or more of linear driving, pulse width modulation (PWM) driving, and dynamic linear driving. In the embodiment of the present application, the driving mode may include multiple modes, such as a linear driving mode without electromagnetic radiation interference but with high power consumption, a dynamic linear driving mode without electromagnetic radiation interference but with low power consumption, a PWM driving mode with electromagnetic radiation but with lower power consumption, etc.

[0009] In a possible implementation, the optical image stabilization device further includes a motor; the motor driver is further used to drive the motor in the target drive state to push the image sensor or lens to make a target displacement. Since the essence of optical image stabilization is to drive the motor through the motor driver, the motor can push the image sensor or lens to make a target displacement to offset the shaking displacement of the human hand, and finally achieve optical image stabilization. That is, a method of correcting the shaking by causing a relative displacement of the image sensor (Sensor). For example, the shaking information of the human hand is obtained through the built-in detection device of the mobile phone, and the displacement required to offset the anti-shake is calculated by the algorithm, so as to control the motor to push the lens or image sensor to make a relative displacement to complete the anti-shake.

[0010] In a possible implementation, when the working state is in the exposure state; the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state. When the image sensor is in the exposure time period of the imaging stage, it has high requirements for optical image stabilization, but is not sensitive to electromagnetic radiation. Specifically, the image sensor is sensitive to human hand shaking in the exposure stage, and keeping the driving frequency in a high-speed state at this time is conducive to improving the imaging quality; in addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the driving mode of the motor driver is a pulse width modulation PWM driving mode, since the current output of the motor driver is a periodic square wave, its output current has a certain frequency and thus generates electromagnetic radiation, which will interfere with the image sensor (may cause image quality degradation), but this method can greatly reduce power consumption, and since the electromagnetic radiation affects the readout circuit of the image sensor (that is, the image sensor in the stage not including the readout state is not affected by the electromagnetic radiation), the above-mentioned image sensor that is only in the exposure state is not sensitive to electromagnetic radiation and is suitable for the PWM driving mode. Therefore, in the embodiment of the present application, when the working state of the image sensor is in the exposure state of the imaging stage, the driving mode of the motor driver can be adjusted to a pulse width modulation (PWM) driving mode (to meet low electromagnetic radiation sensitivity while reducing power consumption) and a high frequency state (to meet high anti-shake requirements) to ensure high imaging quality and low power consumption at the same time.

[0011] In a possible implementation, when the working state is in the exposure state and the readout state at the same time, the target driving state includes a dynamic linear driving mode and a first driving frequency in a high-speed state. When the image sensor is in the exposure and readout time period of the imaging stage, it has high requirements for optical image stabilization and is sensitive to electromagnetic radiation. Specifically, the image sensor is sensitive to human hand shaking during the exposure and readout stages, and maintaining a high-speed driving frequency at this time is beneficial to improving the imaging quality. In addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the driving mode of the motor driver is a dynamic linear driving mode, the current intensity supplied to the motor is always the same as the average current intensity of the motor to maintain its displacement. After the optical image stabilization is turned on, the MOS tube or triode inside the motor driver is always kept on, the voltage at the power supply management terminal is always kept constant, and the power consumption on Ron is (Vin-VO)*I. In this mode, there is no electromagnetic radiation interference to the surrounding devices, and the power consumption is smaller than that of linear driving. Since electromagnetic radiation can affect the readout circuit of the image sensor (that is, the image sensor in the stage including the readout state will be affected by electromagnetic radiation), the above-mentioned image sensor in the exposure and readout time period is sensitive to electromagnetics and is not suitable for PWM driving. Moreover, since the power consumption of the linear driving mode is very large, it is suitable for dynamic linear driving. Therefore, in the embodiment of the present application, when the image sensor is in the exposure and readout state of the imaging stage, the driving mode of the motor driver can be adjusted to a dynamic linear driving mode (to meet high electromagnetic radiation sensitivity while reducing power consumption) and a high frequency state (to meet high anti-shake requirements) to improve imaging quality.

[0012] In a possible implementation, when the working state is in the readout state, the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state. When the image sensor is only in the readout period, it does not have high requirements for optical image stabilization, but is sensitive to electromagnetic radiation. Specifically, the image sensor does not have high requirements for optical image stabilization during the readout stage, and keeping the driving frequency at a low speed at this time is beneficial to reducing power consumption. In addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the driving mode of the motor driver is a dynamic linear driving mode, the current intensity supplied to the motor is always the same as the average current intensity of the motor to maintain its displacement. After the optical image stabilization is turned on, the MOS tube or triode inside the motor driver is always kept on, the voltage at the power supply management terminal is always kept constant, and the power consumption on Ron is (Vin-VO)*I. In this mode, there is no electromagnetic radiation interference to the surrounding devices, and the power consumption is smaller than that of linear driving. Since electromagnetic radiation can affect the readout circuit of the image sensor (that is, the image sensor in the stage including the readout state will be affected by electromagnetic radiation), the above-mentioned image sensor that is in the exposure and readout time period at the same time is sensitive to electromagnetics and is not suitable for PWM driving. Moreover, since the power consumption of the linear driving mode is very large, it is suitable for dynamic linear driving mode. Therefore, in the embodiment of the present application, when the working state of the image sensor is only in the readout state, the driving mode of the motor driver can be adjusted to a dynamic linear driving mode (meeting high electromagnetic radiation sensitivity while reducing power consumption) and a low-speed frequency state (meeting low anti-shake requirements while reducing power consumption) to ensure high imaging quality and low power consumption at the same time. In addition, it can be understood that in the embodiment of the present application, the first driving frequency is greater than the second driving frequency.

[0013] In a possible implementation, when the working state is in an inter-frame idle state; the target drive state includes a PWM drive mode and a second drive frequency in a low-speed state. When the image sensor is in an inter-frame idle stage, it does not require high optical image stabilization and is not sensitive to electromagnetic radiation. Specifically, the image sensor does not require high optical image stabilization in the readout stage, and keeping the drive frequency in a low-speed state at this time is conducive to reducing power consumption; in addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the driving mode of the motor driver is a pulse width modulation PWM driving mode, since the current output of the motor driver is a periodic square wave, its output current has a certain frequency and thus generates electromagnetic radiation, which will interfere with the image sensor (may cause image quality degradation), but this method can greatly reduce power consumption, and since the electromagnetic radiation affects the readout circuit of the image sensor (that is, the image sensor in the stage not including the readout state is not affected by electromagnetic radiation), the above-mentioned image sensor in the inter-frame idle state is insensitive to electromagnetic radiation and is suitable for adopting a PWM driving mode. Therefore, in the embodiment of the present application, when the working state of the image sensor is in the inter-frame idle stage (i.e., the frame interval stage), the driving mode of the motor driver can be adjusted to a PWM driving mode (to meet the low electromagnetic radiation sensitivity while reducing power consumption) and a low-speed frequency state (to meet the low anti-shake requirements while reducing power consumption) to ensure both high imaging quality and low power consumption.

[0014] In one possible implementation, the image sensor is specifically used to: report the exposure status to the motor driver at time t2 before its own exposure start time T2; the motor driver is specifically used to complete switching the drive state to the target drive state before the exposure time T2 after receiving the exposure status reported by the image sensor at time t2.

[0015] In an embodiment of the present application, the image sensor can report its own working status to the motor driver in advance through status output control, so that the motor driver can prepare for switching the driving mode and driving frequency in advance. The reason is that it takes a certain amount of time to switch from one driving state to another. In this way, the motor driver can finally complete the driving mode switching when the image sensor actually enters the corresponding working state, thereby making the driving state of the motor driver and the working state of the image sensor more accurately matched, so as to better achieve high imaging quality and low power consumption.

[0016] In one possible implementation, the image sensor is specifically used to: report the readout status to the motor driver at time t1 before its own start reading time T1; the motor driver is specifically used to complete the switching of the drive state to the target drive state before the readout time T1 after receiving the readout status reported by the image sensor at time t1.

[0017] In an embodiment of the present application, the image sensor can report its own working status to the motor driver in advance through status output control, so that the motor driver can prepare for switching the driving mode and driving frequency in advance. The reason is that it takes a certain amount of time to switch from one driving state to another. In this way, the motor driver can finally complete the driving mode switching when the image sensor actually enters the corresponding working state, thereby making the driving state of the motor driver and the working state of the image sensor more accurately matched, so as to better achieve high imaging quality and low power consumption.

[0018] In a possible implementation, the image sensor is connected to the motor driver via an input / output (IO) interface; the image sensor is specifically configured to report its own working status to the motor driver via the IO interface.

[0019] In an embodiment of the present application, the image sensor and the motor driver are physically connected via an IO interface, and the image sensor can report its own working status to the motor driver in real time through the IO interface, for example, whether it is currently in an exposure state, an exposure and readout state, or an idle state between frames, etc., so that the subsequent motor driver can adjust its own driving mode and driving frequency according to the working state of the image sensor, thereby accurately driving the motor to better achieve high imaging quality and low power consumption.

[0020] In a second aspect, an embodiment of the present application provides an optical image stabilization method, which can be applied to an optical image stabilization device, wherein the optical image stabilization device includes a motor driver and an image sensor physically connected to the motor driver; the method may include:

[0021] Reporting the working status of the image sensor to the motor driver through the image sensor; the working status includes one or more of an exposure state, a readout state, and an inter-frame idle state;

[0022] The working state of the image sensor is monitored by the motor driver, and its own driving state is adjusted to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

[0023] In a possible implementation, the driving mode includes one or more of linear driving, pulse width modulation (PWM) driving, and dynamic linear driving.

[0024] In a possible implementation, the optical image stabilization device further includes a motor; and the method further includes:

[0025] The motor is driven by the motor driver in the target driving state to push the image sensor or the lens to perform target displacement.

[0026] In a possible implementation, when the working state is in an exposure state, the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state.

[0027] In a possible implementation, when the working state is in an exposure state and a readout state at the same time, the target driving state includes a dynamic linear driving mode and a first driving frequency of a high-speed state.

[0028] In a possible implementation, when the working state is in a readout state, the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state.

[0029] In a possible implementation, when the working state is in an inter-frame idle state, the target driving state includes a PWM driving mode and a second driving frequency in a low-speed state.

[0030] In a possible implementation, the reporting of the working status of the image sensor to the motor driver includes: reporting the exposure status to the motor driver at time t2 before the image sensor starts exposure time T2;

[0031] The method of monitoring the working state of the image sensor by the motor driver and adjusting its own driving state to a target driving state based on the working state includes: after receiving the exposure state reported by the image sensor at time t2, the motor driver completes switching the driving state to the target driving state before the exposure time T2.

[0032] In a possible implementation, the reporting of the working status of the image sensor to the motor driver includes: reporting the readout status to the motor driver at time t1 before the image sensor starts to read out the image sensor itself at time T1;

[0033] The method of monitoring the working state of the image sensor by the motor driver and adjusting its own driving state to a target driving state based on the working state includes: after receiving the readout state reported by the image sensor at time t1, the motor driver completes changing the driving state to the target driving state before the readout time T1.

[0034] In a possible implementation, the image sensor is connected to the motor driver via an input / output (IO) interface; and reporting the working status of the image sensor to the motor driver includes:

[0035] The image sensor reports its own working status to the motor driver through the IO interface.

[0036] In a third aspect, an embodiment of the present application provides a camera device, which may include an optical anti-shake device, a motor, an inertial sensor IMU and a motor position detector provided by any one of the implementation methods of the first aspect above; wherein the IMU is used to detect the shake information of the device; the motor driver is used to drive the anti-shake displacement of the motor based on the shake information; and the motor position detector is used to correct the anti-shake displacement of the motor.

[0037] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: an optical image stabilization device and a processor provided by any one of the implementation methods of the first aspect above; the processor is used to send a shooting instruction to an image sensor in the optical image stabilization device; the image sensor is used to enter a corresponding working state according to the received shooting instruction; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state.

[0038] In a possible implementation, the electronic device further includes: a memory for storing program instructions; the program instructions are executed by the processor.

[0039] In a fifth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein:

[0040] The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to execute the following steps:

[0041] Reporting the working status of the image sensor to the motor driver through the image sensor; the working status includes one or more of an exposure state, a readout state, and an inter-frame idle state;

[0042] The working state of the image sensor is monitored by the motor driver, and its own driving state is adjusted to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

[0043] In a sixth aspect, an embodiment of the present application provides a system on chip (SOC), which may include a motor driver and an IO interface; the motor driver is connected to the image sensor through the IO interface;

[0044] The IO interface is used for the image sensor to report its own working state to the motor driver; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state;

[0045] The motor driver is used to monitor the working state of the image sensor through the IO interface, and adjust its own driving state to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

[0046] In a possible implementation, the driving mode includes one or more of linear driving, pulse width modulation (PWM) driving, and dynamic linear driving.

[0047] In a possible implementation, the motor driver is also connected to a motor; and the motor driver is further configured to drive the motor in the target driving state to push the image sensor or the lens to perform target displacement.

[0048] In a possible implementation, when the working state is in an exposure state, the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state.

[0049] In a possible implementation, when the working state is in an exposure state and a readout state at the same time, the target driving state includes a dynamic linear driving mode and a first driving frequency of a high-speed state.

[0050] In a possible implementation, when the working state is in a readout state, the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state.

[0051] In a possible implementation, when the working state is in an inter-frame idle state, the target driving state includes a PWM driving mode and a second driving frequency in a low-speed state.

[0052] In one possible implementation, the IO interface is specifically used for: the image sensor reports the exposure status to the motor driver at time t2 before the image sensor starts exposing itself T2; the motor driver is specifically used for, after receiving the exposure status reported by the image sensor through the IO interface at time t2, completing the switching of the driving state to the target driving state before the exposure time T2.

[0053] In one possible implementation, the IO interface is specifically used for: the image sensor reports the readout status to the motor driver at time t1 before the time T1 at which the image sensor starts reading out itself; the motor driver is specifically used for, after receiving the readout status reported by the image sensor through the IO interface at time t1, completing the switching of the drive state to the target drive state before the readout time T1.

[0054] In a seventh aspect, the present application provides a semiconductor chip, which may include a SOC provided by any one of the implementation methods in the sixth aspect above.

[0055] In an eighth aspect, the present application provides a semiconductor chip, which may include: a SOC provided by any one of the implementations in the sixth aspect above, an internal memory coupled to the SOC, and an external memory.

[0056] In a ninth aspect, the present application provides an optical image stabilization system-on-chip (SOC), which includes the SOC provided by any one of the implementations in the sixth aspect, and an internal memory and an external memory coupled to the SOC. The optical image stabilization SOC may be composed of a chip, or may include a chip and other discrete devices.

[0057] In a tenth aspect, the present application provides a chip system, which may include an optical image stabilization device provided by any one of the implementations of the first aspect. In a possible design, the chip system also includes a memory, which is used to store program instructions and data necessary or related to the optical image stabilization device during operation. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0058] In an eleventh aspect, the present application provides an optical image stabilization device, which has the function of implementing the optical image stabilization method provided by any one of the implementations in the second aspect above. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0059] In a twelfth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable medium is used to store program code, and when the program code is executed by a processor, the optical image stabilization method provided by any one of the implementation methods in the above-mentioned second aspect is implemented.

[0060] In a thirteenth aspect, an embodiment of the present application provides a computer program, which includes instructions. When the instructions are executed by a processor, the processor executes the optical image stabilization method provided by any one of the implementation methods in the above-mentioned second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0062] Figure 1A It is the current path inside the driver.

[0063] Figure 1B It is the equivalent current path inside the driver.

[0064] Figure 2A It is a schematic diagram of linear voltage regulation corresponding to the linear driving method in the prior art.

[0065] Figure 2B It is a schematic diagram of dynamic linear voltage regulation corresponding to the dynamic linear driving method in the prior art.

[0066] Figure 2C It is a schematic diagram of driver current output corresponding to the pulse width modulation driving method in the prior art.

[0067] Figure 2D This is another schematic diagram of driver current output corresponding to the pulse width modulation driving method in the prior art.

[0068] Figure 2E It is a schematic diagram of driver current output corresponding to the combination of PWM and linear drive in the prior art.

[0069] Figure 3A A schematic diagram of a scenario for anti-shake application of mobile phone camera photography provided by an embodiment of the present invention.

[0070] Figure 3B A schematic diagram of a scenario of optical image stabilization for a drone provided by an embodiment of the present invention.

[0071] Figure 3C A schematic diagram of a scenario of intelligent vehicle-mounted optical image stabilization provided by an embodiment of the present invention.

[0072] Figure 4A It is a schematic diagram of the hardware structure of an optical image stabilization device provided in an embodiment of the present application.

[0073] Figure 4B It is a schematic diagram of the hardware structure of another optical image stabilization device provided in an embodiment of the present application.

[0074] Figure 4C This is a schematic diagram of the hardware structure of another optical image stabilization device provided in an embodiment of the present application.

[0075] Figure 4D This is a schematic diagram of the hardware structure of another optical image stabilization device provided in an embodiment of the present application.

[0076] Figure 5A A schematic diagram of a working range of an image sensor provided in an embodiment of the present application.

[0077] Figure 5B A schematic diagram of image sensor status notification and switching timing provided in an embodiment of the present application.

[0078] Figure 5C A schematic diagram of a motor driver working range provided in an embodiment of the present application.

[0079] Figure 5D A schematic diagram of the linkage operation of an image sensor and a motor driver provided in an embodiment of the present application.

[0080] Figure 6 It is a flowchart of an optical image stabilization method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0082] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0083] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0084] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0085] First, some terms in this application are explained to facilitate understanding by those skilled in the art.

[0086] (1) Charge-coupled device (CCD) can be called CCD image sensor, which can convert optical images into digital signals. There are many neatly arranged capacitors on CCD, which can sense light and convert images into digital signals. CCD works like film, but it converts image pixels into digital signals. The tiny photosensitive substances implanted on CCD are called pixels. The more pixels a CCD contains, the higher the image resolution it provides. CCD cameras composed of them are widely used because they are small in size, light in weight, unaffected by magnetic fields, and resistant to vibration and impact.

[0087] (2) Complementary Metal-Oxide-Semiconductor (CMOS) image sensor is a typical solid-state imaging sensor. CMOS image sensor usually consists of image sensitive unit array, row driver, column driver, timing control logic, analog to digital (AD) converter, data bus output interface, control interface and other parts, which are usually integrated on the same silicon chip. Its working process can generally be divided into reset, photoelectric conversion, integration and readout.

[0088] (3) Optical Image Stabilizer (OIS), also known as optical image stabilizer, is a device that corrects "optical axis deviation" through the floating lens of the lens. It is also called optical image stabilization.

[0089] (4) Electronic Image Stabilization (EIS) mainly refers to the anti-shake method used in digital cameras to forcibly increase the CCD photosensitivity parameters while speeding up the shutter and analyzing the image obtained on the CCD, and then using the edge image to compensate. Electronic anti-shake is actually a technology that compensates for jitter by reducing image quality. This technology attempts to achieve a balance between image quality and image jitter. Compared with optical anti-shake, this technology is much cheaper (in fact, it can be achieved by only making some adjustments to the internal software of ordinary digital cameras), and the effect is also worse.

[0090] (5) Pulse width modulation (PWM): Pulse width modulation is an analog control method that modulates the bias of the transistor base or MOS tube gate according to the change of the corresponding load to achieve the change of the transistor or MOS tube conduction time, thereby achieving the change of the output of the switching power supply. This method can keep the output voltage of the power supply constant when the working conditions change. It is a very effective technology for controlling analog circuits using digital signals of microprocessors.

[0091] (6) Exposure area (exposure) and readout area (ReadOut). The image sensor (Sensor) exposes row by row starting from the first row. After one row cycle is completed, the second row starts to be exposed. Similarly, after N-1 rows, the Nth row starts to be exposed. After the first row is exposed, the data is read out. It takes a row cycle time (including the row blanking time, i.e. H_Blank) to read out a row. After the first row is completely read out, the second row starts to be read out. Similarly, after the N-1th row is read out, the Nth row starts to be read out, until the entire image is completely read out. Among them, the exposure area (exposure) and the readout area (ReadOut) will overlap in time.

[0092] First, in order to facilitate the understanding of the embodiments of the present application, the technical problems to be solved by the present application are further analyzed and proposed. In the prior art, there are a variety of technical solutions for managing the current and drive frequency of the motor driver (Driver). The following are four commonly used technical solutions. The details are as follows:

[0093] Solution 1, linear drive:

[0094] See also Figure 2A , Figure 2AThis is a schematic diagram of linear voltage regulation corresponding to the linear drive method in the prior art. The current intensity supplied to the motor is always the same as the average current intensity of the motor to maintain its displacement. After the optical image stabilization is turned on, the MOS tube or triode inside the driver Driver is always turned on, and the voltage at the PMU terminal is always kept constant. The power consumption on Ron is (Vin-VO)*I. In this way, there is no electromagnetic radiation interference to the surrounding devices, but the power consumption is very large.

[0095] Solution 2: Dynamic linear drive:

[0096] See also Figure 2B , Figure 2B The figure is a schematic diagram of dynamic linear voltage regulation corresponding to the dynamic linear drive method in the prior art. The current intensity supplied to the motor is always the same as the average current intensity of the motor to maintain its displacement. After the optical image stabilization is turned on, the MOS tube or triode inside the driver Driver always remains in the open state, and the voltage at the PMU terminal changes with the average current of the motor. The voltage at both ends can always maintain a relatively small difference. In this way, there is no electromagnetic radiation interference to the surrounding devices, and the power consumption is smaller than that of linear drive.

[0097] Solution 3: Pulse Width Modulation (PWM) drive:

[0098] See also Figure 2C and Figure 2D , Figure 2C Schematic diagram of driver current output corresponding to the pulse width modulation driving method in the prior art, Figure 2D This is another driver current output diagram corresponding to the pulse width modulation driving method in the prior art. The current output of the driver is a periodic square wave. By changing the duty cycle or frequency of the square wave, the average current of the motor is adjusted to achieve position change. This method can greatly reduce power consumption, but because the output current has a certain frequency, it will generate electromagnetic radiation, which will interfere with surrounding devices such as image sensors, causing image quality degradation.

[0099] Solution 4: Combination of PWM and linear drive:

[0100] See also Figure 2E , Figure 2E This is a schematic diagram of the driver current output corresponding to the combination of PWM and linear drive in the prior art. In the time period when the peripheral devices are sensitive to electromagnetic radiation, the linear drive method is used, and in the insensitive time period, the PWM drive method is used. At present, this method relies on the sensor output frame or indirect output signal to judge or predict the electromagnetic radiation sensitive time period, resulting in the power consumption optimization not being able to achieve the best, and the space radiation cannot be completely avoided. At the same time, the power consumption of the linear drive stage is relatively large.

[0101] In summary, the above four driving schemes all have their own defects, and cannot achieve a balance between shooting quality and power consumption, and there are problems such as excessive power consumption and poor shooting image quality. Therefore, the technical problems to be solved by this application may include the following aspects: providing a motor driver (Driver) current and driving frequency management method, while reducing the internal power consumption of the motor driver, avoiding the impact on other components such as the image sensor, that is, reducing power consumption while ensuring shooting quality.

[0102] The optical image stabilization device in this application can be applied to a variety of electronic devices with optical image stabilization shooting functions, such as smart phones, smart glasses, smart wearable devices, augmented reality devices, digital cameras, video cameras, portable shooting devices, optical devices, drones, smart bracelets, robots, etc., which are not listed here one by one. The following exemplary lists the scenarios in which the optical image stabilization device in this application is applied, which may include the following three scenarios.

[0103] Scenario 1: The optical image stabilization device in this application is used to realize the anti-shake function of mobile phone photography:

[0104] See also Figure 3A , Figure 3A A schematic diagram of a scenario of a mobile phone camera anti-shake application provided by an embodiment of the present invention. In this application scenario, when the touch screen of the mobile phone detects a user's camera operation, the mobile phone responds to the camera operation and can display Figure 3A The exemplary user photo taking interface 31 is shown. Figure 3A , Figure 3A The user photo interface 31 and the user photo interface 32 of the "camera" application on an electronic device such as a smart phone are exemplarily shown. The user photo interface 31 may include a photo viewing control 301, a photo control 302, a camera switching control 303, and a shooting interface 304. Further, the user photo interface 31 may also include an optical image stabilization control 305. When the user wants to perform anti-shake shooting (including taking photos or videos), the optical image stabilization function can be turned on by clicking the optical image stabilization control 305 (user photo interface 31), such as "optical image stabilization is turned on" shown in the user photo interface 32, that is, the mobile phone can realize the optical image stabilization function for the user through the relevant functions of any optical image stabilization device provided in this application to improve the shooting quality. Optionally, the bottom of the user photo interface 31 or the user photo interface 32 may also include a shooting mode option list, for example, including: night scene mode option, portrait mode option, photo mode option, video mode option and more shooting mode options.

[0105] Scenario 2: The optical image stabilization device in this application is used to realize the shooting function of the drone's shooting system:

[0106] See also Figure 3B , Figure 3B A schematic diagram of a scenario of optical image stabilization of a drone provided in an embodiment of the present invention, wherein the drone in the application scenario may include a fuselage, a power system, a flight control system, a link system, a mission payload, and the like. Among them, the drone's shooting system can not only perform aerial photography, but also can be used for forest fire prevention, earthquake investigation, nuclear radiation detection, border patrol, emergency rescue, crop yield estimation, pipeline inspection, wildlife monitoring in protected areas, military reconnaissance, and the like. However, during the aerial photography process, it is often difficult to ensure the stability of the picture due to high-altitude airflow, resistance, and the vibration of the aircraft itself. The drone can improve the shooting quality of the drone through the relevant functions of any optical image stabilization device provided in this application. For example, the load of an aerial drone generally includes the following: a camera, a gimbal, and an image transmission system, wherein the camera may include any optical image stabilization device provided in this application to improve the shooting quality of the drone.

[0107] Scenario 3: The optical image stabilization device in this application is used to realize the shooting function of the smart car-mounted shooting system:

[0108] See also Figure 3C , Figure 3C A schematic diagram of a scenario of intelligent vehicle-mounted optical image stabilization provided in an embodiment of the present invention. The vehicle-mounted intelligent system in this application scenario may include an intelligent driving system, an intelligent interconnection system, an intelligent monitoring system, etc., wherein the vehicle-mounted monitoring system is a system that monitors the internal and external environment of the vehicle by using technologies such as computer vision, the Internet of Things, and artificial intelligence, and provides real-time alarms and data recording. It can help drivers predict potential dangers and improve driving safety. The monitoring system may include at least cameras, radars, infrared sensors, etc. For example, safety monitoring: monitor the vehicle and the surrounding environment through cameras and sensors inside and outside the vehicle, and provide instant alarms to help drivers avoid potential dangers. For another example, behavior analysis: by analyzing the movement of people and objects inside and outside the vehicle, the intelligent vehicle-mounted monitoring system can help identify abnormal behaviors, such as driver fatigue, distraction, drinking, etc. The cameras inside and outside the vehicle can include any one of the optical image stabilization devices provided in this application to improve the shooting quality of the vehicle-mounted scene.

[0109] Understandably, Figure 3A , Figure 3B and Figure 3C The application scenarios in the above are only several exemplary implementations in the embodiments of the present invention. The application scenarios in the embodiments of the present invention include but are not limited to the above application scenarios. The optical image stabilization device or method in this application can also be applied to application scenarios such as real-time shooting in augmented reality glasses, robot vision systems or professional digital cameras. Other scenarios and examples will not be listed one by one and repeated.

[0110] See also Figure 4A, Figure 4A : is a schematic diagram of the hardware structure of an optical image stabilization device provided in an embodiment of the present application. The optical image stabilization device 40 can be located in any electronic device, such as a computer, a mobile phone, a tablet, and other devices. The hardware structure of the optical image stabilization device 40 can specifically be a chip or a chipset or a circuit board equipped with a chip or a chipset. The chip or chipset or the circuit board equipped with a chip or a chipset can work under the necessary software drive. Figure 4A The optical image stabilization device 40 shown in the figure may include a motor driver 402 and an image sensor 401 physically connected to the motor driver 402; wherein,

[0111] The image sensor 401 is used to report its working state to the motor driver 402; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state. In a possible implementation, see Figure 4B , Figure 4B A schematic diagram of the physical connection between an image sensor and a motor driver provided in an embodiment of the present application, wherein the image sensor 401 is connected to the motor driver 402 via an input / output IO interface; the image sensor 401 is specifically used to generate a readout and exposure signal based on a state output control module, and report its own working state to the motor driver 402 via the IO interface. Among them, the image sensor 401 utilizes the photoelectric conversion function of a photoelectric device to convert the light image on the photosensitive surface into an electrical signal that is proportional to the light image. Optionally, the image sensor 401 may be a charge coupled device (CCD), i.e., a CCD image sensor, or the image sensor 401 may be a complementary metal oxide semiconductor (CMOS) phototransistor, i.e., a CMOS image sensor. Exemplarily, the image sensor 401 converts the light signal into an electrical signal, and then transmits the electrical signal to an image signal processor (ISP) to convert it into a digital image signal. The ISP then outputs the digital image signal to a digital signal processor (DSP) for processing, and finally, the DSP converts the digital image signal into an image signal in a standard RGB, YUV, or other format. Optionally, the image sensor 401 may be inside the camera. For example, the image sensor 401 may belong to a camera module, which may include not only the image sensor 401 but also a lens, a motor, a motor position detector, and the like.

[0112] The motor driver 402 is used to monitor the working state reported by the image sensor 401, and adjust its own driving state to the target driving state based on the working state; the driving state includes a driving mode and a driving frequency. In one possible implementation, the driving mode includes one or more of linear driving, pulse width modulation PWM driving, and dynamic linear driving. Exemplarily, the motor driver 402 is a device for controlling the speed and direction of the motor. The principle is to control the speed and direction of the motor by controlling the output voltage and current. Specifically, the motor driver 402 converts the input power supply voltage into a voltage and current suitable for the motor, and drives the rotation of the motor through the power output in the control circuit.

[0113] For further information, see Figure 4C , Figure 4C 4 is a schematic diagram of the hardware structure of another optical image stabilization device provided in an embodiment of the present application. The optical image stabilization device may also include one or more of a motor 403, an inertial sensor (IMU) 404, and a motor position detector 405. Among them,

[0114] Motor 403, since the essence of optical image stabilization is ultimately to drive motor 403 through motor driver 402, so that motor 403 can push image sensor 401 or lens to make target displacement to offset the shaking displacement of human hand, thereby realizing optical image stabilization. That is, the image sensor (Sensor) component 401 undergoes relative displacement to correct the shaking method. In the optical image stabilization device provided in the embodiment of the present application, motor 403 is a component that causes the lens or image sensor 401 to displace, and can be specifically a motor or an engine. Exemplarily, its working principle can be that the starter rotor is driven to rotate by the energized coil in the magnetic field, and the pinion on the rotor drives the engine flywheel to rotate. In a possible implementation, the optical image stabilization device also includes a motor; the motor driver is also used to drive the motor in the target driving state to push the image sensor or lens to make target displacement. Optionally, the motor 403 is a motor inside the camera module. Exemplarily, the motor 403 may be a voice coil motor (VCM), the main principle of which is to control the stretching position of the spring sheet by changing the magnitude of the DC current of the coil in the motor in a permanent magnetic field, thereby driving the up and down movement. Alternatively, the motor 403 may also be a linear shape memory alloy (SMA) motor, the main principle of which is to drive the relevant components to move by the combined force generated by the power-on contraction of multiple SMA wires. Alternatively, the motor 403 may also be a piezoelectric motor, the principle of which is that under the action of an alternating electric field, piezoelectric ceramics produce movement phenomena such as expansion and contraction, and when a certain type of vibration and ultrasonic vibration is excited in the elastomer, the surface of the elastomer uses friction to push the object in contact with it to move. It is understandable that the motors in the embodiments of the present application may also be of other types, which are not listed here one by one. Setting the drive frequency refers to setting the frequency of the motor displacement, for example, setting a displacement once in 10ms, and trying to maximize the target displacement rate in 1ms.

[0115] The inertial sensor (IMU) 404 can be a detection device for measuring the three-axis angular velocity and acceleration of an object. For example, the inertial sensor 404 may include an accelerometer: detecting the acceleration signals of the three independent axes of the carrier coordinate system; a gyroscope: detecting the angular velocity signal of the carrier relative to the navigation coordinate system; a magnetometer: using algorithms such as Kalman or complementary filtering to provide users with pitch angles, roll angles, and heading angles with absolute references. In the embodiment of the present application, the IMU can sense the shaking direction of the optical anti-shake device and how much position it has shaken (i.e., the shaking information of the human hand) to determine in which direction the motor is pushed and how much the displacement is. Furthermore, the inertial sensor 404 can obtain the shaking information of the human hand, and calculate the displacement required to offset the anti-shake through the algorithm, so that the motor driver 402 can control the motor 403 to push the image sensor 401 or the lens to cause relative displacement to complete the anti-shake.

[0116] The motor position detector 405 can correct the displacement of the motor by building a motor position detector in the optical image stabilization device 40 (such as a mobile phone), that is, correct the displacement error of the motor during the image stabilization process to accurately determine the motor image stabilization displacement. Optionally, the motor is a position detector inside the camera module.

[0117] Further, see Figure 4D , Figure 4D 4 is a schematic diagram of the hardware structure of another optical image stabilization device provided in an embodiment of the present application. The optical image stabilization device may also include one or more of a processor 400, a lens 406, a power management unit (Power Management Unit, PMU) 407, and a battery 408. Among them,

[0118] The processor 400 may include one or more processing units, for example, the processor 400 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0119] The lens 406 may belong to the camera module, and the object generates an optical image through the lens 406 and projects it onto the image sensor 401. In addition, the motor 403 may be driven by the motor driver 402, so that the motor 403 can push the image sensor 401 or the lens 406 to make a target displacement to offset the shaking displacement of the human hand, thereby achieving optical image stabilization.

[0120] The power management unit (PMU) 407 may include power management functions and charging management functions, that is, it integrates power management and charging management functions. The power management unit 407 can be used to connect the battery 408, the processor 400, and the motor driver 402. The power management unit 407 receives input from the battery 408 and supplies power to the processor 400, the motor driver 402, the image sensor 401, etc. In the optical image stabilization device of the embodiment of the present application, since a relatively large current is required in the process of pushing the lens 406 or the image sensor 401, and the average current flowing through the motor 403 over a period of time determines the position of the motor, the current can be provided by the power management unit 407, and the motor driver 402 controls the intensity and mode of the current and the driving frequency. Exemplarily, the power management unit 407 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management unit 407 can also be set in the processor 400.

[0121] The embodiment of the present application provides an optical image stabilization device, which mainly includes the design of a structure and drive adjustment strategy of an optical device, wherein the structure includes a motor driver and an image sensor that are physically connected, wherein the image sensor directly reports its different states during operation, such as exposure, readout, or idle state between frames to the motor driver through a physical connection, and the motor driver can adjust its own drive state based on the different working states of the image sensor, specifically including different drive modes and different drive frequencies, so as to better adjust the motor, greatly reduce power consumption and improve motor performance while ensuring imaging quality. In summary, the embodiment of the present application provides a new low-power management solution for the motor driver, which adopts an image sensor (Sensor) component to be linked with a motor driver (Driver), and the motor driver accurately obtains the working state of the image sensor, and accurately controls the drive mode and drive frequency of the motor according to the different working states of the image sensor, thereby reducing power consumption while ensuring shooting quality.

[0122] It is understood that the structure illustrated in the above application embodiment does not constitute a specific limitation on the optical image stabilization device 40. In other embodiments of the present application, the optical image stabilization device 40 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0123] like Figure 5A As shown, Figure 5A A schematic diagram of the working range of an image sensor provided in an embodiment of the present application. Specifically, the image sensor (Sensor) is divided into an exposure area (exposure), a readout area (ReadOut), and an idle area between frames in the working range, wherein the exposure area (exposure) and the readout area (ReadOut) may overlap in time. In an embodiment of the present application, when the driver Driver obtains the working status of the image sensor (Sensor), it can accurately control the driving mode and driving frequency of the motor. Based on Figure 5A The working range of the image sensor shown in the figure can be divided into the following four states: only in the exposure state, in the exposure state and the readout state at the same time, only in the readout state, and in the inter-frame idle state. The following describes four driving schemes of the motor driver in the embodiment of the present application according to the above four working states of the image sensor.

[0124] (1) Driving scheme 1, the image sensor is only in the exposure state:

[0125] When the current working state of the image sensor is in the exposure state, the motor driver needs to adjust its own driving state to the target state, and the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state.

[0126] When the image sensor is in the exposure time period of the imaging stage, it has high requirements for optical image stabilization, but is not sensitive to electromagnetic radiation. Specifically, the image sensor is sensitive to human hand shaking during the exposure stage, and keeping the driving frequency at a high speed at this time is conducive to improving the imaging quality; in addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the driving mode of the motor driver is a pulse width modulation PWM driving mode, since the current output of the motor driver is a periodic square wave, its output current has a certain frequency and thus generates electromagnetic radiation, which will interfere with the image sensor (may cause image quality degradation), but this method can greatly reduce power consumption, and since electromagnetic radiation affects the readout circuit of the image sensor (that is, the image sensor in the stage that does not include the readout state is not affected by electromagnetic radiation), the above-mentioned image sensor that is only in the exposure state is not sensitive to electromagnetic radiation and is suitable for the PWM driving mode. Therefore, in the embodiment of the present application, when the working state of the image sensor is in the exposure state of the imaging stage, the driving mode of the motor driver can be adjusted to a pulse width modulation (PWM) driving mode (to meet low electromagnetic radiation sensitivity while reducing power consumption) and a high frequency state (to meet high anti-shake requirements) to ensure high imaging quality and low power consumption at the same time.

[0127] In one possible implementation, the image sensor is specifically used to: report the exposure status to the motor driver at time t2 before its own exposure start time T2; the motor driver is specifically used to complete switching the drive state to the target drive state before the exposure time T2 after receiving the exposure status reported by the image sensor at time t2.

[0128] For example, see Figure 5B , Figure 5BA schematic diagram of an image sensor state notification and switching time provided in an embodiment of the present application. When the driver Driver obtains the working state of the Sensor from its I / O interface, it decides whether to switch from linear drive to PWM, or from PWM to dynamic linear drive, and at the same time decides whether the frequency of the drive motor is switched from high speed to low speed, or from low speed to high speed. Regardless of switching the drive mode or switching the drive frequency, the driver Driver needs a certain amount of time. The Sensor is sensitive to electromagnetic radiation at the readout time. If the switch starts from the actual readout time, the Sensor will also be subject to electromagnetic interference during the switching time. The Sensor is sensitive to human hand shaking during the exposure stage. At this time, it is hoped that the drive frequency will remain at a high speed. If the switch is made from low speed to high speed from the actual exposure time, the anti-shake effect will be affected during the switching time. In order to minimize electromagnetic radiation during the drive mode switching process and ensure optical anti-shake performance when switching the drive frequency, the Sensor can output status information ahead of its actual state change time. Since the switching time of different drivers is inconsistent, the advance time can be configured externally.

[0129] In an embodiment of the present application, the image sensor can report its own working status to the motor driver in advance through status output control, so that the motor driver can prepare for switching the driving mode and driving frequency in advance. The reason is that it takes a certain amount of time to switch from one driving state to another. In this way, the motor driver can finally complete the driving mode switching when the image sensor actually enters the corresponding working state, thereby making the driving state of the motor driver and the working state of the image sensor more accurately matched, so as to better achieve high imaging quality and low power consumption.

[0130] (2) Driving scheme 2, the image sensor is in exposure state and readout state at the same time:

[0131] When the current working state of the image sensor is in the exposure state and the readout state at the same time, the motor driver needs to adjust its own driving state to the target state, and the target driving state includes a dynamic linear driving mode and a first driving frequency in a high-speed state.

[0132] When the image sensor is in the exposure and readout period of the imaging stage, it has high requirements for optical image stabilization and is sensitive to electromagnetic radiation. Specifically, the image sensor is sensitive to human hand shaking during the exposure and readout stages, and maintaining a high-speed driving frequency at this time is beneficial to improving the imaging quality. In addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the driving mode of the motor driver is a dynamic linear driving mode, the current intensity supplied to the motor is always the same as the average current intensity of the motor to maintain its displacement. After the optical image stabilization is turned on, the MOS tube or triode inside the motor driver is always kept on, the voltage at the power supply management terminal is always kept constant, and the power consumption on Ron is (Vin-VO)*I. In this mode, there is no electromagnetic radiation interference to the surrounding devices, and the power consumption is smaller than that of linear driving. Since electromagnetic radiation can affect the readout circuit of the image sensor (that is, the image sensor in the stage including the readout state will be affected by electromagnetic radiation), the above-mentioned image sensor in the exposure and readout time period is sensitive to electromagnetics and is not suitable for PWM driving. Moreover, since the power consumption of the linear driving mode is very large, it is suitable for dynamic linear driving. Therefore, in the embodiment of the present application, when the image sensor is in the exposure and readout state of the imaging stage, the driving mode of the motor driver can be adjusted to a dynamic linear driving mode (to meet high electromagnetic radiation sensitivity while reducing power consumption) and a high frequency state (to meet high anti-shake requirements) to improve imaging quality.

[0133] (3) Driving scheme three, the image sensor is only in the readout state:

[0134] When the current working state of the image sensor is in the readout state, the motor driver needs to adjust its own driving state to the target state, and the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state.

[0135] When the image sensor is only in the readout period, it does not require high optical image stabilization, but is sensitive to electromagnetic radiation. Specifically, the image sensor does not require high optical image stabilization in the readout stage, and keeping the driving frequency at a low speed at this time is conducive to reducing power consumption; in addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the driving mode of the motor driver is a dynamic linear driving mode, the current intensity supplied to the motor is always the same as the average current intensity of the motor to maintain its displacement. After the optical image stabilization is turned on, the MOS tube or triode inside the motor driver is always kept in the open state, the voltage of the power supply management terminal is always kept constant, and the power consumption on Ron is (Vin-VO)*I. In this way, it has no electromagnetic radiation interference on the surrounding devices, and the power consumption is smaller than that of linear driving. Since electromagnetic radiation can affect the readout circuit of the image sensor (that is, the image sensor in the stage including the readout state will be affected by electromagnetic radiation), the above-mentioned image sensor in the exposure and readout period at the same time is sensitive to electromagnetics and is not suitable for PWM driving mode. And since the power consumption of the linear driving mode is very large, it is suitable for dynamic linear driving mode. Therefore, in the embodiment of the present application, when the working state of the image sensor is only in the readout state, the driving mode of the motor driver can be adjusted to a dynamic linear driving mode (meeting high electromagnetic radiation sensitivity while reducing power consumption) and a low-speed frequency state (meeting low anti-shake requirements while reducing power consumption) to ensure high imaging quality and low power consumption at the same time. In addition, it can be understood that in the embodiment of the present application, the first driving frequency is greater than the second driving frequency.

[0136] In a possible implementation, the image sensor is specifically used to: report the readout state to the motor driver at time t1 before the start of the readout time T1; the motor driver is specifically used to complete the change of the drive state to the target drive state before the readout time T1 after receiving the readout state reported by the image sensor at time t1. In the embodiment of the present application, the image sensor can report its own working state to the motor driver in advance through the state output control, so that the motor driver can prepare for the switching of the driving mode and driving frequency in advance, because it takes a certain amount of time to switch from one driving state to another driving state. In this way, the motor driver can finally complete the switching of the driving mode when the image sensor actually enters the corresponding working state, so that the driving state of the motor driver and the working state of the image sensor are more accurately matched, so as to better achieve high imaging quality and low power consumption.

[0137] (4) Drive scheme 4, the image sensor is in an idle state between frames:

[0138] When the current working state of the image sensor is in an idle state between frames, the motor driver needs to adjust its own driving state to a target state, and the target driving state includes a PWM driving mode and a second driving frequency in a low-speed state.

[0139] When the image sensor is in the inter-frame idle stage, it does not require high optical image stabilization and is not sensitive to electromagnetic radiation. Specifically, the image sensor does not require high optical image stabilization in the readout stage, and keeping the driving frequency at a low speed at this time is conducive to reducing power consumption; in addition, since the motor driver will generate different degrees of power consumption and cause electromagnetic radiation interference in different situations under different driving modes, for example, when the motor driver is driven in a pulse width modulation PWM driving mode, since the current output of the motor driver is a periodic square wave, its output current has a certain frequency and thus generates electromagnetic radiation, which will interfere with the image sensor (may cause image quality degradation), but this method can greatly reduce power consumption, and since electromagnetic radiation affects the readout circuit of the image sensor (that is, the image sensor in the stage that does not include the readout state is not affected by electromagnetic radiation), the above-mentioned image sensor in the inter-frame idle state is not sensitive to electromagnetic radiation and is suitable for PWM driving mode. Therefore, in the embodiment of the present application, when the working state of the image sensor is in the inter-frame idle stage (i.e., the frame interval stage), the driving mode of the motor driver can be adjusted to a PWM driving mode (to meet the low electromagnetic radiation sensitivity while reducing power consumption) and a low-speed frequency state (to meet the low anti-shake requirements while reducing power consumption) to ensure both high imaging quality and low power consumption.

[0140] Based on the structure of the optical image stabilization device 40 provided in the above embodiment of the present application, the following exemplary embodiment describes a specific implementation method in which the motor driver in the device 40 adjusts the driving state according to the working state fed back by the image sensor:

[0141] See also Figure 5C , Figure 5C A schematic diagram of a motor driver working range provided in an embodiment of the present application. After the motor driver (Driver) obtains the real working state of the image sensor (Sensor), it determines the driving mode and driving frequency of the motor according to different working states. Figure 5CIn the working range, the image sensor (Sensor) is divided into an exposure area (exposure), a readout area (ReadOut), and an idle area between frames, wherein the exposure area (exposure) and the readout area (ReadOut) may overlap in time. Optical image stabilization (OIS) sensitive means that the optical image stabilization requirement is high, and OIS insensitive means that the optical image stabilization requirement is not high; Sensor insensitive means that it is insensitive to electromagnetic radiation, and Sensor sensitive means that it is sensitive to electromagnetic radiation. Vertical block (VerticalBlock, VB) refers to the frame interval, and motor image stabilization means that the motor is in a high-speed state and the corresponding motor driver's driving frequency is in a high-speed state; motor following means that the motor is in a low-speed state and the corresponding motor driver's driving frequency is in a high-speed state. PWM refers to pulse width modulation drive, and Linear refers to dynamic linear drive. The readout frame of the readout area (ReadOut) corresponds to the dynamic linear drive mode between the start mark (Start of frame, SOF) and the end mark (End of frame, EOF) of the frame, while other working intervals correspond to pulse width modulation drive modes. Specifically,

[0142] (1) The image sensor is only in the exposure period, which is the imaging stage. It has high requirements for optical image stabilization (OIS sensitive), but is not sensitive to electromagnetic radiation, that is, the sensor is insensitive. At this time, pulse width modulation (PWM) can be used to drive the motor. In order to improve the imaging quality, the motor needs to turn on the anti-shake function, that is, the motor is in a high-speed state. Correspondingly, the motor driving frequency must be in a high-speed state.

[0143] (2) The image sensor device Sensor is in the exposure (Exposure) and readout (ReadOut) time periods at the same time. It is in the imaging stage, which has high requirements for optical image stabilization (OIS), that is, OIS is sensitive, and it is also in the electromagnetic radiation sensitive stage (Sensor is sensitive). At this time, the motor is driven by dynamic linear drive. In order to improve the imaging quality, the motor needs to turn on the anti-shake function, that is, the motor is in a high-speed state. Correspondingly, the motor driving frequency must be in a high-speed state.

[0144] (3) The image sensor device Sensor is only in the ReadOut time period and is not in the imaging stage. It does not have high requirements for optical image stabilization (OIS is not sensitive), but is sensitive to electromagnetic radiation (Sensor is sensitive). At this time, dynamic linear voltage regulation is used to drive the motor. In order to reduce power consumption, the motor only needs to turn on the follow-up function, that is, the motor can be in a low-speed state. Correspondingly, the motor driving frequency is in a low-speed state.

[0145] (4) The image sensor device Sensor is neither in Exposure nor ReadOut time period, which is the idle area between frames of the Sensor. At this time, it is not sensitive to electromagnetic radiation (Sensor is insensitive) and is not in the imaging stage (OIS is insensitive). The PWM method can be used to drive the motor. The motor only needs to turn on the follow-up function, that is, the motor can be in a low-speed state. Correspondingly, the motor driving frequency is in a low-speed state.

[0146] In order to further reduce power consumption, the embodiment of the present application no longer adopts the traditional PWM and linear voltage regulation switching solution, but adopts the PWM and dynamic linear voltage regulation switching solution. In order to further reduce power consumption, starting from the actual reading time, the motor drive switches to a low speed and enters a follow-up state, and no longer enters an IDLE state at the beginning of the idle time.

[0147] See also Figure 5D , Figure 5D A schematic diagram of the linkage operation of an image sensor and a motor driver provided in an embodiment of the present application is provided. The following is an exemplary description of a specific implementation method in which the motor driver in the device 40 adjusts the driving state according to the working state fed back by the image sensor:

[0148] 1. Measure the time t1 when the motor driver switches from PWM to dynamic linear drive, and the time t2 when the dynamic linear switches to PWM.

[0149] 2. t1 and t2 can be configured as advance times in the image sensor.

[0150] 3. When taking a photo, the image sensor notifies the motor driver in advance that it enters the exposure and readout states, with the lead time being t2 and t1 respectively.

[0151] 4. The image sensor outputs the exposure status t2 in advance of its actual exposure start time. The motor driver receives this status and adjusts the drive frequency from low speed to high speed. The switching end time is guaranteed to be synchronized with the actual exposure time, or in advance of the actual exposure time. After receiving the exposure end status, the high-speed mode enters the low-speed mode.

[0152] 5. The image sensor outputs the readout status t1 earlier than the actual readout start time. The driver receives this status and adjusts the drive mode from PWM to dynamic linear drive. The switching end time is guaranteed to be synchronized with the actual readout time, or ahead of the actual readout time. This prevents electromagnetic radiation from still existing after the actual readout time. After receiving the readout end status, the dynamic linear mode enters the PWM mode.

[0153] Compared with the prior art, in order to reduce power consumption in the readout stage, the embodiment of the present application changes from a linear drive mode to a dynamic linear drive mode.

[0154] See also Figure 6 , Figure 6 is a flow chart of an optical image stabilization method provided in an embodiment of the present application, which can be applied to the above Figure 4A , Figure 4B or Figure 4C In the optical image stabilization device described in the specification, the optical image stabilization device includes a motor driver and an image sensor physically connected to the motor driver. The optical image stabilization method may include method flow steps S601 to S602.

[0155] Step S601: reporting the working state of the image sensor to the motor driver through the image sensor; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state;

[0156] Step S602: monitoring the working state of the image sensor through the motor driver, and adjusting its own driving state to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

[0157] In a possible implementation, the driving mode includes one or more of linear driving, pulse width modulation (PWM) driving, and dynamic linear driving.

[0158] In a possible implementation, the optical image stabilization device further includes a motor; and the method further includes:

[0159] The motor is driven by the motor driver in the target driving state to push the image sensor or the lens to perform target displacement.

[0160] In a possible implementation, when the working state is in an exposure state, the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state.

[0161] In a possible implementation, when the working state is in an exposure state and a readout state at the same time, the target driving state includes a dynamic linear driving mode and a first driving frequency of a high-speed state.

[0162] In a possible implementation, when the working state is in a readout state, the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state.

[0163] In a possible implementation, when the working state is in an inter-frame idle state, the target driving state includes a PWM driving mode and a second driving frequency in a low-speed state.

[0164] In a possible implementation, the reporting of the working status of the image sensor to the motor driver includes: reporting the exposure status to the motor driver at time t2 before the image sensor starts exposure time T2;

[0165] The method of monitoring the working state of the image sensor by the motor driver and adjusting its own driving state to a target driving state based on the working state includes: after receiving the exposure state reported by the image sensor at time t2, the motor driver completes switching the driving state to the target driving state before the exposure time T2.

[0166] In a possible implementation, the reporting of the working status of the image sensor to the motor driver includes: reporting the readout status to the motor driver at time t1 before the image sensor starts to read out the image sensor itself at time T1;

[0167] The method of monitoring the working state of the image sensor by the motor driver and adjusting its own driving state to a target driving state based on the working state includes: after receiving the readout state reported by the image sensor at time t1, the motor driver completes changing the driving state to the target driving state before the readout time T1.

[0168] In a possible implementation, the image sensor is connected to the motor driver via an input / output (IO) interface; and reporting the working status of the image sensor to the motor driver includes:

[0169] The image sensor reports its own working status to the motor driver through the IO interface.

[0170] It should be noted that the various steps of the optical image stabilization method described in the embodiments of the present application can be referred to in the above Figures 3A-5D The relevant description in the device embodiment described in will not be repeated here.

[0171] The embodiment of the present application further provides a system on chip SOC, which may include a motor driver and an IO interface; the motor driver is connected to the image sensor through the IO interface;

[0172] The IO interface is used for the image sensor to report its own working state to the motor driver; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state;

[0173] The motor driver is used to monitor the working state of the image sensor through the IO interface, and adjust its own driving state to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

[0174] In a possible implementation, the embodiment of the present application does not specifically limit the specific type of the communication interface between the motor driver and the image sensor, and different communication interfaces can be used for connection and communication according to different application scenarios or transmission requirements. For example, the motor driver can also query the register through the interface protocol to obtain the working status value of the image sensor.

[0175] In a possible implementation, the driving mode includes one or more of linear driving, pulse width modulation (PWM) driving, and dynamic linear driving.

[0176] In a possible implementation, the motor driver is also connected to a motor; and the motor driver is further configured to drive the motor in the target driving state to push the image sensor or the lens to perform target displacement.

[0177] In a possible implementation, when the working state is in an exposure state, the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state.

[0178] In a possible implementation, when the working state is in an exposure state and a readout state at the same time, the target driving state includes a dynamic linear driving mode and a first driving frequency of a high-speed state.

[0179] In a possible implementation, when the working state is in a readout state, the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state.

[0180] In a possible implementation, when the working state is in an inter-frame idle state, the target driving state includes a PWM driving mode and a second driving frequency in a low-speed state.

[0181] In one possible implementation, the IO interface is specifically used for: the image sensor reports the exposure status to the motor driver at time t2 before the image sensor starts exposing itself T2; the motor driver is specifically used for, after receiving the exposure status reported by the image sensor through the IO interface at time t2, completing the switching of the driving state to the target driving state before the exposure time T2.

[0182] In one possible implementation, the IO interface is specifically used for: the image sensor reports the readout status to the motor driver at time t1 before the time T1 at which the image sensor starts reading out itself; the motor driver is specifically used for, after receiving the readout status reported by the image sensor through the IO interface at time t1, completing the switching of the drive state to the target drive state before the readout time T1.

[0183] It should be noted that the functions of the SOC described in the embodiments of the present application can be found in the above Figures 3A-5D The relevant description in the device embodiment described in will not be repeated here.

[0184] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program includes part or all of the steps of any one of the steps recorded in the above method embodiments.

[0185] An embodiment of the present application also provides a computer program, which includes instructions. When the computer program is executed by a computer, the computer can execute part or all of the steps of any one of the above method embodiments.

[0186] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0187] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, some steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0188] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0189] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0190] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0191] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0192] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An optical image stabilization device, characterized in that: including a motor driver, and an image sensor physically connected to the motor driver; The image sensor is used to report its own working state to the motor driver; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state; The motor driver is used to monitor the working state of the image sensor and adjust its own driving state to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

2. The device according to claim 1, characterized in that The driving mode includes one or more of linear driving, pulse width modulation (PWM) driving, and dynamic linear driving.

3. The device according to claim 1 or 2, characterized in that: The optical image stabilization device further includes a motor; and the motor driver is further configured to drive the motor in the target driving state to drive the image sensor or the lens to cause target displacement.

4. The device according to any one of claims 1 to 3, characterized in that: When the working state is in the exposure state; the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state.

5. The device according to any one of claims 1 to 3, characterized in that: When the working state is in an exposure state and a readout state at the same time; the target driving state includes a dynamic linear driving mode and a first driving frequency in a high-speed state.

6. The device according to any one of claims 1 to 3, characterized in that: When the working state is in the readout state; the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state.

7. The device according to any one of claims 1 to 3, characterized in that: When the working state is in an inter-frame idle state; the target driving state includes a PWM driving mode and a second driving frequency in a low-speed state.

8. The device according to any one of claims 1 to 5, characterized in that: The image sensor is specifically used for: Reporting the exposure status to the motor driver at time t2 before the exposure start time T2; The motor driver is specifically configured to complete switching the driving state to the target driving state before the exposure time T2 after receiving the exposure state reported by the image sensor at time t2.

9. The device according to any one of claims 1-3 and 5-6, characterized in that: The image sensor is specifically used for: Reporting the read status to the motor driver at time t1 before the time T1 at which the motor driver starts reading; The motor driver is specifically configured to complete switching the driving state to the target driving state before the reading time T1 after receiving the readout state reported by the image sensor at time t1.

10. The device according to any one of claims 1 to 9, characterized in that: The image sensor is connected to the motor driver via an input / output (IO) interface; The image sensor is specifically used to report its own working status to the motor driver through the IO interface.

11. An optical image stabilization method, characterized in that: The method is applied to an optical image stabilization device, wherein the optical image stabilization device comprises a motor driver and an image sensor physically connected to the motor driver; the method comprises: Reporting the working status of the image sensor to the motor driver through the image sensor; the working status includes one or more of an exposure state, a readout state, and an inter-frame idle state; The working state of the image sensor is monitored by the motor driver, and its own driving state is adjusted to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

12. The method according to claim 11, characterized in that The driving mode includes one or more of linear driving, pulse width modulation (PWM) driving, and dynamic linear driving.

13. The method according to claim 11, characterized in that The optical image stabilization device further includes a motor; and the method further includes: The motor is driven by the motor driver in the target driving state to push the image sensor or the lens to perform target displacement.

14. The method according to any one of claims 11 to 13, characterized in that: When the working state is in the exposure state; the target driving state includes a PWM driving mode and a first driving frequency in a high-speed state.

15. The method according to any one of claims 11 to 14, characterized in that: When the working state is in an exposure state and a readout state at the same time; the target driving state includes a dynamic linear driving mode and a first driving frequency in a high-speed state.

16. The method according to any one of claims 11 to 14, characterized in that: When the working state is in the readout state; the target driving state includes a dynamic linear driving mode and a second driving frequency in a low-speed state.

17. The method according to any one of claims 11 to 14, characterized in that: When the working state is in an inter-frame idle state; the target driving state includes a PWM driving mode and a second driving frequency in a low-speed state.

18. The method according to any one of claims 11 to 15, characterized in that: The reporting of the working status of the image sensor to the motor driver includes: reporting the exposure status to the motor driver at time t2 before the image sensor starts exposure time T2; The method of monitoring the working state of the image sensor by the motor driver and adjusting its own driving state to a target driving state based on the working state includes: after receiving the exposure state reported by the image sensor at time t2, the motor driver completes switching the driving state to the target driving state before the exposure time T2.

19. The method according to any one of claims 11-13 and 15-16, characterized in that: The reporting of the working status of the image sensor to the motor driver includes: reporting the readout status to the motor driver at time t1 before the image sensor starts to read out time T1; The method of monitoring the working state of the image sensor by the motor driver and adjusting its own driving state to a target driving state based on the working state includes: after receiving the readout state reported by the image sensor at time t1, the motor driver completes changing the driving state to the target driving state before the readout time T1.

20. The method according to any one of claims 11 to 19, characterized in that: The image sensor is connected to the motor driver via an input / output (IO) interface; and the image sensor reports its own working status to the motor driver, including: The image sensor reports its own working status to the motor driver through the IO interface.

21. A camera device, characterized in that: The optical image stabilization device, a motor, an inertial sensor IMU and a motor position detector are included in any one of claims 1 to 10; wherein: The IMU is used to detect the vibration information of the device; The motor driver is used to drive the anti-shake displacement of the motor based on the shake information; The motor position detector is used to correct the anti-shake displacement of the motor.

22. An electronic device, characterized in that: include: The optical image stabilization device and processor according to any one of claims 1 to 10; The processor is used to send a shooting instruction to the image sensor in the optical image stabilization device; The image sensor is used to enter a corresponding working state according to a received shooting instruction; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state.

23. The electronic device according to claim 22, characterized in that: Also includes: The memory is used to store program instructions; the program instructions are executed by the processor.

24. An electronic device, characterized in that: include: a processor and a memory; wherein The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to execute the following steps: Reporting the working status of the image sensor to the motor driver through the image sensor; the working status includes one or more of an exposure state, a readout state, and an inter-frame idle state; The working state of the image sensor is monitored by the motor driver, and its own driving state is adjusted to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

25. A system on chip SOC, characterized in that: It includes a motor driver and an IO interface; the motor driver is connected to the image sensor through the IO interface; The IO interface is used for the image sensor to report its own working state to the motor driver; the working state includes one or more of an exposure state, a readout state, and an inter-frame idle state; The motor driver is used to monitor the working state of the image sensor through the IO interface, and adjust its own driving state to a target driving state based on the working state; the driving state includes a driving mode and a driving frequency.

26. A computer-readable storage medium, characterized in that: The computer-readable medium is used to store program codes, and when the program codes are executed by a processor, the method according to any one of claims 11 to 20 is implemented.

27. A computer program, characterized in that The computer program comprises instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 11-20.

Images