Motor control method and electronic equipment
By moving the initial position of the motor back in the frame gap between the electronic device, the image blur and tremor caused by user jitter is solved, and balanced compensation between video frames is achieved, image stability and user experience are improved.
Patent Information
- Application Number
- CN202310609461.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
AI Technical Summary
When users use electronic devices to take pictures, the image is blurred due to jitter caused by grip or movement, and the degree of blurring of adjacent frames is inconsistent, resulting in a tremor, affecting the video quality.
By moving the initial position back to the middle when the optical image stabilization system is powered up during the frame gap, ensuring the level of OIS compensation for each frame equalization and reducing tremor between video frames. The specific method includes moving the motor backwards after the exposure is completed, and adjusting the exposure time and frame rate according to the device status and ambient brightness to extend the frame gap, ensuring that the motor has sufficient time to move backwards.
It effectively reduces the vibration between video frames, improves the processing effect of video frames, and ensures image stability and user experience.
Smart Images

Figure CN119031245A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to a motor control method and electronic equipment. Background Art
[0002] When users use electronic devices to record videos, the images may become blurred due to the shaking caused by the user holding or moving the device. Generally speaking, the shaking caused by the user holding or moving the device is irregular, so the blurring of the image is also irregular. Since a video is composed of a series of continuous frames, if the blurring of adjacent frames is inconsistent, it will give the user a sense of tremor. Summary of the invention
[0003] The present application provides a motor control method and an electronic device, which return the motor of an optical image stabilization (OIS) system to its initial position when powered on during frame intervals, so that each frame has a relatively balanced OIS compensation degree, reduces the tremor between video frames, and improves the processing effect of video frames.
[0004] In a first aspect, the present application provides a motor control method for an electronic device. During the exposure of the current image frame, the electronic device controls the motor to move to achieve the anti-shake function. After the current image frame is exposed, the electronic device controls the motor to move back to the initial position when it was powered on. During the exposure of the next image frame, the electronic device controls the motor to move from the position after the return to the center to achieve the anti-shake function.
[0005] The motor control method provided in the present application can control the motor to move back in the direction of the initial position when it is powered on, thereby expanding the movement range of the motor in the next image frame as much as possible, and making the movement range of the motor when performing OIS compensation in the current image frame and the next image frame as similar as possible, thereby making the current image frame and the next image frame have a relatively balanced degree of OIS compensation, thereby reducing the tremor between video frames.
[0006] In a possible implementation, the electronic device controls the motor to move back to the initial position when powered on during the frame interval between the last line of the current image frame being exposed and the first line of the next frame being exposed.
[0007] The motor control method provided in the present application can control the motor to move back to the direction of its initial position when it was powered on after the exposure of the current image frame is completed and before the exposure of the next image frame begins. This can prevent the motor from moving in the direction opposite to the anti-shake movement during the exposure of the image frame, thereby avoiding interference and reduction of the image anti-shake effect caused by the motor returning to the center.
[0008] In a possible implementation, the electronic device obtains spatial posture parameters and ambient brightness parameters, and then controls the motor to move back to the initial position direction when it is powered on based on the spatial posture parameters and ambient brightness parameters.
[0009] The motor control method provided in the present application can control the motor to move back to the center in the direction of the initial position when the electronic device is powered on, according to whether the electronic device is currently in a stable state or a moving state and the current ambient brightness. After comprehensively considering the movement mode of the motor in the current frame according to the state of the electronic device and the ambient brightness, the motor's return movement mode is further determined to better achieve the motor return effect.
[0010] In a possible implementation, when the device is in motion and in a dark scene, the electronic device shortens the exposure time and controls the motor to move back and forth during the frame interval.
[0011] When an electronic device is in a dark scene, it takes a long time for the electronic device to be exposed, so the frame interval is relatively short. When the electronic device is in motion, the motor has a long travel distance for anti-shake movement, so it takes enough time to control the motor to return to the center. This method appropriately shortens the exposure time, sacrifices the exposure amount, and extends the frame interval, thereby ensuring that the motor has enough time to return to the center.
[0012] In one possible implementation, when in motion and in a dark scene, the electronic device can shorten the exposure time from a first exposure time to a second exposure time, and the second exposure time is greater than a third time, and the third time is the minimum time to ensure the image exposure quality in the dark scene.
[0013] In order to extend the frame interval, the electronic device shortens the exposure time from the first exposure time to the second exposure time. However, considering that dark light scenes require a certain exposure time to ensure the imaging effect, the exposure time cannot be shortened indefinitely. Therefore, the second exposure time needs to be no less than the third exposure time to ensure the image exposure quality in dark light scenes.
[0014] In one possible implementation, when in motion and in a dark scene, the electronic device can reduce the frame rate and further extend the frame duration from a first frame duration to a second frame duration, and the difference between the second frame duration and the first frame duration is greater than the difference between the second exposure time and the third exposure time.
[0015] In order to extend the frame interval, the electronic device shortens the exposure time from the first exposure time to the second exposure time. However, considering that a certain exposure time is required to ensure the imaging effect in a dark scene, the exposure time cannot be shortened indefinitely. Therefore, this method further reduces the frame rate. When the exposure time cannot be further shortened, the frame length is extended to extend the frame interval so that the motor has enough time to return to the center.
[0016] In a possible implementation, the third time is 10 ms.
[0017] To ensure image exposure quality in a dark light scene, the exposure time of the electronic device may be 10 ms, which is the minimum time to ensure image exposure quality in a dark light scene.
[0018] In a second aspect, the present application provides an electronic device, which includes a processor, a memory and a camera module, wherein computer instructions are stored in the memory, and when the processor executes the computer instructions, the electronic device executes any one of the motor control methods in the first aspect.
[0019] In one possible implementation, the camera module includes an image sensor and a driver chip connected to the image sensor. The image sensor sends a first level signal to the driver chip during exposure time, and the driver chip further controls the motor to move for anti-shake. The image sensor sends a second level signal to the driver chip during non-exposure time, and the driver chip further controls the motor to move back to the center.
[0020] In a third aspect, the present application provides a chip system, which includes a processor and a communication interface. When the processor runs a computer program, it executes any one of the motor control methods in the first aspect.
[0021] In a fourth aspect, the present application provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes any one of the motor control methods in the first aspect.
[0022] In a fifth aspect, the present application provides a computer program product storing program codes. When the computer program product runs on an electronic device, the electronic device executes any one of the motor control methods in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 is a schematic diagram of a hardware system suitable for the present application;
[0024] Figure 2 It is a schematic diagram of the process of exposing a moving object using an image sensor in a rolling shutter exposure mode;
[0025] Figure 3 It is a schematic diagram of the process of exposing a moving object by an image sensor in a rolling shutter exposure mode when an optical image stabilization system is present;
[0026] Figure 4 It is a schematic diagram of two lines being simultaneously exposed during the exposure of a moving object by an image sensor in a rolling shutter exposure mode when an optical image stabilization system is present;
[0027] Figure 5It is a schematic diagram of the exposure time and readout time of two adjacent frames of an image sensor in a rolling shutter exposure mode;
[0028] Figure 6 is a structural schematic diagram of a camera module applicable to the present application;
[0029] Figure 7 This is a schematic diagram of the travel of the OIS motor changing over time when it touches the boundary;
[0030] Figure 8 is a flow chart of a motor control method provided in an embodiment of the present application;
[0031] Fig. 9 This is a schematic diagram of the travel of the OIS motor changing over time after it returns to center during the frame interval;
[0032] Fig.10 It is a schematic diagram of the exposure time and readout time of two adjacent frames of an image sensor in another rolling shutter exposure mode;
[0033] Fig.11 is a module schematic diagram of an image stabilization method provided in an embodiment of the present application;
[0034] Fig.12 is a timing diagram of the image stabilization method provided in an embodiment of the present application;
[0035] Fig.13 The figure is a schematic diagram of the exposure time and readout time of two adjacent frames of an image sensor using a global shutter exposure method. DETAILED DESCRIPTION
[0036] The terms used in the following embodiments are for the purpose of describing the embodiments and are not intended to be limitations on the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above" and "the" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "one or more" refers to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.
[0037] References to "some embodiments" and the like described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "include", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0038] The following describes embodiments of electronic devices and motor control methods for such electronic devices. In some embodiments, the electronic device may be a portable electronic device that also includes other functions such as a personal digital assistant and / or a music player, such as a mobile phone, a tablet computer, a wearable electronic device with wireless communication functions (such as a smart watch), etc. Exemplary embodiments of portable electronic devices include but are not limited to Or a portable electronic device with other operating systems. The portable electronic device may also be other portable electronic devices, such as a laptop computer, etc. It should also be understood that in some other embodiments, the electronic device may not be a portable electronic device, but a desktop computer.
[0039] For example, Figure 1 The structure diagram of the electronic device 100 is shown. The electronic device 100 may include a processor 110, a display screen 120, a camera 130, an internal memory 140, a subscriber identification module (SIM) card interface 150, a universal serial bus (USB) interface 160, a charging management module 170, a power management module 171, a battery 172, a sensor module 180, a mobile communication module 190, a wireless communication module 200, an antenna 1 and an antenna 2, etc. The sensor module 180 may include a pressure sensor 180A, a fingerprint sensor 180B, a touch sensor 180C, an ambient light sensor 180D, etc.
[0040] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0041] The processor 110 may include one or more processing units, for example: the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent components or integrated into one or more processors. The central processing unit is also called a central processing unit, and the graphics processing unit is also called a graphics processor. In some embodiments, the electronic device 100 may also include one or more processors 110. Among them, the controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. In some other embodiments, a memory may also be provided in the processor 110 for storing instructions and data. Exemplarily, the memory in the processor 110 may be a cache memory. The memory may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory, thus avoiding repeated access and reducing the waiting time of the processor 110, thereby improving the efficiency of the electronic device 100 in processing data or executing instructions.
[0042] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface and / or a USB interface, etc. Among them, the USB interface 160 is an interface that complies with the USB standard specification, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. The USB interface 160 can also be used to connect headphones to play audio through the headphones.
[0043] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is for illustrative purposes only and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0044] The electronic device 100 implements the display function through a GPU, a display screen 120, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 120 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
[0045] The camera 130 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red green blue (RGB), YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 130, where N is a positive integer greater than 1.
[0046] The internal memory 140 can be used to store one or more computer programs, which include instructions. The processor 110 can execute the above instructions stored in the internal memory 140, so that the electronic device 100 performs the display method provided in some embodiments of the present application, as well as various applications and data processing. The internal memory 140 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system; the program storage area can also store one or more applications (such as a gallery, contacts, etc.). The data storage area can store data (such as photos, contacts, etc.) created during the use of the electronic device 100. In addition, the internal memory 140 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (UFS), etc. In some embodiments, the processor 110 can execute the image stabilization method provided in the embodiment of the present application, as well as other applications and data processing by running instructions stored in the internal memory 140 and / or instructions stored in a memory provided in the processor 110.
[0047] The internal memory 140 can be used to store the relevant programs of the image stabilization method provided in the embodiment of the present application, and the processor 110 can be used to call the relevant programs of the image stabilization method stored in the internal memory 140 during image processing to execute the image stabilization method of the embodiment of the present application.
[0048] The sensor module 180 may include a gyro sensor 180B, an ambient light sensor 180D, and the like.
[0049] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.
[0050] The ambient light sensor 180D is used to sense the brightness value of the ambient light. The electronic device 100 can adaptively adjust the brightness value of the display screen 120 according to the sensed brightness value of the ambient light. The ambient light sensor 180D can also be used to automatically adjust the white balance when shooting. The ambient light sensor 180D can also transmit the environment information of the device to the GPU.
[0051] Figure 2 It is a schematic diagram of the exposure process of an image sensor using a rolling shutter exposure method. Figure 2 (a) to Figure 2 (e) Arranged in time order, as shown by the black horizontal bars moving downward row by row in the figure, at the beginning of exposure, the image sensor scans the imaging units row by row and exposes them row by row. Figure 2 The shaded part in (a) is the object being photographed. The object moves from left to right, so its position on the sensor imaging unit will be different at different times. Figure 2 As shown in (e), during the exposure time, the exposure result of the object in the imaging area is similar to Figure 3 Similarly, when the user's hand shakes, the following wedge shape will appear Figure 2 The effect when the object shown is in motion.
[0052] If the electronic device has an optical image stabilization system, the vibration of the lens module can be detected through the spatial attitude parameters, and then the processor can calculate the displacement that needs to be compensated, and further instruct the OIS motor to drive the OIS base to move in the XY plane to compensate for the vibration of the lens. Compensation based on the movement of the lens can effectively overcome the image blur caused by the vibration of the electronic device. Figure 2 (a) to Figure 2 As shown in (e), due to the compensation of the OIS system, the exposure result of the object will still maintain the original rectangle.
[0053] The image sensor starts to expose from the first row, and the next row starts to be exposed after a row cycle is completed. At the same time, the data is read out after the first row is exposed. It takes a row cycle to read out a row, that is, after the first row is completely read out, the second row just starts to be read out. Therefore, when the Nth row (N>3) is exposed, the N-1th row is reading out data. At this time, the imaging unit of the first row has completed exposure and readout, so the next round of exposure can be started. Figure 4 As shown in (e), the first and fourth rows are exposed at the same time. Figure 5 As shown, the exposure time of the next frame will overlap with the exposure time of the current frame.
[0054] Figure 6 A camera structure is shown, such as Figure 6 As shown, the camera includes a lens bearing portion 61, an auto focus (AF) base 62 and an optical image stabilization (OIS) base 63. The electronic device detects the shake of the lens module through the spatial posture parameters, and then calculates the displacement that needs to be compensated through the processor, and further instructs the OIS motor to drive the OIS base to move in the XY plane, thereby compensating for the shake of the lens. Compensation based on the movement of the lens can effectively overcome the image blur caused by the vibration of the electronic device. According to the instructions of the processor, the AF system drives the auto focus base to move in the z-axis direction, thereby changing the distance between the lens and the image sensor to improve the focus clarity.
[0055] For electronic devices, the use of an OIS system can effectively improve the stability of the lens during shooting, thereby extending the exposure time and further improving the shooting effect under low-light conditions. Therefore, when the image sensor is exposed, the OIS must be in a working state to compensate for the lens shake in real time, that is, the OIS needs to be enabled during the exposure time. Figure 2 (e) and Figure 5 As shown, when the current frame has not finished exposing the fourth row, the next frame has already started exposing the first row, so the OIS is always in the working state, that is, the OIS is always enabled.
[0056] However, the travel of the OIS motor is limited and can only compensate for lens shake within a certain range. If the lens shakes in the same direction beyond the travel of the OIS motor, the OIS motor will no longer be able to drive the lens or sensor to move in the opposite direction, and the OIS system will fail. Figure 7As shown in the figure, once the movement range of the OIS motor reaches the position of point A, the OIS system will no longer be able to compensate for this direction during the subsequent shooting process. As shown in segment AB, even if the lens is detected to continue shaking in the previous direction, the OIS motor cannot move again because it has reached its maximum range. The OIS system fails and the image clarity becomes low.
[0057] When users use electronic devices to record videos, if the blur levels of adjacent frames are inconsistent, a judder phenomenon (sharpness difference between frames) will occur, causing image clarity to jump and affecting user experience. It should be understood that judder has nothing to do with blur or not, but is related to the difference in blur levels between adjacent frames. In other words, if the images of adjacent frames are equally clear or equally blurry, there will be no judder. Therefore, when recording, if the images of adjacent frames cannot be compensated by the same OIS, the blur levels of adjacent frames will be different, and the final video will have a juddering feeling.
[0058] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 1 Taking the electronic device with the structure shown as an example, the motor control method provided in the embodiment of the present application is specifically described in conjunction with the accompanying drawings.
[0059] This embodiment provides a motor control method, such as Figure 8 As shown, the steps of the motor control method provided by this embodiment are described below.
[0060] Step 1: During the exposure of the current image frame, the electronic device controls the OIS motor to move for anti-shake.
[0061] When an electronic device is exposed, it is necessary to overcome the shaking or movement of the user's hand to keep the electronic device stable. The OIS motor is controlled to move in the opposite direction of the electronic device in the plane to ensure the relative stability of the lens, thereby achieving the purpose of anti-shake.
[0062] Step 2: After the current image frame is exposed, the electronic device controls the OIS motor to move back to the initial position when it was powered on.
[0063] In order to allow the OIS motor to have a larger range of motion in different directions when exposing the next frame, thereby reducing the vibration phenomenon, the electronic device needs to control the OIS motor to return to the initial position according to the exposure situation. When the camera module is not working, the motor is not powered on, and the lens moves with the electronic device in the housing; when the camera module is working, the motor is powered on, and in order to compensate for the vibration in all directions, the initial position of the power-on is the middle position of the motor travel range.
[0064] In order to eliminate the tremor and make the current frame and the next frame get the same degree of OIS compensation, it is necessary to make the range of motion of the OIS motor as similar as possible during the exposure of each frame. Therefore, after the current frame is exposed and before the next frame is exposed, the OIS motor can be returned to the initial position (i.e., the middle position of the travel range) when it was powered on (i.e., "return to the center") after the current frame is exposed and before the next frame is exposed. In one implementation of this embodiment, the OIS may not completely return to the initial position, as long as the OIS returns in the direction of the initial position.
[0065] For example, Fig. 9 As shown in the figure, point C is the position of the OIS motor when the current frame exposure ends, and point D is the position of the OIS motor when the next frame exposure begins. The process of the OIS motor from point C to point D is the OIS return to center. Segment DE is the movement direction of the OIS motor in the next frame. If point C is the maximum point of the OIS motor movement stroke and segment CD does not perform the OIS return to center operation, then OIS compensation will not be possible during the process of shooting the next frame.
[0066] However, if the exposure time between consecutive frames is Figure 3 If there is overlap as shown, there will be no time for OIS to return to center. Therefore, the motor control method provided in the embodiment of the present application needs to have a frame gap, that is, there needs to be a period of time between the end of the current frame exposure and the start of the next frame exposure. At this time, even if the OIS motor reaches the maximum stroke when shooting the current frame, the OIS motor returns to center during the frame gap, and even if the OIS still needs to move in the same direction in the next frame, the OIS can still provide compensation and reduce the tremor phenomenon.
[0067] Step 3: During the exposure of the next image frame, the electronic device controls the OIS motor to start anti-shake movement from the position after the return to the center.
[0068] Similar to step 1, in order to ensure clear imaging, the electronic device needs to control the OIS motor to drive the lens to move in the opposite direction of the shake. Figure 7 As shown, segment CD is the OIS return-to-center stage, point D is the OIS motor return-to-center position, and during the exposure of the next frame, the electronic device controls the OIS motor to move from point D.
[0069] In one implementation of this embodiment, the above step 2 further includes:
[0070] Step 21: The electronic device obtains space posture parameters and ambient brightness parameters.
[0071] Electronic devices can obtain the current ambient brightness parameters through devices such as ambient light sensors, and the current ambient brightness parameters represent the brightness level of the currently captured image. The angular velocity of the electronic device moving around the three axes x, y, and z transmitted by the gyroscope determines whether the electronic device is in a stable state or in motion. The three-axis angular velocity parameters can be further divided into numerical intervals to further distinguish the electronic device as stable, moving with a small amplitude, moving with a large amplitude, etc.
[0072] Step 22: The electronic device determines the exposure parameters of the current image frame based on the ambient brightness parameters and the spatial posture parameters.
[0073] The exposure parameter is related to the ambient brightness parameter. Generally, when the environment is bright, a shorter exposure time can obtain a better imaging frame, while when the environment is dark, a longer exposure time is required to obtain as much detail information as possible. The electronic device can obtain the exposure parameter based on a first comparison table of ambient brightness parameters and exposure parameters. For example, in the first exposure scene, the ambient brightness parameter is lower than the first threshold, and the first exposure parameter is used, which may be 30ms; in the second exposure scene, when the ambient brightness parameter is between the first threshold and the second threshold, the second exposure parameter is used, which may be 20ms; in the third exposure scene, when the ambient brightness parameter is higher than the second threshold, the third exposure parameter is used, which may be 10ms.
[0074] At the same time, if it is detected that the electronic device is in motion, the exposure time can be reduced based on a second comparison table of ambient brightness parameters and exposure parameters in the motion state. For example, in the first exposure scenario, when the ambient brightness parameter is lower than the first threshold and the electronic device is in motion, the first correction exposure parameter is used, and the first correction exposure parameter may be 20ms; in the second exposure scenario, when the ambient brightness is between the first threshold and the second threshold and the electronic device is in motion, the second exposure correction parameter is used, and the second correction exposure parameter may be 10ms.
[0075] Step 23: The electronic device controls the motor to return to the center based on the spatial posture parameters and the ambient brightness parameters.
[0076] The frame rate is a measure of the number of frames displayed per unit time. The unit of measurement is "frames per second" (FPS). The frame duration corresponds to the frame rate. The interframe interval is the time between the completion of the last line exposure of the current frame and the start of the first line exposure of the next frame.
[0077] In the first exposure scene, the ambient brightness parameter is lower than the first threshold value. If the electronic device is in a stable state, the first exposure parameter is used, such as an exposure time of 30ms, OIS is continuously enabled, and the OIS movement range is not large, and OIS re-centering is no longer required. If the electronic device is in motion, the first correction exposure parameter is used, such as an exposure time of 20ms, while reducing the frame rate and extending the frame duration to obtain a frame gap, so that OIS can re-center during the frame gap.
[0078] like Fig.10 As shown in (a), the frame rate can be reduced and the frame duration can be extended, so that the extended time can be used to generate frame gaps. For example, when the frame rate is 30fps, the frame duration is 33ms. If the frame rate is reduced to 24fps, the frame duration becomes 40ms accordingly. Even if the exposure time and readout time remain unchanged, there can be 7ms of time for OIS to return to center. In a possible implementation, when the frame rate is 30fps and the frame duration is 33ms, the readout time is 7ms, the exposure time can be 20ms, and the OIS return time can be 4-7ms.
[0079] In the second exposure scenario, the ambient brightness parameter is higher than the first threshold and lower than the second threshold. If the electronic device is in a stable state, the second exposure parameter is used, such as an exposure time of 20ms, OIS is continuously enabled, and the OIS movement range is not large, so OIS re-centering is no longer required. If the electronic device is in motion, the second correction exposure parameter is used, such as an exposure time of 10ms, so as to obtain a frame gap, so that OIS can re-center in the frame gap. Fig.10 As shown in (b), when the frame rate remains unchanged, the frame length remains unchanged, and the exposure time can be shortened to produce a frame gap.
[0080] In the third exposure scene, the ambient brightness parameter is higher than the second threshold. Regardless of whether the electronic device is in a stable state or in motion, the third exposure parameter is used. For example, the exposure time is 10ms. The exposure time is very short, the shooting effect is better, and the OIS is continuously centered and not enabled, so there is no need to return to the center.
[0081] In a possible implementation, the ambient brightness of the dark scene is lower than the second threshold, that is, the first exposure scene and the second exposure scene are both dark scenes. In the dark scene, in order to ensure image quality, the shortest exposure time may be 10 ms.
[0082] In a possible implementation, the correspondence between the movement mode of the OIS and the state of the electronic device and the exposure scene is shown in Table 1.
[0083] Table 1 Correspondence between OIS motion mode, electronic device status and exposure scene
[0084]
[0085] Fig.11 is a hardware structure diagram of the motor control method provided in this embodiment, Fig.12 is a timing diagram of the motor control method provided in this embodiment. Fig.10 The hardware structure shown is used to implement the above method flow.
[0086] Step 501: The Hall sensor sends Hall data to the OIS / AF driver chip.
[0087] Step 502: The OIS / AF driver chip sends Hall data to a sensing system in a system-on-chip (SoC).
[0088] Step 503: The gyroscope sends the spatial attitude parameters to the OIS / AF driver chip and the perception system.
[0089] Step 504: The perception system sends the spatial posture parameters to the motion detection module in the system-level chip.
[0090] Step 505: The motion detection module determines the stability of the electronic device according to the spatial posture parameters. The motion detection module includes a motion detection algorithm, which can determine whether the electronic device is in a stable state or in a moving state according to the spatial posture parameters.
[0091] Furthermore, the motion detection algorithm in the motion detection module can classify the current stability of the electronic device, such as classifying the stability into stable, small-amplitude motion, large-amplitude motion, etc.
[0092] Step 506: The motion detection module sends the stability of the electronic device to the automatic exposure module and the electronic image stabilization module.
[0093] Step 507: The automatic exposure module performs exposure control of the image sensor according to the stability of the electronic device.
[0094] Generally, the original exposure time is related to the ambient brightness. For example, when the ambient brightness is bright, the original exposure time is short exposure; when the ambient brightness is normal, the original exposure time is medium exposure; when the ambient brightness is dark, the original exposure time is long exposure. The automatic exposure module can adaptively adjust the original exposure time according to the stability of the electronic device, such as shortening the exposure time when the electronic device is in motion.
[0095] Step 508: The perception system sends the spatial attitude parameters and Hall data to the electronic image stabilization (EIS) module.
[0096] Step 509: The electronic image stabilization module controls the OIS / AF driver chip to perform OIS compensation according to the spatial posture parameters and the Hall data as well as the stability of the electronic device.
[0097] Step 510: The image sensor (camera sensor) sends an OIS enable signal to the OIS / AF driver chip.
[0098] The image sensor is connected to the OIS / AF driver chip and sends different signals during exposure time and non-exposure time.
[0099] For example, the image sensor sends a high level signal to the OIS / AF driver chip during the exposure time, and sends a low level signal to the OIS / AF driver chip during the non-exposure time.
[0100] Alternatively, the image sensor sends a low level signal to the OIS / AF driver chip during the exposure time, and sends a high level signal to the OIS / AF driver chip during the non-exposure time.
[0101] It should be understood that the connection between the image sensor and the OIS driver chip can be achieved by changing the current, voltage or other methods so that the OIS / AF driver chip receives different signals.
[0102] Step 511: The OIS / AF driver chip drives the motor to move according to the instruction of the electronic image stabilization module and the signal of the image sensor.
[0103] The OIS / AF driver chip controls the motor to perform OIS compensation normally during the exposure time according to the different signals received, and controls the motor to return to the center during the non-exposure time.
[0104] It should be understood that the method provided in the embodiment of the present application can be executed only when the electronic device is in motion, that is, only when the motion control module confirms that the electronic device is in motion, the EIS system will instruct the OIS system to return the motor to center.
[0105] The above is an introduction to the method provided in the embodiment of the present application applied to an image sensor using a rolling shutter exposure mode. The following will introduce the method provided in the embodiment of the present application applied to an image sensor using a global shutter exposure mode.
[0106] Fig.12It is a schematic diagram of the exposure time and readout time of a camera image sensor using a global shutter exposure method. It should be understood that the main difference between the global shutter exposure method and the rolling shutter exposure method is that the image sensor using the global shutter exposure method can expose each row at the same time, so the exposure forms a rectangular area instead of a wedge-shaped area, and it is still read out row by row. Similar to the first embodiment, the OIS system only needs to work during exposure to achieve an optically stable effect, so the OIS system only needs to be enabled during the exposure time and does not need to be enabled during the readout time, so the OIS can perform OIS return during the readout time without leaving a frame gap. Longer exposure times can be supported while the frame rate remains unchanged.
[0107] The above mainly describes the solution provided by the embodiment of the present application from the perspective of method steps. It is understandable that, in order to achieve the above functions, the electronic device implementing the method includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of protection of the present application.
[0108] The present application also provides a chip, which is coupled to a memory and is used to read and execute computer programs or instructions stored in the memory to execute the methods in the above embodiments.
[0109] The present application also provides an electronic device, which includes a chip, and the chip is used to read and execute a computer program or instruction stored in a memory, so that the method in each embodiment is executed.
[0110] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the motor control method in the above-mentioned embodiment.
[0111] This embodiment further provides a computer program product, which stores program codes. When the computer program product runs on an electronic device, the electronic device executes the above-mentioned related steps to implement the motor control method in the above-mentioned embodiment.
[0112] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the image capture method in the above-mentioned method embodiments.
[0113] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.
[0114] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A motor control method, the method being applied to an electronic device, the electronic device comprising an optical image stabilization motor, characterized in that: The method comprises: During the exposure of the current image frame, controlling the motor to perform anti-shake movement; After the current image frame is exposed, the motor is controlled to move back to the initial position when it is powered on; During the exposure of the next image frame, the motor is controlled to start the anti-shake movement from the position after the return movement.
2. The method according to claim 1, characterized in that The controlling the motor to move back to the initial position direction when it is powered on, comprises: In the frame gap between the current image frame and the next image frame, the motor is controlled to move back to the initial position, and the frame gap is the time interval between the end of exposure of the last row of the current frame and the start of exposure of the first row of the next frame.
3. The method according to claim 1, characterized in that: After the current image frame is exposed, controlling the motor to move back to the initial position when it is powered on, comprises: Get space attitude parameters and environment brightness parameters; Based on the spatial posture parameter and the ambient brightness parameter, the motor is controlled to move back to the initial position direction when it is powered on.
4. The method according to claim 3, characterized in that The controlling the motor to move back to the initial position direction when it is powered on based on the spatial posture parameter and the ambient brightness parameter comprises: When the electronic device is in motion and in a dark scene, the exposure time is shortened and the motor is controlled to move back and forth during frame intervals.
5. The method according to claim 4, characterized in that The shortening of the exposure time includes shortening the exposure time from the first exposure time to the second exposure time, and the second exposure time is not less than the third exposure time, and the third exposure time is the minimum time to ensure the image exposure quality in the dark light scene.
6. The method according to claim 5, characterized in that The second exposure time is equal to the third exposure time, and the method further includes: Reduce the frame rate and extend the frame duration from the first frame duration to the second frame duration; The difference between the second frame duration and the first frame duration is greater than the difference between the second exposure time and the third exposure time.
7. The method according to claim 5, characterized in that The third exposure time is 10 ms.
8. An electronic device, characterized in that: The electronic device includes one or more processors, one or more memories, and a camera module; the one or more memories store computer instructions; When the one or more processors execute the computer instructions, the electronic device is caused to perform the method as claimed in any one of claims 1 to 7.
9. The electronic device according to claim 8, characterized in that: The camera module includes an image sensor and a driving chip connected to the image sensor; The image sensor is used to send a first level signal to the driving chip during the exposure time so that the driving chip controls the motor to perform anti-shake movement during the exposure time; The image sensor is used to send a second level signal to the driving chip during a non-exposure time so that the driving chip controls the motor to move in a circular motion during the non-exposure time.
10. A chip system, characterized in that: The chip system includes a processor and a communication interface; the processor is used to call and run a computer program stored in a storage medium from the storage medium to execute the method as described in any one of claims 1-7.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.
12. A computer program product, characterized in that The computer program product stores program codes, and when the computer program product is run on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Imaging device and imaging method
CN103718096A
Optical image stabilization techniques
CN112930678A
Image processing method, electronic equipment and storage medium
CN113473027A
Blur correcting device and imaging apparatus
JP2010078635A
Optical image stabilization techniques
US20200137309A1
