Image anti-shake control method, electronic device, chip system and storage medium

Abstract

The application provides an image anti-shake control method, an electronic device, a chip system and a storage medium, and relates to the technical field of terminals. In a shooting scene of a camera night scene mode, a first anti-shake control parameter suitable for a current scene can be determined based on scene brightness information, such as a shake information filtering mode, an image frame output mode and / or an image fusion mode, and a second anti-shake control parameter suitable for the current scene can be determined based on shake information, such as an exposure compensation amount, different shake information corresponding to different exposure compensation amounts. The application scheme can perform exposure compensation of different degrees in the shooting stage for night scene shooting scenes with different shake intensities, realize the image anti-shake purpose, and realize comprehensive anti-shake control and image processing in combination with the shake information filtering mode, the image frame output mode and / or the image fusion mode, thereby improving the image anti-shake effect of the night scene shooting scene and improving the definition of the night scene photo.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, and in particular to an image anti-shake control method, an electronic device, a chip system and a storage medium. BACKGROUND

[0002] With the wide use of electronic devices, using electronic devices for shooting has become a daily behavior in people's life. Taking a mobile phone as an example, when people hold the mobile phone for shooting, the mobile phone will shake in the shooting process due to self-hand shaking, walking vibration and other reasons, and thus the imaging will be blurred.

[0003] To this end, the related technology adds an optical image stabilization (OIS) controller to the electronic device for anti-shake. The principle of the OIS controller is to use an OIS motor to push the lens to move, and thus offset the displacement caused by shaking.

[0004] However, for night scene shooting scenes, the current optical anti-shake performance is weak, and there is still a case of imaging blur, which cannot meet the user's demand for shooting clear night scene photos. Therefore, there is an urgent need for an image anti-shake control method more suitable for night scene shooting scenes. SUMMARY

[0005] The present application provides an image anti-shake control method, an electronic device, a chip system and a storage medium, which can improve the image anti-shake effect of the night scene shooting scene and improve the clarity of the night scene photo.

[0006] In a first aspect, the present application provides an image anti-shake control method, which comprises: starting a camera application program; enabling a camera night scene mode; acquiring first scene brightness information and first shaking information in the case of enabling the camera night scene mode; determining a first anti-shake control parameter according to the first scene brightness information, the first anti-shake control parameter comprising a shaking information filtering mode, an image out-frame mode and / or an image fusion mode, different scene brightness information corresponding to different anti-shake control parameters; determining a second anti-shake control parameter according to the first shaking information, the second anti-shake control parameter comprising a first exposure compensation amount, different shaking information corresponding to different exposure compensation amounts; receiving a first operation of a user triggering shooting; in response to the first operation, performing anti-shake control and image processing based on the first anti-shake control parameter, and performing anti-shake control and image processing based on the second anti-shake control parameter, to obtain a target image.

[0007] By the image anti-shake control method provided in the embodiments of the present application, in the shooting scene of the camera night scene mode, the first anti-shake control parameter suitable for the current scene can be determined based on the scene brightness information, such as the shaking information filtering mode, the image out-frame mode and / or the image fusion mode, and the second anti-shake control parameter suitable for the current scene can be determined based on the shaking information, such as the second anti-shake control parameter including the exposure compensation amount, different shaking information corresponding to different exposure compensation amounts. The present application can perform comprehensive anti-shake control and image processing based on the first anti-shake control parameter and the second anti-shake control parameter in the shooting stage, and different degrees of exposure compensation are performed in the shooting stage for different shaking intensity of the night scene shooting scene, which improves the image anti-shake effect of the night scene shooting scene and improves the clarity of the night scene photo.

[0008] It should be noted that in the process of shooting by using the camera night scene mode of the electronic device, due to the insufficient scene illumination in the night scene, the photo is easy to be blurred due to hand shaking. In this case, the image anti-shake control method provided in the embodiments of the present application can avoid or alleviate the problem of blurred night scene photo caused by shaking, and can effectively prolong the shutter time, so as to achieve the purpose of obtaining more exposure information, reducing the noise of the night scene photo and improving the clarity.

[0009] In the embodiments of the present application, the shooting process can be divided into a preview stage and a shooting stage. Specifically, after the camera application program is started, the preview stage is entered, when it is detected that the user triggers the shooting, the camera application program is switched from the preview stage to the shooting stage, and after the shooting is completed, the camera application program is switched from the shooting stage to the preview stage.

[0010] The possible implementation manner of the image anti-shake control method in the preview stage will be described first.

[0011] In some possible implementation manners, before receiving the first operation of the user triggering the shooting after starting the camera application program, the method further includes: in the preview stage, performing anti-shake control and image processing based on the fourth anti-shake control parameter.

[0012] In the preview stage, the electronic device can obtain the first shaking information through the gyroscope sensor.

[0013] For example, the first shaking information can be the offset angular velocity or the offset angle of the electronic device.

[0014] In some possible implementation manners, the anti-shake control and image processing based on the fourth anti-shake control parameter includes: (1) performing band-pass filtering on the first shake information acquired in the preview stage, and controlling the anti-shake device of the electronic device to move based on the first shake information filtered by the band-pass filtering, so as to offset the influence of shaking on the image; (2) acquiring an image by using a reference exposure parameter to obtain a reference exposure image; wherein the reference exposure parameter includes a reference exposure time length and a reference ISO value; (3) outputting an image sequence in the preview stage by using a ZSL mode, and the image sequence in the preview stage includes one or more reference exposure images.

[0015] In the preview stage, the band-pass filtering on the shake information and the ZSL mode for outputting frames have the beneficial effect of good preview real-time performance.

[0016] In the preview stage, the image sequence includes images subjected to anti-shake control. The image sequence in the preview stage can be used for scene recognition, so as to determine the brightness of the scene, and can also be used to determine whether the current scene meets the night scene shooting condition, and can also be used to determine the dynamic range of the image and enable variable exposure shooting in the case of a high dynamic range.

[0017] In some possible implementation manners, the image sequence in the preview stage and the image sequence in the shooting stage can be fused to obtain a night scene photo, so as to improve the image anti-shake effect and improve the definition of the photo.

[0018] The possible implementation manners of the image anti-shake control method in the shooting stage are described below.

[0019] In the shooting stage, the anti-shake control and image processing can be performed based on the first anti-shake control parameter, and the anti-shake control and image processing can be performed based on the second anti-shake control parameter to obtain a target image.

[0020] In the shooting stage, the anti-shake control and image processing can be performed based on the first anti-shake control parameter, and the anti-shake control and image processing can be performed based on the second anti-shake control parameter to obtain a target image.

[0021] In some possible implementation manners, the anti-shake control and image processing based on the first anti-shake control parameter can include: (1) performing filtering processing on the second shake information acquired in the shooting stage by using a first filtering manner, and the first filtering manner is a band-pass filtering or a low-pass filtering; (2) outputting an image sequence by using a first frame output mode, and the first frame output mode is a zero-lag shooting ZSL mode or a non-ZSL mode; (3) performing image fusion by using a first image sequence, and the first image sequence is the image sequence in the preview stage and the image sequence in the shooting stage, or is the image sequence in the shooting stage.

[0022] Specifically, the possible implementation manner of the image processing and the anti-shake control based on the first anti-shake control parameter can be determined based on scene brightness information. Different scene brightness information corresponds to different anti-shake control parameters.

[0023] In some possible implementation manners, different scene brightness information corresponds to different illumination levels, and different illumination levels correspond to different anti-shake control parameters. For example, three illumination levels are divided, which are referred to as a first illumination level, a second illumination level, and a third illumination level, and the illumination of the three illumination levels increases in turn. That is, the illumination of the first illumination level is less than the illumination of the second illumination level, and the illumination of the second illumination level is less than the illumination of the third illumination level.

[0024] The anti-shake control and the image processing based on the first anti-shake control parameter include the following three possible implementation manners.

[0025] Possible implementation manner one: when the first scene brightness information corresponds to the first illumination level (extremely low-illumination scene), the second jitter information is filtered by using a low-pass filtering manner, and the ZSL mode is switched to the non-ZSL mode, and the image fusion is performed by using the image sequence in the shooting stage.

[0026] Possible implementation manner two: when the first scene brightness information corresponds to the second illumination level (low-illumination scene), the second jitter information is filtered by using a low-pass filtering manner, and the ZSL mode is switched to the non-ZSL mode, and the image fusion is performed by using the image sequence in the preview stage and the image sequence in the shooting stage.

[0027] Possible implementation manner three: when the first scene brightness information corresponds to the third illumination level (medium-illumination scene), the second jitter information is filtered by using a band-pass filtering manner, and the ZSL mode is used to output frames, and the image fusion is performed by using the image sequence in the preview stage and the image sequence in the shooting stage.

[0028] The above scenes are divided based on the illumination level. In actual situations, a medium-illumination scene can be divided into a medium-illumination, high-dynamic-range scene or a medium-illumination, non-high-dynamic-range scene. The medium-illumination, high-dynamic-range scene and the medium-illumination, non-high-dynamic-range scene are different in the manner of the anti-shake control and the image processing based on the first anti-shake control parameter. In the medium-illumination, high-dynamic-range scene, the image can be acquired by using a variable exposure parameter to obtain a variable exposure image sequence.

[0029] In some possible implementation manners, the method further includes: determining, according to the first scene brightness information, that the current scene is a high-dynamic-range scene; and in a case where a first operation of triggering photographing by a user is received, setting an exposure parameter to a variable exposure parameter.

[0030] In some possible implementation manners, the variable exposure parameter comprises a first exposure duration and a second exposure duration; and the image sequence of the shooting stage comprises a first exposure image and a second exposure image. Alternatively, the variable exposure parameter comprises a first exposure duration, a second exposure duration and a reference exposure duration; and the image sequence of the shooting stage comprises a first exposure image, a second exposure image and a reference exposure image. The first exposure duration is greater than the reference exposure duration, and the reference exposure duration is greater than the second exposure duration.

[0031] Through the above scheme, image fusion based on a variable exposure image sequence can improve the image effect.

[0032] The above describes possible implementation manners of performing anti-shake control and image processing based on first anti-shake control parameters such as a dithering information filtering manner, an image out-frame mode and an image fusion manner. The following describes possible implementation manners of performing anti-shake control and image processing based on second anti-shake control parameters.

[0033] The second anti-shake control parameter comprises an exposure compensation amount. Different dithering information corresponds to different exposure compensation amounts.

[0034] In some possible implementation manners, performing anti-shake control and image processing based on the second anti-shake control parameter comprises: adjusting a reference exposure parameter based on the first exposure compensation amount to obtain a first exposure parameter; and collecting an image by using the first exposure parameter to obtain an image sequence of a shooting stage.

[0035] In some possible implementation manners, the first exposure compensation amount comprises a compensation amount of an exposure duration and a compensation amount of an ISO value of a light sensitivity, different dithering information corresponds to different dithering levels, and different dithering levels correspond to different exposure compensation amounts. For example, two dithering levels, a first dithering level (low dithering intensity) and a second dithering level (high dithering intensity), are taken as an example for description. The dithering intensity of the first dithering level is less than the dithering intensity of the second dithering level.

[0036] The adjusting the reference exposure parameter based on the first exposure compensation amount comprises: when the first dithering information corresponds to the first dithering level (low dithering intensity), increasing a first duration based on a reference exposure duration and decreasing a first light sensitivity value based on a reference ISO value of a light sensitivity.

[0037] Through the above scheme, in a scenario with low dithering intensity, generation of a night scene image is less affected by dithering. Therefore, the exposure duration can be appropriately increased and the light sensitivity can be appropriately reduced, which can effectively prolong the shutter time, thereby achieving the purpose of obtaining more exposure information, reducing noise of a night scene photo and improving definition.

[0038] The adjusting based on the first exposure compensation on the basis of the reference exposure parameter comprises: when the first shake information corresponds to a second shake level (high shake intensity), decreasing a second time length on the basis of a reference exposure time length and increasing a second ISO value on the basis of a reference ISO value.

[0039] Through the above scheme, in a high shake intensity scene, the generation of a night scene image is greatly affected by shaking, and therefore the exposure time length can be appropriately decreased and the ISO value can be appropriately increased, so that the problem of blur of a night scene photo caused by shaking can be avoided or mitigated, thereby improving the clarity and brightness of the night scene photo.

[0040] It should be noted that the first shake level (low shake intensity) can also be further subdivided, for example, the first shake level is divided into a case where the shake intensity is equal to 0 (tripod shooting scene) and a case where the shake intensity is greater than 0 and less than a first shake threshold (micro-shake scene).

[0041] In some possible implementations, the increasing the first time length on the basis of the reference exposure time length and the decreasing the first ISO value on the basis of the reference ISO value when the first shake information corresponds to the first shake level comprises:

[0042] (1) When the shake intensity indicated by the first shake information is equal to 0 (tripod shooting scene), the first shake information is within the first shake level range, the first time length is increased by N steps on the basis of the reference exposure time length, and the first ISO value is decreased by N steps on the basis of the reference ISO value.

[0043] (2) When the shake intensity indicated by the first shake information is greater than 0 and less than the first shake threshold, the first shake information is within the first shake level range, the first time length is increased by M steps on the basis of the reference exposure time length, and the first ISO value is decreased by M steps on the basis of the reference ISO value.

[0044] Wherein, N is greater than M, and M is greater than or equal to 0.

[0045] Through the above scheme, considering the tripod shooting scene and the micro-shake scene, the exposure time length and the ISO value are adjusted according to actual requirements and experience values respectively, thereby improving the clarity and brightness of the night scene photo.

[0046] It should be noted that the second shake level (high shake intensity) can also be further subdivided, for example, the second shake level is divided into a case where the shake intensity is greater than the first shake threshold and less than a second shake threshold (medium-intensity shake scene) and a case where the shake intensity is greater than or equal to the second shake threshold (high-intensity shake scene).

[0047] In some possible implementation manners, when the first shake information corresponds to a second shake level, a second time length is reduced on the basis of the reference exposure time length, and a second ISO value is increased on the basis of the reference ISO value, including:

[0048] (1) When the shake intensity indicated by the first shake information is greater than or equal to a first shake threshold and less than a second shake threshold, the first shake information is within the second shake level range, an S-grade exposure time length is reduced on the basis of the reference exposure time length, and an S-grade ISO value is increased on the basis of the reference ISO value.

[0049] (2) When the shake intensity indicated by the first shake information is greater than or equal to the second shake threshold, the first shake information is within the second shake level range, a T-grade exposure time length is reduced on the basis of the reference exposure time length, and a T-grade ISO value is increased on the basis of the reference ISO value.

[0050] Wherein, S is less than T, and S is greater than zero.

[0051] Through the above scheme, considering the medium-intensity shake scene and the high-intensity shake scene, the exposure time length and the ISO value are adjusted according to the actual demand and the experience value respectively, so that the clarity and the brightness of the night scene photo can be improved.

[0052] It should be noted that the corresponding limit exposure time can be pre-calibrated for different shake strengths. For example, a shake limit exposure time table can be obtained according to experience values. The limit exposure time corresponding to the shake strength value is determined by querying the shake limit exposure time table according to the shake strength value.

[0053] If the exposure time length calculated by the exposure compensation amount is greater than the limit exposure time found by the table lookup, the exposure time length and the ISO value need to be adjusted again.

[0054] In some possible implementation manners, the first exposure compensation amount is used to adjust the reference exposure parameter to obtain the first exposure parameter, including: the first exposure compensation amount is used to adjust the reference exposure parameter to obtain a first adjusted exposure time length and a first adjusted ISO value; if the first adjusted exposure time length is greater than a preset limit exposure time length, the exposure time length is set to the limit exposure time length, and the ISO value is adjusted again according to a first increment on the basis of the first adjusted ISO value. Wherein, the first exposure parameter includes the limit exposure time length and the ISO value after the second adjustment.

[0055] In some possible implementation manners, the adjusting the ISO value by the first increment based on the first adjusted ISO value comprises: calculating a first ratio according to the first adjusted exposure time and the limit exposure time, multiplying the first ratio by the first adjusted ISO value to obtain the first increment; and adding the first increment to the first adjusted ISO value to obtain a second adjusted ISO value.

[0056] For example, assuming that the preset limit exposure time is 500 ms, the exposure compensation amount is determined according to the shake intensity level, and the first adjustment is performed, for example, the first adjusted exposure time is 600 ms, and the first adjusted ISO value is 100. The first adjusted exposure time 600 ms is greater than the preset limit exposure time 500 ms. In this case, according to the scheme of the present application, the exposure time and the ISO value are second adjusted. On the one hand, the finally used exposure time is set to 500 ms. On the other hand, the first ratio is calculated according to the first adjusted exposure time and the limit exposure time: (600-500) / 600 = 1 / 6, and the first ratio is 1 / 6. The ISO value is adjusted by the first ratio, and it can be determined by calculation that the adjustment amount is 100*(1 / 6)≈17, and the second adjusted ISO value is 100+17 = 117.

[0057] Through the above scheme, in the case that the exposure time calculated by the exposure compensation amount is greater than the limit exposure time found by the table lookup, the exposure time and the ISO value are second adjusted, so that the motion blur of the image caused by exceeding the optical image stabilization capability can be avoided.

[0058] In addition, the embodiments of the present application can also perform exposure compensation based on face detection information in the shooting stage to improve the image stabilization effect. The possible implementation manners of performing exposure compensation based on face detection information in the shooting stage are described below.

[0059] In some possible implementation manners, the method further comprises: in the case that the camera night scene mode is enabled, obtaining face detection information; determining a third anti-shake control parameter according to the face detection information, different face detection information corresponding to different anti-shake control parameters; in the case that a first operation of triggering photographing by a user is received, performing anti-shake control and image processing based on the third anti-shake control parameter.

[0060] In some possible implementation manners, the third anti-shake control parameter described above comprises a second exposure compensation amount. In this case, the anti-shake control and image processing based on the third anti-shake control parameter comprises: adjusting a first exposure parameter based on the second exposure compensation amount to obtain a second exposure parameter; and collecting an image by using the second exposure parameter to obtain an image sequence in the shooting stage.

[0061] It should be noted that, since the second exposure parameter is adjusted on the basis of the first exposure parameter, that is, the second exposure parameter replaces the first exposure parameter, therefore, when the image is collected by using the second exposure parameter, the image is not collected by using the first exposure parameter.

[0062] In some possible implementation manners, different face detection information corresponds to different exposure compensation amounts. The first exposure parameter includes a first exposure duration and a first ISO value. In this case, the above-mentioned adjusting the second exposure compensation amount on the basis of the first exposure parameter includes: when the face detection information satisfies the portrait shooting condition, reducing an R-stop exposure duration on the basis of the first exposure duration and increasing an R-stop ISO value on the basis of the first ISO value. Wherein, R is greater than zero. When the face detection information does not satisfy the portrait shooting condition, the second exposure compensation amount is zero.

[0063] Through the above scheme, in the portrait shooting scene of the camera night scene mode, not only the first anti-shake control parameter determined based on the scene brightness information and the second anti-shake control parameter determined based on the shaking information are used for comprehensive anti-shake control and image processing, but also the third anti-shake control parameter determined based on the face detection information is used for further anti-shake control and image processing, which improves the image anti-shake control effect of the portrait shooting scene in the night scene mode and improves the clarity of the night scene portrait photo.

[0064] In some possible implementation manners, the enabling the camera night scene mode includes: acquiring an image sequence in a preview stage; performing scene recognition based on the image sequence in the preview stage; and in a case where a scene recognition result satisfies a camera night scene mode triggering condition, automatically enabling the camera night scene mode.

[0065] In some possible implementation manners, the enabling the camera night scene mode includes: in response to a second operation of the user starting the camera night scene mode, enabling the camera night scene mode.

[0066] Through the above scheme, whether the camera night scene mode is triggered by the user or automatically triggered by the electronic device according to the scene recognition result, the image anti-shake control method provided in the embodiments of the present application can be used for anti-shake control and image processing, a clearer night scene photo is obtained, and the user experience is improved.

[0067] In the optical anti-shake control and night scene photo generation process in the night scene shooting scene, the night scene optical anti-shake control is performed through the scheme of the present application, which can improve the clarity of the night scene photo and reduce the noise of the night scene photo.

[0068] In a second aspect, the present application provides an image anti-shake control apparatus, which comprises units for performing the method in the first aspect. The apparatus can correspond to performing the method described in the first aspect, and the related description of the units in the apparatus can refer to the description of the first aspect, which will not be repeated here for brevity.

[0069] The method described in the first aspect can be implemented by hardware, or the corresponding software can be executed by hardware. The hardware or software comprises one or more modules or units corresponding to the above functions. For example, processing modules or units, display modules or units, etc.

[0070] In a third aspect, the present application provides an electronic device, which comprises a processor, a processor and a computer program or instructions stored in the memory, and the processor is used to execute the computer program or instructions, so that the method in the first aspect is executed.

[0071] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program (also referred to as instructions or codes) for implementing the method in the first aspect. For example, when the computer program is executed by a computer, the computer can execute the method in the first aspect.

[0072] In a fifth aspect, the present application provides a chip comprising a processor. The processor is used to read and execute the computer program stored in the memory to execute the method in the first aspect and any possible implementation manner thereof. Optionally, the chip further comprises a memory, and the memory is connected to the processor through a circuit or a wire.

[0073] In a sixth aspect, the present application provides a chip system comprising a processor. The processor is used to read and execute the computer program stored in the memory to execute the method in the first aspect and any possible implementation manner thereof. Optionally, the chip system further comprises a memory, and the memory is connected to the processor through a circuit or a wire.

[0074] In a seventh aspect, the present application provides a computer program product, which comprises a computer program (also referred to as instructions or codes). When the computer program is executed by an electronic device, the electronic device can implement the method in the first aspect.

[0075] It can be understood that the beneficial effects of the above-mentioned second aspect to seventh aspect can refer to the related description in the first aspect, which will not be repeated here for brevity. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 The schematic diagram of the image sequence acquired after closing the image anti-shake and the image sequence acquired after opening the image anti-shake for the same shooting scene provided by the embodiment of the present application;

[0077] Figure 2 A schematic diagram of an image generated after anti-shake processing and an image generated without anti-shake processing for the same shooting scene is provided for an embodiment of the present application;

[0078] Figure 3 A schematic diagram of a hardware structure of an electronic device is provided for an embodiment of the present application;

[0079] Figure 4 A schematic diagram of a side view structure of a camera module is provided for an embodiment of the present application;

[0080] Figure 5 A schematic diagram of an anti-shake system architecture is provided for an embodiment of the present application;

[0081] Figure 6 A schematic diagram of a system architecture of an electronic device is provided for an embodiment of the present application;

[0082] Figure 7 A flowchart of an image anti-shake control method is provided for an embodiment of the present application;

[0083] Figure 8 A schematic diagram of an optical anti-shake control loop is provided for an embodiment of the present application;

[0084] Figure 9 A flowchart of an electronic device determining an optical anti-shake working mode is provided for an embodiment of the present application;

[0085] Figure 10 A flowchart of generating a night scene photo according to an image sequence output in a preview stage and an image sequence output in a shooting stage in a non-high dynamic scene is provided for an embodiment of the present application;

[0086] Figure 11 A flowchart of generating a night scene photo according to an image sequence output in a preview stage and an image sequence output in a shooting stage in a high dynamic scene is provided for an embodiment of the present application;

[0087] Figure 12 A flowchart of an image anti-shake control method is provided for an embodiment of the present application;

[0088] Figure 13 A schematic diagram of exposure compensation amounts corresponding to a plurality of shaking levels is provided for an embodiment of the present application;

[0089] Figure 14 A schematic diagram of a specific implementation of image anti-shake control in a medium-illumination, non-high dynamic scene in a night scene mode is provided for an embodiment of the present application;

[0090] Figure 15 A schematic diagram of a specific implementation of image anti-shake control in a medium-illumination, high dynamic scene in a night scene mode is provided for an embodiment of the present application;

[0091] Figure 16 A specific implementation of the image anti-shake control in a low-illumination scene in a night scene mode provided by an embodiment of the present application is shown in the following figure;

[0092] Figure 17 A specific implementation of the image anti-shake control in an extremely low-illumination scene in a night scene mode provided by an embodiment of the present application is shown in the following figure;

[0093] Figure 18 A schematic block diagram of the image anti-shake control device provided by an embodiment of the present application is shown in the following figure. DETAILED DESCRIPTION

[0094] The embodiments of the present application are further explained in detail below with specific embodiments and with reference to the accompanying drawings.

[0095] First, some terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0096] 1. Sensitivity

[0097] Sensitivity, also known as ISO value, refers to the sensitivity of a camera to light. The larger the ISO value, the higher the sensitivity, and the brighter the photos taken will be, and vice versa. However, too high an ISO value will also increase the noise of the photos.

[0098] 2. Exposure time

[0099] Exposure time refers to the time for which the shutter is open and closed. The length of the exposure time will affect the clarity of the photos. The longer the exposure time, the higher the image brightness, but it is also more likely to cause image blur.

[0100] 3. Exposure compensation

[0101] Exposure compensation refers to increasing or decreasing the amount of exposure in the current light measurement data. In actual implementation, exposure compensation can be achieved by adjusting at least one of the sensitivity ISO, exposure time, and aperture. Currently, EV (exposure value) is used to represent the amount of exposure. For example, when the sensitivity ISO is 100, the aperture value is F1, and the exposure time is 1 second, the resulting exposure amount is EV0. +1 EV of exposure compensation means increasing the exposure amount by one step, for example, by extending the exposure time by one time, or increasing the amount of light by one time, or increasing the ISO sensitivity by one time.

[0102] 4. Long-exposure shooting

[0103] Long exposure photography, also known as slow shutter photography, involves extending the exposure time to capture more light and details. However, when taking long exposure photos with handheld devices, factors such as device stability and air turbulence can cause blurring or motion blur, affecting the clarity of the photo. Additionally, long exposure times can also increase noise, degrading image quality.

[0104] 5. Variable exposure photography

[0105] Variable exposure photography involves capturing both long and short exposure images simultaneously using a sensor, and then combining them to create a high dynamic range (HDR) image. In scenes with large differences in brightness, traditional CMOS / CCD image sensors may not be able to capture the full range of light, resulting in overexposed highlights and underexposed shadows. Variable exposure photography can help address this issue by capturing both long and short exposure images and then combining them to create a single image that preserves both the highlights and shadows.

[0106] 6. Zero shutter lag (ZSL) mode

[0107] In zero shutter lag (ZSL) mode, the camera module buffers the captured image frames in memory to allow for quick access to the image captured just before the shutter button is pressed. This reduces the shutter lag, allowing for faster capture of dynamic scenes or fleeting moments. For example, in ZSL mode, the exposure time is relatively short, typically up to 1 / 10s or around 100ms. In non-ZSL mode, the exposure time is relatively longer, typically greater than 100ms.

[0108] 7. Motion blur

[0109] During photography, each frame of an image is generated by accumulating photons over the exposure time, converting them to electrons through photoelectric conversion, and further transforming them into an image recognizable by the human eye. During this time, if the electronic device experiences a large amplitude of motion, this motion information will also be recorded, resulting in a strong motion blur in the generated image.

[0110] 8. Optical image stabilization (OIS)

[0111] Optical image stabilization can also be referred to as optical stabilization, and OIS technology refers to detecting the shaking of an electronic device during exposure by a motion sensor (for example, a gyroscope, an accelerometer) during exposure, and an OIS controller controls a motor that pushes the OIS to move the lens or image sensor according to the shaking data detected by the motion sensor, so that the light path remains stable as much as possible during the entire exposure, and a clear exposure image is obtained.

[0112] Optical image stabilization includes two anti-shake modes, the first being lens movement optical image stabilization, and the second being photosensitive element movement optical image stabilization. The first lens movement optical image stabilization principle is to detect small movements by a gyroscope sensor in the lens, then transmit the signal to a microprocessor, which immediately calculates the displacement amount that needs to be compensated, and then compensates for the lens group according to the shaking direction and displacement amount of the lens, thereby effectively overcoming image blur caused by camera vibration. The second photosensitive element movement optical image stabilization uses image sensor offset to achieve anti-shake, and its principle is: first, place the CCD on a bracket that can move up, down, left and right, and then when the gyroscope sensor detects shaking, the direction, speed and movement of the shaking are processed to calculate the CCD movement amount sufficient to offset the shaking.

[0113] With the widespread use of electronic devices, using electronic devices to take pictures has become a daily behavior in people's lives. During the use of electronic devices by users to take pictures, there is inevitably shaking; for example, the shaking can be hand or touch shaking when the user takes pictures, or the shaking can be that the electronic device is moved during the taking process. Due to the shaking during the taking process, the video or image obtained by taking is blurred, affecting the clarity of the image. Taking a mobile phone as an example, when people hold the mobile phone to take pictures, the mobile phone will shake during the taking process due to hand shaking, walking vibration, etc., and thus the image is blurred.

[0114] To this end, the related art adds an optical image stabilization (OIS) controller to the electronic device to prevent shaking, and the principle of OIS controller anti-shake is to use an OIS motor to push the lens to move, thereby offsetting the displacement caused by shaking.

[0115] Figure 1 is a schematic diagram of an image sequence obtained by closing image stabilization and an image sequence obtained by opening image stabilization for the same shooting scene provided by an embodiment of the present application. As shown in (a) of Figure 1 , a user holds an electronic device (such as a mobile phone) to take pictures. Figure 1 The image sequence shown in (b) of Figure 1The image sequence shown in (c) in FIG. 7 can represent an image sequence acquired by the electronic device after the image stabilization is turned on. In combination with Figure 1 (b) in FIG. 7 and Figure 1 (c) in FIG. 7 can show that, after the image stabilization is turned on in the electronic device, the electronic device performs optical stabilization processing during the shooting process, which can improve the stability of the image sequence and avoid large shaking between image frames in the image sequence, thereby generating a relatively stable image sequence.

[0116] Figure 2 FIG. 7 is a schematic diagram of an image generated after stabilization processing and an image generated without stabilization processing for the same shooting scene, according to an embodiment of the present application. Figure 2 (a) in FIG. 7 shows an image generated based on an image sequence acquired after the image stabilization is turned on. This image is generated after stabilization processing. Figure 2 (b) in FIG. 7 shows an image generated based on an image sequence acquired after the image stabilization is turned off. This image is not generated after stabilization processing. In comparison, the image generated after stabilization processing has higher definition than the image not generated after stabilization processing.

[0117] However, for night scene shooting scenarios, the current optical stabilization performance is weak, and there is still imaging blur, which cannot meet the user's demand for shooting clear night scene photos. The reason why the optical stabilization performance for night scene shooting scenarios is weak is that, in the current related technology, the scene perception ability and the optical stabilization control ability are limited, and the night scene mode stabilization ability is insufficient to support a longer exposure time, resulting in that the night scene photo is greatly affected by hand shaking, causing quality problems such as night scene photo blur and large noise, and affecting the overall experience of the night scene mode.

[0118] To this end, there is an urgent need for an image stabilization control method more suitable for night scene shooting scenarios to improve the stabilization effect and improve the image quality.

[0119] To solve the above problems, the present application improves scene perception, optical stabilization control, and night scene image exposure processing. In the shooting scenario of the camera night scene mode, the first stabilization control parameter suitable for the current scene can be determined based on the scene brightness information, such as the shaking information filtering mode, the image frame output mode, and / or the image fusion mode, and the second stabilization control parameter suitable for the current scene can be determined based on the shaking information, such as the second stabilization control parameter including the exposure compensation amount, and different shaking information corresponds to different exposure compensation amounts. The present application can perform comprehensive stabilization control and image processing based on the first stabilization control parameter and the second stabilization control parameter in the shooting stage, and different degrees of exposure compensation are performed in the shooting stage for different shaking intensity of the night scene shooting scenario, which improves the image stabilization effect of the night scene shooting scenario in the night scene mode and improves the definition of the night scene photo.

[0120] The implementation process of the image anti-shake control method provided in the embodiments of the present application can include: performing scene sensing according to an image sequence output in a preview stage to obtain scene information; then determining an optical anti-shake working mode of a shooting stage according to the scene information; then switching to the optical anti-shake working mode of the shooting stage in response to a user triggering a shooting operation, setting a shake information filtering mode according to the optical anti-shake working mode, setting an image frame output mode, adjusting an exposure parameter, and outputting an image sequence of the shooting stage; and then performing image registration and fusion according to the image sequence of the preview stage and / or the image sequence of the shooting stage to obtain a night scene photo (target image).

[0121] In a shooting process using a camera night scene mode of an electronic device, due to insufficient scene illumination in the night scene, the photo is easily blurred due to hand shaking. In this case, the image anti-shake control method provided in the embodiments of the present application can avoid or alleviate the problem of night scene photo blurring caused by shaking, and can effectively prolong the shutter time, thereby achieving the purpose of obtaining more exposure information, reducing night scene photo noise, and improving the clarity.

[0122] The present application scheme can be applied to a night scene photo shooting scene. Optionally, the present application scheme can be applied to a shooting scene in which the user triggers to start the night scene mode, and can also be applied to a shooting scene in which the electronic device automatically starts the night scene mode. For ease of description, the shooting scene in which the user triggers to start the night scene mode and the shooting scene in which the electronic device automatically starts the night scene mode are collectively referred to as a night scene shooting scene.

[0123] The following first combines the accompanying Figure 3 The hardware system of the electronic device provided in the embodiments of the present application is described.

[0124] Figure 3 A hardware structure schematic diagram of an electronic device provided in the embodiments of the present application is shown. As Figure 3 shown, the electronic device 100 includes a camera module 110, an inertial measurement module (IMU) 120, a processor 130, a memory 140, and a display screen 150.

[0125] The processor 130 can include one or more processing units. For example, the processor 130 can include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU). Among them, different processing units can be independent devices, or can be integrated devices. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0126] The memory in the processor 130 can also be configured to store instructions and data. In some embodiments, the memory in the processor 130 is a cache memory. The memory can save instructions or data that the processor 130 has just used or repeatedly uses. If the processor 130 needs to use the instructions or data again, it can directly call from the memory. Avoiding repeated access reduces the waiting time of the processor 130, thereby improving the efficiency of the system.

[0127] The memory 140 can store programs that can be run by the processor 130, so that the processor 130 executes the method provided in the present application. The memory 140 can also store data. The processor 130 can read the data stored in the memory 140. The memory 140 and the processor 130 can be separately provided. Alternatively, the memory 140 can also be integrated in the processor 130.

[0128] In the embodiments of the present application, the code for implementing the image anti-shake control method described in the embodiments of the present application can be stored on a non-volatile memory. When the camera application is running, the electronic device 100 can load the executable code stored in the non-volatile memory to the random access memory.

[0129] In the embodiments of the present application, the processor 130 can be configured to start a camera application and enable a camera night scene mode; acquire first scene brightness information and first jitter information in a case where the camera night scene mode is enabled; determine a first anti-shake control parameter according to the first scene brightness information, the first anti-shake control parameter including a jitter information filtering mode, an image out-frame mode and / or an image fusion mode, different scene brightness information corresponding to different anti-shake control parameters; determine a second anti-shake control parameter according to the first jitter information, the second anti-shake control parameter including a first exposure compensation amount, different jitter information corresponding to different exposure compensation amounts; receive a first operation of a user triggering photographing; and in response to the first operation, perform anti-shake control and image processing based on the first anti-shake control parameter and perform anti-shake control and image processing based on the second anti-shake control parameter, to obtain a target image.

[0130] The camera module 110 can be configured to capture images. For example, the camera module 110 can include a camera module 111 and an optical image stabilization (OIS) system 112. The camera module 111 can be configured to generate an image at a certain time, which can include a plurality of pixels. The camera module 111 includes a lens and a light sensing element. During the generation of the image by the camera module 111, the optical image stabilization system 112 can dynamically adjust the relative position between the light sensing element and the lens to achieve mechanical stabilization.

[0131] The electronic device 100 can implement a photographing function through an image signal processor (ISP), a camera module 111, a video codec, a GPU, a display screen 150, and an application processor.

[0132] In the embodiments of the present application, the image signal processor (ISP) is configured to process data fed back by the camera module 111. For example, when photographing, the shutter is opened, light is transmitted to the camera light sensing element through the lens, the optical signal is converted into an electrical signal, and the camera light sensing element transmits the electrical signal to the image signal processor (ISP) for processing to convert it into an image visible to the naked eye. The image signal processor (ISP) can also optimize the algorithm for noise, brightness, and skin color of the image. The image signal processor (ISP) can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the image signal processor (ISP) can be disposed in the camera 193.

[0133] In embodiments of the present application, the camera module 111 is configured to capture still images or videos. An object projects an optical image through a lens onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then passed to an image signal processor (ISP) to convert into a digital image signal. The image signal processor (ISP) outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, or the like format image signal. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than one.

[0134] The inertial measurement module 120 can be configured to record the rotation and translation of the body of the electronic device 100 in real time, output the pose of the body at a certain time (i.e., the representation of the rotation and translation data), and output the pose of the body over a period of time (i.e., the motion path of the body over the period of time). For example, the inertial measurement module 120 can include a gyroscope and / or an accelerometer, etc.

[0135] The gyroscope sensor, also known as an angular velocity sensor, can be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyroscope sensor. The gyroscope sensor can be used in scenarios such as anti-shake photography, navigation, and motion sensing games, etc.

[0136] The acceleration sensor can detect the magnitude of acceleration of the electronic device 100 in various directions (generally the x-axis, the y-axis, and the z-axis). When the electronic device 100 is stationary, the acceleration sensor can detect the magnitude and direction of gravity. The acceleration sensor can also be used to identify the attitude of the electronic device 100, as an input parameter for applications such as landscape / portrait screen switching and pedometers, etc.

[0137] For ease of illustration, the following embodiments are described by way of example using a gyroscope sensor.

[0138] In embodiments of the present application, the gyroscope sensor can be used to collect jitter information, which can be used to represent the change in the pose of the electronic device during the shooting process. The jitter information can include the rotational angular velocity when the electronic device is deflected or tilted. Then, by discretely integrating the rotational angular velocity, the angle is obtained.

[0139] In the embodiments of the present application, the gyroscope sensor can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor detects the angle of shaking of the electronic device 100, calculates the distance that needs to be compensated by the camera module according to the angle, and makes the lens offset the shaking of the electronic device 100 by reverse movement, so as to realize anti-shake.

[0140] The display screen 150 is used to display images or videos captured by the camera module 111, etc. The display screen 150 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0141] The electronic device can realize the image display function through the GPU, the display screen 150, the application processor, etc. The GPU is a microprocessor for image processing, connected with the display screen 150 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0142] In addition, the electronic device 100 also includes an ambient light sensor. In the embodiments of the present application, the ambient light sensor can perceive the ambient light brightness, and the processor 130 can determine whether the current scene is a night scene shooting scene based on the ambient light brightness.

[0143] In addition, the electronic device 100 also includes a touch sensor. In the embodiments of the present application, the electronic device 100 can detect the user's clicking, sliding, etc. operation on the display screen 150 by using the touch sensor. For example, the touch sensor can detect the user's touch operation on the shooting control, and the processor 130 can execute the corresponding anti-shake strategy and image shooting in response to the touch operation.

[0144] It can be understood that the present scheme Figure 3The schematic structure does not constitute a specific limitation on the electronic device 100. In other embodiments of the present scheme, the electronic device 100 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0145] It can be understood that, in the embodiments of the present application, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, and / or a smart city device, and the specific type of the electronic device is not specially limited in the embodiments of the present application.

[0146] The camera module of the electronic device will be described in detail below. Figure 4 A side view structure schematic diagram corresponding to the camera module provided in the embodiments of the present application is shown.

[0147] Exemplarily, the camera module (or camera) usually includes an optical lens and a photosensitive element (also referred to as an image sensor), etc. The optical lens can include one or more lenses. The optical lens transmits light by using the refraction principle of the lens, so that the light is imaged on the photosensitive element.

[0148] It should be understood that the "lens" described in the present application can be understood as a whole lens, which can include one or more lenses. The "lens" can be understood as a lens in a lens structure or a lens or lens used to compose a lens.

[0149] In addition, there can be a filter (not shown in the figure) between the lens and the photosensitive element. The filter is used to filter out the unnecessary wave band in the light, to prevent the photosensitive element from producing false colors or moiré, so as to improve its effective resolution and color restoration. Of course, this is only an example, and the camera can also include other structures, which are not limited in the present application.

[0150] In combination with the example, in order to perform optical image stabilization, an OIS controller and an OIS motor can be added to the lens of the camera. The OIS controller is configured to obtain the shaking data of the electronic device collected by the gyroscope sensor, such as angular velocity, and generate a control signal for controlling the movement of the OIS motor according to the shaking data collected by the gyroscope sensor. The OIS motor is configured to move the lens under the control of the control signal, thereby offsetting the displacement caused by the shaking.

[0151] Referring to Figure 4 The optical image stabilization implementation of the OIS motor is described, and the coordinate system shown in Figure 4 The OIS motor in the OIS controller can push the lens to move left and right along the x-axis or y-axis or z-axis direction to offset the displacement caused by the shaking. It should be understood that the OIS controller can control the lens to move in various directions to offset the displacement caused by the shaking according to the compensation requirement, and the specific moving direction and distance can be determined according to the requirement, which is not limited by the embodiments of the present application.

[0152] It should be noted that the OIS algorithm module can send an anti-shake instruction to the OIS controller, and the OIS controller sends the anti-shake instruction to the OIS motor, and the OIS motor controls the camera lens to move according to the anti-shake instruction. The OIS algorithm module and the OIS controller can be two independent modules, or can be integrated into one module.

[0153] In combination with Figure 5 The anti-shake system architecture provided by the embodiments of the present application is described, Figure 5 The relationship structure diagram between the gyroscope sensor, the OIS controller and the lens is shown.

[0154] As Figure 4 indicated, the gyroscope sensor can be connected to the OIS controller. The gyroscope sensor obtains the angular velocity of the shaking of the electronic device and transmits the angular velocity of the shaking of the electronic device to the OIS controller.

[0155] The OIS controller can be connected to the OIS motor (optical image stabilization motor), and the OIS motor can be connected to the lens. The OIS controller calculates the compensation angle according to the angular velocity of the shaking of the electronic device, and instructs the OIS motor to adjust the angle of the lens to stabilize the imaging.

[0156] The OIS controller can also be connected to the application processor AP / image signal processor ISP, and the application processor AP / image signal processor ISP can be connected to the CMOS sensor, and the CMOS sensor can be connected to the lens. The OIS controller can call the application processor AP to calculate the compensation angle. The OIS controller can call the image signal processor ISP to correct the compensation angle.

[0157] The OIS controller can also be connected to a Hall sensor connected to the lens. The Hall sensor functions to feed back position information of the lens to the OIS controller, forming a closed loop control to accurately move the lens to the required position.

[0158] The OIS controller can also be referred to as an OIS control driver or an OIS driving chip.

[0159] Reference Figure 5 The basic principle of optical image stabilization is shown in the structural schematic diagram. When the gyroscope sensor is vibrated by the external environment, the vibration signal is fed back to the OIS controller, the OIS controller drives the OIS motor to move the lens, thereby offsetting the image offset caused by the vibration, and ensuring that the camera can still maintain stable imaging in the vibration environment.

[0160] The hardware system of the electronic device 100 is described in detail above, and the software system of the electronic device 100 is introduced below. The software operating system is run on the hardware system, and the software operating system can be any one or more computer operating systems that implement business processing through processes. Application programs can be installed and run on the software operating system.

[0161] Figure 6 FIG. 1 is a schematic diagram of a system architecture of an electronic device provided by an embodiment of the present application. As shown in FIG. 1, the system architecture can include an application layer 210, an application framework layer 220, a hardware abstraction layer (HAL) 230, a driver layer 240, and a hardware layer 250 from top to bottom. Figure 6

[0162] The application layer 210 can include a series of application program packages. In the embodiment of the present application, the application program packages can include a camera application program, a gallery, and the like.

[0163] The application framework layer 220 provides application program interfaces (APIs) and programming frameworks for the application programs of the application layer; the application framework layer can include some predefined functions.

[0164] In the embodiment of the present application, the application framework layer 220 can include a camera access interface; the camera access interface is used to provide application programming interfaces and programming frameworks for the camera application. The camera access interface can include camera management and camera devices. The camera management can be used to provide an access interface for managing the camera; the camera device can be used to provide an interface for accessing the camera.

[0165] The hardware abstraction layer 230 is an interface layer between the application framework layer and the driver layer, and provides a virtual hardware platform for the operating system.​

[0166] In the embodiment of the present application, the camera hardware abstraction layer and the camera algorithm are included in the hardware abstraction layer 230. The camera hardware abstraction layer can call the camera algorithm; the camera algorithm can include a software algorithm for image processing. The camera hardware abstraction layer 230 can provide virtual hardware of the camera device.

[0167] Exemplarily, the camera algorithm library can include the running code and data of the image anti-shake control method (or OIS algorithm) provided by the embodiment of the present application.

[0168] Exemplarily, the algorithm in the camera algorithm can refer to an implementation independent of specific hardware; for example, code that can usually run in the CPU, etc.

[0169] The driver layer 240 is a layer between hardware and software. The driver layer includes drivers of various hardware, which are used to provide drivers for different hardware devices.

[0170] In the embodiment of the present application, the driver layer can include camera device drivers, digital signal processor drivers, and image processor drivers, etc. The camera device drivers (for example, camera drivers) are used to drive the camera sensor to collect images and drive the image signal processor to pre-process the images. The digital signal processor drivers are used to drive the digital signal processor to process images. The image processor drivers are used to drive the graphics processor to process images.

[0171] The hardware layer 250 is located at the bottom layer of the operating system. As shown in the figure, the hardware layer 250 can include a camera, a display screen, and an OIS motor. Figure 6

[0172] It should be noted that, although the embodiment of the present application is described by taking the Android system as an example, the basic principles are also applicable to electronic devices based on iOS or Windows operating systems.

[0173] The working process of the software system and the hardware system of the electronic device 100 will be described below in conjunction with a photographing scenario.

[0174] ​When the user performs a touch operation on the touch sensor, a corresponding hardware interrupt is sent to the kernel layer, and the kernel layer processes the touch operation into a raw input event, which includes, for example, touch coordinates and a timestamp of the touch operation. The raw input event is stored in the kernel layer, and the application framework layer obtains the raw input event from the kernel layer, identifies the control corresponding to the raw input event, and notifies the application (APP) corresponding to the control. For example, the touch operation described above is a single-click operation, the APP corresponding to the control is a camera APP, and after the camera APP is awakened by the single-click operation, the camera APP can call the kernel layer camera driver through an API to control the camera module to perform photographing through the camera driver.

[0175] Taking a photographing scenario as an example, in response to a user operation of opening a camera application, for example, an operation of clicking a camera application icon, the camera application calls a camera access interface of the application framework layer to start the camera application, and then sends an instruction of starting a camera to a camera device in the camera hardware abstraction layer (camera device 1 and / or another camera device). The camera hardware abstraction layer sends the instruction to a camera device driver in the kernel layer. The camera device driver can start a corresponding camera sensor and acquire an image light signal through the sensor. One camera device in the camera hardware abstraction layer corresponds to one camera sensor in the hardware layer.

[0176] Then, the camera sensor can transmit the acquired image light signal to an image signal processor for preprocessing to obtain an image electrical signal (original image), and transmit the original image to the camera hardware abstraction layer through the camera device driver.

[0177] The camera hardware abstraction layer can send the original image to a camera algorithm library. The camera algorithm library stores program codes for implementing the image anti-shake control method provided in the embodiments of the present application. The camera algorithm library executes the codes based on a digital signal processor and an image processor. In a photographing scenario of a camera night scene mode, a first anti-shake control parameter suitable for a current scenario can be determined based on scene brightness information, such as a jitter information filtering mode, an image frame output mode, and / or an image fusion mode, and a second anti-shake control parameter suitable for the current scenario can be determined based on jitter information, such as an exposure compensation amount. Different exposure compensation amounts correspond to different jitter information. Comprehensive anti-shake control and image processing are performed based on the first anti-shake control parameter and the second anti-shake control parameter in the photographing phase, different degrees of exposure compensation are performed for night scene photographing scenarios with different jitter intensities in the photographing phase, and a night scene photo is generated.

[0178] The camera algorithm library can send the night scene photo to the camera hardware abstraction layer. Then, the camera hardware abstraction layer can display the night scene photo and store the night scene photo in a gallery.

[0179] The embodiment of the present application provides a kind of image anti-shake control method and electronic equipment, the software of electronic equipment is improved, in the shooting scene of camera night scene mode, in shooting stage, based on the first anti-shake control parameter and the second anti-shake control parameter, comprehensive anti-shake control and image processing are carried out, for different intensity of night scene shooting scene of shaking, different degree of exposure compensation is carried out in shooting stage, improve the image anti-shake effect of night scene mode shooting scene, improve the definition of night scene photo.

[0180] The execution subject of the image anti-shake control method provided by the embodiment of the present application can be the electronic device described above, or the functional module and / or functional entity capable of implementing the image anti-shake control method in the electronic device, and the present application can be implemented by hardware and / or software, and the specific implementation can be determined according to actual use requirements, which is not limited by the embodiment of the present application. In the following, the image anti-shake control method provided by the embodiment of the present application is exemplarily described with reference to the electronic device.

[0181] The embodiment of the present application will be illustrated by the following multiple exemplary embodiments with reference to the accompanying drawings. The methods in the following embodiments can be implemented in the electronic device with the above-mentioned hardware structure and software architecture. The hardware structure diagram of the electronic device can be as shown in Figure 3 to Figure 5 The software structure block diagram of the electronic device can be as shown in Figure 6 but the embodiment of the present application is not limited thereto. For ease of illustration, the electronic device is taken as the execution subject in the embodiment of the present application.

[0182] The image anti-shake control method provided by the embodiment of the present application will be introduced below in combination with specific embodiments.

[0183] In the embodiment of the present application, the working stage of the camera module can include a preview stage and a shooting stage. The camera module switches from the preview stage to the shooting stage after the user triggers shooting.

[0184] In the embodiment of the present application, the camera module can collect a preview image sequence in ZSL mode in the preview stage. The camera module can collect an image sequence in ZSL mode or non-ZSL mode in the shooting stage.

[0185] For ease of illustration, the preview image sequence output by the camera module in the preview stage is collectively referred to as front frame image or image sequence in the preview stage, and the image sequence output by the camera module in the shooting stage is collectively referred to as rear frame image or image sequence in the shooting stage.

[0186] In the embodiment of the present application, since the user holds the electronic device for shooting, there are various possible shaking in the preview mode or in the shooting mode, which will affect the image definition. That is, anti-shake processing is needed in the preview mode and the shooting mode.

[0187] Figure 7 is a flowchart of an image anti-shake control method provided by an embodiment of the present application. Referring to FIG. 3, the image anti-shake control method includes the following steps S301-S304. Figure 7

[0188] S301, scene recognition is performed according to an image sequence output in a preview stage to obtain scene information.

[0189] In an embodiment of the present application, the scene information includes scene brightness information, shake information and scene perception information.

[0190] In an embodiment of the present application, after the electronic device starts a camera program, the electronic device enters a preview stage. An image sensor outputs a raw image sequence, which is processed by a hardware ISP to obtain a processed preview image sequence. Each frame of the preview image sequence is down-sampled, for example, the resolution of the image is reduced to VGA resolution, and the image sequence obtained after the down-sampling is output as a tiny stream. The electronic device processes the tiny stream by using a statistical algorithm and a perception algorithm to obtain scene brightness information, shake information and scene perception information.

[0191] The scene brightness information (denoted as Lux_Idx) can be obtained based on automatic exposure statistics and an automatic exposure algorithm.

[0192] The mobile phone shake information (denoted as Gyro_Info) is from an IMU sensor.

[0193] The scene perception information includes scene classification information. For example, it can be determined whether the current scene is a night scene shooting scene according to the scene classification information. For example, when the scene classification information is Scene_Idx, it indicates that the current scene is a night scene shooting scene.

[0194] In an embodiment of the present application, different scene perception information Scene_Idx can represent different categories of shooting scenes. For example, different categories of scenes can include sunny shooting scenes, overcast shooting scenes, night scene shooting scenes, food shooting scenes, indoor shooting scenes, pet shooting scenes, etc.

[0195] The scene perception information can also include face detection information (denoted as Face_detect_info) obtained by processing the tiny stream by using a face detection algorithm. For example, it can be determined whether the current scene is a portrait night scene shooting scene according to the face detection information.

[0196] S302, determining an optical anti-shake working mode of a shooting stage according to the scene information.

[0197] ​It should be noted that, since the optical anti-shake system has a good effect on reducing the shaking blur in the handheld shooting scene, the optical anti-shake system is widely used on mobile phones. Among them, the optical anti-shake system has several forms, for example, the optical anti-shake system can adopt a lens moving mode, or can adopt a photosensitive element moving mode. The control of the optical anti-shake system also has multiple forms, for example, the optical anti-shake system can adopt a control mode based on a driver IC (Driver IC), or can adopt a control mode based on a sensor hub (Sensor Hub).

[0198] In practical applications, in addition to the optical and mechanical design of the high-quality optical anti-shake system itself, the most important thing is the control system, that is, how to realize the optical anti-shake control through the OIS controller.

[0199] Figure 8 A schematic diagram of the optical anti-shake control loop realized based on the OIS controller provided in the embodiments of the present application is shown.

[0200] On the one hand, referring to ① in Figure 8 , in the shooting process, the Hall element detects the lens position information, and feeds back the lens position information after amplifier processing to the operator. It should be noted that the lens position information indicates the lens position in the case of shaking of the electronic device. In some embodiments, the offset of the lens position relative to the initial lens position in the case of shaking of the electronic device is defined as a positive offset (+).

[0201] On the other hand, referring to ② in Figure 8 , in the shooting process, the gyroscope sensor detects the shaking information of the electronic device, and the shaking information is the rotation angular velocity when the electronic device is deflected and tilted. After the rotation angular velocity is discretely integrated by the integrator, the angle is obtained. The angle can be the offset angle around the X / Y axis. The gyroscope sensor feeds back the offset angle around the X / Y axis to the OIS controller, and then the OIS controller feeds back the offset angle around the X / Y axis to the operator. In some embodiments, the offset of the offset angle around the X / Y axis relative to the initial lens position in the case of shaking of the electronic device is defined as a negative offset (-).

[0202] It can be understood that the operator obtains the current lens position in the case of shaking of the electronic device and the offset angle of the lens around the X / Y axis. The operator can calculate the distance to be compensated by the camera module according to the offset angle, and the OIS controller can control the lens to compensate the distance by reverse offset on the basis of the current lens position, to offset the shaking of the electronic device and realize anti-shake.

[0203] Then, referring to Figure 8In the step ③ of the method, the OIS controller drives the actuator to drive the OIS motor to move the lens reversely, so as to offset the image offset caused by the shaking, and ensure that the camera can still keep imaging stable in the shaking environment.

[0204] It should be noted that, in different shooting scenes and different shaking conditions, there are different requirements for the signal processing and anti-shake control mode of the optical anti-shake system. For example, in the urban night scene shooting scene with good illumination, the reference exposure time is relatively short, the requirement for optical anti-shake is relatively low, and the preview real-time and followability are required to be high. For another example, in the night scene shooting scene with low illumination, the reference exposure time is relatively long, the requirement for optical anti-shake is relatively high, and the preview real-time and followability are required to be low.

[0205] In the embodiments of the present application, in view of different requirements, the parameters of the optical anti-shake system can be adaptively adjusted to some extent in combination with the scene brightness information, the shaking information and the scene sensing information, so as to better adapt to the anti-shake requirements of each scene.

[0206] It should be noted that the control signal of the optical anti-shake system is obtained by signal processing based on the detection signal of the gyroscope (Gyro) sensor. For the convenience of description, the detection signal of the gyroscope sensor is referred to as the gyroscope signal. Before using the gyroscope signal, a series of filtering processing needs to be performed on the gyroscope signal to retain the effective information of the gyroscope signal and eliminate noise.

[0207] For example, for the shooting preview or short exposure shooting scene, the optical anti-shake system needs to perform band-pass filtering on the gyroscope signal to avoid the influence of the low-frequency response part on the response speed and followability of the shooting preview and short exposure shooting.

[0208] For another example, for the long exposure shooting scene, the optical anti-shake system needs to retain the low-frequency component of the gyroscope signal to better compensate for the low-frequency shaking such as human breathing, and partially abandon some followability experience.

[0209] In combination with the above scene brightness information, shaking information and scene sensing information, the shooting stage optical anti-shake working mode can be determined in combination with prior information for pre-calibration or in combination with algorithms for real-time determination.

[0210] Figure 9 The flowchart of the method for determining the optical anti-shake working mode of the electronic device provided in the embodiments of the present application is shown. As shown in FIG. 4, the method comprises the following steps. Figure 9As shown, the night scene algorithm module includes a night scene stabilization decision module and a night scene shooting decision module. In this embodiment, the night scene stabilization decision module is a decision module for controlling the stabilization device. The night scene stabilization decision module can combine scene perception information, shake information, and scene brightness information to determine the working mode of the stabilization device and the filtering method for shake information. The night scene shooting decision module can combine scene brightness information and dynamic range information to determine frame mode switching, exposure compensation algorithm, and image fusion algorithm.

[0211] First, after the camera program is activated, the electronic device acquires scene perception information, filtered shake information, and scene brightness information, and then passes this information to the night scene stabilization decision module. For example, the filtered shake information Gyro_Info, scene brightness information Lux_Idx, scene perception information Scene_Idx, and face detection information Face_detect_info are passed to the night scene stabilization decision module.

[0212] Then, the night scene stabilization decision module determines whether to switch the optical image stabilization mode based on the scene perception information Scene_Idx, the shaking information Gyro Filter, the scene brightness information Lux_Idx, and the face detection information Face_detect_info.

[0213] It should be noted that, in this embodiment, the electronic device can determine whether the current scene is a night scene shooting scene based on the scene awareness information Scene_Idx. For example, when the scene awareness information Scene_Idx == Night_Scene, it is determined that the current scene is a night scene shooting scene. When the current scene is determined to be a night scene shooting scene, processing such as switching the optical image stabilization working mode is performed based on the scene brightness information Lux_Idx, the filtered jitter information Gyro_Info, and the face detection information Face_detect_info. It can be understood that if the current scene is determined to be a non-night scene shooting scene, then processing such as switching the optical image stabilization working mode is not performed.

[0214] Finally, the night scene image stabilization decision module transmits the determination result of the optical image stabilization working mode to the OIS HAL layer driver, which then transmits it to the OIS bottom layer driver, which in turn transmits it to the OIS motor. Finally, the OIS motor switches the optical image stabilization working mode.

[0215] In some embodiments, the optical anti-shake working mode can be switched in response to a switching event from the ZSL mode to the non-ZSL mode. The optical anti-shake working mode includes an anti-shake device working mode, whether the dither information filtering mode is switched, whether the out-of-frame mode is switched, an exposure compensation algorithm, an image fusion algorithm, and the like. Exemplarily, in the shooting phase, in the case of switching from the ZSL mode to the non-ZSL mode, the night scene anti-shake decision module switches the filtering processing mode of the dither information. For example, in the ZSL mode, the dither information is subjected to band-pass filtering processing, and in the non-ZSL mode, the dither information is subjected to low-pass filtering processing.

[0216] The following describes an embodiment of the present application in selecting a filtering processing mode of dither information in the ZSL mode or the non-ZSL mode.

[0217] In some embodiments, in the ZSL mode, the exposure time is relatively short, up to 1 / 10 s, about 100 ms or so; at this time, the gyro signal is subjected to band-pass filtering processing, so as to avoid the introduction of low-frequency and step signals to cause excessive anti-shake of the preview picture, and the hand-following feeling of the preview panning can be improved.

[0218] In some embodiments, in the non-ZSL mode, the exposure time is relatively long, which will be greater than 100 ms, at this time, the gyro signal is subjected to low-pass filtering processing, and more low-frequency signals are introduced into the optical anti-shake control, mainly solving the low-frequency dithering problem caused by the natural breathing of a person during long-exposure shooting, and the long-exposure imaging clarity can be improved.

[0219] In some embodiments, in the special ZSL mode, if it is necessary to enter a super optical anti-shake working mode supporting low frequency, mode switching can be performed by combining relevant information and the night scene anti-shake decision module. It should be noted that there will be 1 to 2 frames unavailable in the mode switching process, which will affect the preview effect and performance.

[0220] The following describes a process of switching between the ZSL mode and the non-ZSL mode according to the light condition in an embodiment of the present application.

[0221] In an embodiment of the present application, the electronic device can switch between the ZSL mode and the non-ZSL mode according to the light condition. It is assumed that the lower limit threshold Thd_w1 and the upper limit threshold Thd_w2 are used to judge the light condition.

[0222] When the light condition is Lux_Idx>Thd_w2, the night scene shooting decision module determines that the current scene is an extremely low-illumination scene. The night scene shooting decision module can switch from the ZSL mode to the non-ZSL mode.

[0223] When the illumination condition is Thd_w1 < Lux_Idx ≤ Thd_w2, the night scene shooting decision module determines that the current scene is a low-illumination scene. The night scene shooting decision module can fuse the ZSL out-frame image and the non-ZSL out-frame image to obtain a target image.

[0224] When the illumination condition is Lux_Idx ≤ Thd_w1, the night scene shooting decision module determines that the current scene is a medium-high-illumination scene. In some embodiments, if the current scene is a medium-high-illumination scene but not a high-dynamic scene, the night scene shooting decision module only performs ZSL out-framing. In other embodiments, if the current scene is a medium-high-illumination scene and a high-dynamic scene, the night scene shooting decision module can perform variable-exposure shooting to improve the dynamic range.

[0225] It should be noted that the value of Lux_Idx is inversely proportional to the scene brightness. For example, the larger the value of Lux_Idx, the lower the scene brightness; the smaller the value of Lux_Idx, the higher the scene brightness. Here, the scene brightness is exemplarily illustrated by using Lux_Idx, and in actual implementation, the scene brightness can also be represented in other manners, which is not limited in the embodiments of the present application.

[0226] The image anti-shake control according to the scene brightness information in the night scene shooting scene implemented by the embodiments of the present application is described below. The image anti-shake control includes the control of the filtering mode of the jitter information, the image out-frame mode, and the image fusion mode.

[0227] Firstly, it should be noted that the image out-frame mode includes the ZSL mode and the non-ZSL mode. The image output in the ZSL mode can be referred to as ZSL out-frame, and the image output in the non-ZSL mode can be referred to as non-ZSL out-frame. The image output in the ZSL mode can be referred to as ZSL out-frame image, and the image output in the non-ZSL mode can be referred to as non-ZSL out-frame image.

[0228] In some embodiments, when Lux_Idx > Thd_w2, it is determined that the current scene is an extremely low-illumination scene. In this case, the image sequence in the shooting stage is used to generate a night scene photo, and the shooting stage uses the non-ZSL mode. Moreover, when switching from the ZSL mode to the non-ZSL mode, the electronic device sets the filtering mode of the jitter information to the low-pass filtering mode GYRO_FILTER_LOWPASS_MODE. When the non-ZSL out-frame is completed and the preview is restarted, the electronic device sets the filtering mode of the jitter information to the band-pass filtering mode GYRO_FILTER_BANDPASS_MODE.

[0229] In some embodiments, when Thd_w2≥Lux_Idx>Thd_w1, the current scene is determined as a low-illumination scene. In this case, the ZSL out-frame image and the non-ZSL out-frame image are acquired, and the ZSL out-frame image and the non-ZSL out-frame image are fused to obtain the target image. That is, the preview stage adopts the ZSL mode and the shooting stage adopts the non-ZSL mode. Moreover, when switching from the ZSL mode to the non-ZSL mode, the electronic device sets the filtering mode for the jitter information as the low-pass filtering mode GYRO_FILTER_LOWPASS_MODE. When the non-ZSL out-frame is completed and the preview is restarted, the electronic device sets the filtering mode for the jitter information as the high-pass filtering mode GYRO_FILTER_BANDPASS_MODE.

[0230] In the embodiments of the present application, the night scene anti-shake decision module can determine the exposure compensation amount in the shooting stage according to the filtered jitter information Gyro_Info.

[0231] The exposure parameters involved in the embodiments of the present application will be introduced first.

[0232] In the embodiments of the present application, in the night scene shooting scene, the exposure parameters such as the exposure time and / or the ISO can be adjusted to adjust the exposure amount in the shooting process.

[0233] For the exposure time, a plurality of gears can be provided. Exemplarily, the gears of the exposure time (also referred to as the shutter time) include 1 second, 1 / 2 second, 1 / 4 second, 1 / 8 second, 1 / 15 second, 1 / 30 second, 1 / 60 second, 1 / 125 second, 1 / 250 second, 1 / 500 second, 1 / 1000 second, etc. The exposure amount can be adjusted by increasing or decreasing the gear of the exposure time, for example, the exposure amount is doubled if the exposure time is doubled. It can be understood that the exposure time can also have other gear settings.

[0234] For the ISO, a plurality of gears are also provided. Exemplarily, the gears of the ISO can include 100, 200, 400, 800, 1600, 3200, 6400, etc. The exposure amount can be adjusted by increasing or decreasing the gear of the ISO, for example, the exposure amount is doubled if the adjacent gears differ by a factor of two in value. It can be understood that the ISO can also have other gear settings.

[0235] It should be noted that the electronic device pre-sets a reference exposure time and a reference ISO. In the preview stage and the shooting stage, the electronic device can acquire an image sequence according to the reference exposure time and the reference ISO. In the embodiments of the present application, the exposure time and / or the ISO value can be adjusted under different jitter intensities to improve the image anti-shake effect.

[0236] The following further illustrates possible implementation manners of determining a shake intensity level according to the filtered shake information and adjusting the exposure time and / or the ISO value according to the shake intensity level.

[0237] Exemplarily, the embodiments of the present application provide the following five possible cases.

[0238] Case 1: When the filtered shake information is greater than the second shake threshold, i.e., Gyro_Info > Thd_gyro_shake_w2, it is determined that the current shake amplitude / intensity belongs to a large-amplitude shake case. In this case, the offset / adjustment of the reference exposure time of the shooting stage is set according to the preset parameter corresponding to the large-amplitude shake case. For example, the exposure time is reduced by 2 steps, and the ISO value is increased by 2 steps.

[0239] Case 2: When the filtered shake information is less than or equal to the second shake threshold and greater than the first shake threshold, i.e., Thd_gyro_shake_w2≥Gyro_Info>Thd_gyro_shake_w1, it is determined that the current shake amplitude / intensity belongs to a medium-amplitude shake case. In this case, the exposure time offset of the post-out frame is set according to the preset parameter corresponding to the medium-amplitude shake case. For example, the exposure time is reduced by 1 step, and the ISO value is increased by 1 step.

[0240] Case 3: When the filtered shake information is less than or equal to the first shake threshold, i.e., Gyro_Info≤Thd_gyro_shake_w1, it is determined that the current shake amplitude / intensity belongs to a small-amplitude shake case. In this case, the reference exposure time offset of the post-out frame is set according to the preset parameter corresponding to the small-amplitude shake case. For example, in some embodiments, the exposure time is not reduced, and the ISO value is kept unchanged. In other embodiments, the exposure time is increased by 1 step to 2 steps, and the ISO value is reduced by 1 step to 2 steps.

[0241] Case 4: When the filtered shake information is close to 0, i.e., Gyro_Info≈0, it is determined that the current shake amplitude / intensity belongs to a tripod case. In this case, the reference exposure offset of the post-out frame is set according to the preset parameter corresponding to the tripod. For example, in some embodiments, the exposure time is increased by 2 steps or more, and the ISO value is reduced by 2 steps or more. In other embodiments, when the filtered shake information is close to 0, it can also be selected to continue to execute the default optical anti-shake working mode without switching the optical anti-shake working mode.

[0242] Case 5: When the face detection information meets the preset face detection condition, such as is_face_detect = 1, it is determined that the current scene is a portrait night scene. In this case, on the basis of the exposure compensation based on the shake condition correction, the exposure time of the next frame image is reduced by 0.5 to 1 stop, and the ISO value of the sensitivity is increased by 0.5 to 1 stop, so as to better capture the night portrait image and improve the picture rate.

[0243] wherein the first shake threshold Thd_gyro_shake_w1 is less than the second shake threshold Thd_gyro_shake_w2, and the first shake threshold Thd_gyro_shake_w1 is greater than 0.

[0244] The following describes how the application adjusts the exposure time when the adjusted exposure time exceeds the limit exposure time.

[0245] In the embodiments of the application, the corresponding limit exposure time can be pre-calibrated for different shake intensities. That is, a corresponding relationship between the shake intensity level and the limit exposure time is established, for example, the corresponding relationship between the shake intensity level and the limit exposure time is embodied by a limit exposure time table.

[0246] In the embodiments of the application, in combination with the anti-shake capability of the optical anti-shake system and the pre-calibrated limit exposure time, the limit exposure time under different anti-shake intensities can be obtained.

[0247] In some embodiments, after the above exposure time offset calculation is completed, the limit exposure time OIS_Max_Exposure_Time matching the shake information Gyro_Info is searched in the limit exposure time table according to the shake information Gyro_Info.

[0248] If the exposure time obtained after the above exposure time offset calculation is greater than the limit exposure time OIS_Max_Exposure_Time, the exposure time is adjusted to the limit exposure time, and the ISO value is adjusted in proportion to avoid image motion blur due to exceeding the optical anti-shake capability.

[0249] Exposure Time is 500 ms, the exposure compensation determined according to the shake intensity level is to increase the exposure time and reduce the ISO value, for example, the first adjusted exposure time is 600 ms (for example, 300 ms + 300 ms = 600 ms), and the first adjusted ISO value is 100. The first adjusted exposure time 600 ms is greater than the limit exposure time 500 ms, which is equivalent to a reduction of 1 / 6 in the first adjusted exposure time. In this case, according to the scheme of the present application, the exposure time and the ISO value are adjusted twice, for example, the finally adopted exposure time is set to the limit exposure time 500 ms; accordingly, the adjustment amount of the ISO is calculated in proportion, for example, since the exposure time is reduced by 1 / 6, the ISO value is increased by 1 / 6 based on the first adjusted ISO value, that is, 100 + 100*(1 / 6)≈117, so the ISO value can be set to 117.

[0250] It should be noted that the threshold, gear and limit exposure time can be set according to actual use requirements. For example, the threshold, gear and limit exposure time can be adjusted to a preset empirical value in combination with the sensor light sensitivity and night scene algorithm capability, and the threshold, gear and limit exposure time can be flexibly adjusted according to actual requirements to obtain the best effect.

[0251] S303, in response to the user triggering the photographing operation, switching to the optical image stabilization working mode of the shooting stage, setting the shake information filtering mode according to the optical image stabilization working mode of the shooting stage, setting the image frame output mode, adjusting the exposure parameter, and outputting the image sequence of the shooting stage.

[0252] In some embodiments, the user triggering the photographing operation can be the user clicking the photographing button operation. In some embodiments, the user triggering the photographing operation can be the user triggering the photographing operation by voice. In some embodiments, the user triggering the photographing operation can be the user triggering the photographing operation by gesture. In some embodiments, the user triggering the photographing operation can be the user triggering the photographing operation by setting the countdown. In actual implementation, the user triggering the photographing operation can also be other forms of operation, which is not limited in the embodiments of the present application.

[0253] It should be noted that in the embodiments of the present application, considering that mode switching will affect performance, for example, 1 frame or 2 frames of images will be lost during mode switching, and 1 frame to 2 frames of time will be added on the basis of the freeze screen when switching from the preview stage to the long exposure shooting stage, which will affect the overall experience. In the embodiments of the present application, mode switching is not performed during the night scene preview process, but is performed at the time of night scene shooting.

[0254] In the embodiment of the present application, the optical anti-shake working mode is switched, and images are output according to the optical anti-shake working mode, which includes: setting the filtering mode and setting the exposure parameter, then performing image exposure collection according to the set exposure parameter, filtering the anti-shake information using the set filtering mode during the image exposure process, and performing anti-shake processing according to the filtered anti-shake information. Then, the image sequence is output in the shooting stage.

[0255] Specifically, in response to the operation of the user pressing the shooting button, the electronic device reconfigures the camera preview data stream to complete the output of the post-out frame data required by the night scene image fusion algorithm. In this process, the filtering processing mode adopted for the anti-shake information is set, such as the band-pass filtering mode GYRO_FILTER_BANDPASS_MODE or the low-pass filtering mode GYRO_FILTER_LOWPASS_MODE or other modes.

[0256] It should be noted that for the 1-2 frame time required for mode switching, the previous image information can be used for freeze display, which can avoid the echo of data in the switching process to the preview interface.

[0257] At the same time, the automatic exposure algorithm combines the exposure time required in the shooting stage and issues it to the image sensor, preparing to perform exposure processing on the post-out frame image.

[0258] In some embodiments, the image sequence output in the shooting stage can include a long frame image (referred to as a long frame L) with a long exposure time. The purpose of outputting the long frame L is to obtain more night scene exposure information.

[0259] Optionally, the image sequence output in the shooting stage can also include a normal exposure frame image (referred to as a normal exposure frame N) and a short frame image (referred to as a short frame S) with a short exposure time.

[0260] The purpose of outputting the short frame S is to provide better highlight suppression effect for the highlight area in a city night scene shooting scenario.

[0261] The purpose of outputting the N frame is to cooperate with the ZSL N frame output in the preview stage for multi-frame noise reduction fusion processing.

[0262] It should be noted that the frame number and exposure sequence of the long frame L, the short frame S and the normal exposure frame N can be determined according to actual use requirements, and the present application does not limit the comparison.

[0263] It should be further noted that, for the sake of convenience, the image frame output in the preview stage is referred to as a normal exposure frame. The exposure time of the long frame N is greater than that of the normal exposure frame N. The exposure time of the short frame S is less than that of the normal exposure frame N.

[0264] In the shooting stage, the optical image stabilization mode switching is completed first (i.e., the setting of the filtering mode and the setting of the exposure parameter are completed), then the images are captured according to the set exposure parameter and the image sequence is output, and the optical image stabilization mode set is used for anti-shake processing during the image exposure process, so that the image blur caused by the shaking of the mobile phone during the exposure time can be avoided. In this way, it can be ensured that the image sequence can be output according to the determined optical image stabilization mode after the shooting is triggered in the night scene mode.

[0265] S304, generating a target image according to the image sequence in the preview stage and / or the image sequence in the shooting stage.

[0266] In some embodiments, the image sequence output in the preview stage can be N, N, N, N, N.

[0267] In some embodiments, the image sequence output in the shooting stage in a non-high dynamic scene can be N, L.

[0268] In some embodiments, the image sequence output in the shooting stage in a high dynamic scene can be S, N, S, L.

[0269] It should be noted that the above image sequence is exemplary, and in actual implementation, the number of frames can not be limited to the above scheme, and can also be other numbers, and the combination of frames can also not be limited to the above scheme, and can also be other combinations of long and short frames.

[0270] In the embodiments of the present application, the image sequence in the preview stage and / or the image sequence in the shooting stage is subjected to image registration and fusion according to the optical image stabilization mode in the shooting stage, to generate a night scene photo (target image).

[0271] In some embodiments, a key frame (referred to as a selected frame) is selected from the image sequence output in the preview stage (front frame image) and the image sequence output in the shooting stage (rear frame image), and a night scene photo (i.e., a target image) is obtained through a night scene image registration and fusion algorithm.

[0272] In the embodiments of the present application, the key frame can be selected according to the image definition in the image sequence and the time interval from the time when the user presses the shutter button. For example, the clearer the image frame, the higher the corresponding score; the closer the image frame to the time when the shutter button is pressed, the higher the corresponding score.

[0273] In some embodiments, for each image in the image sequence, the scores corresponding to the image definition and the time interval from the time when the shutter button is pressed are weighted and summed, and the normal exposure frame N with the highest score in the image sequence is determined as the key frame.

[0274] In some embodiments, if the clarity of the normal exposure frame N output in the preview stage is lower than that of the normal exposure frame N output after the anti-shake processing in the shooting stage due to jitter, the key frame can be replaced from the normal exposure frame N output in the preview stage to the normal exposure frame N output after the anti-shake processing in the shooting stage to improve the clarity of the key frame of the night scene photo.

[0275] In the embodiments of the present application, after the key frame is determined, the image registration is performed between the key frame and other image frames in the image sequence except the key frame based on the key frame, and after the image registration is completed, the fusion is performed on each image in the image sequence to obtain the night scene photo.

[0276] Figure 10 A flowchart for generating a night scene photo according to the image sequence output in the preview stage and the image sequence output in the shooting stage in a non-high dynamic scene is shown.

[0277] As shown in Figure 10 , the image sequence output in the preview stage is N, N, N, N, N, the electronic device acquires scene perception information (the night scene mode can be enabled when the scene perception information meets the night scene mode triggering condition), scene brightness information, dynamic range information, etc. according to the image sequence output in the preview stage, and then determines that the current scene is a night scene and a non-high dynamic scene according to these information. When the user triggers the shooting, the preview stage is switched to the shooting stage. The electronic device switches to the optical anti-shake working mode of the shooting stage in response to the user operation, that is, the filtering mode is switched from the band-pass filtering mode to the low-pass filtering mode, and the exposure parameters such as the exposure time and the sensitivity OIS are adjusted according to the jitter intensity, the image is collected and the image sequence is output according to the set exposure parameters, for example, the image sequence output in the shooting stage is N, L, to ensure that the long exposure image sequence can be output according to the determined optical anti-shake working mode after the shooting is triggered in the night scene and the non-high dynamic scene. Then, the night scene shooting decision module performs key frame selection, multi-frame image registration, multi-frame noise reduction fusion, etc. based on the image sequence N, N, N, N, N output in the preview stage and the long exposure image sequence N, L output in the shooting stage to obtain the night scene photo.

[0278] As shown in Figure 10 , after the shooting is completed, the shooting stage is switched to the preview stage, and the optical anti-shake working mode of the preview stage is switched back: the filtering mode is switched from the low-pass filtering mode to the band-pass filtering mode, and the exposure parameters such as the exposure time and the sensitivity OIS are restored to the baseline exposure parameters, the image is collected and the image sequence is output according to the baseline exposure parameters, that is, the preview stage continues to output the image sequence of the normal exposure frame N.

[0279] Figure 11A flowchart for generating a night scene photo according to an image sequence output in a preview stage and an image sequence output in a shooting stage in a high dynamic scene is shown.

[0280] As shown in Figure 11 , the image sequence output in the preview stage is N, N, N, N, N, the electronic device acquires scene perception information, scene brightness information, dynamic range information, etc. according to the image sequence output in the preview stage, and then determines that the current scene is a night scene and a high dynamic scene according to these information. When the user triggers the shooting, the preview stage is switched to the shooting stage. The electronic device switches to the optical image stabilization working mode in the shooting stage in response to the user operation, that is, switches the filtering mode from the band-pass filtering mode to the low-pass filtering mode, and adjusts the exposure time and the OIS exposure parameter such as the sensitivity according to the shaking intensity. The image is collected according to the set exposure parameter and the image sequence is output, for example, the image sequence output in the shooting stage is S, N, S, L, which ensures that the variable exposure image sequence can be output according to the determined optical image stabilization working mode after the shooting is triggered in the night scene and the high dynamic scene. Then, the night scene shooting decision module performs key frame selection, multi-frame image registration, multi-frame noise reduction fusion, and HDR fusion based on the image sequence N, N, N, N, N output in the preview stage and the variable exposure image sequence S, N, S, L output in the shooting stage, and obtains a night scene photo.

[0281] The image anti-shake control method provided by the embodiments of the present application optimizes the OIS algorithm part, determines the optical image stabilization working mode in the shooting stage according to the scene perception information, the shaking information and the face detection information, adjusts the shaking information filtering mode, adjusts the image frame output mode, adjusts the exposure parameter, and completes the frame selection, image registration and fusion processing in the optical image stabilization working mode in the shooting stage, and finally obtains a high-quality night scene photo.

[0282] Figure 12 A flowchart of the image anti-shake control method provided by the embodiments of the present application is shown. As shown in Figure 12 , the image anti-shake control method includes the following steps S401-S412.

[0283] S401, in the preview stage, anti-shake control is performed according to the shaking information after band-pass filtering processing, and images are collected using the reference exposure parameter during the anti-shake control, and an image sequence is output.

[0284] S402, scene recognition is performed according to the image sequence output in the preview stage.

[0285] In some embodiments, the scene recognition result can include scene brightness information, which is used to represent the illumination intensity of the current scene.

[0286] For example, if the luminance indicated by the scene luminance information is within a first luminance range, it indicates that the current scene is an extremely low-illumination scene.

[0287] For example, if the luminance indicated by the scene luminance information is within a second luminance range, it indicates that the current scene is a low-illumination scene.

[0288] For example, if the luminance indicated by the scene luminance information is within a third luminance range, it indicates that the current scene is a medium-illumination scene.

[0289] For example, if the luminance indicated by the scene luminance information is within a fourth luminance range, it indicates that the current scene is a high-illumination scene.

[0290] It should be noted that, in this embodiment, the scenes are divided into extremely low-illumination scenes, low-illumination scenes, medium-illumination scenes, and high-illumination scenes according to the illumination size. In actual implementation, other possible divisions of the scenes can also be performed according to actual requirements, and the embodiments of the present application do not limit this.

[0291] In some embodiments, the scene luminance information can be represented by an illumination value (in Lux). The larger the illumination value, the greater the illumination. The smaller the illumination value, the smaller the illumination.

[0292] In some other embodiments, the scene luminance information can be represented by an index value (denoted as Lux_Idx) corresponding to the illumination. The larger the index value Lux_Idx, the smaller the illumination. The smaller the index value Lux_Idx, the greater the illumination.

[0293] In some embodiments, the scene recognition result can include dynamic range information, which is used to indicate the dynamic range of the current scene.

[0294] The electronic device can determine, according to the dynamic range information, whether the current scene is a high-dynamic scene or a non-high-dynamic scene.

[0295] For example, if the dynamic range information indicates that the dynamic range of the current scene is within a first dynamic range, i.e., the image does not contain obvious light and dark areas, the electronic device determines that the current scene is a non-high-dynamic scene.

[0296] For example, if the dynamic range information indicates that the dynamic range of the current scene exceeds the first dynamic range, i.e., the image contains obvious light and dark areas, the electronic device determines that the current scene is a high-dynamic scene.

[0297] S403, determine whether to enable the night scene mode.

[0298] In some embodiments, the electronic device enables the night scene mode when the electronic device receives a user trigger to start the camera night scene mode.

[0299] In some embodiments, in a case where the scene recognition result meets the night scene mode triggering condition, the electronic device determines to automatically enable the night scene mode when shooting.

[0300] For example, in a case where it is determined according to the scene brightness information that the current scene is an extremely low-illumination scene or a low-illumination scene or a medium-illumination scene, i.e., the scene recognition result meets the night scene mode triggering condition, the electronic device determines to automatically enable the night scene mode when shooting.

[0301] In a case where it is determined that the night scene mode is not enabled, the anti-shake control is completed according to the default anti-shake control strategy of the camera. For example, in the preview stage and the shooting stage, the band-pass filtering method is used to perform band-pass filtering processing on the jitter information, and then the anti-shake control is performed according to the jitter information after the band-pass filtering processing.

[0302] S404, in a case where it is determined that the night scene mode is enabled, the scene illumination level is determined according to the scene brightness information, and the anti-shake control strategy corresponding to the scene illumination level is determined.

[0303] Here, the scene brightness information is an index value Lux_Idx, and three illumination levels are taken as an example for illustrative description. The electronic device compares the index value Lux_Idx with the index threshold value to determine the scene illumination level. It should be noted that the larger the index value Lux_Idx, the smaller the illumination; the smaller the index value Lux_Idx, the larger the illumination.

[0304] When Lux_Idx≥Thd_w2, the electronic device determines that the scene illumination level is the first level, i.e., an extremely low-illumination scene.

[0305] When Thd_w2

[0306] When Lux_Idx

[0307] Wherein, Thd_w2 is greater than Thd_w1. The larger the index threshold value, the smaller the illumination; the smaller the index threshold value, the larger the illumination.

[0308] For example, three illumination levels (the first level, the second level and the third level) and the corresponding image anti-shake control strategies of each level are exemplarily given below:

[0309] The first illumination level - low-illumination scene: the image anti-shake control strategy in the shooting stage: after triggering the shooting, the low-pass filtering is used to filter the jitter information, and the non-ZSL mode output frame is switched, and the image fusion is performed based on the image sequence output by the non-ZSL mode (referred to as non-ZSL mode output frame).

[0310] The second illumination level - low-illumination scene: the image anti-shake control strategy in the shooting stage: after triggering the shooting, the low-pass filtering is used to filter the jitter information, and the non-ZSL mode output frame is switched, and the image fusion is performed based on the image sequence output by the non-ZSL mode (referred to as non-ZSL mode output frame).

[0311] The third illumination level - medium-illumination scene: the image anti-shake control strategy in the shooting stage: after triggering the shooting, the band-pass filtering is still used to filter the jitter information, and the ZSL mode output frame is used for image fusion.

[0312] S405, determine the jitter intensity according to the jitter information, and determine the first exposure compensation corresponding to the jitter intensity.

[0313] Wherein, the jitter information is the jitter information after the band-pass filtering processing, which is represented as Gyro_Info.

[0314] Exemplarily, the jitter information can be an angle value. Wherein, the larger the angle value, the stronger the jitter; the smaller the angle value, the weaker the jitter.

[0315] In the embodiment of the application, the first exposure compensation includes an exposure time adjustment and an ISO value adjustment.

[0316] Figure 13 The schematic diagram of the exposure compensation corresponding to the various jitter levels provided by the embodiment of the application. The following will be described in combination with Figure 13 Exemplarily, the various jitter levels and the first exposure compensation corresponding to each jitter level provided by the embodiment of the application are described.

[0317] The first jitter level - tripod-assisted shooting condition:

[0318] In some embodiments, when Gyro_Info≈0, it is determined that the current jitter amplitude / intensity is basically zero, which belongs to the tripod shooting condition. In this case, the exposure compensation can be set according to the preset parameters corresponding to the tripod condition in the shooting stage.

[0319] For example, when it is judged to belong to the tripod shooting condition, the exposure time can be increased by 2 or more than 2, and the ISO value of the sensitivity can be reduced by 2 or more than 2.

[0320] For example, when it is determined that the shooting condition belongs to the tripod shooting condition, the mode switching can be selected not to be performed, i.e., the ZSL mode is used for the preview stage and the shooting stage, and the non-ZSL mode is not switched to for the shooting stage.

[0321] Second shake level - small shake condition:

[0322] In some embodiments, when Gyro_Info≤Thd_gyro_shake_w1, it is determined that the current shake amplitude / intensity belongs to the small shake condition. In this case, the exposure time adjustment amount can be set according to the preset parameters corresponding to the small shake condition in the shooting stage.

[0323] For example, the exposure time is not reduced and the ISO value is kept unchanged. That is, the exposure time adjustment amount and the ISO value adjustment amount are both 0.

[0324] In other embodiments, the exposure time is increased by 1-2 stops based on the reference exposure time, and the ISO value is reduced by 1-2 stops based on the reference ISO value.

[0325] Wherein, Thd_gyro_shake_w1 is greater than 0.

[0326] Third shake level - medium shake condition:

[0327] In some embodiments, when Thd_gyro_shake_w1≤Gyro_Info<Thd_gyro_shake_w2, it is determined that the current shake amplitude / intensity belongs to the medium shake condition. In this case, the exposure compensation amount can be set according to the preset parameters corresponding to the medium shake condition in the shooting stage. For example, the exposure time is reduced by 1 stop, and the ISO value is increased by 1 stop.

[0328] Wherein, Thd_gyro_shake_w2 is greater than Thd_gyro_shake_w1.

[0329] Fourth shake level - large shake condition:

[0330] In some embodiments, when Gyro_Info≥Thd_gyro_shake_w2, it is determined that the current shake amplitude / intensity belongs to the large shake condition. In this case, the exposure compensation amount can be set according to the preset parameters corresponding to the large shake condition in the shooting stage. For example, the exposure time is reduced by 2 stops, and the ISO value is increased by 2 stops.

[0331] S406, determining a second exposure compensation amount according to the face detection information.

[0332] In some embodiments, the current scene is determined as a portrait night scene according to the face detection information, in which case, in the shooting stage, the exposure parameter is adjusted not only according to the shaking intensity, but also based on the portrait night scene.

[0333] For example, in some embodiments, when the face detection information (such as is_face_detect = 1) meets the preset face detection condition, the current scene is determined as a portrait night scene. In this case, in the shooting stage, on the basis of the exposure compensation / adjustment based on the shaking condition correction, the exposure time is further reduced by 0.5 to 1 stop, and the ISO value of the sensitivity is increased by 0.5 to 1 stop, so as to better capture the night scene portrait image and improve the image yield.

[0334] In some embodiments, the current scene is determined as a night scene without a portrait according to the face detection information, in which case, the step of adjusting the exposure parameter based on the portrait night scene is not performed, i.e., the second exposure compensation amount is 0.

[0335] It should be noted that the execution order of S405 and S406 in the embodiments of the present application is not limited, for example, S405 can be executed first, and then S406 can be executed; or S406 can be executed first, and then S405 can be executed; or S405 and S406 can be executed simultaneously.

[0336] S407, whether the electronic device receives a user operation of triggering photographing.

[0337] For example, the electronic device detects that the user clicks the photographing button, i.e., the electronic device receives the user operation of triggering photographing.

[0338] In the case where the electronic device receives the user operation of triggering photographing, the electronic device switches from the preview stage to the shooting stage.

[0339] S408, in the case where the electronic device receives the user operation of triggering photographing, the first filtering mode is used to filter the shaking information, and the anti-shaking control is performed according to the filtered shaking information.

[0340] In the case where the electronic device receives the user operation of triggering photographing, the first filtering mode is determined by the step of S404.

[0341] For example, in the first illumination level (extremely low illumination scene), the first filtering mode can be a low-pass filtering mode.

[0342] For example, in the second illumination level (low illumination scene), the first filtering mode can be a low-pass filtering mode.

[0343] For example, in the third illumination level (medium illumination scene), the first filtering mode can be a band-pass filtering mode.

[0344] S409, in a case where the electronic device receives an operation of triggering photographing by a user, switching to a non-ZSL mode frame output or not switching, adjusting an exposure parameter based on the first exposure compensation amount and the second exposure compensation amount, and acquiring an image sequence of a photographing stage by using the adjusted exposure parameter.

[0345] In some embodiments, in an extremely low-illumination scene or a low-illumination scene, when the electronic device receives an operation of triggering photographing by a user, the camera is controlled to switch to a non-ZSL mode.

[0346] In some other embodiments, in a medium-illumination scene, when the electronic device receives an operation of triggering photographing by a user, the camera is still controlled to use a ZSL mode, without mode switching.

[0347] It should be noted that the electronic device simultaneously performs S408 and S409. It can be understood that, while performing the anti-shake control, the image is collected by using the adjusted exposure parameter, so as to achieve the image anti-shake purpose.

[0348] In the embodiments of the present application, in a case where the exposure parameter is set based on the jitter intensity and the image is exposed according to the exposure parameter, the jitter information is processed by using the set low-pass filtering mode and the anti-shake control processing is performed according to the filtered jitter information, so that the image blur caused by the jitter of the mobile phone during the exposure time length can be avoided.

[0349] S410, performing image registration and image fusion according to the image sequence of the preview stage and / or the image sequence of the photographing stage, to obtain a night scene photo.

[0350] In some embodiments, in an extremely low-illumination scene, the image registration and image fusion are performed according to the image sequence output in the ZSL mode in the photographing stage, to obtain a night scene photo.

[0351] In some embodiments, in a low-illumination scene, the preview stage uses the ZSL mode for frame output, and the photographing stage is switched to use the non-ZSL mode for frame output. Correspondingly, the image registration and image fusion can be performed according to the image sequence output in the ZSL mode in the preview stage and the image sequence output in the non-ZSL mode in the photographing stage, to obtain a night scene photo.

[0352] In some embodiments, in a medium-illumination scene, the preview stage and the photographing stage both use the ZSL mode for frame output. Correspondingly, the image registration and image fusion can be performed according to the image sequence output in the ZSL mode in the preview stage and the image sequence output in the ZSL mode in the photographing stage, to obtain a night scene photo.

[0353] In the embodiments of the present application, in the case that the image sequence in the preview stage and the image sequence in the shooting stage are acquired, frame selection, multi-frame image registration, multi-frame noise reduction fusion, HDR fusion, and intelligent AI noise reduction can be performed based on the image sequence in the preview stage and the image sequence in the shooting stage, and a night scene photo is obtained.

[0354] S411, determining whether the photographing is ended.

[0355] For example, in the case that the electronic device obtains a night scene photo, it is determined that the photographing is ended.

[0356] For another example, after the image sequence in the shooting stage is acquired by using the adjusted exposure parameter, it is determined that the photographing is ended.

[0357] S412, in the case that the photographing is determined to be ended, switching from the shooting stage to the preview stage, switching to the ZSL mode for frame output, restoring the exposure parameter to the reference exposure parameter, and switching the filtering mode of the shake information to the band-pass filtering.

[0358] After S412, return to continue to execute S401 described above.

[0359] Through the image anti-shake control method provided in the embodiments of the present application, in the shooting scene of the camera night scene mode, the first anti-shake control parameter suitable for the current scene can be determined based on the scene brightness information, such as the shake information filtering mode, the image frame output mode, and / or the image fusion mode, and the second anti-shake control parameter suitable for the current scene can be determined based on the shake information, such as the second anti-shake control parameter including the exposure compensation amount, and different shake information corresponds to different exposure compensation amounts. The present application can perform comprehensive anti-shake control and image processing based on the first anti-shake control parameter and the second anti-shake control parameter in the shooting stage, and different degrees of exposure compensation are performed in the shooting stage for different shake intensity night scene shooting scenes, which improves the image anti-shake effect of the night scene mode shooting scene and improves the clarity of the night scene photo.

[0360] The implementation process of the image anti-shake control method provided in the embodiments of the present application is described above, and the specific implementation mode of the image anti-shake control method provided in the embodiments of the present application in different cases will be described below with reference to the accompanying drawings.

[0361] It should be noted that in the medium illumination scene, it can be a non-high dynamic scene or a high dynamic scene, so the image anti-shake control in the medium illumination, non-high dynamic scene in the night scene mode and the image anti-shake control in the medium illumination, high dynamic scene in the night scene mode will be described in combination with case one and case two.

[0362] Case one: image anti-shake control in the medium illumination, non-high dynamic scene in the night scene mode

[0363] Figure 14 The specific implementation of the image anti-shake control in the medium-illumination, non-high-dynamic scene in the night scene mode is shown. After the camera application program is started and enters the preview stage, the camera application program switches from the preview stage to the shooting stage when detecting that the user triggers photographing. After the photographing is completed, the camera application program switches from the shooting stage to the preview stage.

[0364] As shown in the Figure 14 , in the medium-illumination, non-high-dynamic scene in the night scene mode, the image anti-shake control strategy adopted by the electronic device includes the following four aspects:

[0365] 1) Filter mode: the band-pass filter mode is used to perform band-pass filtering on the jitter information in the preview stage and the shooting stage, and the anti-shake control is performed based on the anti-shake information after the band-pass filtering.

[0366] 2) Frame output mode: the ZSL mode is used to output frames in the preview stage and the shooting stage.

[0367] 3) Exposure parameter: the reference exposure parameter is used to collect images in the preview stage, and a reference exposure image sequence is obtained. The exposure parameter is adjusted according to the jitter information in the shooting stage, and the adjusted exposure parameter is used to collect images, and a compensation exposure image sequence is obtained.

[0368] 4) Image fusion: image registration and fusion are performed based on the reference exposure image sequence in the preview stage and the compensation exposure image sequence in the shooting stage, and a night scene photo is obtained.

[0369] Case two: image anti-shake control in the medium-illumination, high-dynamic scene in the night scene mode

[0370] Figure 15 The specific implementation of the image anti-shake control in the medium-illumination, high-dynamic scene in the night scene mode is shown. As shown in the Figure 15 , in the medium-illumination, high-dynamic scene in the night scene mode, the image anti-shake control strategy adopted by the electronic device includes the following four aspects:

[0371] 1) Filter mode: the band-pass filter mode is used to perform band-pass filtering on the jitter information in the preview stage and the shooting stage, and the anti-shake control is performed based on the anti-shake information after the band-pass filtering.

[0372] 2) Frame output mode: the ZSL mode is used to output frames in the preview stage and the shooting stage.

[0373] 3) Exposure parameter: the reference exposure parameter is used to collect images in the preview stage, and a reference exposure image sequence is obtained. The exposure parameter is adjusted according to the jitter information in the shooting stage, and the adjusted exposure parameter is used to collect images, and a compensation exposure image sequence is obtained. The compensation exposure image sequence is a variable exposure image sequence.

[0374] It should be noted that the variable exposure image sequence can include long exposure images and short exposure images. Alternatively, the variable exposure image sequence includes long exposure images, short exposure images, and reference exposure images.

[0375] 4) Image fusion: image registration and fusion based on the reference exposure image sequence in the preview stage and the compensation exposure image sequence in the shooting stage to obtain a night scene photo.

[0376] Case two and case one have the same image anti-shake control strategy, which is that when switching from the preview stage to the shooting stage, the filtering mode is not switched and the frame output mode is not switched.

[0377] Case two and case one have the same image anti-shake control strategy, which is that when switching from the preview stage to the shooting stage, the filtering mode is not switched and the frame output mode is not switched.

[0378] That is, in case two, the shooting stage outputs a variable exposure image sequence after exposure compensation; while in case one, the shooting stage outputs a compensation exposure image sequence, but not a variable exposure image sequence.

[0379] Case three: image anti-shake control in a low-illumination scene in a night scene mode

[0380] Figure 16 The specific implementation of the image anti-shake control in a low-illumination scene in a night scene mode is shown. As shown in Figure 16 In a low-illumination scene in a night scene mode, the image anti-shake control strategy of the electronic device includes the following four aspects:

[0381] 1) Filtering mode: in the preview stage, band-pass filtering is used to filter the anti-shake information, and anti-shake control is performed based on the band-pass filtered anti-shake information. In the shooting stage, low-pass filtering is switched, and anti-shake control is performed based on the low-pass filtered anti-shake information.

[0382] 2) Frame output mode: ZSL mode is used for frame output in the preview stage. In the shooting stage, non-ZSL mode is switched for frame output.

[0383] 3) Exposure parameter: in the preview stage, reference exposure parameters are used to capture images to obtain a reference exposure image sequence. In the shooting stage, exposure compensation is performed according to the anti-shake information, that is, the exposure parameters are adjusted based on the anti-shake strength, and the adjusted exposure parameters are used to capture images to obtain a compensation exposure image sequence.

[0384] 4) Image fusion: image registration and fusion based on the reference exposure image sequence in the preview stage and the compensation exposure image sequence in the shooting stage, to obtain the night scene photo.

[0385] Compared with case one and case two, the difference and the same points of the image anti-shake control strategy used in case three include:

[0386] In case three, the filtering mode is switched and the frame output mode is switched when switching from the preview stage to the shooting stage. In case one and case two, neither the filtering mode nor the frame output mode is switched.

[0387] In terms of exposure parameters and image fusion, case three is basically the same as case one and case two, that is, the reference exposure parameters are used to collect images in the preview stage, and exposure compensation is performed according to the shake information in the shooting stage, that is, the exposure parameters are adjusted based on the shake strength, and the images are collected using the adjusted exposure parameters, and image registration and fusion are performed based on the reference exposure image sequence in the preview stage and the compensation exposure image sequence in the shooting stage.

[0388] Case four: image anti-shake control in a very low-illuminance scene in night mode

[0389] Figure 17 The specific implementation of the image anti-shake control in a very low-illuminance scene in night mode provided by the embodiments of the present application is shown. As shown in Figure 17 In a very low-illuminance scene in night mode, the image anti-shake control strategy used by the electronic device includes the following four aspects:

[0390] 1) Filtering mode: in the preview stage, band-pass filtering is used to filter the shake information, and anti-shake control is performed based on the band-pass filtered anti-shake information. In the shooting stage, low-pass filtering is switched, and anti-shake control is performed based on the low-pass filtered anti-shake information.

[0391] 2) Frame output mode: ZSL mode is used for frame output in the preview stage. In the shooting stage, non-ZSL mode is switched for frame output.

[0392] 3) Exposure parameters: in the preview stage, reference exposure parameters are used to collect images to obtain a reference exposure image sequence. In the shooting stage, exposure compensation is performed according to the shake information, that is, the exposure parameters are adjusted based on the shake strength, and the images are collected using the adjusted exposure parameters to obtain a compensation exposure image sequence.

[0393] 4) Image fusion: image registration and fusion based on the compensation exposure image sequence in the shooting stage, to obtain the night scene photo.

[0394] Compared with case three, the difference and the same points of the image anti-shake control strategy used in case four include:

[0395] The fourth case and the third case adopt the same image anti-shake control strategy in terms of filtering mode, frame output mode and exposure parameter, that is, when switching from the preview stage to the shooting stage, switching from band-pass filtering to low-pass filtering, switching from ZSL mode frame output to non-ZSL mode frame output, and adjusting the exposure parameter on the basis of the reference exposure parameter.

[0396] In terms of image fusion, the fourth case is different from the third case. In the third case, image registration and fusion are performed based on the reference exposure image sequence in the preview stage and the compensation exposure image sequence in the shooting stage. In the fourth case, image registration and fusion are performed based on the compensation exposure image sequence in the shooting stage.

[0397] By the scheme provided in the present application, in the process of photographing by using the camera night scene mode of the electronic device, due to insufficient scene illumination in the night scene, it is easy to cause photo blur due to hand jitter. In this case, the image anti-shake control method provided in the embodiments of the present application can avoid or alleviate the problem of night scene photo blur caused by jitter, and can effectively prolong the shutter time, so as to achieve the purpose of obtaining more exposure information, reducing night scene photo noise and improving clarity.

[0398] It should be noted that in the embodiments of the present application, "greater than" can be replaced by "greater than or equal to", "less than or equal to" can be replaced by "less than", or "greater than or equal to" can be replaced by "greater than", and "less than" can be replaced by "less than or equal to".

[0399] Each of the embodiments described in the present application can be an independent scheme, or can be combined according to the inherent logic, and these schemes all fall within the protection scope of the present application.

[0400] The above mainly describes the scheme provided by the embodiments of the present application from the perspective of method steps. It can be understood that, in order to realize the above functions, the electronic device implementing the method contains the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed in the present application, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is realized in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical scheme. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the protection scope of the present application.

[0401] The embodiments of the present application can divide the function modules of the electronic device according to the above method examples. For example, each function module can be divided according to each function, or two or more functions can be integrated in one processing module. The integrated module can be realized in the form of hardware or in the form of a software function module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical function division. Other feasible division manners can be used in actual implementation. The following will be described by taking the division of each function module according to each function as an example.

[0402] Figure 18 A schematic block diagram of an image anti-shake control apparatus 500 provided by the embodiments of the present application is shown. The apparatus 500 can be used to perform the actions performed by the electronic device in the above method embodiments. The apparatus 500 includes a camera unit 510, a processing unit 520, and an image anti-shake control unit 530.

[0403] The camera unit 510 is configured to start a camera application program; and enable a camera night scene mode.

[0404] The processing unit 520 is configured to, in a case where the camera night scene mode is enabled, acquire first scene brightness information and first shake information; determine a first anti-shake control parameter according to the first scene brightness information, the first anti-shake control parameter including a shake information filtering manner, an image frame output mode, and / or an image fusion manner, different scene brightness information corresponding to different anti-shake control parameters; and determine a second anti-shake control parameter according to the first shake information, the second anti-shake control parameter including a first exposure compensation amount, different shake information corresponding to different exposure compensation amounts.

[0405] The camera unit 510 is further configured to receive a first operation triggered by a user to take a picture, and perform image acquisition and frame output based on the first anti-shake control parameter and the second anti-shake control parameter in response to the first operation.

[0406] The image anti-shake control unit 530 is configured to perform anti-shake control based on the first anti-shake control parameter and the second anti-shake control parameter in response to the first operation.

[0407] The processing unit 520 is further configured to perform image processing based on the first anti-shake control parameter and the second anti-shake control parameter in response to the first operation, to obtain a target image.

[0408] Through the image anti-shake control device provided in the embodiments of the present application, in the shooting scene of the camera night scene mode, the first anti-shake control parameter suitable for the current scene can be determined based on the scene brightness information, such as the shaking information filtering mode, the image out-frame mode and / or the image fusion mode, and the second anti-shake control parameter suitable for the current scene can be determined based on the shaking information, such as the second anti-shake control parameter including the exposure compensation amount, different shaking information corresponding to different exposure compensation amounts. The present application can perform comprehensive anti-shake control and image processing based on the first anti-shake control parameter and the second anti-shake control parameter in the shooting stage, and for different shaking intensity of the night scene shooting scene, different degrees of exposure compensation are performed in the shooting stage, which improves the image anti-shake effect of the night scene shooting scene and improves the clarity of the night scene photo.

[0409] The apparatus 500 according to the embodiments of the present application can correspond to performing the methods described in the embodiments of the present application, and the above and other operations and / or functions of the units in the apparatus 500 are respectively for realizing the corresponding flows of the methods, and for brevity, will not be described here.

[0410] The present application also provides a chip, which is coupled with a memory, and the chip is used to read and execute a computer program or instructions stored in the memory to perform the method in each of the above embodiments.

[0411] The present application also provides an electronic device, which includes a chip, and the chip is used to read and execute a computer program or instructions stored in the memory, so that the method in each of the embodiments is performed.

[0412] The present application also provides a computer readable storage medium, which stores computer instructions, when the computer instructions run on an electronic device, the electronic device performs the above related method steps to realize the image anti-shake control method in the above embodiments.

[0413] The present application also provides a computer program product, which stores program codes, when the computer program product runs on a computer, the computer executes the above related steps to realize the image anti-shake control method in the above embodiments.

[0414] Among them, the electronic device, the computer readable storage medium, the computer program product or the chip provided by the present application are all used to execute the corresponding method provided above, so the beneficial effects they can achieve can refer to the beneficial effects in the corresponding method provided above, which will not be described here.

[0415] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the division of the apparatus embodiments is only a logical function division, and there can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different apparatuses can be indirect couplings or communication connections through some interfaces, apparatuses or units, and can be in electrical, mechanical or other forms.

[0416] The term "and / or" used in this document is a description of an association relationship between associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The symbol " / " in this document represents an or relationship of associated objects, for example, A / B represents A or B.

[0417] The terms "first" and "second" and the like in the description and claims of this document are used to distinguish different objects, and are not used to describe a specific order. In the description of the embodiments of the present application, unless otherwise specified, "a plurality of" means two or more, for example, a plurality of processing units means two or more processing units, and the like.

[0418] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image anti-shake control method applied to an electronic device with a camera application, characterized in that, The method comprises: starting the camera application; enabling a camera night scene mode; obtaining first scene brightness information and first shake information in a preview stage when the camera night scene mode is enabled; determining first anti-shake control parameters according to the first scene brightness information, the first anti-shake control parameters comprising a shake information filtering mode, an image out-frame mode and / or an image fusion mode; different scene brightness information corresponds to different anti-shake control parameters; determining second anti-shake control parameters according to the first shake information, the second anti-shake control parameters comprising a first exposure compensation amount; different shake information corresponds to different exposure compensation amounts; receiving a first operation of a user triggering photographing; in response to the first operation, obtaining second shake information in a shooting stage, performing anti-shake control and image processing based on the first anti-shake control parameters and performing anti-shake control and image processing based on the second anti-shake control parameters to obtain a target image; the performing anti-shake control and image processing based on the second anti-shake control parameters comprises: adjusting a reference exposure parameter based on the first exposure compensation amount to obtain a first exposure parameter; and collecting images using the first exposure parameter to obtain an image sequence in the shooting stage; the performing anti-shake control and image processing based on the first anti-shake control parameters comprises: when the first scene brightness information corresponds to a first illumination level, filtering the second shake information using a low-pass filtering mode, switching an out-frame mode from a ZSL mode to a non-ZSL mode, and performing image fusion using the image sequence in the shooting stage; when the first scene brightness information corresponds to a second illumination level, filtering the second shake information using a low-pass filtering mode, and switching from the ZSL mode to the non-ZSL mode, and performing image fusion using the image sequence in the preview stage and the image sequence in the shooting stage; when the first scene brightness information corresponds to a third illumination level, filtering the second shake information using a band-pass filtering mode, using the ZSL mode to frame, and performing image fusion using the image sequence in the preview stage and the image sequence in the shooting stage; wherein the illumination of the first illumination level is less than the illumination of the second illumination level, and the illumination of the second illumination level is less than the illumination of the third illumination level.

2. The method of claim 1, wherein, The first exposure compensation amount comprises a compensation amount of an exposure time length and a compensation amount of an ISO value of a light sensitivity; different shake information corresponds to different shake levels, and different shake levels correspond to different exposure compensation amounts; wherein the adjusting the reference exposure parameter based on the first exposure compensation amount comprises: when the first shake information corresponds to a first shake level, increasing a first time length based on a reference exposure time length and decreasing a first ISO value based on a reference ISO value of a light sensitivity; when the first shake information corresponds to a second shake level, decreasing a second time length based on the reference exposure time length and increasing a second ISO value based on the reference ISO value of the light sensitivity; wherein the shake intensity of the first shake level is less than the shake intensity of the second shake level.

3. The method of claim 2, wherein, The first time length is increased based on the reference exposure time length and the first ISO value is decreased based on the reference ISO value when the first shake information corresponds to the first shake level. When the shake intensity indicated by the first shake information is equal to 0, the first shake information increases N-grade exposure time length based on the reference exposure time length and decreases N-grade ISO value based on the reference ISO value within the first shake level range. When the shake intensity indicated by the first shake information is greater than 0 and less than the first shake threshold, the first shake information increases M-grade exposure time length based on the reference exposure time length and decreases M-grade ISO value based on the reference ISO value within the first shake level range. Wherein, N is greater than M, and M is greater than or equal to 0.

4. The method of claim 2, wherein, The second time length is decreased based on the reference exposure time length and the second ISO value is increased based on the reference ISO value when the first shake information corresponds to the second shake level. When the shake intensity indicated by the first shake information is greater than or equal to the first shake threshold and less than the second shake threshold, the first shake information decreases S-grade exposure time length based on the reference exposure time length and increases S-grade ISO value based on the reference ISO value within the second shake level range. When the shake intensity indicated by the first shake information is greater than or equal to the second shake threshold, the first shake information decreases T-grade exposure time length based on the reference exposure time length and increases T-grade ISO value based on the reference ISO value within the second shake level range. Wherein, S is less than T, and S is greater than 0.

5. The method of claim 1, wherein, The first exposure parameter is obtained by adjusting the reference exposure parameter based on the first exposure compensation. The first adjusted exposure time length and the first adjusted ISO value are obtained by adjusting the reference exposure parameter based on the first exposure compensation. If the first adjusted exposure time length is greater than a preset limit exposure time length, the exposure time length is set to the limit exposure time length, and the second adjusted ISO value is obtained by adjusting the first adjusted ISO value by a first increment. The first exposure parameter includes the limit exposure time length and the second adjusted ISO value.

6. The method of claim 5, wherein, The first exposure parameter includes the limit exposure time length and the second adjusted ISO value. The first exposure parameter includes the limit exposure time length and the second adjusted ISO value. Different scene brightness information corresponds to different illumination levels, and different illumination levels correspond to different anti-shake control parameters. The method further comprises:

7. The method of claim 1, wherein, determining that the current scene is a high dynamic range scene according to the first scene brightness information; 8. The method according to any one of claims 1 to 7, characterized in that, setting the exposure parameter to a variable exposure parameter when receiving a first operation triggered by a user to take a photo. ​ ​ 9. The method of claim 8, wherein the varying exposure parameter comprises a first exposure length and a second exposure length; and wherein the image sequence of the shooting phase comprises a first exposure image and a second exposure image.

8. The method of claim 1, wherein the varying exposure parameter comprises a first exposure length, a second exposure length, and a reference exposure length; and wherein the image sequence of the shooting phase comprises a first exposure image, a second exposure image, and a reference exposure image.

7. The method of claim 1, wherein the first exposure length is greater than the reference exposure length, and the reference exposure length is greater than the second exposure length. The method further comprises:

10. The method according to any one of claims 1 to 7, characterized in that, obtaining face detection information in a case where the camera night scene mode is enabled; determining a third anti-shake control parameter according to the face detection information, different face detection information corresponding to different anti-shake control parameters; performing anti-shake control and image processing based on the third anti-shake control parameter in a case where the first operation of triggering photographing by a user is received. The third anti-shake control parameter comprises a second exposure compensation amount; and the performing anti-shake control and image processing based on the third anti-shake control parameter comprises:

11. The method of claim 10, wherein, adjusting a first exposure parameter based on the second exposure compensation amount to obtain a second exposure parameter; collecting an image by using the second exposure parameter to obtain an image sequence of a shooting phase. Different face detection information corresponds to different exposure compensation amounts; and the first exposure parameter comprises a first exposure length and a first ISO value.

12. The method of claim 11, wherein, The adjusting the first exposure parameter based on the second exposure compensation amount comprises: when the face detection information meets a portrait shooting condition, reducing an R-stop exposure length based on the first exposure length and increasing an R-stop light sensitivity value based on the first ISO value; wherein R is greater than zero; when the face detection information does not meet the portrait shooting condition, the second exposure compensation amount is zero. Before the receiving the first operation of triggering photographing by a user after the opening the camera application, the method further comprises:

13. The method according to any one of claims 1 to 7, characterized in that, performing anti-shake control and image processing based on a fourth anti-shake control parameter in a preview phase. The performing anti-shake control and image processing based on the fourth anti-shake control parameter comprises:

14. The method of claim 13, wherein, performing band-pass filtering on first jitter information obtained in the preview phase, and controlling a movement of an anti-shake device of the electronic device based on the first jitter information after the band-pass filtering; collecting an image by using a reference exposure parameter to obtain a reference exposure image; wherein the reference exposure parameter comprises a reference exposure length and a reference light sensitivity ISO value; outputting an image sequence of the preview phase by using a ZSL mode, the image sequence of the preview phase comprising one or more reference exposure images. The enabling the camera night scene mode comprises:

15. The method of any one of claims 1 to 7, wherein, obtaining an image sequence of a preview phase; performing scene recognition based on the image sequence of the preview phase; automatically enabling the camera night scene mode in a case where a scene recognition result meets a camera night scene mode triggering condition. The enabling the camera night scene mode comprises:

16. The method of any one of claims 1 to 7, wherein, enabling the camera night scene mode in response to a second operation of a user opening the camera night scene mode. ​ 17. The method according to any one of claims 1 to 7, characterized in that, The first jitter information is an offset angular velocity or an offset angle of the electronic device, and the first jitter information is acquired by a gyroscope sensor.

18. An electronic device, comprising: The electronic device includes one or more processors and a memory. The memory is coupled to the one or more processors, and the memory is configured to store computer program codes including computer instructions, and the one or more processors are configured to invoke the computer instructions to cause the electronic device to perform the method according to any one of claims 1 to 17.

19. A chip system, characterized by The chip system is applied to an electronic device, and the chip system includes one or more processors configured to invoke computer instructions to cause the electronic device to perform the method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium includes instructions configured to cause an electronic device to perform the method according to any one of claims 1 to 17 when the instructions are executed on the electronic device.

Images