Photographic image stabilization method, electronic device, and storage medium

Abstract

The present application relates to the field of image processing, and provides a photographic image stabilization method, an electronic device, and a storage medium. In a preview phase, whether the current scene is a specific-exposure photography scene is identified; for the specific-exposure photography scene (such as long-exposure or variable-exposure photography), since the impact of shake is significant, an image stabilization reset function is disabled when the start of photographing is triggered, avoiding image blurring caused by image stabilization reset; and full-band filtering processing is performed on shake data and the orientation of a camera lens is adjusted on the basis of the shake data, thereby achieving a full-band image stabilization effect. Image stabilization processing can be performed for high-frequency shake such as movement and touchscreen shake, and low-frequency shake such as breathing and heartbeat; upon triggering the start of photographing and waiting for a first duration, reset of all parameters is completed; when the state is stabilized, a camera sensor is controlled to output an image frame on the basis of adjusted exposure parameters and generate an image, so that the impact of various potential shakes in specific-exposure photography is greatly reduced, improving the sharpness of captured images.

Description

Shooting anti-shake method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 30, 2024, with application number 202410385922.2 and application name “Shooting anti-shake method, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image processing, and in particular to a shooting anti-shake method, electronic device and storage medium. Background Art

[0003] With the rapid development of imaging technology, users' demands for clarity and stability in video recording are constantly increasing. During the shooting process, jitter is inevitable. For example, jitter can be caused by hand-held or touch jitter, or by movement of the electronic device during the shooting process. Due to jitter during the shooting process, motion blur may appear in the captured video or image.

[0004] To this end, related technologies implement anti-shake by adding an optical image stabilization (OIS) controller to electronic devices. The principle of anti-shake of the OIS controller is to use an OIS motor to move the lens, thereby offsetting the displacement caused by shaking.

[0005] However, in certain specific exposure shooting scenes, the amount of jitter is relatively large, which reduces the anti-shake performance of the OIS controller and causes blurred images. Summary of the Invention

[0006] The present application provides a shooting anti-shake method, electronic device and storage medium, which greatly reduces the influence of various possible shakes on specific exposure shooting and improves the clarity of image shooting.

[0007] In a first aspect, the present application provides a shooting anti-shake method, the method comprising: starting a camera application; identifying in a preview stage that a current first scene satisfies a preset shooting condition; receiving a first operation triggered by a user to start shooting, and in response to the first operation, turning off an anti-shake reset function, and adjusting the orientation of the camera lens according to first jitter data; the first jitter data is data obtained after a first filtering process is performed on the jitter data of the electronic device during the shooting stage; waiting for a first time after triggering the start of shooting, controlling the camera sensor to output image frames according to adjusted exposure parameters, and generating a first image based on the output image frames; wherein the adjusted exposure parameters include a first exposure parameter, and the exposure duration corresponding to the first exposure parameter is greater than the exposure duration corresponding to the initial exposure parameter of the camera sensor; wherein the first filtering process is full-band filtering.

[0008] The shooting stabilization method provided in the embodiments of the present application identifies whether the current scene is a specific exposure shooting scene during the preview phase. For specific exposure shooting scenes (such as long exposure shooting or variable exposure), which are significantly affected by shaking, a series of stabilization processes are immediately executed after the shooting is triggered. The stabilization reset function is disabled to avoid image blur caused by the stabilization reset. Specific full-band filtering is performed on the shaking data, and the orientation of the camera lens is adjusted based on the processed shaking data to achieve a full-band stabilization effect. For example, stabilization can be performed for high-frequency shaking such as irregular shaking and touch screen shaking, as well as low-frequency shaking such as breathing and heartbeat. After the shooting is triggered, a first time period is waited to complete the parameter reset. After the state stabilizes, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters. This can avoid image blur caused by various possible shaking, thereby greatly reducing the impact of various possible shaking on specific exposure shooting and improving image clarity.

[0009] In some possible implementations, after the first image is generated based on the output image frame, the first image may be stored in a gallery, wherein the first image may be a picture or a video.

[0010] In some possible implementations, the preset shooting condition is a long exposure shooting condition; or the preset shooting condition is a variable exposure shooting condition.

[0011] In related technologies, the specific anti-shake processing method during the preview phase is as follows: During the preview phase, the jitter data is bandpass filtered, and the camera lens position is adjusted based on the processed jitter data. The preview image is captured while the anti-shake processing is performed. This can avoid image blur caused by high-frequency jitter such as irregular jitter and touch screen jitter. In addition, the anti-shake reset function is enabled by default. When the anti-shake effect is not satisfactory or the anti-shake function fails, the electronic device will activate the anti-shake reset function and reset the lens module to its initial position via the OIS motor, for example, placing the lens module in the center axis position to restore the anti-shake function.

[0012] In related technologies, the anti-shake processing method used during the capture phase is the same as that used during the preview phase. After the capture is triggered, the jitter data is bandpass filtered, and the camera lens position is adjusted based on the processed jitter data. Image capture is performed while the anti-shake processing is in place. This prevents image blur caused by high-frequency jitter, such as irregular jitter and touch screen jitter. Furthermore, the anti-shake reset function is enabled. When the anti-shake effect fails to meet requirements or the anti-shake function fails, the electronic device activates the anti-shake reset function to restore the anti-shake function.

[0013] It should be noted that when the shaking amplitude is large, the pushing distance of the optical image stabilization motor on the lens module may reach its limit, causing the optical image stabilization function to fail. In this case, the electronic device will use the anti-shake reset function.

[0014] Compared with related technologies, the OIS algorithm module of this application adopts different anti-shake strategies for preview mode and different shooting scenes.

[0015] First, in preview mode, the gyroscope sensor acquires jitter data, which is then bandpass filtered. Optical image stabilization is then performed based on the processed jitter data. Simultaneously, the image is captured using preset exposure parameters to generate a preview image, which is displayed on the preview interface. Furthermore, in preview mode, scene detection is performed to determine whether the current shooting scene meets the requirements for long exposure or variable exposure shooting. Different image stabilization strategies are then applied to different shooting scenarios.

[0016] On the one hand, the current shooting scene meets the conditions for long exposure / variable exposure shooting:

[0017] When the electronic device receives a capture command, it switches from preview mode to capture mode. If the current scene meets the requirements for long-exposure / variable-exposure shooting, the corresponding anti-shake strategy includes: disabling the anti-shake reset function, acquiring shake data, and low-pass filtering the shake data to achieve full-band anti-shake effects; then, adjusting the camera exposure parameters to the long-exposure / variable-exposure parameters and capturing images based on these adjusted exposure parameters. Due to the corresponding anti-shake processing, the captured long-exposure frames in this case are less affected by shake. After the long-exposure / variable-exposure shooting is completed, the device switches from capture mode to preview mode, reactivates the optical image stabilization function, and resets the exposure parameters.

[0018] On the other hand, the current shooting scene does not meet the conditions for long exposure / variable exposure shooting:

[0019] When the electronic device receives a capture command, it switches from preview mode to capture mode, maintaining the same exposure parameters. The current capture scene is a standard capture scene. The corresponding stabilization strategy includes: using the stabilization reset function as normal, acquiring jitter data, and performing bandpass filtering on the jitter data to achieve stabilization for high-frequency jitter; and capturing the image using the preset exposure parameters. The normally exposed image captured in this scenario achieves stabilization for some high-frequency jitter and exhibits good hand tracking. After the capture is complete, the device switches from capture mode back to preview mode.

[0020] In some embodiments, scenes such as low-light or night scenes meet the conditions for long-exposure shooting.

[0021] For example, if the electronic device detects that the preview image contains night scene features, the electronic device determines that the current scene meets the conditions for long exposure shooting. For example, if the electronic device detects that the ambient light brightness is lower than a preset brightness value, the electronic device determines that the current scene meets the conditions for long exposure shooting.

[0022] In some embodiments, low-light or backlit scenes, such as backlit shots, or scenes like sunrises, sunsets, and skyscapes, meet the requirements for variable exposure shooting. This is because such scenes contain both very bright and very dark areas (i.e., both overexposed and underexposed areas). Such scenes are suitable for HDR shooting mode, which achieves variable exposure shooting by fusing long and short exposure frames.

[0023] For example, if the electronic device detects that the preview image contains a distinct bright and dark area, the electronic device determines that the current scene meets the conditions for variable exposure shooting. For example, if the electronic device detects that the ambient light is distinct bright and dark, the electronic device determines that the current scene meets the conditions for variable exposure shooting.

[0024] In other embodiments, the electronic device may determine, based on a triggering operation by the user, whether to trigger and enable long exposure or variable exposure shooting, thereby determining whether the current scene meets the conditions for long exposure or variable exposure shooting.

[0025] For example, after a user taps the "Professional Mode" control, the electronic device can use long exposure or variable exposure shooting. For another example, when the electronic device receives a user trigger to enable "long exposure shooting," the electronic device will use long exposure shooting. For another example, when the electronic device receives a user trigger to enable "HDR shooting," the electronic device will use variable exposure shooting.

[0026] In some possible implementations, when the first scene meets the long exposure shooting condition, the adjusted exposure parameters include the first exposure parameters; when the first scene meets the variable exposure shooting condition, the adjusted exposure parameters also include a second exposure parameter, and the exposure duration corresponding to the second exposure parameter is shorter than the exposure duration corresponding to the initial exposure parameter of the camera sensor.

[0027] In some possible implementations, generating the first image based on the output image frames includes: when the first scene meets the long-exposure shooting condition, capturing long-exposure image frames using the first exposure parameters, and generating the first image based on the long-exposure image frames; when the first scene meets the variable-exposure shooting condition, capturing long-exposure image frames using the first exposure parameters, capturing short-exposure image frames using the second exposure parameters, and generating the first image based on the long-exposure image frames and the short-exposure image frames.

[0028] In some possible implementations, the method further includes: switching from a preview mode to a shooting mode in response to the first operation; and controlling an optical image stabilization motor of the electronic device to reset the camera lens to an initial position.

[0029] Because long-exposure or variable-exposure shooting is significantly affected by camera shake, it is necessary to disable the image stabilization reset function when triggering long-exposure or variable-exposure shooting to avoid image blur caused by image stabilization reset and large-scale lens movement. You can also reset the camera lens to its initial position to ensure that the image stabilization function can function normally after disabling the image stabilization reset function.

[0030] In some possible implementations, the first filtering process is a low-pass filtering process, and the first jitter data includes the attitude angle of the electronic device jitter. For example, an infinite impulse response (IIR) filter such as a Butterworth filter or a Chebyshev filter can be used. An IIR filter can implement the filtering process through recursive operations. The filter frequency band can be modified by modifying each order parameter.

[0031] In some possible implementations, after launching the camera application, the method further includes: adjusting the orientation of the camera lens of the electronic device based on the second jitter data during a preview phase; controlling the camera sensor to capture image frames according to the initial exposure parameters; and generating a preview image based on the captured image frames. The second jitter data is data obtained by performing a second filtering process on the jitter data of the electronic device during the preview phase. The filtering frequency band of the second filtering process is smaller than the filtering frequency band of the first filtering process.

[0032] In some possible implementations, the second filtering process is a bandpass filtering process. Bandpass filtering can be used to perform anti-shake processing on high-frequency jitters such as irregular jitter and touch screen jitter, with fast response speed and good hand tracking.

[0033] In some possible implementations, the method further includes: controlling the camera sensor to output image frames according to second exposure parameters within the first time period after triggering the start of shooting; wherein the exposure time corresponding to the second exposure parameter is less than the exposure time corresponding to the first exposure parameter.

[0034] In some possible implementations, the second exposure parameter is an initial exposure parameter of the camera sensor.

[0035] In some possible implementations, the method further includes discarding the image frame output by the camera sensor according to the second exposure parameter. Generating the first image based on the output image frame includes generating the first image based on the image frame output by the camera sensor according to the first exposure parameter.

[0036] In some possible implementations, the first duration is determined based on a second duration, where the first duration is greater than or equal to the second duration, and the second duration represents the time required to reset various parameters after the start of capture is triggered. After the first duration, the filter mode is switched and the anti-shake reset function is disabled, reaching a stable state. Capturing long-exposure frames in this condition effectively avoids blurring of long-exposure image frames caused by anti-shake reset, significantly reducing the impact of shake on long-exposure capture.

[0037] Exemplarily, the first duration may be 100 milliseconds.

[0038] In some possible implementations, resetting the various parameters includes: modifying a filtering method for jitter data; and disabling the anti-shake reset function. In this case, the second duration is the sum of the time required to modify the filtering method for jitter data and the time required to disable the anti-shake reset function.

[0039] Exemplarily, after triggering long exposure shooting or variable exposure shooting, the filtering mode for the jitter data is modified to a low-pass filtering mode.

[0040] In some possible implementations, resetting the various parameters further includes resetting the camera lens to an initial position. In this case, the second duration is the sum of the time required to modify the filtering method for the jitter data, the time required to disable the anti-shake reset function, and the time required to reset the camera lens to its initial position.

[0041] Through the above solution, after triggering the start of shooting, the first time period is waited for to complete the resetting of various parameters. After the state stabilizes, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters. This can avoid image blurring caused by various possible jitters, thereby greatly reducing the impact of various possible jitters on specific exposure shooting and improving image clarity.

[0042] In some possible implementations, identifying that the current first scene meets the preset shooting conditions in the preview stage includes: obtaining ambient light brightness information and / or brightness information of the preview image in the preview stage; performing scene detection based on the ambient light brightness information and / or the brightness information of the preview image, and identifying that the first scene meets the preset shooting conditions.

[0043] In some possible implementations, the brightness information includes a brightness value and a brightness range.

[0044] The performing scene detection based on the ambient light brightness information and / or the brightness information of the preview image to identify that the first scene meets the preset shooting condition includes: if the brightness value of the preview image collected in the preview stage is less than or equal to a first brightness threshold, and / or the ambient light brightness is less than or equal to a second brightness threshold, then identifying that the first scene meets the long exposure shooting condition; if the brightness range of the preview image collected in the preview stage is greater than a first brightness range threshold, and / or the ambient light brightness range is greater than a second brightness range threshold, then identifying that the first scene meets the variable exposure shooting condition.

[0045] In some possible implementations, the method further includes: determining exposure parameters required for photographing the first scene when it is identified in the preview stage that the first scene meets the preset photographing condition.

[0046] In some possible implementations, determining the exposure parameters required for photographing the first scene includes: determining the exposure parameters required for photographing the first scene according to ambient light brightness information and / or brightness information of the preview image.

[0047] In some possible implementations, before controlling the camera sensor to output image frames according to the adjusted exposure parameters, the method further includes: adjusting the exposure parameters of the camera sensor according to the determined exposure parameters required for capturing the first scene.

[0048] In some possible implementations, after generating the first image based on the output image frame, the method further includes: switching from a shooting mode to a preview mode; adjusting the camera exposure parameters to the initial exposure parameters; and re-enabling the anti-shake reset function.

[0049] It should be noted that after the current shooting is completed, you will switch back to preview mode from long exposure shooting or variable exposure shooting. Therefore, it is necessary to restart the anti-shake reset function to ensure that the anti-shake function corresponding to the preview mode can be used normally during the preview stage.

[0050] Through the solution of this application, in preview mode, the jitter data is bandpass filtered and the camera lens movement is controlled based on the processed jitter data to achieve an anti-shake effect. At the same time, zero shutter lag (ZSL) is used to capture the preview image in preview mode. In addition, scene detection is performed during the preview phase, and based on the scene detection results, it is determined whether to initiate long-exposure or variable-exposure shooting. When long-exposure or variable-exposure shooting is initiated, the iOS algorithm module switches from preview mode to shooting mode, filters the jitter data through a full-band low-pass filter, and controls the camera lens movement based on the processed jitter data to achieve an anti-shake effect. After waiting for a first period of time, the sensor exposure parameters are changed, and a long-exposure image is captured using non-ZSL. This greatly reduces the impact of various possible jitters on long-exposure or variable-exposure shooting, improving image clarity. After the long-exposure or variable-exposure shooting is completed, the shooting mode is switched to preview mode. For long-exposure and variable-exposure scenes, the solution of this application can effectively improve the image anti-shake effect, reduce motion blur in the image, and ensure image clarity.

[0051] In some possible implementations, the method further includes: receiving a second operation triggered by the user to start shooting when it is identified in the preview stage that the first scene does not meet the preset shooting conditions; in response to the second operation, adjusting the orientation of the camera lens based on third jitter data, controlling the camera sensor to output image frames according to the initial exposure parameters, and generating a second image based on the output image frames; wherein the third jitter data is data obtained after a second filtering process is performed on the jitter data of the electronic device during the shooting stage.

[0052] In some possible implementations, the method further includes: adjusting the orientation of the camera lens using the anti-shake reset function.

[0053] In a second aspect, the present application provides a camera shake reduction device, comprising a unit for executing the method described in the first aspect. The device can be configured to execute the method described in the first aspect. For a description of the units in the device, please refer to the description of the first aspect above and will not be repeated here for the sake of brevity.

[0054] The method described in the first aspect above can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a processing module or unit, a display module or unit, etc.

[0055] In a third aspect, the present application provides an electronic device comprising a processor, a computer program or instructions stored in the processor and a memory, wherein the processor is configured to execute the computer program or instructions so that the method in the first aspect is executed.

[0056] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program (also referred to as instructions or code) for implementing the method in the first aspect. For example, when the computer program is executed by a computer, the computer can perform the method in the first aspect.

[0057] In a fifth aspect, the present application provides a chip comprising a processor. The processor is configured to read and execute a computer program stored in a memory to perform the method of the first aspect and any possible implementation thereof. Optionally, the chip further comprises a memory, the memory being connected to the processor via a circuit or wire.

[0058] In a sixth aspect, the present application provides a chip system comprising a processor. The processor is configured to read and execute a computer program stored in a memory to perform the method of the first aspect and any possible implementation thereof. Optionally, the chip system further comprises a memory, the memory being connected to the processor via a circuit or wire.

[0059] In a seventh aspect, the present application provides a computer program product, which includes a computer program (also referred to as instructions or codes). When the computer program is executed by an electronic device, the electronic device implements the method in the first aspect.

[0060] It can be understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] FIG1 is a schematic diagram of the interface of the image capture module due to shaking during actual shooting;

[0062] FIG2 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0063] FIG3 is a schematic diagram of the optical image stabilization principle provided by an embodiment of the present application;

[0064] FIG4 is a schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0065] FIG5 is a schematic diagram of jitter data detected by a gyroscope sensor according to an embodiment of the present application;

[0066] FIG6 is a schematic diagram of a flow chart of a method for anti-shake shooting provided by an embodiment of the present application;

[0067] FIG7 is a schematic diagram of a flow chart of a method for anti-shake shooting provided by an embodiment of the present application;

[0068] FIG8 is a schematic diagram of an application scenario of a shooting anti-shake method provided by an embodiment of the present application;

[0069] FIG9 is a schematic diagram of an application scenario of another shooting anti-shake method provided by an embodiment of the present application;

[0070] FIG10 is a schematic diagram of an application scenario interface of another shooting anti-shake method provided by an embodiment of the present application;

[0071] FIG11 is a schematic structural diagram of a shooting anti-shake device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the objectives, technical solutions and advantages of this application more clear, the embodiments of this application are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0073] First, some terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0074] 1. Exposure time

[0075] Exposure time refers to the time it takes for the shutter to open and close. The length of exposure time will affect the clarity of the photo.

[0076] 2. Long exposure shooting

[0077] Long exposure photography, also known as slow shutter photography, creates a flowing or blurred effect by capturing the trajectory of a moving object over a period of time by extending the exposure time (for example, a long exposure time of 5 seconds or longer).

[0078] When users use a handheld device for long exposures, factors such as device stability and slight disturbances in the air may cause blur or vibration in the photo, affecting its clarity. Furthermore, long exposure times can increase noise, affecting image quality.

[0079] 3. Variable exposure shooting

[0080] Variable exposure shooting means that the sensor simultaneously outputs long-exposure images and short-exposure images, and forms a high dynamic range (HDR) image by fusing the long and short exposure images.

[0081] In some shooting scenes with large differences in brightness levels, ordinary CMOS / CCD image sensors cannot fully represent the true scene. The brighter areas of the scene will appear bright white due to overexposure, while the darker areas will appear dark due to underexposure. In such cases, variable exposure shooting can be used to present the dark areas while ensuring that the bright areas are not overexposed, preserving the brightness information.

[0082] 4. Motion Blur

[0083] When an electronic device captures a picture, each frame is generated by accumulating photons during the exposure time, converting them into electrons through photoelectric conversion, and then transforming them into an image recognizable to the human eye. If the device experiences significant movement during this time, this motion information will be accumulated and recorded, resulting in a significant amount of motion blur in the resulting image.

[0084] 5. Optical image stabilization (OIS)

[0085] Optical image stabilization, also known as optical image stabilization, refers to the use of motion sensors (such as gyroscopes and accelerometers) to detect the vibration of electronic devices during photo exposure. The OIS controller controls the motor that drives the OIS and moves the lens or image sensor based on the vibration data detected by the motion sensor, so that the optical path remains as stable as possible during the entire exposure period, thereby obtaining a clear exposed image.

[0086] Optical image stabilization includes two types: lens-shift optical image stabilization and sensor-shift optical image stabilization. The first lens-shift optical image stabilization method uses a gyroscope sensor inside the lens to detect tiny movements. This signal is then transmitted to a microprocessor, which immediately calculates the amount of displacement that needs to be compensated. The microprocessor then uses a compensating lens group to compensate for the direction and amount of lens movement, effectively overcoming image blur caused by camera vibration. The second sensor-shift optical image stabilization method uses image sensor offset to achieve image stabilization. The principle is to first place the CCD on a bracket that can move up, down, left, and right. Then, when the gyroscope sensor detects vibration, it processes parameters such as the direction, speed, and amount of movement to calculate the amount of CCD movement sufficient to offset the vibration.

[0087] Optionally, the OIS controller includes a two-axis and a three-axis optical image stabilizer. The embodiment of the present application takes a two-axis, lens-moving optical image stabilization OIS as an example to illustrate, involving two-axis data, which will not be repeated below.

[0088] When a user uses an electronic device to shoot, jitter is inevitable. For example, jitter may refer to hand-held jitter or touch jitter when the user is shooting, or jitter may refer to the electronic device moving during the shooting process. As shown in Figure 1, a user holds a handheld electronic device (such as a mobile phone) to shoot, and due to jitter during the shooting process, the captured video or image is blurred, affecting the clarity of the image.

[0089] Specifically, jitter mainly comes from:

[0090] 1. Irregular internal and external shaking, such as muscle tremors, pedestrians passing by, and the slightest movement of the wind.

[0091] 2. Touching the screen at the moment of taking a photo causes shaking.

[0092] 3. Regular disturbances of human breathing, heartbeat, etc.

[0093] Table 1

[0094] Table 1 shows the frequencies and amplitudes of the three jitter sources.

[0095] Among them, irregular jitter corresponds to the widest frequency range and the smallest amplitude. This part of jitter will affect the imaging quality to a certain extent. The current anti-shake technology can achieve partial anti-shake for irregular jitter.

[0096] Among them, the frequency of jitter caused by the touch screen is higher than the frequency of regular jitter, and the amplitude of the jitter caused by the touch screen is relatively large. This part of the jitter usually seriously affects the image quality. The current anti-shake technology mainly targets the jitter caused by the touch screen, which greatly reduces the impact of the jitter caused by the touch screen on the image quality.

[0097] Among them, the frequency of regular jitter is relatively low and the amplitude is large, which will also have a certain degree of impact on the imaging quality.

[0098] To mitigate the effects of hand shake, electronic devices typically use optical image stabilization (OIS) to reduce motion blur caused by hand shake and improve the success rate of photo capture. Furthermore, to enhance the hand-tracking experience, current OIS algorithms have added a bandpass filter function, which provides almost no stabilization for low-frequency vibrations. Furthermore, when the user significantly moves the electronic device, the OIS motor pushes the lens to its maximum distance to overcome the effects of shaking. To avoid affecting the subsequent stabilization effect, the current OIS algorithm has added an stabilization reset (pan / tilt) function that pulls the lens back to the center and restores stabilization capabilities.

[0099] Current camera stabilization features can help produce clearer images and minimize blur caused by camera shake. Stabilization technology uses sensors to detect shake and compensates by moving the lens element or the sensor itself. However, when shooting long exposures, if the camera stabilization feature is running, even if the camera is not moving, the stabilization system will try to compensate for the shake by moving the lens element or sensor, which will actually cause the image to shake or blur.

[0100] In some special scenes, image clarity can still be affected by significant camera jitter. For example, in bright scenes, short exposure times result in minimal camera jitter and a high image quality. However, in low-light scenes, prolonged exposure times can dramatically increase camera jitter, leading to poor image clarity and a significant drop in image quality. Similarly, when shooting with variable exposure, long-exposure images can be easily blurred due to the dramatic increase in camera jitter.

[0101] Therefore, the current technical challenges are: on the one hand, when users use handheld electronic devices to shoot long or variable exposures, the camera must be kept stable, otherwise even the slightest shake will affect the photo, so effective anti-shake technology is needed for this scenario. On the other hand, when using long or variable exposures, the amount of shake can increase dramatically, significantly affecting image clarity. However, current camera anti-shake functions require frequent refocusing of the lens, which causes delays and results in poor anti-shake effectiveness.

[0102] Currently, there is an urgent need for an anti-shake method for long exposure shooting and variable exposure shooting to improve the anti-shake effect and enhance image quality.

[0103] In view of the above problems, an embodiment of the present application proposes a shooting stabilization method. In the preview stage, it is identified whether the current scene is a specific exposure shooting scene. For specific exposure shooting scenes (such as long exposure shooting or variable exposure), since they are significantly affected by shaking, a series of stabilization processes are immediately started after the shooting is triggered. The stabilization reset function is disabled to avoid image blur caused by the stabilization reset. In addition, a specific full-band filtering process is performed on the shaking data. The orientation of the camera lens is adjusted according to the processed shaking data to achieve a full-band stabilization effect. For example, stabilization can be performed for high-frequency shaking such as irregular shaking and touch screen shaking, as well as low-frequency shaking such as breathing and heartbeat. After the shooting is triggered, a first time period is waited for to complete the reset of various parameters. After the state stabilizes, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters. This can avoid image blur caused by various possible shaking, thereby greatly reducing the impact of various possible shaking on specific exposure shooting and improving image shooting clarity.

[0104] In some embodiments, when the iOS algorithm module switches from the preview mode to the shooting mode, if it is detected that the iOS motor has turned on the anti-shake reset function, the iOS algorithm module can turn off the anti-shake reset function.

[0105] In some embodiments, when the IOS algorithm module switches from the preview mode to the shooting mode, if it is detected that the camera lens of the electronic device is not in the center position, the IOS algorithm module centers the camera lens through the IOS motor.

[0106] In some embodiments, when the iOS algorithm module switches from preview mode to capture mode, the iOS algorithm module waits for a first duration before changing the sensor exposure parameters and acquiring a long-exposure image. For example, the first duration can be 100 milliseconds. It should be noted that mode switching requires a certain amount of stabilization time. Changing the sensor exposure parameters and acquiring a long-exposure image when the state is stable can help improve image stabilization.

[0107] In some embodiments, when the iOS algorithm module switches from the preview mode to the shooting mode, the iOS algorithm module notifies the upper-layer camera application to delay for the first duration.

[0108] In some embodiments, when the iOS algorithm module switches from preview mode to capture mode, the iOS algorithm module uses an infinite impulse response (IIR) filter for real-time filtering. The IIR filter implements the filtering process through recursive operation. For example, the IIR filter can be a Butterworth filter or a Chebyshev filter, and the filter frequency band can be modified by modifying the parameters of each order.

[0109] The hardware system of the electronic device provided in the embodiment of the present application is described below with reference to FIG2 .

[0110] Illustratively, the electronic devices provided in the embodiments of the present application may be mobile phones, smart screens, tablet computers, wearable electronic devices, in-vehicle electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), projectors, and the like. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0111] For the convenience of explanation, FIG2 takes the electronic device 100 as a mobile phone as an example and illustrates its hardware system.

[0112] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0113] It should be noted that the structure shown in FIG2 does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in FIG2, or the electronic device 100 may include a combination of some of the components shown in FIG2, or the electronic device 100 may include sub-components of some of the components shown in FIG2. The components shown in FIG2 may be implemented in hardware, software, or a combination of software and hardware.

[0114] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices. The controller may generate an operation control signal based on the instruction opcode and timing signal to complete the control of instruction fetching and execution.

[0115] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0116] In the embodiment of the present application, the code for implementing the anti-shake method for shooting described in the embodiment of the present application can be stored in a non-volatile memory. When running the camera application, the electronic device 100 can load the executable code stored in the non-volatile memory into the random access memory.

[0117] In an embodiment of the present application, the processor 110 can execute the following steps: starting a camera application; identifying that the current first scene meets a preset shooting condition in a preview stage; receiving a first operation triggered by a user to start shooting, and in response to the first operation, turning off an anti-shake reset function, and adjusting the orientation of the camera lens according to first jitter data; the first jitter data is data obtained after a first filtering process is performed on the jitter data of the electronic device in the shooting stage; waiting for a first time period after triggering the start of shooting, controlling the camera sensor to output image frames according to adjusted exposure parameters, and generating a first image based on the output image frames; wherein the adjusted exposure parameters include a first exposure parameter, and the exposure duration corresponding to the first exposure parameter is greater than the exposure duration corresponding to the initial exposure parameter of the camera sensor; wherein the first filtering process is full-band filtering.

[0118] 2 is merely a schematic illustration and does not limit the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of the multiple connection modes in the above embodiments.

[0119] Electronic device 100 can implement display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information. Display screen 194 can be used to display images or videos.

[0120] The electronic device 100 can implement a shooting function through an image signal processor ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0121] In an embodiment of the present application, an image signal processor ISP is used to process data fed back by the camera 193. For example, when shooting, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the image signal processor ISP for processing and converts it into an image visible to the naked eye. The image signal processor ISP can also perform algorithmic optimization on the noise, brightness, and skin color of the image. The image signal processor ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the image signal processor ISP can be set in the camera 193.

[0122] In an embodiment of the present application, the camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the image signal processor ISP to convert it into a digital image signal. The image signal processor ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0123] In the embodiments of the present application, the digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0124] Gyro sensor 180B, also known as an angular velocity sensor, can be used to determine the motion posture of electronic device 100. In some embodiments, gyro sensor 180B can be used to determine the angular velocity of electronic device 100 around three axes (i.e., x-axis, y-axis, and z-axis). Gyro sensor 180B can be used for scenarios such as image stabilization, navigation, and motion-sensing gaming.

[0125] In embodiments of the present application, gyroscope sensor 180B can be used to collect jitter information, which can be used to indicate changes in the electronic device's posture during filming. The jitter information can include the angular velocity of the electronic device when it is tilted or deflected. The angle is then obtained by performing a discretized integration of the angular velocity.

[0126] In the embodiment of the present application, the gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and then allows the lens to offset the shaking of the electronic device 100 through reverse movement, thereby achieving anti-shake.

[0127] Accelerometer 180E can detect the magnitude of the acceleration of electronic device 100 in various directions (generally the x-axis, y-axis, and z-axis). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. Accelerometer 180E can also be used to identify the posture of electronic device 100, which can serve as an input parameter for applications such as landscape / portrait switching and pedometers.

[0128] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0129] In the embodiment of the present application, the ambient light sensor 180L can sense the brightness of the ambient light, and the processor 110 can determine whether to perform long exposure or variable exposure shooting based on the brightness of the ambient light.

[0130] The touch sensor 180K is also referred to as a touch-sensitive device. The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also referred to as a touch screen. The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor 180K can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100 and at a different location from the display screen 194.

[0131] In the embodiment of the present application, the electronic device 100 can use the touch sensor 180K to detect the user's click, slide, and other operations on the display screen 194. For example, the touch sensor 180K can detect the user's touch operation on the shooting control, and the processor 110 can execute the corresponding anti-shake strategy and image capture in response to the touch operation.

[0132] Taking a mobile phone with the above hardware system as an example, the camera of the mobile phone is introduced in detail below.

[0133] FIG3 shows a schematic diagram of a side view structure corresponding to a camera.

[0134] For example, a camera generally includes an optical lens and a photosensitive element (also called an image sensor), etc. The optical lens may include one or more lenses. The optical lens uses the refraction principle of the lens to transmit light so that the light forms an image on the photosensitive element.

[0135] It should be understood that the “lens” mentioned in this application can be understood as a whole lens, which may include one or more lenses, and the “lens” can be understood as a lens in a lens structure or a lens or lens used to constitute a lens.

[0136] In addition, a filter (not shown) may be included between the lens and the photosensitive element. The filter is used to filter out unwanted wavelengths in the light, preventing the photosensitive element from generating false colors or ripples, thereby improving its effective resolution and color reproduction. Of course, this is only an example, and the camera may also include other structures, which are not limited in this application.

[0137] In conjunction with this example, in order to perform optical image stabilization, an OIS controller can be added to the camera lens. The OIS controller includes an OIS motor (not shown in the figure). The OIS controller is used to obtain the jitter data of the electronic device collected by the gyroscope sensor, such as angular velocity, and generate a control signal for controlling the movement of the OIS motor based on the jitter data collected by the gyroscope sensor. The OIS motor is used to drive the lens to move under the control of the control signal, thereby offsetting the displacement caused by the jitter.

[0138] Referring to FIG3 , the optical image stabilization implementation method of the OIS motor is explained. Taking the coordinate system shown in FIG3 as an example, the OIS motor in the OIS controller can push the lens to move left and right, that is, to move in the x-axis direction, to offset the displacement caused by the shake in the x-axis direction. The OIS motor in the OIS controller can push the lens to move up and down, that is, to move in the y-axis direction, to offset the displacement caused by the shake in the y-axis direction. It should be understood that the OIS controller can control the lens to move in both the x-axis direction and the y-axis direction in combination with compensation requirements, thereby offsetting the displacement caused by the shake. The specific direction and distance of movement can be determined according to requirements and are not limited in the embodiments of the present application.

[0139] The above introduces the implementation method of optical image stabilization. The reset function of optical image stabilization is described below. When the camera is enabled, the default anti-shake reset function is on. Specifically for this application, when long exposure or variable exposure shooting is adopted, the IOS algorithm module switches from preview mode to shooting mode. In this case, the IOS algorithm module can turn off the anti-shake reset function, that is, turn off the conventional optical image stabilization function. This is because: for specific exposure shooting scenes (such as long exposure shooting or variable exposure), it is significantly affected by the jitter. Therefore, a series of anti-shake processing is immediately started after the shooting is triggered, and the anti-shake reset function is turned off to avoid image blur caused by anti-shake reset. Then, after the photo is taken, switch back from shooting mode to preview mode. In this case, the IOS algorithm module can restart the anti-shake reset function.

[0140] It should be noted that the IOS algorithm module can send anti-shake commands to the OIS controller, and the OIS motor of the OIS controller controls the movement of the camera lens according to the anti-shake commands. The IOS algorithm module and OIS controller can be two independent modules or an integrated module.

[0141] The hardware system of electronic device 100 has been described in detail above. The software system of electronic device 100 is now described. A software operating system runs on the hardware system. The software operating system can be any one or more computer operating systems that implement business processing through processes. Application programs can be installed and run on the software operating system.

[0142] Figure 4 is a schematic diagram of the system architecture of an electronic device provided by an embodiment of the present application. As shown in Figure 4, the system architecture may include, from top to bottom, an application layer 210, an application framework layer 220, a hardware abstraction layer (HAL) 230, a driver layer 240, and a hardware layer 250.

[0143] The application layer 210 may include a series of application packages. In the embodiment of the present application, the application package may include a camera application, a gallery, etc.

[0144] The application framework layer 220 provides an application programming interface (API) and a programming framework for application programs in the application layer; the application framework layer may include some predefined functions.

[0145] In an embodiment of the present application, the application framework layer 220 may include a camera access interface, which is used to provide an application programming interface and programming framework for camera applications. The camera access interface may include camera management and camera devices. The camera management may be used to provide an access interface for managing the camera, and the camera device may be used to provide an interface for accessing the camera.

[0146] The hardware abstraction layer 230 is an interface layer located between the application framework layer and the driver layer, and provides a virtual hardware platform for the operating system.

[0147] In this embodiment of the present application, the hardware abstraction layer 230 includes a camera hardware abstraction layer and camera algorithms. The camera hardware abstraction layer can call camera algorithms; the camera algorithms may include software algorithms for image processing. The camera hardware abstraction layer 230 can provide virtual hardware for camera devices.

[0148] Exemplarily, the camera algorithm library may include the operating code and data of the shooting anti-shake method (or OIS algorithm) provided in the embodiment of the present application.

[0149] For example, the algorithm in the camera algorithm may refer to a code that does not rely on specific hardware implementation; for example, a code that can generally be run in a CPU, etc.

[0150] The driver layer 240 is a layer between hardware and software. The driver layer includes various hardware drivers and is used to provide drivers for different hardware devices.

[0151] In an embodiment of the present application, the driver layer may include a camera device driver, a digital signal processor driver, and an image processor driver. The camera device driver (e.g., a camera driver) is used to drive the camera sensor to capture images and drive the image signal processor to pre-process the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0152] The hardware layer 250 is located at the bottom layer of the operating system; as shown in FIG4 , the hardware layer 250 may include a camera.

[0153] In the embodiment of the present application, the application layer may further include a context awareness module, a business logic processing module, and a business presentation module. The context awareness module, business logic processing module, and business presentation module may be independent apps, or may be integrated into different apps, or may be integrated into the same app, and this application does not limit this.

[0154] Among them, the context awareness module runs permanently or in a low-power form and has the ability to perceive external facts or environments. The context awareness module can detect related events and obtain the status of events from other applications in the application layer or the application framework layer or the system layer or the kernel layer through the application programming interface (API), such as detecting Bluetooth connections, network connections, monitoring user text messages, custom timers, etc. In an embodiment of the present application, the main function of the context awareness module is to monitor whether the camera application is started and perform scene detection in the camera preview mode and identify whether long exposure or variable exposure shooting is used, as well as monitor whether the user triggers shooting. The context awareness module can notify the business logic processing module of the monitored "long exposure or variable exposure shooting" event, and notify the business logic processing module of the "user triggered shooting" event.

[0155] The business logic processing module (e.g., a computing engine) has business logic processing capabilities and is used to perform data processing or logical judgment based on events monitored by the context perception module and preset logical algorithms to achieve anti-shake shooting. Specifically, when it is determined that "long exposure or variable exposure shooting" and "user-triggered shooting" are used, the bandpass filter in preview mode is switched to the full-band low-pass filter in shooting mode. The lens is centered through the OIS motor and the anti-shake reset function is turned off. After waiting for a first period of time, long exposure or variable exposure parameters are used to capture long exposure image frames, and the first image is obtained based on the long exposure image frames. The specific judgment process will be described in detail below.

[0156] The business presentation module (eg, YOYO suggestion) is configured to store the first image in a gallery and display a thumbnail of the first image on a camera preview interface according to instructions from the business logic processing module.

[0157] It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0158] The following describes the workflow of the software system and hardware system of the electronic device 100 in conjunction with a photo-taking scenario.

[0159] When a user performs a touch operation on touch sensor 180K, a corresponding hardware interrupt is sent to the kernel layer, which processes the touch operation into a raw input event. The raw input event includes, for example, information such as the touch coordinates and the timestamp of the touch operation. The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer, identifies the control corresponding to the raw input event, and notifies the application (APP) corresponding to the control. For example, if the touch operation is a single click, and the APP corresponding to the control is a camera APP, after the camera APP is awakened by the single click operation, it can call the kernel layer's camera driver through the API, and the camera driver controls the camera 193 to take pictures.

[0160] Taking the photo-taking scenario as an example, in response to the user opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface of the application framework layer to start the camera application, and then sends an instruction to start the camera by calling the camera device (camera device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends this instruction to the camera device driver in the kernel layer. The camera device driver can start the corresponding camera sensor and collect image light signals through the sensor. A camera device in the camera hardware abstraction layer corresponds to a camera sensor in the hardware layer.

[0161] Then, the camera sensor can transmit the collected image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the above raw image to the camera hardware abstraction layer through the camera device driver.

[0162] The camera hardware abstraction layer can send the original image to the camera algorithm library. The camera algorithm library stores the program code for implementing the shooting anti-shake method provided by the embodiment of the present application. Based on the digital signal processor and the image processor, the camera algorithm library executes the above code. In the preview mode, the jitter data is band-pass filtered, and then the anti-shake processing is performed based on the processed jitter data, thereby avoiding the impact of the jitter on the preview image; and the scene detection is performed in the preview stage to determine whether to perform long exposure or variable exposure shooting; when long exposure or variable exposure shooting, the anti-shake reset function is turned off, the jitter data is low-pass filtered, and then the anti-shake processing is performed based on the processed jitter data; and the long exposure or variable exposure image is collected after a certain period of time from the triggering of the photo, thereby avoiding the impact of the jitter on the long exposure or variable exposure image.

[0163] The camera algorithm library can send long-exposure or variable-exposure image frames captured by the camera to the camera hardware abstraction layer. The camera hardware abstraction layer can then display the long-exposure or variable-exposure image frames and store them in the image library.

[0164] To reduce the impact of various possible jitters on specific exposure shots, this application implements the following shooting anti-shake strategies:

[0165] In the preview mode, scene detection is performed, and the scene detection results are: 1) not shooting with long exposure or shooting with variable exposure, 2) shooting with long exposure or shooting with variable exposure.

[0166] On the one hand, during the preview phase, it is recognized that long exposure or variable exposure shooting is not being used. Accordingly, the preview mode and shooting mode use the same anti-shake method: bandpass filtering is used in both preview mode and shooting mode, and the anti-shake reset function is turned on. The OIS controller obtains the jitter data (angular velocity) of the electronic device collected by the gyroscope sensor and generates a control signal for controlling the movement of the OIS motor based on the jitter data collected by the gyroscope sensor. Under the control of the control signal, the OIS motor pushes the lens to move or reset to offset the displacement caused by the jitter, achieving optical image stabilization. This optical image stabilization processing can be applied to both preview mode and shooting mode.

[0167] On the other hand, in the preview stage, it is recognized that long exposure or variable exposure shooting will be used. Accordingly, different anti-shake methods are used in preview mode and shooting mode respectively:

[0168] 1) In preview mode, bandpass filtering is used, and the image stabilization reset function is enabled. The OIS controller acquires the electronic device's vibration data (angular velocity) from the gyroscope sensor and generates a control signal based on this vibration data to control the movement of the OIS motor. Driven by this control signal, the OIS motor moves the lens up and down / left and right, thereby offsetting the displacement caused by vibration and achieving optical image stabilization. The camera sensor captures the image, producing a preview image with optical image stabilization.

[0169] 2) Switch to low-pass filtering in shooting mode and turn off the anti-shake reset function in shooting mode to avoid image blur caused by anti-shake reset. Center the lens through the OIS motor to ensure that the anti-shake function can be used normally after turning off the anti-shake reset function.

[0170] Long-exposure images from long or variable-exposure photography have long exposure times and noticeable jitter. Therefore, by low-pass filtering the jitter data and controlling the camera lens movement based on this processed jitter data, full-band stabilization can be applied to address both high- and low-frequency jitter in electronic devices. This significantly reduces the impact of various possible jitters on specific exposures, such as long or variable-exposure photography. With this stabilization process, the camera sensor captures long-exposure image frames, generating a first image that has been optically stabilized, improving image clarity.

[0171] The embodiments of the present application provide a shooting anti-shake method and an electronic device. By improving the software of the electronic device, the influence of shaking on long-exposure shooting can be effectively avoided, and the blurring of the image caused by shaking can be avoided, thereby improving the shooting anti-shake performance.

[0172] The execution subject of the shooting anti-shake method provided in the embodiments of the present application can be the above-mentioned electronic device, or a functional module and / or functional entity in the electronic device that can implement the shooting anti-shake method. The present application solution can be implemented through hardware and / or software. The specific implementation can be determined according to actual usage requirements and is not limited by the embodiments of the present application. The following uses an electronic device as an example and combines the accompanying drawings to illustrate the shooting anti-shake method provided in the embodiments of the present application.

[0173] The following embodiments of the present application will be illustrated by the following exemplary embodiments in conjunction with the accompanying drawings. The methods in the following embodiments can all be implemented in an electronic device having the above-mentioned hardware structure and software architecture. The hardware structure diagram of the electronic device can be shown in Figure 2, and the software structure block diagram of the electronic device can be shown in Figure 4, but the embodiments of the present application are not limited thereto. For ease of explanation, the embodiments of the present application are all based on the example of a mobile phone as shown in Figure 2 as an electronic device.

[0174] First, the anti-shake system architecture provided by an embodiment of the present application is described with reference to FIG5 , which shows a schematic structural diagram of the gyroscope sensor, the OIS controller, and the lens.

[0175] As shown in Figure 5, a gyroscope sensor can be connected to an OIS controller. The gyroscope sensor obtains the angular velocity of the electronic device's vibration and transmits the angular velocity of the electronic device's vibration to the OIS controller.

[0176] The OIS controller is connected to a drive motor (also known as an optical image stabilization motor), which is then connected to the lens. The OIS controller calculates the compensation angle based on the angular velocity of the electronic device's vibration and directs the drive motor to adjust the lens angle for stable imaging.

[0177] The OIS controller can also be connected to an application processor (AP) or image signal processor (ISP), which can be connected to a CMOS sensor, which can be connected to a lens. The OIS controller can call on the application processor (AP) to calculate the compensation angle. The OIS controller can also call on the image signal processing signal from the image signal processor (ISP) to correct the compensation angle.

[0178] The OIS controller can be connected to a Hall effect sensor, which in turn is connected to the lens. The Hall effect sensor is used to feed back the lens's position information to the OIS controller, forming a closed-loop control loop to accurately move the lens to the desired position.

[0179] The OIS controller may also be referred to as an OIS control driver or an OIS driver chip.

[0180] Referring to the structural diagram shown in Figure 6, the basic principle of optical image stabilization is that when the gyroscope sensor is subjected to external vibrations, the vibration signal is fed back to the OIS controller. The OIS controller drives the OIS motor to move the lens, thereby offsetting the image offset caused by the vibration and ensuring that the camera can still maintain stable imaging in a shaking environment.

[0181] The following describes the effect of jitter on imaging with reference to the jitter data detected by the gyroscope sensor shown in FIG6 .

[0182] FIG6 shows the time domain jitter data detected by the gyroscope sensor during the preview stage and the shooting stage. Various jitter factors existing in the shooting process will affect the image quality, so it is necessary to analyze the jitter data and perform anti-shake processing by comprehensively considering various jitter factors. The time domain jitter data can be converted into frequency domain jitter data, and the frequency domain jitter data includes high-frequency jitter signals and low-frequency jitter signals. It should be noted that the high frequency and low frequency described here are relative. For example, the high frequency jitter signal may include irregular jitter and jitter caused by touching the screen. The low frequency jitter signal may include regular jitter such as breathing and heartbeat.

[0183] As shown in Figure 6, the electronic device detects jitter data through a gyroscope sensor. After analyzing the jitter data, it is found that the jitter amplitude suddenly increases in long-exposure or variable-exposure shooting scenarios. This is because: the exposure time is prolonged, the jitter amount increases sharply, and the impact of jitter on image quality is more obvious.

[0184] In the related art, when the shutter is pressed, the gyroscope sensor detects the angle of the electronic device's shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens module to offset the shaking of the electronic device through reverse movement to achieve anti-shake. In actual implementation, the shaking data detected by the gyroscope sensor corresponds to a high-frequency shaking signal. Based on the high-frequency shaking signal, the OIS controller controls the OIS motor to move the lens module to offset the high-frequency shaking of the electronic device through reverse movement to achieve optical image stabilization. Such optical image stabilization can only compensate for the impact of high-frequency shaking, but ignores low-frequency shaking. Therefore, this optical image stabilization process is not effective when applied to long-exposure or variable-exposure shooting scenes.

[0185] This application solution can be applied to long-exposure or variable-exposure shooting scenes, and the effects of various possible jitters on long-exposure or variable-exposure shooting scenes and other specific exposure shooting scenes are greatly reduced. Through this application solution, not only relatively high-frequency irregular jitters and jitters caused by the touch screen are anti-shake processed, but also relatively low-frequency regular jitters such as breathing and heartbeats are anti-shake processed, overcoming the effects of jitters in long-exposure or variable-exposure shooting scenes and improving image quality. The specific anti-shake method will be described in detail below.

[0186] The following describes the anti-shake method for shooting provided by the embodiments of the present application in conjunction with specific embodiments.

[0187] It should be noted that, from the perspective of the camera operating state, the camera operating state includes a preview state and a capture state. From the perspective of the camera operating mode, the camera operating mode includes a preview mode and a capture mode. The preview state corresponds to the preview mode, and the capture state corresponds to the capture mode. For ease of explanation, the following embodiments are illustrative examples of camera operating modes including the preview mode and the capture mode.

[0188] In the embodiment of the present application, since the user holds the electronic device to shoot, there are various possible shakes in both preview mode and shooting mode, which will affect the image clarity. In other words, anti-shake processing is required in both preview mode and shooting mode.

[0189] FIG7 is a flow chart of a method for stabilizing a photographic image provided by an embodiment of the present application. Referring to FIG7 , the method for stabilizing a photographic image includes the following steps S301 to S316.

[0190] S301: In response to a user operation, start a camera application and display a preview interface.

[0191] In some embodiments, the electronic device receives a click operation on a camera application icon by a user. In response to the click operation, the electronic device starts the camera application, starts a preview mode, and displays a camera preview interface.

[0192] In other embodiments, the electronic device runs a foreground operation such as An application such as an application with a shooting entrance, and a shooting control is displayed in the current interface of the electronic device. When the electronic device receives a click operation on the shooting control by the user, the electronic device responds to the click operation, starts the camera application, starts the preview mode and displays the camera preview interface.

[0193] S302 : In the preview mode, obtain jitter data, perform bandpass filtering on the jitter data, and perform optical image stabilization according to the processed jitter data and an anti-shake reset function.

[0194] Due to the influence of jitter, the image captured in preview mode will appear blurred, so it is necessary to perform anti-shake processing in preview mode. This application can implement optical anti-shake processing in preview mode according to the anti-shake strategy corresponding to the preview scene.

[0195] The above-mentioned shaking data may include the attitude angle of the electronic device shaking.

[0196] In an embodiment of the present application, in preview mode, the electronic device can obtain the gyroscope angular velocity through the gyroscope sensor, and obtain the angle by performing discretized integration processing on the gyroscope angular velocity. The angle is the posture angle of the electronic device jitter.

[0197] Bandpass filtering is a signal processing technique used to select signals within a specific frequency range from a frequency spectrum when processing a signal. The principle is to selectively attenuate or retain certain frequencies in the signal, so that the filtered signal only contains signals within a specific frequency range.

[0198] In the embodiment of the present application, bandpass filtering of the jitter data includes bandpass filtering of the "posture angle of electronic device jitter." The bandpass filtering process retains the posture angle corresponding to high-frequency jitter (such as irregular jitter and touch screen jitter) and filters out the posture angle corresponding to low-frequency jitter (such as regular jitter such as breathing and heartbeat). In other words, the anti-shake processing in preview mode only targets high-frequency jitter and ignores low-frequency jitter.

[0199] Then, the electronic device calculates the distance that the lens module needs to compensate based on the posture angle corresponding to high-frequency jitter (such as irregular jitter, touch screen jitter), allowing the lens module to offset the jitter of the electronic device through reverse movement to achieve anti-shake.

[0200] The advantage of optical image stabilization (OIS) processing, which performs bandpass filtering on jitter data, is its fast response time, resulting in better chirality in preview mode. Chirality refers to the ability of the screen's content to react to a user's touch. Higher chirality results in a more immediate screen response, resulting in a smoother visual experience.

[0201] In some cases, the anti-shake reset function is enabled by default. When the anti-shake effect is not satisfactory or the anti-shake function fails, the electronic device will use the anti-shake reset function. For example, when the shaking amplitude is large, the OIS motor may reach its limit in pushing the lens module, causing the optical image stabilization function to fail. In this case, the electronic device will use the anti-shake reset function, that is, using the OIS motor to reset the lens module to its initial position, such as placing the lens module in the neutral position, to restore the anti-shake function.

[0202] Therefore, in the preview mode, according to the anti-shake strategy corresponding to the preview scene, the optical anti-shake purpose is achieved based on the jitter data after bandpass filtering and the anti-shake reset function.

[0203] S303: Capture a preview image using preset exposure parameters, and display the preview image on a preview interface.

[0204] Among them, in the preview mode, the electronic device will capture an image and display the captured preview image on the camera preview interface.

[0205] In the embodiment of the present application, optical image stabilization is performed in the preview mode to avoid the impact of high-frequency jitter on image quality, and the response speed is fast to ensure hand tracking.

[0206] S304: Perform scene detection in preview mode.

[0207] In some embodiments, the electronic device may perform scene detection based on brightness information of the preview image.

[0208] In other embodiments, the electronic device may also perform scene detection based on ambient light brightness information collected by the ambient light sensor.

[0209] In other embodiments, the electronic device may perform scene detection based on brightness information of the preview image and ambient light brightness information collected by the ambient light sensor.

[0210] S305: Determine whether the current scene meets the conditions for long exposure or variable exposure shooting.

[0211] In some embodiments, the electronic device detects the current scene and determines whether the current scene meets the conditions for long exposure or variable exposure shooting.

[0212] In some embodiments, scenes such as low-light or night scenes meet the conditions for long-exposure shooting.

[0213] For example, if the electronic device detects that the preview image contains night scene features, the electronic device determines that the current scene meets the conditions for long exposure shooting. For example, if the electronic device detects that the ambient light brightness is lower than a preset brightness value, the electronic device determines that the current scene meets the conditions for long exposure shooting.

[0214] In some embodiments, low-light or backlit scenes, such as backlit shots, or scenes like sunrises, sunsets, and skyscapes, meet the requirements for variable exposure shooting. This is because such scenes contain both very bright and very dark areas (i.e., both overexposed and underexposed areas). Such scenes are suitable for HDR shooting mode, which achieves variable exposure shooting by fusing long and short exposure frames.

[0215] For example, if the electronic device detects that the preview image contains a distinct bright and dark area, the electronic device determines that the current scene meets the conditions for variable exposure shooting. For example, if the electronic device detects that the ambient light is distinct bright and dark, the electronic device determines that the current scene meets the conditions for variable exposure shooting.

[0216] In other embodiments, the implementation method of "determining whether the current scene meets the conditions for long exposure or variable exposure shooting" may also include: the electronic device may determine whether to use long exposure or variable exposure shooting based on the user's trigger operation, thereby determining whether the current scene meets the conditions for long exposure or variable exposure shooting.

[0217] For example, after a user taps the "Professional Mode" control, the electronic device can use long exposure or variable exposure shooting. For another example, when the electronic device receives a user trigger to enable "long exposure shooting," the electronic device will use long exposure shooting. For another example, when the electronic device receives a user trigger to enable "HDR shooting," the electronic device will use variable exposure shooting.

[0218] It should be noted that there are two types of judgment results in S305:

[0219] One is that the current scene meets the conditions for long exposure or variable exposure shooting, and after S305 , the following S306 to S312 are continued to be executed, that is, the anti-shake strategy corresponding to the long exposure or variable exposure shooting scene is executed.

[0220] The other is that the current scene does not meet the conditions for long exposure or variable exposure shooting, and the following S313 to S316 are continued to be executed after S305, that is, the anti-shake strategy corresponding to the conventional shooting scene is executed.

[0221] Anti-shake strategies for long exposure or variable exposure shooting scenarios:

[0222] In some embodiments, when the electronic device determines that the current scene meets the conditions for long exposure or variable exposure shooting, the electronic device automatically implements the corresponding anti-shake strategy according to S306 to S312 below. In other embodiments, the electronic device may display a prompt box to prompt the user to confirm whether to use long exposure or variable exposure shooting; when the electronic device receives the user's confirmation operation to use long exposure or variable exposure shooting, the electronic device may implement the corresponding anti-shake strategy according to S306 to S312 below.

[0223] S306: Receive a first operation of the user triggering shooting.

[0224] When the electronic device receives a user-triggered shooting operation, the electronic device switches from the preview mode to the shooting mode, and then performs optical image stabilization according to the anti-shake strategy corresponding to the long exposure or variable exposure shooting scene in the shooting mode.

[0225] S307 : In response to the first operation, obtain jitter data and disable the anti-shake reset function.

[0226] In shooting mode, the electronic device can obtain the gyroscope angular velocity through the gyroscope sensor and obtain the attitude angle of the electronic device's shaking by performing discretized integration processing on the gyroscope angular velocity. The attitude angle of the electronic device's shaking is the shaking data.

[0227] It should be noted that, as mentioned above, in long-exposure or variable-exposure shooting scenarios, the exposure time is lengthened and the amount of jitter increases sharply. If the anti-shake reset function is used at this time, the lens reset action may cause image blur. To avoid this situation, the anti-shake reset function can be turned off.

[0228] S308 , performing low-pass filtering on the jitter data, and performing optical image stabilization processing based on the processed jitter data.

[0229] It should be noted that in long-exposure or variable-exposure shooting scenarios, the exposure time is prolonged and the amount of jitter increases dramatically. To improve the anti-shake effect, the present application scheme performs low-pass filtering on the jitter data. This low-pass filtering process removes high-frequency components and allows only low-frequency components to pass through. By low-pass filtering the jitter data and then performing optical image stabilization based on the processed jitter data, both high-frequency and low-frequency jitter are compensated, thereby improving the anti-shake effect in long-exposure or variable-exposure shooting scenarios.

[0230] Therefore, in the shooting mode, according to the anti-shake strategy corresponding to the long exposure or variable exposure shooting scene, optical anti-shake is performed based on the jitter data after bandpass filtering to achieve the anti-shake purpose.

[0231] S309 : Adjust camera exposure parameters (including long exposure parameters after adjustment), and capture an image based on the adjusted exposure parameters to obtain a first image.

[0232] In an embodiment of the present application, when the current scene meets the conditions for long exposure or variable exposure shooting, if the electronic device receives a first operation of the user triggering shooting, the electronic device adjusts the camera exposure parameters after the first time period and captures an image based on the adjusted exposure parameters.

[0233] It should be noted that the value of the first duration is determined according to the sum of the time taken to switch the filtering mode and turn off the anti-shake reset function (referred to as the second duration). The first duration is greater than or equal to the second duration.

[0234] It should also be noted that the electronic device discards the images captured within the first time period. After the first time period, the electronic device captures long-exposure image frames based on the adjusted exposure parameters and generates the first image based on the long-exposure image frames.

[0235] It can be understood that after the first duration, the filtering mode switching has been completed and the anti-shake reset function has been turned off, reaching a stable state. In this case, capturing long exposure frames can effectively avoid blurring of long exposure image frames caused by anti-shake reset, greatly reducing the impact of shake on long exposure shooting.

[0236] The adjusted camera exposure parameters include long exposure parameters, which include exposure duration. The shooting anti-shake method provided in the embodiment of the present application adjusts the exposure parameters for the following shooting scenes and captures images based on the adjusted exposure parameters:

[0237] Scenario 1: During long exposure shooting, the electronic device sets long exposure parameters and captures long exposure frames based on the long exposure parameters. For example, the exposure time in the long exposure parameters can be set to 5 seconds or other values.

[0238] Scenario 2: During variable exposure shooting, the electronic device sets long exposure parameters and short exposure parameters, and captures long exposure frames and short exposure frames based on the long exposure parameters and short exposure parameters, respectively. For example, the exposure time in the long exposure parameters can be set to 5 seconds, and the exposure time in the short exposure parameters can be set to 1 second. It should be noted that the long exposure parameters and short exposure parameters are relative, and the exposure time is not limited here.

[0239] For long-exposure or variable-exposure shooting scenarios, a series of anti-shake processing begins immediately after switching from preview mode to shooting mode. For example, the system switches from bandpass filtering to low-pass filtering of the jitter data to achieve full-band anti-shake effects. The anti-shake reset function is also disabled to avoid blurring of long-exposure image frames caused by anti-shake reset, greatly reducing the impact of jitter when capturing long-exposure frames.

[0240] S310: Store the first image in a gallery.

[0241] S311: Restart the anti-shake reset function.

[0242] It should be noted that after the current shooting is completed, you will switch back to preview mode from long exposure shooting or variable exposure shooting. Therefore, it is necessary to restart the anti-shake reset function to ensure that the anti-shake function corresponding to the preview mode can be used normally during the preview stage.

[0243] S312: Restore the camera exposure parameters to the preset exposure parameters.

[0244] It should be noted that after the long exposure shooting or variable exposure shooting is completed, the camera exposure parameters need to be restored to the preset exposure parameters.

[0245] It should be noted that this application does not limit the execution order of the above S311 and S312.

[0246] After S311 and S312, the process returns to the above steps S302. That is, after the shooting is completed, the electronic device switches from the shooting mode to the preview mode, and then performs optical image stabilization according to the anti-shake strategy corresponding to the preview mode.

[0247] The above describes the implementation of optical image stabilization according to the corresponding stabilization strategy for long-exposure or variable-exposure shooting scenarios when the current scene meets the conditions for long-exposure or variable-exposure shooting. The following describes the implementation of optical image stabilization according to the stabilization strategy for conventional shooting scenarios when the current scene does not meet the conditions for long-exposure or variable-exposure shooting.

[0248] Anti-shake strategies for common shooting scenarios:

[0249] S313: Receive a second operation of the user triggering shooting.

[0250] When the electronic device receives a user-triggered shooting operation, the electronic device switches from the preview mode to the shooting mode, and then performs optical image stabilization processing according to the anti-shake strategy corresponding to the conventional shooting scene in the shooting mode.

[0251] S314 : In response to the second operation, obtain jitter data, perform bandpass filtering on the jitter data, and perform optical image stabilization according to the processed jitter data and an anti-shake reset function.

[0252] In an embodiment of the present application, in response to the second operation, the electronic device can obtain the gyroscope angular velocity through the gyroscope sensor, and obtain the attitude angle of the electronic device jitter, that is, the jitter data, by performing discretized integral processing on the gyroscope angular velocity. Then, the jitter data is band-pass filtered. The anti-shake processing in conventional shooting scenarios only targets high-frequency jitter and ignores low-frequency jitter. Then, the electronic device calculates the distance that the lens module needs to compensate based on the high-frequency jitter data (such as irregular jitter, touch screen jitter) retained after band-pass filtering, and allows the lens module to offset the jitter of the electronic device through reverse motion to achieve anti-shake.

[0253] When the image stabilization effect fails to meet expectations or the image stabilization function fails, the electronic device will activate the image stabilization reset function. For example, when the vibration amplitude is large, the OIS motor may reach its limit in pushing the lens module, causing the optical image stabilization function to fail. In this case, the electronic device will activate the image stabilization reset function, that is, using the OIS motor to reset the lens module to its original position, such as placing the lens module in the neutral position, to restore the image stabilization function.

[0254] Therefore, the anti-shake strategy corresponding to the normal shooting scene in the shooting mode is the same as the anti-shake strategy corresponding to the preview mode, and optical anti-shake processing is performed based on the jitter data after bandpass filtering and the anti-shake reset function.

[0255] S315: Capture an image using preset exposure parameters to obtain a second image.

[0256] S316: Store the second image in the gallery.

[0257] After S316, the process returns to the above steps S302. That is, after the shooting is completed, the electronic device switches from the shooting mode to the preview mode, and then performs optical image stabilization according to the anti-shake strategy corresponding to the preview mode.

[0258] In some embodiments, in preview mode, the zero-delay photography mode (ZSL) is enabled to capture images, and in shooting mode, the zero-delay photography mode (ZSL) is disabled to capture images.

[0259] When ZSL mode is turned on, the camera caches a short clip of video in memory, allowing you to capture the scene before pressing the shutter button. When you press the shutter button, the camera can immediately save the previously cached scene, reducing shutter lag. This makes it more convenient to shoot dynamic scenes or capture the moment.

[0260] It can be seen from the above method flow that in the embodiment of the present application, the OIS algorithm module adopts different anti-shake strategies for the preview mode and different shooting scenes.

[0261] First, in preview mode, the gyroscope sensor acquires jitter data, which is then bandpass filtered. Optical image stabilization is then performed based on the processed jitter data. Simultaneously, the image is captured using preset exposure parameters to generate a preview image, which is displayed on the preview interface. Furthermore, in preview mode, scene detection is performed to determine whether the current shooting scene meets the requirements for long exposure or variable exposure shooting. Different image stabilization strategies are then applied to different shooting scenarios.

[0262] On the one hand, the current shooting scene meets the conditions for long exposure / variable exposure shooting:

[0263] When the electronic device receives a capture command, it switches from preview mode to capture mode. If the current scene meets the requirements for long-exposure / variable-exposure shooting, the corresponding anti-shake strategy includes: disabling the anti-shake reset function, acquiring shake data, and low-pass filtering the shake data to achieve full-band anti-shake effects; then, adjusting the camera exposure parameters to the long-exposure / variable-exposure parameters and capturing images based on these adjusted exposure parameters. Due to the corresponding anti-shake processing, the captured long-exposure frames in this case are less affected by shake. After the long-exposure / variable-exposure shooting is completed, the device switches from capture mode to preview mode, reactivates the optical image stabilization function, and resets the exposure parameters.

[0264] On the other hand, the current shooting scene does not meet the conditions for long exposure / variable exposure shooting:

[0265] When the electronic device receives a capture command, it switches from preview mode to capture mode. The current capture scenario is a standard capture scenario. The corresponding stabilization strategy includes: using the stabilization reset function as normal, acquiring jitter data, and performing bandpass filtering on the jitter data to achieve stabilization for high-frequency jitter. Furthermore, the image is captured using preset exposure parameters. The normally exposed image captured in this scenario achieves stabilization for some high-frequency jitter and exhibits good hand tracking. After the capture is complete, the device switches from capture mode back to preview mode.

[0266] The following describes the anti-shake method for shooting provided by the embodiment of the present application from the perspective of upper-layer software.

[0267] Specifically, during the preview phase, scene detection is performed to determine whether to enter long-exposure or variable-exposure shooting. If it is determined that long-exposure or variable-exposure shooting has been entered, the upper-level software's HAL layer sends a switching instruction to the OIS algorithm module, instructing it to switch to long-exposure or variable-exposure shooting mode. After waiting for a certain period of time (the first time), the camera exposure parameters are changed (including the long-exposure parameters after the change). After the long-exposure or variable-exposure shooting is completed, the HAL layer sends a switching instruction to the OIS algorithm module, instructing it to switch back to preview mode.

[0268] The switching instructions issued by the HAL layer can be transmitted to the OIS algorithm module through the driver software. For example, the driver software writes the instructions to the register through the inter-integrated circuit (IIC) protocol, and the OIS algorithm module polls the register to obtain the instructions.

[0269] The following describes the shooting anti-shake method provided by the embodiment of the present application from the perspective of the OIS algorithm module.

[0270] Specifically, the OIS algorithm module can use the following filtering strategies: high-frequency jitter filtering (band-pass filtering) corresponding to the preview mode, and full-band filtering (low-pass filtering) corresponding to the shooting mode. That is, in the camera preview stage, the OIS algorithm module uses band-pass filtering for the jitter data to achieve the purpose of anti-shake for high-frequency jitter; when shooting with long exposure or variable exposure, the OIS algorithm module uses low-pass filtering for the jitter data to achieve full-band anti-shake. Among them, the OIS algorithm module can use an IIR filter to filter the jitter data in real time, and the filter frequency band can be modified by modifying the parameters of each order.

[0271] After receiving instructions from the upper layer, the OIS algorithm module switches to long exposure / variable exposure shooting mode, disables the anti-shake reset function, and resets the lens through the OIS motor. Because mode switching requires a certain stabilization time (approximately 100ms), it waits a certain period of time before changing the camera exposure parameters and capturing images based on the modified exposure parameters. Accordingly, the OIS algorithm module can notify the upper layer software to wait for a fixed delay.

[0272] Among them, the OIS algorithm module switches back to preview mode after receiving the upper-layer instruction, restarts the anti-shake reset function, and resets the camera exposure parameters.

[0273] Figure 8 illustrates the implementation of the anti-shake method for shooting in a long-exposure shooting scenario, as provided in an embodiment of the present application. As shown in Figure 8, the anti-shake strategy in preview mode and shooting mode is explained through the upper-layer algorithm module, sensor image output, OIS status, and shake data.

[0274] Anti-shake strategy in preview mode:

[0275] 1) Upper-level algorithm module: Detects the scene through preview and determines whether to enter long-exposure or variable-exposure shooting.

[0276] 2) Sensor output: Use the preset exposure parameters to capture the image frame of regular exposure (marked as N in the figure).

[0277] 3) OIS status:

[0278] a) acquiring jitter data, performing bandpass filtering on the jitter data, and performing optical image stabilization based on the processed jitter data;

[0279] b) The anti-shake reset function is on. When the anti-shake function fails, use the anti-shake reset function to reset the lens to its initial state to restore the anti-shake function.

[0280] Assume that the upper-layer algorithm module determines through scene detection that the conditions for entering long-exposure shooting are met. If the upper-layer algorithm module receives an operation to trigger taking a photo, the upper-layer algorithm module sends an instruction to the OIS algorithm module, instructing it to switch to long-exposure shooting mode and adopt the anti-shake strategy in long-exposure shooting mode for anti-shake.

[0281] Anti-shake strategy in long exposure shooting mode:

[0282] 1) Upper-level algorithm module: performs long-exposure shooting.

[0283] 2) Sensor output: The image frames collected within the first duration are discarded; after the first duration, the exposure parameters are adjusted and long-exposure image frames (marked as L in the figure) are collected.

[0284] 3) OIS status:

[0285] a) acquiring jitter data and performing low-pass filtering on the jitter data, and performing optical image stabilization based on the processed jitter data;

[0286] b) Turn off the anti-shake reset function to avoid blurring of long-exposure image frames caused by anti-shake reset.

[0287] c) After the long exposure shooting is completed, the anti-shake reset function is restarted to reset the exposure parameters and automatically switch to preview mode.

[0288] Figure 9 illustrates the implementation of the anti-shake method for shooting in a variable exposure shooting scenario, as provided in an embodiment of the present application. As shown in Figure 9, the anti-shake strategy in preview mode and shooting mode is explained through the upper-layer algorithm module, sensor image output, OIS status, and shake data.

[0289] The anti-shake strategy in preview mode is as described above and will not be repeated here.

[0290] Assume that the upper-layer algorithm module determines through scene detection that the conditions for entering variable exposure shooting are met. If the upper-layer algorithm module receives an operation to trigger taking a photo, the upper-layer algorithm module sends an instruction to the OIS algorithm module, instructing it to switch to variable exposure shooting mode and adopt the anti-shake strategy in variable exposure shooting mode for anti-shake.

[0291] Anti-shake strategy in variable exposure shooting mode:

[0292] 1) Upper-level algorithm module: performs variable exposure shooting.

[0293] 2) Sensor output: The image frames collected within the first duration are discarded; after the first duration, the exposure parameters are adjusted. The adjusted exposure parameters include long exposure parameters and short exposure parameters. The camera sensor alternately outputs long-exposure image frames and short-exposure image frames. The long-exposure image frames are marked as L, and the short-exposure image frames are marked as S.

[0294] 3) OIS status:

[0295] a) acquiring jitter data and performing low-pass filtering on the jitter data, and performing optical image stabilization based on the processed jitter data;

[0296] b) Turn off the anti-shake reset function to avoid blurring of long-exposure image frames caused by anti-shake reset.

[0297] c) After the long exposure shooting is completed, the anti-shake reset function is restarted to reset the exposure parameters and automatically switch to preview mode.

[0298] It should be noted that the difference between long-exposure shooting and variable-exposure shooting lies in the sensor output. During long-exposure shooting, long-exposure image frames are output, and images are generated based on these long-exposure image frames. During variable-exposure shooting, long-exposure image frames and short-exposure image frames are alternately output, and images are generated based on these long-exposure and short-exposure image frames.

[0299] The shooting anti-shake method provided in the embodiment of the present application can be applied to shooting scenes such as professional mode, night scene, and high dynamic range.

[0300] FIG10 shows a schematic diagram of an interface for applying the shooting anti-shake method provided in an embodiment of the present application.

[0301] As shown in Figure 10, scene detection is performed in preview mode to determine whether the current shooting scene meets the conditions for long exposure / variable exposure shooting. If the shooting scene meets the conditions for long exposure / variable exposure shooting, a specific anti-shake strategy is adopted for the shooting scene. When the electronic device receives a shooting instruction, it performs a series of anti-shake actions: switching from preview mode to shooting mode, disabling the anti-shake reset function, and obtaining jitter data. The jitter data is low-pass filtered to achieve a full-band anti-shake effect; and after waiting for a first period of time, the camera exposure parameters are adjusted to long exposure / variable exposure parameters, and the image is captured based on the adjusted exposure parameters. Due to the corresponding anti-shake processing, the long exposure frame captured in this case is less affected by jitter, and the final image is clear.

[0302] The shooting anti-shake method provided by the embodiment of the present application is used to identify whether the current scene is a specific exposure shooting scene in the preview stage. For specific exposure shooting scenes (such as long exposure shooting or variable exposure), since they are significantly affected by shaking, the anti-shake reset function is turned off after the shooting is triggered to avoid image blur caused by the anti-shake reset. The shaking data is filtered in the full frequency band and the orientation of the camera lens is adjusted based on the shaking data to achieve a full-band anti-shake effect. It can perform anti-shake processing for high-frequency shaking such as movement and touch screen shaking, as well as low-frequency shaking such as breathing and heartbeat. After the shooting is triggered to start, a first time period is waited to complete the reset of various parameters. After the state is stable, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters, so that the specific exposure shooting is greatly reduced from being affected by various possible shaking, thereby improving the clarity of image shooting.

[0303] It should be noted that, in the embodiments of the present application, "greater than" can be replaced by "greater than or equal to", "less than or equal to" can be replaced by "less than", or "greater than or equal to" can be replaced by "greater than", and "less than" can be replaced by "less than or equal to".

[0304] The various embodiments described herein may be independent solutions or may be combined according to internal logic, and all of these solutions fall within the scope of protection of this application.

[0305] The above mainly describes the solution provided by the embodiment of the present application from the perspective of method steps. It is understandable that, in order to achieve the above functions, the electronic device implementing the method includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of protection of this application.

[0306] In the embodiment of the present application, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other feasible division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0307] FIG11 is a schematic block diagram of a camera anti-shake device 400 according to an embodiment of the present application. The device 400 can be used to perform the actions performed by the electronic device in the above method embodiment. The device 400 includes an anti-shake unit 410 and a camera unit 420.

[0308] The anti-shake unit 410 is used to, after starting the camera application, identify in the preview phase that the current first scene meets the preset shooting conditions; receive a first operation triggered by the user to start shooting, and in response to the first operation, turn off the anti-shake reset function and adjust the orientation of the camera lens according to first jitter data; the first jitter data is data obtained by performing a first filtering process on the jitter data of the electronic device during the shooting phase; wherein the first filtering process is full-band filtering.

[0309] The shooting unit 420 is used to wait for a first time period after triggering the start of shooting, control the camera sensor to output image frames according to the adjusted exposure parameters, and generate a first image based on the output image frames; wherein the adjusted exposure parameters include a first exposure parameter, and the exposure time corresponding to the first exposure parameter is greater than the exposure time corresponding to the initial exposure parameter of the camera sensor.

[0310] The shooting anti-shake device provided in the embodiment of the present application identifies whether the current scene is a specific exposure shooting scene during the preview phase. For specific exposure shooting scenes (such as long exposure shooting or variable exposure), which are significantly affected by shaking, a series of anti-shake processing is immediately started after the shooting is triggered. The anti-shake reset function is disabled to avoid image blur caused by the anti-shake reset. In addition, a specific full-band filtering process is performed on the shaking data. The orientation of the camera lens is adjusted according to the processed shaking data to achieve a full-band anti-shake effect. For example, anti-shake processing can be performed for high-frequency shaking such as irregular shaking and touch screen shaking, as well as low-frequency shaking such as breathing and heartbeat. After the shooting is triggered, a first time period is waited to complete the reset of various parameters. After the state stabilizes, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters. This can avoid image blur caused by various possible shaking, thereby greatly reducing the impact of various possible shaking on specific exposure shooting and improving image clarity.

[0311] The device 400 according to the embodiment of the present application may be corresponding to executing the method described in the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0312] The present application also provides a chip, which is coupled to a memory and is used to read and execute computer programs or instructions stored in the memory to perform the methods in the above embodiments.

[0313] The present application also provides an electronic device, which includes a chip, and the chip is used to read and execute computer programs or instructions stored in a memory, so that the methods in each embodiment are executed.

[0314] This embodiment also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions run anti-shake reset on an electronic device, the electronic device executes the above-mentioned related method steps to implement the shooting anti-shake method in the above-mentioned embodiment.

[0315] This embodiment further provides a computer program product, wherein the computer-readable storage medium stores program code. When the computer program product runs on a computer, the computer executes the above-mentioned related steps to implement the shooting anti-shake method in the above-mentioned embodiment.

[0316] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0317] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0318] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0319] The term "user interface (UI)" in this application refers to the interface between an application or operating system and a user for interaction and information exchange. It converts information from its internal form into a user-friendly form. An application's UI is source code written in a specific computer language, such as Java or Extensible Markup Language (XML). This source code is parsed and rendered on the terminal device, ultimately presenting user-readable content such as images, text, buttons, and other controls.

[0320] A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operations that uses graphics. It can be an icon, window, control, or other interface element displayed on the display of an electronic device. In the camera preview interface, controls can include visual interface elements such as shooting controls.

[0321] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0322] In the specification and claims herein, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. In the description of the embodiments of this application, unless otherwise specified, the meaning of "plurality" refers to two or more. For example, "multiple processing units" refers to two or more processing units, etc.; "multiple components" refers to two or more components, etc.

[0323] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A shooting anti-shake method, applied to electronic equipment, characterized in that: The method comprises: Launch the Camera app; In the preview stage, it is identified that the current first scene meets the preset shooting conditions; receiving a first operation triggered by a user to start shooting, disabling an anti-shake reset function in response to the first operation, and adjusting the orientation of a camera lens according to first jitter data obtained by performing a first filtering process on jitter data of the electronic device during the shooting phase; After triggering to start shooting, wait for a first time period, control the camera sensor to output image frames according to the adjusted exposure parameters, and generate a first image based on the output image frames; The adjusted exposure parameter includes a first exposure parameter, and the exposure duration corresponding to the first exposure parameter is greater than the exposure duration corresponding to the initial exposure parameter of the camera sensor; The first filtering process is full-band filtering.

2. The method according to claim 1, characterized in that The preset shooting condition is a long exposure shooting condition; or the preset shooting condition is a variable exposure shooting condition.

3. The method according to claim 1 or 2, characterized in that The method further comprises: In response to the first operation, switching from a preview mode to a capture mode; The optical image stabilization motor of the electronic device is controlled to reset the camera lens to an initial position.

4. The method according to any one of claims 1 to 3, characterized in that The first filtering process is a low-pass filtering process, and the first shaking data includes a posture angle of the shaking of the electronic device.

5. The method according to any one of claims 1 to 4, characterized in that After starting the camera application, the method further includes: adjusting the orientation of the camera lens of the electronic device based on the second jitter data during the preview phase; wherein the second jitter data is data obtained by performing a second filtering process on the jitter data of the electronic device during the preview phase; Controlling the camera sensor to capture image frames according to the initial exposure parameters, and generating a preview image based on the captured image frames; The filtering frequency band of the second filtering process is smaller than the filtering frequency band of the first filtering process.

6. The method according to claim 5, characterized in that The second filtering process is a band-pass filtering process.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Controlling the camera sensor to output image frames according to second exposure parameters within the first time period after the trigger starts shooting; The exposure duration corresponding to the second exposure parameter is shorter than the exposure duration corresponding to the first exposure parameter.

8. The method according to claim 7, characterized in that The second exposure parameter is an initial exposure parameter of the camera sensor.

9. The method according to claim 7 or 8, characterized in that The method further includes: discarding the image frame output by the camera sensor according to the second exposure parameter; The generating the first image based on the output image frame includes: generating the first image based on the image frame output by the camera sensor according to the first exposure parameter.

10. The method according to any one of claims 1 to 9, characterized in that The first duration is determined based on the second duration, the first duration is greater than or equal to the second duration, and the second duration is the time required to complete the resetting of various parameters after the shooting is triggered; The resetting of various parameters includes: Modify the filtering method for jitter data; The anti-shake reset function is set to off.

11. The method according to claim 10, characterized in that The parameter reset also includes: Reset the camera lens to its initial position.

12. The method according to any one of claims 1 to 11, characterized in that The step of identifying that the current first scene meets the preset shooting condition during the preview phase includes: Acquiring ambient light brightness information and / or preview image brightness information during the preview phase; Scene detection is performed based on the ambient light brightness information and / or the brightness information of the preview image to identify that the first scene meets the preset shooting condition.

13. The method according to claim 12, characterized in that The brightness information includes a brightness value and a brightness range; The performing scene detection according to the ambient light brightness information and / or the brightness information of the preview image, and identifying that the first scene meets the preset shooting condition, includes: If the brightness value of the preview image collected in the preview stage is less than or equal to a first brightness threshold, and / or the ambient light brightness is less than or equal to a second brightness threshold, then it is recognized that the first scene meets the long exposure shooting condition; If the brightness range of the preview image collected in the preview stage is greater than the first brightness range threshold, and / or the ambient light brightness range is greater than the second brightness range threshold, it is recognized that the first scene meets the variable exposure shooting condition.

14. The method according to any one of claims 1 to 13, characterized in that The method further comprises: When it is identified in the preview stage that the first scene meets the preset shooting condition, exposure parameters required for shooting the first scene are determined.

15. The method according to claim 14, characterized in that The determining of exposure parameters required for photographing the first scene includes: Determine exposure parameters required for photographing the first scene according to the ambient light brightness information and / or the brightness information of the preview image.

16. The method according to claim 15, characterized in that Before controlling the camera sensor to output the image frame according to the adjusted exposure parameters, the method further includes: Adjusting exposure parameters of the camera sensor according to the determined exposure parameters required for photographing the first scene.

17. The method according to any one of claims 1 to 16, characterized in that In a case where the first scene meets a long exposure shooting condition, the adjusted exposure parameters include the first exposure parameters; When the first scene meets the variable exposure shooting condition, the adjusted exposure parameter further includes a second exposure parameter, and the exposure duration corresponding to the second exposure parameter is shorter than the exposure duration corresponding to the initial exposure parameter of the camera sensor.

18. The method according to claim 17, characterized in that Generating a first image based on the output image frame includes: When the first scene meets the long-exposure shooting condition, capturing a long-exposure image frame using the first exposure parameter, and generating the first image based on the long-exposure image frame; When the first scene meets the variable exposure shooting condition, long-exposure image frames are captured using the first exposure parameters, short-exposure image frames are captured using the second exposure parameters, and the first image is generated based on the long-exposure image frames and the short-exposure image frames.

19. The method according to any one of claims 1 to 18, characterized in that After generating the first image based on the output image frame, the method further includes: Switch from capture mode to preview mode; Adjusting the camera exposure parameters to the initial exposure parameters; Re-enable the anti-shake reset function.

20. The method according to any one of claims 1 to 19, characterized in that The method further comprises: In the case where it is identified during the preview phase that the first scene does not meet the preset shooting condition, receiving a second operation triggered by the user to start shooting; In response to the second operation, adjusting the orientation of the camera lens based on third jitter data, controlling the camera sensor to output image frames according to the initial exposure parameters, and generating a second image based on the output image frames; The third jitter data is data obtained by performing a second filtering process on the jitter data of the electronic device during the shooting phase.

21. The method according to claim 20, characterized in that The method further comprises: The anti-shake reset function is used to adjust the orientation of the camera lens.

22. The method according to any one of claims 1 to 19, characterized in that The first image is a picture or a video.

23. An electronic device, characterized in that: The electronic device includes: one or more processors, and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to perform the method according to any one of claims 1 to 22.

24. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 22.

25. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises instructions, which, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 22.

26. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 22.

Images