Shooting anti-shake method, electronic equipment and storage medium

Abstract

The invention provides a shooting anti-shake method, electronic equipment and a storage medium, and relates to the field of image processing. Whether a current scene is a specific exposure shooting scene or not is identified in a preview stage, and for the specific exposure shooting scene (such as long exposure shooting or variable exposure), because the specific exposure shooting scene is obviously influenced by jitter, an anti-jitter reset function is closed after shooting is started by triggering, so that image blurring caused by anti-jitter reset is avoided, and the image quality is improved. The method comprises the following steps: acquiring jitter data of a camera, performing full-band filtering processing on the jitter data, adjusting the orientation of a camera lens based on the jitter data, realizing a full-band anti-jitter effect, performing anti-jitter processing on high-frequency jitter such as mobile and touch screen jitter and low-frequency jitter such as breathing and heartbeat, waiting for a first time length after triggering to start shooting, and completing resetting of various parameters. And after the state is stable, the camera sensor is controlled to output the image frame according to the adjusted exposure parameter and generate the image, so that the influence of various possible jitters on specific exposure shooting is greatly reduced, and the image shooting definition is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a shooting anti-shake method, electronic device and storage medium. Background Art

[0002] With the rapid development of imaging technology, users' demands for clarity and stability in video recording are constantly increasing. During the shooting process, jitter is inevitable. For example, jitter can be caused by hand-held or touch jitter, or by movement of the electronic device during the shooting process. Due to jitter during the shooting process, motion blur may appear in the captured video or image.

[0003] To this end, related technologies implement anti-shake by adding an optical image stabilization (OIS) controller to electronic devices. The principle of anti-shake of the OIS controller is to use an OIS motor to move the lens, thereby offsetting the displacement caused by shaking.

[0004] However, in certain specific exposure shooting scenes, the amount of jitter is relatively large, which reduces the anti-shake performance of the OIS controller and causes blurred images. Summary of the Invention

[0005] The present application provides a shooting anti-shake method, electronic device and storage medium, which greatly reduces the impact of various possible shakes on specific exposure shooting and improves the clarity of image shooting.

[0006] In a first aspect, the present application provides a shooting anti-shake method, the method comprising: starting a camera application; identifying in a preview stage that a current first scene satisfies a preset shooting condition; receiving a first operation triggered by a user to start shooting, and in response to the first operation, turning off an anti-shake reset function, and adjusting the orientation of the camera lens according to first jitter data; the first jitter data is data obtained after a first filtering process is performed on the jitter data of the electronic device during the shooting stage; waiting for a first time after triggering the start of shooting, controlling the camera sensor to output image frames according to adjusted exposure parameters, and generating a first image based on the output image frames; wherein the adjusted exposure parameters include a first exposure parameter, and the exposure duration corresponding to the first exposure parameter is greater than the exposure duration corresponding to the initial exposure parameter of the camera sensor; wherein the first filtering process is full-band filtering.

[0007] The shooting stabilization method provided in the embodiments of the present application identifies whether the current scene is a specific exposure shooting scene during the preview phase. For specific exposure shooting scenes (such as long exposure shooting or variable exposure), which are significantly affected by shaking, a series of stabilization processes are immediately executed after the shooting is triggered. The stabilization reset function is disabled to avoid image blur caused by the stabilization reset. Specific full-band filtering is performed on the shaking data, and the orientation of the camera lens is adjusted based on the processed shaking data to achieve a full-band stabilization effect. For example, stabilization can be performed for high-frequency shaking such as irregular shaking and touch screen shaking, as well as low-frequency shaking such as breathing and heartbeat. After the shooting is triggered, a first time period is waited to complete the parameter reset. After the state stabilizes, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters. This can avoid image blur caused by various possible shaking, thereby greatly reducing the impact of various possible shaking on specific exposure shooting and improving image clarity.

[0008] In some possible implementations, after the first image is generated based on the output image frame, the first image may be stored in a gallery, wherein the first image may be a picture or a video.

[0009] In some possible implementations, the preset shooting condition is a long exposure shooting condition; or the preset shooting condition is a variable exposure shooting condition.

[0010] In related technologies, the specific anti-shake processing method during the preview phase is as follows: During the preview phase, the jitter data is bandpass filtered, and the camera lens position is adjusted based on the processed jitter data. The preview image is captured while the anti-shake processing is performed. This can avoid image blur caused by high-frequency jitter such as irregular jitter and touch screen jitter. In addition, the anti-shake reset function is enabled by default. When the anti-shake effect is not satisfactory or the anti-shake function fails, the electronic device will activate the anti-shake reset function and reset the lens module to its initial position via the OIS motor, for example, placing the lens module in the center axis position to restore the anti-shake function.

[0011] In related technologies, the anti-shake processing method used during the capture phase is the same as that used during the preview phase. After the capture is triggered, the jitter data is bandpass filtered, and the camera lens position is adjusted based on the processed jitter data. Image capture is performed while the anti-shake processing is in place. This prevents image blur caused by high-frequency jitter, such as irregular jitter and touch screen jitter. Furthermore, the anti-shake reset function is enabled. When the anti-shake effect fails to meet requirements or the anti-shake function fails, the electronic device activates the anti-shake reset function to restore the anti-shake function.

[0012] It should be noted that when the shaking amplitude is large, the pushing distance of the optical image stabilization motor on the lens module may reach its limit, causing the optical image stabilization function to fail. In this case, the electronic device will use the anti-shake reset function.

[0013] Compared with related technologies, the OIS algorithm module of this application adopts different anti-shake strategies for preview mode and different shooting scenes.

[0014] First, in preview mode, the gyroscope sensor acquires jitter data, which is then bandpass filtered. Optical image stabilization is then performed based on the processed jitter data. Simultaneously, the image is captured using preset exposure parameters to generate a preview image, which is displayed on the preview interface. Furthermore, in preview mode, scene detection is performed to determine whether the current shooting scene meets the requirements for long exposure or variable exposure shooting. Different image stabilization strategies are then applied to different shooting scenarios.

[0015] On the one hand, the current shooting scene meets the conditions for long exposure / variable exposure shooting:

[0016] When the electronic device receives a capture command, it switches from preview mode to capture mode. If the current scene meets the requirements for long-exposure / variable-exposure shooting, the corresponding anti-shake strategy includes: disabling the anti-shake reset function, acquiring shake data, and low-pass filtering the shake data to achieve full-band anti-shake effects; then, adjusting the camera exposure parameters to the long-exposure / variable-exposure parameters and capturing images based on these adjusted exposure parameters. Due to the corresponding anti-shake processing, the captured long-exposure frames in this case are less affected by shake. After the long-exposure / variable-exposure shooting is completed, the device switches from capture mode to preview mode, reactivates the optical image stabilization function, and resets the exposure parameters.

[0017] On the other hand, the current shooting scene does not meet the conditions for long exposure / variable exposure shooting:

[0018] When the electronic device receives a capture command, it switches from preview mode to capture mode, maintaining the same exposure parameters. The current capture scene is a standard capture scene. The corresponding stabilization strategy includes: using the stabilization reset function as normal, acquiring jitter data, and performing bandpass filtering on the jitter data to achieve stabilization for high-frequency jitter; and capturing the image using the preset exposure parameters. The normally exposed image captured in this scenario achieves stabilization for some high-frequency jitter and exhibits good hand tracking. After the capture is complete, the device switches from capture mode back to preview mode.

[0019] In some embodiments, scenes such as low-light or night scenes meet the conditions for long-exposure shooting.

[0020] For example, if the electronic device detects that the preview image contains night scene features, the electronic device determines that the current scene meets the conditions for long exposure shooting. For example, if the electronic device detects that the ambient light brightness is lower than a preset brightness value, the electronic device determines that the current scene meets the conditions for long exposure shooting.

[0021] In some embodiments, low-light or backlit scenes, such as backlit shots, or scenes like sunrises, sunsets, and skyscapes, meet the requirements for variable exposure shooting. This is because such scenes contain both very bright and very dark areas (i.e., both overexposed and underexposed areas). Such scenes are suitable for HDR shooting mode, which achieves variable exposure shooting by fusing long and short exposure frames.

[0022] For example, if the electronic device detects that the preview image contains a distinct bright and dark area, the electronic device determines that the current scene meets the conditions for variable exposure shooting. For example, if the electronic device detects that the ambient light is distinct bright and dark, the electronic device determines that the current scene meets the conditions for variable exposure shooting.

[0023] In other embodiments, the electronic device may determine, based on a triggering operation by the user, whether to trigger and enable long exposure or variable exposure shooting, thereby determining whether the current scene meets the conditions for long exposure or variable exposure shooting.

[0024] For example, after a user taps the "Professional Mode" control, the electronic device can use long exposure or variable exposure shooting. For another example, when the electronic device receives a user trigger to enable "long exposure shooting," the electronic device will use long exposure shooting. For another example, when the electronic device receives a user trigger to enable "HDR shooting," the electronic device will use variable exposure shooting.

[0025] In some possible implementations, when the first scene meets the long exposure shooting condition, the adjusted exposure parameters include the first exposure parameters; when the first scene meets the variable exposure shooting condition, the adjusted exposure parameters also include a second exposure parameter, and the exposure duration corresponding to the second exposure parameter is shorter than the exposure duration corresponding to the initial exposure parameter of the camera sensor.

[0026] In some possible implementations, generating the first image based on the output image frames includes: when the first scene meets the long-exposure shooting condition, capturing long-exposure image frames using the first exposure parameters, and generating the first image based on the long-exposure image frames; when the first scene meets the variable-exposure shooting condition, capturing long-exposure image frames using the first exposure parameters, capturing short-exposure image frames using the second exposure parameters, and generating the first image based on the long-exposure image frames and the short-exposure image frames.

[0027] In some possible implementation manners, the method further includes: switching from a preview mode to a shooting mode in response to the first operation; and controlling an optical image stabilization (OIS) motor of the electronic device to reset the camera lens to an initial position.

[0028] Since the long-exposure shooting or the variable-exposure shooting is obviously affected by the shaking, it is necessary to close the OIS reset function at the moment of triggering the long-exposure shooting or the variable-exposure shooting, so as to avoid the image blur caused by the OIS reset and the large movement of the camera lens, and the camera lens can be reset to the initial position, so that the OIS function can be normally used after the OIS reset function is closed.

[0029] In some possible implementation manners, the first filter processing is low-pass filter processing, and the first shaking data includes an attitude angle of shaking of the electronic device. For example, an infinite impulse response (IIR) filter such as a Butterworth filter or a Chebyshev filter can be used. The IIR filter can realize the filter process through recursive operation. By modifying the parameters of each order, the modification of the filter band can be realized.

[0030] In some possible implementation manners, after the camera application is started, the method further includes: adjusting the orientation of the camera lens of the electronic device based on second shaking data in a preview stage; and controlling the camera sensor to collect an image frame according to the initial exposure parameter, and generating a preview image based on the collected image frame. The second shaking data is data obtained by performing second filter processing on shaking data of the electronic device in the preview stage. The filter band of the second filter processing is smaller than the filter band of the first filter processing.

[0031] In some possible implementation manners, the second filter processing is band-pass filter processing. Through the band-pass filter processing, the high-frequency shaking such as irregular shaking and touch screen shaking can be processed, the response speed is fast, and the hand following property is good.

[0032] In some possible implementation manners, the method further includes: controlling the camera sensor to output an image frame according to a second exposure parameter within the first time length after triggering the shooting; and the exposure time corresponding to the second exposure parameter is less than the exposure time corresponding to the first exposure parameter.

[0033] In some possible implementation manners, the second exposure parameter is an initial exposure parameter of the camera sensor.

[0034] In some possible implementation manners, the method further includes discarding the image frame output by the camera sensor according to the second exposure parameter. The generating the first image based on the output image frame includes generating the first image based on the image frame output by the camera sensor according to the first exposure parameter.

[0035] In some possible implementation manners, the first time length is determined according to a second time length, and the first time length is greater than or equal to the second time length, the second time length being a time required for completing the resetting of the parameters after triggering the start of shooting. After the first time length, the filter mode switching is completed and the anti-shake reset function is closed, reaching a stable state, and in this case, the long-exposure frame is collected, which can effectively avoid the blur of the long-exposure image frame caused by the anti-shake reset, greatly reducing the influence of the shaking on the long-exposure shooting.

[0036] Exemplarily, the first time length can be 100 milliseconds.

[0037] In some possible implementation manners, the resetting of the parameters includes modifying a filter mode of the shaking data and setting the anti-shake reset function to a closed state. In this case, the second time length is a sum of a time required for modifying the filter mode of the shaking data and a time required for setting the anti-shake reset function to the closed state.

[0038] Exemplarily, after triggering the long-exposure shooting or the variable exposure shooting, the filter mode of the shaking data is modified to a low-pass filter mode.

[0039] In some possible implementation manners, the resetting of the parameters further includes resetting the camera lens to an initial position. In this case, the second time length is a sum of a time required for modifying the filter mode of the shaking data, a time required for setting the anti-shake reset function to the closed state, and a time required for resetting the camera lens to the initial position.

[0040] Through the above scheme, after triggering the start of shooting, the first time length is waited, the resetting of the parameters is completed, and after the state is stable, the camera sensor is controlled to output the image frame according to the adjusted exposure parameter and generate the image, which can avoid the image blur caused by various possible shaking, thereby greatly reducing the influence of the specific exposure shooting by various possible shaking and improving the image shooting clarity.

[0041] In some possible implementation manners, the identifying that the current first scene satisfies the preset shooting condition in the preview stage includes: acquiring environment light brightness information and / or brightness information of a preview image in the preview stage; and performing scene detection according to the environment light brightness information and / or the brightness information of the preview image to identify that the first scene satisfies the preset shooting condition.

[0042] In some possible implementations, the brightness information includes a brightness value and a brightness range.

[0043] The performing scene detection based on the ambient light brightness information and / or the brightness information of the preview image to identify that the first scene meets the preset shooting condition includes: if the brightness value of the preview image collected in the preview stage is less than or equal to a first brightness threshold, and / or the ambient light brightness is less than or equal to a second brightness threshold, then identifying that the first scene meets the long exposure shooting condition; if the brightness range of the preview image collected in the preview stage is greater than a first brightness range threshold, and / or the ambient light brightness range is greater than a second brightness range threshold, then identifying that the first scene meets the variable exposure shooting condition.

[0044] In some possible implementations, the method further includes: determining exposure parameters required for photographing the first scene when it is identified in the preview stage that the first scene meets the preset photographing condition.

[0045] In some possible implementations, determining the exposure parameters required for photographing the first scene includes: determining the exposure parameters required for photographing the first scene according to ambient light brightness information and / or brightness information of the preview image.

[0046] In some possible implementations, before controlling the camera sensor to output image frames according to the adjusted exposure parameters, the method further includes: adjusting the exposure parameters of the camera sensor according to the determined exposure parameters required for capturing the first scene.

[0047] In some possible implementations, after generating the first image based on the output image frame, the method further includes: switching from a shooting mode to a preview mode; adjusting the camera exposure parameters to the initial exposure parameters; and re-enabling the anti-shake reset function.

[0048] It should be noted that after the current shooting is completed, you will switch back to preview mode from long exposure shooting or variable exposure shooting. Therefore, it is necessary to restart the anti-shake reset function to ensure that the anti-shake function corresponding to the preview mode can be used normally during the preview stage.

[0049] By the scheme, in the preview mode, the dithering data is band-pass filtered through band-pass filtering, and the camera lens is controlled to move based on the processed dithering data to achieve an anti-shake effect; meanwhile, a preview image is acquired by using zero shutter lag (ZSL) in the preview mode. In addition, scene detection is performed in the preview stage, and it is determined whether to start long exposure or variable exposure shooting according to the scene detection result. In the case of starting long exposure or variable exposure shooting, the IOS algorithm module switches from the preview mode to the shooting mode, the dithering data is filtered through full-band low-pass filtering, and the camera lens is controlled to move based on the processed dithering data to achieve an anti-shake effect. After waiting for a first time length, the sensor exposure parameter is changed, and a long exposure image is acquired by using non-ZSL. In this way, the influence of long exposure or variable exposure shooting on various possible dithers is greatly reduced, and the image shooting clarity is improved. After the long exposure or variable exposure shooting ends, the shooting mode is switched to the preview mode again. For long exposure scenes and variable exposure scenes, the scheme can effectively improve the anti-shake effect of the image, reduce the motion blur in the image, and ensure the image clarity.

[0050] In some possible implementation ways, the method further includes: in the case that the first scene is identified not to satisfy the preset shooting condition in the preview stage, receiving a second operation of a user triggering to start shooting; and in response to the second operation, adjusting the orientation of the camera lens based on third dithering data, and controlling the camera sensor to output an image frame according to the initial exposure parameter, and generating a second image based on the output image frame; wherein the third dithering data is data obtained by performing second filtering processing on dithering data of the electronic device in the shooting stage.

[0051] In some possible implementation ways, the method further includes: adjusting the orientation of the camera lens by using the anti-shake reset function.

[0052] In a second aspect, the present application provides a shooting anti-shake device, which includes units for executing the method in the first aspect. The device can correspond to the method described in the first aspect, and the related description of the units in the device is referred to the description of the first aspect. For brevity, the description is not repeated here.

[0053] The method described in the first aspect can be implemented by hardware, or can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, processing modules or units, display modules or units, and the like.

[0054] In a third aspect, the present application provides an electronic device, which comprises a processor, and a computer program or instructions stored in the memory and readable by the processor, wherein the processor is configured to execute the computer program or instructions so as to perform the method in the first aspect.

[0055] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program (also referred to as instructions or codes) for implementing the method in the first aspect. For example, when the computer program is executed by a computer, the computer can perform the method in the first aspect.

[0056] In a fifth aspect, the present application provides a chip, which comprises a processor. The processor is configured to read and execute a computer program stored in a memory so as to perform the method in the first aspect and any possible implementation manner thereof. Optionally, the chip further comprises the memory, which is connected to the processor by a circuit or a wire.

[0057] In a sixth aspect, the present application provides a chip system, which comprises a processor. The processor is configured to read and execute a computer program stored in a memory so as to perform the method in the first aspect and any possible implementation manner thereof. Optionally, the chip system further comprises the memory, which is connected to the processor by a circuit or a wire.

[0058] In a seventh aspect, the present application provides a computer program product, which comprises a computer program (also referred to as instructions or codes). When the computer program is executed by an electronic device, the electronic device can implement the method in the first aspect.

[0059] It can be understood that the beneficial effects of the second aspect to the seventh aspect described above can be referred to the related description of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 An interface diagram of a photographing image module caused by shaking during actual photographing;

[0061] Figure 2 A structural diagram of an electronic device provided by an embodiment of the present application;

[0062] Figure 3 An optical anti-shaking principle diagram provided by an embodiment of the present application;

[0063] Figure 4 A software architecture diagram of an electronic device provided by an embodiment of the present application;

[0064] Figure 5 A shaking data diagram detected by a gyroscope sensor provided by an embodiment of the present application;

[0065] Figure 6 A flowchart of a method for stabilizing shooting provided by an embodiment of the present application;

[0066] Figure 7 A flowchart of a method for stabilizing shooting provided by an embodiment of the present application;

[0067] Figure 8 A schematic diagram of an application scenario of a shooting anti-shake method provided in an embodiment of the present application;

[0068] Figure 9 A schematic diagram of an application scenario of another shooting anti-shake method provided in an embodiment of the present application;

[0069] Figure 10 A schematic diagram of an application scenario interface of another shooting anti-shake method provided in an embodiment of the present application;

[0070] Figure 11 A schematic structural diagram of a shooting anti-shake device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the objectives, technical solutions and advantages of this application more clear, the embodiments of this application are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0072] First, some terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0073] 1. Exposure time

[0074] Exposure time refers to the time it takes for the shutter to open and close. The length of exposure time will affect the clarity of the photo.

[0075] 2. Long exposure shooting

[0076] Long exposure photography, also known as slow shutter photography, creates a flowing or blurred effect by capturing the trajectory of a moving object over a period of time by extending the exposure time (for example, a long exposure time of 5 seconds or longer).

[0077] When users use a handheld device for long exposures, factors such as device stability and slight disturbances in the air may cause blur or vibration in the photo, affecting its clarity. Furthermore, long exposure times can increase noise, affecting image quality.

[0078] 3. Variable exposure shooting

[0079] Variable exposure shooting means that the sensor simultaneously outputs long-exposure images and short-exposure images, and forms a high dynamic range (HDR) image by fusing the long and short exposure images.

[0080] In some shooting scenes with large differences in brightness levels, the ordinary CMOS / CCD image sensor cannot fully present the real scene. The brighter area in the scene will appear a piece of white due to overexposure, and the darker area in the scene will appear a piece of darkness due to underexposure. For this case, variable exposure shooting can be used to present dark part information and ensure that the bright part is not overexposed, and the brightness information is retained.

[0081] 4. Motion blur

[0082] In the process of shooting by an electronic device, the generation of each frame of image relies on the accumulation of photons in the exposure time, and the photons are converted into electrons through photoelectric conversion, and further transformed into an image recognizable by the human eye. During this time, if the electronic device has a large amplitude of motion, the motion information will also be accumulated and recorded, and the generated image will be accompanied by strong motion blur.

[0083] 5. Optical image stabilization (OIS)

[0084] Optical image stabilization can also be referred to as optical stabilization. OIS technology refers to detecting the shaking of an electronic device by a motion sensor (for example, a gyroscope, an accelerometer) during shooting exposure. An OIS controller controls a motor that pushes OIS to move a lens or an image sensor according to the shaking data detected by the motion sensor, so that the light path remains as stable as possible during the entire exposure period, and a clear exposure image is obtained.

[0085] Optical image stabilization includes two anti-shake modes. The first is lens moving optical image stabilization, and the second is photosensitive element moving optical image stabilization. The principle of the first lens moving optical image stabilization is to detect a small movement by a gyroscope sensor in the lens, then transmit the signal to a microprocessor, the microprocessor immediately calculates the displacement amount that needs to be compensated, and then compensates the lens group according to the shaking direction and displacement amount of the lens, so as to effectively overcome the image blur caused by the shaking of the camera. The second photosensitive element moving optical image stabilization uses image sensor offset to achieve anti-shake. The principle is as follows: first, place the CCD on a bracket that can move up, down, left and right, and then when the gyroscope sensor detects shaking, the direction, speed and movement amount of the shaking are processed to calculate the CCD movement amount sufficient to offset the shaking.

[0086] Optionally, the OIS controller includes two-axis and three-axis optical image stabilizers. The embodiments of the present application take the two-axis lens moving optical image stabilization OIS as an example for description, and involve two-axis data, which will not be described hereinafter.

[0087] When users use electronic devices to shoot, jitter is inevitable. For example, jitter can refer to hand-held jitter or touch jitter when the user is shooting, or jitter can refer to the movement of the electronic device during the shooting process. Figure 1 As shown, a user holds an electronic device (such as a mobile phone) to shoot, and due to shaking during the shooting process, the captured video or image is blurred, affecting the clarity of the image.

[0088] Specifically, jitter mainly comes from:

[0089] 1. Irregular internal and external shaking, such as muscle tremors, pedestrians passing by, and the slightest movement of the wind.

[0090] 2. Touching the screen at the moment of taking a photo causes shaking.

[0091] 3. Regular disturbances of human breathing, heartbeat, etc.

[0092] Table 1

[0093] Source of jitter Frequency (Hz) Amplitude (°) Random jitter 0～10 0.1～0.2 Jitter caused by touching the screen 3～5 0.2～2 Regular jitter 0.5～2 0.4～1

[0094] Table 1 shows the frequencies and amplitudes of the three jitter sources.

[0095] Among them, irregular jitter corresponds to the widest frequency range and the smallest amplitude. This part of jitter will affect the imaging quality to a certain extent. The current anti-shake technology can achieve partial anti-shake for irregular jitter.

[0096] Among them, the frequency of jitter caused by the touch screen is higher than the frequency of regular jitter, and the amplitude of the jitter caused by the touch screen is relatively large. This part of the jitter usually seriously affects the image quality. The current anti-shake technology mainly targets the jitter caused by the touch screen, which greatly reduces the impact of the jitter caused by the touch screen on the image quality.

[0097] Among them, the frequency of regular jitter is relatively low and the amplitude is large, which will also have a certain degree of impact on the imaging quality.

[0098] To mitigate the effects of hand shake, electronic devices typically use optical image stabilization (OIS) to reduce motion blur caused by hand shake and improve the success rate of photo capture. Furthermore, to enhance the hand-tracking experience, current OIS algorithms have added a bandpass filter function, which provides almost no stabilization for low-frequency vibrations. Furthermore, when the user significantly moves the electronic device, the OIS motor pushes the lens to its maximum distance to overcome the effects of shaking. To avoid affecting the subsequent stabilization effect, the current OIS algorithm has added an stabilization reset (pan / tilt) function that pulls the lens back to the center and restores stabilization capabilities.

[0099] Current camera stabilization features can help produce clearer images and minimize blur caused by camera shake. Stabilization technology uses sensors to detect shake and compensates by moving the lens element or the sensor itself. However, when shooting long exposures, if the camera stabilization feature is running, even if the camera is not moving, the stabilization system will try to compensate for the shake by moving the lens element or sensor, which will actually cause the image to shake or blur.

[0100] In some special scenes, image clarity can still be affected by significant camera jitter. For example, in bright scenes, short exposure times result in minimal camera jitter and a high image quality. However, in low-light scenes, prolonged exposure times can dramatically increase camera jitter, leading to poor image clarity and a significant drop in image quality. Similarly, when shooting with variable exposure, long-exposure images can be easily blurred due to the dramatic increase in camera jitter.

[0101] Therefore, the current technical challenges are: on the one hand, when users use handheld electronic devices to shoot long or variable exposures, the camera must be kept stable, otherwise even the slightest shake will affect the photo, so effective anti-shake technology is needed for this scenario. On the other hand, when using long or variable exposures, the amount of shake can increase dramatically, significantly affecting image clarity. However, current camera anti-shake functions require frequent refocusing of the lens, which causes delays and results in poor anti-shake effectiveness.

[0102] Currently, there is an urgent need for an anti-shake method for long exposure shooting and variable exposure shooting to improve the anti-shake effect and enhance image quality.

[0103] In view of the above problems, an embodiment of the present application proposes a shooting stabilization method. In the preview stage, it is identified whether the current scene is a specific exposure shooting scene. For specific exposure shooting scenes (such as long exposure shooting or variable exposure), since they are significantly affected by shaking, a series of stabilization processes are immediately started after the shooting is triggered. The stabilization reset function is disabled to avoid image blur caused by the stabilization reset. In addition, a specific full-band filtering process is performed on the shaking data. The orientation of the camera lens is adjusted according to the processed shaking data to achieve a full-band stabilization effect. For example, stabilization can be performed for high-frequency shaking such as irregular shaking and touch screen shaking, as well as low-frequency shaking such as breathing and heartbeat. After the shooting is triggered, a first time period is waited for to complete the reset of various parameters. After the state stabilizes, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters. This can avoid image blur caused by various possible shaking, thereby greatly reducing the impact of various possible shaking on specific exposure shooting and improving image shooting clarity.

[0104] In some embodiments, when the iOS algorithm module switches from the preview mode to the shooting mode, if it is detected that the iOS motor has turned on the anti-shake reset function, the iOS algorithm module can turn off the anti-shake reset function.

[0105] In some embodiments, when the IOS algorithm module switches from the preview mode to the shooting mode, if it is detected that the camera lens of the electronic device is not in the center position, the IOS algorithm module centers the camera lens through the IOS motor.

[0106] In some embodiments, when the iOS algorithm module switches from preview mode to capture mode, the iOS algorithm module waits for a first duration before changing the sensor exposure parameters and acquiring a long-exposure image. For example, the first duration can be 100 milliseconds. It should be noted that mode switching requires a certain amount of stabilization time. Changing the sensor exposure parameters and acquiring a long-exposure image when the state is stable can help improve image stabilization.

[0107] In some embodiments, when the iOS algorithm module switches from the preview mode to the shooting mode, the iOS algorithm module notifies the upper-layer camera application to delay for the first duration.

[0108] In some embodiments, when the iOS algorithm module switches from preview mode to capture mode, the iOS algorithm module uses an infinite impulse response (IIR) filter for real-time filtering. The IIR filter implements the filtering process through recursive operation. For example, the IIR filter can be a Butterworth filter or a Chebyshev filter, and the filter frequency band can be modified by modifying the parameters of each order.

[0109] Below we first combine the attached Figure 2 The hardware system of the electronic device provided in the embodiments of the present application is described.

[0110] Illustratively, the electronic device provided in the embodiments of the present application may be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc. The embodiments of the present application do not impose any restrictions on the specific type of electronic device.

[0111] For the sake of convenience, Figure 2Taking the electronic device 100 as a mobile phone as an example, its hardware system is illustrated.

[0112] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0113] It should be noted that Figure 2 The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 2 More or fewer components than those shown, or the electronic device 100 may include Figure 2 Combinations of some of the components shown, or alternatively, the electronic device 100 may include Figure 2 Subassemblies of some of the components shown. Figure 2 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0114] The processor 110 can include one or more processing units. For example, the processor 110 can include at least one of an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated devices. The controller can generate operation control signals according to instruction opcodes and timing signals, and complete control of fetching and executing instructions.

[0115] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that have just been used or are recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0116] In the embodiments of the present application, the code for implementing the anti-shake photographing method described in the embodiments of the present application can be stored on a non-volatile memory. When the camera application is running, the electronic device 100 can load the executable code stored in the non-volatile memory to the random access memory.

[0117] In the embodiments of the present application, the processor 110 can execute the following steps: starting a camera application; identifying that a current first scene satisfies a preset photographing condition in a preview stage; receiving a first operation of triggering a user to start photographing, in response to the first operation, closing an anti-shake reset function, and adjusting an orientation of a camera lens according to first jitter data; the first jitter data is data obtained by performing first filtering processing on jitter data of the electronic device in a photographing stage; waiting for a first time duration after triggering the start of photographing, controlling a camera sensor to output an image frame according to an adjusted exposure parameter, and generating a first image based on the output image frame; wherein the adjusted exposure parameter includes a first exposure parameter, and an exposure time corresponding to the first exposure parameter is greater than an exposure time corresponding to an initial exposure parameter of the camera sensor; wherein the first filtering processing is full-band filtering.

[0118] Figure 2The connection relationship between the illustrated modules is only illustrative and does not constitute a limitation on the connection relationship between the modules of the electronic device 100. Alternatively, the modules of the electronic device 100 can also adopt a combination of the above-mentioned various connection modes.

[0119] The electronic device 100 can realize the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information. The display screen 194 can be used to display images or videos.

[0120] The electronic device 100 can realize the shooting function through the image signal processor ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0121] In the embodiments of the present application, the image signal processor ISP is used to process the data fed back by the camera 193. For example, when shooting, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the image signal processor ISP for processing to convert it into an image visible to the naked eye. The image signal processor ISP can also optimize the algorithm of the noise, brightness, and skin color of the image. The image signal processor ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the image signal processor ISP can be arranged in the camera 193.

[0122] In the embodiments of the present application, the camera 193 is used to capture still images or videos. Objects generate optical images through lenses and project them onto photosensitive elements. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the image signal processor ISP to convert it into a digital image signal. The image signal processor ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, or other format image signal. In some embodiments, the electronic device 100 can include one or N cameras 193, N being a positive integer greater than 1.

[0123] In embodiments of the present application, the digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0124] The gyroscope sensor 180B, also known as an angular velocity sensor, can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used in scenarios such as anti-shake shooting, navigation, and motion sensing games.

[0125] In embodiments of the present application, the gyroscope sensor 180B can be used to collect jitter information, which can be used to represent the pose change of the electronic device during shooting. The jitter information can include the rotational angular velocity when the electronic device is deflected and tilted. Then, by discretely integrating the rotational angular velocity, the angle is obtained.

[0126] In embodiments of the present application, the gyroscope sensor 180B can be used for shooting anti-shake. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 jitter, and calculates the distance that the lens module needs to compensate according to the angle, so that the lens offsets the jitter of the electronic device 100 through reverse motion, realizing anti-shake.

[0127] The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 100 in each direction (generally the x-axis, the y-axis, and the z-axis). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 180E can also be used to identify the posture of the electronic device 100, as an input parameter for horizontal and vertical screen switching and step counter applications.

[0128] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the display screen 194 brightness according to the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when shooting. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent false touch.

[0129] In embodiments of the present application, the ambient light sensor 180L can sense the ambient light brightness, and the processor 110 can determine whether to perform long exposure or variable exposure shooting based on the ambient light brightness.

[0130] The touch sensor 180K is also referred to as a touch-sensitive device. The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also referred to as a touch screen. The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor 180K can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100 and at a different location from the display screen 194.

[0131] In the embodiment of the present application, the electronic device 100 can use the touch sensor 180K to detect the user's click, slide, and other operations on the display screen 194. For example, the touch sensor 180K can detect the user's touch operation on the shooting control, and the processor 110 can execute the corresponding anti-shake strategy and image capture in response to the touch operation.

[0132] Taking a mobile phone with the above hardware system as an example, the camera of the mobile phone is introduced in detail below.

[0133] Figure 3 A schematic diagram of a side view structure corresponding to a camera is shown.

[0134] For example, a camera generally includes an optical lens and a photosensitive element (also called an image sensor), etc. The optical lens may include one or more lenses. The optical lens uses the refraction principle of the lens to transmit light so that the light forms an image on the photosensitive element.

[0135] It should be understood that the “lens” mentioned in this application can be understood as a whole lens, which may include one or more lenses, and the “lens” can be understood as a lens in a lens structure or a lens or lens used to constitute a lens.

[0136] In addition, a filter (not shown) may be included between the lens and the photosensitive element. The filter is used to filter out unwanted wavelengths in the light, preventing the photosensitive element from generating false colors or ripples, thereby improving its effective resolution and color reproduction. Of course, this is only an example, and the camera may also include other structures, which are not limited in this application.

[0137] In conjunction with this example, in order to perform optical image stabilization, an OIS controller can be added to the camera lens. The OIS controller includes an OIS motor (not shown in the figure). The OIS controller is used to obtain the jitter data of the electronic device collected by the gyroscope sensor, such as angular velocity, and generate a control signal for controlling the movement of the OIS motor based on the jitter data collected by the gyroscope sensor. The OIS motor is used to drive the lens to move under the control of the control signal, thereby offsetting the displacement caused by the jitter.

[0138] See also Figure 3 Explain how the OIS motor's optical image stabilization is implemented. Figure 3 Taking the coordinate system shown in as an example, the OIS motor in the OIS controller can push the lens to move left and right, that is, to move in the x-axis direction, to offset the displacement caused by the jitter in the x-axis direction. The OIS motor in the OIS controller can push the lens to move up and down, that is, to move in the y-axis direction, to offset the displacement caused by the jitter in the y-axis direction. It should be understood that the OIS controller can control the lens to move in both the x-axis direction and the y-axis direction in combination with compensation requirements, thereby offsetting the displacement caused by the jitter. The specific direction and distance of movement can be determined according to requirements and are not limited in the embodiments of the present application.

[0139] The above introduces the implementation method of optical image stabilization. The reset function of optical image stabilization is described below. When the camera is enabled, the default anti-shake reset function is on. Specifically for this application, when long exposure or variable exposure shooting is adopted, the IOS algorithm module switches from preview mode to shooting mode. In this case, the IOS algorithm module can turn off the anti-shake reset function, that is, turn off the conventional optical image stabilization function. This is because: for specific exposure shooting scenes (such as long exposure shooting or variable exposure), it is significantly affected by the jitter. Therefore, a series of anti-shake processing is immediately started after the shooting is triggered, and the anti-shake reset function is turned off to avoid image blur caused by anti-shake reset. Then, after the photo is taken, switch back from shooting mode to preview mode. In this case, the IOS algorithm module can restart the anti-shake reset function.

[0140] It should be noted that the IOS algorithm module can send anti-shake instructions to the OIS controller, and the OIS motor of the OIS controller controls the movement of the camera lens according to the anti-shake instructions. The IOS algorithm module and OIS controller can be two independent modules or an integrated module.

[0141] The hardware system of electronic device 100 has been described in detail above. The software system of electronic device 100 will now be described. A software operating system runs on top of the hardware system. The software operating system can be any one or more computer operating systems that implement business processing through processes. Application programs can be installed and run on the software operating system.

[0142] Figure 4 Schematic diagram of the system architecture of the electronic device provided in the embodiment of the present application. Figure 4 As shown, the system architecture may include, from top to bottom, an application layer 210 , an application framework layer 220 , a hardware abstraction layer (HAL) 230 , a driver layer 240 and a hardware layer 250 .

[0143] The application layer 210 can include a series of application packages. In the embodiments of the present application, the application packages can include a camera application, a gallery, and the like.

[0144] The application framework layer 220 provides an application programming interface (API) and a programming framework for the application programs of the application layer; the application framework layer can include some predefined functions.

[0145] In the embodiments of the present application, the application framework layer 220 can include a camera access interface; the camera access interface is used to provide an application programming interface and a programming framework for the camera application. The camera access interface can include camera management and a camera device. The camera management can be used to provide an access interface for managing the camera; the camera device can be used to provide an interface for accessing the camera.

[0146] The hardware abstraction layer 230 is an interface layer between the application framework layer and the driver layer, and provides a virtual hardware platform for the operating system.

[0147] In the embodiments of the present application, the hardware abstraction layer 230 includes a camera hardware abstraction layer and a camera algorithm. The camera hardware abstraction layer can call the camera algorithm; the camera algorithm can include a software algorithm for image processing. The camera hardware abstraction layer 230 can provide a virtual hardware of the camera device.

[0148] Exemplarily, the camera algorithm library can include a running code and data of the shooting anti-shake method (or referred to as an OIS algorithm) provided in the embodiments of the present application.

[0149] Exemplarily, the algorithm in the camera algorithm can refer to an implementation independent of specific hardware; for example, a code that can usually run in a CPU, and the like.

[0150] The driver layer 240 is a layer between hardware and software. The driver layer includes drivers of various hardware, and is used to provide drivers for different hardware devices.

[0151] In the embodiments of the present application, the driver layer can include a camera device driver, a digital signal processor driver, and an image processor driver, and the like. The camera device driver (for example, a camera driver) is used to drive the sensor of the camera to collect images and drive the image signal processor to pre-process the images. The digital signal processor driver is used to drive the digital signal processor to process the images. The image processor driver is used to drive the graphics processor to process the images.

[0152] The hardware layer 250 is located at the bottom of the operating system; as shown in the figure, the hardware layer 250 can include a camera. Figure 4 ​

[0153] In the embodiment of the present application, the application layer may further include a context awareness module, a business logic processing module, and a business presentation module. The context awareness module, business logic processing module, and business presentation module may be independent apps, or may be integrated into different apps, or may be integrated into the same app, and this application does not limit this.

[0154] Among them, the context awareness module runs permanently or in a low-power form and has the ability to perceive external facts or environments. The context awareness module can detect related events and obtain the status of events from other applications in the application layer or the application framework layer or the system layer or the kernel layer through the application programming interface (API), such as detecting Bluetooth connections, network connections, monitoring user text messages, custom timers, etc. In an embodiment of the present application, the main function of the context awareness module is to monitor whether the camera application is started and perform scene detection in the camera preview mode and identify whether long exposure or variable exposure shooting is used, as well as monitor whether the user triggers shooting. The context awareness module can notify the business logic processing module of the monitored "long exposure or variable exposure shooting" event, and notify the business logic processing module of the "user triggered shooting" event.

[0155] The business logic processing module (e.g., a computing engine) has business logic processing capabilities and is used to perform data processing or logical judgment based on events monitored by the context perception module and preset logical algorithms to achieve anti-shake shooting. Specifically, when it is determined that "long exposure or variable exposure shooting" and "user-triggered shooting" are used, the bandpass filter in preview mode is switched to the full-band low-pass filter in shooting mode. The lens is centered through the OIS motor and the anti-shake reset function is turned off. After waiting for a first period of time, long exposure or variable exposure parameters are used to capture long exposure image frames, and the first image is obtained based on the long exposure image frames. The specific judgment process will be described in detail below.

[0156] The business presentation module (eg, YOYO suggestion) is configured to store the first image in a gallery and display a thumbnail of the first image on a camera preview interface according to instructions from the business logic processing module.

[0157] It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0158] The following describes the workflow of the software system and hardware system of the electronic device 100 in conjunction with a photo-taking scenario.

[0159] When a user performs a touch operation on touch sensor 180K, a corresponding hardware interrupt is sent to the kernel layer, which processes the touch operation into a raw input event. The raw input event includes, for example, information such as the touch coordinates and the timestamp of the touch operation. The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer, identifies the control corresponding to the raw input event, and notifies the application (APP) corresponding to the control. For example, if the touch operation is a single click, and the APP corresponding to the control is a camera APP, after the camera APP is awakened by the single click operation, it can call the kernel layer's camera driver through the API, and the camera driver controls the camera 193 to take pictures.

[0160] Taking the photo-taking scenario as an example, in response to the user opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface of the application framework layer to start the camera application, and then sends an instruction to start the camera by calling the camera device (camera device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends this instruction to the camera device driver in the kernel layer. The camera device driver can start the corresponding camera sensor and collect image light signals through the sensor. A camera device in the camera hardware abstraction layer corresponds to a camera sensor in the hardware layer.

[0161] Then, the camera sensor can transmit the collected image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the above raw image to the camera hardware abstraction layer through the camera device driver.

[0162] The camera hardware abstraction layer can send the original image to the camera algorithm library. The camera algorithm library stores the program code for implementing the shooting anti-shake method provided by the embodiment of the present application. Based on the digital signal processor and the image processor, the camera algorithm library executes the above code. In the preview mode, the jitter data is band-pass filtered, and then the anti-shake processing is performed based on the processed jitter data, thereby avoiding the impact of the jitter on the preview image; and the scene detection is performed in the preview stage to determine whether to perform long exposure or variable exposure shooting; when long exposure or variable exposure shooting, the anti-shake reset function is turned off, the jitter data is low-pass filtered, and then the anti-shake processing is performed based on the processed jitter data; and the long exposure or variable exposure image is collected after a certain period of time from the triggering of the photo, thereby avoiding the impact of the jitter on the long exposure or variable exposure image.

[0163] The camera algorithm library can send long-exposure or variable-exposure image frames captured by the camera to the camera hardware abstraction layer. The camera hardware abstraction layer can then display the long-exposure or variable-exposure image frames and store them in the image library.

[0164] In order to reduce the influence of specific exposure shooting by various possible jitters, the present application implements the following shooting anti-jitter strategy:

[0165] In the preview mode, scene detection is performed, and the scene detection result is: 1) not using long exposure shooting or variable exposure shooting, 2) using long exposure shooting or using variable exposure shooting.

[0166] On the one hand, in the preview stage, it is identified that long exposure shooting or variable exposure shooting is not used, and correspondingly, the preview mode and the shooting mode use the same anti-jitter mode: in the preview mode and the shooting mode, band-pass filtering is used, and the anti-jitter reset function is in the open state, the OIS controller obtains the jitter data (angular velocity) of the electronic device collected by the gyroscope sensor, and generates a control signal for controlling the movement of the OIS motor according to the jitter data collected by the gyroscope sensor. The OIS motor moves or resets the lens under the control of the control signal to offset the displacement caused by jitter, realizing optical anti-jitter. This optical anti-jitter processing can be applied to the preview mode, and can also be applied to the shooting mode.

[0167] On the other hand, in the preview stage, it is identified that long exposure shooting or variable exposure shooting will be used, and correspondingly, the preview mode and the shooting mode use different anti-jitter modes respectively:

[0168] 1) In the preview mode, band-pass filtering is used, and the anti-jitter reset function is in the open state, the OIS controller obtains the jitter data (angular velocity) of the electronic device collected by the gyroscope sensor, and generates a control signal for controlling the movement of the OIS motor according to the jitter data collected by the gyroscope sensor. The OIS motor moves up and down / left and right under the control of the control signal, and then offsets the displacement caused by jitter, realizing optical anti-jitter. The camera sensor collects images to obtain a preview image subjected to optical anti-jitter.

[0169] 2) In the shooting mode, low-pass filtering is switched, and the anti-jitter reset function is closed in the shooting mode to avoid image blur caused by anti-jitter reset, and the lens is centered by the OIS motor to ensure that the anti-jitter function can be normally used after the anti-jitter reset function is closed.

[0170] For long exposure images obtained by long exposure or variable exposure shooting, the exposure time is long, and the jitter is obvious, so by low-pass filtering the jitter data and controlling the camera lens movement based on the processed jitter data, full-band anti-jitter processing can be performed for high-frequency jitter and low-frequency jitter of the electronic device, so that the influence of specific exposure shooting such as long exposure or variable exposure shooting by various possible jitters is greatly reduced. Under this anti-jitter processing, the camera sensor collects long exposure image frames to obtain a first image subjected to optical anti-jitter, which improves the image shooting clarity.

[0171] The embodiments of the present application provide a shooting anti-shake method and an electronic device. By improving the software of the electronic device, the influence of shaking on long-exposure shooting can be effectively avoided, and the blurring of the image caused by shaking can be avoided, thereby improving the shooting anti-shake performance.

[0172] The execution subject of the shooting anti-shake method provided in the embodiments of the present application can be the above-mentioned electronic device, or a functional module and / or functional entity in the electronic device that can implement the shooting anti-shake method. The present application solution can be implemented through hardware and / or software. The specific implementation can be determined according to actual usage requirements and is not limited by the embodiments of the present application. The following uses an electronic device as an example and combines the accompanying drawings to illustrate the shooting anti-shake method provided in the embodiments of the present application.

[0173] The following will be combined with the accompanying drawings to illustrate the embodiments of the present application through the following exemplary embodiments. The methods in the following embodiments can be implemented in an electronic device with the above hardware structure and software architecture. The hardware structure diagram of the electronic device can be as follows: Figure 2 As shown, the software structure diagram of the electronic device can be as follows Figure 4 For the sake of convenience, the embodiments of the present application are all based on electronic devices as shown in FIG. Figure 2 Take the mobile phone shown as an example.

[0174] First combine Figure 5 The anti-shake system architecture provided by the embodiment of the present application is described. Figure 5 A schematic diagram of the structure between the gyroscope sensor, OIS controller and lens is shown.

[0175] like Figure 5 As shown, the gyroscope sensor can be connected to the OIS controller. The gyroscope sensor obtains the angular velocity of the electronic device's vibration and transmits the angular velocity of the electronic device's vibration to the OIS controller.

[0176] The OIS controller is connected to a drive motor (also known as an optical image stabilization motor), which is then connected to the lens. The OIS controller calculates the compensation angle based on the angular velocity of the electronic device's vibration and directs the drive motor to adjust the lens angle for stable imaging.

[0177] The OIS controller can also be connected to an application processor (AP) or image signal processor (ISP), which can be connected to a CMOS sensor, which can be connected to a lens. The OIS controller can call on the application processor (AP) to calculate the compensation angle. The OIS controller can also call on the image signal processing signal from the image signal processor (ISP) to correct the compensation angle.

[0178] The OIS controller can be connected to a Hall sensor, and the Hall sensor is connected to the lens. The Hall sensor functions to feed back position information of the lens to the OIS controller, to form a closed-loop control to accurately move the lens to a required position.

[0179] The OIS controller can also be referred to as an OIS control driver or an OIS driving chip.

[0180] Referring to Figure 6 As shown in the structural schematic diagram, the basic principle of optical image stabilization is that, when the gyroscope sensor is subjected to external vibration, the vibration signal is fed back to the OIS controller, the OIS controller drives the OIS motor to move the lens, thereby offsetting the image offset caused by the vibration, to ensure that the camera can still maintain stable imaging in a vibrating environment.

[0181] The following will be described in combination with Figure 6 The vibration data detected by the gyroscope sensor as shown in the figure illustrates the influence of the vibration on imaging.

[0182] Figure 6 The time-domain vibration data detected by the gyroscope sensor in the preview stage and the shooting stage is shown. Various vibration factors existing in the shooting process will affect the imaging quality, so it is necessary to analyze the vibration data and consider various vibration factors for anti-vibration processing. The time-domain vibration data can be converted into frequency-domain vibration data, and the frequency-domain vibration data includes high-frequency vibration signals and low-frequency vibration signals. It should be noted that the high frequency and low frequency described herein are relative. Exemplarily, the high-frequency vibration signal can include irregular vibration and vibration caused by touching the screen. The low-frequency vibration signal can include regular vibration such as breathing and heartbeat.

[0183] Referring to Figure 6 As shown in the figure, the electronic device detects vibration data through the gyroscope sensor, and after analyzing the vibration data, it is found that in a long exposure or variable exposure shooting scene, the vibration amplitude suddenly increases, because: the exposure time is lengthened, the vibration amount is increased, and the influence of the vibration on the image quality is more obvious.

[0184] In the related art, when the shutter is pressed, the gyroscope sensor detects the angle of the electronic device vibration, calculates the distance that the lens module needs to compensate according to the angle, and makes the lens module offset the vibration of the electronic device through reverse motion to achieve anti-vibration. In actual implementation, the vibration data detected by the gyroscope sensor corresponds to high-frequency vibration signals, and the OIS controller controls the OIS motor to move the lens module to offset the high-frequency vibration of the electronic device through reverse motion based on the high-frequency vibration signals, to achieve optical image stabilization processing. Such optical image stabilization processing can only compensate for the influence of high-frequency vibration, and ignores low-frequency vibration, so such optical image stabilization processing has poor anti-vibration effect when applied to a long exposure or variable exposure shooting scene.

[0185] The application scheme can be applied to long exposure or variable exposure shooting scenes, and the influence of specific exposure shooting scenes such as long exposure or variable exposure shooting scenes on various possible shaking is greatly reduced. Through the application scheme, not only irregular shaking of relatively high frequency and shaking caused by a touch screen are processed, but also regular shaking such as breathing and heartbeat of relatively low frequency is processed. The influence of shaking in long exposure or variable exposure shooting scenes is overcome, and the image quality is improved. The specific anti-shaking method will be described in detail below.

[0186] The shooting anti-shaking method provided by the application embodiment will be described below in combination with specific embodiments.

[0187] It should be noted that, from the perspective of the camera working state, the camera working state includes a preview state and a shooting state. From the perspective of the camera working mode, the camera working mode includes a preview mode and a shooting mode. The preview state corresponds to the preview mode, and the shooting state corresponds to the shooting mode. For ease of illustration, the following embodiments are exemplarily described with the camera working mode including the preview mode and the shooting mode.

[0188] In the application embodiment, since the user holds the electronic device for shooting, various possible shaking exists whether in the preview mode or in the shooting mode, and will affect the image clarity. That is, anti-shaking processing is needed in the preview mode and the shooting mode.

[0189] Figure 7 is a flowchart of the shooting anti-shaking method provided by the application embodiment. Referring to Figure 7 , the shooting anti-shaking method includes the following steps S301-S316.

[0190] S301, in response to a user operation, starting a camera application, and displaying a preview interface.

[0191] In some embodiments, the electronic device receives a click operation of a user on a camera application icon, and in response to the click operation, the electronic device starts the camera application, and starts the preview mode to display a camera preview interface.

[0192] In another embodiment, the electronic device runs an application program such as application in the foreground, and the electronic device displays a shooting control in the current interface. In the case that the electronic device receives a click operation of a user on the shooting control, the electronic device starts the camera application in response to the click operation, starts the preview mode, and displays the camera preview interface.

[0193] S302, in the preview mode, obtain the jitter data, and perform band-pass filtering processing on the jitter data, and perform optical anti-shake processing according to the processed jitter data and the anti-shake reset function.

[0194] Due to the influence of jitter, the image collected in the preview mode will appear image blur, so it is necessary to perform anti-shake processing in the preview mode. The present application can perform optical anti-shake processing according to the anti-shake strategy corresponding to the preview scene in the preview mode.

[0195] The jitter data can include the attitude angle of the jitter of the electronic device.

[0196] In the embodiments of the present application, in the preview mode, the electronic device can obtain the gyro angular velocity through the gyro sensor, and obtain the angle by performing discrete integration processing on the gyro angular velocity. The angle is the attitude angle of the jitter of the electronic device.

[0197] The band-pass filtering processing is a signal processing technology, which is used to select signals within a certain frequency range from the frequency spectrum when processing signals. Its principle is to selectively attenuate or retain part of the frequency in the signal, so that the filtered signal only contains signals within a certain frequency range.

[0198] In the embodiments of the present application, the band-pass filtering processing on the jitter data includes: performing band-pass filtering processing on the attitude angle of the jitter of the electronic device. The band-pass filtering processing includes: retaining the attitude angle corresponding to high-frequency jitter (such as irregular jitter, touch screen jitter), and filtering out the attitude angle corresponding to low-frequency jitter (such as breathing, heartbeat and other regular jitter). That is, the anti-shake processing in the preview mode is only for high-frequency jitter, and low-frequency jitter is ignored.

[0199] Then, the electronic device calculates the distance that needs to be compensated by the lens module according to the attitude angle corresponding to the high-frequency jitter (such as irregular jitter, touch screen jitter), so that the lens module offsets the jitter of the electronic device through reverse motion, and realizes anti-shake.

[0200] The advantage of this optical anti-shake processing by band-pass filtering the jitter data is fast response speed, which makes the preview mode have better hand following property. The hand following property refers to the characteristic that when the user touches the screen, the screen content can respond in time according to the user's touch operation. The higher the hand following property, the more timely the screen response will be, so that the picture feeling is more smooth.

[0201] In some cases, the anti-shake reset function is enabled by default, and when the anti-shake effect cannot meet the requirements or the anti-shake function fails, the electronic device uses the anti-shake reset function. For example, when the jitter amplitude is large, the pushing distance of the OIS motor to the lens module may reach the limit, causing the optical anti-shake function to fail, in which case the electronic device uses the anti-shake reset function, that is, resets the lens module to the initial position, for example, places the lens module at the central axis position, to restore the anti-shake function.

[0202] Thus, in the preview mode, according to the anti-shake strategy corresponding to the preview scene, the optical anti-shake purpose is achieved according to the band-pass filtered jitter data and the anti-shake reset function.

[0203] S303, collect a preview image using a preset exposure parameter, and display the preview image on a preview interface.

[0204] In the preview mode, the electronic device collects an image and displays the collected preview image on the camera preview interface.

[0205] In the embodiments of the present application, optical anti-shake processing is performed in the preview mode, which avoids the influence of high-frequency jitter on image quality, has fast response speed, and can ensure followability.

[0206] S304, scene detection is performed in the preview mode.

[0207] In some embodiments, the electronic device can perform scene detection according to the brightness information of the preview image.

[0208] In other embodiments, the electronic device can also perform scene detection according to the ambient light brightness information collected by the ambient light sensor.

[0209] In other embodiments, the electronic device can perform scene detection according to the brightness information of the preview image and the ambient light brightness information collected by the ambient light sensor.

[0210] S305, determine whether the current scene meets the conditions for long exposure or variable exposure shooting.

[0211] In some embodiments, the electronic device detects the current scene and determines whether the current scene meets the conditions for long exposure or variable exposure shooting.

[0212] In some embodiments, scenes such as weak light or night scene meet the conditions for long exposure shooting.

[0213] For example, if the electronic device detects that the preview image contains a night scene feature, the electronic device determines that the current scene meets the conditions for long exposure shooting. For example, if the electronic device detects that the ambient light brightness is lower than a preset brightness value, the electronic device determines that the current scene meets the conditions for long exposure shooting.

[0214] In some embodiments, a weak light or back light scene such as back light shooting, or a scene such as sunrise, sunset and sky landscape meets the condition of variable exposure shooting. This is because, for such scenes, there are very bright and very dark regions in the picture at the same time (i.e. overexposure and underexposure regions exist at the same time), and such shooting scenes are suitable for using the HDR shooting mode to realize variable exposure shooting by fusing long exposure frames and short exposure frames.

[0215] For example, if the electronic device detects that the preview image contains regions with distinct brightness and darkness, the electronic device determines that the current scene meets the condition of variable exposure shooting. For example, if the electronic device detects that the ambient light has distinct brightness and darkness, the electronic device determines that the current scene meets the condition of variable exposure shooting.

[0216] In other embodiments, the implementation of "judging whether the current scene meets the condition of long exposure or variable exposure shooting" can also include that the electronic device can determine to use long exposure or variable exposure shooting according to the user's trigger operation, thereby determining that the current scene meets the condition of long exposure or variable exposure shooting.

[0217] For example, after the user clicks the "professional mode" control, the electronic device can use long exposure or variable exposure shooting. For another example, when the electronic device receives a user trigger operation to start "long exposure shooting", the electronic device will use long exposure shooting. For another example, when the electronic device receives a user trigger operation to start "HDR shooting", the electronic device will use variable exposure shooting.

[0218] It should be noted that the judgment result of S305 has two kinds:

[0219] One is that the current scene meets the condition of long exposure or variable exposure shooting, and after S305, the following S306 to S312 are executed, that is, the anti-shake strategy corresponding to the long exposure or variable exposure shooting scene is executed.

[0220] The other is that the current scene does not meet the condition of long exposure or variable exposure shooting, and after S305, the following S313 to S316 are executed, that is, the anti-shake strategy corresponding to the normal shooting scene is executed.

[0221] The anti-shake strategy corresponding to the long exposure or variable exposure shooting scene:

[0222] In some embodiments, in a case where the electronic device determines that the current scene meets the condition of long exposure or variable exposure shooting, the electronic device automatically performs the corresponding anti-shake strategy according to S306 to S312 below. In other embodiments, the electronic device can display a prompt box to prompt the user to confirm whether to use long exposure or variable exposure shooting; in a case where the electronic device receives an operation of the user confirming to use long exposure or variable exposure shooting, the electronic device can perform the corresponding anti-shake strategy according to S306 to S312 below.

[0223] S306, receiving a first operation of the user triggering shooting.

[0224] In a case where the electronic device receives the operation of the user triggering shooting, the electronic device switches from the preview mode to the shooting mode. Then, in the shooting mode, optical anti-shake processing is performed according to the anti-shake strategy corresponding to the long exposure or variable exposure shooting scene.

[0225] S307, in response to the first operation, obtaining the jitter data and closing the anti-shake reset function.

[0226] In the shooting mode, the electronic device can obtain the gyro angular velocity through the gyro sensor, and obtain the attitude angle of the jitter of the electronic device through the integral processing of the discretization of the gyro angular velocity. The attitude angle of the jitter of the electronic device is the jitter data.

[0227] It should be noted that, as described above, in the long exposure or variable exposure shooting scene, the exposure time is lengthened and the jitter amount is increased. If the anti-shake reset function is used at this time, the lens reset action may cause image blur. In order to avoid this situation, the anti-shake reset function can be closed.

[0228] S308, performing low-pass filtering processing on the jitter data, and performing optical anti-shake processing according to the processed jitter data.

[0229] It should be noted that, in the long exposure or variable exposure shooting scene, the exposure time is lengthened and the jitter amount is increased. In order to improve the anti-shake effect, the jitter data is processed by low-pass filtering in the present application. The effect of the low-pass filtering processing includes filtering out high-frequency components and only allowing low-frequency components to pass. By performing low-pass filtering processing on the jitter data and then performing optical anti-shake processing according to the processed jitter data, high-frequency jitter is compensated, and low-frequency jitter is also compensated, thereby improving the anti-shake effect in the long exposure or variable exposure shooting scene.

[0230] Therefore, in the shooting mode, according to the anti-shake strategy corresponding to the long exposure or variable exposure shooting scene, optical anti-shake is performed according to the jitter data processed by the band-pass filtering, and the anti-shake purpose is achieved.

[0231] S309, adjust the camera exposure parameter (including the long exposure parameter after adjustment), and collect an image based on the adjusted exposure parameter to obtain a first image.

[0232] In the embodiments of the present application, if the electronic device receives a first operation triggered by a user to start shooting in a case where the current scene meets the condition for long exposure or variable exposure shooting, the electronic device adjusts the camera exposure parameter after a first time period and collects an image based on the adjusted exposure parameter.

[0233] It should be noted that the value of the first time period is determined according to the sum of the time period consumed by switching the filtering mode and closing the anti-shake reset function (referred to as the second time period). The first time period is greater than or equal to the second time period.

[0234] It should also be noted that the electronic device discards the image collected within the first time period. The electronic device collects a long exposure image frame based on the adjusted exposure parameter after the first time period and generates a first image based on the long exposure image frame.

[0235] It can be understood that after the first time period, the filtering mode has been switched and the anti-shake reset function has been closed, reaching a stable state, and in this case, the long exposure frame is collected, which can effectively avoid the blurring of the long exposure image frame caused by the anti-shake reset, greatly reducing the influence of the shaking on the long exposure shooting.

[0236] Among them, the adjusted camera exposure parameter includes the long exposure parameter, and the long exposure parameter includes the exposure time. The shooting anti-shake method provided by the embodiments of the present application adjusts the exposure parameter and collects an image based on the adjusted exposure parameter for the following shooting scenes:

[0237] Scene one: when shooting with long exposure, the electronic device sets the long exposure parameter and collects a long exposure frame based on the long exposure parameter. Illustratively, the exposure time in the long exposure parameter can be set to 5 seconds or other values.

[0238] Scene two: when shooting with variable exposure, the electronic device sets the long exposure parameter and the short exposure parameter, and collects a long exposure frame and a short exposure frame based on the long exposure parameter and the short exposure parameter respectively. Illustratively, the exposure time in the long exposure parameter can be set to 5 seconds, and the exposure time in the short exposure parameter can be set to 1 second. It should be noted that the long exposure parameter and the short exposure parameter are relative, and the exposure time is not limited here.

[0239] For long exposure shooting or variable exposure shooting scenes, a series of anti-shake processes are immediately started after switching from preview mode to shooting mode, such as switching from band-pass filtering of the shaking data to low-pass filtering to achieve full-band anti-shake effect, and closing the anti-shake reset function to avoid blurring of the long exposure image frame caused by the anti-shake reset, so that the influence of shaking on the collection of the long exposure frame is greatly reduced.

[0240] S310, store the first image to the gallery.

[0241] S311, restart the anti-shake reset function.

[0242] It should be noted that after the current shooting is completed, the long exposure shooting or the variable exposure shooting is switched back to the preview mode, and therefore it is necessary to restart the anti-shake reset function to ensure that the anti-shake function corresponding to the preview mode can be normally used in the preview stage.

[0243] S312, restore the camera exposure parameter to the preset exposure parameter.

[0244] It should be noted that after the current long exposure shooting or variable exposure shooting is completed, the camera exposure parameter needs to be restored to the preset exposure parameter.

[0245] It should be noted that the present application does not limit the execution order of S311 and S312.

[0246] After S311 and S312, return to execute the above S302 and the like. That is, after shooting is completed, the electronic device switches from the shooting mode to the preview mode, and then performs optical anti-shake processing according to the anti-shake strategy corresponding to the preview mode.

[0247] The above illustrates the implementation process of performing optical anti-shake processing according to the anti-shake strategy corresponding to the long exposure or variable exposure shooting scene when the current scene meets the conditions of long exposure or variable exposure shooting. Next, the implementation process of performing optical anti-shake processing according to the anti-shake strategy corresponding to the normal shooting scene when the current scene does not meet the conditions of long exposure or variable exposure shooting is described.

[0248] The anti-shake strategy corresponding to the normal shooting scene:

[0249] S313, receive a second operation triggered by a user to shoot.

[0250] When the electronic device receives the operation triggered by the user to shoot, the electronic device switches from the preview mode to the shooting mode. Then, in the shooting mode, optical anti-shake processing is performed according to the anti-shake strategy corresponding to the normal shooting scene.

[0251] S314, in response to the second operation, obtain the jitter data, perform band-pass filtering processing on the jitter data, and perform optical anti-shake processing according to the processed jitter data and the anti-shake reset function.

[0252] In the embodiment of the present application, in response to the second operation, the electronic device can obtain the gyro angular velocity through the gyro sensor, obtain the attitude angle of the shaking of the electronic device, that is, the shaking data, by performing the integral processing of the discretization of the gyro angular velocity. Then, the shaking data is subjected to the band-pass filtering processing. In the anti-shake processing in the conventional shooting scene, only the high-frequency shaking is processed, and the low-frequency shaking is ignored. Then, the electronic device calculates the distance that needs to be compensated by the lens module according to the high-frequency shaking data (such as irregular shaking and touch screen shaking) reserved after the band-pass filtering processing, so that the lens module offsets the shaking of the electronic device through the reverse movement, thereby realizing the anti-shake.

[0253] When the anti-shake effect cannot meet the requirement or the anti-shake function is invalid, the electronic device starts the anti-shake reset function. For example, when the shaking amplitude is large, the pushing distance of the OIS motor to the lens module can reach the limit, so that the optical anti-shake function is invalid. In this case, the electronic device starts the anti-shake reset function, that is, resets the lens module to the initial position through the OIS motor, for example, places the lens module at the central axis position, so as to restore the anti-shake function.

[0254] Therefore, in the shooting mode, the anti-shake strategy corresponding to the conventional shooting scene is the same as the anti-shake strategy corresponding to the preview mode, and the optical anti-shake processing is performed according to the shaking data after the band-pass filtering processing and the anti-shake reset function.

[0255] S315, acquiring the image by using the preset exposure parameter to obtain a second image.

[0256] S316, storing the second image to the gallery.

[0257] After S316, return to execute the above-mentioned S302 and the like. That is, after the current shooting is completed, the electronic device switches from the shooting mode to the preview mode, and then performs the optical anti-shake processing according to the anti-shake strategy corresponding to the preview mode.

[0258] In some embodiments, in the preview mode, the zero shutter lag (ZSL) mode is started, and the image is acquired. In the shooting mode, the zero shutter lag (ZSL) mode is closed, and the image is acquired.

[0259] When the ZSL mode is set to be started, the camera can cache a short video in the memory, so as to obtain the picture captured before the shutter button is pressed. When the shutter button is pressed, the camera can immediately save the previously cached picture, thereby reducing the shutter delay. This makes it more convenient to shoot dynamic scenes or capture moments.

[0260] It can be known from the above method flow that the OIS algorithm module adopts different anti-shake strategies for the preview mode and different shooting scenes.

[0261] Firstly, in the preview mode, the shaking data is acquired through the gyro sensor, and the shaking data is band-pass filtered, and then the optical anti-shake processing is performed according to the processed shaking data; meanwhile, the image is collected by using the preset exposure parameter, and the preview image is obtained and displayed on the preview interface. Moreover, scene detection is performed in the preview mode, whether the current shooting scene meets the condition of long exposure / variable exposure shooting is judged, and then different anti-shake strategies are adopted for different shooting scenes.

[0262] On the one hand, the current shooting scene meets the condition of long exposure / variable exposure shooting:

[0263] When the electronic device receives a shooting instruction, it switches from the preview mode to the shooting mode, the current shooting scene meets the condition of long exposure / variable exposure shooting, and the corresponding anti-shake strategy includes: closing the anti-shake reset function, acquiring shaking data, and low-pass filtering the shaking data to achieve full-band anti-shake effect; and adjusting the camera exposure parameter to the long exposure / variable exposure parameter, and collecting the image based on the adjusted exposure parameter. Since the corresponding anti-shake processing is performed, the long exposure frame collected in this case is less affected by shaking. After the long exposure / variable exposure shooting is completed, the shooting mode is switched to the preview mode, the optical anti-shake function is restarted, and the exposure parameter is reset.

[0264] On the other hand, the current shooting scene does not meet the condition of long exposure / variable exposure shooting:

[0265] When the electronic device receives a shooting instruction, it switches from the preview mode to the shooting mode, the current shooting scene is a normal shooting scene, and the corresponding anti-shake strategy includes: normally using the anti-shake reset function, acquiring shaking data, and band-pass filtering the shaking data to have an anti-shake effect on high-frequency shaking; and collecting the image by using the preset exposure parameter. In this case, the normal exposure image collected has an anti-shake effect on part of the high-frequency shaking, and at the same time has good hand-following property. After the shooting is completed, the shooting mode is switched back to the preview mode.

[0266] The following explains the shooting anti-shake method provided by the embodiments of the application from the perspective of upper-layer software.

[0267] Specifically, scene detection is performed in the preview stage to determine whether to enter long exposure or variable exposure shooting. In the case of determining to enter long exposure or variable exposure shooting, the HAL layer of the upper-layer software issues a switching instruction to the OIS algorithm module, instructing to switch to the long exposure or variable exposure shooting mode, and changes the camera exposure parameter (after the change, the long exposure parameter is included) after waiting for a certain time (the first time). After the long exposure or variable exposure shooting is completed, the HAL layer issues a switching instruction to the OIS algorithm module, instructing to switch back to the preview mode.

[0268] The switching instructions issued by the HAL layer can be transmitted to the OIS algorithm module through the driver software. For example, the driver software writes the instructions to the register through the Inter-Integrated Circuit (IIC) protocol, and the OIS algorithm module polls the register to obtain the instructions.

[0269] The following describes the shooting anti-shake method provided by the embodiment of the present application from the perspective of the OIS algorithm module.

[0270] Specifically, the OIS algorithm module can use the following filtering strategies: high-frequency jitter filtering (band-pass filtering) corresponding to the preview mode, and full-band filtering (low-pass filtering) corresponding to the shooting mode. That is, in the camera preview stage, the OIS algorithm module uses band-pass filtering for the jitter data to achieve the purpose of anti-shake for high-frequency jitter; when shooting with long exposure or variable exposure, the OIS algorithm module uses low-pass filtering for the jitter data to achieve full-band anti-shake. Among them, the OIS algorithm module can use an IIR filter to filter the jitter data in real time, and the filter frequency band can be modified by modifying the parameters of each order.

[0271] After receiving instructions from the upper layer, the OIS algorithm module switches to long exposure / variable exposure shooting mode, disables the anti-shake reset function, and resets the lens through the OIS motor. Because mode switching requires a certain stabilization time (approximately 100ms), it waits a certain period of time before changing the camera exposure parameters and capturing images based on the modified exposure parameters. Accordingly, the OIS algorithm module can notify the upper layer software to wait for a fixed delay.

[0272] Among them, the OIS algorithm module switches back to preview mode after receiving the upper-layer instruction, restarts the anti-shake reset function, and resets the camera exposure parameters.

[0273] Figure 8 The embodiment of the present application shows the implementation process of the anti-shake method for shooting in a long exposure shooting scene. Figure 8 As shown in the figure, the anti-shake strategy in preview mode and shooting mode is explained through the upper-layer algorithm module, sensor image output, OIS status, and jitter data.

[0274] Anti-shake strategy in preview mode:

[0275] 1) Upper-level algorithm module: Detects the scene through preview and determines whether to enter long-exposure or variable-exposure shooting.

[0276] 2) Sensor output: Use the preset exposure parameters to capture the image frame of regular exposure (marked as N in the figure).

[0277] 3) OIS status:

[0278] a) acquiring jitter data, performing bandpass filtering on the jitter data, and performing optical image stabilization based on the processed jitter data;

[0279] b) The anti-shake reset function is on. When the anti-shake function fails, use the anti-shake reset function to reset the lens to its initial state to restore the anti-shake function.

[0280] Assume that the upper-layer algorithm module determines through scene detection that the conditions for entering long-exposure shooting are met. If the upper-layer algorithm module receives an operation to trigger taking a photo, the upper-layer algorithm module sends an instruction to the OIS algorithm module, instructing it to switch to long-exposure shooting mode and adopt the anti-shake strategy in long-exposure shooting mode for anti-shake.

[0281] Anti-shake strategy in long exposure shooting mode:

[0282] 1) Upper-level algorithm module: performs long-exposure shooting.

[0283] 2) Sensor output: The image frames collected within the first duration are discarded; after the first duration, the exposure parameters are adjusted and long-exposure image frames (marked as L in the figure) are collected.

[0284] 3) OIS status:

[0285] a) acquiring jitter data and performing low-pass filtering on the jitter data, and performing optical image stabilization based on the processed jitter data;

[0286] b) Turn off the anti-shake reset function to avoid blurring of long-exposure image frames caused by anti-shake reset.

[0287] c) After the long exposure shooting is completed, the anti-shake reset function is restarted to reset the exposure parameters and automatically switch to preview mode.

[0288] Figure 9 The embodiment of the present application shows the implementation process of the anti-shake method for shooting in a variable exposure shooting scene. Figure 9 As shown in the figure, the anti-shake strategy in preview mode and shooting mode is explained through the upper-layer algorithm module, sensor image output, OIS status, and jitter data.

[0289] The anti-shake strategy in preview mode is as described above and will not be repeated here.

[0290] Assume that the upper-layer algorithm module determines through scene detection that the conditions for entering variable exposure shooting are met. If the upper-layer algorithm module receives an operation to trigger taking a photo, the upper-layer algorithm module sends an instruction to the OIS algorithm module, instructing it to switch to variable exposure shooting mode and adopt the anti-shake strategy in variable exposure shooting mode for anti-shake.

[0291] Anti-shake strategy in variable exposure shooting mode:

[0292] 1) Upper algorithm module: perform variable exposure shooting.

[0293] 2) Sensor output: the image frames collected in the first time length are discarded; the exposure parameters are adjusted after the first time length, the adjusted exposure parameters include long exposure parameters and short exposure parameters, the camera sensor alternately outputs long exposure image frames and short exposure image frames, the long exposure image frames are marked as L, and the short exposure image frames are marked as S.

[0294] 3) OIS state:

[0295] a) Obtain the jitter data and perform low-pass filtering on the jitter data, and perform optical anti-shake processing according to the processed jitter data;

[0296] b) Close the anti-shake reset function to avoid blur of long exposure image frames caused by anti-shake reset.

[0297] c) After the long exposure shooting is completed, restart the anti-shake reset function and reset the exposure parameters, and automatically switch to the preview mode.

[0298] It should be noted that the difference between long exposure shooting and variable exposure shooting lies in sensor output. In long exposure shooting, long exposure image frames are output, and an image is generated based on the long exposure image frames. In variable exposure shooting, long exposure image frames and short exposure image frames are alternately output, and an image is generated based on the long exposure image frames and the short exposure image frames.

[0299] The shooting anti-shake method provided by the embodiments of the present application can be applied to professional mode, night scene, high dynamic range and other shooting scenes.

[0300] Figure 10 An interface schematic diagram of applying the shooting anti-shake method provided by the embodiments of the present application is shown.

[0301] As shown in Figure 10 , scene detection is performed in the preview mode, and it is judged whether the current shooting scene meets the conditions of long exposure / variable exposure shooting. If the shooting scene meets the conditions of long exposure / variable exposure shooting, a specific anti-shake strategy is adopted for the shooting scene. When the electronic device receives a shooting instruction, a series of anti-shake actions are performed: switching from the preview mode to the shooting mode, closing the anti-shake reset function, and obtaining the jitter data, performing low-pass filtering on the jitter data to achieve full-band anti-shake effect; and adjusting the camera exposure parameters to long exposure / variable exposure parameters after waiting for a first time length, and collecting images based on the adjusted exposure parameters. Since the corresponding anti-shake processing is performed, the long exposure frames collected in this case are less affected by jitter, and the finally generated image is clear.

[0302] The shooting anti-shake method provided by the embodiment of the present application is used to identify whether the current scene is a specific exposure shooting scene in the preview stage. For specific exposure shooting scenes (such as long exposure shooting or variable exposure), since they are significantly affected by shaking, the anti-shake reset function is turned off after the shooting is triggered to avoid image blur caused by the anti-shake reset. The shaking data is filtered in the full frequency band and the orientation of the camera lens is adjusted based on the shaking data to achieve a full-band anti-shake effect. It can perform anti-shake processing for high-frequency shaking such as movement and touch screen shaking, as well as low-frequency shaking such as breathing and heartbeat. After the shooting is triggered to start, a first time period is waited to complete the reset of various parameters. After the state is stable, the camera sensor is controlled to output image frames and generate images according to the adjusted exposure parameters, so that the specific exposure shooting is greatly reduced from being affected by various possible shaking, thereby improving the clarity of image shooting.

[0303] It should be noted that, in the embodiments of the present application, "greater than" can be replaced by "greater than or equal to", "less than or equal to" can be replaced by "less than", or "greater than or equal to" can be replaced by "greater than", and "less than" can be replaced by "less than or equal to".

[0304] The various embodiments described herein may be independent solutions or may be combined according to internal logic, and all of these solutions fall within the scope of protection of this application.

[0305] The above mainly describes the solution provided by the embodiment of the present application from the perspective of method steps. It is understandable that, in order to achieve the above functions, the electronic device implementing the method includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of protection of this application.

[0306] In the embodiment of the present application, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other feasible division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0307] Figure 11 A schematic block diagram of a photographing anti-shake device 400 is provided for an embodiment of the present application. The device 400 can be used to perform the actions performed by the electronic device in the above method embodiments. The device 400 includes an anti-shake unit 410 and a photographing unit 420.

[0308] The anti-shake unit 410 is configured to, after starting a camera application, identify that a current first scene meets a preset photographing condition in a preview stage; receive a first operation of a user triggering starting photographing, in response to the first operation, close an anti-shake reset function, and adjust an orientation of a camera lens according to first shake data, the first shake data being data obtained by performing first filtering processing on shake data of the electronic device in a photographing stage; and the first filtering processing is full-band filtering.

[0309] The photographing unit 420 is configured to, after triggering starting photographing, wait for a first time length, control a camera sensor to output an image frame according to an adjusted exposure parameter, and generate a first image based on the output image frame; and the adjusted exposure parameter includes a first exposure parameter, and an exposure time length corresponding to the first exposure parameter is greater than an exposure time length corresponding to an initial exposure parameter of the camera sensor.

[0310] Through the photographing anti-shake device provided by the embodiments of the present application, whether the current scene is a specific exposure photographing scene is identified in the preview stage. For the specific exposure photographing scene (such as long exposure photographing or variable exposure), since the influence of shaking is obvious, a series of anti-shake processing is immediately started after triggering starting photographing, the anti-shake reset function is closed, image blur caused by anti-shake reset is avoided, specific full-band filtering processing is performed on the shake data, the orientation of the camera lens is adjusted according to the processed shake data, a full-band anti-shake effect is achieved, for example, irregular shaking, touch screen shaking and other high-frequency shaking can be prevented, and breathing, heartbeat and other low shaking can also be prevented, and after triggering starting photographing, a first time length is waited for, various parameter resets are completed, and after the state is stable, the camera sensor is controlled to output an image frame according to an adjusted exposure parameter and generate an image, which can avoid image blur caused by various possible shaking, thereby greatly reducing the influence of various possible shaking on specific exposure photographing, and improving image photographing clarity.

[0311] The device 400 according to the embodiments of the present application can correspond to performing the methods described in the embodiments of the present application, and the above and other operations and / or functions of the units in the device 400 are respectively for realizing the corresponding flows of the methods, and for brevity, will not be described herein.

[0312] The present application also provides a chip, which is coupled with a memory, and is used to read and execute a computer program or instructions stored in the memory to perform the method in each of the above embodiments.

[0313] The present application also provides an electronic device, which includes a chip, and the chip is used to read and execute computer programs or instructions stored in a memory, so that the methods in each embodiment are executed.

[0314] This embodiment also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions run anti-shake reset on an electronic device, the electronic device executes the above-mentioned related method steps to implement the shooting anti-shake method in the above-mentioned embodiment.

[0315] This embodiment further provides a computer program product, wherein the computer-readable storage medium stores program code. When the computer program product runs on a computer, the computer is caused to execute the above-mentioned related steps to implement the shooting anti-shake method in the above-mentioned embodiment.

[0316] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0317] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0318] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0319] The "user interface (UI)" in the present application is a medium interface for interaction and information exchange between an application program or an operating system and a user, which realizes conversion between an internal form of information and a form acceptable by the user. The user interface of an application program is source code written in a specific computer language such as Java or extensible markup language (XML), and the interface source code is parsed and rendered on a terminal device to finally present content recognizable by the user, such as pictures, words, buttons and other controls.

[0320] A common form of the user interface is a graphic user interface (GUI), which refers to a user interface related to computer operation displayed in a graphic manner. It can be an icon, a window, a control element and other interface elements displayed in a display screen of an electronic device, wherein in a camera preview interface, the control element can include a visible interface element such as a shooting control element.

[0321] The term "and / or" in the present document is used to describe an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The symbol " / " in the present document represents an or relationship of associated objects, for example, A / B represents A or B.

[0322] The terms "first" and "second" and the like in the description and claims of the present document are used to distinguish different objects, rather than to describe a specific order of the objects. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like.

[0323] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A shooting anti-shake method, applied to electronic equipment, characterized in that: The method comprises: Launch the Camera app; In the preview stage, it is identified that the current first scene meets the preset shooting conditions; receiving a first operation triggered by a user to start shooting, disabling an anti-shake reset function in response to the first operation, and adjusting the orientation of a camera lens according to first jitter data obtained by performing a first filtering process on jitter data of the electronic device during the shooting phase; After triggering to start shooting, wait for a first time period, control the camera sensor to output image frames according to the adjusted exposure parameters, and generate a first image based on the output image frames; The adjusted exposure parameter includes a first exposure parameter, and the exposure duration corresponding to the first exposure parameter is greater than the exposure duration corresponding to the initial exposure parameter of the camera sensor; The first filtering process is full-band filtering.

2. The method according to claim 1, characterized in that The preset shooting condition is a long exposure shooting condition; or the preset shooting condition is a variable exposure shooting condition.

3. The method according to claim 1 or 2, characterized in that The method further comprises: In response to the first operation, switching from a preview mode to a capture mode; The optical image stabilization motor of the electronic device is controlled to reset the camera lens to an initial position.

4. The method according to any one of claims 1 to 3, characterized in that The first filtering process is a low-pass filtering process, and the first shaking data includes a posture angle of the shaking of the electronic device.

5. The method according to any one of claims 1 to 4, characterized in that After starting the camera application, the method further includes: Adjusting the orientation of the camera lens of the electronic device based on the second jitter data during the preview phase; wherein the second jitter data is data obtained by performing a second filtering process on the jitter data of the electronic device during the preview phase; Controlling the camera sensor to capture image frames according to the initial exposure parameters, and generating a preview image based on the captured image frames; The filtering frequency band of the second filtering process is smaller than the filtering frequency band of the first filtering process.

6. The method according to claim 5, characterized in that The second filtering process is a band-pass filtering process.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Within the first time period after the trigger starts shooting, controlling the camera sensor to output image frames according to the second exposure parameter; The exposure duration corresponding to the second exposure parameter is shorter than the exposure duration corresponding to the first exposure parameter.

8. The method according to claim 7, characterized in that The second exposure parameter is an initial exposure parameter of the camera sensor.

9. The method according to claim 7 or 8, characterized in that The method further includes: discarding the image frame output by the camera sensor according to the second exposure parameter; The generating the first image based on the output image frame includes: generating the first image based on the image frame output by the camera sensor according to the first exposure parameter.

10. The method according to any one of claims 1 to 9, characterized in that The first duration is determined based on the second duration, the first duration is greater than or equal to the second duration, and the second duration is the time required to complete the resetting of various parameters after the shooting is triggered; The resetting of various parameters includes: Modify the filtering method for jitter data; The anti-shake reset function is set to off.

11. The method according to claim 10, characterized in that The parameter reset also includes: Reset the camera lens to its initial position.

12. The method according to any one of claims 1 to 11, characterized in that The step of identifying that the current first scene meets the preset shooting condition during the preview phase includes: Acquiring ambient light brightness information and / or brightness information of the preview image during the preview phase; Scene detection is performed based on the ambient light brightness information and / or the brightness information of the preview image to identify that the first scene meets the preset shooting condition.

13. The method according to claim 12, characterized in that The brightness information includes a brightness value and a brightness range; The performing scene detection according to the ambient light brightness information and / or the brightness information of the preview image, and identifying that the first scene meets the preset shooting condition, includes: If the brightness value of the preview image collected in the preview stage is less than or equal to a first brightness threshold, and / or the ambient light brightness is less than or equal to a second brightness threshold, then it is recognized that the first scene meets the long exposure shooting condition; If the brightness range of the preview image collected in the preview stage is greater than the first brightness range threshold, and / or the ambient light brightness range is greater than the second brightness range threshold, it is recognized that the first scene meets the variable exposure shooting condition.

14. The method according to any one of claims 1 to 13, characterized in that The method further comprises: When it is identified in the preview stage that the first scene meets the preset shooting condition, exposure parameters required for shooting the first scene are determined.

15. The method according to claim 14, characterized in that The determining of exposure parameters required for photographing the first scene includes: Determine exposure parameters required for photographing the first scene according to the ambient light brightness information and / or the brightness information of the preview image.

16. The method according to claim 15, characterized in that Before controlling the camera sensor to output the image frame according to the adjusted exposure parameters, the method further includes: Adjusting exposure parameters of the camera sensor according to the determined exposure parameters required for photographing the first scene.

17. The method according to any one of claims 1 to 16, characterized in that In a case where the first scene meets a long exposure shooting condition, the adjusted exposure parameters include the first exposure parameters; When the first scene meets the variable exposure shooting condition, the adjusted exposure parameter further includes a second exposure parameter, and the exposure duration corresponding to the second exposure parameter is shorter than the exposure duration corresponding to the initial exposure parameter of the camera sensor.

18. The method according to claim 17, characterized in that Generating a first image based on the output image frame includes: When the first scene meets the long-exposure shooting condition, capturing a long-exposure image frame using the first exposure parameter, and generating the first image based on the long-exposure image frame; When the first scene meets the variable exposure shooting condition, long-exposure image frames are captured using the first exposure parameters, short-exposure image frames are captured using the second exposure parameters, and the first image is generated based on the long-exposure image frames and the short-exposure image frames.

19. The method according to any one of claims 1 to 18, characterized in that After generating the first image based on the output image frame, the method further includes: Switch from capture mode to preview mode; Adjusting the camera exposure parameters to the initial exposure parameters; Re-enable the anti-shake reset function.

20. The method according to any one of claims 1 to 19, characterized in that The method further comprises: In the case where it is identified during the preview phase that the first scene does not meet the preset shooting condition, receiving a second operation triggered by the user to start shooting; In response to the second operation, adjusting the orientation of the camera lens based on third jitter data, controlling the camera sensor to output image frames according to the initial exposure parameters, and generating a second image based on the output image frames; The third jitter data is data obtained by performing a second filtering process on the jitter data of the electronic device during the shooting phase.

21. The method according to claim 20, characterized in that The method further comprises: The anti-shake reset function is used to adjust the orientation of the camera lens.

22. The method according to any one of claims 1 to 19, characterized in that The first image is a picture or a video.

23. An electronic device, characterized in that: The electronic device includes: one or more processors, and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to perform the method according to any one of claims 1 to 22.

24. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 22.

25. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises instructions, which, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 22.

Images