Shooting method and device and storage medium

Abstract

The invention relates to a shooting method and device and a storage medium. The shooting method comprises the following steps: in response to flicker of ambient light in a zero-delay shooting process, acquiring the flicker frequency of the ambient light, and acquiring an exposure image frame sequence which is an image frame needing to be exposed in a zero-delay cache queue; first exposure and second exposure are sequentially and alternately carried out on image frames in the exposure image frame sequence, the exposure time of the first exposure is the default exposure time of zero-delay photographing, the exposure time of the second exposure is integral multiple of the period corresponding to the ambient light flicker frequency, and an exposed image frame sequence is obtained; and previewing and displaying the post-exposure image frame subjected to the second exposure in the post-exposure image frame sequence. According to the invention, the stroboflash phenomenon of the image displayed on the screen in the zero-delay photographing process can be eliminated.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a shooting method, device and storage medium. Background Art

[0002] Flicker light sources refer to light sources that directly use alternating current, which are very common in daily life. In an environment with a light source with a specific alternating flickering frequency, since the camera CMOS (Complementary Metal-Oxide-Semiconductor) sensor is exposed row by row, the exposure starting point of each pixel in the same row of the image sensor is approximately the same when the image sensor is exposed, but the exposure starting points of the rows are different. When the row exposure time is not an integer multiple of the light source flickering cycle, there will be a difference in the exposure integral between the rows inside the image sensor, resulting in the formation of light and dark stripes in the image. This phenomenon is called stroboscopic phenomenon. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a shooting method, a device and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a shooting method is provided, including: in response to flickering of ambient light during zero-delay photography, obtaining a flickering frequency of the ambient light and obtaining an exposure image frame sequence, wherein the exposure image frame sequence is image frames that need to be exposed in a zero-delay cache queue; performing a first exposure and a second exposure alternately on the image frames in the exposure image frame sequence, wherein the exposure time of the first exposure is a default exposure time for the zero-delay photography, and the exposure time of the second exposure is an integer multiple of a period corresponding to the flickering frequency of the ambient light, to obtain an exposed image frame sequence; and previewing and displaying the exposed image frames that undergo the second exposure in the exposed image frame sequence.

[0005] In one embodiment, the method further includes: in response to detecting a photo-taking instruction, determining a plurality of continuous image frames for image fusion processing from the exposed image frame sequence; determining a reference image frame from the plurality of continuous image frames, and generating a photographed image based on the reference image frame.

[0006] In one embodiment, acquiring the exposure image frame sequence includes: determining a first shooting frame rate, where the first shooting frame rate is a default shooting frame rate for the zero-delay photography; determining a second shooting frame rate based on the first shooting frame rate; and acquiring the exposure image frame sequence based on the second shooting frame rate.

[0007] In one embodiment, determining a reference image frame from the continuous multiple image frames includes: performing image fusion processing on the continuous multiple image frames, wherein the number of image frames after the second exposure in the continuous multiple image frames is the same as the number of image frames after the first exposure and the exposure times are adjacent, and determining the continuous multiple image frames after the image fusion processing; eliminating the stroboscopic band in the continuous multiple image frames after the image fusion processing to obtain multiple image frames with the stroboscopic band eliminated; and determining an image frame among the multiple image frames with the stroboscopic band eliminated as the reference image frame.

[0008] In one embodiment, determining an image frame from the multiple image frames for eliminating the stroboscopic band as the reference image frame includes: determining an image frame for performing the first exposure after eliminating the stroboscopic band as the reference image frame from the multiple image frames for eliminating the stroboscopic band.

[0009] According to a second aspect of an embodiment of the present disclosure, a shooting device is provided, including: a receiving module, for obtaining an ambient light flicker frequency and an exposure image frame sequence in response to flickering of ambient light during zero-delay shooting, wherein the exposure image frame sequence is image frames that need to be exposed in a zero-delay cache queue; a processing module, for alternately performing a first exposure and a second exposure on image frames in the exposure image frame sequence, wherein the exposure time of the first exposure is a default exposure time for the zero-delay shooting, and the exposure time of the second exposure is an integer multiple of a period corresponding to the ambient light flicker frequency, so as to obtain an exposed image frame sequence and preview and display the exposed image frames subjected to the second exposure in the exposed image frame sequence.

[0010] In one embodiment, the processing module is also used to: in response to detecting a photo-taking instruction, determine a plurality of continuous image frames for image fusion processing from the exposed image frame sequence; determine a reference image frame from the plurality of continuous image frames, and generate a photographed image based on the reference image frame.

[0011] In one embodiment, the receiving module obtains the exposure image frame sequence in the following manner: determining a first shooting frame rate, where the first shooting frame rate is the default shooting frame rate for the zero-delay photography; determining a second shooting frame rate based on the first shooting frame rate; and obtaining the exposure image frame sequence based on the second shooting frame rate.

[0012] In one embodiment, the processing module determines a reference image frame from the continuous multiple image frames in the following manner: performing image fusion processing on the continuous multiple image frames, wherein the number of image frames after the second exposure in the continuous multiple image frames is the same as the number of image frames after the first exposure and the exposure times are adjacent, and the continuous multiple image frames after the image fusion processing are determined; eliminating the stroboscopic band in the continuous multiple image frames after the image fusion processing to obtain multiple image frames with the stroboscopic band eliminated; and determining an image frame among the multiple image frames with the stroboscopic band eliminated as the reference image frame.

[0013] In one embodiment, the processing module determines an image frame among the multiple image frames for eliminating the stroboscopic band as the reference image frame in the following manner: among the multiple image frames for eliminating the stroboscopic band, determines an image frame for performing the first exposure as the reference image frame.

[0014] According to a third aspect of an embodiment of the present disclosure, a shooting device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to: execute the shooting method described in the first aspect or any one of the implementations of the first aspect.

[0015] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the shooting method described in the first aspect or any one of the implementations of the first aspect.

[0016] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: changing the exposure time of the image frame according to the flickering frequency of the ambient light, and previewing the image frame after exposure with the changed exposure time, thereby avoiding the situation where the preview display image has a flickering band.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] Figure 1 The figure shows a schematic diagram of a civil alternating current wave according to an exemplary embodiment.

[0020] Figure 2 It is a schematic diagram showing the cause of the stroboscopic phenomenon according to an exemplary embodiment.

[0021] Figure 3It is a schematic diagram showing a film of superimposed stroboscopic images according to an exemplary embodiment.

[0022] Figure 4 is a schematic diagram of an image frame processing flow during zero-delay and non-zero-delay shooting according to an exemplary embodiment.

[0023] Figure 5a The figure is a flowchart of a shooting method according to an exemplary embodiment.

[0024] Figure 5b The diagram is a schematic diagram showing a method of extracting a long exposure image frame for preview display according to an exemplary embodiment.

[0025] Figure 6 The present invention is a flowchart of generating a photographed image according to an exemplary embodiment.

[0026] Figure 7 The flowchart of acquiring an exposure image frame sequence is shown according to an exemplary embodiment.

[0027] Figure 8 The present invention is a flowchart showing a method for determining a reference image frame according to an exemplary embodiment.

[0028] Fig. 9 The invention is a flowchart showing a method of determining a reference frame among multiple image frames according to an exemplary embodiment.

[0029] Fig.10 is a block diagram of a shooting device according to an exemplary embodiment.

[0030] Fig.11 The invention is a block diagram of a device for photographing according to an exemplary embodiment. DETAILED DESCRIPTION

[0031] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0032] In some scenarios, civilian power supply systems generally use 50HZ or 60HZ alternating current. If the artificial light source is not converted from AC to DC, then this light source will produce a light source that flickers alternately at 100HZ or 120HZ due to the characteristics of AC. LED (light-emitting diode) lamps are high-frequency modulated light sources, such as the lighting systems on subways and high-speed railways. This light source has a flickering frequency that is much higher than the industrial frequency and can reach between 200HZ and 1000HZ. The screen light source of electronic products, such as the screen of a lit mobile phone or TV, also has a flickering frequency. When using a terminal to take pictures in the above flickering light source scenario, it is easy for light and dark stripes to appear on the photos, resulting in poor photo effects. This phenomenon of light and dark stripes appearing on photos is called stroboscopic phenomenon.

[0033] The present disclosure will explain the causes of stroboscopic light in conjunction with the accompanying drawings. Figure 1 FIG. 1 is a schematic diagram of a civilian AC wave according to an exemplary embodiment. The civilian AC wave is a sine wave, and the output current at each moment will change, so the light source using AC will flicker. Figure 2 FIG. 1 is a schematic diagram showing the cause of the stroboscopic phenomenon according to an exemplary embodiment. When the holder of the photographing device is in an environment with a light source having a specific alternating flickering frequency, and the motion capture function is enabled, the exposure time of the camera is changed so that the exposure time is not an integer multiple of the light source flickering cycle, stroboscopic stripes will appear on the camera's film. Figure 2 As shown in the figure, when the AC power fluctuation cycle is 1 / 100s, the ambient light source flicker cycle is also 1 / 100s, and the number of frames for taking photos at this time is 30 frames, that is, the length of each frame is 1 / 30s. The line exposure time of each frame image is 1 / 250s. At this time, the exposure time is not an integer multiple of the ambient light source flicker cycle, so stroboscopic phenomenon will occur, and stroboscopic bands will appear on the image. Figure 3 FIG. 1 is a schematic diagram of a superimposed stroboscopic image according to an exemplary embodiment. In the picture, the flickering of the ambient light source causes dark stripes and bright stripes in the image, which is the stroboscopic phenomenon.

[0034] In related technologies, such as Figure 4 Shown is an image frame processing flow during zero-delay and non-zero-delay shooting according to an exemplary embodiment. Figure 4In the preview stage, the motion detection module will perform motion detection on the data of each frame, and pass the motion detection information to the automatic exposure control module, and control the exposure time adjustment of the next frame through the motion information. The automatic exposure control module determines whether there is flicker in the ambient light through the camera, and selects zero-delay photography or non-zero-delay photography according to whether there is flicker in the ambient light. When there is no flicker in the ambient light, the RAW image (Bayer array image) is obtained through the camera and stored in the zero-delay buffer, and the RAW image is also format converted as a preview image for preview display. When the photo is triggered, and there is no flicker in the ambient light during the shooting process, the cached RAW image will be directly processed by the multi-frame noise reduction algorithm and synthesized into a photo image for the user. When there is flicker in the ambient light, when the photo is triggered, the frame is taken backward from the shooting moment, the selected image frame skips the preview display, and the exposure processing and flicker reduction processing are modified, and finally the final photo image is generated by multi-frame noise reduction and given to the user.

[0035] When there is no flicker in the ambient light, the exposure time of each frame in the preview stage can be changed to reduce motion blur during the shooting process, and the image frames can be acquired from the moment of taking the photo forward through the zero-delay shooting method, so that the delay between the filming and the photo-taking moment is low, and the preview interface is not stuck. However, in scenes where there is flicker in the ambient light, changing the exposure time only according to the motion information will cause stroboscopic stripes to appear on the image frames in the preview stage and the filmed image when shooting through the zero-delay shooting method, affecting the shooting effect. When taking photos through the non-zero-delay shooting method, a long wait is required after the shooting is triggered. If there is shooting shaking during the process, the filming effect will be reduced, and the effect of instant photography cannot be achieved, affecting the user experience.

[0036] In view of this, the present disclosure provides a shooting method, which sets long and short staggered exposure frames, displays long exposure image frames in the preview stage, and does not display short exposure image frames, and eliminates the stroboscopic bands in the image through an algorithm, so that the real picture imaging during the shooting process will not produce stroboscopic bands due to the flickering of ambient light, thereby improving the user's shooting experience.

[0037] In one embodiment, after determining that the ambient light flickers, each acquired image frame is subjected to long and short staggered exposure, and the long exposure image frame is previewed and displayed, while the short exposure image frame is not displayed. The following is an explanation of how to eliminate the stroboscopic band process during the shooting process. For the convenience of description, the long exposure image frame described below is the image frame after the second exposure, and the short exposure image frame is the image frame after the first exposure.

[0038] Figure 5a is a flowchart of a shooting method according to an exemplary embodiment. Figure 5aAs shown, the shooting method is used in a terminal and includes the following steps.

[0039] In step S11, in response to the presence of ambient light flickering during zero-delay photography, the ambient light flickering frequency is acquired, and an exposure image frame sequence is acquired, where the exposure image frame sequence is the image frames that need to be exposed in the zero-delay cache queue.

[0040] In the embodiment of the present disclosure, during the zero-delay photography process, the ambient light is detected. If the ambient light flickers, it means that the photographic imaging may experience stroboscopic phenomena, and subsequent processing is performed using the shooting method in the embodiment of the present disclosure.

[0041] In the disclosed embodiment, the ambient light flicker frequency is obtained and determined for adjusting the photographing process. During the photographing process, image frames are obtained and all the obtained image frames are cached in a zero-delay cache queue for subsequent exposure processing. The zero-delay cache queue includes cached image frames that need to be exposed.

[0042] In step S12, the image frames in the exposure image frame sequence are subjected to a first exposure and a second exposure alternately in sequence, wherein the exposure time of the first exposure is a default exposure time for zero-delay photography, and the exposure time of the second exposure is an integer multiple of the period corresponding to the ambient light flicker frequency, to obtain an image frame sequence after exposure.

[0043] In the disclosed embodiment, the exposure time of the first exposure is the default exposure time for zero-delay photography, and the default exposure time for the current zero-delay photography can be read by the terminal device to determine the exposure time of the first exposure. The exposure time of the second exposure is an integer multiple of the period corresponding to the ambient light flicker frequency, thereby preventing the existence of stroboscopic phenomena in the image frames for the second exposure. The image frame sequence after exposure is composed of the image frames for the first exposure and the image frames for the second exposure, and the image frames for the first exposure and the image frames for the second exposure are sorted according to the time when the exposure is alternately performed.

[0044] In step S13, a plurality of exposed image frames in the exposed image frame sequence that undergo the second exposure are previewed and displayed.

[0045] In the embodiment of the present disclosure, multiple exposed image frames in the exposed image frame sequence that undergo the second exposure are previewed, while the exposed image frames that undergo the first exposure are not previewed. Since there is no stroboscopic phenomenon in the exposed image frames that undergo the second exposure, there will be no stroboscopic phenomenon in the preview display at this time, thereby eliminating the stroboscopic phenomenon in the preview display screen.

[0046] In one implementation, after the exposed image frame sequence is obtained, the exposed image frame sequence is also saved in a zero-delay cache queue.

[0047] In one embodiment, during zero-delay photography, the terminal device detects whether there is flicker in the ambient light. After confirming that there is flicker in the ambient light, the terminal device obtains the flicker frequency of the ambient light and obtains an exposure image frame sequence for exposure. The exposure time of the second exposure is determined according to the flicker frequency of the ambient light, so that the exposure time of the second exposure is an integer multiple of the ambient light flicker period. The image frames in the exposure image frame sequence are exposed alternately for long and short exposures to obtain an exposed image frame sequence. The exposed image frame sequence is cached, and the exposed image frames for the second exposure in the exposed image frame sequence are previewed and displayed, thereby eliminating the stroboscopic phenomenon in the preview display screen during zero-delay photography caused by the flicker of the ambient light.

[0048] Figure 5b The diagram is a schematic diagram showing a method of extracting a long exposure image frame for preview display according to an exemplary embodiment.

[0049] like Figure 5b As shown, NE is a long exposure image frame, and SE is a short exposure image frame that may have stroboscopic stripes.

[0050] In one embodiment, if Figure 5b As shown, when previewing, the current number of frames in the zero-delay buffer queue in the same period is 2xfps. The long-exposure image frame (NE) is selected from the long and short-exposure image frames after multiple exposures in the zero-delay buffer queue, while the short-exposure image frame (SE) is not previewed. At this time, the frame rate of the preview display is the default frame rate xfps, and the preview display will not be missing frames or stuck because only the long-exposure image frame is displayed. Where x is an arbitrary number.

[0051] In one embodiment, when performing zero-delay photography, multiple frames of long-exposure image frames and short-exposure image frames are cached in the zero-delay cache of the terminal device. Among them, the long-exposure image frame is the image frame after the second exposure, and the short-exposure image frame is the image frame after the first exposure. When preview display is required, the long-exposure image frame is extracted from the cached multiple frames for preview display, and the short-exposure image frame is not previewed. Among them, the extracted long-exposure image frame rate is the same as the original default shooting frame rate of the camera. During preview display, the long-exposure image frame can be previewed and displayed by the shooting method in the above embodiment, while the short-exposure frame is not previewed. However, when the long-exposure frame is generated for shooting and imaging, motion blur will increase due to the long exposure time. In the following embodiments of the present disclosure, the short-exposure frame is processed, and a frame of image frames is determined from the processed short-exposure frame. Based on this frame of image frames, a photographed image is generated, thereby reducing the motion blur of the photographed image. The process of generating a photographed image is described below.

[0052] Figure 6The present invention is a flowchart of generating a photographed image according to an exemplary embodiment.

[0053] In step S21, in response to detecting a photographing instruction, a plurality of continuous image frames to be subjected to image fusion processing are determined from the exposed image frame sequence.

[0054] In the embodiment of the present disclosure, each image frame in the sequence of exposed image frames is an image frame cached before the photo-taking instruction is detected. The continuous multiple image frames for image fusion processing are determined from the sequence of exposed image frames, that is, multiple image frames are selected from the moment when the photo-taking instruction is detected, and the selected multiple image frames are multiple image frames adjacent to the shooting moment.

[0055] In step S22, a reference image frame is determined from a plurality of consecutive image frames, and a photographed image is generated based on the reference image frame.

[0056] In the disclosed embodiment, the reference image frame is one of the continuous multiple image frames, and is an exposed image frame for the second exposure.

[0057] In one embodiment, in response to a photographing instruction, multiple consecutive image frames are extracted from a cached sequence of exposed image frames for processing, and a frame is determined from the processed consecutive multiple image frames as a reference frame. A photographed image is generated based on the reference frame. The stroboscopic phenomenon that may exist in the image frames in the sequence of exposed image frames is removed, thereby avoiding the stroboscopic phenomenon in the photographed image finally captured.

[0058] Since in the shooting method in the above embodiment, after obtaining the exposed image frame sequence according to the exposed image frame sequence, only the exposed image frame for the second exposure is selected from the exposed image frame sequence for preview display, the display frame rate may be affected at this time. The following is a description of the process of obtaining the exposed image frame sequence in order to prevent the display frame rate from being affected.

[0059] Figure 7 The flowchart of acquiring an exposure image frame sequence is shown according to an exemplary embodiment.

[0060] In step S31, a first shooting frame rate is determined, where the first shooting frame rate is a default shooting frame rate for zero-delay photography.

[0061] In the disclosed embodiment, the first shooting frame rate is the default shooting frame rate for zero-delay shooting, which is automatically acquired by the terminal device according to zero-delay shooting. When the image frame is displayed after shooting, the shooting frame rate will affect the display frame rate of the display screen.

[0062] In step S32, a second shooting frame rate is determined according to the first shooting frame rate.

[0063] In the embodiment of the present disclosure, the second shooting frame rate is greater than the first shooting frame rate, that is, greater than the default shooting frame rate. The second shooting frame rate can be obtained by increasing the first shooting frame rate.

[0064] In step S33, an exposure image frame sequence is acquired based on the second shooting frame rate.

[0065] In the embodiment of the present disclosure, because the second shooting frame rate is twice the default shooting frame rate, the number of image frames shot in the same time period is twice the default frame rate. That is, in the same time period, the number of image frames in the exposure image frame sequence obtained based on the second shooting frame rate is twice the number of image frames in the exposure image frame sequence obtained at the default shooting frame rate.

[0066] In one implementation, doubling the default shooting frequency can prevent preview display jamming. For example, the default shooting frame rate is 30fps, that is, 30 image frames are shot in 1s, and the default display frame rate is 30fps. If long and short exposures are required to be alternating, the number of image frames after exposure in 1s is 15 long exposure image frames and 15 short exposure image frames. At this time, if only long exposure image frames are selected for preview display, the display frame rate is 15fps, which will cause jamming. After doubling the default shooting frame rate, after long and short alternating exposures, the number of image frames after exposure in 1s is 30 long exposure image frames and 30 short exposure image frames. At this time, if only long exposure image frames are selected for preview display, the display frame rate is 30fps, and no jamming will occur. Among them, the default frame rate can be other frame rates other than 30fps, and the image frames can also be obtained by doubling the shooting frame rate to prevent display jamming. Among them, based on the actual shooting needs, the multiple of doubling the default shooting frequency can be other multiples other than 2 times.

[0067] In one embodiment, when only a single camera is used to shoot, the camera can only obtain one frame of image at a time. When a portion of the exposed image frames are selected from the acquired multiple frames for preview display, the preview display may be jittery due to the reduced frame rate. By using the method proposed in the above embodiment of the present disclosure, the shooting frame rate is increased so that the frame rate selected for preview display is the same as the original display frame rate, thereby avoiding preview display jitter.

[0068] In one implementation, the current default shooting frame rate is confirmed and increased to twice the default frame rate to obtain a doubled frame rate. The exposure image frame sequence is acquired according to the doubled frame rate, so that when the long and short alternating exposures are subsequently performed, the preview display screen frame rate will not be reduced and jammed due to the preview display of the long exposure image frame.

[0069] In the above-mentioned embodiment of the present disclosure, in the process of generating photographic imaging, it is necessary to determine a reference image frame from multiple frames of images to be used in generating photographic imaging. The process of determining the reference image frame will be described below.

[0070] Figure 8 The present invention is a flowchart showing a method for determining a reference image frame according to an exemplary embodiment.

[0071] In step S41, image fusion processing is performed on the continuous multiple image frames. The number of image frames after the second exposure in the continuous multiple image frames is the same as the number of image frames after the first exposure and the exposure times are adjacent, and the continuous multiple image frames after image fusion processing are determined.

[0072] In the embodiment of the present disclosure, the continuous multiple image frames include multiple image frames after exposure for the second exposure and multiple image frames after exposure for the first exposure. Among the continuous multiple image frames, the image frames after exposure for the second exposure and the image frames after exposure for the first exposure are all subjected to image processing to obtain the processed continuous multiple image frames.

[0073] In step S42, the stroboscopic bands in the continuous multiple image frames after the image fusion processing are eliminated to obtain multiple image frames with the stroboscopic bands eliminated.

[0074] In the disclosed embodiment, multiple image frames of a continuous multiple image frames are subjected to stroboscopic band elimination, and the obtained multiple image frames with stroboscopic band eliminated are used to determine the reference image frames. The multiple image frames with stroboscopic band eliminated include multiple image frames after exposure for the first exposure with stroboscopic band eliminated and multiple image frames after exposure for the second exposure with stroboscopic band eliminated.

[0075] In step S43, one image frame is determined from the plurality of image frames from which the stroboscopic band is eliminated as a reference image frame.

[0076] In the embodiment of the present disclosure, the reference image frame is determined from a plurality of image frames from which the stroboscopic band is eliminated, so there is no stroboscopic band in the reference image frame.

[0077] In one implementation, the image fusion processing may be performed on the continuous multiple image frames by using an image noise reduction algorithm, or the image fusion processing may be performed on the continuous multiple image frames by using other image fusion noise reduction methods.

[0078] In one implementation, the stroboscopic bands in a plurality of consecutive image frames may be eliminated by using a stroboscopic elimination algorithm.

[0079] In one implementation, based on the long exposure image frames in the continuous multiple image frames, the stroboscopic band can be eliminated from the short exposure image frames by using a stroboscopic band elimination algorithm, while the long exposure image frames are not subjected to stroboscopic band elimination processing.

[0080] In one implementation, a continuous plurality of image frames may be subjected to image fusion processing by an image noise reduction algorithm to obtain a continuous plurality of image frames after processing. A stroboscopic band existing in the continuous plurality of image frames may be eliminated by a stroboscopic elimination algorithm to obtain a plurality of image frames with the stroboscopic band eliminated. A frame may be selected from the plurality of image frames with the stroboscopic band eliminated as a reference frame for subsequent photographic image generation, so that the photographic images generated subsequently do not have the stroboscopic phenomenon.

[0081] Fig. 9 The invention is a flowchart showing a method of determining a reference frame among multiple image frames according to an exemplary embodiment.

[0082] In step S51, a plurality of image frames from which stroboscopic bands are eliminated are determined.

[0083] In the embodiment of the present disclosure, each image frame in the plurality of image frames from which the stroboscopic band is eliminated is determined.

[0084] In step S52 , among the multiple image frames from which the stroboscopic band is eliminated, one image frame for the first exposure is determined as a reference image frame.

[0085] In the embodiment of the present disclosure, the image frame for the first exposure after eliminating the stroboscopic band is the short-exposure image frame after eliminating the stroboscopic band. At this time, the image frame for the second exposure, that is, the long-exposure image frame is not used as the reference image frame.

[0086] In one implementation, the short-exposure image frame with the highest confidence level among the multiple image frames with the stroboscopic band removed can be determined as the reference image frame based on the long-exposure image frame among the multiple image frames with the stroboscopic band removed. The one with the highest confidence level has the best restoration effect.

[0087] In one implementation, a long-exposure image frame in multiple image frames without stroboscopic bands may be eliminated through a correlation image fusion algorithm, and a short-exposure image frame may be determined as a reference image frame from the short-exposure image frames in the multiple image frames without stroboscopic bands.

[0088] In one implementation, for example, three consecutive adjacent short-exposure image frames and three long-exposure image frames can be selected from a zero-delay cache queue for image processing using an image noise reduction algorithm, and the stroboscopic band in the short-exposure frame can be eliminated using a stroboscopic elimination algorithm to obtain three short-exposure frames with the stroboscopic band eliminated. By comparing the three short-exposure frames with the stroboscopic band eliminated with the adjacent long-exposure frames, the short-exposure frame with the best restoration effect is selected as the reference image frame, and a photographic image is generated based on the reference image frame. When performing image processing, short-exposure image frames and long-exposure image frames of other frame numbers can also be selected as continuous multiple image frames as required.

[0089] In the embodiment of the present disclosure, by determining a short exposure frame from multiple image frames with flicker eliminated as a reference frame for generating a photographic image, motion blur in the photographic image generated due to long exposure is reduced, and the flicker phenomenon that may exist in the generated image is avoided.

[0090] In one embodiment, in response to the presence of flickering in ambient light during zero-delay photography, the ambient light flicker frequency is obtained. The shooting frame rate is doubled to obtain an exposure image sequence and cache it. The exposure image frame sequence is an image frame that needs to be exposed in the zero-delay cache queue. The short exposure time is determined according to the default exposure time for zero-delay photography, and the long exposure time is determined according to the ambient light flicker cycle. The long exposure time is an integer multiple of the ambient light flicker cycle. The image frames in the exposure image sequence are exposed alternately in long and short order to obtain an exposed image frame sequence and cache it. The long exposure image frames in the exposed image frame sequence are previewed and displayed, while the short exposure frames are skipped for preview display. In response to detecting a photography instruction, multiple continuous image frames are determined from the exposed image frame sequence for image fusion processing to obtain processed continuous multiple image frames. The processed continuous multiple image frames are processed by a stroboscopic elimination algorithm to eliminate the stroboscopic bands in the processed continuous multiple image frames to obtain multiple image frames with the stroboscopic bands eliminated. A short-exposure frame is determined from the multiple image frames with the stroboscopic band eliminated as a reference image frame, and a final photographed image is generated according to the reference image frame.

[0091] In the disclosed embodiment, during the zero-delay photography process, by alternating long and short exposures and only previewing the long-exposure image frames, the stroboscopic phenomenon in the preview display is avoided. By doubling the shooting frame rate, the preview display freezes caused by only previewing the long-exposure image frames. By determining the reference image frame from the multi-frame image with the stroboscopic elimination to generate the photographed image, the stroboscopic phenomenon that may exist in the photographed image is avoided, the motion blur is reduced, and the clarity of the photographed image is ensured.

[0092] Based on the same concept, an embodiment of the present disclosure also provides a photographing device.

[0093] It is understandable that in order to realize the above functions, the shooting device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0094] Fig.10 is a block diagram of a photographing device according to an exemplary embodiment. Fig.10 The device includes a receiving module 101 and a processing module 102.

[0095] The receiving module 101 is configured to obtain the ambient light flicker frequency and the exposure image frame sequence in response to the presence of ambient light flickering during zero-delay photography, where the exposure image frame sequence is the image frames that need to be exposed in the zero-delay cache queue.

[0096] The processing module 102 is configured to perform a first exposure and a second exposure alternately on the image frames in the exposure image frame sequence, wherein the exposure time of the first exposure is a default exposure time for zero-delay photography, and the exposure time of the second exposure is an integer multiple of the period corresponding to the ambient light flicker frequency, to obtain an exposed image frame sequence; and preview and display the exposed image frames that undergo the second exposure in the exposed image frame sequence.

[0097] In one implementation provided by an embodiment of the present disclosure, the processing module 102 is also used to: in response to detecting a photo-taking instruction, determine a plurality of continuous image frames for image fusion processing from the exposed image frame sequence; determine a reference image frame from the plurality of continuous image frames, and generate a photographed image based on the reference image frame.

[0098] In one implementation provided by an embodiment of the present disclosure, the receiving module 101 obtains an exposure image frame sequence in the following manner: determining a first shooting frame rate, where the first shooting frame rate is a default shooting frame rate for zero-delay photography; determining a second shooting frame rate based on the first shooting frame rate; and obtaining an exposure image frame sequence based on the second shooting frame rate.

[0099] In one implementation manner provided by an embodiment of the present disclosure, the processing module 102 determines a reference image frame from a continuous plurality of image frames in the following manner: performing image fusion processing on the continuous plurality of image frames to determine a continuous plurality of image frames after the image fusion processing, wherein the number of image frames after the second exposure and the number of image frames after the first exposure in the continuous plurality of image frames are the same and the exposure times are adjacent; eliminating stroboscopic bands in the continuous plurality of image frames after the image fusion processing to obtain a plurality of image frames from which the stroboscopic bands are eliminated; and determining an image frame in the plurality of image frames from which the stroboscopic bands are eliminated as the reference image frame.

[0100] In one implementation provided by an embodiment of the present disclosure, the processing module 102 determines an image frame among multiple image frames from which the stroboscopic band is eliminated as a reference image frame in the following manner: among the multiple image frames from which the stroboscopic band is eliminated, determines an image frame for performing a first exposure as a reference image frame.

[0101] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0102] Fig.11 2 is a block diagram of a device 200 for photographing according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0103] Reference Fig.11 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0104] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0105] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0106] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0107] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0108] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0109] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0110] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0111] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0112] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0113] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0114] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0115] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0116] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0117] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0118] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0119] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A shooting method, characterized in that: include: In response to the presence of flickering of ambient light during zero-delay photography, obtaining a flickering frequency of the ambient light and obtaining an exposure image frame sequence, wherein the exposure image frame sequence is an image frame that needs to be exposed in the zero-delay cache queue; Performing a first exposure and a second exposure alternately on the image frames in the exposure image frame sequence, wherein the exposure time of the first exposure is a default exposure time for the zero-delay photography, and the exposure time of the second exposure is an integer multiple of a period corresponding to the ambient light flicker frequency, to obtain an exposed image frame sequence; The exposed image frames subjected to the second exposure in the exposed image frame sequence are previewed and displayed.

2. The shooting method according to claim 1, characterized in that: The method further comprises: In response to detecting a photographing instruction, determining a plurality of continuous image frames to be subjected to image fusion processing from the exposed image frame sequence; A reference image frame is determined from the continuous multiple image frames, and a photographed image is generated based on the reference image frame.

3. The shooting method according to claim 1, characterized in that: The step of acquiring an exposure image frame sequence comprises: Determine a first shooting frame rate, where the first shooting frame rate is a default shooting frame rate for the zero-delay photography; determining a second shooting frame rate according to the first shooting frame rate; Based on the second shooting frame rate, the exposure image frame sequence is acquired.

4. The shooting method according to claim 2, characterized in that: The step of determining a reference image frame from the continuous plurality of image frames comprises: performing image fusion processing on the continuous multiple image frames, wherein the number of image frames after the second exposure in the continuous multiple image frames is the same as the number of image frames after the first exposure and the exposure times are adjacent, and determining the continuous multiple image frames after the image fusion processing; Eliminating the stroboscopic band in the continuous multiple image frames after the image fusion processing, to obtain the multiple image frames without the stroboscopic band; An image frame is determined from the multiple image frames for eliminating the stroboscopic band as the reference image frame.

5. The shooting method according to claim 4, wherein determining one image frame from the multiple image frames from which the stroboscopic band is eliminated as the reference image frame comprises: Among the multiple image frames for eliminating the stroboscopic band, an image frame for performing the first exposure is determined as a reference image frame.

6. A photographing device, characterized in that: include: A receiving module, used for obtaining the ambient light flicker frequency and an exposure image frame sequence in response to the ambient light flickering during the zero-delay photography process, wherein the exposure image frame sequence is the image frames that need to be exposed in the zero-delay cache queue; A processing module is used to alternately perform a first exposure and a second exposure on the image frames in the exposure image frame sequence, wherein the exposure time of the first exposure is a default exposure time for the zero-delay photography, and the exposure time of the second exposure is an integer multiple of the period corresponding to the ambient light flickering frequency, so as to obtain a post-exposure image frame sequence and preview and display the post-exposure image frames that undergo the second exposure in the post-exposure image frame sequence.

7. The photographing device according to claim 6, characterized in that: The processing module is also used for: In response to detecting a photographing instruction, determining a plurality of continuous image frames to be subjected to image fusion processing from the exposed image frame sequence; A reference image frame is determined from the continuous multiple image frames, and a photographed image is generated based on the reference image frame.

8. The photographing device according to claim 6, characterized in that: The receiving module obtains the exposure image frame sequence in the following manner: Determine a first shooting frame rate, where the first shooting frame rate is a default shooting frame rate for the zero-delay photography; determining a second shooting frame rate according to the first shooting frame rate; Based on the second shooting frame rate, the exposure image frame sequence is acquired.

9. The photographing device according to claim 7, characterized in that: The processing module determines the reference image frame from the continuous multiple image frames in the following manner: performing image fusion processing on the continuous multiple image frames, wherein the number of image frames after the second exposure in the continuous multiple image frames is the same as the number of image frames after the first exposure and the exposure times are adjacent, and determining the continuous multiple image frames after the image fusion processing; Eliminating the stroboscopic band in the continuous multiple image frames after the image fusion processing, to obtain the multiple image frames without the stroboscopic band; An image frame is determined from the multiple image frames for eliminating the stroboscopic band as the reference image frame.

10. The photographing device according to claim 9, characterized in that: The processing module determines an image frame from the multiple image frames for eliminating the stroboscopic band as the reference image frame in the following manner: Among the multiple image frames for eliminating the stroboscopic band, an image frame for performing the first exposure is determined as a reference image frame.

11. A photographing device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the shooting method described in any one of claims 1 to 5.

12. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor of the terminal, the terminal is enabled to execute the shooting method described in any one of claims 1 to 5.

Images