Photographing method and device and storage medium

Abstract

The invention relates to a photographing method, a photographing device and a storage medium. The photographing method comprises the following steps: in response to a photographing request received in a process of providing a preview picture by a camera application of the electronic equipment, acquiring a first exposure image frame in a preview image cache queue before the current moment when the photographing request is received, the preview image cache queue comprises preview image frames which are output according to a time sequence and are stored in a cache region when the camera application provides a preview image, and a second exposure image frame and a third exposure image frame are obtained after the current moment, wherein the preview image cache queue comprises the preview image frames which are output according to the time sequence and are stored in the cache region when the camera application provides the preview image; generating a photographed image based on the first exposure image frame, the second exposure image frame and the third exposure image frame; the second exposure is smaller than the first exposure, and the third exposure is larger than the first exposure. According to the invention, the problem of too long photographing time delay can be improved.

Description

Technical Field

[0001] The present disclosure relates to photographing, and more particularly to a photographing method, device and storage medium. Background Art

[0002] With the continuous advancement of technology, more and more users are beginning to use terminal devices with camera functions. However, due to factors such as hardware configuration and user shooting skills, the images obtained by users directly using the terminal to take pictures are often not clear enough and the image quality is also poor. In order to improve this situation, the terminal usually processes the images obtained by taking pictures to obtain clearer and higher quality images.

[0003] In the related art, although image processing can improve image quality, the processing process will cause a high delay, which means that the user needs to wait patiently for a while before seeing the final image result after pressing the shutter, which will undoubtedly reduce the user experience. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a photographing method, device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a photographing method is provided, comprising: in response to receiving a photographing request during a preview screen provided by a camera application of an electronic device, acquiring a first exposure image frame in a preview image cache queue before a current moment when the photographing request is received, and acquiring a second exposure image frame and a third exposure image frame after the current moment, wherein the preview image cache queue includes preview image frames that are output in time sequence and stored in a cache area when the camera application provides a preview screen; generating a photographed image based on the first exposure image frame, the second exposure image frame and the third exposure image frame; the second exposure is less than the first exposure, and the third exposure is greater than the first exposure.

[0006] In one embodiment, the acquiring of a first exposure image frame from a preview image cache queue before a current moment when the photo request is received, and the acquiring of a second exposure image frame and a third exposure image frame after the current moment, comprises: in response to the photo request, generating a first sub-request and a second sub-request in sequence in a time sequence after the current moment, the first sub-request being used to request the acquisition of the first exposure image frame and the second exposure image frame, and the second sub-request being used to request the acquisition of the third exposure image frame; in response to the first sub-request, acquiring the first exposure image frame from the preview image cache queue before the current moment when the photo request is received, and acquiring the second exposure image frame after the current moment; in response to the second sub-request, acquiring the third exposure image frame after the second exposure image frame.

[0007] In one implementation, a preview image is generated based on the second exposure image frame and the third exposure image frame.

[0008] In one embodiment, a third sub-request is generated after the second sub-request is generated, and the third sub-request is used to request to obtain a first exposure image frame; in response to the third sub-request, the first exposure image frame is obtained after the third exposure image frame; and the preview image is generated based on the second exposure image frame and the third exposure image frame, including: generating a preview image based on the second exposure image frame, the third exposure image frame and the first exposure image frame.

[0009] In one embodiment, the acquiring of a first exposure image frame from a preview image cache queue before a current moment when the photo request is received, and the acquiring of a second exposure image frame and a third exposure image frame after the current moment, comprises: in response to the photo request, generating a fourth sub-request, a fifth sub-request and a sixth sub-request in sequence in time sequence after the current moment, the fourth sub-request being used to request the acquisition of the first exposure image frame, the fifth sub-request being used to request the acquisition of the third exposure image frame, and the sixth sub-request being used to request the acquisition of the second exposure image frame; in response to the fourth sub-request, acquiring the first exposure image frame from the preview image cache queue before the current moment when the photo request is received; in response to the fifth sub-request, acquiring the third exposure image frame after the first exposure image frame; and in response to the sixth sub-request, acquiring the second exposure image frame after the third exposure image frame.

[0010] According to a second aspect of an embodiment of the present disclosure, a photographing device is provided, comprising a processing unit, which, in response to receiving a photographing request during a process in which a camera application of an electronic device provides a preview screen, obtains a first exposure image frame in a preview image cache queue before a current moment when the photographing request is received, and obtains a second exposure image frame and a third exposure image frame after the current moment, wherein the preview image cache queue includes preview image frames that are output in time sequence and stored in a cache area when the camera application provides a preview screen; a generating unit, which generates a photographed image based on the first exposure image frame, the second exposure image frame and the third exposure image frame; the second exposure is less than the first exposure, and the third exposure is greater than the first exposure.

[0011] In one embodiment, the generating unit acquires a first exposure image frame in a preview image cache queue before the current moment when the photo request is received, and acquires a second exposure image frame and a third exposure image frame after the current moment in the following manner: in response to the photo request, a first sub-request and a second sub-request are generated in sequence in a time sequence after the current moment, the first sub-request being used to request the acquisition of the first exposure image frame and the second exposure image frame, and the second sub-request being used to request the acquisition of the third exposure image frame; in response to the first sub-request, the first exposure image frame is acquired in the preview image cache queue before the current moment when the photo request is received, and the second exposure image frame is acquired after the current moment; in response to the second sub-request, the third exposure image frame is acquired after the second exposure image frame.

[0012] In one implementation, the generating unit is further configured to generate a preview image based on the second exposure image frame and the third exposure image frame.

[0013] In one embodiment, the generation unit is further used to: generate a third sub-request after generating the second sub-request, the third sub-request being used to request obtaining a first exposure image frame; in response to the third sub-request, obtaining the first exposure image frame after the third exposure image frame; and generating a preview image based on the second exposure image frame and the third exposure image frame, comprising: generating a preview image based on the second exposure image frame, the third exposure image frame and the first exposure image frame.

[0014] In one embodiment, the generating unit acquires a first exposure image frame in a preview image cache queue before the current moment when the photo request is received, and acquires a second exposure image frame and a third exposure image frame after the current moment in the following manner: in response to the photo request, a fourth sub-request, a fifth sub-request and a sixth sub-request are generated in sequence in time sequence after the current moment, the fourth sub-request being used to request the first exposure image frame, the second sub-request being used to request the third exposure image frame, and the sixth sub-request being used to request the second exposure image frame; in response to the fourth sub-request, the first exposure image frame is acquired in the preview image cache queue before the current moment when the photo request is received; in response to the fifth sub-request, the third exposure image frame is acquired after the first exposure image frame; in response to the sixth sub-request, the second exposure image frame is acquired after the third exposure image frame.

[0015] According to a third aspect of an embodiment of the present disclosure, a photographing device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to: execute the photographing method described in the first aspect or any one of the implementations of the first aspect.

[0016] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor, the processor is enabled to execute the photographing method described in the first aspect or any one of the embodiments of the first aspect.

[0017] The technical solution provided by the embodiment of the present disclosure may include the following beneficial effects: when the present disclosure receives a photo request, it obtains the first exposure image frame before the current moment from the preview image cache queue, obtains the second and third exposure image frames after the current moment, and generates a photo image based on the obtained first, second and third exposure image frames. The method optimizes the frame acquisition strategy in the photo acquisition process and reduces the time consumption of frame acquisition.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] Figure 1 A flow chart of a fast HDR photography method is shown.

[0021] Figure 2 A flow chart showing an HDR processing flow is shown.

[0022] Figure 3 A schematic diagram showing a traditional sensor image output method.

[0023] Figure 4 A schematic diagram of a fast HDR photography method for taking frames is shown.

[0024] Figure 5 The figure is a flow chart of a photographing method according to an exemplary embodiment.

[0025] Figure 6 The flowchart of acquiring three image frames with different exposure levels is shown according to an exemplary embodiment.

[0026] Figure 7 The figure is a flow chart of a photographing method according to an exemplary embodiment.

[0027] Figure 8 The figure is a flow chart of a photographing method according to an exemplary embodiment.

[0028] Fig. 9 The figure is a flow chart of a photographing method according to an exemplary embodiment.

[0029] Fig.10 The figure is a schematic diagram showing a frame acquisition strategy according to an exemplary embodiment.

[0030] Fig.11 The figure is a flow chart of a photographing method according to an exemplary embodiment.

[0031] Fig.12 The figure is a schematic diagram showing a frame acquisition strategy according to an exemplary embodiment.

[0032] Fig.13 A flow chart of an ISP module implementing multi-frame noise reduction and image synthesis is shown.

[0033] Fig.14 It is a block diagram of a device for taking photos according to an exemplary embodiment.

[0034] Fig.15 It is a block diagram of a device for taking photos according to an exemplary embodiment. DETAILED DESCRIPTION

[0035] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0036] In the accompanying drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limitations on the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure. The embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings.

[0037] High dynamic range photography technology (HDR) has become a common feature used by current photography users. This feature can bring users a picture quality experience and a better visual experience than non-high dynamic range technology. Therefore, HDR photography technology has a high practical value. Excellent HDR photography technology can bring both a visual experience and a good photography experience.

[0038] Figure 1 A flowchart of a fast HDR photography method is shown. Figure 1 As shown in the figure, the HDR photo processing flow includes the following steps: When the user presses the shutter button, a photo request message is sent. The underlying system sets the exposure parameters of the sensor according to the received photo request. The sensor obtains the original image frame according to the set exposure parameters and generates a raw image. The RAW image is processed by the image signal processing (ISP) hardware and software algorithm to generate multiple photo images. The multi-frame high dynamic fusion algorithm synthesizes the multiple received photo images to generate a photo image with a high dynamic range.

[0039] Depend on Figure 1 It can be seen that the length of time from taking pictures to outputting pictures using the HDR photography technology is affected by the following links: the bottom layer responding to the photo request, taking pictures, exposing, the algorithm receiving the image and the algorithm synthesizing.

[0040] Figure 2 A flow chart showing an HDR processing flow is shown.

[0041] like Figure 2 As shown in the figure. After the sensor generates raw images with different exposures, they need to be processed by the image signal processor (ISP) to generate multiple images for algorithm processing. After ISP processing, multiple images with different exposures are sent to the HDR fusion algorithm to generate a JPG image with a high dynamic range effect.

[0042] Depend on Figure 2 From the content shown, we can see that when the hardware platform is basically the same, the main factors affecting the time length from shooting to image output are exposure and image acquisition. When the exposure parameters are constant, shortening the image output time of the sensor can shorten the time from shooting to image output.

[0043] In view of this, this article discloses a photographing method, in which the frame taking time is shortened by optimizing the frame taking strategy, thereby shortening the time from photographing to image output and improving the photographing efficiency.

[0044] Figure 3A schematic diagram showing a traditional sensor image output method.

[0045] Depend on Figure 3 As shown in the figure, a traditional sensor can only produce one image per frame, and the exposure of this image is determined by the exposure parameters. In the same frame, a traditional sensor can only produce one image with determined exposure parameters.

[0046] Since the HDR photography method requires raw data images with different exposure parameters to determine the photographed image, for traditional sensors, the HDR photography method will set a frame acquisition scheme to obtain multiple frames of raw images with different exposure parameters for determining the photographed image.

[0047] Figure 4 A schematic diagram showing a fast HDR photo taking method frame taking method

[0048] Depend on Figure 4 It can be seen that in the related art, for example, there is a 6-frame delay from the time the user presses the shutter button at frame #30 to the time the sensor starts to output images, and a total of 6 image frames are required from the start of the sensor output to the end of the output. Here, the average time for each frame is 33ms, and a total of 396ms is required to complete the image acquisition stage.

[0049] It is understandable that, since HDR technology requires at least three images with different exposures to generate a photographic image, the image acquisition can only be completed when images with three exposures are obtained.

[0050] Figure 5 is a flow chart of a photographing method according to an exemplary embodiment. Figure 5 As shown, the photographing method includes the following steps S11 to S13.

[0051] In step S11, in response to receiving a photo request during the process of providing a preview image by a camera application of the electronic device, a first exposure image frame is obtained from a preview image cache queue before the current moment when the photo request is received.

[0052] In the embodiment of the present disclosure, the first exposure image frame is obtained from the preview image cache queue before the current moment. It is understandable that when the camera application of the electronic device is awakened, the application will capture real-time images and store these images as preview image frames in the preview cache queue. These preview image frames will not be discarded directly after being previewed by the user, but will continue to exist in the preview cache queue until the preview cache queue is full, and then they will be discarded. Therefore, at the current moment when the photo request is received, there are some preview image frames in the preview cache queue that have not been discarded. It is understandable that these preview image frames have the same exposure. For the convenience of description, the exposure of the preview image frame is referred to as the first exposure in the present disclosure.

[0053] In step S12, a second exposure image frame and a third exposure image frame are acquired after the current moment, and the preview image cache queue includes preview image frames output in time sequence and stored in the cache area when the camera application provides a preview screen. The second exposure is less than the first exposure, and the third exposure is greater than the first exposure.

[0054] In an embodiment of the present disclosure, after receiving a request to take a photo, the camera application of the electronic device will adjust the exposure parameters and take a photo. The photos obtained through the above process will also be stored in the preview cache queue. It can be understood that due to the change in exposure parameters, the exposure of the captured image will also change. Therefore, in the preview cache queue after the current moment, there are image frames with different exposures from the first exposure image frames. For the convenience of description, in the present disclosure, image frames with an exposure less than the first exposure are collectively referred to as second exposure image frames, and image frames with an exposure greater than the first exposure are collectively referred to as third exposure image frames.

[0055] In step S13, a photographic image is generated based on the first exposure image frame, the second exposure image frame, and the third exposure image frame.

[0056] In the related technology, after receiving the current moment of taking a photo request, the camera application of the electronic device continuously shoots image frames of the first exposure level, the second exposure level and the third exposure level in a time sequence. When all the image frames of the three exposure levels are shot, the electronic device can generate a photo image based on the image frames of the three exposure levels.

[0057] In the disclosed embodiment, the first exposure image frame that already exists in the preview cache queue before the current moment of receiving the photo request is used as the first exposure image frame for generating the photo image. At the current moment of receiving the photo request, there is no need to repeatedly shoot the first exposure image frame. The second exposure image frame and the third exposure image frame can be directly shot to collect three image frames of different exposures and generate the photo image based on these image frames. The disclosed embodiment greatly shortens the frame acquisition time by omitting the step of shooting the first exposure image frame, thereby shortening the time from shooting to outputting the image, and improving the shooting efficiency.

[0058] In the above embodiment, three image frames with different exposures need to be obtained from the pre-buffer queue. In view of this, the embodiment of the present disclosure provides a method for obtaining three image frames with different exposures.

[0059] Figure 6 is a flow chart of obtaining three image frames with different exposures according to an exemplary embodiment. Figure 6 As shown, the steps include steps S21 to S23.

[0060] In step S21, in response to the photo request, a first sub-request and a second sub-request are generated in sequence after the current moment in time, the first sub-request is used to request to obtain a first exposure image frame and a second exposure image frame, and the second sub-request is used to request to obtain a third exposure image frame.

[0061] In the embodiment of the present disclosure, after receiving a photo request, the camera application generates two sub-requests in chronological order: a first sub-request and a second sub-request. The first sub-request is used to obtain a first exposure image frame and a second exposure image frame, while the second sub-request is used to obtain a third exposure image frame. These sub-requests are generated in chronological order to ensure that the required image frames of different exposures can be obtained before multi-frame fusion is performed.

[0062] In step S22, in response to the first sub-request, a first exposure image frame is acquired from the preview image cache queue before the current moment when the photo-taking request is received, and a second exposure image frame is acquired after the current moment.

[0063] In the disclosed embodiment, after receiving a photo request, the camera application determines the current moment according to the current time. In the preview image cache queue, the preview image frames captured previously are saved in chronological order. For the first sub-request, the first exposure image frame closest to the current moment is found from the preview image cache queue before the current moment. After the current moment, the second exposure image frame is obtained from the preview image cache queue. Through the above steps, the first sub-request is successfully responded to, and the image frames of the first exposure and the second exposure are obtained.

[0064] In step S23, in response to the second sub-request, a third exposure image frame is acquired after the second exposure image frame.

[0065] In the disclosed embodiment, after the first sub-request is responded to, the image frame of the first exposure and the image frame of the second exposure have been successfully acquired, and only the image frame of the third exposure is missing. By responding to the second sub-request, the electronic device acquires the third image frame after the image frame of the second exposure is acquired. After the second sub-request is responded to, the electronic device has collected all the image frames of the three exposures, and has collected all the image frames required to generate the photographed image.

[0066] The embodiment of the present disclosure provides a method for obtaining image frames of three different exposures. Compared with the method in the prior art in which an electronic device shoots image frames of different exposures after receiving a photo-taking instruction and randomly obtains three image frames of different exposures from a set of shot image frames, the method provided by the embodiment of the present disclosure can obtain image frames of all three exposures by executing two sub-instructions in a time sequence, and collects all image frames required to generate a photographed image in a concise and efficient manner.

[0067] In the above embodiment, when the frame acquisition is completed, the exposure of the last image frame is the second exposure or the third exposure. Since the frame is acquired in the pre-cache queue, no new image frames will be stored in the pre-cache queue after the frame acquisition process is completed. However, the last image frame in the pre-cache queue will be displayed to the user as a preview image. Since the exposure of the last image frame is the second exposure or the third exposure at this time, the image frame is not clear, so it is inappropriate to display the image frame as a preview image to the customer. In view of this, a photographing method is provided in the embodiment of the present disclosure, based on which an image frame with appropriate exposure can be displayed to the customer as a preview image.

[0068] Figure 7 is a flow chart of a photographing method according to an exemplary embodiment. Figure 7 As shown, the photographing method includes the following steps S31 to S33.

[0069] In step S31, in response to receiving a photo request during the process of providing a preview image by a camera application of the electronic device, a first exposure image frame is obtained from a preview image cache queue before the current moment when the photo request is received.

[0070] In step S32, a second exposure image frame and a third exposure image frame are acquired after the current moment, and the preview image cache queue includes preview image frames output in time sequence and stored in the cache area when the camera application provides a preview screen. The second exposure is less than the first exposure, and the third exposure is greater than the first exposure.

[0071] In step S33, a photographic image is generated based on the first exposure image frame, the second exposure image frame, and the third exposure image frame.

[0072] The above steps have been Figure 2 The corresponding embodiments are described in detail and will not be repeated here.

[0073] In step S34 , a preview image is generated based on the second exposure image frame and the third exposure image frame.

[0074] In the embodiment of the present disclosure, since the image frame of the first exposure is no longer stored in the cache queue after the current moment when the photo request is received, there is no ready-made image frame of the first exposure that can be used to display to the user as a preview image. Therefore, in order to generate a clear preview image, the embodiment of the present disclosure generates a preview image based on the second exposure image frame and the third exposure image frame. Since the exposure of the second exposure image frame is lower than that of the first exposure image frame, and the exposure of the third exposure image frame is higher than that of the first exposure image frame, the exposure of the image generated based on the two will be in an appropriate range, so that the image is clear enough to be displayed to the customer as a preview image.

[0075] Figure 8 is a flow chart of a photographing method according to an exemplary embodiment. Figure 8 As shown, the photographing method includes the following steps S35 to S38.

[0076] In step S35, in response to the photo request, a first sub-request and a second sub-request are generated in sequence after the current moment, the first sub-request is used to request the acquisition of a first exposure image frame and a second exposure image frame, and the second sub-request is used to request the acquisition of a third exposure image frame.

[0077] In the embodiment of the present disclosure, after receiving a photo request, the camera application generates two sub-requests in chronological order: a first sub-request and a second sub-request. The first sub-request is used to obtain a first exposure image frame and a second exposure image frame, while the second sub-request is used to obtain a third exposure image frame. These sub-requests are generated in chronological order to ensure that the required image frames of different exposures can be obtained before multi-frame fusion is performed.

[0078] In step S36, in response to the first sub-request, a first exposure image frame is acquired from the preview image cache queue before the current moment when the photo-taking request is received, and a second exposure image frame is acquired after the current moment.

[0079] In the disclosed embodiment, after receiving a photo request, the camera application determines the current moment according to the current time. In the preview image cache queue, the preview image frames captured previously are saved in chronological order. For the first sub-request, the first exposure image frame closest to the current moment is found from the preview image cache queue before the current moment. After the current moment, the second exposure image frame is obtained from the preview image cache queue. Through the above steps, the first sub-request is successfully responded to, and the image frames of the first exposure and the second exposure are obtained.

[0080] In step S37 , in response to the second sub-request, a third exposure image frame is acquired after the second exposure image frame.

[0081] In step S38 , a preview image is generated based on the second exposure image frame and the third exposure image frame.

[0082] The above steps have been Figure 3 and Figure 4 The corresponding embodiments are described in detail and will not be repeated here.

[0083] In the above embodiment, although the exposure of the preview image is within a suitable range, it may still not be so clear. In view of this, the embodiment of the present disclosure provides a photographing method, based on which an image frame with a more suitable exposure can be displayed to a customer as a preview image.

[0084] Fig. 9 is a flow chart of a photographing method according to an exemplary embodiment. Fig. 9 As shown, the photographing method includes the following steps S41 to S45.

[0085] In step S41, in response to a photo-taking request, a first sub-request and a second sub-request are generated in sequence after the current moment, the first sub-request is used to request a first exposure image frame and a second exposure image frame, and the second sub-request is used to request a third exposure image frame.

[0086] In step S42, in response to the first sub-request, a first exposure image frame is acquired from the preview image cache queue before the current moment when the photo-taking request is received, and a second exposure image frame is acquired after the current moment.

[0087] In step S43, in response to the second sub-request, a third exposure image frame is acquired after the second exposure image frame.

[0088] The above steps have been Figure 2 The corresponding embodiments are described in detail and will not be repeated here.

[0089] In step S44, a third sub-request is generated after the second sub-request is generated, and the third sub-request is used to request to obtain the first exposure image frame. In response to the third sub-request, the first exposure image frame is obtained after the third exposure image frame.

[0090] In step S45 , a preview image is generated based on the second exposure image frame, the third exposure image frame, and the first exposure image frame.

[0091] In the disclosed embodiment, after the second sub-request is generated, a third sub-request is generated according to the timing, and the request is intended to obtain a first exposure image frame. The response to the third sub-request is to obtain the first exposure image frame after the third exposure image frame. The advantage of this is that since the request needs to take the first exposure image frame from the pre-cache queue after the current moment of the photo request, the first exposure image frame will be supplemented in the pre-cache queue. At this time, the pre-cache queue has the first exposure image frame, the second exposure image frame and the third exposure image frame at the same time. Based on the three image frames, a preview image with a more suitable exposure can be generated.

[0092] Fig.10The figure is a schematic diagram showing a frame acquisition strategy according to an exemplary embodiment.

[0093] In one implementation, the frame-taking strategy is as follows: Fig.10 As shown. Assume that the exposure combination of the HDR photo request is fixed to 4 frames of EV0, 1 frame of EV- and 1 frame of EV+. After the user presses the shutter, the APP sends the request to the bottom layer in the first sub-request (#Req1), the second sub-request (#Req2) and the third sub-request (#Req3). Each #Req contains a preview stream and a photo stream. The preview stream is mainly responsible for feeding the image back to the preview interface, and the photo stream provides the image to the subsequent multi-frame HDR algorithm for fusion processing.

[0094] Among them, EV0 is a normally exposed image frame, and its exposure corresponds to the first exposure, EV- is an underexposed image frame, and its exposure corresponds to the second exposure, and EV+ is an overexposed image frame, and its exposure corresponds to the third exposure. It can be understood that since EV0 does not perform exposure compensation, there is more noise in the image frame, so 4 frames of EV0 are needed to perform noise reduction processing on the first exposure image frame.

[0095] In one implementation, when #Req1 is issued, the preview stream will output a frame #37 to the preview cache queue, and the photo stream will take out the original RAW images #33, #34, #35, and #36 in the cache for subsequent ISP processing. When #Req2 is issued, the preview stream generates a preview image #38 and stores it in the preview cache queue, and the photo stream takes out #37 for processing; when #Req3 is issued, the preview stream generates a preview image #39 and stores it in the preview cache queue, and the photo stream takes out #38 for processing. At the same time, the exposure parameters also need to be adjusted accordingly to ensure that the last frame #39 is not overexposed or too dark when displayed in the preview. Compared with the traditional frame-taking strategy, the frame-taking time in this article is optimized from the previous 396ms to 264ms, with an improvement rate of 33.3%.

[0096] Among them, #33, #34, #35, and #36 are EV0, that is, the exposure of these four frames of images is the first exposure. #37 is EV-, and the exposure of this frame is the second exposure. #38 is EV+, and the exposure of this frame is the third exposure. #39 is EV0, and the exposure of this frame is the first exposure. In response to the first sub-instruction, the image frames #33, #34, #35, and #36 with the first exposure are taken, and the image frame #37 with the second exposure is taken. In response to the second sub-instruction, the image frame #38 with the third exposure is taken. In response to the third sub-instruction, the image frame #39 with the first exposure is taken.

[0097] After executing the first, second, and third sub-instructions in sequence, 7 image frames are obtained from the pre-cache queue. Among them, #33, #34, #35, #36, and #39 have the first exposure, #33, #34, #35, and #36 are only used to generate photo images, and #39 is only used to generate preview images. The exposure of #37 is the second exposure, and this frame is used to generate both photo images and preview images. The exposure of #38 is the third exposure, and this frame is used to generate both photo images and preview images. When the first and second sub-instructions are executed, #33, #34, #35, #36, #37, and #38 obtained from the pre-cache queue are sufficient to generate photo images. Therefore, the image acquisition process for the purpose of taking pictures starts from frame #31 and ends at frame #38. The frame acquisition time is optimized from the previous 396ms to 264ms, and the improvement rate reaches 33.3%. The purpose of taking the #39 frame is to generate a preview image. The process of generating the preview image and the process of generating the photo image can be carried out simultaneously, so the time consumed in taking the #39 frame is not included in the image taking process.

[0098] The present disclosure also provides another photographing method that can provide a suitable preview image.

[0099] Fig.11 is a flow chart of a photographing method according to an exemplary embodiment. Fig.11 As shown, the photographing method includes the following steps S51 to S54.

[0100] In step S51, in response to the photo request, a fourth sub-request, a fifth sub-request and a sixth sub-request are generated in sequence after the current moment in time. The fourth sub-request is used to request to obtain a first exposure image frame, the fifth sub-request is used to request to obtain a third exposure image frame, and the sixth sub-request is used to request to obtain a second exposure image frame.

[0101] In step S52, in response to the fourth sub-request, a first exposure image frame is obtained from the preview image cache queue before the current moment when the photo-taking request is received.

[0102] In step S53 , in response to the fifth sub-request, a third exposure image frame is acquired after the first exposure image frame.

[0103] In step S54, in response to the sixth sub-request, a second exposure image frame is acquired after the third exposure image frame.

[0104] In the embodiment of the present disclosure, after the current moment of receiving the photo request, the fourth sub-request is first generated, and the fourth sub-request is used to request to obtain the first exposure image frame. In this way, after the current moment of receiving the photo request, the first exposure image frame will exist in the pre-cache queue. Then, the fifth sub-request is generated to request to obtain the third exposure image frame and the sixth sub-request is generated to request to obtain the second exposure image frame. In this way, the pre-cache queue collects three different exposure image frames, and a preview image with suitable exposure can be generated.

[0105] It is understandable that the fourth sub-request is generated in the embodiment of the present disclosure, and the fourth sub-request is used to request the acquisition of the first exposure image frame, only to ensure that the first exposure image frame exists in the pre-cached queue after the current moment when the photo request is received, and it is not necessary to obtain all the required first exposure image frames in the pre-cached queue after the current moment. Therefore, in the solution in the embodiment of the present disclosure, after obtaining the first exposure image frame, the time from taking the photo to outputting the image is still shorter than the time used in the related art.

[0106] Fig.12 The figure is a schematic diagram showing a frame acquisition strategy according to an exemplary embodiment.

[0107] In one implementation, the specific frame grabbing strategy is as follows. When the fourth sub-request (#Req4) is issued, the preview stream will output a frame #37 to the preview cache queue, and the photo stream will take out the original RAW images #33, #34, #35, #36 and the images taken by #37 in the cache for subsequent ISP processing. When the fifth sub-request #Req5 is issued, the preview stream will output a frame #38 to the preview cache queue, and the photo stream will take out #38 for processing; when the sixth sub-request #Req6 is issued, the preview stream will output a frame #39 to the preview cache queue, and the photo stream will take out #39 for processing. Compared with the traditional frame grabbing strategy, the frame grabbing time in this article is optimized from the previous 396ms to 297ms, with an improvement rate of 25%.

[0108] Among them, #33, #34, #35, and #36 are EV0, that is, the exposure of these four frames of images is the first exposure. #37 is EV0, and the exposure of this frame is the first exposure. #38 is EV-, and the exposure of this frame is the second exposure. #39 is EV+, and the exposure of this frame is the first exposure. In response to the fourth sub-instruction, the four image frames #33, #34, #35, and #36 with the first exposure are taken, and at the same time, the image frame #37 with the first exposure is taken. In response to the fifth sub-instruction, the image frame #38 with the second exposure is taken. In response to the third sub-instruction, the image frame #39 with the third exposure is taken. After executing the fourth, fifth, and sixth sub-instructions in sequence, 7 image frames are obtained from the pre-cache queue. Among them, #33, #34, #35, #36, and #37 have the first exposure. #33, #34, #35, and #36 are only used to generate the photo image. #37 is used to generate both the preview image and the photo image. #38 has the second exposure, and this frame is used to generate both the photo image and the preview image. #39 has the third exposure, and this frame is used to generate both the photo image and the preview image. The image acquisition process for the purpose of taking photos starts from #31 and ends at #38. The frame acquisition time is optimized from 396ms to 297ms, with an improvement rate of 25%.

[0109] Compared to Fig.10 The frame-taking strategy in Fig.12 Although the frame-taking strategy in can obtain a preview image with a more appropriate exposure by synthesizing three exposure image frames, the total number of frames taken is 7, which is less than that in Fig.10 The overall improvement rate dropped from 33.3% to 25% by using the frame extraction strategy in the video. Although the improvement rate can be increased by removing the #37 frame, this will increase the probability of generating ghost images. Fig.10 The frame-taking strategy shown is a better solution.

[0110] Fig.13 A flow chart of an ISP module implementing multi-frame noise reduction and image synthesis is shown.

[0111] like Fig.13 As shown in the figure, the main processes of the ISP module to achieve multi-frame noise reduction and image synthesis are as follows: PDPC (phase focus), Demosaic (demosaicing), CC (color correction), GTM (local brightening), Gamma (global brightening), HNR (grid noise reduction), Down Scaler (down sampling), and finally the RAW image is processed into a YUV format image.

[0112] In the disclosed embodiment, the first exposure image frame already existing in the preview cache queue is used to generate the photographed image. When a photographing request is received, there is no need to shoot the first exposure image frame again. It is only necessary to shoot the image frames of the second exposure and the third exposure to obtain three image frames of different exposures, and generate the photographed image based on them. By omitting the step of shooting the first exposure image frame, the frame acquisition time is greatly shortened, thereby speeding up the speed from shooting to outputting the image, and improving the efficiency of shooting.

[0113] Based on the same concept, an embodiment of the present disclosure also provides a photographing device.

[0114] It is understandable that the photographing device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0115] Fig.14 is a block diagram of a photographing device according to an exemplary embodiment. Fig.14 , the device includes a processing unit 101 and a generating unit 102.

[0116] The processing unit 101 is configured to: in response to receiving a photo request during the process of providing a preview screen by a camera application of the electronic device, obtain a first exposure image frame in a preview image cache queue before the current moment when the photo request is received, and obtain a second exposure image frame and a third exposure image frame after the current moment, wherein the preview image cache queue includes preview image frames that are output in time sequence and stored in the cache area when the camera application provides a preview screen.

[0117] The generating unit 102 is configured to generate a photographed image based on a first exposure image frame, a second exposure image frame and a third exposure image frame; the second exposure is less than the first exposure, and the third exposure is greater than the first exposure.

[0118] In one embodiment, the processing unit 101 obtains a first exposure image frame in a preview image cache queue before the current moment when a photo request is received, and obtains a second exposure image frame and a third exposure image frame after the current moment in the following manner: in response to the photo request, a first sub-request and a second sub-request are generated in sequence in a time sequence after the current moment, the first sub-request being used to request the first exposure image frame and the second exposure image frame, and the second sub-request being used to request the third exposure image frame; in response to the first sub-request, the first exposure image frame is obtained in the preview image cache queue before the current moment when the photo request is received, and the second exposure image frame is obtained after the current moment; in response to the second sub-request, the third exposure image frame is obtained after the second exposure image frame.

[0119] In one implementation, the generating unit 102 is further configured to generate a preview image based on the second exposure image frame and the third exposure image frame.

[0120] In one embodiment, the generation unit 102 is further used to: generate a third sub-request after generating the second sub-request, the third sub-request being used to request obtaining the first exposure image frame; in response to the third sub-request, obtaining the first exposure image frame after the third exposure image frame; and generating a preview image based on the second exposure image frame and the third exposure image frame, including: generating a preview image based on the second exposure image frame, the third exposure image frame and the first exposure image frame.

[0121] In one embodiment, the processing unit 101 obtains a first exposure image frame in a preview image cache queue before the current moment when a photo request is received, and obtains a second exposure image frame and a third exposure image frame after the current moment in the following manner: in response to the photo request, a fourth sub-request, a fifth sub-request and a sixth sub-request are generated in sequence in time sequence after the current moment, the fourth sub-request is used to request to obtain the first exposure image frame, the second sub-request is used to request to obtain the third exposure image frame, and the sixth sub-request is used to request to obtain the second exposure image frame; in response to the fourth sub-request, the first exposure image frame is obtained in the preview image cache queue before the current moment when the photo request is received; in response to the fifth sub-request, the third exposure image frame is obtained after the first exposure image frame; in response to the sixth sub-request, the second exposure image frame is obtained after the third exposure image frame.

[0122] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0123] Fig.152 is a block diagram of a device 200 for taking photos according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0124] Reference Fig.15 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0125] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0126] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, images, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0127] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0128] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0129] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0130] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0131] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0132] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0133] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0134] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0135] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar thereto. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include the plural forms, unless the context clearly indicates other meanings.

[0136] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0137] It will be further understood that the terms “center”, “longitudinal”, “lateral”, “front”, “back”, “up”, “down”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0138] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0139] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0140] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0141] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A photographing method, characterized in that: The photographing method comprises: In response to receiving a photo request during the process of providing a preview screen by a camera application of the electronic device, acquiring a first exposure image frame in a preview image cache queue before a current moment when the photo request is received, and acquiring a second exposure image frame and a third exposure image frame after the current moment, wherein the preview image cache queue includes preview image frames output in time sequence and stored in a cache area when the camera application provides a preview screen; Generate a photographic image based on the first exposure image frame, the second exposure image frame, and the third exposure image frame; The second exposure level is lower than the first exposure level, and the third exposure level is higher than the first exposure level.

2. The photographing method according to claim 1, characterized in that: The method of acquiring a first exposure image frame from a preview image cache queue before the current moment of receiving the photo request, and acquiring a second exposure image frame and a third exposure image frame after the current moment, comprises: In response to the photographing request, a first sub-request and a second sub-request are generated in sequence after the current moment in time, wherein the first sub-request is used to request to obtain a first exposure image frame and a second exposure image frame, and the second sub-request is used to request to obtain a third exposure image frame; In response to the first sub-request, acquiring a first exposure image frame from a preview image cache queue before a current moment when the photo-taking request is received, and acquiring a second exposure image frame after the current moment; In response to the second sub-request, a third exposure image frame is acquired after the second exposure image frame.

3. The photographing method according to claim 1 or 2, characterized in that: The method further comprises: A preview image is generated based on the second exposure image frame and the third exposure image frame.

4. The photographing method according to claim 3, characterized in that: The method further comprises: Generate a third sub-request after the timing of generating the second sub-request, wherein the third sub-request is used to request to obtain the first exposure image frame; In response to the third sub-request, acquiring a first exposure image frame after the third exposure image frame; The generating a preview image based on the second exposure image frame and the third exposure image frame comprises: A preview image is generated based on the second exposure image frame, the third exposure image frame, and the first exposure image frame.

5. The photographing method according to claim 1, characterized in that: The method of acquiring a first exposure image frame from a preview image cache queue before the current moment of receiving the photo request, and acquiring a second exposure image frame and a third exposure image frame after the current moment, comprises: In response to the photographing request, a fourth sub-request, a fifth sub-request and a sixth sub-request are generated in sequence after the current moment in time, wherein the fourth sub-request is used to request to obtain a first exposure image frame, the fifth sub-request is used to request to obtain a third exposure image frame, and the sixth sub-request is used to request to obtain a second exposure image frame; In response to the fourth sub-request, obtaining a first exposure image frame from a preview image cache queue before the current moment when the photo request is received; In response to the fifth sub-request, acquiring a third exposure image frame after the first exposure image frame; In response to the sixth sub-request, a second exposure image frame is acquired after the third exposure image frame.

6. A photographing device, characterized in that: include: a processing unit, in response to receiving a photo request during a process in which a camera application of the electronic device provides a preview image, acquiring a first exposure image frame from a preview image cache queue before a current moment when the photo request is received, and acquiring a second exposure image frame and a third exposure image frame after the current moment, wherein the preview image cache queue includes preview image frames that are output in time sequence and stored in a cache area when the camera application provides a preview image; A generating unit, which generates a photographic image based on the first exposure image frame, the second exposure image frame and the third exposure image frame; The second exposure level is lower than the first exposure level, and the third exposure level is higher than the first exposure level.

7. The photographing device according to claim 6, characterized in that: The processing unit obtains a first exposure image frame from a preview image cache queue before the current moment of receiving the photo request, and obtains a second exposure image frame and a third exposure image frame after the current moment in the following manner: In response to the photographing request, a first sub-request and a second sub-request are generated in sequence after the current moment in time, wherein the first sub-request is used to request to obtain a first exposure image frame and a second exposure image frame, and the second sub-request is used to request to obtain a third exposure image frame; In response to the first sub-request, acquiring a first exposure image frame from a preview image cache queue before a current moment when the photo-taking request is received, and acquiring a second exposure image frame after the current moment; In response to the second sub-request, a third exposure image frame is acquired after the second exposure image frame.

8. The photographing device according to claim 6 or 7, characterized in that: The generating unit is further configured to: A preview image is generated based on the second exposure image frame and the third exposure image frame.

9. A photographing device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the photographing method described in any one of claims 1 to 5.

10. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by the processor, the processor is enabled to execute the photographing method described in any one of claims 1 to 5.

Images