Electronic device and photography time control method thereof

Abstract

An electronic device and a photography time control method thereof are provided. The method is adapted to the electronic device including an image sensor and includes the following steps. A photographing interval time is determined according to a predetermined number of continuous shots and a maximum storage capacity of a frame queue. The photographing interval time is used to determine whether a user photographing request is accepted. When the user photographing request is accepted, multiple image frames captured by the image sensor are temporarily stored in the frame queue. A composite image is generated based on the image frames in the frame queue, and the composite image is stored in a format that complies with an image storage format.

Description

[Technical Field]

[0001] This disclosure relates to a method for controlling the time of taking a photograph and an electronic device using this method. [Previous Technology]

[0002] With the continuous advancement of technology, electronic devices with image capture capabilities have become deeply integrated into modern people's daily lives. To further improve photo quality, many devices employ multi-frame synthesis algorithms to optimize imaging effects. However, during the process of multi-frame image capture, the multiple frames captured by the image sensor are usually first stored in the frame queue, and then processed by a multi-threaded system for image enhancement and synthesis. However, due to limited system resources, the frame queue may become full, severely impacting the user's shooting experience. For example, when a user takes a rapid burst of shots, the first four photos may be captured normally, but when the fifth photo is taken, the shooting function may not be able to continue operating because the frame queue is full (i.e., the image enhancement and synthesis processing of the first four photos has not yet been completed), preventing the user from completing the expected shooting needs in time. This situation confuses the user and interrupts the photography experience, significantly reducing the user experience. [Summary of the Invention]

[0003] This disclosure provides a method for controlling the time of taking photos, which is used in an electronic device including an image sensor. The method includes the following steps: determining a photo interval based on a preset number of consecutive shots and the available storage capacity of an image frame queue; deciding whether to accept a user's photo request based on the photo interval; temporarily storing multiple image frames captured by the image sensor into the image frame queue when the user's photo request is accepted; generating a composite image based on the multiple image frames in the image frame queue, and storing the composite image in accordance with an image storage format.

[0004] This disclosure also provides an electronic device including an image sensor and a processor. The processor is coupled to the image sensor. The processor is configured to perform the following operations: determine a shooting interval based on a preset number of consecutive shots and the available storage capacity of an image frame queue; determine whether to accept a user's shooting request based on the shooting interval; when the user's shooting request is accepted, temporarily store multiple image frames captured by the image sensor in the image frame queue; generate a composite image based on the multiple image frames in the image frame queue, and store the composite image conforming to an image storage format.

[0005] Based on the above, in this disclosed embodiment, the photo-taking interval can be determined according to the preset number of consecutive shots and the available storage capacity of the image frame queue. Then, the system can decide whether to accept or reject the user's photo-taking request based on the photo-taking interval. This avoids the unpleasant experience of unpredictable long-term photography interruptions due to a full image frame queue, thereby allowing users to obtain a better photography experience.

Implementation Method

[0006] Exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Wherever possible, the same component reference numerals are used in the drawings and description to denote the same or similar parts. These embodiments are only a part of the present disclosure and do not reveal all possible implementations of the present disclosure. Rather, these embodiments are merely examples of apparatuses and methods within the scope of the patent application claims of this disclosure.

[0007] Referring to Figure 1, the electronic device 100 may include an image sensor 110, an image signal processor (ISP) 120, a storage device 130, a display 140, and a processor 150. The electronic device 100 may be, for example, a smartphone, a digital camera, a tablet computer, a game console, a wearable electronic device, or a photographic device, or other electronic devices with image capture capabilities, and the type of electronic device 100 is not limited to these.

[0008] The image sensor 110 is used to capture images and may include a lens, an image sensing element, and other components. The lens may include an optical lens for optical path control. The image sensing element provides image sensing functionality. The image sensing element may include a photosensitive element, such as a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) element, or other elements, which are not limited herein. The lens can converge imaging light onto the image sensing element to achieve the purpose of capturing images.

[0009] The image signal processor (ISP) 120 is used to process image data in real time. The image signal processor 120 can obtain raw image data from the image sensor 110 and perform front-end image processing on the raw image data. For example, the image signal processor 120 can perform image optimization processing on the raw image data such as contrast enhancement, color correction, sharpening, and noise reduction.

[0010] Storage device 130 is used to store data such as files, images, instructions, code, software modules, etc., and may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar device, integrated circuit or combination thereof.

[0011] The display 140 can be any type of display, such as a liquid crystal display (LCD), a light-emitting diode (LED) display, or an organic light-emitting diode (OLED), and this disclosure is not limited thereto. The display 140 can be used to display the program operation interface of the camera application, the shooting preview screen, and the shooting result screen, etc.

[0012] Processor 150 is coupled to display 140, image sensor 110, and storage device 130. It is, for example, a central processing unit (CPU), application processor (AP), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), image signal processor (ISP), graphics processing unit (GPU), or other similar devices, integrated circuits, or combinations thereof. In some embodiments, processor 150 may execute instructions or program code in storage device 130 to implement the steps of the time-of-capture method in this disclosed embodiment.

[0013] FIG2 is a flowchart illustrating a photographing time control method according to an embodiment of the present disclosure. Referring to FIG2, the method of this embodiment can be executed by the electronic device 100 of FIG1. ​​The details of each step in FIG2 are explained below with reference to the components shown in FIG1.

[0014] In step S210, the processor 150 determines a shooting interval based on a preset number of consecutive shots and the available storage capacity of an image frame queue. Specifically, the image frame queue is used to buffer image frames captured by the image sensor 110 so that the processor 150 can perform subsequent image post-processing on these image frames. In some embodiments, the image frame queue may be established by a photography application or image processing framework. The aforementioned image post-processing is, for example, HDR compositing to generate High Dynamic Range Imaging (HDR) images, or image compositing to generate long exposure images, etc., and this disclosure is not limited thereto. When performing image post-processing, the processor 150 needs to use multiple image frames to generate a composite image. For example, the processor 150 needs 5 image frames for HDR compositing to generate one HDR image.

[0015] It should be noted that the number of image frames that can be temporarily stored in the image frame queue is limited, that is, the image frame queue has available storage capacity. For example, the image frame queue can only store a maximum of multiple image frames used to generate N composite images. The available storage capacity of the image frame queue may depend on factors such as the computing and storage capabilities of the electronic device 100 and the complexity of image post-processing. When the image frame queue has already stored multiple image frames used to generate 2 composite images, the available storage capacity of the image frame queue only allows for storing (N-2) more image frames for composite images.

[0016] In some embodiments, the preset burst shot count can be a fixed preset value. Alternatively, the preset burst shot count can be a dynamic value determined by the user or the shooting context. Furthermore, the preset burst shot count can be manually set by the user. Alternatively, the preset burst shot count can be determined by the processor 150 based on the shooting context or the shooting mode selected by the user. The preset burst shot count is at least greater than 1.

[0017] In some embodiments, the processor 150 may determine the shooting interval based on a preset number of consecutive shots and the available storage capacity of the image frame queue, wherein the shooting interval may be the shooting time required to execute one shooting procedure. Specifically, to avoid the image frame queue becoming full, causing the electronic device 100 to be unable to respond to the user's shooting request and capture images, the processor 150 may extend the shooting time required for each shooting procedure to ensure that the image frame queue has sufficient space to temporarily store new image frames each time. In some embodiments, the processor 150 may determine the shooting interval by comparing a preset number of consecutive shots and the available storage capacity of the image frame queue.

[0018] In step S220, the processor 150 determines whether to accept the user's photo-taking request based on the photo-taking interval. In some embodiments, the processor 150 can detect whether the shutter button is pressed to detect the user's photo-taking request. The shutter button can be a physical button or a virtual button displayed on a touch screen. When the shutter button is pressed, the processor 150 can receive the user's photo-taking request. When a user's photo-taking request is received, the processor 150 will determine whether the previous photo-taking process triggered by the previous user's photo-taking request has ended based on the photo-taking interval. If the processor 150 determines that the previous photo-taking process has not ended based on the photo-taking interval, the processor 150 will not accept the user's photo-taking request until the previous photo-taking process is completed. If the processor 150 determines that the previous photo-taking process has ended based on the photo-taking interval, the processor 150 will accept the user's photo-taking request.

[0019] For example, referring to Figure 3A, the processor 150 can receive the previous user's photo-taking request at time point t1 and record M image frames ImgF1_1 to ImgF1_M into image frame queue Q1 according to the previous user's photo-taking request. Then, if the processor 150 receives a user's photo-taking request at time point t2, which is before time point t3, the processor 150 will not accept this user's photo-taking request. As shown in Figure 3A, since the user issued the photo-taking request within the photo-taking interval ΔTG between time point t1 and time point t3, the processor 150 will not accept the user's photo-taking request at time point t2. The photo-taking interval ΔTG can be the program execution time of the previous photo-taking procedure for the previous user's photo-taking request.

[0020] In step S230, when a user's request to take a photo is accepted, the processor 150 temporarily stores the multiple image frames captured by the image sensor 110 into the image frame queue.

[0021] For example, please refer to Figure 3B. The processor 150 may receive the previous user photo-taking request at time point t1 and record M image frames ImgF1_1 to ImgF1_1 into image frame queue Q1 according to the previous user photo-taking request. Subsequently, if the processor 150 receives the user photo-taking request at time point t4 after time point t3, the processor 150 will accept the current user photo-taking request because the previous photo-taking procedure has ended. As shown in Figure 3B, since the user submits the photo-taking request after the photo-taking interval ΔTG required by the previous photo-taking procedure, the processor 150 will accept the user photo-taking request at time point t4. Subsequently, the processor 150 may record M image frames ImgF2_1 to ImgF2_M into image frame queue Q1 according to the user photo-taking request at time point t4.

[0022] In step S240, the processor 150 generates a composite image based on multiple image frames in the image frame queue and stores the composite image in an image saving format. The processor 150 can read multiple image frames used to generate a composite image from the image frame queue. Then, the processor 150 can perform image post-processing on the multiple image frames to synthesize the composite image and compress it into an image saving format for saving. For example, the image saving format is JPEG, but it is not limited to this. When the processor 150 completes the generation and storage of a composite image, the image frame queue can release the multiple image frames associated with that composite image. By releasing image frames that are no longer needed, the image frame queue can be kept within a reasonable capacity range.

[0023] FIG4 is a flowchart illustrating a photographing time control method according to an embodiment of the present disclosure. Referring to FIG4, the method of this embodiment can be executed by the electronic device 100 of FIG1. ​​The details of each step in FIG4 are explained below with reference to the components shown in FIG1.

[0024] In step S420, the processor 150 displays a photo preview screen on the display 140. In step S430, the processor 150 receives a photo request from the user. Furthermore, the processor 150 can control the display 140 to display the photo preview screen based on multiple images continuously captured by the image sensor 110, facilitating the user's framing and decision-making regarding the photo content. When the user presses the shutter button, the processor 150 receives the user's photo request. The shutter button can be a physical button or a virtual button displayed on a touchscreen.

[0025] In step S440, the processor 150 decides whether to accept the user's photo-taking request based on the photo-taking interval. In this embodiment, step S440 can be implemented as steps S441 to S444.

[0026] In step S441, in response to receiving a user's photo-taking request, the processor 150 determines whether the image frame queue has available queue space. If step S441 determines no, it means the image frame queue is full and cannot store any more image data. Therefore, if step S441 determines no, proceed to step S444. In step S444, when the image frame queue does not have available queue space, the processor 150 does not accept the user's photo-taking request.

[0027] In step S442, when the image frame queue has available queue space, the processor 150 determines whether the previous photo-taking process has ended based on the photo-taking interval time.

[0028] If step S442 is incorrect, it means the previous photo-taking process has not yet ended, and proceed to step S444. In some embodiments, when a timer's timing from receiving the previous user photo-taking request to receiving the current user photo-taking request is less than the photo-taking interval, the processor 150 can determine that the previous photo-taking process has not yet ended. In step S444, when the previous photo-taking process has not yet ended, the processor 150 does not accept the user photo-taking request. If step S442 is correct, it means the time interval between two consecutive user photo-taking requests is long enough, and proceed to step S443. When a timer's timing from receiving the previous user photo-taking request to receiving the current user photo-taking request is greater than the photo-taking interval, the processor 150 can determine that the previous photo-taking process has ended. In step S443, when the previous photo-taking process has ended, the processor 150 decides to accept the user photo-taking request.

[0029] In step S450, when a user's photo-taking request is received, the processor 150 temporarily stores the multiple image frames captured by the image sensor 110 into an image frame queue. Specifically, when the image frame queue has available queue space and the timer has counted to the photo-taking interval, the Frame Capture Thread starts working, and the processor 150 controls the image sensor 110 to capture multiple image frames through the Frame Capture Thread.

[0030] In step S460, the processor 150 generates a composite image based on multiple image frames in the image frame queue and stores the composite image in accordance with the image storage format. When the processor 150 completes the generation and storage of a composite image, the image frame queue can release the multiple image frames associated with that composite image. In detail, after a set of images is captured and recorded in the image frame queue, the image enhancement post-processing execution thread retrieves the set of images from the image frame queue for image enhancement processing or image compositing processing.

[0031] Subsequently, in step S480, the processor 150 determines a shooting interval time based on a preset number of consecutive shots and the available storage capacity of an image frame array.

[0032] Please refer to Figure 5, which is a flowchart illustrating the determination of the photo interval according to an embodiment of the present disclosure. In some embodiments, step S480 may be implemented as steps S481 to S484.

[0033] In step S481, the processor 150 compares the preset number of consecutive shots with the available storage capacity of the image frame array. In step S482, the processor 150 determines whether the preset number of consecutive shots is greater than the available storage capacity of the image frame array.

[0034] It should be noted that the image frame queue can only temporarily store a maximum of multiple image frames used to generate Q composite images. In this embodiment of the invention, the available storage capacity of the image frame queue is defined as the maximum number of composite images that the image frame queue can support.

[0035] If step S482 is determined to be yes, proceed to step S483. In step S483, when the preset number of consecutive shots is greater than the available storage capacity of the image frame array, the processor 150 determines the shooting interval time according to a first calculation formula associated with the image post-processing time. If step S482 is determined to be no, proceed to step S484. In step S484, when the preset number of consecutive shots is less than or equal to the available storage capacity of the image frame array, the processor 150 determines the shooting interval time according to a second calculation formula not associated with the image post-processing time. Further, depending on whether the preset number of consecutive shots is greater than the available storage capacity of the image frame array, the processor 150 may use different calculation formulas to determine the shooting interval time. In some embodiments, the image post-processing time is the time required for the processor 150 to perform image post-processing on multiple image frames to generate a composite image.

[0036] In some embodiments, the first calculation formula is calculated based on an image post-processing time, while the second calculation formula is not calculated based on the image post-processing time. That is, when the preset number of consecutive shots is greater than the available storage capacity of the image frame array, the processor 150 needs to consider the image post-processing time, so the processor 150 can determine the shooting interval time based on the image post-processing time. When the preset number of consecutive shots is less than or equal to the available storage capacity of the image frame array, the processor 150 may not determine the shooting interval time based on the image post-processing time.

[0037] In some embodiments, when the preset number of consecutive shots is greater than the available storage capacity of the image frame array, the processor 150 can determine the shooting interval time based on the preset number of consecutive shots, the available storage capacity of the image frame array, the image capture time, and the image post-processing time. The first calculation formula can be represented by the following formula (1). , where N > Q Formula (1) Wherein, T represents the shooting interval time; N represents the preset number of consecutive shots; Q represents the available storage capacity of the image frame array; F represents an image capture time; P represents the image post-processing time. It should be noted that the image capture time can be the exposure time of a single image multiplied by the number of frames used to generate a single composite image.

[0038] For example, assuming the exposure time of a single image is 33ms and the number of frames used to generate a single composite image is 10, then the image capture time can be 330ms (i.e., 3310 = 330). In this case, assuming the preset number of shots is 6, the available storage capacity of the image frame array is 4, and the image post-processing time is 2500ms, then the shooting interval time T = (3304 + 2500(6-4)) / (6-1) = 1264ms.

[0039] In some embodiments, when the preset number of consecutive shots is less than or equal to the available storage capacity of the image frame array, the processor 150 can determine the shooting interval time based on the preset number of consecutive shots and the image capture time. The second calculation formula can be represented as the following formula (2). , where N ≤ Q Formula (1) Where, T represents the shooting interval time; N represents the preset number of consecutive shots; F represents an image capture time.

[0040] For example, assuming the exposure time of a single image is 33ms and the number of frames used to generate a single composite image is 10, then the image capture time can be 330ms (i.e., 3310 = 330). In this case, assuming the preset burst shot count is 3, the available storage capacity of the image frame array is 4, and the image post-processing time is 2500ms, then the shooting interval time T = (3303) / (3-1) = 495ms.

[0041] In step S470, when the user's request to take a photo is not accepted, the processor 150 provides a photo-disable notification through an output device. The output device may be, for example, a display 140, a lighting device, or a speaker. The photo-disable notification may be a visual notification or an audio notification. In some embodiments, the processor 150 may display a photo-disable notification prompting the user to wait on the camera application's interface via the display 140. The photo-disable notification may be text, an icon, or a combination thereof.

[0042] In summary, in the embodiments disclosed herein, the photo-taking interval can be determined based on the preset number of consecutive shots and the available storage capacity of the image frame array. When the preset number of consecutive shots exceeds the available storage capacity of the image frame array, the photo-taking interval can be determined based on the image post-processing time. Then, the decision to accept the user's photo-taking request can be made based on the photo-taking interval. This avoids the unpleasant experience of unpredictable long-term photography interruptions due to a full image frame array, thereby providing users with a better photography experience. In shooting scenarios with a high number of consecutive shots, the photo-taking results generated after image post-processing can be produced at a consistent and stable speed, thus avoiding the predicament of having to wait a long time before taking a photo.

[0043] Although the invention has been described above with reference to embodiments, it is not intended to limit the invention. Anyone skilled in the art can make some modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the invention shall be determined by the appended claims. [Simplified Explanation of the Diagram]

[0044] Figure 1 is a block diagram of an electronic device according to an embodiment of the present disclosure. Figure 2 is a flowchart of a photo-taking time control method according to an embodiment of the present disclosure. Figure 3A is a schematic diagram of not accepting a user's photo-taking request according to an embodiment of the present disclosure. Figure 3B is a schematic diagram of accepting a user's photo-taking request according to an embodiment of the present disclosure. Figure 4 is a flowchart of a photo-taking time control method according to an embodiment of the present disclosure. Figure 5 is a flowchart of determining the photo-taking interval time according to an embodiment of the present disclosure.

Claims

1. A method for controlling the time of taking a picture, used in an electronic device including an image sensor, the method comprising: A shooting interval is determined based on a preset number of consecutive shots and the available storage capacity of an image frame array; The process involves determining whether to accept a user's photo-taking request based on the photo-taking interval; when the user's photo-taking request is received, temporarily storing multiple image frames captured by the image sensor in the image frame queue; generating a composite image based on the multiple image frames in the image frame queue, and storing the composite image in accordance with the image storage format; wherein the step of determining the photo-taking interval based on the preset number of consecutive shots and the available storage capacity of the image frame queue includes: comparing the preset number of consecutive shots with the available storage capacity of the image frame queue; When the preset number of consecutive shots is greater than the available storage capacity of the image frame series, the shooting interval time is determined according to a first calculation formula; and when the preset number of consecutive shots is less than or equal to the available storage capacity of the image frame series, the shooting interval time is determined according to a second calculation formula.

2. The photo capture time control method as described in claim 1, wherein the first calculation formula is calculated based on an image post-processing time, and the second calculation formula is not calculated based on the image post-processing time.

3. The photo-taking time control method as described in claim 1, wherein when the preset number of consecutive shots is greater than the available storage capacity of the image frame array, the step of determining the photo-taking interval time according to the first calculation formula includes: The shooting interval is determined based on the preset number of consecutive shots, the available storage capacity of the image frame array, the image capture time, and the image post-processing time.

4. The photo capture time control method as described in claim 1, wherein the first calculation formula is characterized as follows: , where T represents the photo capture interval time; N represents the preset number of consecutive shots; Q represents the available storage capacity of the image frame array; F represents an image capture time; and P represents the image post-processing time.

5. The photo-taking time control method as described in claim 1, wherein the step of determining the photo-taking interval time according to the second calculation formula when the preset number of consecutive shots is less than or equal to the available storage capacity of the image frame series includes: The shooting interval is determined based on the preset number of consecutive shots and the image capture time.

6. The photo-taking time control method as described in claim 1, wherein the step of determining whether to accept the user's photo-taking request based on the photo-taking interval includes: In response to receiving the user's photo-taking request, determine whether the previous photo-taking process has ended based on the photo-taking interval time; If the previous photo-taking process has ended based on the photo-taking interval, the decision is made to accept the user's photo-taking request. And if it is determined from the photo-taking interval that the previous photo-taking process has not yet ended, it is decided not to accept the user's photo-taking request.

7. The photo-taking time control method as described in claim 6, wherein the timer is triggered to start timing upon allowing the previous user's photo-taking request.

8. The photo-taking time control method as described in claim 6, wherein the step of determining whether the previous photo-taking process has ended based on the photo-taking interval time in response to receiving the user's photo-taking request includes: In response to receiving the user's photo-taking request, determine whether the image frame queue has available queue space; When the image frame queue has available queue space, it is determined whether the previous photo-taking process has ended based on the photo-taking interval time; and when the image frame queue does not have the available queue space, it is decided not to accept the user's photo-taking request.

9. The photo-taking time control method as described in claim 1 further includes: When the user's request to take a photo is not accepted, a photo-disable notification is provided through an output device.

10. An electronic device comprising: An image sensor; The system also includes a processor coupled to the image sensor and configured to: determine a shooting interval based on a preset number of consecutive shots and the available storage capacity of an image frame queue; determine whether to accept a user's photo-taking request based on the shooting interval; when the user's photo-taking request is accepted, temporarily store multiple image frames captured by the image sensor in the image frame queue; and generate a composite image based on the multiple image frames in the image frame queue, and store the composite image in an image storage format; wherein the step of determining the shooting interval based on the preset number of consecutive shots and the available storage capacity of the image frame queue includes: comparing the preset number of consecutive shots with the available storage capacity of the image frame queue; When the preset number of consecutive shots is greater than the available storage capacity of the image frame series, the shooting interval time is determined according to a first calculation formula; and when the preset number of consecutive shots is less than or equal to the available storage capacity of the image frame series, the shooting interval time is determined according to a second calculation formula.

Images