Image shooting method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of electronic equipment, and provides an image shooting method, electronic equipment and a computer readable storage medium. The method is applied to the electronic equipment supporting a preset shooting mode, and comprises the following steps: when the electronic equipment carries out image shooting in the preset shooting mode, if a second click operation used for triggering to end shooting by a user is received, the electronic equipment synthesizes a preview image currently collected by a camera to obtain a third image and stores the third image. Namely, after the electronic equipment receives the second click operation, the electronic equipment gives up to wait for the preview images which have not yet been collected, and only the preview images currently collected by the camera are directly synthesized and stored, so that the image output efficiency can be improved, the waiting time of a user is shortened, and the shooting experience of the user is improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of electronic devices, and in particular, to an image capturing method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of the image capturing technology of electronic devices and the increasing demands of users for capturing, more and more image capturing modes can be provided by existing electronic devices. For example, portrait mode, panoramic mode, time-lapse photography, night mode, slow motion, high dynamic range imaging (HDR), and light painting shutter. Among them, the light painting shutter can also be called long exposure.

[0003] Currently, the light painting shutter mode can continuously capture images for a long time to capture the movement track of light. Therefore, when using the light painting shutter mode to capture images, the electronic device needs to respond to two click operations of the user on the capture shutter to complete the capture. The first click operation of the user triggers the electronic device to start capturing images, and the second click operation of the user triggers the electronic device to end the image capture. However, after the electronic device starts to capture images, it will continuously call the camera to collect images.

[0004] Therefore, when the user triggers the electronic device to end the image capture, there may still be some images that have not been completely collected by the camera. Currently, for these incompletely collected images, the electronic device often waits for the camera to complete the collection and then generates the final image using the images collected by the camera to end the current image capture. Thus, when capturing images using the light painting shutter mode, the image output speed of the electronic device is relatively slow, and the image output efficiency is low, causing the user to wait for a long time for the image output, resulting in a poor user experience. Summary of the Invention

[0005] Embodiments of the present application provide an image capturing method, an electronic device, and a computer-readable storage medium, which are used to solve the problems of slow image output speed and low image output efficiency in the light painting shutter mode, resulting in the user waiting for a long time for the image output and a poor user experience.

[0006] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, an image capturing method is provided. This method is applied to an electronic device which includes a camera and supports a preset shooting mode that can be used to capture star trails. The method includes: the electronic device first responds to a user's operation of enabling the preset shooting mode within a camera application and displays a shooting interface corresponding to the preset shooting mode; wherein, the shooting interface corresponding to the preset shooting mode includes a shooting shutter. Subsequently, whenever the user triggers image capture on this shooting interface, the electronic device considers it as capturing in the preset shooting mode. That is, the electronic device captures an image in the image capture manner corresponding to the preset shooting mode and performs image processing on the captured image according to the processing manner corresponding to the preset shooting mode.

[0008] In this preset shooting mode, the electronic device first receives a user's first click operation on the shooting shutter, displays a first image at a first moment, and displays a second image at a second moment. Among them, the first image is an image obtained from the first frame of preview image collected by the camera based on the first preview request generated in response to the first click operation. And the second image is an image synthesized from the first to k frames of preview images collected by the camera based on the first to k preview requests generated in response to the first click operation. Wherein, k > 1 and k is a positive integer. The second moment is before the first moment. It can be understood that the camera will collect one frame of corresponding preview image based on each generated preview request.

[0009] Then, at a third moment after the second moment, the electronic device receives a user's second click operation on the shooting shutter, and this second click operation is used to trigger the end of taking pictures. The electronic device then saves the third image and ends taking pictures.

[0010] In the first aspect, at the third moment, if the electronic device has called the camera to collect the i-th frame of preview image but has generated the (i + p)-th preview request in response to the first click operation, the electronic device only synthesizes the first to i frames of preview images to obtain the third image for saving. That is, in the first aspect, if the electronic device receives the second click operation, the electronic device directly abandons waiting for those preview images that have not been collected in time and directly synthesizes and saves only the preview images currently collected by the camera. That is to say, the electronic device abandons waiting for the out-framing of the above p frames of images and directly synthesizes the first to i frames of preview images to obtain the third image to be saved, thereby being able to avoid the problem of reduced image output speed caused by waiting for out-framing, thus improving the image output efficiency, reducing the user's waiting time, and enhancing the user's shooting experience.

[0011] In a possible implementation of the first aspect, when the electronic device receives the second click operation triggered by the user to end the photo shooting at the third moment, it is possible that all the collected preview images have been synthesized and displayed in time, or it is possible that the electronic device still has some preview images that have been collected but have not yet been synthesized and displayed. Then, the fourth image displayed at the third moment may be an image synthesized from all the collected preview images, or may be an image that does not include preview images that have not been displayed. That is, the above-mentioned image shooting method may also include:

[0012] At the third moment, the electronic device displays a fourth image; the fourth image is synthesized from the 1st to jth preview images captured by the camera based on the 1st to jth preview requests generated in response to the first click operation. Wherein, k<j≤i, j is a positive integer. Specifically, if j<i, it indicates that the fourth image displayed at the third moment is an image synthesized from only the 1st to jth preview images. Furthermore, the third image can be obtained by synthesizing the fourth image and the preview image frames after the jth frame. And if j=i, it indicates that the fourth image displayed at the third moment is an image synthesized from the 1st to ith preview images, and thus the third image can be the fourth image.

[0013] In another possible implementation of the first aspect, in terms of the processing capability of current electronic devices, there is a high probability that there is only one frame of preview images that cannot be sent for display in time. Therefore, at the third moment, the electronic device displays a fourth image; the fourth image is synthesized from the preview images of the first to j frames captured by the camera based on the preview requests of the first to j frames generated in response to the first click operation; wherein j=i or j=i-1.

[0014] In a possible implementation of the first aspect, in order to comply with the image storage format, saving the third image may include: performing format conversion on the third image, and saving the third image after the format conversion. In another possible implementation of the first aspect, the third image before the format conversion may be in a YUV format, and the third image after the format conversion may be in a JPEG format.

[0015] In a possible implementation of the first aspect, the electronic device may synthesize multiple preview images by accumulating them frame by frame. Based on this, synthesizing the preview images from the 1st to the kth frames captured by the camera to obtain the second image may include: synthesizing the fifth image and the kth preview image to obtain the second image. The fifth image is synthesized based on the preview images from the 1st to the k-1st frames.

[0016] In another possible implementation of the first aspect, synthesizing the third image based on the 1st to i-th preview images captured by the camera may include: synthesizing the sixth image and the i-th preview image to obtain the third image, wherein the sixth image is an image synthesized based on the 1st to i-1-th preview images.

[0017] In a possible implementation of the first aspect, to avoid abnormal image capture caused by other unnecessary problems due to no corresponding returned image for the preview request, the above image capture method may further include: generating corresponding virtual preview images for p preview requests after the i-th preview request; wherein, the virtual preview images include virtual image data, image metadata, and discard frame flags.

[0018] In another possible implementation of the first aspect, to avoid abnormal image capture caused by continuous exposure decision-making after the shooting ends, the above image capture method may further include: interrupting the exposure decision-making after obtaining the third image.

[0019] In a possible implementation of the first aspect, the electronic device includes an automatic exposure module, a frame output module, a fast frame return module, an image processing module, a selection module, and a display transmission module; the above image capture method may further include:

[0020] The camera application continuously sends preview requests to the frame output module in response to the first click operation; the frame output module periodically sends preview requests to the camera according to the exposure time of the camera to drive the camera to capture a frame of preview image; wherein, the preview images are stored in the first image queue in the order of collection; the exposure time is determined by the automatic exposure module according to the ambient brightness and sent to the frame output module. Then, the fast frame return module extracts the preview images frame by frame from the first image queue and transmits them to the image processing module. The image processing module synthesizes the first frame of preview image to obtain the first image, and synthesizes the preview images from the first to the k-th frame to obtain the second image; the image processing module transmits the first image and the second image to the camera application via the selection module and the display transmission module, and the camera application displays the first image and the second image.

[0021] In a possible implementation of the first aspect, the electronic device further includes an image generation module. The above image capturing method may further include: in response to a second click operation at a third moment, the camera application sends a photographing request to the fast frame return module via the frame output module; at the third moment, if the preview images after the j-th frame among the first to i-th frame preview images have not been transmitted to the image processing module, the fast frame return module extracts the preview images after the j-th frame and transmits them to the image processing module in response to the photographing request; k < j < i, and j is a positive integer; the image processing module synthesizes the preview images after the j-th frame with a fourth image to obtain a third image; wherein, the fourth image is synthesized by the image processing module based on the first to j-th frame preview images; the image processing module transmits the third image to the image generation module via the selection module, and the image generation module performs format conversion on the third image and then sends it to the camera application for saving. Thereby, by quickly responding to the photographing request by the fast frame return module and synthesizing and saving the third image based on the currently captured preview images, the image output efficiency is improved, the user waiting time is reduced, and the user shooting experience is enhanced.

[0022] In another possible implementation of the first aspect, at the third moment, if all the first to i-th frame preview images have been extracted and transmitted frame by frame to the image processing module, the fast frame return module sends a picture generation instruction to the selection module in response to the photographing request; the selection module transmits the third image to the image generation module, and the image generation module performs format conversion on the third image and then sends it to the camera application for saving. Thereby, the third image that can be saved can be obtained quickly, and the image output efficiency is improved.

[0023] In a possible implementation of the first aspect, the image capturing method may further include: the fast frame return module receives a flag bit sent by the camera application, and the flag bit is sent when the camera application receives the third image; the fast frame return module generates corresponding virtual preview images for p preview requests after the i-th preview request in response to the flag bit.

[0024] In a possible implementation of the first aspect, the image capturing method may further include: the fast frame return module sends an interrupt instruction to the automatic exposure module in response to the flag bit sent by the camera application, instructing the automatic exposure module to interrupt the exposure decision.

[0025] In a second aspect, the present application provides an electronic device, including: a camera, one or more processors and a memory, the camera and the memory are respectively coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device is caused to perform the following steps:

[0026] In response to a user's operation of enabling a preset shooting mode within a camera application, display a shooting interface corresponding to the preset shooting mode; wherein, the shooting interface corresponding to the preset shooting mode includes a shooting shutter; receive the user's first click operation on the shooting shutter; at a first moment, display a first image; wherein, the first image is obtained from the first frame of preview image captured by the camera based on the first preview request generated in response to the first click operation; the camera captures one frame of preview image based on each preview request; at a second moment, display a second image; wherein, the second image is synthesized from the first to k frames of preview images captured by the camera based on the first to k preview requests generated in response to the first click operation; the second moment is after the first moment; k > 1, and k is a positive integer; at a third moment, receive the user's second click operation on the shooting shutter and save a third image; wherein, the third moment is after the second moment; at the third moment, in response to the first click operation, the (i + p)th preview request has been generated, and the camera has captured the i-th frame of preview image; the third image is synthesized from the first to i frames of preview images captured by the camera; i > k, and p is a positive integer.

[0027] In a possible implementation manner of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: at the third moment, the electronic device displays a fourth image; the fourth image is synthesized from the first to j frames of preview images captured by the camera based on the first to j preview requests generated in response to the first click operation; k < j ≤ i, and j is a positive integer; if j < i, the third image is synthesized with the fourth image and the preview images after the j-th frame; if j = i, the fourth image is the third image. In another possible implementation manner of the second aspect, j = i or j = i - 1.

[0028] In a possible implementation manner of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: perform format conversion on the third image and save the third image after format conversion. In another possible implementation manner of the second aspect, the third image before format conversion is in YUV format, and the third image after format conversion is in JPEG format.

[0029] In a possible implementation manner of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: synthesize the second image by combining the fifth image and the k-th frame of preview image. Wherein, the fifth image is synthesized based on the first to k - 1 frames of preview images.

[0030] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: synthesizing a sixth image and an i-th frame preview image to obtain a third image, where the sixth image is an image synthesized based on the first to i-1 frame preview images.

[0031] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: generating corresponding virtual preview images for p preview requests after the i-th preview request; where the virtual preview images include virtual image data, image metadata, and a dropped frame flag.

[0032] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: after obtaining the third image, interrupting the exposure decision.

[0033] In a possible implementation of the second aspect, the electronic device includes an automatic exposure module, a frame output module, a fast frame return module, an image processing module, a selection module, and a display sending module. When the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: the camera application continuously sends preview requests to the frame output module in response to a first click operation; the frame output module periodically sends preview requests to the camera according to the exposure time of the camera to drive the camera to capture a frame of preview image; where the preview images are stored in a first image queue in the order of acquisition; the exposure time is determined by the automatic exposure module according to the ambient brightness and sent to the frame output module. Then, the fast frame return module extracts the preview images from the first image queue frame by frame and transmits them to the image processing module, and the image processing module synthesizes the first frame preview image to obtain a first image, and the image processing module synthesizes the first to k frame preview images to obtain a second image; the image processing module transmits the first image and the second image to the camera application via the selection module and the display sending module, and the camera application displays the first image and the second image.

[0034] In a possible implementation of the second aspect, the electronic device further includes an image generation module. When the above computer instructions are executed by the processor, the electronic device further performs the following steps: The camera application responds to the second click operation at the third moment, and sends a photographing request to the fast frame return module via the frame output module; at the third moment, if the preview images after the j-th frame among the first to i-th frame preview images have not been transmitted to the image processing module, the fast frame return module responds to the photographing request, extracts the preview images after the j-th frame and transmits them to the image processing module; k < j < i, and j is a positive integer; the image processing module synthesizes the preview images after the j-th frame with the fourth image to obtain the third image; wherein, the fourth image is synthesized by the image processing module based on the first to j-th frame preview images; the image processing module transmits the third image to the image generation module via the selection module, and the image generation module performs format conversion on the third image and then sends it to the camera application for saving.

[0035] In a possible implementation of the second aspect, when the above computer instructions are executed by the processor, the electronic device further performs the following steps: at the third moment, if the first to i-th frame preview images have all been extracted and transmitted frame by frame to the image processing module, the fast frame return module responds to the photographing request and sends a picture generation instruction to the selection module; the selection module transmits the third image to the image generation module, and the image generation module performs format conversion on the third image and then sends it to the camera application for saving. Thus, the third image that can be saved can be obtained quickly, improving the image output efficiency.

[0036] In a possible implementation of the second aspect, when the above computer instructions are executed by the processor, the electronic device further performs the following steps: The fast frame return module receives the flag bit sent by the camera application, and the flag bit is sent when the camera application receives the third image; the fast frame return module responds to the flag bit and generates corresponding virtual preview images for p preview requests after the i-th preview request respectively.

[0037] In a possible implementation of the second aspect, when the above computer instructions are executed by the processor, the electronic device further performs the following steps: The fast frame return module responds to the flag bit sent by the camera application and sends an interrupt instruction to the automatic exposure module, instructing the automatic exposure module to interrupt the exposure decision.

[0038] In a third aspect, a computer-readable storage medium of the present application stores a computer program, and when the computer program is executed by a processor in an electronic device, the electronic device is caused to execute the image capture method as described in the first aspect and any of its possible implementations.

[0039] Fourthly, the present application provides a computer program product which, when running on a computer, causes the computer to execute the method as described in the first aspect and any possible implementation thereof. The computer may be the above-mentioned electronic device.

[0040] It can be understood that for the beneficial effects that can be achieved by the electronic device in any possible implementation of the second aspect, the computer-readable storage medium in the third aspect, and the computer program product in the fourth aspect, reference may be made to the beneficial effects in the first aspect and any possible implementation thereof, which will not be elaborated here. Description of the Drawings

[0041] Figure 1 FIG. is a schematic diagram of a scenario where a user triggers the opening of the light painting shutter provided by an embodiment of the present application;

[0042] Figure 2 FIG. is a schematic diagram of a star trail image provided by an embodiment of the present application;

[0043] Figure 3 FIG. is a schematic diagram of a scenario where a user triggers the opening of the gorgeous star trail sub-mode provided by an embodiment of the present application;

[0044] Figure 4 FIG. is a schematic diagram of an interface waiting for an image to be taken in the light painting shutter mode provided by an embodiment of the present application;

[0045] Figure 5 FIG. is a flowchart of an image shooting method provided by an embodiment of the present application;

[0046] Figure 6 FIG. is a structural block diagram of the software and hardware architecture of an electronic device provided by an embodiment of the present application;

[0047] Figure 7 FIG. is the flow of another image shooting method provided by an embodiment of the present application Figure 1 ;

[0048] Figure 8 FIG. is the flow of another image shooting method provided by an embodiment of the present application Figure 2 ;

[0049] Figure 9 FIG. is a flowchart of a traditional image shooting method provided by an embodiment of the present application;

[0050] Figure 10 FIG. is the flow of another image shooting method provided by an embodiment of the present application Figure 3 ;

[0051] Figure 11 FIG. is the flow of another image shooting method provided by an embodiment of the present application Figure 4 ;

[0052] Figure 12 Schematic diagram of a structure of an electronic device 1200 provided by an embodiment of the present application;

[0053] Figure 13 Schematic diagram of a structure of a chip system provided by an embodiment of the present application. Detailed implementation manners

[0054] Next, the technical solutions of the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to limit the present application. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, if the same items or similar items with basically the same functions and effects are distinguished by using terms such as "first" and "second". Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily limit to be different. Also, in the description of the embodiments of the present application, unless otherwise stated, the meaning of "a plurality" means two or more.

[0055] With the development of the image shooting technology of electronic devices and the increasing user demands for shooting, the existing electronic devices can provide more and more image shooting modes to users. For example, portrait mode, panoramic mode, time-lapse photography, night mode, slow motion, high dynamic range imaging (HDR), and light painting shutter.

[0056] Among them, the light painting shutter can also be called long exposure, which is a shooting mode used to capture the movement trajectory of light and automatically extend the shutter time. Simply put, using the light painting shutter mode to shoot an image can capture the movement trajectory of light. From the shooting effect, the light shot in the light painting shutter mode will be in a line shape.

[0057] As Figure 1 shown, taking the mobile phone 10 as an example, an embodiment of the present application shows a schematic diagram of a scenario where a user triggers the opening of the light painting shutter. Hereinafter, the operation process of the user triggering the opening of the light painting shutter will be described in conjunction with Figure 1 the following.

[0058] Referring to Figure 1 , when the mobile phone 10 is in the powered-on state, the main interface 100 as shown in Figure 1 can be displayed. The main interface 100 may include application icons of application programs such as "Clock", "Calendar", "Camera", "Gallery", "Memo", "File Management", "Email", "Music", "Settings", "Contacts", "Phone", and "Messages".

[0059] Understandably, according to actual requirements, the home screen 100 of the mobile phone 10 may further include application icons of more applications, such as various third-party applications downloaded by the user. The embodiments of the present application do not make any limitations in this regard.

[0060] In response to the user's click operation on the "Camera" application icon 101 in the home screen 100, the mobile phone 10 enters the camera application and displays the shooting interface 102.

[0061] The shooting interface 102 includes multiple shooting options. As shown in the shooting interface 102, there are shooting options such as "AI", "HDR", "Settings", "Aperture", "Night Scene", "Portrait", "Photo", "Video", "Professional", and "More".

[0062] In response to the user's click operation on the "More" shooting option 103 in the shooting interface 102, the mobile phone 10 enters the more shooting options interface 104.

[0063] The more shooting options interface 104 includes other shooting options provided by the camera. As shown in the more shooting options interface 104, there are "Micro Film", "Time-lapse Photography", "Panorama", "Document Scan", "Watermark", and "Light Painting Shutter". Similarly, according to actual requirements, the more shooting options interface 104 may further include more shooting options, such as "High Pixel", "Multi-lens Video", "Super Macro", etc. The embodiments of the present application do not make any limitations in this regard.

[0064] In response to the user's click operation on the "Light Painting Shutter" shooting option 105 in the more shooting options interface 104, the mobile phone 10 enters the light painting shutter shooting interface 106. In the light painting shutter shooting interface 106, in response to the user's click operation on the shooting shutter, the mobile phone 10 can be triggered to start image shooting.

[0065] It should be noted that the embodiments of the present application Figure 1 The shown scenarios do not constitute a scene limitation for the user to trigger the light painting shutter. The user can also trigger the light painting shutter through more ways. For example, the user can also turn on the light painting shutter in the camera settings provided by "Settings". The embodiments of the present application do not make any limitations in this regard.

[0066] In addition, for different shooting scene requirements, there may currently be multiple sub-modes provided in the light painting shutter mode. For example, there are four sub-modes: flowing cars and horses, light painting graffiti, silk-like flowing water, and gorgeous star trails. The user can select the corresponding sub-mode for image shooting according to the actual shooting scene requirements.

[0067] Among them, heavy traffic can be used to shoot vehicle flows. Light painting graffiti can be used to shoot large light painting works. Silky water can be used to shoot all dynamic water elements, including flowing water, ocean waves, waterfalls, etc. And gorgeous star trails can be used to shoot the starry sky. Through the gorgeous star trails, the movement trajectories of the stars, that is, star trails, can be clearly captured.

[0068] Exemplarily, as Figure 2 shown, an embodiment of the present application shows a star trail image. Figure 2 The shown star trail image is an image taken using the gorgeous star trails sub-mode and processed in black and white later. The white lines therein are the movement trajectories of the stars.

[0069] As Figure 3 shown, taking the mobile phone 10 shown in Figure 1 as an example, an embodiment of the present application shows a schematic diagram of a scenario where a user triggers to turn on the gorgeous star trails sub-mode. Hereinafter, the operation process of the user triggering to turn on the gorgeous star trails will be described in combination with Figure 3 this.

[0070] After the user triggers to turn on the light painting shutter module, the mobile phone 10 displays a light painting shutter shooting interface 106. The light painting shutter shooting interface 106 includes a sub-mode selection control 107.

[0071] The mobile phone 10 responds to the user's click operation on the sub-mode selection control 107 and enters a sub-mode selection interface 108.

[0072] The sub-mode selection interface 108 includes various sub-modes provided by the light painting shutter. When a certain sub-mode is selected, the sub-mode selection interface 108 correspondingly displays the name of the sub-mode. Figure 3 The currently default selected sub-mode in the shown sub-mode selection interface 108 is heavy traffic.

[0073] The mobile phone 10 responds to the user's left swipe operation 109 to switch sub-modes. For example, after the user selects the gorgeous star trails mode by left swiping, the user can stop swiping to complete the sub-mode switch and display a gorgeous star trails selected interface 110.

[0074] The mobile phone 10 responds to the user's click operation on the blank area of the gorgeous star trails selected interface 110 and redisplay the light painting shutter shooting interface 106. On this light painting shutter shooting interface 106, the mobile phone 10 responds to the user's first click operation 111 on the shooting shutter and enters a shooting interface 112 to start image shooting.

[0075] It can be understood that since the mobile phone 10 has switched from the traffic scene to the gorgeous star trail in response to the user's operation before redisplaying the light painting shutter shooting interface 106. Therefore, when the first click operation 111 of the user is triggered under the light painting shutter shooting interface 106 to perform image shooting, the mobile phone 10 will process the image according to the image processing method corresponding to the gorgeous star trail.

[0076] In addition, the shooting time "00:00" and shooting precautions "Please keep the camera stable" are displayed in the shooting interface 112. The shooting time "00:00" will increase sequentially by seconds as the shooting time elapses, such as "00:01", "00:02", "00:03"... The elapsed shooting time can be determined through the shooting time. At the same time, the shooting shutter presents a shooting state in the shooting interface 112, and the currently captured image will be displayed in real time in the viewfinder 114 for the user to preview.

[0077] When the user observes a satisfactory star trail effect in the viewfinder 114, the user can perform a second click operation 113 on the shooting shutter presenting a shooting state in the shooting interface 112. The mobile phone 10 responds to the user's second click operation 113 and ends the image shooting.

[0078] It should be noted that the embodiments of the present application Figure 3 The shown scenario does not constitute a scenario limitation for the user to trigger the opening of the gorgeous star trail. The user can also trigger the opening of the gorgeous star trail through more ways. For example, the user can also directly click on the gorgeous star trail in the sub-mode selection interface 108 to switch the sub-mode of the light painting shutter from the traffic scene to the gorgeous star trail. The embodiments of the present application do not make any limitations in this regard.

[0079] According to Figure 3 The shown scenario, when the electronic device performs image shooting in the light painting shutter mode, it needs to respond to two click operations of the user on the shooting shutter to complete the shooting.

[0080] Among them, the first click operation of the user on the shooting shutter that the electronic device responds to, such as Figure 3 The first click operation 111 shown, is used to trigger the electronic device to start image shooting (that is, the first click operation is used to trigger the electronic device to start taking pictures). The electronic device processes the collected image data in real time and returns it to the camera application for display (that is, for preview), so as to allow the user to preview the image shooting effect.

[0081] The second click operation of the user on the shooting shutter that the electronic device responds to, such as Figure 3The second click operation 113 shown is used to trigger the electronic device to end image capture (i.e., the second click operation is used to trigger the electronic device to end taking pictures), and the electronic device directly stores the processed image in the gallery after image processing.

[0082] Among them, the timing when the user triggers the electronic device to end taking pictures can be any moment, and the embodiments of the present application do not make any restrictions on this. Generally speaking, the user can observe the image effect of the sent display image in real time. If the image effect reaches the expected shooting effect, the user can click the shooting shutter a second time to trigger the electronic device to end taking pictures.

[0083] Simply put, in the light painting shutter mode, the first click operation of the user on the shooting shutter is regarded by the electronic device as a preview request in the light painting shutter mode. And the second click operation of the user on the shooting shutter is regarded by the electronic device as a taking picture request. The difference between the preview request and the taking picture request is that the image corresponding to the preview request will be processed and then sent back to the camera application for display, that is, sent for display. While the image corresponding to the taking picture request will not be sent for display after being synthesized and processed, but will be directly saved. For example, the image corresponding to the taking picture request is directly stored in the gallery after being synthesized and processed.

[0084] That is to say, when the light painting shutter mode is in the on state, if the camera application in the electronic device receives the first click operation of the user on the shooting shutter, the camera application will continuously send preview requests to the lower layer. Furthermore, the lower layer will respond to these preview requests sent by the camera application to capture the corresponding images (which can be called preview images). Then, the lower layer will return the captured preview images to the upper layer camera application for display after corresponding image processing for the user to preview.

[0085] If the camera application in the electronic device receives the second click operation of the user on the shooting shutter, then the camera application will not continuously send preview requests to the lower layer, but will respond to this second click operation and send a corresponding taking picture request to the lower layer. Furthermore, after receiving this taking picture request, the lower layer will also respond to this taking picture request to capture the corresponding image (which can be called a taking picture image). However, the difference is that after the lower layer returns the captured taking picture image to the upper layer after image processing, the upper layer will choose to directly save it, and the upper layer camera application will not display this taking picture image. And the taking picture image saved by the electronic device is the image obtained by taking pictures in the light painting shutter mode.

[0086] It should be noted that the preview in the light painting shutter mode is different from the normal preview (i.e., the preview image collected and displayed in the viewfinder after the camera is turned on). The requirements for image processing in the normal preview are usually relatively low. However, the preview in the light painting shutter mode is triggered by the user clicking the shooting shutter, so the requirements for image processing depend on the requirements of the light painting shutter mode for the image. It can be understood that the requirements for image processing in the light painting shutter mode are much higher than those in the normal preview. And generally speaking, the image processing process corresponding to the preview request in the light painting shutter mode is usually the same as that corresponding to the shooting request in the light painting shutter mode, and the only difference is whether to display the image.

[0087] Thus, it can be simply understood that the light painting shutter obtains the movement trajectory of the object in the corresponding shooting scene through long-time continuous shooting. For example, shooting the starry sky for a long time to obtain star trails. Therefore, when the electronic device takes pictures in the light painting shutter mode, as the shooting time extends, the number of images collected by the electronic device will be more and more. However, after the shooting is completed, the electronic device will only generate one image and save it in the photo gallery. So, when taking pictures in the light painting shutter mode, the principle of the electronic device for image processing is: to perform image synthesis on the collected images. That is, before the electronic device receives the second click operation of the user on the shooting shutter, the electronic device will continuously perform image synthesis on the collected images and display the synthesized image obtained. When the electronic device receives the second click operation of the user on the shooting shutter, the electronic device will directly save the synthesized image after image synthesis without displaying it.

[0088] Specifically, after the electronic device responds to the user's opening operation to turn on the light painting shutter mode, if it receives the first click operation of the user on the shooting shutter, the electronic device will start shooting in response to this first click operation. During the shooting process, every time the electronic device collects a frame of image (outputs a frame), it will perform image synthesis on the current frame of image and the previously collected images, and fuse the image information of the previous images through pairwise synthesis of the images. At the same time, before receiving the second click operation of the user on the shooting shutter, the electronic device will display the synthesized image obtained by image synthesis for the user to preview the effect of image shooting in real time. If it receives the second click operation of the user on the shooting shutter, the electronic device will respond to this second click operation, save the latest synthesized image without displaying it, and end this image shooting.

[0089] For example, after the electronic device starts taking pictures, it acquires the first frame of image. Since it is the first frame without any previous image, the electronic device processes the first frame of image and displays it separately. When it comes to the acquired second frame of image, the electronic device synthesizes the second frame of image with the first frame of image to obtain a new second frame of image. This new second frame of image incorporates the image information of the first frame of image. Then, the electronic device displays the new second frame of image.

[0090] Similarly, the third frame image needs to be synthesized with the new second frame image before being displayed. The fourth frame image, the fifth frame image, and the nth frame image are synthesized in the same manner.

[0091] However, when the nth frame is reached, that is, the last frame captured, the nth frame is the frame corresponding to the user triggering the end of the photo shooting (i.e., the user clicks the shutter button for the second time to issue a photo shooting request). It will also be synthesized with the synthesized image of the previous frame, but will not be displayed, but directly stored in the gallery. That is, the first frame, the second frame, and the n-1th frame will be displayed, but the nth frame will not be displayed.

[0092] As a result, the third frame image sent for display will fuse the image information of the first frame image and the second frame image. The fourth frame image sent for display will fuse the image information of the first frame image, the second frame image and the third frame image. The same is true for subsequent images and will not be repeated here. The frame image stored in the gallery is a fusion of the image information of all the previous frames, that is, the image information of the first frame image, the second frame image... and the n-1th frame image. That is, in the streamer shutter mode, the electronic device continuously shoots for a long time and synthesizes all images to obtain an image with a motion trajectory.

[0093] However, the automatic exposure (AE) module in current electronic devices will determine the exposure time based on the brightness of the surrounding environment. As the surrounding environment gets darker, the exposure time determined by the AE module will be longer. When the surrounding environment is particularly dark, the exposure time determined by the AE module can be as long as 10 seconds. In other words, when the surrounding environment is particularly dark, the electronic device may take up to 10 seconds to capture a frame of image (output frame).

[0094] Then, for the light painting shutter mode, especially for scenes like the magnificent star trails that require long - time shooting and are usually carried out in a dark environment, when the electronic device receives the user's second click operation on the shooting shutter to trigger the end of taking pictures, it is very likely that due to the too - long frame output time, there is still a part of the image being produced (hereinafter referred to as the frame being produced). Because the underlying layer needs to respond to each request sent by the upper - layer camera application and feedback the corresponding image. So when the electronic device receives the instruction from the user to trigger the end of taking pictures, the underlying layer may be responding to a preview request to collect an image. At the same time, there may still be some preview requests waiting for response processing at the underlying layer.

[0095] That is to say, when the electronic device receives the instruction from the user to trigger the end of taking pictures (the user's second click operation on the shooting shutter), the preview requests sent by the camera application before may still be being processed. There may also be some preview requests that have been sent by the camera application but have not had time to be responded to and processed, which will then cause there to be some frames being produced.

[0096] Therefore, the frame being produced may be an image that has been partially collected but not completely, or it may be an image that has not been collected at all.

[0097] According to the traditional processing method, after the electronic device receives the user's second click operation on the shooting shutter, in order to ensure that all the preview requests sent by the camera application are responded to and there are corresponding images, the electronic device will wait for these frames being produced to complete frame output, and then respond to the taking - picture request to generate the last frame of the taken - picture composite image (i.e., the composite image corresponding to the taking - picture request) and store it in the photo library.

[0098] From the user's perspective, the shooting shutter on the shooting interface displayed by the electronic device will always be in a spinning state, indicating that the electronic device is generating an image (outputting an image). Exemplarily, as Figure 4 shown, an embodiment of the present application shows a schematic diagram of an interface waiting for image output. Figure 4 This is an interface diagram shown by taking the user's second click operation 113 in the shooting interface 112 as an example. That is, the mobile phone 10 enters the waiting - for - image - output interface 113 in response to the user's second click operation 113 on the shooting shutter. Figure 3 In the waiting - for - image - output interface 113, the outer - circle of the shooting shutter is in a spinning state. It should be noted that

[0099] the arrow shown is a schematic diagram of the spinning direction of the outer - circle of the shooting shutter in an embodiment of the present application, which does not constitute a limitation on the waiting - for - image - output interface 113. That is to say, the arrow may not be present on the actual waiting - for - image - output interface 113 displayed by the electronic device. Figure 4 the arrow shown is a schematic diagram of the spinning direction of the outer - circle of the shooting shutter in an embodiment of the present application, which does not constitute a limitation on the waiting - for - image - output interface 113. That is to say, the arrow may not be present on the actual waiting - for - image - output interface 113 displayed by the electronic device.

[0100] And, Figure 4 The clockwise rotation shown by the arrow in Figure 4 is also an example of the embodiment of the present application. According to actual requirements, it can also be a counterclockwise rotation. The embodiment of the present application does not make any limitation on this.

[0101] Exemplarily, if the exposure time is 10 seconds and there are 5 frames being currently output, then the user needs to wait approximately 5 * 10 seconds = 50 seconds. That is to say, from the user's perspective, the user will watch the circular outer ring of the shooting shutter rotate for 50 seconds.

[0102] It can be understood that this 50 seconds is only used as an example in the embodiment of the present application. The specific waiting duration depends on the actually determined exposure time and the number of frames being output. The embodiment of the present application does not make any limitation on this.

[0103] Therefore, traditionally in the light painting shutter mode, after the electronic device receives the user's second click operation on the shooting shutter, because it needs to wait for the frames to be output, the time for the electronic device to output an image is relatively long, and the image output efficiency is low. At the same time, the user needs to wait for a long time to obtain the captured image, resulting in the user being in a long waiting state all the time, thus leading to a poor user experience.

[0104] Based on this, in order to reduce the image output time in the light painting shutter mode, improve the image output efficiency, and reduce the user's waiting duration to enhance the user experience, the embodiment of the present application provides an image shooting method.

[0105] The image shooting method provided by the embodiment of the present application is applied to an electronic device. At the same time, this electronic device can support a preset shooting mode, and the preset shooting mode includes a star trail shooting mode. That is, the preset shooting mode can be used to shoot star trails.

[0106] In some embodiments, the preset shooting mode can be the light painting shutter mode or the long exposure mode. It can be understood that based on different custom names of different manufacturers, the preset shooting mode can be the above light painting shutter mode or the long exposure mode. Of course, according to the actual naming of the manufacturer, the preset shooting mode can also be other names. The embodiment of the present application does not make any limitation on this.

[0107] For the convenience of understanding and description of the solution, the following embodiments will uniformly refer to the preset shooting mode as the light painting shutter mode.

[0108] In the embodiment of the present application, when the light painting shutter mode is in the on state, if the electronic device receives a command from the user to end the photo taking, that is, after the electronic device receives the user's second click operation on the shooting shutter, the electronic device responds to this second click operation and directly uses the currently generated frames to generate an image and stores it in the photo library.

[0109] That is to say, the electronic device in the embodiments of the present application no longer waits for the frames being output to complete the frame output, but directly abandons the frames being output. Thus, the electronic device can avoid the long image output time caused by waiting for the frame output to complete, thereby reducing the image output time, improving the image output efficiency, reducing the user waiting time, and enhancing the user experience.

[0110] It can be understood that although abandoning the frames being output will cause the finally generated image to lack the image information of these preview images, the frames being output are usually the image information of the last few preview images. Therefore, even if the image information of the last few frames is missing, the impact on the final image effect is not significant.

[0111] In some embodiments, the preset shooting mode in the embodiments of the present application can also be used for shooting scenes such as heavy traffic, silk-like flowing water, and light painting graffiti. However, it should be noted that the shooting duration of heavy traffic and light painting graffiti is usually relatively short, so the number of images collected during the entire shooting process will not be too large. Then, the original image output speed will not be very slow.

[0112] At the same time, silk-like flowing water is usually shot during the day, so the light in the surrounding environment is not too dark, and the corresponding exposure time will not be too long. Therefore, if the image shooting method provided in the embodiments of the present application is applied to scenes such as heavy traffic, silk-like flowing water, and light painting graffiti, although the image output efficiency can also be improved by abandoning the frames being output, the effect is not as obvious as that of star trail shooting.

[0113] Therefore, it can be configured according to actual business requirements, that is, the image shooting method provided in the embodiments of the present application can be configured to be only applied to gorgeous star trails for star trail shooting. It can also be configured to be simultaneously applied to any one or more sub-modes among heavy traffic, light painting graffiti, silk-like flowing water, and gorgeous star trails.

[0114] It can be understood that no matter which sub-mode it is applied to, the implementation principle is the same. The electronic device can be configured according to the actual situation, and the embodiments of the present application do not make any limitations in this regard.

[0115] As Figure 5 shown, the embodiments of the present application show a flowchart of an image shooting method, including steps S501 - S503. Hereinafter, the image shooting method provided in the embodiments of the present application will be described in detail with reference to Figure 5 the shown flowchart.

[0116] S501, the electronic device responds to the user's operation of enabling the preset shooting mode in the camera application, and displays the shooting interface corresponding to the preset shooting mode. Among them, the shooting interface corresponding to the preset shooting mode includes a shooting shutter.

[0117] The process of the electronic device in the embodiment of the present application to turn on the preset shooting mode in response to the user's turn-on operation can refer to the above description of the scenarios related to Figure 1 and Figure 3 The shooting interface corresponding to the preset shooting mode can be the Figure 3 shown light painting shutter shooting interface 106.

[0118] S502, the electronic device receives the user's first click operation on the shooting shutter, displays the first image at the first moment, and displays the second image at the second moment.

[0119] After the electronic device receives the user's first click operation on the shooting shutter (such as the first click operation 111 shown in Figure 3 ), the electronic device calls the camera to start collecting images. It can be understood that since the first click operation in the light painting shutter mode corresponds to a preview request, the images collected by the electronic device in response to this first click operation can be regarded as preview images in the light painting shutter mode.

[0120] Moreover, in the light painting shutter mode, before the electronic device receives the user's second click operation on the shooting shutter (such as the second click operation 113 shown in Figure 3 ), because the electronic device needs to continuously collect preview images, the electronic device will continuously generate multiple preview requests in response to the first click operation.

[0121] At the same time, the electronic device will call the camera to collect 1 frame of corresponding preview image in response to each preview request. That is to say, the number of preview images is the same as the number of preview requests. That is, the first preview request corresponds to the first frame of preview image, the second preview request corresponds to the second frame of preview image, the third preview request corresponds to the third frame of preview image...

[0122] For the first frame of preview image collected by the camera, since there is no preview image before the first frame of preview image, the first frame of preview image can be directly displayed at the first moment after image processing. After the first frame of preview image, for each frame of preview image collected by the camera, the electronic device will perform image synthesis with the previous preview image and then display it.

[0123] That is, the electronic device can display the second image obtained by image synthesis at the second moment. This second image can be an image obtained by performing image synthesis on the first to k frames of preview images. k is greater than 1 and k is a positive integer.

[0124] It can be understood that the specific value of k depends on the number of preview images actually captured by the electronic device at the second moment. For example, at the second moment, if the electronic device actually responds to 3 preview requests and captures 3 frames of preview images, then k = 3. Another example is that at the second moment, if the electronic device actually responds to 10 preview requests and captures 10 frames of preview images, then k = 10.

[0125] It can be understood that since the electronic device captures preview images in response to each preview request in chronological order and performs image synthesis to display the images, the first image must be displayed before the second image. Therefore, the second moment in the embodiments of the present application must be after the first moment.

[0126] Specifically, if the electronic device captures the second frame of preview image and at this time k = 2, then the electronic device performs image synthesis on the first frame of preview image and the second frame of preview image to obtain the second image 1, and displays this second image 1 at the second moment 1.

[0127] If the electronic device captures the third frame of preview image and at this time k = 3, then the electronic device can perform image synthesis on the first frame of preview image, the second frame of preview image, and the third frame of preview image to obtain the second image 2, and can display this second image 2 at the second moment 2.

[0128] Similarly, if the electronic device captures the kth frame of preview image, then the electronic device can perform image synthesis on these k frames of preview images to obtain the second image k - 1. That is, the electronic device synthesizes the first to kth frames of preview images to obtain the second image k - 1. At the same time, the electronic device can display this second image k - 1 at the second moment k - 1. It can be understood that the second images include the second image 1, the second image 2... the second image k - 1.

[0129] It should be noted that since the exposure time affects the acquisition speed of preview images, and the longer the exposure time, the longer the acquisition time for each frame of preview image. Therefore, at the second moment, the number of preview requests actually generated by the electronic device may be greater than the number of preview requests actually responded to by the electronic device. For example, at the second moment, the electronic device actually generates 14 preview requests, but the electronic device can only respond to 10 preview requests and capture 10 frames of preview images.

[0130] S503. At the third moment, the electronic device receives the user's second click operation on the shooting shutter and saves the third image.

[0131] Among them, the third moment is after the second moment, and the third moment can depend on the user. Exemplarily, the third moment can be the moment when the user observes a satisfactory image capture effect in the second image displayed on the electronic device. For example, by observing the second image displayed on the electronic device, the user determines that the electronic device has currently captured a satisfactory star trail effect. Then the user can perform a second click operation on the capture shutter at this moment (i.e., the third moment) to send an end photo-taking instruction to the electronic device.

[0132] Similarly, since the exposure time affects the acquisition speed of the preview image, the actual number of generated preview requests will be greater than the actual number of responded preview requests. Therefore, at the third moment, the electronic device may have generated the (i + p)-th preview request in response to the first click operation, but the electronic device may actually have only processed the i-th preview request.

[0133] That is, at the third moment, the electronic device may have generated a total of (i + p) preview requests, but the electronic device has only responded and processed the i-th preview request. Correspondingly, at the third moment, the electronic device should only call the camera to capture the i-th frame of the preview image, that is, the electronic device has currently captured a total of i frames of preview images. At this time, the electronic device still has p preview requests not responded, that is, the electronic device still has p frames of preview images not called the camera to capture.

[0134] However, after the electronic device receives the second click for photo-taking from the user at the third moment, it indicates that the electronic device has received the instruction triggered by the user to end photo-taking. According to the user's requirements, the electronic device needs to respond to this second click operation to end photo-taking at this time. According to the traditional processing method, the electronic device will continue to respond to these p preview requests. After waiting for the electronic device to call the camera to capture the p frames of preview images corresponding to these p preview requests, the electronic device then synthesizes them with the previously captured i frames of preview images to obtain the third image for saving.

[0135] Thus, the duration for the traditional electronic device to respond to these p preview requests and capture p frames of preview images is the additional duration that the image needs to wait, that is, the duration that the user needs to wait. If the exposure time is longer, then this waiting duration will be longer, thereby reducing the image output efficiency, resulting in the user waiting for a long time, and the user's shooting experience is poor.

[0136] In the embodiment of the present application, if the electronic device receives the second click operation at the third moment, since only i frames of images are captured at the third moment, the electronic device abandons the p frames of images corresponding to those p preview requests and directly synthesizes the existing i frames of preview images to obtain the third image for saving.

[0137] Therefore, the third image in the traditional processing method is an image synthesized from the first to the (i + p)-th preview images. However, the third image in the embodiment of the present application is an image synthesized from the first to the i-th preview images. Thus, it can be seen that although the third image in the embodiment of the present application lacks the image information of p preview images, the embodiment of the present application does not need to wait for an additional p preview images to be output, thereby being able to accelerate the image output speed and reduce the waiting duration. At the same time, for long-term shooting, the lack of the image information of the last p images has little impact on the image effect of the final third image. Therefore, compared with the traditional method, the embodiment of the present application can accelerate the image output speed, improve the image output efficiency, reduce the user's waiting duration, and enhance the user's shooting experience while ensuring the image effect.

[0138] It can be understood that the third image saved in the embodiment of the present application is the image generated corresponding to the user's triggering to end the photo-taking. That is, since the second click operation in the light painting shutter mode corresponds to a photo-taking request, when the electronic device receives this second click operation, it is equivalent to receiving the photo-taking request issued by the user. Then, the saved third image is the image corresponding to the photo-taking request.

[0139] In some embodiments, when receiving the instruction from the user to trigger the end of photo-taking at the third moment, due to differences in the processing speeds of different threads or abnormal situations, it may be that all the preview images collected by the electronic device have been timely synthesized and sent for display, or it may be that the electronic device still has some preview images that have been collected but not yet synthesized and sent for display. For example, when the electronic device receives the second click operation at the third moment, the camera may have just collected one or more frames of preview images at that moment, and then these one or more frames of preview images are the preview images that have not yet been synthesized and sent for display. Or, due to abnormal situations, one or more frames of preview images that have been collected and could be displayed at the third moment fail to be successfully displayed at the third moment.

[0140] Then, at the third moment, the image displayed by the electronic device (hereinafter referred to as the fourth image) may be an image synthesized from the first to the j-th preview images. Among them, k < j ≤ i, and j is a positive integer.

[0141] For the case where j < i, it means that although the electronic device has collected the i-th preview image at the third moment, due to speed differences, the preview images from the (j + 1)-th to the i-th may all be the preview images just collected at the third moment. Therefore, in this case, the fourth image displayed at the third moment is an image synthesized only from the first to the j-th preview images.

[0142] In this case, it is understandable that since these preview image frames after the j-th frame are already captured frames, there is no need to spend extra time waiting for the electronic device to perform image synthesis. Therefore, in order to ensure the image output efficiency while retaining as much image information of the preview images as possible, the electronic device can obtain the third image by synthesizing the fourth image and the preview images after the j-th frame. Then, save this third image.

[0143] For example, at the third moment, the electronic device captures the 5(i)-th frame of the preview image. However, since the 4th and 5th frames of the preview images among these 5(i) frames are the preview images just captured at the third moment. Therefore, the fourth image displayed by the electronic device at the third moment should be the image synthesized only from the 1st to 3(j)-th frames of the preview images. Furthermore, after the electronic device receives the second click operation, since there is no need to wait for the frame output for these two preview images, the electronic device can synthesize this fourth image with the 4th and 5th frames of the preview images to obtain the third image.

[0144] For the case of j = i, it means that the difference between the capture and display speeds is not significant, and each captured preview image is synthesized and displayed in a timely manner. Therefore, in this case, the fourth image displayed at the third moment is the image synthesized from the 1st to i-th frames of the preview images. For example, at the third moment, the 5(i)-th frame of the preview image captured by the electronic device, because the display is timely, the fourth image displayed at the third moment may be the image synthesized from the 1st to 5(j)-th frames of the preview images. That is, i = j.

[0145] For this case, it means that when the user triggers the end of taking a photo, all the preview images captured by the electronic device have just been successfully displayed. At this time, if the electronic device wants to perform image synthesis with the newly captured preview images, it needs to wait for the camera to output a frame, so it needs to spend time waiting. Therefore, in order to improve the image output speed and reduce the user's waiting time, the electronic device can directly save the fourth image as the third image. It is understandable that in this case, the third image saved by the electronic device is the fourth image.

[0146] At the same time, although the fourth image is an image synthesized in response to a preview request, the image processing flow corresponding to the preview request in the light painting shutter mode is usually the same as that corresponding to the photo-taking request in the light painting shutter mode. Therefore, whether directly using the fourth image as the third image corresponding to the photo-taking request or re-synthesizing to obtain the third image corresponding to the photo-taking request, the final image effect is the same. And directly using the fourth image as the third image can also save processing time, thereby further improving the image output efficiency and enhancing the user experience.

[0147] In other embodiments, in terms of the processing power of current electronic devices, the speed difference between acquisition and display is generally not very large. Therefore, even if there are preview images that cannot be displayed in time, the number will not be large, and there may be only one frame in most cases. Therefore, j is most likely equal to i, or equal to i-1. That is, in some embodiments, j=i or j=i-1.

[0148] In some embodiments, the image synthesized by the electronic device for the preview image frame in the streamer shutter mode is usually in YUV format. For example, the first image, the second image, the fourth image, and the third image synthesized by the preview image after the jth frame and the fourth image may all be in YUV format. Moreover, in general, the electronic device will only display the image in YUV format, and will not directly save the image in YUV format to the gallery.

[0149] Therefore, in order to comply with the image storage format, the third image in the embodiment of the present application can be saved by converting the third image in YUV format into a format acceptable to the electronic device library before being stored in the library. In some embodiments, after the third image is format converted, the converted format can be JPEG format. That is, the third image in YUV format is converted into a third image in JPEG format and then saved.

[0150] In some embodiments, the electronic device may synthesize multiple preview images frame by frame. That is, after the first preview image is synthesized with the second preview image to obtain the second image 1, the third preview image can be synthesized with the second image 1 to obtain the second image 2. The fourth preview image can be synthesized with the second image 2 to obtain the second image 3. Similarly, the kth preview image can be synthesized with the second image k-2 to obtain the second image k-1. The specific process is the same and will not be repeated here.

[0151] Therefore, synthesizing the preview images of the first to k frames captured by the camera to obtain the second image may include: synthesizing the fifth image and the preview image of the kth frame to obtain the second image. Among them, the fifth image of the embodiment of the present application can be synthesized based on the preview images of the first to k-1 frames.

[0152] Similarly, synthesizing the third image based on the preview images of the 1st to i-th frames captured by the camera can also include: synthesizing the sixth image and the i-th preview image to obtain the third image. The sixth image is an image synthesized based on the preview images of the 1st to i-1st frames. The principle is the same as that of the synthesis of the second image, which is cumulative synthesis frame by frame, and will not be repeated here.

[0153] In some embodiments, the software system of the above-mentioned electronic device may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Hereinafter, taking the Android TM system in a layered architecture as an example, the software and hardware architectures of the electronic device will be exemplarily described. Figure 6 FIG. shows a structural block diagram of the software and hardware architectures of an electronic device.

[0154] As Figure 6 shown, the structural block diagram of the software and hardware architectures of the electronic device includes a software layered architecture and a hardware layer. Among them, the hardware layer of the electronic device includes hardware required for the electronic device such as a camera, a display screen, a speaker, etc. It can be understood that according to actual needs, the electronic device may further include more hardware, such as buttons, motors, etc. Embodiments of the present application Figure 6 do not limit the hardware composition of the electronic device.

[0155] The software layered architecture of the electronic device divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely the application layer, the application framework layer, the Hardware Abstract Layer (HAL), and the kernel layer.

[0156] The application layer may include a series of application packages. As Figure 6 shown, the application packages may include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, etc.

[0157] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0158] As Figure 6 shown, the application framework layer may include a camera service, a window manager, an activity manager, an input manager, a resource manager, a notification manager, a view system, and a content provider, etc.

[0159] The camera service plays a bridging role. In response to requests from the camera application, it sends them to the camera HAL to call the camera to capture images. Specifically, in the embodiments of this application, the camera service can respond to preview requests (generated by responding to the user's first click on the shooting shutter) and capture requests (generated by responding to the user's second click on the shooting shutter) sent by the camera application, and send the preview requests and capture requests to the camera HAL to call the camera to capture images, thereby obtaining preview images corresponding to the preview requests (such as the first image, the second image, and the fourth image mentioned above) and capture images corresponding to the capture requests (such as the third image mentioned above).

[0160] The window manager provides the Window Manager Service (WMS). The WMS can be used for window management, window animation management, surface management, and as a transfer station for the input system.

[0161] The activity manager can provide the Activity Manager Service (AMS). The AMS can be used for the startup, switching, scheduling of system components (such as activities, services, content providers, and broadcast receivers), and the management and scheduling of application processes.

[0162] The input manager can provide the Input Manager Service (IMS). The IMS can be used to manage system inputs, such as touch screen inputs, key inputs, sensor inputs, etc. The IMS retrieves events from input device nodes and distributes the events to appropriate windows through interaction with the WMS.

[0163] The resource manager provides various resources for application programs, such as localized strings, icons, pictures, layout files, video files, and so on.

[0164] The notification manager enables application programs to display notification information in the status bar. It can be used to convey notification-type messages, which can disappear automatically after a short stay without user interaction. For example, the notification manager is used to inform that the download is complete, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as the notification of a background-running application program, or a notification that appears in the form of a dialogue window on the screen. For example, it prompts text information in the status bar, emits a prompt sound, vibrates the electronic device, and blinks the indicator light, etc.

[0165] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon can include a view for displaying text and a view for displaying pictures.

[0166] The content provider is used to store and obtain data, and make this data accessible to applications. The data can include videos, images, audio, incoming and outgoing calls, browsing history and bookmarks, phone books, etc.

[0167] The hardware abstraction layer runs in the user space, encapsulates the kernel layer drivers, and provides call interfaces to the upper layer. As Figure 6 shown, the hardware abstraction layer can include the camera HAL, the display HAL, the audio HAL, and the Bluetooth HAL.

[0168] The kernel layer is the layer between hardware and software. As Figure 6 shown, the kernel layer can include the camera driver, the display driver, the audio driver, and the Bluetooth driver.

[0169] Taking Figure 6 the software and hardware architecture of the electronic device shown as an example, the embodiments of the present application Figure 7 and Figure 8 respectively show the flowcharts of an image capture method.

[0170] Next, in combination with Figures 7 - 8 the image capture method provided by the embodiments of the present application will be described in detail.

[0171] In the state where the long exposure shutter mode is turned on, the user's first click operation on the capture shutter is used to trigger the electronic device to start taking pictures. That is to say, after the camera application in the application layer receives the user's first click operation on the capture shutter, the camera application starts to continuously send preview requests to the lower layer, that is, to the camera service in the application framework layer.

[0172] In the embodiments of the present application, the requests sent by the camera application are maintained in a request queue that follows the first-in, first-out rule in the order of the sending time.

[0173] As Figures 7 - 8For the request queue shown, preview request 1 (the first preview request) is the first preview request to be queued, preview request 2 (the second preview request) is the second preview request to be queued, …, preview request k (the k-th preview request) is the k-th preview request to be queued, …, preview request i (the i-th preview request) is the i-th preview request to be queued, …, preview request i + p (the (i + p)-th preview request) is the (i + p)-th preview request to be queued. The order of preview requests queued corresponds to their numbering order, which will not be elaborated here. Then, according to the first-in, first-out characteristic of the queue, each time the camera service finishes processing a preview request, that preview request can be dequeued.

[0174] After the camera service receives the preview requests successively sent by the camera application, it sends the preview requests to the camera HAL successively. The frame output module in the camera HAL maintains the preview requests sent from the upper layer. At the same time, the camera HAL also includes an automatic exposure module for determining the exposure time. The frame output module receives the exposure time determined by the automatic exposure module at the same time.

[0175] In some embodiments, the automatic exposure module can also determine exposure parameters such as the light input amount and the ISO value. The embodiments of the present application do not make any limitations in this regard. That is, when the automatic exposure module transmits the exposure time to the frame output module, it can transmit exposure parameters such as the light input amount and the ISO value together.

[0176] After the frame output module receives the preview request from the upper layer and the exposure time determined by the automatic exposure module, it sends the preview request to the camera driver in the lower layer according to the exposure time determined by the automatic exposure module. The camera driver calls the camera to capture the preview images corresponding to these preview requests.

[0177] As Figures 7 - 8 shown, the exposure time determined by the automatic exposure mode is 10 seconds (i.e., 10s). Then, the frame output module calls the camera to capture the preview images corresponding to the preview requests at 10-second intervals. It can be understood that the frame output module sends a preview request to the camera driver every 10 seconds to drive the camera to capture preview images. That is, the time interval between preview request 1 and preview request 2 is 10 seconds, the time interval between preview request 2 and preview request 3 is also 10 seconds, and the time intervals between subsequent preview requests are all 10 seconds, which will not be elaborated here.

[0178] After the camera driver receives the preview requests from the frame output module, in response to these preview requests, it drives the camera to capture one frame of preview image every 10 seconds and returns it to the camera driver. The preview images in the embodiments of the present application are stored in the first image queue.

[0179] As Figures 7 - 8As shown, since there is a 10-second interval between the preview request 1 (the first preview request) and the preview request 2 (the second preview request), there is also a 10-second interval between the preview image 1 (the first-frame preview image) and the preview image 2 (the second-frame preview image) in the first image queue. Similarly, there is a 10-second interval between the preview image 2 (the second-frame preview image) and the preview image 3 (the third-frame preview image) in the first image queue, and the time intervals of the subsequently acquired preview images are all 10 seconds, which will not be elaborated here.

[0180] In the embodiment of the present application, after the camera driver receives the preview image returned by the camera capture, the fast frame return module extracts the preview image from the first image queue frame by frame and transmits it to the image processing module. Then, the image processing module performs image synthesis on the preview image transmitted by the fast frame return module to obtain a corresponding synthesized image (such as the second image) and saves it to the second image queue. In some embodiments, the preview image in the first image queue may be dequeued because it is extracted by the fast frame return module, or because there is a newly enqueued preview image after the queue is full.

[0181] It can be understood that since the preview image 1 is the first-frame preview image captured by the camera after the first click operation and there is no image that can be synthesized before it, the preview image 1 is processed separately by the image processing module to obtain the first image. That is, the image content of the first image is the same as the image content of the preview image 1 (the first-frame preview image).

[0182] Subsequently, after the camera captures the preview image 2 (the second-frame preview image), the fast frame return module transmits the preview image 2 (the second-frame preview image) to the image processing module, and the image processing module performs image synthesis on the preview image 2 (the second-frame preview image) and the first image to obtain the second image 1.

[0183] Similarly, the fast frame return module transmits the preview image 3 (the third-frame preview image) to the image processing module, and the image processing module performs image synthesis on the preview image 3 (the third-frame preview image) and the second image 1 to obtain the second image 2. The fast frame return module transmits the preview image 4 (the fourth-frame preview image) to the image processing module, and the image processing module performs image synthesis on the preview image 4 and the second image 2 to obtain the second image 3. Subsequent images are all synthesized in the same synthesis manner, which will not be elaborated here.

[0184] It can be seen that the image corresponding to the preview request 1 (the first preview request) for display is the first image, the image corresponding to the preview request 2 (the second preview request) for display is the second image 1, the image corresponding to the preview request 3 (the third preview request) for display is the second image 2... The image corresponding to the preview request k (the kth preview request) for display is the second image k - 1... The image corresponding to the preview request i (the ith preview request) for display is the second image i - 1...

[0185] After that, the image processing module can transmit the first image and the second image in the second image queue to the selection module. The selection module selects the first image and the second image for display or sends them to the picture generation module for format conversion and storage in the picture library. In the embodiments of the present application, the images corresponding to the preview requests need to be displayed, and the images corresponding to the photographing requests do not need to be displayed.

[0186] Therefore, when the images transmitted by the image processing module are the first image and the second image corresponding to the preview requests, the selection module will return the first image and the second image to the upper-layer camera application via the display module for display.

[0187] In addition, if the image transmitted by the image processing module is the third image corresponding to the photographing request, the selection module will transmit the third image to the picture generation module for format conversion and then return it to the upper layer for storage in the picture library. In some embodiments, storing in the picture library can be returning to the camera application, and the camera application stores it in the picture library. It can also be directly returned to the picture library application for storage in the picture library. The illustrations in the embodiments of the present application are all returning to the picture library application for storage in the picture library.

[0188] Then, after the user observes a relatively satisfactory image effect through the second image displayed by the camera application, the user can perform a second click operation on the shooting shutter (for example Figure 3 the second click operation 113 shown), triggering the electronic device to end the photographing.

[0189] That is to say, after the camera application receives the second click operation of the user on the shooting shutter, the camera application starts to send the corresponding photographing request to the lower-layer camera service. It can be understood that since the second click operation triggers the end of photographing, the photographing request sent by the camera application is the last request in the request queue.

[0190] After the camera service receives the photographing request sent by the camera application, it also sends the photographing request to the frame output module in the camera HAL. At this time, because the exposure time is up to 10 seconds, there may be unprocessed preview requests before the photographing request.

[0191] Such as Figures 7 - 8As shown, the preview requests represented by dashed lines in the request queue are the preview requests that were not processed before the capture request. The unprocessed preview requests can be understood as the preview requests for which the camera has not completed the corresponding preview image capture. That is, in the embodiments of the present application, as long as the camera has not captured the preview image corresponding to this preview request, it indicates that this preview request is an unprocessed preview request.

[0192] Correspondingly, the images represented by dashed lines in the first image queue are the preview images corresponding one by one to these unprocessed preview requests, that is, the frames being output in the embodiments of the present application.

[0193] It should be noted that the unprocessed preview requests are actually sent by the upper-layer camera application, but the lower layer is still queuing up waiting to be processed. Therefore, these preview requests are actually included in the request queue. However, the frames being output are actually images for which the corresponding data has not been captured yet. Therefore, there should actually be no such images represented by dashed lines in the first image queue or the second image queue.

[0194] It can be understood that Figures 7 - 8 showing these images with dashed lines is for the convenience of understanding and describing the solution, and does not mean that these images actually exist in the first image queue or the second image queue. However, the preview requests represented by dashed lines are real requests that have not been responded to and processed.

[0195] According to the traditional processing method, after the upper-layer camera application sends a capture request, this capture request will wait in the request queue to be processed sequentially. Figure 9 The flowchart of a traditional image capture method is shown.

[0196] As Figure 9 shown, traditionally, after the camera captures an image, it is directly transmitted to the image processing module by the camera driver. Thus, if there are unprocessed preview requests after the user triggers the end of capture, it is necessary to wait for those unprocessed preview requests before the capture request to be processed before starting to process this capture request.

[0197] That is, as Figure 9As shown, after the camera application issues a photo-taking request, if the camera has not yet captured the p preview images after the preview image i, it means that the p preview requests after the preview request i are all unprocessed preview requests. Traditionally, it is necessary to wait for the camera to capture these p preview images after the preview request i before it is considered that these p preview requests after the preview request i have been processed. Furthermore, only when all p preview requests after the preview request i are processed can the electronic device start to process the photo-taking request. That is, in the chronological order of the preview requests, traditionally, it is necessary to wait until the preview request i + p (the (i + p)-th preview request) is also processed before the photo-taking request can be processed and responded to.

[0198] Therefore, if there are more unprocessed preview requests before the photo-taking request, that is, the more preview requests after the preview request i, which means the larger p is, the longer it will take for the electronic device to output an image, the lower the image output efficiency will be, and the longer the user will have to wait for the image to be output. That is, as Figure 9 shown, traditionally, after the camera application issues a photo-taking request, it is necessary to wait for the camera to capture the (i + p)-th preview image (preview image i + p) before it can capture the photo image corresponding to the photo-taking request and perform image synthesis to obtain the third image and return it to the gallery.

[0199] However, in the embodiments of the present application, in order to improve the image output efficiency and reduce the user waiting time, after the lower layer receives the photo-taking request issued by the camera application, the frame output module sends the photo-taking request to the fast frame return module. The fast frame return module quickly responds to the photo-taking request of the upper-layer camera application.

[0200] In the embodiments of the present application, after the fast frame return module receives the photo-taking request, it immediately responds to the photo-taking request and determines whether there are any un-displayed preview images in the first image queue. At the current processing speed of the electronic device, if there are un-displayed preview images in the first image queue, there is usually only one frame. This frame of preview image is usually the latest preview image captured by the camera. That is, the last frame of preview image in the first image queue. Therefore, in some embodiments, the fast frame return module can directly extract the last frame of preview image in the first image queue.

[0201] As Figures 7 - 8 shown, since the preview images after the preview image i (the i-th preview image) in the first image queue are all represented by dotted lines, that is, there are actually no preview images, the last frame of preview image is the preview image i (the i-th preview image). Therefore, the fast frame return module can directly extract the preview image i (the i-th preview image).

[0202] If the last preview image in the first image queue is an image that has not been sent for display yet (i.e., there is a preview image that has been captured by the camera but has not been sent for display in time), that is, there is currently a preview image in the first image queue that has been captured by the camera but has not been sent for display in time. Then, the fast frame return module transmits this last preview image and the capture request to the image processing module. The image processing module synthesizes the last preview image with the previous preview images to obtain a third image. Then, the image processing module sends this synthesized third image to the selection module.

[0203] After the selection module receives the third image transmitted by the image processing module, since this third image is the synthesized image corresponding to the capture request. Therefore, the selection module sends this third image to the picture generation module, and the picture generation module performs format conversion on this third image and returns the format-converted third image to the application for saving to the gallery.

[0204] It can be understood that this third image saved in the gallery is the image obtained by the user using the light painting shutter mode for image capture. For example, the third image can be Figure 2 the star trail image shown. It can be understood that since Figure 2 is a star trail image that has been processed in black and white later. Therefore, if the third image saved in the gallery has not been processed in black and white later, the third image in the gallery can be a color star trail image.

[0205] Reference Figure 7 , the last preview image is preview image i, but preview image i (the i-th preview image) is the preview image that has been newly captured by the camera and has not been sent to the image processing module for image synthesis and display in time. At this time, preview image i (the i-th preview image) is a preview image that has already been out of the frame and does not require additional waiting for out-of-frame. Therefore, in order to quickly output the image while ensuring that more image information is fused, the fast frame return module can extract preview image i (the i-th preview image) and send it to the image processing module for image synthesis to obtain the corresponding second image i-1. Then, the image processing module transmits this second image i-1 to the selection module. Since the second image i-1 is the image corresponding to the capture request, the selection module sends the second image i-1 to the picture generation module for format conversion and returns it to the upper layer for saving to the gallery.

[0206] If the last preview image in the first image queue is a preview image that has already been transmitted to the image processing module for processing, then this last preview image is already an image that has been synthesized and displayed (i.e., there is no preview image that has been captured by the camera but has not been sent for display in time). That is, all the preview images in the first image queue have been synthesized and displayed in a timely manner, indicating that there is currently no preview image that can be synthesized without time-consuming waiting.

[0207] Meanwhile, since the second image synthesized by the image processing module is determined by the selection module whether to be displayed or stored in the picture library, the selection module usually retains the latest frame of the second image. Therefore, in order to quickly generate an image, the fast frame return module can directly send an image generation instruction to the selection module. In response to the image generation instruction, the selection module directly sends the second image newly transmitted by the image processing module to the image generation module for format conversion and then returns it to the application layer to be stored in the picture library.

[0208] Reference Figure 8 , the last frame of the preview image is the preview image i (the i-th frame of the preview image), and its corresponding synthesized image is the second image i-1. This second image i-1 is the currently latest second image sent for display received by the selection module from the image processing module. Then, in response to the image generation instruction, the selection module directly sends this second image i-1 to the image generation module. After format conversion by the image generation module, it is stored in the picture library.

[0209] In summary, by comparing Figure 7 , Figure 8 and Figure 9 , it can be seen that in the embodiment of the present application, regardless of Figure 7 and Figure 8 , in either case, the second image i-1 will be advanced to become the third image corresponding to the photographing request. Thus, in the embodiment of the present application, there is no need to wait in sequence for a large amount of time for preview images such as preview image i+1, preview image i+2... preview image i+p to be output as in the traditional method shown in Figure 9 before the third image corresponding to the photographing request can be synthesized.

[0210] That is to say, the difference between the embodiment of the present application and the traditional method shown in Figure 9 is that after the camera application issues a photographing request, the embodiment of the present application directly abandons waiting for the preview images after the preview image i (the i-th frame of the preview image) to be output.

[0211] Therefore, compared with the traditional shooting method, the embodiment of the present application can achieve fast image generation. The image generation speed will be faster, thereby improving the image generation efficiency, reducing the waiting time of the user, and enhancing the user experience.

[0212] In summary, in the embodiment of the present application, once the camera application issues a photographing request, the fast frame return module quickly responds to the photographing request, instead of waiting in sequence for the processing of the photographing request as in the traditional method. The embodiment of the present application pre-processes the photographing request through the fast frame return module, abandons the frames being output, and directly sends the already output frames (i.e., the preview images collected by the camera) to the image generation module for format conversion to obtain the third image that can be stored in the picture library. As Figures 7 - 8As shown, the acquisition of preview images i+1, i+2, …, i+p by the camera is abandoned. In this way, the embodiments of the present application can reduce the image output time, achieve fast image output, thereby reducing the waiting time of users and improving the user experience.

[0213] In some embodiments, in addition to image synthesis, the image processing module may also perform image processing such as image compensation, image correction, and image denoising on images. The embodiments of the present application do not make any limitations in this regard.

[0214] As Figures 10 - 11 shown, the embodiments of the present application illustrate a flowchart of another image capture method.

[0215] Since each request issued by the camera application requires a corresponding image to be returned, if the frames being produced are directly discarded without feedback to the upper layer, it will cause some preview requests issued by the camera application to have no corresponding returned images, thus easily causing other unnecessary problems. Therefore, in order to avoid other unnecessary problems caused by the lack of corresponding returned images for preview requests, in the embodiments of the present application, for those frames being produced that are abandoned, virtual data and discard frame flags are filled and then returned to the upper-layer camera application. Among them, the virtual data includes virtual image data (buffer) and image metadata. By means of the virtual image data and image metadata, this frame of image can be successfully returned to the upper layer.

[0216] Meanwhile, when the camera application receives this frame of image, the camera application can determine that this frame of image is an incorrect frame, that is, an image that does not need to be processed for display, based on the discard frame flag. Thus, the camera application directly abandons the display of this frame of image.

[0217] Specifically, in the embodiments of the present application, after the upper-layer gallery application receives the third image returned by the lower layer (for example Figures 7 - 8 the second image i-1 in, that is, the third image in the embodiments of the present application), the gallery application (or the camera application, shown as the gallery application) issues a flag bit to the fast frame return module. The gallery application feeds back to the fast frame return module that it has received the third image through the issued flag bit.

[0218] Furthermore, in response to this flag bit, the fast frame return module generates corresponding virtual preview images for each unprocessed preview request. These virtual preview images are generated by filling virtual data, that is, filling virtual image data and image metadata. At the same time, the virtual preview images also include discard frame flags.

[0219] Exemplarily, Figure 10The p preview requests after the preview request i shown are unprocessed preview requests. That is, the camera has not captured the p preview images after the preview image i (that is, has not captured the preview image i + 1, preview image i + 2,..., preview image i + p). Therefore, in response to the flag bit, the fast frame - back module generates a corresponding virtual preview image for each of these p preview requests, so as to ensure that there is a corresponding returned image for each of these p preview requests issued by the camera application.

[0220] Meanwhile, the virtual preview images all carry discard - frame flags. Further, after the camera application receives the virtual preview images corresponding to these p preview requests, it can discard and not display these virtual preview images to prevent the display of incorrect frames.

[0221] That is to say, in the embodiment of the present application, because the fast frame - back module responds to the capture request in advance, the current image capture is already completed. However, in the actual capture process, there are still unprocessed preview requests (that is, the p preview requests after the preview request i), and the capture is actually not over yet. Therefore, if the capture is not further finalized, it may cause the capture process to be abnormal.

[0222] Therefore, in the embodiment of the present application, the fast frame - back module is instructed by the flag bit sent from the upper layer to finalize the capture to prevent the capture process from being abnormal. Therefore, after the fast frame - back module receives the flag bit sent from the upper layer, the fast frame - back module responds to this flag bit and starts to finalize the capture work, that is, generates virtual preview images filled with virtual image data, image metadata, and discard - frame flags for the unprocessed preview requests.

[0223] Among them, the filled virtual image data and image metadata can be configured according to actual needs, and the embodiment of the present application does not make any limitations. The discard - frame flag can also be agreed according to actual needs. For example, an error flag can be added to the virtual preview image to indicate that this image frame is an incorrect frame. Thus, not only can it be ensured that each preview request has a corresponding returned image to the upper layer, but also the upper layer can be informed through the discard - frame flag not to perform any display processing on this image, thereby avoiding incorrect display.

[0224] In some embodiments, after the image capture process starts, the automatic exposure module is also making decisions on exposure parameters as required for use in capture. Therefore, the finalization work of the fast frame - back module should also include interrupting the exposure decision work of the automatic exposure module.

[0225] Therefore, as Figures 10 - 11As shown, after the quick frame return module receives the flag bit sent by the gallery application (or camera application), in order to prevent the automatic exposure module from continuously making exposure time decisions for the current image capture, the quick frame return module responds to this flag bit and simultaneously sends an interrupt instruction to the automatic exposure module. The quick frame return module interrupts the exposure decision of the automatic exposure module through the interrupt instruction and reclaims the corresponding resources. At the same time, it aborts and discards the frames captured by the camera. Thereby, it is ensured that no abnormalities occur after the image capture ends prematurely, causing other unnecessary problems.

[0226] In the embodiments of the present application, the above-mentioned electronic device may include at least one of a mobile phone, a foldable electronic device (such as a foldable mobile phone), a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, a camera, a video camera, a video recorder, or a smart city device. The embodiments of the present application do not impose special restrictions on the specific type of the electronic device.

[0227] As Figure 12 shown, the embodiments of the present application illustrate a schematic structural diagram of an electronic device 1200.

[0228] The electronic device 1200 may include a processor 1210, an external memory interface 1220, an internal memory 1221, a universal serial bus (USB) connector 1230, a charging management module 1240, a power management module 1241, a battery 1242, antennas 1201, 1202, a mobile communication module 1250, a wireless communication module 1260, an audio module 1270, a speaker 1270A, a receiver 1270B, a microphone 1270C, a headphone jack 1270D, a sensor module 1280, keys 1290, a motor 1291, an indicator 1292, a camera module 1293, a display screen 1294, and a subscriber identification module (SIM) card interface 1295, etc. Among them, the sensor module 1280 may include a pressure sensor 1280A and a touch sensor 1280B.

[0229] It can be understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the electronic device 1200. In other embodiments of this application, the electronic device 1200 may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0230] The processor 1210 may include one or more processing units. For example, the processor 1210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0231] The processor may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions. Exemplarily, the image capture method described in the embodiments of this application may be implemented by the processor 1210.

[0232] A memory may also be provided in the processor 1210 for storing instructions and data. In some embodiments, the memory in the processor 1210 may be a cache memory. This memory may store instructions or data that have been used by the processor 1210 or are used frequently. If the processor 1210 needs to use this instruction or data, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor 1210, and thus improves the efficiency of the system.

[0233] In some embodiments, the processor 1210 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. The processor 1210 may be connected to modules such as a touch sensor, an audio module, a wireless communication module, a display screen, and a camera module through at least one of the above interfaces.

[0234] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only for illustrative purposes and does not constitute a structural limitation on the electronic device 1200. In other embodiments of the present application, the electronic device 1200 may also adopt different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0235] The external memory interface 1220 may be used to connect to an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the electronic device 1200. The external memory card communicates with the processor 1210 through the external memory interface 1220 to implement the data storage function. For example, music, video and other files are saved in the external memory card. Or music, video and other files are transferred from the electronic device to the external memory card.

[0236] The internal memory 1221 can be used to store computer-executable program code, and the executable program code includes instructions. The internal memory 1221 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 1200 (such as audio data, a phone book, etc.). In addition, the internal memory 1221 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 1210 executes various functional methods or data processing of the electronic device 1200 by running the instructions stored in the internal memory 1221 and / or the instructions stored in the memory provided in the processor.

[0237] The electronic device 1200 can implement a display function through a GPU, a display screen 1294, an application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 1294 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 1210 may include one or more GPUs, which execute program instructions to generate or change display information. For example, the GPU can be used for rendering and the display screen 1294 can be used to display the star trail images generated in the embodiments of the present application (such as the first preview image and the second image stored in the gallery).

[0238] The display screen 1294 is used to display images, videos, etc. In some embodiments, it can be used to display the captured star trail images. The display screen 1294 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 1200 can include one or more display screens 1294.

[0239] The electronic device 1200 can implement the camera function through the camera module 1293, ISP, video codec, GPU, display screen 1294, application processor AP, neural network processor NPU, etc. For example, the camera module 1293 and ISP are used to capture and generate the star trail images (such as the first preview image and the second image stored in the gallery) in the embodiments of the present application.

[0240] The camera module 1293 can be used to collect the color image data and depth data of the photographed object. The ISP can be used to process the color image data collected by the camera module 1293. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element (i.e., the image sensor), where the optical signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene.

[0241] In some embodiments, the ISP can be disposed in the camera module 1293.

[0242] In some embodiments, the camera module 1293 can be composed of a color camera module and a 3D sensing module.

[0243] In some embodiments, the photosensitive element of the camera of the color camera module can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats.

[0244] In some embodiments, the 3D sensing module may be a (time of flight, TOF) 3D sensing module or a structured light 3D sensing module. Among them, structured light 3D sensing is an active depth sensing technology. The basic components of the structured light 3D sensing module may include an infrared (Infrared) emitter, an IR camera module, etc. The working principle of the structured light 3D sensing module is to first emit a light spot with a specific pattern to the object to be photographed, then receive the light coding of the light spot pattern on the surface of the object, and then compare the similarities and differences with the original projected light spot, and use the triangulation principle to calculate the three-dimensional coordinates of the object. The three-dimensional coordinates include the distance between the electronic device 1200 and the object to be photographed. Among them, TOF 3D sensing can be an active depth sensing technology. The basic components of the TOF 3D sensing module may include an infrared (Infrared) emitter, an IR camera module, etc. The working principle of the TOF 3D sensing module is to calculate the distance (i.e., depth) between the TOF 3D sensing module and the object to be photographed through the time of infrared return, so as to obtain a 3D depth of field map.

[0245] The structured light 3D sensing module can also be applied to fields such as face recognition, somatosensory game consoles, and industrial machine vision detection. The TOF 3D sensing module can also be applied to fields such as game consoles, augmented reality (AR) / virtual reality (VR), etc.

[0246] In some other embodiments, the camera module 1293 can also be composed of two or more cameras. These two or more cameras may include a color camera, and the color camera can be used to collect color image data of the object to be photographed. These two or more cameras can use stereo vision technology to collect depth data of the object to be photographed. Stereo vision technology is based on the principle of human eye parallax. Under natural light, images of the same object are taken from different angles through two or more cameras, and then operations such as triangulation are performed to obtain the distance information between the electronic device 1200 and the object to be photographed, that is, depth information.

[0247] In some embodiments, the electronic device 1200 may include one or more camera modules 1293. Specifically, the electronic device 1200 may include one front camera module 1293 and one rear camera module 1293. Among them, the front camera module 1293 is usually used to collect color image data and depth data of the photographer facing the display screen 1294, and the rear camera module can be used to collect color image data and depth data of the object to be photographed (such as star trails) faced by the photographer.

[0248] In some embodiments, the CPU, GPU, or NPU in the processor 1210 may process the color image data and depth data collected by the camera module 1293. In some embodiments, the NPU may identify the color image data collected by the camera module 1293 (specifically, the color camera module) through a neural network algorithm based on skeleton point recognition technology, such as a convolutional neural network algorithm (CNN), to determine the skeleton points of the photographed person. The CPU or GPU may also run the neural network algorithm to determine the skeleton points of the photographed person based on the color image data.

[0249] The digital signal processor is used to process digital signals and can also process other digital signals. For example, when the electronic device 1200 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0250] The video codec is used to compress or decompress digital videos. The electronic device 1200 may support one or more video codecs. In this way, the electronic device 1200 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0251] The NPU is a neural-network (NN) computing processor. By learning from the biological neural network structure, such as learning from the transmission mode between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 1200 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0252] The pressure sensor 1280A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor 1280A can be disposed on the display screen 1294. There are many types of pressure sensors 1280A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 1280A, the capacitance between the electrodes changes. The electronic device 1200 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 1294, the electronic device 1200 detects the intensity of the touch operation according to the pressure sensor 1280A. The electronic device 1200 can also calculate the position of the touch based on the detection signal of the pressure sensor 1280A. In some embodiments, touch operations with the same touch position but different touch operation intensities can correspond to different operation instructions. For example: When a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, the instruction to view short messages is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0253] The touch sensor 1280B, also known as a "touch control device". The touch sensor 1280B can be disposed on the display screen 1294. The touch sensor 1280B and the display screen 1294 form a touch screen, also known as a "touch control screen". The touch sensor 1280K is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 1294. In other embodiments, the touch sensor 1280K can also be disposed on the surface of the electronic device 1200, in a different position from the display screen 1294.

[0254] Specific to the embodiments of this application, the electronic device 1200 can detect the user's click operation on the camera shutter and the opening operation of the user to open the camera application through the pressure sensor 1280A and the touch sensor 1280B.

[0255] The USB receiver 1230 is an interface that complies with the USB standard specification and can be used to connect the electronic device 1200 and peripheral devices. Specifically, it can be a Mini USB connector, a Micro USB connector, a USB Type C connector, etc. The charging management module 1240 is used to receive the charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. The power management module 1241 is used to connect the battery 1242, the charging management module 1240, and the processor 1210.

[0256] The wireless communication function of the electronic device 1200 can be implemented by the antenna 1201, antenna 1202, mobile communication module 1250, wireless communication module 1260, modulation and demodulation processor, baseband processor, etc.

[0257] The electronic device 1200 can implement audio functions through the audio module 1270, speaker 1270A, receiver 1270B, microphone 1270C, headphone jack 1270D, and application processor, etc. Such as music playback, recording, etc.

[0258] The keys 1290 may include a power-on key, volume keys, etc. The motor 1291 can generate a vibration prompt. The indicator 1292 can be an indicator light, which can be used to indicate the charging status, power change, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 1295 is used to connect the SIM card.

[0259] It should be noted that the image capture methods in the following embodiments can all be implemented in the electronic device 1200 with the above hardware structure.

[0260] Another embodiment of the present application provides an electronic device, including: a camera, one or more processors, and a memory. The camera and the memory are respectively coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device implements the image capture method described in any of the above embodiments.

[0261] Another embodiment of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program is executed by a processor in the electronic device, the electronic device implements the image capture method described in any of the above embodiments.

[0262] The embodiment of the present application also provides a computer program product. When the computer program product runs on a computer, the computer executes each function or step in the above method embodiments.

[0263] The embodiment of the present application also provides a chip system, as Figure 13 shown, the chip system 1300 includes at least one processor 1301 and at least one interface circuit 1302. The processor 1301 and the interface circuit 1302 can be interconnected by lines. For example, the interface circuit 1302 can be used to receive signals from other devices (such as the memory of a computer). For another example, the interface circuit 1302 can be used to send signals to other devices (such as the processor 1301).

[0264] Exemplarily, the interface circuit 1302 can read the instructions stored in the memory and send the instructions to the processor 1301. When the instructions are executed by the processor 1301, the computer can be made to execute the various steps in the above embodiments. Of course, the chip system may further include other discrete devices, and the embodiments of the present application do not make specific limitations thereto.

[0265] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0266] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0267] The units described as separate components may or may not be physically separated. The components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0268] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0269] When an integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0270] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image shooting method, characterized in that, Applied to an electronic device, the electronic device includes a camera, and the electronic device supports a preset shooting mode, and the preset shooting mode includes a star trail shooting mode; the method includes: In response to a user's operation of enabling the preset shooting mode within the camera application, display a shooting interface corresponding to the preset shooting mode; wherein, the shooting interface corresponding to the preset shooting mode includes a shooting shutter; Receive a first click operation of the user on the shooting shutter; At a first moment, display a first image; wherein, the first image is obtained from the first frame of preview image collected by the camera based on the first preview request generated in response to the first click operation; the camera collects one frame of preview image based on each preview request; At a second moment, display a second image; wherein, the second image is synthesized from the first to k frames of preview images collected by the camera based on the first to k preview requests generated in response to the first click operation; the second moment is after the first moment; k>1, and k is a positive integer; At a third moment, receive a second click operation of the user on the shooting shutter, and save a third image; wherein, the third moment is after the second moment; at the third moment, in response to the first click operation, the (i + p)th preview request has been generated, and the camera has collected the i-th frame of preview image; the third image is synthesized from the first to i frames of preview images collected by the camera; i>k, and p is a positive integer.

2. The method according to claim 1, wherein The method further includes: At the third moment, the electronic device displays a fourth image; the fourth image is synthesized from the first to j frames of preview images collected by the camera based on the first to j preview requests generated in response to the first click operation; k<j≤i, and j is a positive integer; if j<i, the third image is synthesized with the fourth image and the preview images after the j-th frame; if j = i, the fourth image is the third image.

3. The method according to claim 2, wherein j = i or j = i - 1.

4. The method according to any one of claims 1 to 3, characterized in that The saving of the third image includes: Perform format conversion on the third image, and save the third image after format conversion; wherein, the third image before format conversion is in YUV format, and the third image after format conversion is in JPEG format.

5. The method according to any one of claims 1-4, characterized in that The second image synthesized from the first to k frames of preview images collected by the camera includes: Synthesize a fifth image and the k-th frame of preview image to obtain the second image; wherein, the fifth image is synthesized based on the first to k - 1 frames of preview images.

6. The method according to any one of claims 1-5, characterized in that, The third image is synthesized based on the first to i frames of preview images collected by the camera, including: Synthesize a sixth image and the i-th frame of preview image to obtain the third image; wherein, the sixth image is synthesized based on the first to i - 1 frames of preview images.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Generate corresponding virtual preview images for p preview requests after the i-th preview request; wherein, the virtual preview images include virtual image data, image metadata, and discard frame flags.

8. The method according to any one of claims 1-7, characterized in that The method further includes: after obtaining the third image, interrupt the exposure decision.

9. The method according to any one of claims 1 - 8, characterized in that The electronic device includes an automatic exposure module, a frame output module, a fast frame return module, an image processing module, a selection module, and a display sending module; The camera application continuously sends the preview request to the frame output module in response to the first click operation; The frame output module periodically sends the preview request to the camera according to the exposure time of the camera to drive the camera to capture a frame of preview image; wherein, the preview images are stored in the first image queue in the order of acquisition; the exposure time is determined by the automatic exposure module according to the ambient brightness and sent to the frame output module; The fast frame return module extracts the preview images from the first image queue frame by frame and transmits them to the image processing module. The image processing module synthesizes the first frame of preview image to obtain the first image, and synthesizes the first to k frames of preview images to obtain the second image; The image processing module transmits the first image and the second image to the camera application via the selection module and the display sending module, and the camera application displays the first image and the second image.

10. The method according to claim 9, wherein The electronic device further includes a picture generation module; The camera application sends a photographing request to the fast frame return module via the frame output module in response to the second click operation at the third moment; At the third moment, if the preview images after the j-th frame among the first to i-th frame preview images have not been transmitted to the image processing module, the fast frame return module extracts the preview images after the j-th frame and transmits them to the image processing module in response to the photographing request; k < j < i, and j is a positive integer; The image processing module synthesizes the preview images after the j-th frame with the fourth image to obtain the third image; wherein, the fourth image is synthesized by the image processing module based on the first to j-th frame preview images; The image processing module transmits the third image to the picture generation module via the selection module, and the picture generation module performs format conversion on the third image and then sends it to the camera application for saving.

11. The method according to claim 10, wherein The method further includes: At the third moment, if all the first to i-th frame preview images have been extracted and transmitted to the image processing module frame by frame, the fast frame return module sends a picture generation instruction to the selection module in response to the photographing request; The selection module transmits the third image to the picture generation module, and the picture generation module performs format conversion on the third image and then sends it to the camera application for saving.

12. The method according to any one of claims 9-11, characterized in that The method further includes: The fast frame return module receives the flag bit sent by the camera application, and the flag bit is sent when the camera application receives the third image; The fast frame return module generates corresponding virtual preview images for p preview requests after the i-th preview request in response to the flag bit.

13. The method according to any one of claims 9-12, characterized in that, The method further includes: The fast frame return module sends an interrupt instruction to the automatic exposure module in response to the flag bit sent by the camera application, instructing the automatic exposure module to interrupt the exposure decision.

14. An electronic device, characterized in that, Including: A camera, one or more processors, and a memory, wherein the camera and the memory are respectively coupled to the processor; The camera is configured to capture images; One or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device is caused to execute the image capturing method according to any one of claims 1-13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the electronic device, the electronic device is caused to execute the image capturing method according to any one of claims 1-13.

Images