Image capturing method, electronic device, and computer-readable storage medium

Abstract

The present application relates to the technical field of electronic devices, and provides an image capturing method, an electronic device, and a computer-readable storage medium. The method is applied to an electronic device supporting a preset photographing mode, and comprises: when the electronic device performs image capturing in the preset photographing mode, if a second click operation of a user for triggering ending of photographing is received, the electronic device synthesizes preview images currently acquired by a camera to obtain a third image for storage. That is, upon receiving the second click operation, the electronic device gives up waiting for preview images that have not yet been captured, and directly synthesizes and saves only preview images that have been currently captured by the camera, thereby improving image output efficiency, shortening the waiting time of a user, and enhancing the photographing experience of the user.

Description

[Corrected 28.10.2024 according to Rule 26] Image capturing method, electronic device, and computer-readable storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 29, 2023, with application number 202311873537.4 and invention name “Image capture method, electronic device and computer-readable storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the technical field of electronic devices, and in particular, to an image capturing method, an electronic device, and a computer-readable storage medium. Background Art

[0003] With the advancement of electronic device image capture technology and the increasing demand for photography from users, existing electronic devices are now offering users a growing number of image capture modes. These include portrait mode, panorama mode, time-lapse photography, night mode, slow motion, high dynamic range imaging (HDR), and light streamer shutter. Light streamer shutter is also known as long exposure.

[0004] Currently, the light streamer shutter mode allows for long-term continuous image capture to capture the movement of light. Therefore, to capture images using the light streamer shutter mode, the electronic device must respond to two user clicks on the shutter button to complete the capture. The first click triggers the electronic device to start capturing the image, and the second click triggers the electronic device to end capturing the image. However, once the electronic device begins capturing, it will continue to use the camera for image acquisition.

[0005] Therefore, when the user triggers the electronic device to end image capture, the camera may still have some uncaptured images. Currently, for these uncaptured images, the electronic device often waits for the camera to complete the capture before generating the final image from the captured images, thus ending the capture process. Consequently, when using the streamer shutter mode to capture images, the electronic device's image output speed is relatively slow and inefficient, forcing the user to wait for a long time for the image to be output, resulting in a poor user experience.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide an image capture method, an electronic device, and a computer-readable storage medium for solving the problem of slow image output speed and low image output efficiency in streamer shutter mode, which causes users to wait for a long time for image output and a poor user experience.

[0008] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0009] In a first aspect, an image capture method is provided. The method is applied to an electronic device, the electronic device including a camera, and the electronic device supporting a preset capture mode for capturing star trails. The preset capture mode may be a mode for continuously capturing multiple frames of continuous images within the period between two shutter clicks, and synthesizing a single image frame based on the multiple frames of continuous images. The preset capture mode may include at least one of a streamer shutter mode, a long exposure mode, and a star trail capture mode. Alternatively, the preset capture mode may also be other capture modes involving long exposure or other modes for capturing star trails, without limitation.

[0010] The method includes: first, in response to a user's activation of a preset shooting mode within a camera application, the electronic device displays a shooting interface corresponding to the preset shooting mode; wherein the shooting interface corresponding to the preset shooting mode includes a shooting shutter. Subsequently, when the user triggers image capture on the shooting interface, the electronic device is deemed to be shooting in the preset shooting mode. That is, the electronic device captures the image using the image capture method corresponding to the preset shooting mode and processes the captured image according to the processing method corresponding to the preset shooting mode.

[0011] In this preset shooting mode, the electronic device first receives the user's first click operation on the shooting shutter, displays the first image at the first moment, and displays the second image at the second moment. The first image is an image obtained by capturing the first frame of preview image based on the first preview request generated in response to the first click operation. The second image is an image synthesized from the first to k frames of preview image captured by the camera based on the first to k preview requests generated in response to the first click operation. Wherein, k>1, k is a positive integer. The second moment is before the first moment. It can be understood that the camera will capture a corresponding preview image frame based on each preview request generated.

[0012] Then, at a third moment after the second moment, the electronic device receives a second click operation on the shooting shutter by the user, and the second click operation is used to trigger the end of the shooting. The electronic device then saves the third image and ends the shooting.

[0013] In the first aspect, at the third moment, if the electronic device has called the camera to capture the i-th frame preview image, but the i+p-th preview request has been generated in response to the first click operation, the electronic device only synthesizes the preview images from the 1st to the i-th frames to obtain the third image for storage. That is, in the first aspect, if the electronic device receives the second click operation, the electronic device directly gives up waiting for the preview images that have not yet been captured, and directly synthesizes and saves only the preview images currently captured by the camera. In other words, the electronic device gives up waiting for the above-mentioned p-frame images to be framed, and directly synthesizes the preview images from the 1st to the i-th frames to obtain the third image to be saved, thereby avoiding the problem of reduced image output speed caused by waiting for frame output, thereby improving image output efficiency, reducing user waiting time, and enhancing user shooting experience.

[0014] In one possible implementation of the first aspect, when the electronic device receives the user's second click operation to end the photo capture at the third moment, it is possible that all captured preview images have already been synthesized and displayed, or the electronic device may still have some preview images that have been captured but have not yet been synthesized and displayed. In this case, the fourth image displayed at the third moment may be an image synthesized from all captured preview images, or it may be an image that does not include preview images that have not yet been displayed. That is, the above-mentioned image capture method may further include:

[0015] At the third moment, the electronic device displays a fourth image; the fourth image is synthesized from the preview images of frames 1 to j captured by the camera based on the preview requests 1 to j generated in response to the first click operation, where k < j ≤ i, and j is a positive integer.

[0016] Specifically, if j < i, the fourth image displayed at the third moment is an image synthesized from only the preview images of frames 1 to j. In other words, the fourth image is the last frame synthesized and displayed by the electronic device at the third moment, and there are currently preview images from frames j+1 to i that have been captured but not synthesized and displayed. Furthermore, the third image can be obtained by synthesizing the fourth image and the preview images from frames j+1 to i that have not been synthesized and displayed. In this way, the number of images used to directly synthesize the third image is less than frames i, and the computational complexity is reduced. In some cases, the difference between i and j is not too large, for example, the difference is 1 or 2.

[0017] If j=i, it indicates that the fourth image displayed at the third moment is an image obtained by synthesizing the preview images of the 1st to i frames, and thus the third image can be the fourth image.

[0018] In another possible implementation of the first aspect, given the current processing capabilities of electronic devices, there is a high probability that only one preview image frame will be displayed in time, for example, j = i-1. Therefore, at the third moment, the electronic device displays a fourth image; the fourth image is synthesized from the preview images of frames 1 to j captured by the camera based on the preview requests 1 to j generated in response to the first click operation; where j = i or j = i-1.

[0019] In one possible implementation of the first aspect, to conform to an image storage format, saving the third image may include: converting the format of the third image and saving the converted third image. In another possible implementation of the first aspect, the third image before the format conversion may be in YUV format, and the third image after the format conversion may be in JPEG format.

[0020] In a possible implementation of the first aspect, the electronic device may synthesize multiple preview images on a frame-by-frame basis. Based on this, synthesizing the preview images (1st to kth frames) captured by the camera to obtain the second image may include synthesizing the fifth image with the preview image (kth frame) to obtain the second image. The fifth image is synthesized based on the preview images (1st to k-1th frames).

[0021] In another possible implementation of the first aspect, synthesizing the third image based on preview images (1 to i) captured by the camera may include synthesizing a sixth image with the i-th preview image to obtain the third image, wherein the sixth image is synthesized based on preview images (1 to i-1).

[0022] In a possible implementation of the first aspect, to avoid unnecessary problems and image capture abnormalities caused by a preview request lacking a corresponding returned image, the image capture method may further include: generating corresponding virtual preview images for each of the p preview requests following the i-th preview request; wherein the virtual preview images include virtual image data, image metadata, and a frame discard flag. The frame discard flag is used to indicate that the corresponding virtual preview image can be discarded, or to indicate that the corresponding virtual preview image is an error frame.

[0023] In another possible implementation of the first aspect, in order to avoid abnormal shooting caused by the exposure decision being continued after the shooting is completed, the above-mentioned image shooting method may further include: after obtaining the third image, interrupting the exposure, or interrupting the continued execution of the exposure decision.

[0024] In a possible implementation of the first aspect, the electronic device includes an automatic exposure module, a frame output module, a fast frame return module, an image processing module, a selection module, and a display module; and the image capture method may further include:

[0025] In response to the first click, the camera application continuously sends preview requests to the frame output module. The frame output module periodically sends preview requests to the camera according to the camera's exposure time, driving the camera to capture a preview image frame. The preview images are stored in the first image queue in the order in which they were captured. The exposure time is determined by the automatic exposure module based on ambient brightness and sent to the frame output module. The fast frame return module then extracts preview images frame by frame from the first image queue and transmits them to the image processing module. The image processing module synthesizes the first preview image to obtain the first image, and the image processing module synthesizes the preview images from frames 1 to k to obtain the second image. The image processing module transmits the first and second images to the camera application via the selection module and the display module, and the camera application displays the first and second images.

[0026] In a possible implementation of the first aspect, the electronic device further includes an image generation module. The image capture method may further include: the camera application, in response to a second click operation at a third moment, sending a photo request to the quick frame return module via the frame output module; at the third moment, if the preview images after the jth frame among the preview images of frames 1 to i have not been transmitted to the image processing module, the quick frame return module, in response to the photo request, extracts the preview images after the jth frame and transmits them to the image processing module; k < j < i, where j is a positive integer; the image processing module synthesizes the preview images after the jth frame with a fourth image to obtain a third image; wherein the fourth image is synthesized by the image processing module based on the preview images of frames 1 to j; the image processing module transmits the third image to the image generation module via the selection module, the image generation module converts the format of the third image and then sends it to the camera application for storage. Thus, the quick frame return module quickly responds to the photo request and synthesizes the third image based on the currently captured preview images for storage, thereby improving image output efficiency, reducing user waiting time, and enhancing the user's shooting experience.

[0027] In another possible implementation of the first aspect, at the third moment, if the first through i preview images have been extracted and transmitted frame by frame to the image processing module, the quick frame return module sends an image generation instruction to the selection module in response to the capture request. The selection module transmits the third image to the image generation module, which converts the format of the third image and then sends it to the camera application for storage. This allows for the rapid acquisition of a storable third image, improving image output efficiency.

[0028] In a possible implementation of the first aspect, the image capture method may further include: a fast frame return module receiving a flag sent by a camera application, the flag being sent when the camera application receives the third image; and the fast frame return module generating corresponding virtual preview images for p preview requests after the i-th preview request in response to the flag.

[0029] In a possible implementation manner of the first aspect, the image capturing method may further include: the fast frame return module sending an interrupt instruction to the automatic exposure module in response to a flag sent by the camera application, instructing the automatic exposure module to interrupt the exposure decision.

[0030] In a second aspect, the present application provides an electronic device comprising: a camera, one or more processors, and a memory, wherein the camera and the memory are respectively coupled to the processor; the memory stores one or more computer program codes, wherein the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device performs the following steps:

[0031] In response to a user's operation of turning on a preset shooting mode in a camera application, a shooting interface corresponding to the preset shooting mode is displayed; wherein the shooting interface corresponding to the preset shooting mode includes a shooting shutter; a first click operation of the user on the shooting shutter is received; at a first moment, a first image is displayed; wherein the first image is obtained by collecting the first frame of preview image from the camera based on the first preview request generated in response to the first click operation; the camera collects one frame of preview image based on each preview request; at a second moment, a second image is displayed; wherein the second image is obtained by synthesizing the first to k frames of preview image collected by the camera based on the first to k preview requests generated in response to the first click operation; the second moment is after the first moment; k>1, k is a positive integer; at a third moment, a second click operation of the user on the shooting shutter is received, and a third image is saved; wherein the third moment is after the second moment; at the third moment, the i+pth preview request has been generated in response to the first click operation, and the camera has collected the ith frame of preview image; the third image is obtained by synthesizing the preview images collected by the camera from the first to i frames; i>k, p is a positive integer.

[0032] In one possible implementation of the second aspect, when the computer instructions are executed by a processor, the electronic device further performs the following steps: at a third moment, the electronic device displays a fourth image; the fourth image is synthesized from the first to jth preview images captured by the camera based on the first to jth preview requests generated in response to the first click operation; k < j ≤ i, where j is a positive integer; if j < i, the third image is synthesized from the fourth image and a preview image after the jth frame; if j = i, the fourth image is the third image. In another possible implementation of the second aspect, j = i or j = i-1.

[0033] In one possible implementation of the second aspect, when the computer instructions are executed by a processor, the electronic device further performs the following steps: converting the format of the third image and saving the converted third image. In another possible implementation of the second aspect, the third image before the format conversion is in YUV format, and the third image after the format conversion is in JPEG format.

[0034] In a possible implementation of the second aspect, when the computer instructions are executed by a processor, the electronic device further performs the following steps: synthesizing a fifth image and the kth preview image to obtain a second image, wherein the fifth image is synthesized based on the preview images of frames 1 to k-1.

[0035] In a possible implementation of the second aspect, when the computer instructions are executed by a processor, the electronic device further performs the following steps: synthesizing a sixth image and the i-th preview image to obtain a third image. The sixth image is synthesized based on the preview images of frames 1 to (i-1).

[0036] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: generating corresponding virtual preview images for p preview requests after the i-th preview request; wherein the virtual preview image includes virtual image data, image metadata, and a frame discard flag.

[0037] In a possible implementation manner of the second aspect, when the above computer instructions are executed by the processor, the electronic device further performs the following steps: after obtaining the third image, interrupting the exposure decision.

[0038] In one possible implementation of the second aspect, an electronic device includes an automatic exposure module, a frame output module, a quick frame return module, an image processing module, a selection module, and a display module. When the aforementioned computer instructions are executed by a processor, the electronic device further performs the following steps: a camera application continuously sends preview requests to the frame output module in response to a first click operation; the frame output module periodically sends preview requests to the camera according to the camera's exposure time, driving the camera to capture a frame of preview image; wherein the preview images are stored in a first image queue in the order in which they were captured; the exposure time is determined by the automatic exposure module based on ambient brightness and sent to the frame output module. Then, the quick frame return module extracts preview images frame by frame from the first image queue and transmits them to the image processing module, which synthesizes the first preview image to obtain a first image, and synthesizes the first to k preview images to obtain a second image; the image processing module transmits the first and second images to the camera application via the selection module and the display module, and the camera application displays the first and second images.

[0039] In a possible implementation of the second aspect, the electronic device further includes a picture generation module. When the computer instructions are executed by the processor, the electronic device further performs the following steps: the camera application, in response to a second click operation at a third moment, sends a picture-taking request to the quick frame return module via the frame output module; at the third moment, if the preview images after the jth frame among the preview images of the first to i frames have not been transmitted to the image processing module, the quick frame return module, in response to the picture-taking request, extracts the preview images after the jth frame and transmits them to the image processing module; k < j < i, where j is a positive integer; the image processing module synthesizes the preview images after the jth frame with a fourth image to obtain a third image; wherein the fourth image is synthesized by the image processing module based on the preview images of the first to j frames; the image processing module transmits the third image to the picture generation module via the selection module, and the picture generation module converts the format of the third image and sends it to the camera application for storage.

[0040] In one possible implementation of the second aspect, when the computer instructions are executed by a processor, the electronic device further performs the following steps: at a third moment, if the first through i preview images have been extracted frame by frame and transmitted to the image processing module, the quick frame return module sends an image generation instruction to the selection module in response to the photo capture request; the selection module transmits the third image to the image generation module, and the image generation module converts the format of the third image and then sends it to the camera application for storage. This allows for rapid acquisition of a storable third image, improving image output efficiency.

[0041] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: the fast frame return module receives a flag sent by the camera application, and the flag is sent when the camera application receives the third image; the fast frame return module responds to the flag and generates corresponding virtual preview images for p preview requests after the i-th preview request.

[0042] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: the fast frame return module responds to the flag sent by the camera application and sends an interrupt instruction to the automatic exposure module, instructing the automatic exposure module to interrupt the exposure decision.

[0043] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor in an electronic device, the electronic device executes the image capture method of the first aspect and any possible implementation thereof.

[0044] In a fourth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method of the first aspect and any possible implementation thereof. The computer may be the electronic device described above.

[0045] It can be understood that the beneficial effects that can be achieved by the electronic device of any possible implementation of the second aspect, the computer-readable storage medium of the third aspect, and the computer program product of the fourth aspect can be referred to as the beneficial effects in the first aspect and any possible implementation thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of a scenario in which a user triggers the opening of a streamer shutter according to an embodiment of the present application;

[0047] FIG2 is a schematic diagram of a star trail image provided in an embodiment of the present application;

[0048] FIG3 is a schematic diagram of a scenario in which a user triggers the activation of the gorgeous star track sub-mode according to an embodiment of the present application;

[0049] FIG4 is a schematic diagram of an interface for waiting for image output in a streamer shutter mode provided by an embodiment of the present application;

[0050] FIG5 is a flow chart of an image capturing method provided in an embodiment of the present application;

[0051] FIG6 is a block diagram of the software and hardware architecture of an electronic device provided in an embodiment of the present application;

[0052] FIG7 is a flowchart of another image capturing method provided in an embodiment of the present application;

[0053] FIG8 is a second flowchart of another image capturing method provided by an embodiment of the present application;

[0054] FIG9 is a flow chart of a conventional image capturing method provided in an embodiment of the present application;

[0055] FIG10 is a third flowchart of another image capturing method provided in an embodiment of the present application;

[0056] FIG11 is a fourth flowchart of another image capturing method provided by an embodiment of the present application;

[0057] FIG12 is a schematic structural diagram of an electronic device 1200 provided in an embodiment of the present application;

[0058] FIG13 is a schematic structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions of the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to limit the present application. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, if the words "first", "second" and the like are used to distinguish between the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit them to be different. And, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" means two or more.

[0060] With the development of electronic device image capture technology and the increasing demand for photography from users, existing electronic devices are now able to provide users with a growing number of image capture modes, such as portrait mode, panorama mode, time-lapse photography, night scene mode, slow motion, high dynamic range imaging (HDR), and light streamer shutter.

[0061] Lightstream Shutter, also known as long exposure, is a shooting mode that captures the movement of light and automatically extends the shutter time. Simply put, using Lightstream Shutter mode to capture the movement of light creates a streak of light.

[0062] As shown in Figure 1, taking a mobile phone 10 as an example, the embodiment of the present application shows a schematic diagram of a scenario in which a user triggers to open a light streamer shutter.

[0063] 1 , when the mobile phone 10 is powered on, a main interface 100 may be displayed. The main interface 100 may include application icons for applications such as "Clock," "Calendar," "Camera," "Gallery," "Memo," "File Management," "Email," "Music," "Settings," "Contacts," "Phone," and "Messages."

[0064] It is understandable that, according to actual needs, the main interface 100 of the mobile phone 10 may also include application icons of more applications, such as various third-party applications downloaded by the user, and the embodiment of the present application does not impose any limitation on this.

[0065] In response to the user clicking the “camera” application icon 101 in the main interface 100 , the mobile phone 10 enters the camera application and displays the shooting interface 102 .

[0066] The shooting interface 102 includes a plurality of shooting options. The shooting interface 102 includes shooting options such as "AI", "HDR", "Settings", "Aperture", "Night Scene", "Portrait", "Photo", "Video", "Professional" and "More".

[0067] In response to the user clicking on the “more” shooting option 103 in the shooting interface 102 , the mobile phone 10 enters the more shooting options interface 104 .

[0068] The more shooting options interface 104 includes other shooting options provided by the camera. The more shooting options interface 104 includes: "Micro Movie," "Time-Lapse," "Panorama," "Document Scan," "Watermark," and "Light Streamer." Similarly, the more shooting options interface 104 may include more shooting options, such as "High Pixel," "Multi-Camera Recording," "Super Macro," etc., depending on actual needs. This embodiment of the present application does not impose any limitations on this.

[0069] In response to the user clicking on the "Light Streamer Shutter" shooting option 105 in the more shooting options interface 104, the mobile phone 10 enters the Light Streamer Shutter shooting interface 106. In the Light Streamer Shutter shooting interface 106, the mobile phone 10 can trigger the mobile phone 10 to start image capture in response to the user clicking on the shooting shutter.

[0070] It should be noted that the scenario shown in Figure 1 of the embodiment of this application does not constitute a scenario limit for the user to trigger the light stream shutter. The user can also trigger the light stream shutter in many other ways. For example, the user can also enable the light stream shutter in the camera settings provided by "Settings", and the embodiment of this application does not impose any restrictions on this.

[0071] Additionally, the Light Stream shutter mode may offer multiple sub-modes to suit different shooting scenarios. For example, four sub-modes are available: Traffic Flow, Light Painting Graffiti, Silky Flowing Water, and Magnificent Star Trails. Users can select the appropriate sub-mode to capture the image based on the specific scene.

[0072] Among them, "Traffic Flow" is suitable for capturing traffic flow. "Light Painting Graffiti" is suitable for capturing light painting masterpieces. "Silk Flowing Water" is suitable for capturing all dynamic water elements, including flowing water, waves, and waterfalls. And "Gorgeous Star Trails" is suitable for capturing the starry sky. "Gorgeous Star Trails" allows you to clearly capture the movement of stars, also known as star trails.

[0073] For example, as shown in Figure 2, an embodiment of the present application shows a star trail image. The star trail image shown in Figure 2 was taken using the gorgeous star trail sub-mode and then processed in black and white. The white lines are the movement trajectories of the stars.

[0074] As shown in Figure 3, taking the mobile phone 10 shown in Figure 1 as an example, the embodiment of the present application shows a scenario schematic diagram of a user triggering the opening of the gorgeous star trails sub-mode. Below, the operation process of the user triggering the opening of the gorgeous star trails is described in conjunction with Figure 3.

[0075] After the user triggers the light streamer shutter module, the mobile phone 10 displays a light streamer shutter shooting interface 106. The light streamer shutter shooting interface 106 includes a sub-mode selection control 107.

[0076] In response to the user clicking the sub-mode selection control 107 , the mobile phone 10 enters the sub-mode selection interface 108 .

[0077] The sub-mode selection interface 108 includes various sub-modes provided by the streamer shutter. When a sub-mode is selected, the sub-mode selection interface 108 displays the name of the sub-mode. The sub-mode currently selected by default in the sub-mode selection interface 108 shown in FIG3 is "Traffic".

[0078] The mobile phone 10 switches sub-modes in response to the user's left swipe operation 109. For example, after the user selects the gorgeous star trails mode by swiping left, the user can stop swiping, completing the sub-mode switch and displaying the gorgeous star trails selection interface 110.

[0079] In response to the user clicking on a blank area of ​​the gorgeous star trail selection interface 110, the mobile phone 10 redisplays the streamer shutter shooting interface 106. In the streamer shutter shooting interface 106, the mobile phone 10 responds to the user's first click operation 111 on the shooting shutter to enter the shooting interface 112 and start image capture.

[0080] Understandably, since the mobile phone 10 has already switched from the busy traffic scene to the gorgeous star trails scene in response to the user's operation before the light stream shutter shooting interface 106 is redisplayed, when the image is captured in response to the user's first click operation 111 in the light stream shutter shooting interface 106, the mobile phone 10 will process the image according to the image processing method corresponding to the gorgeous star trails scene.

[0081] In addition, the shooting interface 112 displays the shooting timer "00:00" and the shooting precaution "Please keep the camera steady." The shooting timer "00:00" will increase in seconds as the shooting time passes, for example, "00:01," "00:02," "00:03," and so on. The shooting timer can be used to determine the current shooting duration. At the same time, the shooting shutter shows the shooting status in the shooting interface 112, and the currently captured image is displayed in real time in the viewfinder 114 for the user to preview.

[0082] When the user observes a satisfactory star trail effect in the viewfinder 114 , the user can perform a second click operation 113 on the shooting shutter that is in shooting state in the shooting interface 112 . The mobile phone 10 ends the image shooting in response to the user's second click operation 113 .

[0083] It should be noted that the scenario shown in FIG. 3 of the present embodiment does not constitute a scenario limit for users to trigger the activation of the gorgeous star trails. Users can also trigger the activation of the gorgeous star trails in more ways. For example, users can also directly click on the gorgeous star trails in the sub-mode selection interface 108 to switch the light streamer shutter sub-mode from the busy traffic mode to the gorgeous star trails mode. The present embodiment does not impose any restrictions on this.

[0084] According to the scenario shown in FIG3 , when the electronic device captures an image in the streamer shutter mode, it needs to respond to two clicks of the user on the shooting shutter to complete the capture.

[0085] Among them, the first click operation of the user on the shooting shutter responded by the electronic device, such as the first click operation 111 shown in Figure 3, is used to trigger the electronic device to start image capture (that is, the first click operation is used to trigger the electronic device to start taking pictures), and the electronic device processes the collected image data in real time and returns it to the camera application for display (that is, sending it for display), so that the user can preview the image shooting effect.

[0086] The electronic device responds to the user's second click operation on the shooting shutter, such as the second click operation 113 shown in Figure 3, which is used to trigger the electronic device to end image capture (that is, the second click operation is used to trigger the electronic device to end photo taking), and the electronic device processes the captured image and directly stores it in the gallery.

[0087] The user can trigger the electronic device to end the photo shooting at any time, and the embodiments of the present application do not impose any restrictions on this. Generally speaking, the user can observe the image effect of the displayed image in real time. If the image effect meets the expected shooting effect, the user can click the shutter button a second time to trigger the electronic device to end the photo shooting.

[0088] Simply put, in streamer shutter mode, the electronic device interprets the user's first click of the shutter button as a preview request in streamer shutter mode. The electronic device interprets the user's second click of the shutter button as a photo request. The difference between a preview request and a photo request is that the image captured for a preview request is processed and sent back to the camera app for display. In contrast, the image captured for a photo request is synthesized and not displayed, but is directly saved. For example, the image captured for a photo request is synthesized and directly stored in the gallery.

[0089] That is, when the streamer shutter mode is enabled, if the camera application in the electronic device receives the user's first click on the shutter button, the camera application will continuously send preview requests to the lower layer. In response to these preview requests, the lower layer will capture the corresponding images (which can be called preview images). The lower layer will then perform image processing on the captured preview images and return them to the upper layer camera application for display by the user.

[0090] If the camera application in the electronic device receives a second click operation on the shutter by the user, the camera application will not continue to send preview requests to the lower layer, but will respond to this second click operation and send a corresponding photo request to the lower layer. Furthermore, after the lower layer receives this photo request, it will also respond to this photo request to collect the corresponding image (which can be called a photo image). However, the difference is that after the lower layer processes the collected photo image and returns it to the upper layer, the upper layer will choose to save it directly, and the upper-layer camera application will not display this photo image. The photo image saved by the electronic device is the image obtained by shooting in the streamer shutter mode.

[0091] It should be noted that the preview in the streamer shutter mode is different from the normal preview (i.e., the preview image captured and displayed in the viewfinder after the camera is turned on). The image processing requirements for the normal preview are usually relatively low, but the preview in the streamer shutter mode is triggered by the user clicking the shutter button to enter the preview, so its image processing requirements depend on the image requirements of the streamer shutter mode. It can be understood that the image processing requirements for the preview in the streamer shutter mode are much higher than those for the normal preview. And, generally speaking, the image processing process corresponding to the preview request in the streamer shutter mode is usually the same as the image processing corresponding to the photo request in the streamer shutter mode. The only difference is whether it is sent for display.

[0092] In this way, it can be simply understood that the light streamer shutter obtains the motion trajectory of objects in the corresponding shooting scene through long-term continuous shooting, such as obtaining star trails by shooting the starry sky for a long time. Therefore, when the electronic device shoots images in the light streamer shutter mode, as the shooting time increases, the electronic device will collect more and more images. However, after the shooting is finished, the electronic device will only generate one image and store it in the gallery. Therefore, when shooting images in the light streamer shutter mode, the principle of image processing performed by the electronic device is to synthesize the collected images. That is, before the electronic device receives the second click operation of the user on the shooting shutter, the electronic device will continue to synthesize the collected images and display the obtained synthesized image. When the electronic device receives the second click operation of the user on the shooting shutter, the electronic device will directly save the synthesized image after image synthesis and no longer display it.

[0093] Specifically, after the electronic device turns on the streamer shutter mode in response to the user's turn-on operation, if the electronic device receives the user's first click operation on the shooting shutter, the electronic device starts taking pictures in response to the first click operation. During the shooting process, each time the electronic device acquires a frame of image (outgoing frame), it will synthesize the current frame of image with the previously acquired image, and fuse the image information of the previous image through pairwise synthesis of the images. At the same time, before receiving the user's second click operation on the shooting shutter, the electronic device will display the synthesized image obtained by image synthesis, so that the user can preview the effect of the image shooting in real time. If the user clicks the shooting shutter for the second time, the electronic device will save the latest synthesized synthetic image in response to the second click operation without displaying it, and end the image shooting.

[0094] For example, when an electronic device captures the first frame of image after starting to take a photo, it processes the first frame and displays it separately because it is the first frame without any previous image. When it captures the second frame of image, the electronic device synthesizes the second frame with the first frame to generate a new second frame of image. This new second frame of image incorporates the image information of the first frame of image. The electronic device then displays the new second frame of image.

[0095] Similarly, the third frame image needs to be synthesized with the new second frame image before being displayed. The fourth frame image, the fifth frame image, and the nth frame image are all synthesized in the same way.

[0096] However, the nth frame, the last captured frame, is the one triggered by the user to end the capture (i.e., the second click of the shutter button). It is also composited with the previous frame, but is not displayed and is instead stored directly in the gallery. That is, the first, second, ..., and n-1 frames are displayed, but the nth frame is not.

[0097] As a result, the third frame displayed will incorporate the image information of the first and second frames. The fourth frame displayed will incorporate the image information of the first, second, and third frames. This is true for all subsequent images and will not be repeated here. The image stored in the gallery is a fusion of the image information of all previous frames: the first, second, …, and n-1th frames. In other words, in streamer shutter mode, the electronic device continuously captures images over a long period of time, synthesizing all the images to create a single image with a motion trajectory.

[0098] However, the automatic exposure (AE) module in current electronic devices determines the exposure time based on the ambient brightness. As the ambient light gets darker, the exposure time determined by the AE module increases. In extremely dark environments, the exposure time determined by the AE module can be as long as 10 seconds. This means that in extremely dark environments, it may take up to 10 seconds for the electronic device to capture a single image frame.

[0099] So, for the streamer shutter mode, especially for scenes such as gorgeous star trails that require long-term shooting and usually need to be shot in a dark environment, when the electronic device receives the second click of the shutter by the user to trigger the end of shooting, it is very likely that the electronic device will still be producing some images (hereinafter referred to as frames being produced) because the frame output time is too long. This is because the bottom layer needs to respond to each request sent by the upper-level camera application and feedback the corresponding image. Therefore, when the electronic device receives the instruction from the user to trigger the end of shooting, the bottom layer may be responding to a preview request to collect images. At the same time, there may be some preview requests waiting for response processing at the bottom layer.

[0100] That is, when the electronic device receives the user's instruction to end the photo capture (the user's second click of the shutter button), the preview requests previously issued by the camera application may still be being processed. There may also be some preview requests that the camera application has already issued but has not yet responded to, resulting in some frames being produced.

[0101] Therefore, the frame being produced may be an image that has been partially but not completely acquired, or the frame being produced may be an image that has not been acquired at all.

[0102] According to the traditional processing method, after the electronic device receives the user's second click operation on the shutter, in order to ensure that all preview requests issued by the camera application are responded to and have corresponding images, the electronic device will wait for the frames being produced to be completed, and then respond to the photo request to generate the last frame of the photo composite image (that is, the composite image corresponding to the photo request) and store it in the gallery.

[0103] From the user's perspective, the shooting shutter on the shooting interface displayed by the electronic device will always be in a circling state, indicating that the electronic device is generating an image (outputting a picture). For example, as shown in Figure 4, an embodiment of the present application shows a schematic diagram of an interface for waiting for the output of a picture. Figure 4 is an interface diagram shown as an example of a user performing a second click operation 113 in the shooting interface 112 shown in Figure 3. That is, the mobile phone 10 enters the waiting interface 113 for outputting a picture in response to the user's second click operation 113 on the shooting shutter.

[0104] The circular outer ring of the shutter button in the image output waiting interface 113 is rotating. It should be noted that the arrow shown in FIG4 is a schematic illustration of the direction of the circular rotation of the shutter button in the embodiment of the present application and does not constitute a limitation on the image output waiting interface 113. In other words, the image output waiting interface 113 actually displayed by the electronic device may not have this arrow.

[0105] Also, the arrow in FIG4 indicates a clockwise rotation, which is also an example of an embodiment of the present application. Depending on actual needs, it can also be a counterclockwise rotation, and the embodiment of the present application does not impose any limitation on this.

[0106] For example, if the exposure time is 10 seconds and there are 5 frames currently being produced, the user will need to wait approximately 5*10 seconds = 50 seconds. In other words, from the user's perspective, the user will watch the circular rotation of the shutter ring for 50 seconds.

[0107] It is understandable that the 50 seconds are only used as an example in the embodiment of the present application. The specific waiting time depends on the exposure time of the actual decision and the number of frames being produced. The embodiment of the present application does not impose any restrictions on this.

[0108] Therefore, in traditional streamer shutter mode, after the electronic device receives the user's second click of the shutter button, it has to wait for the frame to be output, which makes the electronic device take a long time to output the image, resulting in low image output efficiency. At the same time, the user has to wait a long time before the captured image is available, which makes the user wait for a long time and leads to a poor user experience.

[0109] Based on this, in order to reduce the image output time in the streamer shutter mode, improve the image output efficiency, reduce the user's waiting time and improve the user experience, an embodiment of the present application provides an image shooting method.

[0110] The image capture method provided in the embodiments of the present application is applied to an electronic device. Furthermore, the electronic device may support a preset capture mode, which includes a star trail capture mode. That is, the preset capture mode may be used to capture star trails.

[0111] In some embodiments, the preset shooting mode can be a streamer shutter mode or a long exposure mode. It is understandable that based on the different customized naming of different manufacturers, the preset shooting mode can be the above-mentioned streamer shutter mode or the long exposure mode. Of course, based on the actual naming of the manufacturer, the preset shooting mode can also be named other ways, and the embodiments of the present application do not impose any restrictions on this.

[0112] To facilitate understanding and description of the solution, the following embodiments collectively refer to the preset shooting modes as streamer shutter modes.

[0113] In an embodiment of the present application, when the streamer shutter mode is on, if the electronic device receives an instruction from the user to end the photo shooting, that is, after the electronic device receives the second click operation of the user on the shooting shutter, the electronic device responds to this second click operation and directly uses the currently generated frame to generate an image and store it in the gallery.

[0114] In other words, the electronic device in the embodiment of the present application no longer waits for frames that are being produced to be completed, but instead directly abandons those frames that are being produced. As a result, the electronic device can avoid the long image production time caused by waiting for frames to be completed, thereby reducing image production time, improving image production efficiency, reducing user waiting time, and improving user experience.

[0115] It is understandable that although abandoning the frames being produced will cause the final generated image to lack the image information of the preview images, the frames being produced are usually the image information of the last few frames of preview images, so even if the last few frames of image information are missing, the impact on the final image effect is not significant.

[0116] In some embodiments, the preset shooting mode of the embodiments of the present application can also be used to shoot scenes such as bustling traffic, flowing water, and light painting graffiti. However, it should be noted that the shooting time of scenes such as bustling traffic and light painting graffiti is usually relatively short, so the number of images collected during the entire shooting process will not be too large, and the image output speed will not be very slow.

[0117] At the same time, silky flowing water is usually photographed during the day, so the ambient light is not too dark, and the corresponding exposure time is not too long. Therefore, if the image capture method provided in the embodiment of the application is applied to scenes such as busy traffic, silky flowing water, and light painting graffiti, although the image output efficiency can be improved by abandoning the frames being produced, the effect will not be as obvious as that of star trail photography.

[0118] Therefore, the image capture method provided in the embodiments of the present application can be configured based on actual business needs. That is, the image capture method provided in the embodiments of the present application can be configured to be applied only to the gorgeous star trails for star trails capture. It can also be configured to be applied to any one or more of the sub-modes of busy traffic, light painting graffiti, silky flowing water, and gorgeous star trails at the same time.

[0119] It can be understood that no matter which sub-mode is applied, the implementation principle is the same, and the electronic device can be configured according to actual conditions. The embodiments of the present application do not impose any limitations on this.

[0120] As shown in FIG5 , the embodiment of the present application shows a flowchart of an image capturing method, including steps S501 to S503 . The image capturing method provided by the embodiment of the present application will be described in detail below with reference to the flowchart shown in FIG5 .

[0121] S501: In response to a user's activation of a preset shooting mode in a camera application, the electronic device displays a shooting interface corresponding to the preset shooting mode, wherein the shooting interface corresponding to the preset shooting mode includes a shooting shutter.

[0122] The process of the electronic device in response to the user's opening operation to open the preset shooting mode in the embodiment of the present application can refer to the above description of the scenes shown in Figures 1 and 3. Among them, the shooting interface corresponding to the preset shooting mode can be the streamer shutter shooting interface 106 shown in Figure 3.

[0123] S502: The electronic device receives a first click operation on a shooting shutter by a user, displays a first image at a first moment, and displays a second image at a second moment.

[0124] After receiving the user's first click operation on the shooting shutter (e.g., first click operation 111 shown in FIG3 ), the electronic device calls the camera to start capturing images. It is understandable that since the first click operation in the streamer shutter mode corresponds to a preview request, the image that the electronic device calls the camera to start capturing in response to this first click operation can be regarded as a preview image in the streamer shutter mode.

[0125] Moreover, in the streamer shutter mode, before the electronic device receives the second click operation of the user on the shooting shutter (for example, the second click operation 113 shown in FIG3 ), the electronic device needs to continuously capture the preview image, so the electronic device will continuously generate multiple preview requests in response to the first click operation.

[0126] At the same time, the electronic device will call the camera to capture a preview image corresponding to each preview request. In other words, the number of preview images is the same as the number of preview requests. That is, the first preview request corresponds to the first preview image frame, the second preview request corresponds to the second preview image frame, the third preview request corresponds to the third preview image frame, and so on.

[0127] For the first preview image captured by the camera, since there is no previous preview image, the first preview image can be displayed directly after image processing. After the first preview image, the electronic device will synthesize each preview image captured by the camera with the previous preview image before displaying it.

[0128] That is, the electronic device may display a second image obtained by performing image synthesis at the second moment. The second image may be an image obtained by synthesizing the preview images of frames 1 to k, where k is greater than 1 and is a positive integer.

[0129] It is understood that the specific value of k depends on the number of preview images actually captured by the electronic device at the second moment. For example, if at the second moment the electronic device actually responds to three preview requests and captures three preview images, then k = 3. For another example, if at the second moment the electronic device actually responds to ten preview requests and captures ten preview images, then k = 10.

[0130] It is understandable that since the electronic device captures preview images in chronological order in response to each preview request, synthesizes the images, and then displays the images, the first image must be displayed before the second image, and the second moment in the embodiment of the present application must be after the first moment.

[0131] Specifically, if the electronic device captures the second preview image, k=2, the electronic device synthesizes the first preview image and the second preview image to obtain a second image 1, and displays the second image 1 at the second time instant 1.

[0132] If the electronic device captures the third preview image, k=3, the electronic device can synthesize the first, second, and third preview images to obtain a second image 2, and display the second image 2 at a second time 2.

[0133] Similarly, if the electronic device captures the kth preview image frame, it can synthesize the kth preview image frame to obtain a second image k-1. That is, the electronic device synthesizes the preview images 1 to k to obtain the second image k-1. Simultaneously, the electronic device can display the second image k-1 at the second time k-1. It is understood that the second images include second image 1, second image 2, and so on, to second image k-1.

[0134] It should be noted that since exposure time affects the speed of preview image acquisition, and the longer the exposure time, the longer it takes to capture each preview image frame. Therefore, at the second moment, the number of preview requests actually generated by the electronic device may be greater than the number of preview requests actually responded to by the electronic device. For example, at the second moment, the electronic device actually generated 14 preview requests, but was only able to respond to 10 preview requests and capture 10 preview frames.

[0135] S503: At a third moment, the electronic device receives a second click operation on the shooting shutter by the user and saves a third image.

[0136] The third moment is after the second moment, and the third moment may depend on the user. For example, the third moment may be the moment when the user observes a satisfactory image shooting effect in the second image displayed by the electronic device. For example, the user determines that the electronic device has currently captured a satisfactory star trail effect by observing the second image displayed by the electronic device. Then, the user may click the shutter button a second time at this moment (i.e., the third moment) to send an instruction to the electronic device to end the shooting.

[0137] Similarly, because exposure time affects the speed of preview image acquisition, the number of preview requests actually generated may be greater than the number of preview requests actually responded to. Therefore, at the third moment, the electronic device may have generated the i+pth preview request in response to the first click operation, but may have only actually processed the ith preview request.

[0138] That is, at the third moment, the electronic device may have generated a total of i+p preview requests, but the electronic device has only processed the i-th preview request. Accordingly, at the third moment, the electronic device should only use the camera to capture the i-th preview image frame. In other words, at the third moment, the electronic device has currently captured a total of i preview frames. At this point, the electronic device still has p preview requests unresponsive, meaning that the electronic device still has p preview frames that have not been captured by the camera.

[0139] However, at the third moment, after the electronic device receives the user's second click to take a photo, it indicates that the electronic device has received the user's instruction to trigger the end of the photo capture. According to the user's request, the electronic device now needs to respond to this second click to end the photo capture. According to traditional processing methods, the electronic device will continue to respond to these p preview requests. After the electronic device calls the camera to capture the p frames of preview images corresponding to these p preview requests, the electronic device will then synthesize the third image with the previously captured i frames of preview images to save it.

[0140] Therefore, the time it takes for a traditional electronic device to capture p preview frames in response to these p preview requests is the additional waiting time required for the image to be rendered, which is also the waiting time for the user. Longer exposure times increase this waiting time, reducing image rendering efficiency, causing users to wait for extended periods and a poor shooting experience.

[0141] In an embodiment of the present application, if the electronic device receives a second click operation at the third moment, since only i frames of images are collected at the third moment, the electronic device abandons the p frames of images corresponding to the p preview requests and directly synthesizes the existing i frames of preview images to obtain the third image for storage.

[0142] Therefore, the third image in the traditional processing method is an image synthesized from the 1st to i+p frame preview images. However, the third image in the embodiment of the present application is an image synthesized from the 1st to i frame preview images. It can be seen that although the third image in the embodiment of the present application lacks the image information of the p-frame preview image, the embodiment of the present application does not need to wait for the p-frame preview image to be framed, thereby speeding up the image output speed and reducing the waiting time. At the same time, for long-term shooting, the lack of the last p-frame image information has little effect on the image effect of the final third image. Therefore, compared with the traditional method, the embodiment of the present application can speed up the image output speed while ensuring the image effect, improve the image output efficiency, reduce the user's waiting time, and enhance the user's shooting experience.

[0143] It is understandable that the third image saved in this embodiment of the present application is the image generated when the user triggers the end of the photo capture. That is, since the second click operation in the streamer shutter mode corresponds to a photo capture request, the electronic device receiving this second click operation is equivalent to receiving a photo capture request issued by the user. Therefore, the third image saved is the image corresponding to the photo capture request.

[0144] In some embodiments, when the user triggers the end of the photo shooting instruction at the third moment, the preview images captured by the electronic device may have been synthesized and displayed in time due to differences in processing speeds of different threads or abnormal conditions, or the electronic device may still have some preview images that have been captured but have not yet been synthesized and displayed. For example, when the electronic device receives the second click operation at the third moment, the camera may have just captured one or more preview images, and these one or more preview images are preview images that have not yet been synthesized and displayed. Alternatively, due to an abnormal condition, the one or more preview images that were originally captured and could be displayed at the third moment are not successfully displayed at the third moment.

[0145] Then, at the third moment, the image displayed by the electronic device (hereinafter referred to as the fourth image) may be an image synthesized from the preview images of frames 1 to j. Here, k<j≤i, and j is a positive integer.

[0146] For the case where j < i, even though the electronic device captures the i-th preview image at the third moment, due to speed differences, the preview images after the j-th frame up to the i-th frame may all be preview images captured at the third moment. Therefore, in this case, the fourth image displayed at the third moment is a composite of preview images from frames 1 to j.

[0147] In this case, it's understandable that because the preview image frames after the jth frame are already captured, there's no need to wait for the electronic device to perform image synthesis. Therefore, to ensure image output efficiency while preserving as much preview image information as possible, the electronic device can synthesize the fourth image with the preview images after the jth frame to generate a third image. This third image is then saved.

[0148] For example, at the third moment, the electronic device captures the 5(i)th preview image. However, because the 4th and 5th preview images in these 5(i) frames are preview images that have just been captured at the third moment, the fourth image displayed by the electronic device at the third moment should be an image that only synthesizes the 1st to 3(j) preview images. Furthermore, after the electronic device receives the second click operation, since these two preview images do not need to wait for the frame to be output, the electronic device can synthesize this fourth image with the 4th and 5th preview images to obtain the third image.

[0149] In the case where j = i, the acquisition and display speeds are similar, and each preview frame captured is synthesized and displayed promptly. Therefore, in this case, the fourth image displayed at the third moment is a composite of the preview images from frames 1 to i. For example, at the third moment, the electronic device captures the 5(i)th preview frame. Because it is displayed promptly, the fourth image displayed at the third moment is likely a composite of the preview images from frames 1 to 5(j). That is, i = j.

[0150] In this case, when the user triggers the end of the photo shooting, all the preview images that the electronic device has already captured are successfully displayed. At this time, if the electronic device wants to synthesize the image with the newly captured preview image, it needs to wait for the camera to output the frame, which takes time. Therefore, in order to increase the image output speed and reduce the user's waiting time, the electronic device can directly save the fourth image as the third image. It can be understood that in this case, the third image saved by the electronic device is the fourth image.

[0151] Furthermore, although the fourth image is synthesized in response to a preview request, the image processing flow for a preview request in streamer shutter mode is generally the same as the image processing flow for a photo request in streamer shutter mode. Therefore, whether the fourth image is directly used as the third image for a photo request or resynthesized to obtain the third image for the photo request, the final image effect is the same. Directly using the fourth image as the third image also saves processing time, further improving image output efficiency and enhancing the user experience.

[0152] In other embodiments, given the processing capabilities of current electronic devices, the speed difference between capture and display is generally not significant. Therefore, even if there are preview images that are not displayed in time, the number will generally be small, most likely just a single frame. Therefore, j is most likely equal to i, or i-1. That is, in some embodiments, j = i or j = i-1.

[0153] In some embodiments, the images synthesized from preview image frames by electronic devices in streamer shutter mode are typically in YUV format. For example, the first image, second image, fourth image, and the third image synthesized by combining the preview image after the jth frame and the fourth image may all be in YUV format. Furthermore, electronic devices typically only display images in YUV format and do not directly save images in YUV format to the image library.

[0154] Therefore, to comply with the image storage format, embodiments of the present application may convert the third image in YUV format to a format acceptable to the electronic device's image library before storing it in the image library. In some embodiments, the third image may be converted to JPEG format after format conversion. That is, the third image in YUV format is converted to JPEG format before being saved.

[0155] In some embodiments, the electronic device may synthesize multiple preview images on a frame-by-frame basis. That is, after the first preview image is synthesized with the second preview image to produce second image 1, the third preview image can be synthesized with second image 1 to produce second image 2. The fourth preview image can be synthesized with second image 2 to produce second image 3. Similarly, the kth preview image can be synthesized with second image k-2 to produce second image k-1. The specific process is the same and will not be further described here.

[0156] Therefore, synthesizing the preview images of frames 1 to k captured by the camera to obtain the second image may include synthesizing the fifth image and the preview image of frame k to obtain the second image. In this embodiment of the present application, the fifth image may be synthesized based on the preview images of frames 1 to k-1.

[0157] Similarly, synthesizing the third image based on the preview images (frames 1 to i) captured by the camera can also include synthesizing the sixth image with the preview image (frame i) to obtain the third image. The sixth image is synthesized based on the preview images (frames 1 to i-1). The principle is the same as that of synthesizing the second image described above, with both synthesizing the images frame by frame cumulatively, and this will not be further described.

[0158] In some embodiments, the software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. TM Taking the system as an example, the software and hardware architecture of the electronic device is exemplified. FIG6 shows a structural block diagram of the software and hardware architecture of an electronic device.

[0159] As shown in Figure 6, the block diagram of the software and hardware architecture of an electronic device includes a software layered architecture and a hardware layer. The hardware layer of the electronic device includes hardware required for the electronic device, such as a camera, display, and speaker. It is understood that, depending on actual needs, the electronic device may also include more hardware, such as buttons, motors, and the like. Figure 6 of the embodiment of the present application does not limit the hardware structure of the electronic device.

[0160] The layered software architecture of electronic devices divides the software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer.

[0161] The application layer can include a series of application packages. As shown in Figure 6, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0162] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0163] As shown in FIG6 , the application framework layer may include a camera service, a window manager, an activity manager, an input manager, a resource manager, a notification manager, a view system, and a content provider, etc.

[0164] The camera service plays a connecting role, responding to requests from the camera application and sending them to the camera HAL to call the camera to capture images. Specifically in the embodiment of the present application, the camera service can respond to preview requests (preview requests generated by responding to the user's first click operation on the shooting shutter) and photo requests (photo request generated by responding to the user's second click operation on the shooting shutter) sent by the camera application, and send the preview requests and photo requests to the camera HAL to call the camera to capture images, thereby obtaining preview images corresponding to the preview requests (such as the first image, second image, and fourth image mentioned above) and photo images corresponding to the photo requests (such as the third image mentioned above).

[0165] The window manager provides window management services (WMS). WMS can be used for window management, window animation management, surface management, and as a transfer station for the input system.

[0166] The Activity Manager can provide Activity Management Service (AMS), which can be used to start, switch, and schedule system components (such as activities, services, content providers, and broadcast receivers) as well as manage and schedule application processes.

[0167] The Input Manager provides Input Management Service (IMS), which manages system inputs, such as touch screen input, key input, and sensor input. The IMS retrieves events from input device nodes and, through interaction with the WMS, distributes the events to the appropriate window.

[0168] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0169] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0170] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0171] Content providers are used to store and retrieve data and make it accessible to applications. This data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0172] The hardware abstraction layer (HAL) runs in user space, encapsulates kernel drivers, and provides a calling interface to the upper layer. As shown in Figure 6, the HAL can include the camera HAL, display HAL, audio HAL, and Bluetooth HAL.

[0173] The kernel layer is the layer between hardware and software. As shown in Figure 6, the kernel layer may include camera driver, display driver, audio driver, and Bluetooth driver.

[0174] Taking the hardware and software architecture of the electronic device shown in FIG6 as an example, FIG7 and FIG8 of the embodiment of the present application respectively show flow charts of an image capturing method.

[0175] The image capturing method provided in the embodiment of the present application is described in detail below with reference to FIG. 7 and FIG. 8 .

[0176] When the light streamer shutter mode is enabled, the first click of the shutter button by the user triggers the electronic device to start taking a photo. That is, after the camera application in the application layer receives the first click of the shutter button by the user, it begins to continuously send preview requests to the camera service in the application framework layer.

[0177] In the embodiment of the present application, the requests sent by the camera application are maintained in a request queue that follows a first-in-first-out rule in the order of sending time.

[0178] In the request queues shown in Figures 7 and 8, preview request 1 (the first preview request) is the first preview request to be queued, preview request 2 (the second preview request) is the second preview request to be queued, ..., preview request k (the kth preview request) is the kth preview request to be queued, ..., preview request i (the ith preview request) is the ith preview request to be queued, ..., preview request i+p (the i+pth preview request) is the i+pth preview request to be queued. The order in which the preview requests are queued corresponds to their numbering order and will not be detailed here. Then, according to the first-in-first-out (FIFO) feature of the queue, each preview request can be dequeued after the camera service completes processing it.

[0179] After receiving preview requests from the camera application, the camera service sends them to the camera HAL. The frame output module in the camera HAL maintains these preview requests. The camera HAL also includes an automatic exposure module, which determines the exposure time. The frame output module also receives the exposure time determined by the automatic exposure module.

[0180] In some embodiments, the automatic exposure module can also determine exposure parameters such as the amount of light entering and the sensitivity, which is not limited in this embodiment of the present application. That is, when the automatic exposure module transmits the exposure time to the frame output module, it can also transmit the exposure parameters such as the amount of light entering and the sensitivity.

[0181] After receiving the preview request from the upper layer and the exposure time determined by the automatic exposure module, the frame output module sends a preview request to the lower layer camera driver according to the exposure time determined by the automatic exposure module. The camera driver calls the camera to capture the preview image corresponding to these preview requests.

[0182] As shown in Figures 7 and 8, the exposure time determined by the automatic exposure mode is 10 seconds (i.e., 10s). Therefore, the frame output module calls the camera to capture the preview image corresponding to the preview request at 10-second intervals. This means that the frame output module sends a preview request to the camera driver every 10 seconds to drive the camera to capture the preview image. That is, the interval between preview request 1 and preview request 2 is 10 seconds, and the interval between preview request 2 and preview request 3 is also 10 seconds. The interval between subsequent preview requests is also 10 seconds, which will not be repeated here.

[0183] After receiving the preview request from the frame output module, the camera driver responds to the preview request and drives the camera to capture a preview image frame every 10 seconds and returns it to the camera driver. The preview image in the embodiment of the present application is stored in the first image queue.

[0184] As shown in Figures 7 and 8, since there is a 10-second interval between Preview Request 1 (the first preview request) and Preview Request 2 (the second preview request), there is also a 10-second interval between Preview Image 1 (the first preview image frame) and Preview Image 2 (the second preview image frame) in the first image queue. Similarly, there is also a 10-second interval between Preview Image 2 (the second preview image frame) and Preview Image 3 (the third preview image frame) in the first image queue. The intervals between subsequent preview images are also 10 seconds, which will not be further explained here.

[0185] In an embodiment of the present application, after the camera driver receives the preview image returned by the camera capture, the fast frame return module extracts the preview image frame by frame from the first image queue and transmits it to the image processing module. The image processing module then performs image synthesis on the preview image transmitted by the fast frame return module, obtains a corresponding synthesized image (such as the second image), and saves it to the second image queue. In some embodiments, the preview image in the first image queue can be extracted from the queue by the fast frame return module, or it can be removed from the queue after the queue is full and a new preview image is added to the queue.

[0186] As can be understood, since Preview Image 1 is the first preview image frame captured by the camera after the first click operation, and there is no preceding image to synthesize, Preview Image 1 is processed separately by the image processing module to obtain the first image. In other words, the image content of the first image is the same as that of Preview Image 1 (the first preview image frame).

[0187] Subsequently, after the camera captures preview image 2 (the second frame preview image), the fast frame return module transmits preview image 2 (the second frame preview image) to the image processing module, and the image processing module synthesizes preview image 2 (the second frame preview image) with the first image to obtain the second image 1.

[0188] Similarly, the fast frame rewind module transmits preview image 3 (the third preview image) to the image processing module, which synthesizes preview image 3 (the third preview image) with second image 1 to produce second image 2. The fast frame rewind module transmits preview image 4 (the fourth preview image) to the image processing module, which synthesizes preview image 4 with second image 2 to produce second image 3. Subsequent images are synthesized using the same synthesis method and will not be further described here.

[0189] It can be seen from this that the image displayed for preview request 1 (the first preview request) is the first image, the image displayed for preview request 2 (the second preview request) is the second image 1, the image displayed for preview request 3 (the third preview request) is the second image 2..., the image displayed for preview request k (the kth preview request) is the second image k-1..., the image displayed for preview request i (the i-th preview request) is the second image i-1...

[0190] The image processing module then transmits the first and second images in the second image queue to the selection module. The selection module then selects whether to display the first and second images or to send them to the image generation module for format conversion and storage in the image library. In this embodiment of the present application, images corresponding to preview requests are displayed, while images corresponding to photo requests do not need to be displayed.

[0191] Therefore, when the images transmitted by the image processing module are the first image and the second image corresponding to the preview request, the selection module will return the first image and the second image to the upper-layer camera application via the display module for display.

[0192] Additionally, if the image transmitted by the image processing module is the third image corresponding to the photo request, the selection module transmits this third image to the image generation module for format conversion before returning it to the upper layer for storage in the image library. In some embodiments, storage in the image library can be returned to the camera application, which then stores the image in the image library. Alternatively, the image can be directly returned to the image library application for storage in the image library. The embodiments of this application illustrate the process of returning the image to the image library application for storage in the image library.

[0193] Then, after the user observes a satisfactory image effect through the second image displayed by the camera application, he or she may click the shutter button a second time (eg, the second click operation 113 shown in FIG. 3 ), triggering the electronic device to end taking the photo.

[0194] That is, after the camera app receives the second click of the user's shutter button, it begins sending the corresponding photo request to the underlying camera service. Understandably, because the second click triggers the end of the photo capture, the photo request sent by the camera app is the last request in the request queue.

[0195] After receiving the photo request from the camera application, the camera service also sends the photo request to the frame output module in the camera HAL. At this time, because the exposure time is as long as 10 seconds, there may be unprocessed preview requests before the photo request.

[0196] As shown in Figures 7 and 8, the preview requests indicated by dashed lines in the request queue are preview requests that were not fully processed before the photo request. An unprocessed preview request can be understood as one for which the camera has not yet completed capturing the corresponding preview image. That is, in this embodiment of the present application, as long as the camera has not captured the preview image corresponding to the preview request, the preview request is considered to be unprocessed.

[0197] Correspondingly, the images represented by dotted lines in the first image queue are the preview images corresponding one-to-one to these unprocessed preview requests, that is, the frames being produced in the embodiment of the present application.

[0198] It's important to note that unprocessed preview requests are actually sent by the upper-layer camera application, but are still queued for processing by the lower layer. Therefore, these preview requests are actually included in the request queue. However, the frames being produced are images for which data has not yet been captured, so the images represented by the dashed lines should not be in the first or second image queues.

[0199] It is understood that the images shown in dashed lines in Figures 7 and 8 are for ease of understanding and description of the solution and do not indicate that these images actually exist in the first image queue or the second image queue. However, the preview requests represented by dashed lines are actual requests that have not yet been responded to and processed.

[0200] According to the traditional processing method, after the upper-layer camera application sends a photo request, the photo request will wait in the request queue for processing in sequence. Figure 9 shows a flow chart of a traditional image capture method.

[0201] As shown in Figure 9, after the camera captures an image, the camera driver directly transmits it to the image processing module. Therefore, if there are still unprocessed preview requests after the user triggers the end of the photo, the processing of this photo request will not start until the unprocessed preview requests before the photo request are processed.

[0202] That is, as shown in Figure 9, after the camera application sends a photo request, the camera has not yet captured the p frames of preview images following preview image i, indicating that the p preview requests following preview request i are all preview requests that have not yet been processed. Therefore, the traditional method requires waiting for the camera to capture the p frames of preview images following preview request i before it indicates that the p preview requests following preview request i have been processed. Furthermore, only after all the p preview requests following preview request i have been processed can the electronic device begin to process the photo request. In other words, according to the chronological order of the preview requests, the traditional method requires waiting until preview request i+p (the i+pth preview request) has also been processed before responding to and processing the photo request.

[0203] Therefore, the more preview requests that have not been processed before the photo request, that is, the more preview requests that follow preview request i, and the larger p is, the longer it will take for the electronic device to produce the image, the lower the efficiency of the image production will be, and the longer the user will have to wait for the image to be produced. That is, as shown in Figure 9, after the camera application sends a photo request, it must wait until the camera captures the i+pth preview image (preview image i+p) before it can capture the photo image corresponding to the photo request, perform image synthesis to obtain the third image, and return it to the gallery.

[0204] However, in this embodiment of the present application, in order to improve image output efficiency and reduce user waiting time, after the lower layer receives the photo request sent by the camera application, the frame output module sends the photo request to the fast frame return module. The fast frame return module quickly responds to the photo request of the upper layer camera application.

[0205] In an embodiment of the present application, after receiving a photo request, the fast frame return module immediately responds to the photo request and determines whether there are any preview images in the first image queue that have not yet been displayed. Given the current processing speed of electronic devices, if there are any preview images in the first image queue that have not yet been displayed, they are usually only one frame. This preview image is usually the most recently captured preview image by the camera. In other words, it is the last preview image in the first image queue. Therefore, in some embodiments, the fast frame return module can directly extract the last preview image in the first image queue.

[0206] As shown in Figures 7 and 8, since the preview images after preview image i (the i-th preview image) in the first image queue are all represented by dotted lines, meaning there are no actual preview images, the last preview image is preview image i (the i-th preview image). Therefore, the fast frame return module can directly retrieve preview image i (the i-th preview image).

[0207] If the last preview image in the first image queue is an image that has not yet been displayed (i.e., a preview image that has been captured by the camera but has not yet been displayed), the fast frame rewind module transmits this last preview image and the photo request to the image processing module. The image processing module then synthesizes the last preview image with the previous preview image to generate a third image. The image processing module then sends this synthesized third image to the selection module.

[0208] After the selection module receives the third image from the image processing module, since this third image is the composite image corresponding to the photo request, it sends it to the image generation module. The image generation module converts the third image format, returns the converted third image to the application, and saves it in the image library.

[0209] It's understandable that the third image stored in the gallery is the image captured by the user using the streamer shutter mode. For example, the third image could be the star trail image shown in Figure 2. As you can understand, since Figure 2 is a star trail image that has been post-processed to black and white, if the third image stored in the gallery hasn't been post-processed to black and white, then the third image in the gallery could be a color star trail image.

[0210] Referring to Figure 7, the last preview image is preview image i, but preview image i (the i-th preview image) is the latest image captured by the camera and has not yet been sent to the image processing module for image synthesis and display. At this time, preview image i (the i-th preview image) is a preview image that has already been framed and does not need to wait for additional frame output. Therefore, in order to quickly output the image while ensuring the integration of more image information, the fast frame return module can extract preview image i (the i-th preview image) and send it to the image processing module for image synthesis to obtain the corresponding second image i-1. Then, the image processing module transmits this second image i-1 to the selection module. Since the second image i-1 is the image corresponding to the photo request, the selection module passes the second image i-1 to the picture generation module for format conversion and returns it to the upper layer for storage in the library.

[0211] If the last preview image in the first image queue has already been transmitted to the image processing module for processing, then this last preview image has already been synthesized and displayed (i.e., there is no preview image that has been captured by the camera but has not yet been displayed). In other words, all preview images in the first image queue have been synthesized and displayed in a timely manner, indicating that there are currently no preview images that can be synthesized without wasting time waiting.

[0212] Furthermore, because the selection module determines whether the second image synthesized by the image processing module is displayed or stored in the image library, the selection module typically retains the most recent second image. Therefore, to quickly output the image, the fast frame return module can directly send an image generation instruction to the selection module. The selection module responds to the image generation instruction and directly sends the second image, which has just been transmitted from the image processing module, to the image generation module for format conversion before returning it to the application layer for storage in the image library.

[0213] Referring to Figure 8 , the last preview image is preview image i (the i-th preview image), and its corresponding composite image is second image i-1. This second image i-1 is the most recently displayed second image received by the selection module from the image processing module. In response to the image generation instruction, the selection module directly sends this second image i-1 to the image generation module. The image generation module then converts the format and stores it in the image library.

[0214] In summary, comparing Figures 7, 8, and 9, we can see that in the present embodiment, regardless of the situations in Figures 7 and 8, the second image i-1 will be pre-evaluated as the third image corresponding to the photo request. Therefore, the present embodiment eliminates the need to waste a significant amount of time, as in the conventional method shown in Figure 9, waiting for preview images i+1, i+2, and i+p to be framed before synthesizing the third image corresponding to the photo request.

[0215] That is to say, the embodiment of the present application is different from the traditional method shown in Figure 9 in that after the camera application sends a photo request, the embodiment of the present application directly gives up waiting for the preview image after preview image i (the i-th frame preview image) to be framed out.

[0216] Therefore, compared with traditional shooting methods, the embodiments of the present application can achieve fast image output. The image output speed will be faster, thereby improving image output efficiency, reducing user waiting time, and improving user experience.

[0217] In summary, in an embodiment of the present application, once the camera application issues a photo request, the quick frame return module will quickly respond to the photo request, instead of waiting for the photo request to be processed in sequence as in the traditional way. The embodiment of the present application uses the quick frame return module to process the photo request in advance, abandons the frame being produced, and directly sends the produced frame (that is, the preview image that the camera has collected) to the picture generation module for format conversion to obtain a third image that can be stored in the gallery. As shown in Figures 7-8, waiting for the camera to collect preview image i+1, preview image i+2,..., preview image i+p is abandoned. In this way, the embodiment of the present application can reduce the image output time and achieve fast image output, thereby reducing the user's waiting time to improve the user experience.

[0218] In some embodiments, in addition to image synthesis, the image processing module can also perform image processing such as image compensation, image correction, and image denoising on the image, which is not limited in any way in the embodiments of the present application.

[0219] As shown in FIG. 10 and FIG. 11 , an embodiment of the present application shows a flow chart of another image capturing method.

[0220] Since each request issued by the camera application requires a corresponding image to be returned, if the frame being produced is directly discarded without feedback to the upper layer, some preview requests issued by the camera application will not have corresponding returned images, which may easily cause other unnecessary problems. Therefore, in order to avoid other unnecessary problems caused by the preview request having no corresponding returned image, the embodiment of the present application fills virtual data and a frame abandonment flag for those abandoned frames being produced and returns them to the upper-level camera application. Among them, the virtual data includes virtual image data (buffer) and image metadata (metadata). The virtual image data and image metadata ensure that this frame of image can be smoothly returned to the upper layer.

[0221] At the same time, after the camera application receives this frame of image, the camera application can determine that the frame of image is an erroneous frame according to the frame abandonment flag, that is, an image that does not need to be displayed, so the camera application directly abandons displaying the frame of image.

[0222] Specifically, in the embodiment of the present application, after the upper-layer gallery application receives the third image returned by the lower layer (e.g., the second image i-1 in Figures 7 and 8, i.e., the third image in the embodiment of the present application), the gallery application (or the camera application, shown as the gallery application) sends a flag to the fast frame return module. The gallery application uses the sent flag to provide feedback to the fast frame return module that it has received the third image.

[0223] In response to this flag, the fast frame return module generates a corresponding virtual preview image for each unprocessed preview request. These virtual preview images are generated by filling in virtual data, namely virtual image data and image metadata. They also include a frame discard flag.

[0224] For example, the p preview requests following preview request i shown in Figure 10 are unprocessed preview requests. That is, the camera has not yet captured the p preview images following preview image i (i.e., preview image i+1, preview image i+2, ..., preview image i+p). Therefore, the fast frame return module, in response to the flag bit, generates a corresponding virtual preview image for each of the p preview requests, thereby ensuring that each of the p preview requests issued by the camera application has a corresponding returned image.

[0225] At the same time, the virtual preview images all carry a discard frame flag. Further, after the camera application receives the virtual preview images corresponding to the p preview requests, it can discard and not display these virtual preview images to prevent displaying error frames.

[0226] That is, in this embodiment of the present application, because the fast frame return module responds to the photo request in advance, the image capture is complete. However, in the actual capture process, there are still unprocessed preview requests (i.e., p preview requests after preview request i), and the capture is not actually completed. Therefore, if the capture is not further completed, it may cause an abnormal capture process.

[0227] Therefore, in this embodiment of the present application, a flag sent from an upper layer is used to instruct the fast frame return module to complete the capture process to prevent abnormalities in the capture process. Therefore, after the fast frame return module receives the flag sent from the upper layer, it responds to this flag and begins to complete the capture process, namely, generating a virtual preview image filled with virtual image data, image metadata, and a frame discard flag for unprocessed preview requests.

[0228] Among them, the virtual image data and image metadata filled in can be configured according to actual needs, and the embodiments of the present application do not impose any restrictions on them. The frame abandonment flag can also be agreed upon according to actual needs. For example, an error flag can be added to the virtual preview image to indicate that this image frame is an error frame. In this way, not only can it be ensured that each preview request corresponds to a return image to the upper layer, but the frame abandonment flag can also be used to inform the upper layer not to perform any display processing on the image, thereby avoiding display errors.

[0229] In some embodiments, after the image capture process begins, the automatic exposure module also determines the exposure parameters for the capture as required. Therefore, the finalization of the fast frame return module should also include interrupting the exposure decision work of the automatic exposure module.

[0230] Therefore, as shown in Figures 10 and 11, after the quick frame return module receives the flag sent by the gallery application (or camera application), in order to prevent the automatic exposure module from continuing to determine the exposure time for this image capture, the quick frame return module responds to this flag and simultaneously sends an interrupt instruction to the automatic exposure module. The quick frame return module interrupts the automatic exposure module through the interrupt instruction and reclaims the corresponding resources. At the same time, it aborts and discards the frames captured by the camera. This ensures that no abnormalities occur after the image capture ends prematurely, causing other unnecessary problems.

[0231] In the embodiment of the present application, the electronic device may include at least one of a mobile phone, a foldable electronic device (e.g., a foldable mobile phone), a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device, a camera, a video camera, a video recorder, or a smart city device. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0232] As shown in FIG12 , an embodiment of the present application shows a schematic structural diagram of an electronic device 1200 .

[0233] The electronic device 1200 may include a processor 1210, an external memory interface 1220, an internal memory 1221, a universal serial bus (USB) connector 1230, a charging management module 1240, a power management module 1241, a battery 1242, an antenna 1201, an antenna 1202, a mobile communication module 1250, a wireless communication module 1260, an audio module 1270, a speaker 1270A, a receiver 1270B, a microphone 1270C, an earphone jack 1270D, a sensor module 1280, a button 1290, a motor 1291, an indicator 1292, a camera module 1293, a display 1294, and a subscriber identification module (SIM) card interface 1295. The sensor module 1280 may include a pressure sensor 1280A and a touch sensor 1280B.

[0234] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 1200. In other embodiments of the present application, the electronic device 1200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0235] The processor 1210 may include one or more processing units. For example, the processor 1210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0236] The processor can generate an operation control signal based on the instruction operation code and the timing signal to complete the control of instruction fetching and execution. For example, the image capture method described in the embodiment of the present application can be implemented by the processor 1210.

[0237] Processor 1210 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 1210 may be a cache memory. This memory can store instructions or data that have been used or are frequently used by processor 1210. When processor 1210 needs to use the instruction or data, it can directly access it from this memory. This avoids duplicate accesses, reduces processor 1210 latency, and thus improves system efficiency.

[0238] In some embodiments, the processor 1210 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface. The processor 1210 may be connected to modules such as a touch sensor, an audio module, a wireless communication module, a display screen, and a camera module through at least one of the above interfaces.

[0239] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 1200. In other embodiments of the present application, the electronic device 1200 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0240] External memory interface 1220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of electronic device 1200. The external memory card communicates with processor 1210 via external memory interface 1220 to implement data storage. For example, files such as music and videos can be saved on the external memory card or transferred from the electronic device to the external memory card.

[0241] The internal memory 1221 can be used to store computer executable program code, which includes instructions. The internal memory 1221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 1200 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 1221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 1210 executes various functional methods or data processing of the electronic device 1200 by running instructions stored in the internal memory 1221, and / or instructions stored in a memory provided in the processor.

[0242] The electronic device 1200 can implement a display function through a GPU, a display screen 1294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 1294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 1210 may include one or more GPUs that execute program instructions to generate or change display information. For example, the star trail image generated by the embodiment of the present application (such as the first preview image and the second image stored in the gallery) can be rendered using the GPU and displayed using the display screen 1294.

[0243] Display screen 1294 is used to display images, videos, etc. In some embodiments, it can be used to display captured star trail images. Display screen 1294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (QLED), etc. In some embodiments, electronic device 1200 may include one or more display screens 1294.

[0244] The electronic device 1200 can implement a camera function through a camera module 1293, an ISP, a video codec, a GPU, a display screen 1294, an application processor AP, a neural network processor NPU, etc. For example, the camera module 1293 and the ISP can be used to capture and generate star trail images (such as the first preview image and the second image stored in the gallery) in the embodiment of the present application.

[0245] The camera module 1293 can be used to collect color image data and depth data of the subject. The ISP can be used to process the color image data collected by the camera module 1293. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element (i.e., image sensor). The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene.

[0246] In some embodiments, the ISP may be provided in the camera module 1293 .

[0247] In some embodiments, the camera module 1293 may be composed of a color camera module and a 3D sensing module.

[0248] In some embodiments, the photosensitive element of the camera of the color camera module can be a charge coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV.

[0249] In some embodiments, the 3D sensing module may be a time of flight (TOF) 3D sensing module or a structured light 3D sensing module. Structured light 3D sensing is an active depth sensing technology, and its basic components may include an infrared emitter, an IR camera module, and the like. The operating principle of a structured light 3D sensing module is to first emit a specific pattern of light spots at the object being photographed, then receive the light coding on the object's surface, compare the light pattern with the original projected light spot, and calculate the object's three-dimensional coordinates using triangulation. These three-dimensional coordinates include the distance between the electronic device 1200 and the object being photographed. TOF 3D sensing can be an active depth sensing technology, and its basic components may include an infrared emitter, an IR camera module, and the like. The operating principle of a TOF 3D sensing module is to calculate the distance (i.e., depth) between the TOF 3D sensing module and the object being photographed based on the time it takes for the infrared light to return, thereby generating a 3D depth map.

[0250] Structured light 3D sensing modules can also be used in facial recognition, motion-sensing game consoles, industrial machine vision inspection, and other fields. Time of Flight 3D sensing modules can also be used in game consoles, augmented reality (AR) and virtual reality (VR).

[0251] In other embodiments, the camera module 1293 may also be composed of two or more cameras. The two or more cameras may include a color camera, which can be used to collect color image data of the photographed object. The two or more cameras may use stereo vision technology to collect depth data of the photographed object. Stereo vision technology is based on the principle of human eye parallax. Under natural light, two or more cameras are used to capture images of the same object from different angles, and then triangulation and other calculations are performed to obtain the distance information between the electronic device 1200 and the object, that is, depth information.

[0252] In some embodiments, the electronic device 1200 may include one or more camera modules 1293. Specifically, the electronic device 1200 may include one front camera module 1293 and one rear camera module 1293. The front camera module 1293 may generally be used to capture color image data and depth data of the photographer facing the display screen 1294, while the rear camera module may be used to capture color image data and depth data of the subject (e.g., star trails) the photographer is facing.

[0253] In some embodiments, the CPU, GPU, or NPU in the processor 1210 can process the color image data and depth data collected by the camera module 1293. In some embodiments, the NPU can identify the color image data collected by the camera module 1293 (specifically, the color camera module) through a neural network algorithm based on skeleton point recognition technology, such as a convolutional neural network algorithm (CNN), to determine the skeleton points of the photographed person. The CPU or GPU can also run a neural network algorithm to determine the skeleton points of the photographed person based on the color image data.

[0254] The digital signal processor is used to process digital signals and can also process other digital signals. For example, when the electronic device 1200 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0255] Video codecs are used to compress or decompress digital video. Electronic device 1200 may support one or more video codecs. This allows electronic device 1200 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0256] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 1200, such as image recognition, face recognition, speech recognition, and text comprehension.

[0257] Pressure sensor 1280A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 1280A can be provided on display screen 1294. There are many types of pressure sensors 1280A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force is applied to pressure sensor 1280A, the capacitance between the electrodes changes. Electronic device 1200 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to display screen 1294, electronic device 1200 detects the intensity of the touch operation based on pressure sensor 1280A. Electronic device 1200 can also calculate the location of the touch based on the detection signal from pressure sensor 1280A. In some embodiments, touch operations applied to the same touch location but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to a first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0258] Touch sensor 1280B, also known as a "touch device," can be provided on display screen 1294. Touch sensor 1280B and display screen 1294 form a touch screen, also known as a "touch screen." Touch sensor 1280K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operation to an application processor to determine the type of touch event. Visual output related to the touch operation can be provided via display screen 1294. In other embodiments, touch sensor 1280K can also be provided on the surface of electronic device 1200, at a location different from that of display screen 1294.

[0259] Specifically in the embodiment of the present application, the electronic device 1200 can detect the user's click operation on the shooting shutter and the user's opening operation of opening the camera application through the pressure sensor 1280A and the touch sensor 1280B.

[0260] USB receiver 1230 is an interface compliant with USB standards and can be used to connect electronic device 1200 to peripheral devices. Specifically, it can be a Mini USB connector, Micro USB connector, USB Type-C connector, etc. Charging management module 1240 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. Power management module 1241 is used to connect battery 1242, charging management module 1240, and processor 1210.

[0261] The wireless communication function of the electronic device 1200 can be implemented through the antenna 1201, the antenna 1202, the mobile communication module 1250, the wireless communication module 1260, the modem processor and the baseband processor.

[0262] The electronic device 1200 can implement audio functions such as music playback and recording through the audio module 1270 , the speaker 1270A, the receiver 1270B, the microphone 1270C, the headphone jack 1270D, and the application processor.

[0263] Buttons 1290 may include a power button, volume buttons, etc. Motor 1291 may generate vibration prompts. Indicator 1292 may be an indicator light that can be used to indicate charging status, power level changes, messages, missed calls, notifications, etc. SIM card interface 1295 is used to connect a SIM card.

[0264] It should be noted that the image capturing methods in the following embodiments can all be implemented in the electronic device 1200 having the above hardware structure.

[0265] Another embodiment of the present application provides an electronic device comprising: a camera, one or more processors, and a memory. The camera and the memory are each coupled to the processor; the memory stores one or more computer program codes, each comprising computer instructions; when the processor executes the computer instructions, the electronic device implements the image capture method described in any of the above embodiments.

[0266] Another embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor in an electronic device, the electronic device implements the image capture method described in any of the above embodiments.

[0267] The embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the functions or steps in the above method embodiment.

[0268] The present application also provides a chip system, as shown in FIG13 . The chip system 1300 includes at least one processor 1301 and at least one interface circuit 1302. The processor 1301 and the interface circuit 1302 can be interconnected via a line. For example, the interface circuit 1302 can be used to receive signals from other devices (such as a computer memory). For another example, the interface circuit 1302 can be used to send signals to other devices (such as the processor 1301).

[0269] For example, the interface circuit 1302 can read instructions stored in the memory and send the instructions to the processor 1301. When the instructions are executed by the processor 1301, the computer can execute the various steps in the above embodiment. Of course, the chip system can also include other discrete devices, which are not specifically limited in this embodiment of the application.

[0270] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0271] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0272] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0273] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0274] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0275] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image capturing method, characterized in that, Applied to an electronic device, the electronic device including a camera; the method includes: Receiving a first click operation of the user on the shooting shutter; At a first moment, displaying a first image; wherein, the first image is obtained based on the first frame preview image collected by the camera, the first frame preview image is collected in response to the first preview request, and the first preview request is generated in response to the first click operation; the camera collects one frame of preview image in response to each preview request; At a second moment, displaying a second image; wherein, the second image is synthesized based on the first to k frame preview images collected by the camera, the first to k frame preview images are collected in response to the first to k preview requests, and the first to k preview requests are generated in response to the first click operation; the second moment is after the first moment; k>1, and k is a positive integer; At a third moment, receiving a second click operation of the user on the shooting shutter and saving a third image; wherein, the third moment is after the second moment; the third image is synthesized based on the first to i frame preview images collected by the camera, the first to i frame preview images are collected in response to the first to i preview requests, and the first to i preview requests are the first i preview requests among the first to i + p preview requests generated from the first moment to the third moment in response to the first click operation; i + p>i>k, and p is a positive integer.

2. The method according to claim 1, wherein The method further includes: At the third moment, the electronic device displays a fourth image; the fourth image is synthesized based on the first to j frame preview images, and the first to j frame preview images are collected in response to the first j preview requests among the first to i preview requests; k<j<i, and j is a positive integer; the third image is synthesized based on the fourth image and the j + 1 to the i frame preview images.

3. The method according to claim 2, wherein When j = i, the fourth image is the third image.

4. The method according to any one of claims 1-3, characterized in that The saving of the third image includes: Performing format conversion on the third image and saving the third image after format conversion; wherein, the third image before format conversion is in YUV format, and the third image after format conversion is in JPEG format.

5. The method according to any one of claims 1 to 4, characterized in that Before displaying the second image, the method further includes: Synthesizing a fifth image and the k frame preview image to obtain the second image; wherein, the fifth image is synthesized based on the first to k - 1 frame preview images.

6. The method according to any one of claims 1-5, characterized in that, Before saving the third image, the method further includes: Synthesizing a sixth image and the i frame preview image to obtain the third image; wherein, the sixth image is synthesized based on the first to i - 1 frame preview images.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Generating corresponding virtual preview images for the i + 1 to the p preview requests respectively; wherein, the virtual preview image includes virtual image data, image metadata, and a dropped frame flag, and the dropped frame flag is used to indicate that the corresponding virtual preview image is a droppable frame.

8. The method according to any one of claims 1-7, characterized in that, The electronic device includes an automatic exposure module, a frame output module, a fast frame return module, an image processing module, a selection module, and a display sending module; In response to the first click operation, the camera application continuously sends the preview request to the frame output module; The frame output module periodically sends the preview request to the camera according to the exposure time of the camera to drive the camera to capture a frame of preview image; wherein, the preview images are stored in the first image queue in the order of acquisition; the exposure time is determined by the automatic exposure module according to the ambient brightness and sent to the frame output module; The fast frame return module extracts the preview images from the first image queue frame by frame and transmits them to the image processing module. The image processing module synthesizes the first frame of preview image to obtain the first image, and the image processing module synthesizes the first to k frames of preview images to obtain the second image; The image processing module transmits the first image and the second image to the camera application via the selection module and the display transmission module, and the camera application displays the first image and the second image.

9. The method according to claim 8, characterized in that, The electronic device further includes a picture generation module; In response to the second click operation, the camera application sends a photographing request to the fast frame return module via the frame output module; At the third moment, in the case that the fast frame return module does not transmit the (j + 1)-th to the i-th frames of preview images among the first to i frames of preview images to the image processing module, the fast frame return module transmits the (j + 1)-th to the i-th frames of preview images to the image processing module in response to the photographing request; The image processing module synthesizes the (j + 1)-th to the i-th frames of preview images with the fourth image to obtain the third image; wherein, the fourth image is synthesized by the image processing module based on the first to j frames of preview images; The image processing module transmits the third image to the picture generation module via the selection module, and the picture generation module performs format conversion on the third image and then sends it to the camera application for saving.

10. The method according to claim 9, wherein The method further includes: At the third moment, in the case that the fast frame return module transmits the first to i frames of preview images to the image processing module, the fast frame return module then sends a picture generation instruction to the selection module in response to the photographing request; The selection module transmits the third image to the picture generation module, and the picture generation module sends the third image to the camera application for saving.

11. The method according to any one of claims 8 - 10, characterized in that, The method further includes: When the camera application receives the third image, a flag bit is sent to the fast frame return module; In response to the flag bit, the fast frame return module generates corresponding virtual preview images for the (i + 1)-th to the p-th preview requests; the virtual preview images include discard frame flags, and the discard frame flags are used to indicate that the corresponding virtual preview images can be discarded.

12. The method according to any one of claims 8-11, characterized in that, The method further includes: In response to the flag bit sent by the camera application, the fast frame return module sends an interrupt instruction to the automatic exposure module; the interrupt instruction is used to instruct the automatic exposure module to interrupt the exposure.

13. The method according to any one of claims 1 to 12, characterized in that, Before receiving the user's first click operation on the shooting shutter, the method further includes: In response to a user's operation of enabling a preset shooting mode within a camera application, display a shooting interface corresponding to the preset shooting mode; wherein, the shooting interface corresponding to the preset shooting mode includes a shooting shutter; the preset shooting mode includes at least one of a light painting shutter mode, a long exposure mode, and a star trail shooting mode. Including:

14. An electronic device, characterized in that, A camera, one or more processors and a memory, the camera and the memory are respectively coupled to the processor; The camera is used to collect images; One or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device executes the image shooting method according to any one of claims 1-13. When the computer program is executed by the processor of the electronic device, the electronic device executes the image shooting method according to any one of claims 1-13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, ​

Images