Image capturing method, electronic device, and computer-readable storage medium

Abstract

The application relates to the technical field of electronic equipment, and provides an image shooting method, electronic equipment and a computer readable storage medium. The method is applied to electronic equipment supporting a preset shooting mode, and comprises the following steps: when the electronic equipment shoots an image in the preset shooting mode, if a second click operation for triggering the end of shooting is received by a user, the electronic equipment synthesizes a third image by using a preview image currently collected by a camera and saves the third image. That is, after the second click operation is received by the electronic equipment, the electronic equipment gives up waiting for preview images that have not been collected yet, directly synthesizes and saves only the preview image currently collected by the camera, so that the efficiency of outputting an image can be improved, the waiting time of the user can be reduced, and the shooting experience of the user can be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of electronic devices, and in particular to an image capturing method, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] With the development of image capturing technology of electronic devices and the increasing demand of users for image capturing, the existing electronic devices can provide more and more image capturing modes for users. For example, portrait mode, panorama mode, time-lapse photography, night scene mode, slow motion, high dynamic range imaging (HDR), and light painting. Among them, light painting can also be referred to as long exposure.

[0003] Currently, the light painting mode can continuously capture images for a long time to capture the motion trail of light. Therefore, when image capturing is performed using the light painting mode, the electronic device needs to respond to two click operations of the user on the shutter to complete the capturing. The first click operation of the user triggers the electronic device to start image capturing, and the second click operation of the user triggers the electronic device to end image capturing. However, after the electronic device starts image capturing, it will continuously call the camera to capture images.

[0004] Therefore, when the user triggers the electronic device to end image capturing, the camera can still have some images that have not been captured. Currently, for these images that have not been captured, the electronic device often waits until the camera completes image capturing, and then generates a final image using the images captured by the camera to end this image capturing. Thus, when image capturing is performed using the light painting mode, the electronic device has a slow image output speed and low image output efficiency, which causes the user to be in a long waiting state for image output, resulting in poor user experience. SUMMARY

[0005] Embodiments of the present application provide an image capturing method, an electronic device, and a computer readable storage medium, to solve the problem of slow image output speed, low image output efficiency, long waiting time for image output, and poor user experience in the light painting mode.

[0006] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, an image capturing method is provided. The method is applied to an electronic device. The electronic device comprises a camera and supports a preset capturing mode which can be used to capture a star trail. The method comprises: in response to a user's opening operation on the preset capturing mode in a camera application, the electronic device displays a capturing interface corresponding to the preset capturing mode, wherein the capturing interface corresponding to the preset capturing mode comprises a capturing shutter. Subsequently, when the user triggers image capturing in the capturing interface, the electronic device considers that the image is captured in the preset capturing mode. That is, the electronic device captures the image in the image capturing manner corresponding to the preset capturing mode, and processes the captured image in the processing manner corresponding to the preset capturing mode.

[0008] In the preset capturing mode, the electronic device first receives a first click operation of the user on the capturing shutter, displays a first image at a first time, and displays a second image at a second time. The first image is an image obtained by synthesizing the first to k-th preview images captured by the camera based on the first to k-th preview requests generated in response to the first click operation. The second image is an image obtained by synthesizing the first to k-th preview images captured by the camera based on the first to k-th preview requests generated in response to the first click operation. k>1, and k is a positive integer. The second time is before the first time. It can be understood that the camera captures a corresponding preview image based on each generated preview request.

[0009] Then, the electronic device receives a second click operation of the user on the capturing shutter at a third time after the second time. The second click operation is used to trigger the end of the photographing. The electronic device saves a third image and ends the photographing.

[0010] In the first aspect, at the third time, if the electronic device calls the camera to have captured the i-th preview image but has generated the i+p-th preview request in response to the first click operation, the electronic device only synthesizes the first to i-th preview images to obtain the third image to be saved. That is, in the first aspect, if the electronic device receives the second click operation, the electronic device directly gives up waiting for the preview images that have not yet been captured, and directly synthesizes only the preview images that have been captured by the camera to obtain the third image to be saved. That is, the electronic device gives up waiting for the above-mentioned p frames of images to be captured, directly synthesizes the first to i-th preview images to obtain the third image to be saved, thereby avoiding the problem of reduced image output speed caused by waiting for the images to be captured, thereby improving the image output efficiency, reducing the user waiting time, and improving the user photographing experience.

[0011] In a possible implementation manner of the first aspect, when the electronic device receives the second click operation triggered by the user to end the photographing at the third time, there can be a case that all the captured preview images have been synthesized and sent to display in time, or there can also be a case that the electronic device still has some preview images that have been captured but have not been synthesized and sent to display. Then, the fourth image displayed at the third time can be an image synthesized from all the captured preview images, or can be an image excluding the preview images that have not been sent to display. That is, the image photographing method described above can further include:

[0012] At the third time, the electronic device displays a fourth image; the fourth image is synthesized from the first to jth preview images captured by the camera based on the first to jth preview requests generated in response to the first click operation. Wherein, k < j ≤ i, and j is a positive integer. Specifically, if j < i, it indicates that the fourth image displayed at the third time is an image synthesized from the first to jth preview images. Then, the third image can be obtained by synthesizing the fourth image and the preview images after the jth frame. If j = i, it indicates that the fourth image displayed at the third time is an image synthesized from the first to ith preview images, and then the third image can be the fourth image.

[0013] In another possible implementation manner of the first aspect, in terms of the processing capability of the electronic device, the preview image that cannot be sent to display in time is probably only one frame. Therefore, at the third time, the electronic device displays a fourth image; the fourth image is synthesized from the first to jth preview images captured by the camera based on the first to jth preview requests generated in response to the first click operation; wherein, j = i or j = i-1.

[0014] In a possible implementation manner of the first aspect, in order to conform to the image storage format, saving the third image can include: performing format conversion on the third image, and saving the third image after format conversion. In another possible implementation manner of the first aspect, the third image before format conversion can be in YUV format, and the third image after format conversion can be in JPEG format.

[0015] In a possible implementation manner of the first aspect, the electronic device can perform image synthesis on the multiple preview images in a frame-by-frame cumulative manner. Based on this, synthesizing the second image from the first to kth preview images captured by the camera can include: synthesizing a fifth image and the kth preview image to obtain the second image. Wherein, the fifth image is synthesized from the first to k-1th preview images.

[0016] In another possible implementation manner of the first aspect, synthesizing the third image from the first to ith preview images captured by the camera can include: synthesizing a sixth image and the ith preview image to obtain the third image. Wherein, the sixth image is synthesized from the first to i-1th preview images.

[0017] In a possible implementation manner of the first aspect, to avoid the preview request without corresponding preview image causing other unnecessary problems and leading to image shooting abnormality, the image shooting method further includes: generating a corresponding virtual preview image for each of the p preview requests after the i-th preview request; the virtual preview image includes virtual image data, image metadata, and a frame discard flag.

[0018] In another possible implementation manner of the first aspect, to avoid the exposure decision being made after the shooting is completed, leading to shooting abnormality, the image shooting method further includes: interrupting the exposure decision after the third image is obtained.

[0019] In a possible implementation manner of the first aspect, the electronic device includes an automatic exposure module, a frame output module, a fast frame output module, an image processing module, a selection module, and a display module; the image shooting method further includes:

[0020] The camera application continuously issues a preview request to the frame output module in response to the first click operation; the frame output module periodically sends a preview request to the camera according to an exposure time of the camera to drive the camera to collect a preview image; the preview images are saved in a first image queue according to the order of collection; the exposure time is decided by the automatic exposure module according to the ambient brightness and sent to the frame output module. Then, the fast frame output module extracts the preview images from the first image queue frame by frame and transmits them to the image processing module, the image processing module synthesizes the first preview image to obtain a first image, and synthesizes the first to k-th preview images to obtain a second image; the image processing module transmits the first image and the second image to the camera application via the selection module and the display module, and the camera application displays the first image and the second image.

[0021] In a possible implementation manner of the first aspect, the electronic device further includes a picture generation module. The image capturing method further includes: the camera application sends a photographing request to the fast back frame module via the out-frame module in response to the second click operation at the third time; at the third time, if the preview image after the jth frame of the first i frames of preview images is not transmitted to the image processing module, the fast back frame module extracts the preview image after the jth frame and transmits the preview image to the image processing module in response to the photographing request; k < j < i, and j is a positive integer; the image processing module synthesizes the third image from the preview image after the jth frame and a fourth image; the fourth image is synthesized by the image processing module based on the first j frames of preview images; the image processing module transmits the third image to the picture generation module via the selection module, and the picture generation module sends the third image to the camera application for saving after format conversion. In this way, the fast back frame module quickly responds to the photographing request, the third image is synthesized based on the currently captured preview image and saved, the out-picture efficiency is improved, the user waiting time is reduced, and the user shooting experience is improved.

[0022] In another possible implementation manner of the first aspect, at the third time, if the first i frames of preview images have all been extracted and transmitted to the image processing module frame by frame, the fast back frame module sends a picture generation instruction to the selection module in response to the photographing request; the selection module transends the third image to the picture generation module, and the picture generation module sends the third image to the camera application for saving after format conversion. In this way, the third image that can be saved can be quickly obtained, and the out-picture efficiency is improved.

[0023] In a possible implementation manner of the first aspect, the image capturing method further includes: the fast back frame module receives a flag bit issued by the camera application, and the flag bit is issued when the camera application receives the third image; and the fast back frame module generates a corresponding virtual preview image for each of the p preview requests after the ith preview request in response to the flag bit.

[0024] In a possible implementation manner of the first aspect, the image capturing method further includes: the fast back frame module sends an interrupt instruction to the automatic exposure module in response to the flag bit issued by the camera application, to instruct the automatic exposure module to interrupt exposure decision.

[0025] In a second aspect, the present application provides an electronic device, including: a camera, one or more processors and a memory, the camera, the memory and the processor are coupled respectively; the memory stores one or more computer program codes, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device performs the following steps:

[0026] In response to a user opening operation on a preset shooting mode in a camera application, a shooting interface corresponding to the preset shooting mode is displayed; wherein the shooting interface corresponding to the preset shooting mode includes a shooting shutter; a first click operation of the user on the shooting shutter is received; at a first time, a first image is displayed; wherein the first image is obtained based on a first preview request generated in response to the first click operation, and a first frame of preview image captured by a camera; the camera captures one frame of preview image based on each preview request; at a second time, a second image is displayed; wherein the second image is obtained based on a first to kth preview request generated in response to the first click operation, and a first to kth frame of preview image captured by the camera; the second time is after the first time; k>1, and k is a positive integer; at a third time, a second click operation of the user on the shooting shutter is received, and a third image is saved; wherein the third time is after the second time; at the third time, an ith preview request has been generated in response to the first click operation, and an ith frame of preview image has been captured by the camera; the third image is obtained based on a first to ith frame of preview image captured by the camera; i>k, and p is a positive integer.

[0027] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: at a third time, the electronic device displays a fourth image; the fourth image is obtained based on a first to jth preview request generated in response to the first click operation, and a first to jth frame of preview image captured by the camera; k

[0028] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: the third image is format-converted, and the format-converted third image is saved. In another possible implementation of the second aspect, the third image before format conversion is in YUV format, and the third image after format conversion is in JPEG format.

[0029] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: a fifth image and the kth frame of preview image are synthesized to obtain the second image; wherein the fifth image is synthesized based on the first to k-1th frame of preview image.

[0030] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following step: synthesizing the sixth image and the i-th preview image to obtain a third image, wherein the sixth image is an image synthesized based on the first to (i-1)-th preview images.

[0031] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following step: generating a corresponding virtual preview image for each of the p preview requests after the i-th preview request, wherein the virtual preview image comprises virtual image data, image metadata, and a frame skipping flag.

[0032] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following step: interrupting the exposure decision after the third image is obtained.

[0033] In a possible implementation of the second aspect, the electronic device comprises an automatic exposure module, a frame output module, a fast frame output module, an image processing module, a selection module, and a display module. When the computer instructions are executed by the processor, the electronic device further performs the following steps: the camera application continuously issues a preview request to the frame output module in response to the first click operation; the frame output module periodically sends a preview request to the camera according to the exposure time of the camera to drive the camera to capture a preview image; wherein the preview images are saved in the first image queue according to the order of capture; the exposure time is decided by the automatic exposure module according to the ambient brightness and sent to the frame output module. Then, the fast frame output module extracts the preview images from the first image queue frame by frame and transmits them to the image processing module, the image processing module synthesizes the first preview image to obtain the first image, and the image processing module synthesizes the first to k-th preview images to obtain the second image; the image processing module transmits the first image and the second image to the camera application via the selection module and the display module, and the camera application displays the first image and the second image.

[0034] In a possible implementation manner of the second aspect, the electronic device further includes a picture generation module. When the computer instructions are executed by the processor, the electronic device further performs the following steps: the camera application sends a photographing request to the fast back frame module via the out-frame module in response to the second click operation at the third time; at the third time, if the preview image after the jth preview image in the first i preview images has not been transmitted to the image processing module, the fast back frame module extracts the preview image after the jth preview image and transmits the preview image to the image processing module in response to the photographing request; k < j < i, j is a positive integer; the image processing module synthesizes the third image from the preview image after the jth preview image and a fourth image; the fourth image is synthesized by the image processing module based on the first j preview images; the image processing module transmits the third image to the picture generation module via the selection module, and the picture generation module sends the third image to the camera application for saving after format conversion.

[0035] In a possible implementation manner of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: at the third time, if the first i preview images have all been extracted and transmitted to the image processing module frame by frame, the fast back frame module sends a picture generation instruction to the selection module in response to the photographing request; the selection module transmits the third image to the picture generation module, and the picture generation module sends the third image to the camera application for saving after format conversion. In this way, the third image that can be saved can be quickly obtained, and the out-picture efficiency is improved.

[0036] In a possible implementation manner of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: the fast back frame module receives a flag bit issued by the camera application, and the flag bit is issued when the camera application receives the third image; the fast back frame module generates a corresponding virtual preview image for each of the p preview requests after the ith preview request in response to the flag bit.

[0037] In a possible implementation manner of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: the fast back frame module sends an interrupt instruction to the automatic exposure module in response to the flag bit issued by the camera application, to instruct the automatic exposure module to interrupt exposure decision.

[0038] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor in an electronic device, the electronic device performs the image photographing method in the first aspect or any possible implementation manner thereof.

[0039] In a fourth aspect, the present application provides a computer program product, which, when running on a computer, causes the computer to perform the method according to the first aspect and any possible implementation thereof. The computer can be the electronic device described above.

[0040] It can be understood that the electronic device of the second aspect and any possible implementation thereof, the computer readable storage medium of the third aspect, and the computer program product of the fourth aspect can achieve the beneficial effects as described in the first aspect and any possible implementation thereof, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A scene diagram of triggering the flow light shutter by the user according to an embodiment of the present application;

[0042] Figure 2 A diagram of a star trail image according to an embodiment of the present application;

[0043] Figure 3 A scene diagram of triggering the splendid star trail sub-mode by the user according to an embodiment of the present application;

[0044] Figure 4 An interface diagram of waiting for the image to be output in the flow light shutter mode according to an embodiment of the present application;

[0045] Figure 5 A flowchart of an image shooting method according to an embodiment of the present application;

[0046] Figure 6 A structure block diagram of a software and hardware architecture of an electronic device according to an embodiment of the present application;

[0047] Figure 7 A flowchart of another image shooting method according to an embodiment of the present application Figure 1 ;

[0048] Figure 8 A flowchart of another image shooting method according to an embodiment of the present application Figure 2 ;

[0049] Figure 9 A flowchart of a traditional image shooting method according to an embodiment of the present application;

[0050] Figure 10 A flowchart of another image shooting method according to an embodiment of the present application Figure 3 ;

[0051] Figure 11 A flowchart of another image shooting method according to an embodiment of the present application Figure 4 ;

[0052] Figure 12 This is a schematic diagram of the structure of an electronic device 1200 provided in an embodiment of this application;

[0053] Figure 13 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is only for the purpose of describing specific embodiments and is not intended to limit the application. Furthermore, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., are not necessarily different. Also, in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0055] With the development of image capture technology for electronic devices and the increasing demands of users for photography, existing electronic devices can offer users a wider range of image capture modes. These include, for example, portrait mode, panorama mode, time-lapse photography, night mode, slow motion, high dynamic range imaging (HDR), and light painting.

[0056] Light painting, also known as long exposure, is a shooting mode used to capture the movement of light and automatically extends the shutter speed. Simply put, using light painting mode captures the movement of light. Visually, the light rays captured in light painting mode appear as streaks.

[0057] like Figure 1 As shown, taking mobile phone 10 as an example, this application embodiment illustrates a scenario where a user triggers the activation of the light painting shutter. The following, in conjunction with... Figure 1 The procedure for users to trigger the light painting shutter is explained.

[0058] refer to Figure 1 When the phone is powered on, it can display something like this: Figure 1 The main interface 100 shown may include application icons for applications such as "Clock", "Calendar", "Camera", "Gallery", "Notes", "File Management", "Email", "Music", "Settings", "Contacts", "Phone", and "Messages".

[0059] It can be understood that the main interface 100 of the mobile phone 10 can also include application icons of more application programs according to actual needs, for example, various third-party application programs downloaded by the user, and the embodiments of the present application do not make any limitation in this regard.

[0060] The mobile phone 10 enters the camera application in response to the click operation of the user on the "camera" application icon 101 in the main interface 100, and displays a shooting interface 102.

[0061] The shooting interface 102 includes a plurality of shooting options. The shooting interface 102 shown includes "AI", "HDR", "Settings", "Aperture", "Night scene", "Portrait", "Photo", "Video", "Professional", and "More" shooting options.

[0062] The mobile phone 10 enters a more shooting option interface 104 in response to the click operation of the user on the "More" shooting option 103 in the shooting interface 102.

[0063] The more shooting option interface 104 includes other shooting options provided by the camera. The more shooting option interface 104 shown includes "Micro film", "Time-lapse photography", "Panorama", "Document scanning", "Watermark", and "Flow shutter". Similarly, according to actual needs, the more shooting option interface 104 can also include more shooting options, for example, "High pixel", "Multi-lens video", "Super macro", etc., and the embodiments of the present application do not make any limitation in this regard.

[0064] The mobile phone 10 enters a flow shutter shooting interface 106 in response to the click operation of the user on the "Flow shutter" shooting option 105 in the more shooting option interface 104. In the flow shutter shooting interface 106, the mobile phone 10 can trigger the mobile phone 10 to start image shooting in response to the click operation of the user on the shooting shutter.

[0065] It should be noted that the embodiments of the present application Figure 1 The scene shown does not constitute a limitation on the scene in which the user triggers the flow shutter to be started, and the user can also trigger the flow shutter to be started in more ways. For example, the user can also start the flow shutter in the camera settings provided by the "Settings", and the embodiments of the present application do not make any limitation in this regard.

[0066] In addition, a plurality of sub-modes can be provided in the flow shutter mode according to different shooting scene needs. For example, four sub-modes of "vehicle and people", "light drawing", "silk flowing water", and "splendid star track". The user can select the corresponding sub-mode for image shooting according to the actual shooting scene needs.

[0067] Among them, the car water and horse can be used to shoot the traffic. Light painting graffiti can be used to shoot light painting. Silk water can shoot all dynamic water elements, including running water, waves and waterfalls, etc. And the beautiful star track can be used to shoot the starry sky. Through the beautiful star track, the motion track of the star can be clearly shot, that is, the star track.

[0068] As shown in Figure 2 , the embodiment of the application shows a star track image. Figure 2 The star track image shown is shot by using the beautiful star track sub-mode, and the image is processed by black and white after the later period, and the white line in the image is the motion track of the star.

[0069] As shown in Figure 3 , the embodiment of the application shows a star track image. Figure 1 As shown in Figure 3 , the embodiment of the application shows a scene diagram of the user triggering to start the beautiful star track sub-mode. Hereinafter, the operation process of the user triggering to start the beautiful star track is described.

[0070] After the user triggers to start the light flow shutter module, the mobile phone 10 displays the light flow shutter shooting interface 106. The light flow shutter shooting interface 106 includes a sub-mode selection control 107.

[0071] The mobile phone 10 responds to the click operation of the user on the sub-mode selection control 107, and enters the sub-mode selection interface 108.

[0072] The sub-mode selection interface 108 includes various sub-modes provided by the light flow shutter, and when a certain sub-mode is selected, the sub-mode selection interface 108 displays the name of the sub-mode accordingly. Figure 3 The currently default selected sub-mode in the sub-mode selection interface 108 is car water and horse.

[0073] The mobile phone 10 switches the sub-mode in response to the left sliding operation 109 of the user. For example, after the user selects the beautiful star track mode by left sliding, the user can stop sliding to complete the switching of the sub-mode, and display the beautiful star track selected interface 110.

[0074] The mobile phone 10 responds to the click operation of the user on the blank of the beautiful star track selected interface 110, and re-displays the light flow shutter shooting interface 106. In the light flow shutter shooting interface 106, the mobile phone 10 responds to the first click operation 111 of the user on the shooting shutter, and enters the shooting interface 112 to start image shooting.

[0075] Understandably, since the phone 10 had already switched from traffic scenes to star trails in response to the user's operation before the light trail shooting interface 106 was re-displayed, the phone 10 will process the image according to the image processing method corresponding to star trails when the user's first click operation 111 triggers image capture under the light trail shooting interface 106.

[0076] Additionally, the shooting interface 112 displays a shooting timer "00:00" and a shooting precaution "Please keep the camera stable." The shooting timer "00:00" increases sequentially in seconds as the shooting time progresses, for example, "00:01", "00:02", "00:03", and so on. The shooting timer allows you to determine the current shooting duration. Simultaneously, the shooting shutter speed is displayed on the shooting interface 112, and the captured image is displayed in real-time in the viewfinder 114 for the user to preview.

[0077] When the user observes a satisfactory star trail effect in the viewfinder 114, they can tap the shutter button in the shooting interface 112 a second time 113. The phone 10 responds to the user's second tap 113 and ends the image capture.

[0078] It should be noted that the embodiments of this application Figure 3 The scenario shown does not constitute a limitation for the user to trigger the activation of the Star Trails mode. Users can trigger the activation of the Star Trails mode in many other ways. For example, users can also switch the sub-mode of the Light Painting shutter from Traffic Scene to Star Trails by directly clicking on the Star Trails sub-mode selection interface 108. This application embodiment does not impose any limitations on this.

[0079] according to Figure 3 As shown in the scenario, when an electronic device takes a picture in light painting mode, it needs to respond to two clicks of the shutter button by the user to complete the shooting.

[0080] Among them, the electronic device's response to the user's first click of the shutter button, such as... Figure 3 The first click operation 111 shown is used to trigger the electronic device to start taking pictures (that is, the first click operation is used to trigger the electronic device to start taking pictures). The electronic device processes the acquired image data in real time and returns it to the camera application for display (i.e., display), so that the user can preview the image shooting effect.

[0081] The electronic device's response to the user's second click of the shutter button, such as... Figure 3The second click operation 113 is shown to trigger the electronic device to end image shooting (i.e., the second click operation is used to trigger the electronic device to end shooting), and the electronic device directly stores the image after image processing into the gallery.

[0082] The timing at which the user triggers the electronic device to end shooting can be any time, and the embodiments of the present application do not make any limitation thereon. Generally, the user can observe the image effect of the display image in real time, and if the image effect meets the expected shooting effect, the user can trigger the electronic device to end shooting by clicking the shutter twice.

[0083] Briefly, in the flowing shutter mode, the first click operation of the user on the shooting shutter is regarded by the electronic device as a preview request in the flowing shutter mode. The second click operation of the user on the shooting shutter is regarded by the electronic device as a shooting request. The difference between the preview request and the shooting request is that the image corresponding to the preview request is sent back to the camera application after processing for display, i.e., display. The image corresponding to the shooting request is not displayed after synthesis processing, but is directly saved. For example, the image corresponding to the shooting request is directly stored into the gallery after synthesis processing.

[0084] That is, in the flowing shutter mode in the open state, if the camera application in the electronic device receives the first click operation of the user on the shooting shutter, the camera application will continuously issue preview requests to the lower layer. In turn, the lower layer will collect the corresponding images (which can be referred to as preview images) in response to the preview requests issued by the camera application. Then, the lower layer returns the collected preview images to the upper camera application for display after corresponding image processing, for the user to preview.

[0085] If the camera application in the electronic device receives the second click operation of the user on the shooting shutter, the camera application will not continuously issue preview requests to the lower layer, but will issue a corresponding shooting request to the lower layer in response to the second click operation. In turn, the lower layer will collect the corresponding images (which can be referred to as shooting images) in response to the shooting request after receiving the shooting request. However, the difference is that after the lower layer returns the collected shooting images to the upper layer after image processing, the upper layer will choose to save directly, and the upper camera application will not display the shooting image. The shooting image saved by the electronic device is the image obtained by shooting in the flowing shutter mode.

[0086] It should be noted that the preview in the flow shutter mode is different from the ordinary preview (i.e., the preview image displayed in the viewfinder after the camera is turned on). The requirement for image processing in the ordinary preview is generally low, but the preview in the flow shutter mode is triggered by the user clicking the shooting shutter, and the requirement for image processing depends on the requirement of the flow shutter mode for the image. It can be understood that the image processing requirement of the preview in the flow shutter mode is much higher than that of the ordinary preview. Generally, the image processing flow corresponding to the preview request in the flow shutter mode is the same as that corresponding to the shooting request in the flow shutter mode, and the difference is only whether the display is sent.

[0087] Thus, it can be simply understood that the flow shutter is obtained by long-time continuous shooting to obtain the motion track of the object in the corresponding shooting scene, for example, the star track is obtained by long-time shooting of the starry sky. Therefore, the electronic device performs image shooting in the flow shutter mode, and as the shooting time is prolonged, the images collected by the electronic device will be more and more. However, after the shooting is finished, the electronic device will only generate one image and store it in the gallery. Therefore, when the electronic device performs image shooting in the flow shutter mode, the principle of image processing by the electronic device is to perform image synthesis on the collected images. That is, before the electronic device receives the second click operation of the shooting shutter by the user, the electronic device will continuously perform image synthesis on the collected images, and the obtained synthesized image will be displayed. When the electronic device receives the second click operation of the shooting shutter by the user, the electronic device will directly save the synthesized image after image synthesis without displaying it.

[0088] Specifically, after the electronic device opens the flow shutter mode in response to the opening operation of the user, if the first click operation of the shooting shutter by the user is received, the electronic device starts shooting in response to the first click operation. During the shooting process, the electronic device collects one frame of image (out of frame) every time, and then performs image synthesis on the current frame of image and the previously collected images, and fuses the image information of the previous images through the two-by-two synthesis of images. At the same time, before the second click operation of the shooting shutter by the user is received, the electronic device will display the synthesized image obtained by image synthesis, so that the user can preview the effect of image shooting in real time. If the second click operation of the shooting shutter by the user is received, the electronic device saves the newly synthesized synthesized image without displaying it in response to the second click operation, and ends the image shooting.

[0089] For example, the first frame image collected after the electronic device starts to take a photo is the first frame without a pre-image, so the electronic device displays the first frame image alone after image processing. When the second frame image is collected, the electronic device synthesizes the second frame image with the first frame image to obtain a new second frame image, which fuses the image information of the first frame image. Then, the electronic device displays the new second frame image.

[0090] Similarly, the third frame image needs to be synthesized with the new second frame image before being displayed. The fourth frame image, the fifth frame image, and the nth frame image are synthesized in the same way.

[0091] However, when the nth frame image, that is, the last frame image of the shooting, is reached, the nth frame image is the frame image corresponding to the user's trigger to end the shooting (that is, the shooting request issued by the user's second click operation on the shooting shutter). It is also synthesized with the synthesized image of the previous frame, but will not be displayed, but directly stored in the gallery. That is, the first frame image, the second frame image, and the nth-1 frame image are displayed, and the nth frame image is not displayed.

[0092] Therefore, the third frame image displayed fuses the image information of the first frame image and the second frame image. The fourth frame image displayed fuses the image information of the first frame image, the second frame image, and the third frame image. The subsequent images are the same, and will not be described here. The frame image stored in the gallery fuses the image information of all the previous frames, that is, the image information of the first frame image, the second frame image, and the nth-1 frame image. That is, in the flow shutter mode, the electronic device synthesizes all the images by long-time continuous shooting to obtain an image with a motion trajectory.

[0093] However, the auto exposure (AE) module in the electronic device currently decides the exposure time based on the brightness of the surrounding environment. The longer the surrounding environment is darker, the longer the exposure time decided by the AE module. When the surrounding environment is particularly dark, the exposure time decided by the AE module can be as long as 10 seconds. That is, when the surrounding environment is particularly dark, the electronic device can take a frame image (out frame) for up to 10 seconds.

[0094] So, for the flow light shutter mode, especially for the scene of the beautiful star trail which needs long time shooting and usually needs to be shot in a dark environment, when the electronic device receives the second click operation of the shooting shutter of the user for triggering the end of shooting, the electronic device is likely to have a part of the image being produced (hereinafter referred to as the frame being produced) because the out-frame time is too long. Because, the underlying needs to respond to each request issued by the upper camera application and feed back the corresponding image. Therefore, when the electronic device receives the instruction of the user triggering the end of shooting, the underlying may be collecting the image in response to a preview request. At the same time, the underlying may also have some preview requests waiting for response processing.

[0095] That is, when the electronic device receives the instruction of the user triggering the end of shooting (the second click operation of the shooting shutter of the user), the preview request issued by the camera application before may still be processing. There may also be some preview requests that have been issued by the camera application but have not been responded yet, thereby causing some frames being produced.

[0096] Therefore, the frame being produced may be an image that has been collected for a part but not completely collected, or an image that has not been collected at all.

[0097] According to the traditional processing mode, after receiving the second click operation of the shooting shutter of the user, in order to ensure that the preview request issued by the camera application is responded and has the corresponding image, the electronic device will wait for the frame being produced to complete the out-frame, and then respond to the shooting request to generate the last frame of the shooting synthesis image (i.e. the synthesis image corresponding to the shooting request) and store it into the gallery.

[0098] From the perspective of the user, the shooting shutter on the shooting interface displayed by the electronic device will always be in a rotating state, representing that the electronic device is generating an image (out-image). For example, as shown in Figure 4 , an interface diagram of waiting for out-image is shown. Figure 4 For example, the second click operation 113 of the user in the shooting interface 112 shown in Figure 3 is taken as an example of the interface diagram. That is, the mobile phone 10 enters the waiting for out-image interface 113 in response to the second click operation 113 of the shooting shutter of the user.

[0099] The outer circle of the circle of the shooting shutter in the waiting for out-image interface 113 is in a rotating state. It should be noted that, Figure 4 the arrow shown in does not constitute a limitation on the waiting for out-image interface 113. That is, the electronic device actually displayed on the waiting for out-image interface 113 can not have this arrow.

[0100] and Figure 4 The clockwise rotation indicated by the arrow is also an example of the embodiments of the present application. According to actual needs, it can also be counterclockwise rotation, and the embodiments of the present application do not make any limitation.

[0101] For example, if the exposure time is 10 seconds, and there are 5 frames being output at present, the user needs to wait about 5*10 seconds = 50 seconds. That is, in the user's view, the user will watch the circular rotation of the shooting shutter for 50 seconds.

[0102] It can be understood that this 50 seconds is only used as an example of the embodiments of the present application, and the actual waiting time depends on the actual decision of the exposure time and the number of frames being output, and the embodiments of the present application do not make any limitation.

[0103] Therefore, in the traditional star trail shooting mode, after the electronic device receives the second click operation of the user on the shooting shutter, the electronic device needs to wait for the frame to be output, so that the time for the electronic device to output the image is relatively long, and the image output efficiency is low. At the same time, the user needs to wait for a long time to get the photographed image, so that the user is always in a long waiting state, thereby causing poor user experience.

[0104] Therefore, in order to reduce the image output time in the star trail shooting mode, improve the image output efficiency, reduce the user waiting time, and improve the user experience, the embodiments of the present application provide an image shooting method.

[0105] The image shooting method provided by the embodiments of the present application is applied to an electronic device. At the same time, the electronic device can support a preset shooting mode, and the preset shooting mode includes a star trail shooting mode. That is, the preset shooting mode can be used for shooting a star trail.

[0106] In some embodiments, the preset shooting mode can be a star trail shooting mode or a long exposure mode. It can be understood that based on different custom names of different manufacturers, the preset shooting mode can be the above-mentioned star trail shooting mode or long exposure mode. Of course, according to the actual name of the manufacturer, the preset shooting mode can also be other names, and the embodiments of the present application do not make any limitation.

[0107] In order to facilitate the understanding and description of the scheme, the following embodiments will uniformly refer to the preset shooting mode as the star trail shooting mode.

[0108] In the embodiments of the present application, when the star trail shooting mode is in an open state, after the electronic device receives the instruction of the user to end the shooting, that is, after the electronic device receives the second click operation of the user on the shooting shutter, the electronic device directly generates an image using the currently generated frame and stores it in the gallery in response to the second click operation.

[0109] That is, the electronic device in the embodiment of the present application no longer waits for the frames being produced to complete the frame output, but directly gives up the frames being produced. Thus, the electronic device can avoid the long image output time caused by waiting for the frame output to complete, thereby reducing the image output time, improving the image output efficiency, reducing the user waiting time and improving the user experience.

[0110] It can be understood that giving up the frames being produced will cause the finally generated image to lack the image information of the preview images, but the frames being produced are usually the image information of the last few frames of the preview images, so even if the last few frames of image information is missing, the effect on the final image is not great.

[0111] In some embodiments, the preset shooting mode of the embodiment of the present application can also be used for shooting car water, silk water, light painting and graffiti, etc. However, it should be noted that the shooting time of car water and light painting is usually relatively short, so the number of images collected during the entire shooting process will not be too much, and the original image output speed will not be too slow.

[0112] At the same time, silk water is usually shot during the day, so the surrounding light will not be too dark, and the corresponding exposure time will not be too long. Therefore, if the image shooting method provided in the embodiment of the present application is applied to car water, silk water, light painting and graffiti, etc., although the image output efficiency can be improved by giving up the frames being produced, the effect is not as obvious as star trail shooting.

[0113] Therefore, the image shooting method provided in the embodiment of the present application can be configured to be applied only to the splendid star trail for star trail shooting according to actual business needs. It can also be configured to be applied to any one or more of car water, light painting, silk water and splendid star trail.

[0114] It can be understood that no matter which sub-mode is applied, the implementation principle is the same, and the electronic device can be configured according to the actual situation, which is not limited in the embodiment of the present application.

[0115] As shown in Figure 5 , the embodiment of the present application shows a flowchart of an image shooting method, including steps S501-S503. In the following, the image shooting method provided in the embodiment of the present application is described in detail in combination with the flowchart shown in Figure 5 .

[0116] S501, the electronic device displays a shooting interface corresponding to a preset shooting mode in response to a user's opening operation of the preset shooting mode in the camera application. The shooting interface corresponding to the preset shooting mode includes a shooting shutter.

[0117] The process that the electronic device opens the preset shooting mode in response to the opening operation of the user can refer to the description of the above-mentioned scene. Figure 1 and Figure 3 The shooting interface corresponding to the preset shooting mode can be the flowing shutter shooting interface 106 shown in Figure 3

[0118] S502, the electronic device receives the first click operation of the user on the shooting shutter, displays the first image at the first time, and displays the second image at the second time.

[0119] After the electronic device receives the first click operation of the user on the shooting shutter (for example, the first click operation 111 shown in Figure 3 The electronic device calls the camera to start collecting images. It can be understood that, since the first click operation in the flowing shutter mode corresponds to a preview request, the electronic device calls the camera to start collecting images in response to the first click operation can be regarded as a preview image in the flowing shutter mode.

[0120] In addition, in the flowing shutter mode, before the electronic device receives the second click operation of the user on the shooting shutter (for example, the second click operation 113 shown in Figure 3 The electronic device will continuously generate multiple preview requests in response to the first click operation because it wants to continuously collect preview images.

[0121] At the same time, the electronic device will call the camera to collect 1 frame of corresponding preview image in response to each preview request respectively. That is, the number of preview images is the same as the number of preview requests. That is, the first preview request corresponds to the first frame of preview image, the second preview request corresponds to the second frame of preview image, and the third preview request corresponds to the third frame of preview image.

[0122] For the first frame of preview image collected by the camera, since there is no preview image before the first frame of preview image, the first frame of preview image can be directly displayed at the first time after image processing. After the first frame of preview image, the electronic device will perform image synthesis on each frame of preview image collected by the camera and the previous preview image before displaying.

[0123] That is, the electronic device can display the second image obtained by image synthesis at the second time. This second image can be an image obtained by image synthesis of the first to k frames of preview images. k is greater than 1 and k is a positive integer.

[0124] ​It can be understood that the specific value of k depends on the number of preview images actually collected by the electronic device at the second time. For example, at the second time, the electronic device actually responds to 3 preview requests to collect 3 frames of preview images, so k = 3. For another example, at the second time, the electronic device actually responds to 10 preview requests to collect 10 frames of preview images, so k = 10.

[0125] It can be understood that, since the electronic device responds to each preview request in time sequence to collect preview images and then displays the image after image synthesis. Therefore, the first image must be displayed before the second image, so the second time in the embodiment of the application must be after the first time.

[0126] Specifically, if the electronic device collects the second frame of preview image, k = 2 at this time. Then the electronic device synthesizes the first frame of preview image and the second frame of preview image to obtain the second image 1, and displays this second image 1 at the second time 1.

[0127] If the electronic device collects the third frame of preview image, k = 3 at this time. Then the electronic device can synthesize the first frame of preview image, the second frame of preview image and the third frame of preview image to obtain the second image 2, and can display this second image 2 at the second time 2.

[0128] Similarly, if the electronic device collects the k frame of preview image, the electronic device can synthesize the k frame of preview image to obtain the second image k-1. That is, the electronic device synthesizes the first to k frames of preview images to obtain the second image k-1. At the same time, the electronic device can display this second image k-1 at the second time k-1. It can be understood that the second image includes the second image 1, the second image 2, …, the second image k-1.

[0129] It should be noted that the exposure time will affect the collection speed of the preview image, and the longer the exposure time, the longer the collection time of each frame of preview image. Therefore, at the second time, the number of preview requests actually generated by the electronic device may be greater than the number of preview requests actually responded by the electronic device. For example, at the second time, the electronic device actually generates 14 preview requests, but the electronic device can only respond to 10 preview requests to collect 10 frames of preview images.

[0130] S503, the electronic device receives a second click operation of the user on the shooting shutter at a third time, and saves the third image.

[0131] The third time is after the second time, and the third time can depend on the user. For example, the third time can be the time at which the user observes a satisfactory image shooting effect in the second image displayed by the electronic device. For example, the user determines that the electronic device has currently shot a satisfactory star trail effect by observing the second image displayed by the electronic device, and then the user can click the shooting shutter a second time at this time (i.e., the third time) to issue an end shooting instruction to the electronic device.

[0132] Similarly, because the exposure time affects the collection speed of the preview image, the actual number of generated preview requests is greater than the actual number of responded preview requests. Therefore, at the third time, the electronic device can have generated the ith+p preview request in response to the first click operation, but the electronic device can have actually only processed the ith preview request.

[0133] That is, at the third time, the electronic device can have generated a total of ith+p preview requests, but the electronic device has only responded to and processed the ith preview request. Accordingly, at the third time, the electronic device should only call the camera to collect the ith frame of preview image, that is, the electronic device has currently collected a total of i frames of preview image at the third time. At this time, the electronic device has p preview requests that have not been responded to, that is, the electronic device has p frames of preview image that have not been called to be collected by the camera.

[0134] However, at the third time, the electronic device receives the second click shooting of the user, indicating that the electronic device has received the instruction to trigger the end shooting issued by the user. According to the user's demand, the electronic device needs to respond to the second click operation to end the shooting. According to the conventional processing manner, the electronic device will continue to respond to the p preview requests. After waiting for the electronic device to call the camera to collect the p frames of preview image corresponding to the p preview requests, the electronic device synthesizes the i frames of preview image collected before to obtain a third image for saving.

[0135] Therefore, the time length for the conventional electronic device to respond to the p preview requests to collect the p frames of preview image is the time length for which the image needs to be additionally waited for, that is, the time length for which the user needs to wait. If the exposure time is longer, the waiting time length will be longer, thereby reducing the image output efficiency and causing the user to wait for a long time, and the user's shooting experience is poor.

[0136] In the embodiment of the present application, if the electronic device receives the second click operation at the third time, because only i frames of image are collected at the third time, the electronic device discards the p frames of image corresponding to the p preview requests, and directly synthesizes the i frames of preview image to obtain a third image for saving.

[0137] Therefore, the third image in the traditional processing manner is an image synthesized from the first to i+p frame preview images. The third image in the embodiment of the present application is an image synthesized from the first to i frame preview images. It can be seen that although the third image in the embodiment of the present application lacks the image information of the p frame preview image, the embodiment of the present application does not need to additionally wait for the p frame preview image to be output, thereby accelerating the image output speed and reducing the waiting time. Meanwhile, for long-time shooting, the lack of the last p frame image information has little effect on the image effect of the final third image. Therefore, compared with the traditional manner, the embodiment of the present application can accelerate the image output speed, improve the image output efficiency, reduce the user waiting time, and improve the user shooting experience while ensuring the image effect.

[0138] It can be understood that the third image saved in the embodiment of the present application is the image generated corresponding to the user triggering the end of shooting. That is, since the second click operation in the flow shutter mode corresponds to a shooting request, the electronic device receiving the second click operation is equivalent to receiving the shooting request issued by the user. Therefore, the saved third image is the image corresponding to the shooting request.

[0139] In some embodiments, when the instruction of the user triggering the end of shooting is received at the third time, due to the difference in processing speed of different threads or abnormal situations, the preview images collected by the electronic device have been timely synthesized and displayed, or the electronic device still has some preview images that have been collected but have not been synthesized and displayed. For example, when the electronic device receives the second click operation at the third time, the camera may have just collected one or more frames of preview images at that moment, and the one or more frames of preview images are the preview images that have not been synthesized and displayed. Or, due to abnormal situations, the one or more frames of preview images that have been collected and can be displayed at the third time are not successfully displayed at the third time.

[0140] Therefore, at the third time, the image displayed by the electronic device (hereinafter referred to as the fourth image) can be an image synthesized from the first to j frame preview images. Wherein, k < j ≤ i, and j is a positive integer.

[0141] For the case of j < i, although the electronic device collects the i frame preview image at the third time, due to the speed difference, the preview images from the j frame to the i frame can be the preview images collected at the third time. Therefore, in this case, the fourth image displayed at the third time is an image synthesized from the first to j frame preview images.

[0142] In this case, it can be understood that because the preview image frames after the jth frame are already captured frames, no additional time is spent waiting for the electronic device to perform image synthesis. Therefore, to ensure the efficiency of the image output while retaining as much image information of the preview images as possible, the electronic device can obtain a third image by synthesizing the fourth image and the preview images after the jth frame. Then, the third image is saved.

[0143] For example, at the third time, the electronic device captures a 5(i)th frame preview image. However, because the fourth frame preview image and the fifth frame preview image in the 5(i)th frame image are preview images captured at the third time, the fourth image displayed by the electronic device at the third time should be an image synthesized only from the first to third (j)th frame preview images. Further, after receiving the second click operation, because the two preview images do not need to wait for the image output, the electronic device can obtain a third image by synthesizing the fourth image, the fourth frame preview image, and the fifth frame preview image.

[0144] For the case where j = i, the difference between the capturing speed and the display speed is not large, and each captured frame preview image is synthesized and displayed in time. Therefore, in this case, the fourth image displayed at the third time is an image synthesized from the first to i frame preview images. For example, at the third time, the electronic device captures a 5(i)th frame preview image, and because the display is in time, the fourth image displayed at the third time can be an image synthesized from the first to fifth (j)th frame preview images. That is, i = j.

[0145] For this case, at the time when the user triggers the end of the photographing, all the captured preview images of the electronic device are successfully displayed. At this time, if the electronic device wants to synthesize the new captured preview image, the electronic device needs to wait for the camera to output the image, which requires time. Therefore, to improve the image output speed and reduce the waiting time of the user, the electronic device can directly save the fourth image as the third image. It can be understood that in this case, the third image saved by the electronic device is the fourth image.

[0146] Meanwhile, although the fourth image is an image synthesized in response to a preview request, the image processing flow corresponding to the preview request in the flowing shutter mode is usually the same as the image processing corresponding to the photographing request in the flowing shutter mode. Therefore, whether the fourth image is directly used as the third image corresponding to the photographing request or the third image corresponding to the photographing request is obtained by re-synthesis, the final image effect is the same. Directly using the fourth image as the third image can also save processing time, thereby further improving the image output efficiency and improving the user experience.

[0147] In some embodiments, the difference between the speed of acquisition and the speed of display is not very large in terms of the processing capability of the electronic device. Therefore, even if there are preview images that cannot be displayed in time, the number of such images is usually not large, and there is a high probability that there is only one such image. Therefore, j is equal to i or i-1 with high probability. That is, in some embodiments, j = i or j = i-1.

[0148] In some embodiments, the images synthesized by the electronic device from the preview images in the flow shutter mode are usually in YUV format. For example, the first image, the second image, the fourth image, and the third image obtained by synthesizing the preview image after the jth frame with the fourth image can all be in YUV format. In general, the electronic device only displays images in YUV format, and does not directly save images in YUV format to the gallery.

[0149] Therefore, in order to meet the image storage format, the third image can be saved by converting the YUV format of the third image to a format acceptable by the gallery of the electronic device before being stored in the gallery. In some embodiments, the converted format of the third image can be JPEG format. That is, the YUV format of the third image is converted to a JPEG format of the third image before being saved.

[0150] In some embodiments, the electronic device synthesizes multiple preview images in a frame-by-frame cumulative manner. That is, after the first frame of the preview image and the second frame of the preview image are synthesized to obtain the second image 1, the third frame of the preview image can be synthesized with the second image 1 to obtain the second image 2. The fourth frame of the preview image can be synthesized with the second image 2 to obtain the second image 3. Similarly, the kth frame of the preview image can be synthesized with the second image k-2 to obtain the second image k-1, and the specific process is the same and will not be repeated here.

[0151] Therefore, synthesizing the first to kth frames of the preview image obtained by the camera to obtain the second image can include synthesizing the fifth image and the kth frame of the preview image to obtain the second image. The fifth image of the present embodiment can be synthesized based on the first to k-1th frames of the preview image.

[0152] Similarly, synthesizing the first to i frames of the preview image obtained by the camera to obtain the third image can also include synthesizing the sixth image and the ith frame of the preview image to obtain the third image. The sixth image is synthesized based on the first to i-1th frames of the preview image. The principle is the same as the synthesis of the second image described above, which is a frame-by-frame cumulative synthesis, and will not be repeated here.

[0153] In some embodiments, the software system of the electronic device described above can employ a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Hereinafter, the embodiments of the present application take the Android system with a layered architecture as an example to exemplarily illustrate the software and hardware architecture of the electronic device. TM The structure block diagram of the software and hardware architecture of the electronic device is shown. Figure 6 The structure block diagram of the software and hardware architecture of the electronic device is shown.

[0154] As shown in Figure 6 , the structure block diagram of the software and hardware architecture of the electronic device includes a software layered architecture and a hardware layer. The hardware layer of the electronic device includes a camera, a display screen, a loudspeaker, and other hardware required by the electronic device. It can be understood that the electronic device can further include more hardware according to actual requirements, for example, can further include a key, a motor, and the like. The embodiments of the present application Figure 6 do not limit the hardware of the electronic device.

[0155] The software layered architecture of the electronic device divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer.

[0156] The application layer can include a series of application packages. As shown in Figure 6 , the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message applications.

[0157] The application framework layer provides the application layer with application programming interfaces (APIs) and programming frameworks. The application framework layer includes some pre-defined functions.

[0158] As shown in Figure 6 , the application framework layer can include camera service, window manager, activity manager, input manager, resource manager, notification manager, view system, and content provider.

[0159] The camera service plays a role of bridging the camera application and the camera HAL, and responds to the request of the camera application to invoke the camera to capture images. In the embodiment of the present application, the camera service can respond to the preview request (the preview request is generated in response to the first click operation of the user on the shooting shutter) and the photographing request (the photographing request is generated in response to the second click operation of the user on the shooting shutter) issued by the camera application, and issue the preview request and the photographing request to the camera HAL to invoke the camera to capture images, so as to obtain the preview image (such as the first image, the second image, and the fourth image described above) corresponding to the preview request and the photographing image (such as the third image described above) corresponding to the photographing request.

[0160] The window manager provides a window management service (WMS), which can be used for window management, window animation management, surface management, and as a relay station of an input system.

[0161] The activity manager can provide an activity management service (AMS), which can be used for starting, switching, scheduling of system components (such as activities, services, content providers, broadcast receivers), and management and scheduling of application processes.

[0162] The input manager can provide an input management service (IMS), which can be used for managing the input of the system, such as touch screen input, key input, sensor input, etc. The IMS takes events from the input device node, and through interaction with the WMS, distributes the events to the appropriate window.

[0163] The resource manager provides various resources for the application program, such as localized strings, icons, pictures, layout files, video files, etc.

[0164] The notification manager enables the application program to display notification information in the status bar, which can be used to convey messages of the notification type, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the completion of the download, message reminders, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification in the form of a dialogue window appearing on the screen. For example, the text information is prompted in the status bar, a prompt sound is emitted, the electronic device is vibrated, the indicator light flashes, etc.

[0165] The view system includes visual controls, such as controls that display text, controls that display pictures, and the like. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface that includes a short message notification icon can include a view that displays text and a view that displays a picture.

[0166] The content provider is used to store and retrieve data and make the data accessible to the application. The data can include videos, images, audio, dialed and received calls, browsing history and bookmarks, phonebook, and the like.

[0167] The hardware abstraction layer runs in the user space and encapsulates the kernel layer driver to provide a calling interface to the upper layer. As shown in Figure 6 , the hardware abstraction layer can include a camera HAL, a display HAL, an audio HAL, and a Bluetooth HAL.

[0168] The kernel layer is a layer between hardware and software. As shown in Figure 6 , the kernel layer can include a camera driver, a display driver, an audio driver, and a Bluetooth driver.

[0169] Taking the software and hardware architecture of the electronic device as shown in Figure 6 , the embodiments of the present application Figure 7 and Figure 8 respectively show a flowchart of an image capturing method.

[0170] In the following, the image capturing method provided by the embodiments of the present application will be described in detail. Figures 7-8

[0171] In the state where the flowing shutter mode is turned on, the first click operation of the user on the shooting shutter is used to trigger the electronic device to start photographing. That is, after the camera application of the application layer receives the first click operation of the user on the shooting shutter, the camera application starts to continuously issue a preview request to the lower layer, that is, to the camera service in the application framework layer.

[0172] In the embodiments of the present application, the request issued by the camera application is maintained in a request queue that follows the first-in-first-out rule according to the order of issue time.

[0173] As Figures 7-8 ​The request queue shown has preview request 1 (the first preview request) as the first one enqueued, preview request 2 (the second preview request) as the second one enqueued, and so on, preview request k (the kth preview request) as the kth one enqueued, and so on, preview request i (the ith preview request) as the ith one enqueued, and so on, preview request i+p (the (i+p)th preview request) as the (i+p)th one enqueued. The order in which the preview requests are enqueued corresponds to their number order, which will not be elaborated further here. Then, according to the first-in-first-out (FIFO) characteristic of the queue, each preview request is dequeued after the camera service processes it.

[0174] After receiving preview requests sequentially from the camera application, the camera service forwards these requests to the camera HAL. The frame output module within the camera HAL maintains these preview requests. The camera HAL also includes an automatic exposure module for determining the exposure time. The frame output module simultaneously receives the exposure time determined by the automatic exposure module.

[0175] In some embodiments, the automatic exposure module can also determine exposure parameters such as light intake and ISO sensitivity, but this application embodiment does not limit this. That is, while the automatic exposure module transmits the exposure time to the frame output module, it can also transmit exposure parameters such as light intake and ISO sensitivity.

[0176] After receiving the preview request from the upper layer and the exposure time determined by the automatic exposure module, the frame output module sends the preview request to the lower layer camera driver according to the exposure time determined by the automatic exposure module. The camera driver then calls the camera to capture the preview image corresponding to these preview requests.

[0177] like Figures 7-8 As shown, the automatic exposure mode determines the exposure time to be 10 seconds (i.e., 10s). Therefore, the frame output module calls the camera to capture the preview image corresponding to the preview request at 10-second intervals. This can be understood as the frame output module sending a preview request to the camera driver every 10 seconds to drive the camera to capture a preview image. That is, there is a 10-second interval between preview request 1 and preview request 2, a 10-second interval between preview request 2 and preview request 3, and subsequent preview requests are also spaced 10 seconds apart, which will not be elaborated further here.

[0178] After receiving preview requests from the frame output module, the camera driver responds to these requests by driving the camera to capture a preview image every 10 seconds and returning it to the camera driver. In this embodiment, the preview images are stored in the first image queue.

[0179] like Figures 7-8As shown, since the interval between the preview request 1 (the first preview request) and the preview request 2 (the second preview request) is 10 seconds, the interval between the preview image 1 (the first frame of preview image) and the preview image 2 (the second frame of preview image) in the first image queue is also 10 seconds. Similarly, the interval between the preview image 2 (the second frame of preview image) and the preview image 3 (the third frame of preview image) in the first image queue is also 10 seconds, and the time interval of the subsequently collected preview images is also 10 seconds, which will not be described here.

[0180] In the embodiment of the present application, after the camera driving receives the preview image returned by the camera collection, the preview image is extracted from the first image queue by the fast frame returning module frame by frame and transmitted to the image processing module. Then, the image processing module performs image synthesis on the preview image transmitted by the fast frame returning module to obtain a corresponding synthesized image (such as a second image) and save it to the second image queue. In some embodiments, the preview image in the first image queue can be dequeued because of the fast frame returning module, or can be dequeued because the queue is full and there is a new preview image entering the queue.

[0181] It can be understood that since the preview image 1 is the first frame of preview image collected by the camera after the first click operation, there is no image that can be synthesized in the sequence, so after the preview image 1 is processed by the image processing module alone, the first image is obtained. That is, the image content of the first image is the same as that of the preview image 1 (the first frame of preview image).

[0182] Subsequently, after the camera collects the preview image 2 (the second frame of preview image), the fast frame returning module transmits the preview image 2 (the second frame of preview image) to the image processing module, and the image processing module performs image synthesis on the preview image 2 (the second frame of preview image) and the first image to obtain the second image 1.

[0183] Similarly, the fast frame returning module transmits the preview image 3 (the third frame of preview image) to the image processing module, and the image processing module performs image synthesis on the preview image 3 (the third frame of preview image) and the second image 1 to obtain the second image 2. The fast frame returning module transmits the preview image 4 (the fourth frame of preview image) to the image processing module, and the image processing module performs image synthesis on the preview image 4 and the second image 2 to obtain the second image 3. The subsequent images are synthesized in the same way, which will not be described here.

[0184] It can be seen that the first image corresponding to the preview request 1 (the first preview request) is displayed, the second image 1 corresponding to the preview request 2 (the second preview request) is displayed, the second image 2 corresponding to the preview request 3 (the third preview request) is displayed, and so on.

[0185] Then, the image processing module can transmit the first image and the second image in the second image queue to the selection module. The selection module selects whether to display the first image and the second image or to store the first image and the second image in the gallery after format conversion in the picture generation module. In the embodiment of the present application, the image corresponding to the preview request needs to be displayed, and the image corresponding to the photographing request does not need to be displayed.

[0186] Therefore, when the image transmitted by the image processing module is the first image and the second image corresponding to the preview request, the selection module returns the first image and the second image to the camera application in the upper layer through the display module for display.

[0187] In addition, if the image transmitted by the image processing module is the third image corresponding to the photographing request, the selection module transmits the third image to the picture generation module for format conversion and then returns the third image to the gallery application in the upper layer for storage in the gallery. In some embodiments, the storage in the gallery can be returned to the camera application for storage in the gallery. Alternatively, the storage in the gallery can be directly returned to the gallery application for storage in the gallery. The embodiments of the present application are illustrated by returning to the gallery application for storage in the gallery.

[0188] Then, after the user observes the second image displayed by the camera application and finds a satisfactory image effect, the user can perform a second click operation (for example, the second click operation 113 shown in the figure) on the photographing shutter, triggering the electronic device to end photographing. Figure 3

[0189] That is, after the camera application receives the second click operation of the user on the photographing shutter, the camera application starts to issue a corresponding photographing request to the camera service in the lower layer. It can be understood that, because the second click operation triggers the end of photographing, the photographing request issued by the camera application is the last request in the request queue.

[0190] After the camera service receives the photographing request issued by the camera application, the camera service also issues the photographing request to the frame output module in the camera HAL. At this time, because the exposure time is as long as 10 seconds, there can be a preview request that has not been processed before the photographing request.

[0191] As shown in FIG. 6, the camera service issues the photographing request to the frame output module in the camera HAL, and the frame output module processes the photographing request. Figures 7-8 ​As shown, the preview requests represented by dashed lines in the request queue are those that were not processed before the photo capture request. An unprocessed preview request can be understood as a preview request for which the camera has not yet completed capturing the corresponding preview image. That is, in this embodiment, as long as the camera has not captured the preview image corresponding to the preview request, it indicates that the preview request is an unprocessed preview request.

[0192] Correspondingly, the images represented by dashed lines in the first image queue are the preview images that correspond one-to-one with these unprocessed preview requests, which are the frames being generated in this embodiment of the application.

[0193] It should be noted that unprocessed preview requests are actually sent by the upper-layer camera application, but are still queued for processing at the lower layer. Therefore, the request queue does include these preview requests. However, the frames being generated are actually images for which the corresponding data has not yet been acquired, so the images represented by the dashed lines should not actually be in the first or second image queue.

[0194] Understandable. Figures 7-8 The images are shown with dashed lines for ease of understanding and description, and do not indicate that these images actually exist in the first or second image queue. However, the preview requests represented by the dashed lines are real requests that have not yet been processed.

[0195] In the traditional approach, after the upper-layer camera application sends a photo-taking request, the request will wait in the request queue for processing. Figure 9 A flowchart of a traditional image capturing method is shown.

[0196] like Figure 9 As shown, traditionally, after the camera captures an image, the camera driver directly transmits it to the image processing module. Therefore, if there are still unprocessed preview requests after the user triggers the end of the photo-taking request, the camera will only begin processing the photo-taking request after those unprocessed preview requests have been completed.

[0197] That is, such as Figure 9As shown, after the camera application issues the photograph request, the camera has not yet captured the p frames of preview images after the preview image i, so the p preview requests after the preview request i are all preview requests that have not been processed. Therefore, the conventional method needs to wait until the camera captures the p frames of preview images after the preview request i before indicating that the p preview requests after the preview request i have been processed. Further, the electronic device can start processing the photograph request only after the p preview requests after the preview request i have been processed. That is, according to the time sequence of the preview requests, the conventional method needs to wait until the preview request i+p (the (i+p)th preview request) has also been processed before responding to the photograph request.

[0198] Therefore, the more preview requests that have not been processed before the photograph request, that is, the more preview requests after the preview request i, or the larger the value of p, the longer the electronic device takes to output the image, the lower the efficiency of outputting the image, and the longer the user needs to wait for the image. That is, as shown in FIG. 1, the conventional method needs to wait until the camera captures the (i+p)th preview image (the preview image i+p) after the camera application issues the photograph request before capturing the photograph image corresponding to the photograph request to perform image synthesis to obtain the third image and return the third image to the gallery. Figure 9

[0199] However, in the embodiments of the present application, in order to improve the efficiency of outputting the image and reduce the waiting time of the user, after the lower layer receives the photograph request issued by the camera application, the frame output module sends the photograph request to the fast frame return module. The fast frame return module responds to the photograph request of the upper camera application.

[0200] In the embodiments of the present application, after the fast frame return module receives the photograph request, the fast frame return module immediately responds to the photograph request to determine whether there is a preview image that has not been sent to the display in the first image queue. According to the current processing speed of the electronic device, if there is a preview image that has not been sent to the display in the first image queue, there is usually only one frame of the preview image. This frame of the preview image is usually the latest preview image captured by the camera, that is, the last frame of the preview image in the first image queue. Therefore, in some embodiments, the fast frame return module can directly extract the last frame of the preview image in the first image queue.

[0201] As shown in FIG. 2, because the preview images after the preview image i (the ith frame of the preview image) in the first image queue are all indicated by dashed lines, that is, there are actually no preview images, the last frame of the preview image is the preview image i (the ith frame of the preview image). Therefore, the fast frame return module can directly extract the preview image i (the ith frame of the preview image). Figures 7-8

[0202] ​​If the last preview image in the first image queue is an image that has not yet been displayed (i.e., there is a preview image that the camera has captured but has not yet been displayed), then the fast frame return module transmits this last preview image and the photo capture request to the image processing module. The image processing module then combines the last preview image with the preceding preview image to obtain a third image. Finally, the image processing module sends this combined third image to the selection module.

[0203] After receiving the third image from the image processing module, the selection module, since this third image is a composite image corresponding to the photo request, sends it to the image generation module. The image generation module then performs format conversion on the third image to obtain a format-converted third image, which is returned to the application and saved to the image library.

[0204] Understandably, the third image stored in the photo library is the image obtained by the user using the light painting shutter mode. For example, the third image could be... Figure 2 The star trail image shown. Understandably, due to... Figure 2 It is a star trail image that has been processed into black and white. Therefore, if the third image stored in the image library has not been processed into black and white, then the third image in the image library can be a color star trail image.

[0205] refer to Figure 7 The last preview image is preview image i, but preview image i (the i-th frame preview image) is the latest image captured by the camera and has not yet been sent to the image processing module for image synthesis and display. At this time, preview image i (the i-th frame preview image) is a preview image that has already been generated and does not require additional waiting for frame generation. Therefore, in order to quickly generate images while ensuring the fusion of more image information, the fast frame return module can extract preview image i (the i-th frame preview image) and send it to the image processing module for image synthesis to obtain the corresponding second image i-1. Then, the image processing module transmits this second image i-1 to the selection module. Since the second image i-1 is the image corresponding to the photo request, the selection module passes the second image i-1 to the image generation module for format conversion and returns it to the upper layer for storage in the image library.

[0206] If the last preview image in the first image queue is a preview image that has already been transmitted to the image processing module for processing, then this last preview image has already been synthesized and displayed (i.e., there are no preview images that the camera has captured but have not yet been displayed). In other words, all preview images in the first image queue have been synthesized and displayed in a timely manner, indicating that there are currently no preview images that can be synthesized without waiting.

[0207] Meanwhile, because the second image synthesized by the image processing module is determined by the selection module to be displayed or stored in the gallery, the selection module usually retains the latest second image. Therefore, in order to quickly output the image, the fast frame returning module can directly send an image generation instruction to the selection module, and the selection module directly sends the second image newly transmitted by the image processing module to the image generation module for format conversion and then returns to the application layer to be stored in the gallery in response to the image generation instruction.

[0208] Reference Figure 8 The last preview image is the preview image i (the i-th preview image), and the corresponding synthesized image is the second image i-1. This second image i-1 is the currently latest displayed second image received by the selection module from the image processing module. Then, the selection module directly sends this second image i-1 to the image generation module in response to the image generation instruction. After format conversion by the image generation module, it is stored in the gallery.

[0209] In summary, by comparing Figure 7 , Figure 8 and Figure 9 , it can be seen that in the embodiments of the present application, no matter which of the following two cases Figure 7 and Figure 8 occurs, the second image i-1 will become the third image corresponding to the photographing request in advance. Thus, the embodiments of the present application do not need to waste a lot of time to sequentially wait for the preview images after the preview image i+1, the preview image i+2, …, the preview image i+p to be output, as in the traditional way shown in Figure 9 .

[0210] That is, the difference between the embodiments of the present application and the traditional method shown in Figure 9 is that after the camera application issues a photographing request, the embodiments of the present application directly give up waiting for the preview image after the preview image i (the i-th preview image) to be output.

[0211] Therefore, compared with the traditional photographing method, the embodiments of the present application can realize fast image output. The image output speed will be faster, thereby improving the image output efficiency and reducing the waiting time of the user to improve the user experience.

[0212] In summary, in the embodiments of the present application, once the camera application issues a photographing request, the fast frame returning module quickly responds to the photographing request, instead of sequentially waiting for the processing of the photographing request as in the traditional way. The embodiments of the present application process the photographing request in advance through the fast frame returning module, give up the frame being output, and directly send the already output frame (i.e., the preview image collected by the camera) to the image generation module for format conversion to obtain the third image that can be stored in the gallery. As Figures 7-8As shown, the preview image i+1, the preview image i+2, …, and the preview image i+p are abandoned, that is, the waiting is abandoned. In this way, the application embodiment can reduce the out-of-picture time, realize fast out-of-picture, thereby reducing the waiting time of the user, and improving the user experience.

[0213] In some embodiments, in addition to image synthesis, the image processing module can also perform image compensation, image correction, image denoising, and other image processing on the image, and the application embodiment does not make any limitation on this.

[0214] As shown, the application embodiment shows a flowchart of another image shooting method. Figures 10-11

[0215] Since each request issued by the camera application needs to have a corresponding image returned, if the frames being produced are directly discarded without feedback to the upper layer, it will cause some preview requests issued by the camera application to have no corresponding returned images, thereby easily causing other unnecessary problems. Therefore, in order to avoid unnecessary problems caused by preview requests without corresponding returned images, the application embodiment fills in virtual data and a frame discarding flag for the frames being produced that are abandoned and returns them to the camera application of the upper layer. The virtual data includes virtual image data (buffer) and image metadata (metadata). The virtual image data and the image metadata ensure that the frame image can be successfully returned to the upper layer.

[0216] Meanwhile, after the camera application receives the frame image, the camera application can determine according to the frame discarding flag that the frame image is an incorrect frame, that is, an image that does not need to be displayed, so that the camera application directly abandons to display the frame image.

[0217] Specifically, in the application embodiment, after the upper-layer gallery application receives the third image (for example, the second image i-1 in Figures 7-8 , that is, the third image in the application embodiment) returned by the lower layer, the gallery application (or the camera application, illustrated as the gallery application) issues a flag bit to the fast frame returning module. The gallery application feeds back to the fast frame returning module through the issued flag bit that the third image has been received.

[0218] Further, the fast frame returning module generates a corresponding virtual preview image for each unprocessed preview request in response to the flag bit. The virtual preview image is generated by filling in virtual data, that is, filling in virtual image data and image metadata. Meanwhile, the virtual preview image also includes a frame discarding flag.

[0219] For example, Figure 10 ​The p preview requests after the preview request i are preview requests that are not processed. That is, the camera has not captured the p frames of preview images after the preview image i (that is, the preview image i+1, the preview image i+2, …, and the preview image i+p have not been captured). Therefore, the fast frame return module generates a corresponding virtual preview image for each of the p preview requests in response to the flag, thereby ensuring that the camera application has a corresponding return image for each of the p preview requests.

[0220] Meanwhile, the virtual preview images all carry the frame discard flag. Then, after the camera application receives the virtual preview images corresponding to the p preview requests, the camera application can discard the virtual preview images and prevent the display of incorrect frames.

[0221] That is, in the embodiment of the present application, because the fast frame return module responds to the photographing request in advance, the image capture is already complete. However, in the actual capture process, there are still preview requests that are not processed (that is, the p preview requests after the preview request i), and the capture is not actually complete. Therefore, if the capture is not further completed, the capture process may be abnormal.

[0222] Therefore, the embodiment of the present application prevents the capture process from being abnormal by using the flag sent by the upper layer to instruct the fast frame return module to complete the capture. Therefore, after the fast frame return module receives the flag sent by the upper layer, the fast frame return module responds to the flag to start the completion work of the capture, that is, the fast frame return module generates a virtual preview image filled with virtual image data, image metadata, and a frame discard flag for the preview request that is not processed.

[0223] The virtual image data and image metadata filled in can be configured according to actual requirements, and the embodiment of the present application does not make any limitation. The frame discard flag can also be agreed according to actual requirements, for example, an error flag can be added to the virtual preview image to indicate that the image frame is an incorrect frame. Therefore, not only can the fast frame return module ensure that each preview request has a return image for the upper layer, but also can inform the upper layer not to display the image by using the frame discard flag, thereby avoiding the display of incorrect frames.

[0224] In some embodiments, after the image capture process starts, the auto exposure module also starts to determine exposure parameters according to requirements for use in the capture. Therefore, the completion work of the fast frame return module should also include interrupting the exposure determination work of the auto exposure module.

[0225] Therefore, as described above, the fast frame return module can generate a virtual preview image for each of the p preview requests in response to the flag, thereby ensuring that the camera application has a corresponding return image for each of the p preview requests. Figures 10-11As shown, after the fast frame return module receives the flag bit issued by the gallery application (or the camera application), in order to avoid the automatic exposure module from continuously making exposure time decisions for the current image shooting, the fast frame return module simultaneously issues an interrupt instruction to the automatic exposure module in response to the flag bit. The fast frame return module interrupts the exposure decision of the automatic exposure module through the interrupt instruction, and recycles the corresponding resources. At the same time, the frame captured by the camera is stopped and discarded. In this way, it is ensured that the image shooting ends in advance and does not cause other unnecessary problems due to abnormalities.

[0226] In the embodiments of the present application, the electronic device can include at least one of a mobile phone, a foldable electronic device (e.g., a foldable mobile phone), a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device, a camera, a camcorder, a video recorder, or a smart city device. The embodiments of the present application do not specially limit the specific type of the electronic device.

[0227] As shown in the Figure 12 embodiments of the present application, a structural schematic diagram of an electronic device 1200 is shown.

[0228] The electronic device 1200 can include a processor 1210, an external memory interface 1220, an internal memory 1221, a universal serial bus (USB) joint 1230, a charge management module 1240, a power management module 1241, a battery 1242, an antenna 1201, an antenna 1202, a mobile communication module 1250, a wireless communication module 1260, an audio module 1270, a loudspeaker 1270A, a receiver 1270B, a microphone 1270C, a headset interface 1270D, a sensor module 1280, a key 1290, a motor 1291, an indicator 1292, a camera module 1293, a display screen 1294, and a subscriber identification module (SIM) card interface 1295, etc. The sensor module 1280 can include a pressure sensor 1280A and a touch sensor 1280B.

[0229] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 1200. In some other embodiments of the present application, the electronic device 1200 can include more or fewer components than those shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software or a combination of software and hardware.

[0230] The processor 1210 can include one or more processing units, for example: the processor 1210 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0231] The processor can generate operation control signals according to instruction operation codes and timing signals, complete the control of fetching instructions and executing instructions. For example, the image shooting method described in the embodiments of the present application can be implemented by the processor 1210.

[0232] The memory can also be provided in the processor 1210, used to store instructions and data. In some embodiments, the memory in the processor 1210 can be a cache memory. The memory can save instructions or data that have been used or used frequently by the processor 1210. If the processor 1210 needs to use the instructions or data, it can directly call from the memory. Avoiding repeated access, reducing the waiting time of the processor 1210, thus improving the efficiency of the system.

[0233] In some embodiments, the processor 1210 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. The processor 1210 can connect modules such as a touch sensor, an audio module, a wireless communication module, a display screen, a camera module, etc. through at least one of the above interfaces.

[0234] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 1200. In some other embodiments of the present application, the electronic device 1200 can also use different interface connection modes or a combination of multiple interface connection modes in the above embodiments.

[0235] The external memory interface 1220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 1200. The external memory card communicates with the processor 1210 through the external memory interface 1220 to realize the data storage function. For example, music, video, etc. files are saved in the external memory card. Or the music, video, etc. files are transmitted from the electronic device to the external memory card.

[0236] The internal memory 1221 can be used to store computer executable program code including instructions. The internal memory 1221 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required by a function (such as a sound playing function, an image playing function, etc.), and the like. The data storage area can store data created during use of the electronic device 1200 (such as audio data, a phonebook, etc.), and the like. In addition, the internal memory 1221 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like. The processor 1210 executes various functions, methods, or data processing of the electronic device 1200 by running instructions stored in the internal memory 1221 and / or instructions stored in a memory disposed in the processor.

[0237] The electronic device 1200 can implement a display function through a GPU, a display screen 1294, an application processor, and the like. The GPU is a microprocessor for image processing, connected to the display screen 1294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 1210 can include one or more GPUs that execute program instructions to generate or change display information. For example, the star trail image (e.g., the first image for preview and the second image stored in the gallery) generated by the embodiments of the present application can be rendered using the GPU and displayed using the display screen 1294.

[0238] The display screen 1294 is used to display images, videos, and the like. In some embodiments, it can be used to display a captured star trail image. The display screen 1294 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), and the like. In some embodiments, the electronic device 1200 can include one or more display screens 1294.

[0239] The electronic device 1200 can implement a camera function through the camera module 1293, an ISP, a video codec, a GPU, a display screen 1294, and an application processor (AP), a neural network processing unit (NPU), and the like. For example, the camera module 1293 and the ISP are used to implement the photographing and generation of the star trail image (for example, a first image for preview and a second image for storage in a gallery) in the embodiments of the present application.

[0240] The camera module 1293 can be used to collect color image data and depth data of a photographed object. The ISP can be used to process the color image data collected by the camera module 1293. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element (i.e., image sensor) through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into a visible image to the naked eye. The ISP can also optimize the algorithm for noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the photographed scene.

[0241] In some embodiments, the ISP can be disposed in the camera module 1293.

[0242] In some embodiments, the camera module 1293 can be composed of a color camera module and a 3D sensing module.

[0243] In some embodiments, the photosensitive element of the camera of the color camera module can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, or the like format image signal.

[0244] In some embodiments, the 3D sensing module can be a time of flight (TOF) 3D sensing module or a structured light 3D sensing module. The structured light 3D sensing is an active depth sensing technology, and the basic components of the structured light 3D sensing module can include an infrared (IR) emitter, an IR camera module, and the like. The working principle of the structured light 3D sensing module is to first emit a specific pattern of light spots to the object being photographed, then receive the light spot pattern coding on the surface of the object, and then compare the differences between the original projected light spots, and calculate the three-dimensional coordinates of the object by using the principle of triangle. The three-dimensional coordinates include the distance between the electronic device 1200 and the object being photographed. The TOF 3D sensing is an active depth sensing technology, and the basic components of the TOF 3D sensing module can include an infrared (IR) emitter, an IR camera module, and the like. The working principle of the TOF 3D sensing module is to calculate the distance (i.e., depth) between the TOF 3D sensing module and the object being photographed by the time of infrared return, to obtain a 3D depth map.

[0245] The structured light 3D sensing module can also be applied to face recognition, motion game consoles, industrial machine vision detection, and the like. The TOF 3D sensing module can also be applied to game consoles, augmented reality (AR) / virtual reality (VR), and the like.

[0246] In other embodiments, the camera module 1293 can also be composed of two or more cameras. The two or more cameras can include a color camera, which can be used to collect color image data of the object being photographed. The two or more cameras can use stereo vision technology to collect depth data of the object being photographed. The stereo vision technology is based on the principle of human eye parallax. Under natural light, the same object is photographed from different angles by two or more cameras, and then the distance information between the electronic device 1200 and the object being photographed, i.e., the depth information, is obtained by triangulation method.

[0247] In some embodiments, the electronic device 1200 can include one or more camera modules 1293. Specifically, the electronic device 1200 can include one front camera module 1293 and one rear camera module 1293. The front camera module 1293 can be used to collect color image data and depth data of the photographer facing the display screen 1294, and the rear camera module can be used to collect color image data and depth data of the object being photographed (such as a star track) facing the photographer.

[0248] In some embodiments, the CPU or GPU or NPU in the processor 1210 can process the color image data and depth data collected by the camera module 1293. In some embodiments, the NPU can identify the color image data collected by the camera module 1293 (specifically the color camera module) by a neural network algorithm, such as a convolutional neural network algorithm (CNN), based on a skeleton point recognition technology, to determine the skeleton points of the photographed person. The CPU or GPU can also run the neural network algorithm to determine the skeleton points of the photographed person according to the color image data.

[0249] The digital signal processor is used to process digital signals, and can also process other digital signals. For example, when the electronic device 1200 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0250] The video codec is used to compress or decompress digital video. The electronic device 1200 can support one or more video codecs. In this way, the electronic device 1200 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0251] The NPU is a neural-network (NN) computing processor, which can quickly process input information by drawing on the structure of a biological neural network, such as the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 1200 can implement intelligent cognitive applications, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0252] The pressure sensor 1280A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 1280A can be disposed on the display screen 1294. The pressure sensor 1280A can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 1280A, the capacitance between the electrodes changes. The electronic device 1200 determines the intensity of the force according to the change in capacitance. When a touch operation is applied to the display screen 1294, the electronic device 1200 detects the intensity of the touch operation according to the pressure sensor 1280A. The electronic device 1200 can also calculate the position of the touch according to the detection signal of the pressure sensor 1280A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to the short message application icon, an instruction to view short messages is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0253] The touch sensor 1280B, also referred to as a "touch device". The touch sensor 1280B can be disposed on the display screen 1294, and the touch sensor 1280B and the display screen 1294 form a touch screen, also referred to as a "touch screen". The touch sensor 1280K is configured to detect a touch operation applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 1294. In other embodiments, the touch sensor 1280K can also be disposed on the surface of the electronic device 1200, which is different from the position of the display screen 1294.

[0254] In the embodiments of the present application, the electronic device 1200 can detect the click operation of the user on the shooting shutter and the opening operation of the user to open the camera application through the pressure sensor 1280A and the touch sensor 1280B.

[0255] The USB receiver 1230 is an interface that conforms to the USB standard specification and can be used to connect the electronic device 1200 and a peripheral device. Specifically, the USB receiver 1230 can be a Mini USB connector, a Micro USB connector, a USB Type C connector, etc. The charging management module 1240 is configured to receive a charging input from a charger. The charger can be a wireless charger or a wired charger. The power management module 1241 is configured to connect the battery 1242 and the charging management module 1240 to the processor 1210.

[0256] The wireless communication function of the electronic device 1200 can be implemented by the antenna 1201, the antenna 1202, the mobile communication module 1250, the wireless communication module 1260, the modem processor, the baseband processor, and the like.

[0257] The electronic device 1200 can implement an audio function by the audio module 1270, the speaker 1270A, the receiver 1270B, the microphone 1270C, the earphone interface 1270D, the application processor, and the like. For example, music playing, recording, and the like.

[0258] The key 1290 can include a power-on key, a volume key, and the like. The motor 1291 can generate a vibration prompt. The indicator 1292 can be an indicator light, which can be used to indicate a charging state, a power change, and can also be used to indicate a message, a missed call, a notification, and the like. The SIM card interface 1295 is used to connect a SIM card.

[0259] It should be noted that the image capturing method in the following embodiments can be implemented in the electronic device 1200 with the above hardware structure.

[0260] Another embodiment of the present application provides an electronic device, comprising: a camera, one or more processors and a memory. The camera and the memory are coupled with the processor respectively; the memory stores one or more computer program codes, and the computer program codes comprise computer instructions; when the processor executes the computer instructions, the electronic device implements the image capturing method in any of the above embodiments.

[0261] Another embodiment of the present application provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor in an electronic device, the electronic device implements the image capturing method in any of the above embodiments.

[0262] The embodiments of the present application further provide a computer program product, which, when running on a computer, causes the computer to perform the functions or steps in the above method embodiments.

[0263] The embodiments of the present application further provide a chip system, as shown in the following figure: Figure 13 The chip system 1300 comprises at least one processor 1301 and at least one interface circuit 1302. The processor 1301 and the interface circuit 1302 can be interconnected by a line. For example, the interface circuit 1302 can be used to receive signals from other devices (such as the memory of a computer). For another example, the interface circuit 1302 can be used to send signals to other devices (such as the processor 1301).

[0264] For example, the interface circuit 1302 can read instructions stored in the memory and transmit the instructions to the processor 1301. When the instructions are executed by the processor 1301, the computer can be caused to perform various steps in the above embodiments. Of course, the chip system can also include other discrete devices, and the embodiments of the present application do not make specific limitations thereto.

[0265] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0266] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other ways. For example, the above-described device embodiments are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0267] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place, or can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0268] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0269] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0270] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image capturing method characterized by, The application is applied to an electronic device, the electronic device comprises a camera, the electronic device supports a preset shooting mode, and the preset shooting mode comprises a star trail shooting mode; the method comprises the following steps: In response to a user opening operation of the preset shooting mode in a camera application, a shooting interface corresponding to the preset shooting mode is displayed; wherein the shooting interface corresponding to the preset shooting mode comprises a shooting shutter; A first click operation of the user on the shooting shutter is received; At a first time, a first image is displayed; wherein the first image is obtained by synthesizing the first frame of preview images captured by the camera based on the first preview request generated in response to the first click operation; the camera captures one frame of preview images based on each preview request; At a second time, a second image is displayed; wherein the second image is obtained by synthesizing the first to k frames of preview images captured by the camera based on the first to k preview requests generated in response to the first click operation; the second time is after the first time; k>1, and k is a positive integer; At a third time, a second click operation of the user on the shooting shutter is received, and a third image is saved; wherein the third time is after the second time; at the third time, the i+pth preview request has been generated in response to the first click operation, and the i frame of preview images has been captured by the camera; the third image is obtained by synthesizing the first to i frames of preview images captured by the camera; i>k, and p is a positive integer; And at the third time, the electronic device displays a fourth image; the fourth image is obtained by synthesizing the first to j frames of preview images captured by the camera based on the first to j preview requests generated in response to the first click operation; k 2. The method of claim 1, wherein, The saving of the third image comprises: The third image is format-converted, and the third image after format-conversion is saved; wherein the third image before format-conversion is in YUV format, and the third image after format-conversion is in JPEG format.

3. The method according to claim 1 or 2, characterized in that, The synthesizing of the first to k frames of preview images captured by the camera to obtain the second image comprises: The second image is obtained by synthesizing a fifth image and the kth frame of preview image; wherein the fifth image is obtained by synthesizing the first to k-1 frames of preview images.

4. The method according to claim 1 or 2, characterized in that, The third image is obtained by synthesizing the first to i frames of preview images captured by the camera, comprising: The third image is obtained by synthesizing a sixth image and the i frame of preview image; wherein the sixth image is obtained by synthesizing the first to i-1 frames of preview images.

5. The method according to claim 1 or 2, characterized in that, The method further comprises: Virtual preview images corresponding to the p preview requests after the i preview request are respectively generated; wherein the virtual preview image comprises virtual image data, image metadata and a discard frame flag.

6. The method according to claim 1 or 2, characterized in that, The method further comprises interrupting an exposure decision after the third image is obtained.

7. The method of claim 1, wherein, The electronic device comprises an automatic exposure module, a frame output module, a fast frame return module, an image processing module, a selection module and a display module; The camera application continuously issues the preview request to the frame output module in response to the first click operation; The frame output module periodically sends the preview request to the camera to drive the camera to collect a frame of preview image according to the exposure time of the camera; wherein the preview image is saved in a first image queue according to the order of collection; the exposure time is decided by the automatic exposure module according to the ambient brightness and sent to the frame output module; The fast frame return module extracts the preview image from the first image queue frame by frame and transmits it to the image processing module, synthesizes the first frame of preview image to obtain the first image by the image processing module, and synthesizes the first to k frames of preview image to obtain the second image by the image processing module; The image processing module transmits the first image and the second image to the camera application via the selection module and the display module, and displays the first image and the second image by the camera application.

8. The method of claim 7, wherein, The electronic device further comprises a picture generation module; The camera application sends a photographing request to the fast frame return module via the frame output module in response to the second click operation at the third time; If the preview image after the jth frame of the first to i frames of preview image has not been transmitted to the image processing module at the third time, the fast frame return module extracts the preview image after the jth frame and transmits it to the image processing module in response to the photographing request; k < j < i, j is a positive integer; The image processing module synthesizes the preview image after the jth frame with a fourth image to obtain a third image; wherein the fourth image is synthesized by the image processing module based on the first to j frames of preview image; The image processing module transmits the third image to the picture generation module via the selection module, and the picture generation module saves the third image after format conversion to the camera application.

9. The method of claim 8, wherein, The method further comprises: If the first to i frames of preview image have all been extracted frame by frame and transmitted to the image processing module at the third time, the fast frame return module sends a picture generation instruction to the selection module in response to the photographing request. The selection module transmits the third image to the picture generation module, and the picture generation module saves the third image after format conversion to the camera application.

10. The method according to any one of claims 7-9, characterized in that, The method further comprises: The fast frame return module receives the flag bit issued by the camera application, and the flag bit is issued when the camera application receives the third image; The fast frame return module generates a corresponding virtual preview image for p preview requests after the i th preview request in response to the flag bit.

11. The method according to any one of claims 7-9, characterized in that, The method further comprises: The fast frame return module sends an interrupt instruction to the automatic exposure module in response to the flag bit issued by the camera application, instructing the automatic exposure module to interrupt the exposure decision.

12. An electronic device, comprising: It comprises: a camera, one or more processors and a memory, the camera and the memory being coupled with the processor respectively; the camera is configured to capture an image; the memory stores one or more computer program codes, the computer program codes comprising computer instructions; when the processor executes the computer instructions, the electronic device is caused to perform the image capturing method according to any one of claims 1-11.

13. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by the processor of the electronic device, causes the electronic device to perform the image capturing method according to any one of claims 1-11.

Images