Photographing processing method, electronic device, and computer-readable storage medium

Abstract

Disclosed in embodiments of the present application are a photographing processing method, an electronic device, and a computer-readable storage medium. The method comprises: an electronic device responding to a first photographing operation in a photographing preview interface to select a first photographing frame and issue a first manual exposure parameter to an image sensor, wherein the first photographing frame comprises a first image and a first manual exposure frame, the first manual exposure parameter is used for instructing the image sensor to obtain a second manual exposure frame on the basis of the first manual exposure parameter, and the second manual exposure frame is cached in a cache space; in response to a second photographing operation, selecting a second photographing frame, wherein the second photographing frame comprises a second image and a second manual exposure frame selected from the cache space, and the second photographing operation follows the first photographing operation; and generating a second captured image on the basis of the second photographing frame. In this way, upon receiving a photographing operation, manual exposure parameters are set and issued in advance for next photographing, and the latency brought by the manual exposure parameters taking effect is hidden, thereby improving the photographing speed.

Description

Photo processing method, electronic device and computer-readable storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 29, 2024, with application number 202410389142.5 and application name “Photo processing method, electronic device and computer-readable storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of imaging technology, and in particular to a photographic processing method, an electronic device, and a computer-readable storage medium. Background Art

[0003] With the increasing popularity of electronic devices such as mobile phones and their increasingly powerful functions, more and more users are using the camera function of mobile phones and other electronic devices to take photos.

[0004] When electronic devices such as mobile phones capture High Dynamic Range (HDR) images, they need to fuse multiple images with different exposure levels each time they take a picture to improve the dynamic range of the image. Therefore, in addition to capturing a normally exposed image, each time they take a picture, they also need to set manual exposure parameters and capture at least one manually exposed frame based on the manual exposure parameters. Manually exposed frames have different exposure levels from normally exposed images and can be used to improve the dynamic range of the image. For example, manually exposed frames can include long-exposure images and short-exposure images.

[0005] However, it takes time for the manual exposure parameters to take effect, that is, there is a certain delay between the manual exposure parameter setting and the manual exposure frame output, which reduces the shooting speed. Summary of the Invention

[0006] The present application provides a photographing processing method, an electronic device and a computer-readable storage medium, which can solve the problem in the prior art of low photographing speed caused by a certain delay between manual exposure parameter setting and manual exposure frame output.

[0007] In a first aspect, an embodiment of the present application provides a photo processing method, in which an electronic device displays a photo preview interface; in response to a first photo operation on the photo preview interface, a first photo frame is selected, and a first manual exposure parameter is sent to the image sensor, the first photo frame including at least one first image and at least one first manual exposure frame, and the exposure amount of the first manual exposure frame is different from that of the first image; the first manual exposure parameter is used to instruct the image sensor to perform exposure processing according to the first manual exposure parameter to obtain a second manual exposure frame, and the second manual exposure frame is cached in a cache space; based on the first photo frame, a first photo image is generated; in response to a second photo operation on the photo preview interface, a second photo frame is selected, the second photo frame including at least one second image and at least one second manual exposure frame selected from the cache space, and the exposure amount of the second manual exposure frame is different from that of the second image; the second photo operation is performed after the first photo operation; and based on the second photo frame, a second photo image is generated.

[0008] As can be seen from the above technical solution, after receiving the first photo operation, the embodiment of the present application not only selects the first photo frame required for the first photo operation, but also sends the first manual exposure parameter to the image sensor in advance to control the image sensor to output the manual exposure frame required for the next photo, i.e., the second manual exposure frame, and stores it in the cache space. In this way, when the next photo operation (i.e., the second photo operation) is received, the second manual exposure frame can be directly selected from the cache space, without having to set the manual exposure parameters for the current photo and wait for the manual exposure parameters to take effect. The time between setting the manual exposure parameters and outputting the manual exposure frame is hidden, thereby improving the photo speed. In other words, by setting and sending the manual exposure parameters for the next photo in advance after receiving the photo operation, the time when the manual exposure parameters take effect is hidden, thereby improving the photo speed.

[0009] For example, each HDR capture requires multi-frame fusion of a normal exposure image, a long exposure image, and a short exposure image. These images have different exposure levels, and the manual exposure frame includes both long and short exposure images. In this case, in addition to capturing a normal exposure image, long and short exposure images are also required.

[0010] In the related art, electronic devices can be based on zero-delay photography (ZSL) technology. When displaying the photo preview interface, they pre-cache a normal exposure image in the ZSL queue; when receiving the user's photo operation, they send long exposure parameters and short exposure parameters to the image sensor to instruct the image sensor to output a long exposure image and a short exposure image according to the sent exposure parameters; finally, they perform multi-frame synthesis based on the long exposure image, the short exposure image, and the normal exposure image selected from the ZSL queue to obtain an HDR image for the current photo operation. However, after receiving the user's photo operation, the manual exposure parameters for the current photo are sent, and there is a certain delay between the manual exposure parameter setting and the manual exposure parameter taking effect (i.e., the image sensor outputs the exposure frame), resulting in a long time to obtain the multi-frame composite image of the current photo operation, affecting the photo speed. In particular, in a continuous shooting scenario, that is, a scenario where the user clicks the photo button multiple times in a short period of time, each photo needs to wait for the manual exposure parameters to take effect, which seriously affects the photo speed and reduces the continuous shooting speed.

[0011] In the embodiment of the present application, when the electronic device displays the photo preview interface, it pre-buffers the normal exposure image in the ZSL queue; after receiving the user's photo operation, it not only selects the photo frame required for the current photo, but also sends the manual exposure parameters to the image sensor in advance to instruct the image sensor to output the long exposure image and short exposure image required for the next photo operation, and caches them in the cache space. In this way, after receiving the next photo operation, the long exposure image and short exposure image required for the current photo can be directly selected from the cache space, without having to set the manual exposure parameters for the current photo and waiting for the manual exposure parameters to take effect. The time between setting the manual exposure parameters and the time when the manual exposure parameters take effect is hidden, thereby improving the photo speed. In particular, in the continuous shooting scenario, since the embodiment of the present application sets and sends the manual exposure parameters in advance when taking pictures, the effective time of the manual exposure parameters is hidden, thereby improving the photo speed in the HDR continuous shooting scenario.

[0012] In a possible implementation of the first aspect, before sending the first manual exposure parameter to the image sensor, the electronic device may further determine whether the first manual exposure frame is available for the next photo shoot; if the first manual exposure frame is not available for the next photo shoot, the electronic device may send the first manual exposure parameter to the image sensor to instruct the image sensor to output the manual exposure frame required for the next photo shoot according to the first manual exposure parameter; if the first manual exposure frame is available for the next photo shoot, in response to the second photo shooting operation, the first manual exposure frame and at least one second image may be selected as frames for the second photo shoot.

[0013] In an embodiment of the present application, the electronic device first determines whether the first manual exposure frame used in the current photo is available for the next photo. If it is available, there is no need to pre-set and send manual exposure parameters for the next photo, and the image sensor does not pre-output the manual exposure frame for the next photo. Instead, the first manual exposure frame used in the current photo is directly reused for the next photo. If it is not available, manual exposure parameters are set and sent in advance for the next photo. In this way, the same manual exposure frame can be reused for both photo operations, achieving manual exposure frame reuse, reducing the number of manual exposure parameter transmissions, and further improving photo capture speed.

[0014] In a possible implementation of the first aspect, when determining whether a first manual exposure frame is available for a next photograph, the electronic device may predict the time of the next photograph based on the photographing time and the photographing interval of the first photographing operation. If the time interval between the time of the next photograph or the timestamp of the reference frame of the next photograph and the timestamp of the first manual exposure frame is greater than a preset time threshold, the electronic device determines that the first manual exposure frame is unavailable for the next photograph. If the time interval between the time of the next photograph or the timestamp of the reference frame of the next photograph and the timestamp of the first manual exposure frame is less than or equal to the preset time threshold, the electronic device determines that the first manual exposure frame is available for the next photograph. The timestamp of the reference frame of the next photograph is determined based on the time of the next photograph. For example, after determining the time of the next photograph, the 100ms or 50ms before the time of the next photograph is used as the timestamp of the reference frame of the next photograph.

[0015] In this way, by predicting the shooting interval time, the shooting time of the current shot and the timestamp of the manual exposure frame, the manual exposure frames that are closer to the next shooting time are reused, and the manual exposure frames that are farther away from the next shooting time are not reused. Therefore, while ensuring the shooting effect, the frame reuse of the manual exposure frames can be achieved, thereby improving the shooting speed.

[0016] In one possible implementation of the first aspect, the electronic device may, in response to the second photographing operation, send a second manual exposure parameter to the image sensor, instructing the image sensor to perform exposure processing according to the second manual exposure parameter, thereby obtaining a third manually exposed frame, and buffering the third manually exposed frame in the buffer space. In other words, the electronic device can pre-set and send the manual exposure parameter for the next photograph each time the photograph is taken, thereby improving photographing speed.

[0017] Alternatively, in response to the second photo-taking operation, if the electronic device determines that the second manual exposure frame is unavailable for the next photo-taking operation, the electronic device may send a second manual exposure parameter to the image sensor to instruct the image sensor to perform exposure processing according to the second manual exposure parameter, thereby obtaining a third manual exposure frame, which is then cached in the cache space. In other words, the electronic device need not pre-set and send the manual exposure parameters for the next photo-taking operation each time the electronic device takes a photo. Instead, the electronic device may pre-set and send the manual exposure parameters for the current photo-taking operation only when the manual exposure frame for the current photo-taking operation is unavailable for the next photo-taking operation. This reduces the number of manual exposure parameter transmissions and further improves photo-taking speed.

[0018] In one possible implementation of the first aspect, part of the first image and part of the second image are the same image. In this way, in addition to enabling frame multiplexing of manually exposed frames, frame multiplexing of images different from manually exposed frames can also be enabled, thereby further improving the shooting speed.

[0019] In a possible implementation of the first aspect, when the electronic device displays the picture preview interface shown, it can cache images captured by the image sensor and corresponding to the picture preview interface in a cache space. In this case, the first image is the image cached in the cache space before the first picture operation, the first manual exposure frame is the image cached in the cache space before the first picture operation or before the second picture operation; and the second image is the image cached in the cache space before the second picture operation. In this way, by caching the first and second images in advance before receiving the picture operation, the image required for the current picture can be directly obtained from the cache space when the picture operation is received, without waiting for the image sensor to output a frame, further improving the picture taking speed. In addition, while achieving frame multiplexing of images different from the manual exposure frame, the multiplexed image is also cached in advance, which can further improve the picture taking speed.

[0020] For example, the first and second images are both normally exposed images cached by the electronic device in the ZSL queue, while the manually exposed frames are long- and short-exposure images. In this case, not only the long- and short-exposure images are reused, but also the normally exposed image. This frame reuse reduces the waiting time for the image sensor to output the image required for the current shot. Precaching images also reduces the waiting time for the image sensor to output the image required for the current shot, thereby increasing the speed of the shot.

[0021] In a possible implementation manner of the first aspect, the cache space is an image cache queue for zero-delay photography, that is, a ZSL image queue.

[0022] In one possible implementation of the first aspect, the first photographic frame includes a cached image in the cache space, or includes the cached image in the cache space and a historical cached image, wherein the historical cached image is an image that was cached in the cache space before the first photographing operation but was not cached in the cache space during the first photographing operation and is persistently stored in the storage medium; the second image is a cached image in the cache space, or an image that was cached in the cache space before the second photographing operation but was not cached in the cache space during the second photographing operation and is persistently stored in the storage medium. In this way, by caching the multiplexed image in the cache space or persistently storing the multiplexed image in the storage medium, frame multiplexing is achieved, thereby improving photographing speed.

[0023] In a possible implementation of the first aspect, when the electronic device is selecting a first frame for photographing, if there is a target cache image in the cache space, or there is first target information in the historical frame storage space, the target cache image or the image corresponding to the first target information is selected as the first manual exposure frame, and the historical frame storage space is used to store information of the photographic frames selected for historical photographing. The photographic frames selected for historical photographing are persistently stored in a storage medium, and the target cache image and the image corresponding to the first target information are manual exposure frames available for the current photographing; at least one first image is selected from the historical frame storage space and / or the cache space; for the image selected from the cache space, the selected image is deleted from the cache space, and the selected image is persistently stored in the storage medium, and the information of the selected image is stored in the historical storage space, and the selected image includes the first manual exposure frame and / or the first image.

[0024] In the embodiment of the present application, when selecting frames for a photo, after selecting the frames required for the photo, the selected frames are deleted from the cache, and the information of the frames is stored in the historical frame storage space, and the frames are persistently stored in the storage medium. In this way, even if the frames in the cache are deleted after the photo is selected, the frames used for the previous photo can still be obtained from the information of the frames stored in the historical frame storage space and the frames in the storage medium during the next photo, and the frames used for the previous photo can be reused, thereby achieving frame multiplexing and improving the photo capture speed. In addition, the storage of frames for the photos through the historical frame storage space and the storage medium is not affected by the size of the cache, and can ensure that the frame reuse rate is maintained while saving memory.

[0025] In one possible implementation of the first aspect, if the electronic device determines that the target cache image does not exist in the cache space and the first target information does not exist in the historical frame storage space, it sends a fourth manual exposure parameter to the image sensor to instruct the image sensor to perform exposure processing according to the fourth manual exposure parameter to obtain a manually exposed frame, and selects the manually exposed frame as the first manually exposed frame. In this way, if no reusable manually exposed frame exists, the electronic device can send the manual exposure parameter for the current photo after receiving the current photo operation to obtain the manual exposed frame required for the current photo.

[0026] In a possible implementation of the first aspect, when the number of frames between the manually exposed frame available for the second photo and the reference frame of the first photo operation is less than a preset number, or when the time interval between the timestamp of the manually exposed frame available for the second photo and the timestamp of the reference frame of the first photo operation is less than a preset threshold, this can achieve frame multiplexing while ensuring photo quality.

[0027] In one possible implementation of the first aspect, during the process of selecting a second frame for photographing, the electronic device may select a second manually-exposed frame from a cache, delete the second manually-exposed frame from the cache, persistently store the second manually-exposed frame in a storage medium, and store information about the second manually-exposed frame in a history storage space; select a second image from the cache, delete the second image from the cache, persistently store the second image in a storage medium, and store information about the second image in a history storage space; and / or select second target information from the history storage space, where the image corresponding to the second target information is the second image. In this way, even if the photographing frame in the cache is deleted after photographing and selecting a frame, the next photographing can still obtain the photographing frame used in the previous photographing by using the photographing frame information stored in the history frame storage space and the photographing frames in the storage medium, and reuse the photographing frame used in the previous photographing, thereby achieving frame multiplexing and improving photographing speed. Furthermore, by storing photographing frames in the history frame storage space and the storage medium, the storage of photographing frames is not limited by the size of the cache, ensuring a high frame reuse rate while conserving memory.

[0028] In a possible implementation of the first aspect, during the process of selecting a first frame for photographing, if the electronic device determines that a target cached image exists in a cache space, the electronic device selects the target cached image as a first manual exposure frame, where the target cached image is a manual exposure frame available for photographing the current photograph; selects at least one first image from the cache space; and maintains the first manual exposure frame and the first image cached in the cache space, waiting for removal by a storage mechanism of the cache space.

[0029] Accordingly, when selecting the second frame for photographing, the electronic device can select the second manual exposure frame and the second image from the cache space, and keep the second manual exposure frame and the second image cached in the cache space, waiting to be removed by the storage mechanism of the cache space.

[0030] In this way, after selecting the frames needed for the current photo from the cache, the frames in the cache are not deleted, but wait for the storage mechanism (such as the first-in-first-out mechanism) to remove them. The next time you take a photo, you can still retrieve the frames used for the previous photo from the cache, achieving frame reuse and improving photo taking speed.

[0031] In a possible implementation of the first aspect, when the electronic device is generating a first photographic image based on the first photographic frame, if an operation of viewing a thumbnail is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the first photographic frame is obtained from the storage medium, and the first photographic frame is processed by a photographic algorithm to obtain a first photographic image.

[0032] Accordingly, when the electronic device is generating a second photographic image based on the second photographic frame, if it detects an operation of viewing thumbnails, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the second photographic frame is obtained from the storage medium, and the second photographic frame is processed by a photographic algorithm to obtain a second photographic image.

[0033] In an embodiment of the present application, an electronic device does not immediately perform photo algorithm processing on photo frames during a photo capture process. Instead, the electronic device stores the photo frames in a storage medium and performs photo algorithm processing only after detecting a thumbnail viewing operation, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected. Thus, compared to continuously and uninterruptedly executing all photo algorithm processing during the photo capture process, the embodiment of the present application performs photo algorithm processing only after exiting the photo capture process, reducing computing and memory resource usage during the photo capture process, thereby alleviating problems such as device lag and device overheating, improving photo performance, and enhancing the user photo capture experience. In addition, by storing photo frames in a storage medium and eliminating the need to repeatedly write reused image frames to disk, not only disk space is saved, but also the time spent saving images to disk is reduced, thereby improving photo processing speed. In a possible implementation of the first aspect, the preset trigger condition includes at least one of the following: opening an image access application, viewing an image, sharing an image, editing an image, the system being idle, the screen being off or off, the running memory being above a first threshold, the device temperature being below a first temperature threshold, and the processor load being below a second threshold. In this way, when users need to view, edit or share image data, the photo algorithm processing can be performed in a timely manner, so that users can view the photo images in a timely manner; in addition, when the system status meets the conditions, that is, the system status such as running memory, device temperature and processor load meets the requirements, the photo algorithm processing can be automatically triggered, which can further alleviate problems such as device freezes and device heating.

[0034] In a second aspect, an embodiment of the present application provides a photo processing method, which is applied to an electronic device and includes:

[0035] In the photo preview mode, multiple frames of images captured by the image sensor are cached in the ZSL queue;

[0036] In response to a first photographing operation, selecting a first photographing frame from the ZSL queue, persistently storing the first photographing frame in a storage medium, storing metadata of the first photographing frame in a historical frame queue, and deleting the first photographing frame from the ZSL queue;

[0037] generating a first photographic image according to the first photographic frame in the storage medium;

[0038] In response to the second photographing operation, selecting a target image from the ZSL queue, selecting target metadata from the metadata of the first photographing frame in the historical frame queue, persistently storing the target image in a storage medium, and deleting the target image in the cache queue;

[0039] A second photographic image is generated based on the second photographic frame in the storage medium. The second photographic frame includes the target image and the photographic frame corresponding to the target metadata. Portions of the first photographic frame and the second photographic frame are identical, meaning that the same image frame can be reused for both photographic operations, achieving frame multiplexing.

[0040] As can be seen from the above technical solution, in the embodiment of the present application, when selecting frames for a photo, after selecting the frames for the photo required for the photo, the selected frames for the photo are deleted from the ZSL queue, and the information of the frames for the photo is stored in the historical frame queue, and the frames for the photo are persistently stored in the storage medium. In this way, even if the frames for the photo are deleted from the ZSL queue after the frames for the photo are selected, the metadata of the frames for the photo stored in the historical frame queue and the frames for the photo in the storage medium can still be used to obtain the frames for the photo used in the previous photo the next time the photo is taken, and the frames for the photo used in the previous photo can be reused, thereby achieving frame multiplexing and improving the speed of photo taking. In addition, the frame multiplexing of the frames for the photo is achieved through the historical frame queue and the storage medium, which is not limited by the length of the ZSL queue, and can ensure that the frame multiplexing rate is guaranteed while saving memory.

[0041] In a possible implementation of the second aspect, when the electronic device is generating a first photographic image based on the first photographic frame, if an operation of viewing a thumbnail is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the first photographic frame is obtained from the storage medium, and the first photographic frame is processed by a photographic algorithm to obtain a first photographic image.

[0042] Accordingly, when the electronic device is generating a second photographic image based on the second photographic frame, if it detects an operation of viewing thumbnails, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the second photographic frame is obtained from the storage medium, and the second photographic frame is processed by a photographic algorithm to obtain a second photographic image.

[0043] In the embodiment of the present application, the electronic device does not immediately perform the photo algorithm processing on the photo frames during the photo shooting process. Instead, the photo frames are stored in a storage medium and the photo algorithm processing is performed after the operation of viewing thumbnails is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected. In this way, compared with continuously and uninterruptedly executing all photo algorithm processing during the photo shooting process, the embodiment of the present application performs the photo algorithm processing only after the photo shooting process is exited, which reduces the computing resources and memory resources occupied during the photo shooting process, thereby alleviating problems such as device freezing and device heating, improving photo shooting performance, and enhancing the user's photo shooting experience.

[0044] In one possible implementation of the second aspect, the preset trigger conditions include at least one of the following: opening an image access application, viewing an image, sharing an image, editing an image, the system being idle, the screen being off or off, the running memory being above a first threshold, the device temperature being below a first temperature threshold, and the processor load being below a second threshold. In this way, when a user needs to view, edit, or share image data, the photo algorithm processing can be performed promptly, allowing the user to view the captured image in a timely manner. Furthermore, the photo algorithm processing can be automatically triggered when the system status meets the conditions, namely, when the running memory, device temperature, and processor load meet the requirements, further alleviating issues such as device lag and overheating.

[0045] In a possible implementation of the second aspect, in an HDR photography scenario, the images cached in the ZSL queue are typically normally exposed images. To improve the dynamic range of the image, the electronic device also needs to control the image sensor to output at least one manually exposed frame, such as a long-exposure image or a short-exposure image. In this case, after receiving the first photography operation, the electronic device can not only select a frame for the first photography, but also set manual exposure parameters for the next photography, and send the manual exposure parameters for the next photography to the image sensor, instructing the image sensor to output the manual exposure frame for the next photography according to the manual exposure parameters and cache it in the ZSL queue. In this way, after receiving the next photography operation, i.e., the second photography operation, the electronic device can select the manually exposed frames and images of other exposure amounts (e.g., normally exposed images) available for the current photography from the ZSL queue. There is no need to set the manual exposure parameters for the current photography after receiving the second photography operation, and wait for the manual exposure parameters for the current photography to take effect. The time between setting the manual exposure parameters and taking effect is hidden, further improving the HDR photography speed.

[0046] That is, by setting and issuing manual exposure parameters for the next photo in advance after receiving a photo operation, the time when the manual exposure parameters take effect is hidden, thereby improving the photo taking speed.

[0047] In a possible implementation of the second aspect, the electronic device can set manual exposure parameters for the next photo in advance and send manual exposure parameters each time it takes a photo. This can hide the effective time of the manual exposure parameters for each photo, thereby increasing the photo-taking speed.

[0048] In a possible implementation of the second aspect, the electronic device can also reuse manual exposure frames, eliminating the need to pre-set and distribute manual exposure parameters for the next photo each time a photo is taken, thereby reducing the number of manual exposure parameter distributions and further improving the photo-taking speed. In this case, after selecting a frame for a photo, and before setting and distributing the manual exposure parameters for the next photo, the electronic device can also determine whether the manual exposure frame used for the current photo is available for the next photo. If the manual exposure frame used for the current photo is available for the next photo, then the manual exposure parameters for the next photo do not need to be pre-set and distributed, and the manual exposure frame used for the current photo can be reused for the next photo, i.e., manual exposure frame reuse. If the manual exposure frame used for the current photo is not available for the next photo, then the manual exposure parameters for the next photo are pre-set and distributed.

[0049] That is, after selecting the first and / or second frames for photographing, the electronic device may first determine whether the manual exposure frame used for the current photograph is available for the next photograph, and then determine whether to set and transmit manual exposure parameters for the next photograph in advance. Furthermore, if the manual exposure frame used for the current photograph is obtained based on the manual exposure parameters set and transmitted for the current photograph, it is not necessary to determine whether the manual exposure frame used for the current photograph is available for the next photograph, and it is assumed that the manual exposure frame used for the current photograph is available for the next photograph.

[0050] In one possible implementation of the second aspect, the electronic device can further increase the speed of capturing photos by reusing manual exposure frames. In this case, during the frame selection process, the electronic device can first determine whether there are manual exposure frames available for the current photo in the ZSL queue and / or the historical frame queue, that is, determine whether there are reusable manual exposure frames. If there are reusable manual exposure frames, the electronic device selects the manual exposure frame as the manual exposure frame for the current photo. After selecting the manual exposure frame for the current photo, it further predicts whether the manual exposure frame for the current photo will be available for the next photo. If there are no manual exposure frames available for the current photo in either the ZSL queue or the historical frame queue, it is necessary to set and issue manual exposure parameters to instruct the image sensor to output a manual exposure frame based on the manual exposure parameters and cache the manual exposure frame in the ZSL queue. After the image sensor outputs the manual exposure frame, the electronic device selects the manual exposure frame and other images from the ZSL queue to obtain a frame for the current photo. Furthermore, if the manual exposure parameters for the current photo are issued during the current photo, it is not necessary to determine whether the manual exposure frame for the current photo will be available for the next photo.

[0051] That is to say, when the electronic device is selecting the first frame for taking a picture and / or the second frame for taking a picture, it can first determine whether there is a manual exposure frame available for the current picture in the ZSL queue and / or the historical frame queue. If so, it will be selected as the manual exposure frame for the current picture. If not, the manual exposure parameters will be set and issued to obtain the manual exposure frame for the current picture.

[0052] In a possible implementation of the second aspect, when determining whether a manual exposure frame used in a current photo is available for the next photo, the electronic device may predict the time of the next photo based on the photo time and photo interval of the current photo operation. If the time interval between the time of the next photo or the timestamp of the reference frame of the next photo and the timestamp of the manual exposure frame used in the current photo is greater than a preset time threshold, the electronic device determines that the manual exposure frame used in the current photo is unavailable for the next photo. If the time interval between the time of the next photo or the timestamp of the reference frame of the next photo and the timestamp of the manual exposure frame used in the current photo is less than or equal to the preset time threshold, the electronic device determines that the manual exposure frame used in the current photo is available for the next photo. In this way, by using the predicted photo interval, the time of the current photo, and the timestamp of the manual exposure frame used in the current photo, the electronic device reuses manual exposure frames closer to the next photo time and disregards manual exposure frames farther from the next photo time. This allows for frame reuse of manual exposure frames while ensuring photo quality, thereby improving photo capture speed.

[0053] In one possible implementation of the second aspect, when determining whether a manual exposure frame exists in the ZSL queue and / or the historical frame queue, the electronic device may determine whether the manual exposure frame exists and is available for the current photo in the ZSL queue and / or the historical frame queue based on the number of frames between the manual exposure frame and the reference frame for the current photo. If a manual exposure frame exists and the number of frames between the manual exposure frame and the reference frame for the current photo is less than or equal to a preset number of frames (e.g., 7 frames), the manual exposure frame is considered available for the current photo; conversely, if the number of frames between the manual exposure frame and the reference frame for the current photo is greater than the preset number of frames, the manual exposure frame is considered unavailable for the current photo. Alternatively, the determination may be based on the time interval between the timestamp of the manual exposure frame and the timestamp of the reference frame for the current photo. If a manual exposure frame exists and the time interval between the timestamp of the manual exposure frame and the timestamp of the reference frame for the current photo is less than or equal to a preset threshold, the manual exposure frame is considered available for the current photo; conversely, if the time interval is greater than the preset threshold, the manual exposure frame is considered unavailable for the current photo. That is, the time interval between the timestamp of the manual exposure frame and the timestamp of the reference frame for the current photo is less than the preset threshold.

[0054] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in any one of the first or second aspects above is implemented.

[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of any one of the first or second aspects above.

[0056] In a fifth aspect, embodiments of the present application provide a chip system, comprising a processor coupled to a memory, the processor executing a computer program stored in the memory to implement the method described in either the first or second aspect. The chip system may be a single chip or a chip module consisting of multiple chips.

[0057] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute the method described in any one of the first or second aspects above.

[0058] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first or second aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] FIG1 is a schematic diagram of a mobile phone photo taking scene provided in an embodiment of the present application;

[0060] FIG2A is a schematic diagram of a non-ZSL photography process according to an embodiment of the present application;

[0061] FIG2B is a schematic diagram of the ZSL photography process provided in an embodiment of the present application;

[0062] FIG3 is a schematic diagram of a ZSL queue image caching process according to an embodiment of the present application;

[0063] FIG4A is a schematic diagram of multi-frame synthesis provided in an embodiment of the present application;

[0064] FIG4B is a schematic diagram of multi-frame synthesis based on a ZSL queue according to an embodiment of the present application;

[0065] FIG5 is a schematic diagram showing a comparison of photographing speeds before and after frame multiplexing according to an embodiment of the present application;

[0066] FIG6 is a schematic diagram of a solution for implementing frame multiplexing provided in an embodiment of the present application;

[0067] FIG7 is a schematic diagram of another solution for implementing frame multiplexing provided in an embodiment of the present application;

[0068] FIG8A is a schematic diagram of a multiplexed manual exposure frame according to an embodiment of the present application;

[0069] FIG8B is a schematic block diagram of a process for obtaining a manual exposure frame according to an embodiment of the present application;

[0070] FIG9 is a schematic diagram of setting a manual exposure frame in advance according to an embodiment of the present application;

[0071] FIG10A is a schematic diagram of multiplexing manual exposure frames and pre-setting manual exposure frames according to an embodiment of the present application;

[0072] FIG10B is a schematic block diagram of a process for obtaining a manual exposure frame according to an embodiment of the present application;

[0073] FIG11A is a schematic diagram of photographing speed provided by an embodiment of the present application;

[0074] FIG11B is a schematic diagram of the photographing speed provided in an embodiment of the present application;

[0075] FIG12 is a schematic structural diagram of an electronic device 200 provided in an embodiment of the present application;

[0076] FIG13 is a schematic block diagram of a process of a photo processing method provided in an embodiment of the present application;

[0077] FIG14 is a flow chart showing a method for processing photographs according to an embodiment of the present application. DETAILED DESCRIPTION

[0078] In the following description, for the purpose of illustration rather than limitation, specific details such as particular system structures and technologies are provided to facilitate a thorough understanding of the embodiments of the present application.

[0079] The following describes relevant contents that may be involved in the embodiments of this application.

[0080] Photo shooting scene:

[0081] Portable electronic devices such as mobile phones and tablets can use hardware such as graphics processing units (GPUs), cameras, image signal processors (ISPs), displays, and application processors, along with camera applications, to achieve camera functionality. Camera applications installed on mobile phones and other electronic devices can leverage hardware such as cameras to take photos. These applications can be the operating system's built-in camera application or other third-party applications with camera functionality.

[0082] For example, referring to FIG1 , which shows a schematic diagram of a mobile phone photo-taking scenario provided by an embodiment of the present application, the main interface of a mobile phone 10 displays icons of applications 1 to 10, a gallery application, and a camera application 11. When a user clicks on the camera application 11, the mobile phone 10 responds to the click by opening the camera application 11 and displaying a photo preview interface 12 of the camera application 11 by default.

[0083] The photo preview interface 12 includes a viewfinder and a photo button 13. The viewfinder is used to display a photo preview image. After the camera application 11 is opened, the mobile phone 10 enters the photo preview phase (or photo preview mode) by default. During the photo preview phase, the camera continuously captures preview images and displays the preview images in the viewfinder of the photo preview interface 12.

[0084] The mobile phone 10 is in a photo preview mode. The user clicks the photo button 13 . In response to the click operation, the mobile phone 10 executes a photo operation, obtains a photo image, and displays a thumbnail 14 of the photo image on the photo preview interface 12 .

[0085] The mobile phone 10 can also support capturing HDR images. As shown in FIG1 , the mobile phone 10 includes an HDR mode control 15 . The user can click the HDR mode control 15 to instruct the mobile phone 10 to enter HDR capture mode and capture HDR images. Of course, the mobile phone 10 can also provide an HDR switch control in the photo preview interface 12 for the user to turn HDR mode on and off through the HDR switch control. When HDR mode is turned on through the HDR switch control, the mobile phone 10 enters HDR capture mode; when HDR mode is turned off through the HDR switch control, the HDR capture mode is exited.

[0086] Zero delay photography ZSL:

[0087] In order to reduce the delay in taking photos, mobile phones and other electronic devices can use a zero-delay photo taking method to take photos so that the image obtained by taking the photo is consistent with the picture seen by the user in the photo preview interface.

[0088] During the photo-taking process, there's a certain amount of time between when the user presses the photo button and when the electronic device actually captures the image, which results in a delay. The process from pressing the photo button to image capture involves multiple steps, including focusing, exposure calculation, image capture, and image processing. Each step takes a certain amount of time, resulting in a certain amount of delay.

[0089] For example, referring to FIG2A , which shows a flow chart of non-ZSL photography provided by an embodiment of the present application, based on the scenario shown in FIG1 , when the mobile phone 10 opens the camera application 11, it calls the camera and other related hardware to capture images, obtains a preview stream, and displays the preview stream on the photo preview interface 12. In this way, the user can see the photo preview screen in the viewfinder of the photo preview interface 12. As shown in FIG2A , the preview stream includes frames 1 to 7 (i.e., frame 1 to frame 7). In terms of timing, frame 1 is the earliest and frame 7 is the latest. The mobile phone 10 displays frames 1 to 7 in the preview photo interface 102 in succession.

[0090] At time T1, the user presses the photo button 13 to trigger the photo. At this time, the image frame displayed on the photo preview interface 12 is frame 2. In response to the user's triggering operation to take a photo, the mobile phone 10 performs processes such as autofocus and exposure calculation to determine the image focus and exposure parameters. At time T2, the mobile phone 10 completes the focus and exposure calculation processes and starts taking photos based on the exposure parameters and image focus, obtaining an image frame captured by the image sensor, which is frame 5. At this time, the image frame displayed on the photo preview interface 12 is frame 5. After capturing frame 5 based on the exposure parameters, the mobile phone 10 can perform encoding processing on frame 5 to obtain the final photographed image. Frame 5 in the preview stream and frame 5 captured based on the exposure parameters are not the same image, but the image content of the two is the same.

[0091] As shown in Figure 2A, the user triggers a photo at time T1, and the image displayed on the photo preview interface 12 is frame 2. That is, when the user triggers the photo, the image content they see corresponds to frame 2. However, because the mobile phone 10 needs to perform a series of processing steps such as focusing and exposure, the image frame actually captured by the mobile phone 10 (i.e., the image actually captured) is frame 5, and the image content of frame 5 is inconsistent with the image content of frame 2. Therefore, the image actually captured by the mobile phone 10 is inconsistent with the image seen by the user when the photo is triggered, and there is a certain delay between the two.

[0092] When an electronic device uses zero-delay photography to take pictures, after opening the camera application and entering the picture preview stage, in addition to generating a picture preview image, it also caches image frames in a cache space; when the user triggers a picture, the electronic device can quickly obtain the required picture frames from the cache space and generate a picture image based on the picture frames.

[0093] Zero-delay photography pre-generates and caches image frames during the photo preview stage, allowing the required images to be quickly retrieved from the cache space when the user triggers the photo. This can significantly reduce photo delays and provide a smoother and more immediate photo experience.

[0094] The photo preview image is used to be displayed on the photo preview interface. The image frame cached in the cache space may be a RAW image corresponding to the photo preview image. The frame data of the photo preview image may be smaller than the frame data of the cached image frame.

[0095] For example, referring to the flow chart of ZSL photography provided by an embodiment of the present application shown in FIG2B , based on the scenario shown in FIG1 , when the mobile phone 10 opens the camera application 11 , the photography preview interface 12 is displayed by default and the photography preview stage is entered.

[0096] During the photo preview phase, mobile phone 10 simultaneously generates a preview stream and cached image frames, displays the preview stream on photo preview interface 12, and caches the cached image frames in a ZSL queue. As shown in FIG2B , the preview stream sequentially includes frames 1 through 7 (i.e., frames 1 through 7), which mobile phone 10 sequentially displays on preview photo interface 102. The ZSL queue includes frames 1 through 7. Frames 1 through 7 in the ZSL queue are not identical to frames 1 through 7 in the preview stream, but rather contain images with the same content.

[0097] At time T1, the user presses the photo button 13, triggering a photo. The image frame displayed on the photo preview interface 12 is now frame 2. Because the mobile phone 10 pre-buffers the image frame corresponding to the preview screen in the ZSL queue, in response to the user's triggering action, the mobile phone 10 retrieves the image frame corresponding to the preview screen at time T1 from the ZSL queue and performs encoding and other operations on the retrieved image frame to obtain the final captured image. At this point, the image actually captured by the mobile phone 10 is frame 2, which is consistent with the image seen when the user triggered the photo, achieving a "what you see is what you get" photo effect.

[0098] ZSL Queue:

[0099] The ZSL queue is an image cache queue used to cache image frames corresponding to the photo preview screen during the photo preview phase.

[0100] For example, referring to FIG3 , which shows a schematic diagram of the ZSL queue image caching process provided in an embodiment of the present application, the image captured by the image sensor is transmitted to the ISP; after the ISP performs corresponding processing on the image captured by the image sensor, the image frame is stored in the ZSL queue.

[0101] In Figure 3, the ZSL queue buffers multiple image frames, including frame n, frame n-1, frame n-2, frame n-3, and so on. Here, n is a positive integer greater than or equal to 1, and nm is greater than or equal to 1. Frame n can refer to the nth image captured by the image sensor. Similarly, frame nm refers to the nmth image captured by the image sensor.

[0102] The ZSL queue has a first-in, first-out mechanism, meaning that the image frames that enter the ZSL queue first are removed first. When the ZSL queue is full, or reaches its maximum capacity, the oldest (first-entered) image frame can be removed from the queue using the pop command, allowing the ISP to refill it with new image frames. After the ISP outputs a new image frame, it can be added to the ZSL queue using the push command.

[0103] In the ZSL queue shown in Figure 3, frame n is the newest image frame, meaning it enters the ZSL queue last. Frame nm is the oldest image frame, meaning it enters the ZSL queue first. Based on the first-in-first-out mechanism, frame nm can be removed from the ZSL queue, allowing the ISP to add frame n+1 to the ZSL queue.

[0104] Photographing multi-frame synthesis:

[0105] It is a common photo processing method for mobile phones and other electronic devices to fuse multiple frames of images to obtain a high-quality photo image.

[0106] For example, assume that 8 image frames are selected for multi-frame synthesis each time a photo is taken. Referring to FIG4A , which shows a schematic diagram of multi-frame synthesis provided in an embodiment of the present application, the image frame sequence includes frames 100 to 116. Based on this image frame sequence, the first photo frame selection process and the second photo frame selection process are performed successively. The number after the frame can indicate the frame number of the image output by the image sensor. For example, frame 100 refers to the 100th frame image output by the image sensor, and frame 116 refers to the 116th frame image output by the image sensor.

[0107] The first shot frame selection process is for the first shot, and is used to select the frames for the first shot. The second shot frame selection process is for the second shot, and is used to select the frames for the second shot. The first shot is taken before the second shot.

[0108] As shown in Figure 4A, during the first frame selection, eight frames, namely, frames 100, 101, 102, 103, 104, 105, 106, and 107, are selected from the image frame sequence. These eight frames are then multi-frame fused to obtain the first image. During the second frame selection, eight frames, namely, frames 108, 109, 110, 111, 112, 113, 114, and 115, are selected from the image frame sequence. These eight frames are then multi-frame synthesized to obtain the second image.

[0109] The image frame sequence may be, but is not limited to, multiple frames of images cached in the ZSL queue. When multiple frames of images are selected from the ZSL queue for synthesis, the selected images are deleted from the ZSL queue.

[0110] For example, referring to FIG4B , which shows a schematic diagram of multi-frame synthesis based on a ZSL queue provided in an embodiment of the present application, the ZSL queue caches multiple image frames, such as frame n, frame n-1, frame n-2, frame n-3, ..., and frame nm. Frame n, frame n-1, frame n-2, and frame n-3 are selected from the ZSL queue as frames for photographing, and frames n, frame n-1, frame n-2, and frame n-3 are synthesized to obtain a photographic image. A photographic frame refers to an image frame used for photographing.

[0111] After selecting a capture frame from the ZSL queue, the capture frame is made available for subsequent capture processing. Furthermore, the capture frames in the ZSL queue are deleted. Specifically, frames n, n-1, n-2, and n-3 are deleted from the ZSL queue. In Figure 4B , dashed boxes represent frames n, n-1, n-2, and n-3, indicating that these image frames have been deleted.

[0112] In the related art photo-taking and multi-frame synthesis process, the images used for the multi-frame synthesis in the two photos are different.

[0113] For example, as shown in FIG4A , the images used for multi-frame synthesis in the first photograph are frames 100 to 107 , while the images used for multi-frame synthesis in the second photograph are frames 108 to 115 . The image frames used in the two photographs are different.

[0114] For another example, as shown in FIG4B , after selecting frame n, frame n-1, frame n-2 and frame n-3 from the ZSL queue as the frames for taking the current photo, frame n, frame n-1, frame n-2 and frame n-3 in the ZSL queue will be deleted, so that when selecting the frames for taking the photo from the ZSL queue next time, none of frame n, frame n-1, frame n-2 and frame n-3 can be selected, which will also cause the images used for synthesis in the two photos to be different.

[0115] During their research, the inventors discovered that using different images for the composite between two shots reduces the capture speed. For example, an image sensor outputs frames at a 30fps frame rate, meaning it outputs 30 frames per second. FPS stands for frames per second. Assuming a capture uses six frames, since each shot uses a different image, a maximum of five shots can be taken in one second, resulting in a capture speed of five frames per second.

[0116] To increase capture speed, the number of frames used in multi-frame synthesis can be reduced. For example, if the image sensor outputs frames at a 30fps frame rate, a single shot would use six frames, resulting in five shots per second and a capture speed of five frames per second. Now, a single shot uses five frames, resulting in six shots per second and a higher capture speed. Alternatively, a single-frame direct output method can be used to increase capture speed. However, while reducing the number of frames used in multi-frame synthesis and single-frame direct output methods can increase capture speed, they sacrifice capture quality and reduce the quality of the captured image.

[0117] To address the aforementioned issues, embodiments of the present application utilize a frame multiplexing solution to increase capture speed without compromising image quality. Frame multiplexing allows for the reuse of the same image for two captures, meaning that at least one image frame used in both captures is the same. Typically, the reference frame is not reused.

[0118] For example, referring to FIG5 , which shows a schematic diagram comparing the speed of photographing before and after frame multiplexing provided by an embodiment of the present application, assuming that each photograph is synthesized using 8 frames, and during the preview phase of the photograph, the image sensor outputs frames at a frame rate of 30 fps, meaning that the image sensor outputs 30 frames per second. In FIG5 , in chronological order, the image sequence includes frames 100 to 135, for a total of 36 frames. At this point, the image sensor outputs frames 100 to 129 within 1 second.

[0119] Before frame multiplexing, that is, the image frames used for the two photos are different, a maximum of 4 images can be taken in 1 second, that is, the photo shooting speed is 4 frames per second.

[0120] Specifically, when selecting frames for the first photo, frame 100, frame 101, frame 102, frame 103, frame 104, frame 105, frame 106, and frame 107 are selected as the frames for the first photo. These 8 images are synthesized into multiple frames to obtain the photo image of the first photo.

[0121] When selecting frames for the second photo, frames 108, 109, 110, 111, 112, 113, 114, and 115 are selected as the frames for the second photo. These 8 images are synthesized into multiple frames to obtain the photo image for the second photo.

[0122] When selecting frames for the third photo, frames 116, 117, 118, 119, 120, 121, 122, and 123 are selected as the frames for the third photo. These 8 images are synthesized into multiple frames to obtain the photo image for the third photo.

[0123] When selecting frames for the fourth photo, frames 124, 125, 126, 127, 128, 129, 130, and 131 are selected as the frames for the fourth photo. These 8 images are synthesized into multiple frames to obtain the photo image for the fourth photo.

[0124] After frame multiplexing, that is, adopting the frame multiplexing scheme, the same image frame can be reused for two photos. Up to 30 images can be taken in 1 second, that is, the shooting speed is 30 frames per second, which significantly improves the shooting speed.

[0125] Specifically, when selecting frames for the first photo, frame 100, frame 101, frame 102, frame 103, frame 104, frame 105, frame 106, and frame 107 are selected as the frames for the first photo. These 8 images are synthesized into multiple frames to obtain the photo image of the first photo.

[0126] When selecting frames for the second photo, frame 101, frame 102, frame 103, frame 104, frame 105, frame 106, frame 107, and frame 108 are selected as the frames for the second photo. These 8 images are synthesized into multiple frames to obtain the photo image for the second photo.

[0127] By comparing the frame selection for the first photo and the frame selection for the second photo, we can see that 7 of the image frames used for the first photo and the second photo are the same, namely frame 101, frame 102, frame 103, frame 104, frame 105, frame 106, and frame 107, that is, 7 images are reused.

[0128] Similarly, when selecting frames for the third photo, frames 102, 103, 104, 105, 106, 107, 108, and 109 are selected as the frames for the third photo. These 8 images are synthesized into multiple frames to obtain the photo image for the third photo.

[0129] By comparing the frame selection for the second and third shots, we can see that seven of the image frames used for the second and third shots are the same, namely frame 102, frame 103, frame 104, frame 105, frame 106, frame 107, and frame 108, that is, seven images are reused.

[0130] And so on, the subsequent 4th to 28th photo frame selections are carried out in sequence.

[0131] When selecting frames for the 29th photo, frames 128, 129, 130, 131, 132, 133, 134, and 135 are selected as the frames for the 29th photo. These 8 images are synthesized into multiple frames to obtain the photo image for the 29th photo.

[0132] When selecting frames for the 30th photo, frames 129, 130, 131, 132, 133, 134, 135, and 136 are selected as the frames for the 30th photo. These 8 images are synthesized into multiple frames to obtain the photo image for the 30th photo.

[0133] As can be seen from Figure 5, after frame multiplexing, each shot reuses the 7 images from the previous shot, thereby increasing the shooting speed from 4 frames per second to 30 frames per second.

[0134] It should be noted that Figure 5 shows a comparison of capture speeds before and after frame multiplexing in a non-high-dynamic-range scenario. Furthermore, each capture in Figure 5 reuses seven frames from the previous capture. In other embodiments, the number of frames reused per capture can be six, five, four, or even fewer. In this case, while the capture speed will be lower than 30 frames per second, it will still be higher than four frames per second. Typically, by reused at least one frame from the previous capture, each capture can achieve frame multiplexing and improve capture speed.

[0135] Frame multiplexing improves capture speed without reducing the number of frames synthesized per shot, thus increasing image quality. For example, in Figure 5, before frame multiplexing, eight images are fused per shot; after frame multiplexing, eight images are also fused per shot, without reducing the number of fused images per shot.

[0136] The present application provides two frame reuse schemes. The first scheme involves the electronic device selecting a frame from the ZSL queue for the current photo. Instead of deleting the frame from the ZSL queue, the electronic device maintains the frame in the ZSL queue, awaiting removal via a first-in, first-out mechanism. This allows the electronic device to select the frame from the previous photo when selecting the next photo, ensuring that at least one image in the frames used for the two photos is the same, thus achieving frame reuse.

[0137] The second solution is: After the electronic device selects a frame for the current photo from the ZSL queue, it deletes the frame from the ZSL queue, persistently stores the frame to a storage medium, and stores the metadata of the frame in the historical frame queue. This way, when selecting a frame for the next photo, the electronic device not only selects a frame from the ZSL queue but also selects the frame from the previous photo from the historical frame queue based on the metadata in the historical frame queue, thus achieving frame reuse.

[0138] The historical frame queue may be a buffer space for storing metadata of frames for photographing. According to the metadata stored in the historical frame queue, the frames for photographing corresponding to the metadata may be acquired from the storage medium.

[0139] The following describes these two frame multiplexing implementation schemes in conjunction with the accompanying drawings.

[0140] The first option:

[0141] For example, referring to FIG6 , which shows a schematic diagram of a scheme for implementing frame multiplexing provided by an embodiment of the present application, the ZSL queue includes multiple image frames such as frame n, frame n-1, frame n-2, frame n-3… and frame nm; frame n, frame n-1, frame n-2, and frame n-3 are selected from the ZSL queue as frames for the current photo; after copying frame n, frame n-1, frame n-2, and frame n-3, the copied image is used for photo processing. After the photo is taken and the frames are selected, frame n, frame n-1, frame n-2, and frame n-3 in the ZSL queue are not deleted. In this case, the image frames in the ZSL queue will only be deleted when they are normally rotated according to the first-in-first-out mechanism.

[0142] In this way, by not deleting the frames used for the current photo in the ZSL queue after selecting the frames for the current photo, the frames used for the current photo can still be selected from the ZSL queue for the next photo, thereby realizing frame multiplexing and improving the photo speed without reducing the quality of the photo image.

[0143] For example, when taking a photo, frames n, n-1, n-2, and n-3 are selected as the frames for that photo. When taking the next photo, assuming that frames n, n-1, n-2, and n-3 have not been removed by the first-in-first-out mechanism and are still stored in the ZSL queue, frames n+1, n, n-1, and n-2 can be selected as the frames for the photo. In this way, the three frames of images, frame n, n-1, and n-2, are reused in the two photos.

[0144] Second option:

[0145] For example, see Figure 7, which illustrates another scheme for implementing frame multiplexing according to an embodiment of the present application. It shows the process of taking a previous and next photo. In terms of timing, the previous photo occurs before the next photo. The ZSL queue caches image frames and metadata.

[0146] During the previous photo capture process, the ZSL queue included multiple image frames, including frame n, frame n-1, frame n-2, frame n-3, ..., and frame nm. Frame n, frame n-1, frame n-2, and frame n-3 are selected from the ZSL queue as the frames for the current photo capture process. Frame n, frame n-1, frame n-2, and frame n-3 are used for photo processing. Frame n, frame n-1, frame n-2, and frame n-3 are deleted from the ZSL queue, and frames n, frame n-1, frame n-2, and frame n-3 are written to the file system. In other words, the frames selected for the current photo capture process are deleted from the ZSL queue.

[0147] Flashing to the file system means persistently storing frames n, n-1, n-2, and n-3 on a storage medium. This storage medium can be a hard drive, for example. After persistently storing the capture frames on the storage medium, you can subsequently read them from the storage medium and process them using a capture algorithm to obtain the captured image.

[0148] In addition, metadata for frames n, n-1, n-2, and n-3 (i.e., the frames used for the photo) is retrieved from the ZSL queue and stored as frame shells in the historical frame queue. A frame shell contains the frame's metadata (i.e., metadata) but does not contain image content. Frame metadata can include, but is limited to, the frame number and exposure parameters. Frame shells and frame metadata can be equivalent. The historical frame queue can store metadata for photo frames for a long period of time.

[0149] As shown in Figure 7, the metadata for frame n is copied from the ZSL queue to generate a shell for frame n, i.e., the frame n shell, which is then stored in the historical frame queue. The frame n shell does not include the image content of frame n, but only the metadata for frame n. Similarly, based on the metadata for frames n-1, n-2, and n-3 in the ZSL queue, the frame n-1 shell, frame n-2 shell, and frame n-3 shell are generated, respectively, and the n-1 shell, frame n-2 shell, and frame n-3 shell are stored in the historical frame queue.

[0150] The frame n-1 shell includes metadata of frame n-1 but does not include the image content of frame n-1; the frame n-2 shell includes metadata of frame n-2 but does not include the image content of frame n-2; and the frame n-3 shell includes metadata of frame n-3 but does not include the image content of frame n-3.

[0151] During the next capture, the ZSL queue includes multiple image frames, including frames n+3, n+2, n+1, and nk. nk is greater than or equal to 1. Compared to the previous capture, the ZSL queue rotates normally based on a first-in, first-out mechanism, with newer frames, such as frames n+1, n+2, and n+3, filling the ZSL queue. The historical frame queue includes the shells for frames n, n-1, n-2, and n-3.

[0152] Select frames for the current photo from the ZSL queue and the historical frame queue. Specifically, select frames n+1 and n+2 from the ZSL queue as frames for the photo, and write frames n+1 and n+2 to the file system; delete frames n+1 and n+2 from the ZSL queue, obtain metadata for frames n+1 and n+2 from the ZSL queue, generate a frame n+1 shell based on the metadata for frame n+1, and generate a frame n+2 shell based on the metadata for frame n+2; store the frame n+1 shell and the frame n+2 shell in the historical frame queue, and store the metadata for frames n+1 and n+2 in the historical frame queue.

[0153] The images corresponding to the frame n-1 shell and the frame -2 shell are selected from the historical frame queue as frames for photographing, and the frame n-1 shell and the frame -2 shell are copied from the historical frame queue, and the copied frame n-1 shell and the frame -2 shell are used for subsequent photographing processing.

[0154] When it is necessary to perform photographic algorithm processing on the photographic frames, frame n+1 and frame n+2 can be read from the file system, frame n-1 can be found from the file system according to the frame n-1 shell, and frame n-2 can be found from the file system according to the frame n-2 shell; frame n-2, frame n-1, frame n+1, and frame n+2 can be processed by the photographic algorithm to obtain the photographic image of the current photo.

[0155] It can be seen that the image frames multiplexed between the previous photo-taking and the next photo-taking include frame n-2 and frame n-1.

[0156] It's worth noting that frames selected from the historical frame queue for use in a photo shoot don't need to be written to the file system. By associating metadata selected from the historical frame queue with the current photo shoot, the corresponding image frames can be subsequently retrieved from the file system based on the metadata. This eliminates the need to repeatedly write multiplexed image frames to disk, saving storage space. For example, in the subsequent photo shoot in Figure 7, frame n-1 corresponding to frame n-1 and frame n-2 corresponding to frame n-2 are not written to the file system because frames n-1 and n-2 were already written to the file system during the previous photo shoot, and the file system already has frames n-1 and n-2 stored.

[0157] It should be noted that during the previous photo capture process in Figure 7, no photo frames were selected from the historical frame queue because the historical frame queue did not contain metadata for the photo frames from the previous capture. In actual applications, if the historical frame queue does contain metadata for the photo frames from the previous capture, photo frames would need to be selected from both the historical frame queue and the ZSL queue. If this is the first photo capture, and metadata for the photo frames from the previous capture is not yet in the historical frame queue, then only the ZSL queue can be used to select the photo frames.

[0158] In addition, Figures 6 and 7 exemplarily show that four image frames are selected as frames for each photo-taking. In actual applications, the number of frames for each photo-taking can be determined according to actual needs and is not limited here.

[0159] As can be seen above, the first solution achieves frame reuse by not deleting the camera frames from the ZSL queue, allowing them to be cached in the ZSL queue. However, the camera frames are not permanently cached in the ZSL queue and are limited by the length of the ZSL queue. When the ZSL queue reaches its maximum capacity and new image frames need to be added to the ZSL queue, the old image frames in the ZSL queue are removed. This affects the reuse rate of the camera frames. The reuse rate of the camera frames can refer to the number of times the camera frames are reused.

[0160] For example, in Figure 6, frame n is the frame used for the first shot. During the second shot, frame n is still stored in the ZSL queue and can be selected as the frame for the second shot. During the third shot, frame n has already been removed by the normal first-in-first-out round-robin mechanism and cannot be selected as the frame for the third shot. In this case, the reuse count for frame n is 1.

[0161] The second solution stores the frames used for photo capture in the file system and their metadata in the historical frame queue. Photo frames can be permanently stored in the file system without being restricted by the ZSL queue length; nor is the metadata for photo frames restricted by the historical frame queue length. This approach saves storage space while maintaining frame reuse rates.

[0162] For example, in Figure 7, frame n is the frame used for the first photo. During the second and third photos, frame n is still stored in the file system, and its metadata is still stored in the historical frame queue. Frame n can be selected as the frame used for the second and third photos. In this case, the number of times frame n is reused is 2.

[0163] After determining the frame for photographing, the electronic device can process the frame for photographing using a photographing algorithm to obtain a photographed image. The photographing algorithm can enhance the image data output by the camera hardware to improve the image effect and image quality.

[0164] Exemplarily, the photo-taking algorithm may include at least one of the following: a beauty processing algorithm, a filter processing algorithm, a rotation processing algorithm, a watermark processing algorithm, a blur processing algorithm, a high dynamic range imaging (HDR) processing algorithm, a multi-frame processing algorithm, a denoising algorithm, and a detail enhancement algorithm, etc.

[0165] In some optional embodiments, the electronic device may continuously and uninterruptedly execute a photographing algorithm during the photographing process, process photographing frames, and generate photographic images.

[0166] When an electronic device is in the process of taking a photo, the camera application will run in the foreground and will also need to process the preview image data stream to display the photo preview interface on the photo preview interface. For example, in the scenario shown in Figure 1, after the mobile phone 10 opens the camera application 11, it enters the photo taking process. During the photo taking process, the camera application 11 runs in the foreground and displays the photo preview image on the photo preview interface 12.

[0167] During the research process, the inventors found that continuously and uninterruptedly executing all photographic algorithms during the photographing process to obtain photographic images in real time would reduce photographic performance and affect the user's photographic experience.

[0168] Specifically, during the photo-taking process, the camera application running in the foreground, processing the preview video stream, and running complex photo-taking algorithms all require a significant amount of computing and memory resources. Processing the preview video stream, running the camera application in the foreground, and rendering the image already consume a significant amount of computing and memory resources. Continuously executing all photo-taking algorithms during the photo-taking process increases the amount of computation required, increasing the computational load and memory usage, leading to excessive processor load and high memory usage. Furthermore, the camera application running in the foreground, processing the preview video stream, and running complex photo-taking algorithms all occur simultaneously, competing for computing and memory resources, further increasing system pressure.

[0169] If the processor load and running memory usage are too high, the photo shooting speed will be significantly reduced, and the photo shooting will be stuck. In some scenarios (such as rapid continuous shooting), the photo shooting will even become unresponsive and the photo shooting performance will be poor. In addition, if the processor runs at high load for a long time during the photo shooting process, it will cause significant heating problems of the device. If the device temperature is too high, it will trigger the temperature control strategy. The temperature control strategy reduces the temperature by limiting the use of the processor, for example, reducing the frequency and reducing the use time of the large core. These processor restriction measures reduce the processing power of the processor, which further reduces the photo shooting performance.

[0170] For example, in Figures 6 and 1, after receiving a user click on the photo button 13, the mobile phone 10 selects frames n, n-1, n-2, and n-1 from the ZSL queue as the frames for the current photo. The mobile phone 10 continues to process the preview video stream, displaying the real-time preview image on the photo preview interface 12. Furthermore, the mobile phone 10 continuously and uninterruptedly executes one or more photo algorithms on the frames for the photo to generate a photo image. During this process, the mobile phone 10 may experience problems such as device heating and reduced photo performance due to excessive system pressure and load.

[0171] In particular, in a continuous shooting scenario, when the user clicks the photo button 13 multiple times in succession to take multiple photos, the mobile phone 10 needs to continuously and uninterruptedly execute one or more photo algorithms for each photo frame. If the processor load and running memory usage are too high, related problems such as a decrease in photo shooting speed will occur.

[0172] In other optional embodiments, when the electronic device determines the frame for taking pictures, it may not execute the picture-taking algorithm during the picture-taking process, or may only execute part of the picture-taking algorithm during the picture-taking process, thereby reducing the amount of calculation during the picture-taking process, so as to reduce the processor load and running memory occupancy rate during the picture-taking process, thereby alleviating problems such as lag and device heating, improving picture-taking performance, and enhancing the user's picture-taking experience.

[0173] Unlike the continuous and uninterrupted execution of all photographic algorithms during the photographing process to obtain a photographic image in real time, the segmented photographing solution of the embodiment of the present application does not execute the photographic algorithm during the photographing process, or only executes part of the photographic algorithm during the photographing process, and then triggers the execution of the remaining photographic algorithms at the appropriate time to obtain the photographic image. In this way, there is no need to continuously and uninterruptedly execute all photographic algorithms during the photographing process.

[0174] The appropriate timing may be, for example, when a user or application needs to access image data to view, share, or edit image data, when the system is idle, or when the camera application is closed or running in the background, and when system status data (such as processor load, running memory, and device temperature) is below a preset value. User access to image data may mean that the user taps a thumbnail on the camera preview interface of the electronic device or in the gallery application.

[0175] For example, in Figure 7, after the electronic device writes the selected photographic frame to the file system, if the photographic process is still in progress, it may not execute the photographic algorithm on the photographic frame, or it may read the photographic frame from the file system, execute only a portion of the photographic algorithm on the photographic frame, and write the intermediate results of the partial photographic algorithm to the file system. At an appropriate time, the electronic device may read the photographic frame from the file system again, execute the photographic algorithm on the photographic frame, and obtain the photographic image; or it may read the intermediate results of the partial photographic algorithm from the file system again, and continue to execute the remaining portion of the photographic algorithm to obtain the final photographic image.

[0176] The segmented photography solution can reduce the processor load and running memory usage during the photography process, and can further improve the photography speed. Specifically, if a single-stage photography is used (that is, all photography algorithms are executed continuously and uninterruptedly during the photography process), the running memory will continue to increase with the number of photos taken, until the system memory is exhausted or the prescribed usage limit is reached, and the system slows down. However, if a segmented photography is used, since the frames of each photo are stored in the file system, the system's running memory will not be exhausted as the number of photos continues to increase, which can effectively alleviate the running memory pressure and effectively improve the photography performance. In other words, the segmented photography solution can effectively alleviate the problem of excessive running memory usage during the photography process, thereby alleviating problems such as device freezes.

[0177] In HDR photography scenarios, electronic devices need to capture one or more manually exposed frames. Manually exposed frames are used to improve the dynamic range of an image and are obtained by exposing it according to manual exposure parameters. Manually exposed frames are also called manual AE frames.

[0178] The manual exposure parameter is an exposure parameter determined by the camera system based on current environmental information, and is not manually set by the user. The current environmental information may include, for example, brightness information of the current environment.

[0179] For example, the manual exposure frame includes a long exposure image and a short exposure image. During the photo preview stage, the electronic device collects a normal exposure image and displays a preview screen based on the normal exposure image; in addition, the normal exposure image is cached in the ZSL queue. After receiving the photo operation, the electronic device needs to determine the manual exposure parameters based on the current environmental information, and control the image sensor to output the long exposure image and short exposure image of the photo according to the manual exposure parameters; after selecting the normal exposure image from the ZSL queue, the normal exposure image, the long exposure image and the short exposure image are multi-frame fused to obtain an HDR image.

[0180] In related technologies, only after receiving a photo operation will the electronic device set the manual exposure parameters for that photo and send the manual exposure parameters to the image sensor, instructing the image sensor to output the manually exposed frame for that photo. However, there is a certain delay between the time the manual exposure parameters are set and the time they take effect (i.e., the image sensor outputs the manually exposed frame). If the manual exposure parameters are set and sent after receiving a photo operation for each photo, each photo will require a certain amount of time to wait for the image sensor to output the manually exposed frame, which reduces the speed of photo capture.

[0181] Based on the two schemes for achieving frame multiplexing mentioned above, the embodiment of the present application can increase the shooting speed in the HDR scene by multiplexing manual exposure frames.

[0182] At this time, after receiving the photo operation, the electronic device can first determine whether there is a manual exposure frame available for the current photo in the ZSL queue, or in the ZSL queue and the historical frame queue; if there is a manual exposure frame available for the current photo, then there is no need to set and send the manual exposure parameters for the current photo, but instead the available manual exposure frame is selected as the manual exposure frame for the current photo; if there is no manual exposure frame available for the current photo, then the manual exposure parameters for the current photo are set and sent, and the image sensor is waited for to output the manual exposure frame. In this way, when there is a manual exposure frame available for the current photo, there is no need to set and send the manual exposure parameters for the current photo. Only when there is no manual exposure frame available for the current photo, the manual exposure parameters for the current photo are set and sent, which reduces the number of times the manual exposure parameters are set and sent, reduces the time spent waiting for the image sensor to output the manual exposure frame, and improves the photo shooting speed. Especially in continuous shooting scenarios, there is no need to set and send the manual exposure parameters for each photo, which improves the continuous shooting speed.

[0183] In HDR multi-frame fusion scenarios, each photo has a reference frame. Optionally, the electronic device determines the number of frames between the reference frame and the manual exposure frame for the current photo based on the frame number of the reference frame and the frame number of the manual exposure frame. If the number of frames is greater than a preset number, it indicates that the image content of the manual exposure frame differs significantly from that of the reference frame, and the manual exposure frame is determined to be unusable for the current photo, i.e., no manual exposure frame is available for the current photo. If the number of frames is less than or equal to the preset number, it indicates that the image content of the manual exposure frame differs slightly from that of the reference frame, and the manual exposure frame is determined to be available for the current photo, i.e., no manual exposure frame is available for the current photo. The preset number can be set according to actual needs. For example, the preset number is 7.

[0184] If the image content of the manual exposure frame differs significantly from that of the reference frame, the fusion of the manual exposure frame and the reference frame will result in a poor final image quality. This embodiment of the present application uses the interval between the reference frame and the manual exposure frame to rationally determine whether there are available manual exposure frames for the current photo. This allows for reuse of manual exposure frames while ensuring the quality of the photo, thereby increasing the speed of photo capture.

[0185] Furthermore, the time interval between the reference frame and the manually exposed frame in the current shot can be determined based on the timestamp of the reference frame and the timestamp of the manually exposed frame. If the time interval between the reference frame and the manually exposed frame is less than or equal to a preset threshold, it indicates that the image content of the manually exposed frame differs slightly from that of the reference frame, and the manually exposed frame is therefore usable for the current shot. Conversely, if the time interval between the reference frame and the manually exposed frame is greater than the preset threshold, it indicates that the image content of the manually exposed frame differs significantly from that of the reference frame, and the manually exposed frame is therefore unusable for the current shot. The preset threshold can be set as needed and is not limited here. In HDR scenarios, in addition to manually exposed frames, non-manually exposed frames can also be reused. In this case, when selecting frames for a shot, not only manually exposed frames can be selected from the ZSL queue, or from both the ZSL queue and the historical frame queue, but also non-manually exposed frames. Furthermore, in HDR scenarios, reference frames are generally not reused.

[0186] For example, referring to FIG8A , which illustrates a schematic diagram of multiplexed manual exposure frames provided in an embodiment of the present application, an image sequence includes frames 100 through 117, for a total of 18 image frames. In chronological order, frame 100 is the earliest and frame 117 is the latest. An image sequence can be image frames captured by an image sensor over a period of time.

[0187] Before time T1, the electronic device is in the photo preview stage and continues to cache image frames in the ZSL queue. As shown in FIG8A, frames 100, 101, 102, and 103 are captured before time T1.

[0188] At time T1, the electronic device receives a trigger operation (e.g., pressing a capture button) for the user to take the first photo. In response to this trigger operation, the electronic device begins selecting frames for the first photo. Specifically, frames 100, 101, and 102 are selected as frames for the first photo. Frame 100 is the reference frame for the first photo.

[0189] Furthermore, when selecting frames for the first photo, a manual exposure frame must also be acquired. Specifically, as shown in FIG8B , a schematic flow chart illustrating a process for acquiring a manual exposure frame according to an embodiment of the present application, begins frame selection by determining whether a manual exposure frame is available for the current photo. If so, the available manual exposure frame is selected as the manual exposure frame for the current photo. If not, manual exposure parameters for the current photo are set and issued, and the image sensor waits for a manual exposure frame to be captured based on the manual exposure parameters. In FIG8A , according to the process of FIG8B , it is determined that no manual exposure frame is available for the first photo, and the manual exposure parameters for the first photo are then set and issued. After the manual exposure parameters are issued to the image sensor, the image sensor outputs frame 107 based on the manual exposure parameters. After the image sensor outputs frame 107, frame 107 is selected as the manual exposure frame for the first photo. As shown in FIG8A , when selecting frames for the first photo, the frames selected for the photo include frame 100, frame 101, frame 102, and frame 107.

[0190] As shown in Figure 8A, the electronic device sets and sends manual exposure parameters at frame 104. However, since the manual exposure parameters take time to take effect, the image sensor does not output a manual exposure frame based on the manual exposure parameters until frame 107. In other words, setting and sending manual exposure parameters after receiving a capture operation slows down the capture speed, as the manual exposure parameters take time to take effect.

[0191] Between time T1 and time T2, the electronic device is in the photo preview stage and continues to cache image frames in the ZSL queue.

[0192] At time T2, the electronic device receives a trigger operation (e.g., pressing a capture button) for the second photo. In response to this trigger operation, the electronic device begins selecting frames for the second photo. Specifically, frames 106, 107, 108, and 109 are selected as the frames for the second photo. Frame 108 is the reference frame for the second photo, and frame 107 is reused as the manual exposure frame for the second photo.

[0193] When selecting a frame for the second photo, the electronic device determines that frame 107 is available for the current photo according to the process shown in Figure 8B, and reuses the manual exposure frame of the first photo without setting and sending the manual exposure parameters for the current photo.

[0194] As can be seen from FIG8A , when the electronic device selects a frame for the second photo, it can reuse the manual exposure frame of the previous photo, without having to set and send manual exposure parameters or wait for the manual exposure parameters to take effect, so it can quickly select the frame for the current photo.

[0195] Between time T2 and time T3, the electronic device is in the photo preview stage and continues to cache image frames in the ZSL queue.

[0196] At time T3, the electronic device receives a trigger operation (e.g., pressing a capture button) for the third photo. In response to this trigger operation, the electronic device begins selecting frames for the third photo. Specifically, frames 110, 111, 112, and 116 are selected as frames for the third photo. Frame 110 is the reference frame for the third photo, and frame 117 is the manual exposure frame for the third photo.

[0197] During the third frame selection, the electronic device determines, according to the process shown in FIG8B , that frame 107 is unavailable for the current shot, and then sets and issues manual exposure parameters for the current shot, waiting for the image sensor to output an image frame according to the manual exposure parameters. At this point, the image sensor outputs frame 116 according to the manual exposure parameters.

[0198] It should be noted that in the scenarios of Figures 8A and 8B, either the first or second solution can be used to implement frame multiplexing. When using the first solution, frames are selected from the ZSL queue when taking a photo; when using the second solution, frames are selected from both the ZSL queue and the historical frame queue.

[0199] Based on the above embodiment, after determining that a manual exposure frame is available for the current photo, it is further possible to predict whether the manual exposure frame available for the current photo will be available for the next photo. If the manual exposure frame available for the current photo is available for the next photo, there is no need to set the manual exposure frame for the next photo in advance. If the manual exposure frame available for the current photo is not available for the next photo, the manual exposure frame for the next photo is set in advance.

[0200] At this time, after receiving the photo operation, the electronic device can first determine whether there is a manual exposure frame available for the current photo in the ZSL queue, or the ZSL queue and the historical frame queue; if there is a manual exposure frame available for the current photo, the available manual exposure frame is selected as the manual exposure frame for the current photo. In addition, it also predicts whether the manual exposure frame available for the current photo will be available for the next photo, and determines whether to set the manual exposure frame for the next photo in advance based on the prediction result; if there is no manual exposure frame available for the current photo, the manual exposure parameters for the current photo are set and issued, and the image sensor is waited for to output the manual exposure frame.

[0201] As shown above, in the related art, only after receiving a photo operation will the electronic device set the manual exposure parameters for the current photo and send the manual exposure parameters for the current photo to the image sensor, instructing the image sensor to output the manual exposure frame for the current photo. However, there is a certain delay between the manual exposure parameter being set and the manual exposure parameter taking effect (i.e., the image sensor outputs the manual exposure frame). If the manual exposure parameters for each photo are set and sent after receiving a photo operation, each photo will require a certain amount of time to wait for the image sensor to output the manual exposure frame, which reduces the photo shooting speed.

[0202] In the embodiment of the present application, after receiving a photo-taking operation, manual exposure parameters for the next photo-taking are set and issued in advance to hide the effective time of the manual exposure parameters, thereby improving the HDR photo-taking speed.

[0203] Specifically, after receiving the first photo-taking operation, the electronic device first sets and sends the manual exposure parameters for that photo, and then sets and sends the manual exposure parameters for the next photo. The image sensor outputs the manual exposure parameters for the current photo based on the manual exposure parameters for the current photo; then, based on the manual exposure parameters for the next photo, it outputs the manual exposure frame for the next photo and caches the manual exposure frame for the next photo.

[0204] After receiving the second photo-taking operation, since the manual exposure parameters for the second photo were set and issued in advance during the first photo-taking operation, the electronic device can obtain the pre-stored manual exposure frame for the second photo from the cache, without having to set and issue the manual exposure parameters for the current photo. In addition, the electronic device also sets and issues the manual exposure parameters for the next photo in advance, instructing the image sensor to output the manual exposure frame for the next photo based on the manual exposure parameters for the next photo, and caches the manual exposure frame for the next photo. The second photo-taking operation occurs after the first photo-taking operation. The first photo is the photo corresponding to the first photo-taking operation, and the second photo is the photo corresponding to the second photo-taking operation.

[0205] After receiving the third photo-taking operation, the electronic device can retrieve the manually exposed frame of the current photo from the cache, as the manual exposure parameters for the third photo were set and distributed in advance during the second photo-taking operation. Furthermore, the electronic device also sets and distributes the manual exposure parameters for the next photo-taking operation in advance, instructing the image sensor to output the manually exposed frame for the next photo-taking operation based on the manual exposure parameters for the next photo-taking operation, and cache the manually exposed frame for the next photo-taking operation. The third photo-taking operation occurs after the second photo-taking operation.

[0206] Similarly, each time a photo-taking operation is received, the electronic device sets and sends manual exposure parameters for the next photo in advance, instructs the image sensor to output the manual exposure frame for the next photo, and stores the manual exposure frame for the next photo. When the next photo-taking operation is received, the electronic device can retrieve the pre-stored manual exposure frame.

[0207] For example, referring to FIG9 , which illustrates a schematic diagram of pre-setting manual exposure frames according to an embodiment of the present application, an image sequence includes frames 100 through 117. In chronological order, frame 100 is the earliest and frame 117 is the latest. An image sequence may be image frames captured by an image sensor over a period of time.

[0208] At time T1, the electronic device receives a trigger operation (e.g., pressing a capture button) for the user to take the first photo. In response to this trigger operation, frame selection for the first photo is initiated. Specifically, frames 100, 101, 102, and 107 are selected as frames for the first photo. Frame 100 is the base frame for the first photo, and frame 107 is the manually exposed frame for the first photo.

[0209] After receiving the trigger operation for the first photo, the electronic device sets the manual exposure frame for the current photo and also sets the manual exposure frame for the next photo in advance. Specifically, the electronic device first sends the manual exposure parameters for the current photo to the image sensor, and then sends the manual exposure parameters for the next photo to the image sensor. As shown in Figure 9, the image sensor outputs frame 107 based on the manual exposure parameters for the current photo, and outputs frame 109 based on the manual exposure parameters for the next photo, and caches frame 109.

[0210] At time T2, the electronic device receives a trigger operation for the user to take a second photo (for example, pressing a photo button), and in response to the trigger operation, begins selecting frames for the second photo. Specifically, frames 106, 108, 109, and 110 are selected as the frames for the second photo, where frame 108 is the reference frame for the second photo and frame 109 is the manual exposure frame for the second photo. In addition, the electronic device also sets and issues manual exposure parameters for the next photo in advance to instruct the image sensor to output frame 113 according to the manual exposure parameters for the next photo, and caches frame 113.

[0211] Among them, frame 109 is set in advance when taking the first photo and is pre-cached. Therefore, when selecting a frame for the second photo, frame 109 can be directly obtained from the cache space as the manual exposure frame for the current photo, without having to set and send the manual exposure parameters for the current photo.

[0212] At time T3, the electronic device receives a trigger operation (e.g., pressing a photo button) for the user to take a third photo. In response to this trigger operation, the electronic device begins selecting frames for the third photo. Specifically, frames 110, 111, 112, and 113 are selected as the frames for the third photo, with frame 110 being the reference frame for the third photo and frame 113 being the manual exposure frame for the third photo. Furthermore, the electronic device also sets and issues manual exposure parameters for the next photo in advance, instructing the image sensor to output frame 116 based on the manual exposure parameters for the next photo, and caches frame 116.

[0213] Among them, frame 113 is set in advance during the second photo shooting and pre-cached. Therefore, when selecting a frame for the third photo shooting, frame 113 can be directly obtained from the cache space as the manual exposure frame for the current photo shooting, without having to set and send the manual exposure parameters for the current photo shooting.

[0214] At time T4, the electronic device receives a trigger operation for the user to take a fourth photo (for example, pressing a photo button), and in response to the trigger operation, begins selecting frames for the fourth photo. Specifically, frames 112, 114, 115, and 116 are selected as frames for the fourth photo, where frame 115 is the reference frame for the fourth photo, and frame 116 is the manual exposure frame for the fourth photo. In addition, the electronic device also sets and issues manual exposure parameters for the next photo in advance to instruct the image sensor to output the manual exposure frame for the next photo according to the manual exposure parameters for the next photo, and caches the manual exposure frame for the next photo.

[0215] Among them, frame 116 is set in advance during the third photo shooting and is pre-cached. Therefore, when selecting a frame for the fourth photo shooting, frame 116 can be directly obtained from the cache space as the manual exposure frame for the current photo shooting, without having to set and send the manual exposure parameters for the current photo shooting.

[0216] As shown in Figure 9, except for the need to wait for the manual exposure frame of the first shot to take effect, by setting and issuing the manual exposure parameters for the next shot in advance, the effective time of the manual exposure frames of the second, third and fourth shots is hidden, thereby improving the HDR shooting speed.

[0217] The embodiment of the present application can also provide a photo processing solution for frame multiplexing and pre-setting manual exposure frames to increase the photo speed without reducing the quality of the photo image.

[0218] Specifically, before setting the manual exposure frame for the next photo in advance, the electronic device first predicts whether the manual exposure frame used in the current photo will be available in the next photo. If the manual exposure frame used in the current photo is not available in the next photo, the manual exposure frame for the next photo is set in advance. If the manual exposure frame used in the current photo is available in the next photo, the manual exposure frame for the next photo does not need to be set in advance, and the manual exposure frame used in the current photo is reused in the next photo. In this way, the manual exposure frame for the next photo does not need to be set in advance for each photo. Instead, it is set in advance after predicting that the manual exposure frame used in the current photo will be unavailable for the next photo. This reduces the number of times the manual exposure frame is set and sent, thereby further improving the photo-taking speed.

[0219] For example, referring to FIG. 10A , which shows a schematic diagram of multiplexing manual exposure frames and pre-setting manual exposure frames according to an embodiment of the present application, an image sequence includes frames 100 to 117. In chronological order, frame 100 is the earliest and frame 117 is the latest. An image sequence may be image frames captured successively by an image sensor over a period of time.

[0220] Before time T1, the electronic device is in the photo preview stage and continues to cache image frames in the ZSL queue. As shown in FIG8A, frames 100, 101, 102, and 103 are captured before time T1.

[0221] At time T1, the electronic device receives a trigger operation (e.g., pressing a capture button) for the user's first photo. In response to this trigger operation, frame selection for the first photo begins. Specifically, frames 100, 101, 102, and 107 are selected as the frames for the first photo. Frame 100 is the base frame for the first photo, and frame 107 is the manually exposed frame for the first photo.

[0222] 10B shows a schematic block diagram of a process for obtaining a manual exposure frame according to an embodiment of the present application. Each time a frame is selected for a photo, a determination is first made as to whether a manual exposure frame is currently available. If not, a manual exposure frame is set for the current photo. If so, a prediction is made as to whether a manual exposure frame will be available for the next photo. If no manual exposure frame is available for the next photo, a manual exposure frame is set in advance for the next photo. If a manual exposure frame is available for the next photo, no manual exposure frame is set.

[0223] During frame selection for the first photo, according to the process shown in FIG10B , if it is determined that no manual exposure frame is available for the current photo, a manual exposure frame is set for the current photo. Specifically, the manual exposure parameters for the first photo are sent to the image sensor, and the image sensor outputs frame 107 based on the manual exposure parameters for the first photo.

[0224] Between time T1 and time T2, the electronic device is in the photo preview stage and continues to cache image frames in the ZSL queue.

[0225] At time T2, the electronic device receives a trigger operation (e.g., pressing a capture button) for the second photo. In response to this trigger operation, the electronic device begins selecting frames for the second photo. Specifically, frames 106, 107, 108, and 109 are selected as frames for the second photo. Frame 108 is the reference frame for the second photo, and frame 107 is reused as the manual exposure frame for the second photo.

[0226] When selecting a frame for the second photo, the electronic device determines, according to the process shown in FIG8B , that frame 107 is available for the current photo, i.e., a manual exposure frame is available for the current photo, and reuses the manual exposure frame from the first photo. Furthermore, the electronic device predicts whether a manual exposure frame is available for the next photo, i.e., predicts whether frame 107 is available for the next photo. In this case, if the electronic device predicts that frame 107 is available for the next photo, i.e., a manual exposure frame is available for the next photo, the manual exposure frame for the next photo is not set in advance.

[0227] Here, setting the manual exposure frame for the next photo may be sending a manual exposure parameter to the image sensor, instructing the image sensor to output the manual exposure frame for the next photo according to the manual exposure parameter.

[0228] Between time T2 and time T3, the electronic device is in the photo preview stage and continues to cache image frames in the ZSL queue.

[0229] At time T3, the electronic device receives a trigger operation (e.g., pressing a capture button) for the third photo. In response to this trigger operation, frame selection for the third photo is initiated. Specifically, frames 107, 110, 111, and 112 are selected as the frames for the third photo. Frame 112 is the reference frame for the third photo, and frame 107 is reused as the manual exposure frame for the third photo.

[0230] When selecting frames for the third photo, the electronic device determines, according to the process shown in FIG8B , that frame 107 is available for the current photo, and reuses the manually exposed frame from the first photo. Furthermore, the electronic device predicts whether a manually exposed frame will be available for the next photo, that is, whether frame 107 will be available for the next photo. If the electronic device predicts that frame 107 will not be available for the next photo, i.e., that there are no manually exposed frames available for the next photo, the electronic device sets a manually exposed frame for the next photo in advance. Specifically, the electronic device sends the manual exposure parameters for the fourth photo to the image sensor, which then outputs frame 116 based on the manual exposure parameters for the fourth photo and caches frame 116.

[0231] At time T4, the electronic device receives a trigger operation (e.g., pressing a capture button) for the fourth photo. In response to this trigger operation, frame selection for the fourth photo is initiated. Specifically, frames 113, 114, 115, and 116 are selected as frames for the fourth photo. Frame 115 is the reference frame for the fourth photo, and frame 116 is the manually exposed frame for the fourth photo.

[0232] When selecting a frame for the fourth photo, the electronic device determines, according to the process shown in FIG8B , that frame 116 is available for the current photo, and selects frame 116 as the manual exposure frame for the current photo. Furthermore, the electronic device predicts whether a manual exposure frame will be available for the next photo, that is, whether frame 116 will be available for the next photo. If not, a manual exposure frame is set for the next photo in advance; if available, no manual exposure frame is set.

[0233] Optionally, when determining whether a manual exposure frame is available for the current photo, the electronic device may determine the number of frames between the reference frame and the manual exposure frame for the current photo based on the frame number of the reference frame and the frame number of the manual exposure frame. If the number of frames is greater than a preset number, it indicates that the image content of the manual exposure frame differs significantly from that of the reference frame, and the electronic device determines that the manual exposure frame is unavailable for the current photo, i.e., no manual exposure frame is available for the current photo. If the number of frames is less than or equal to the preset number, it indicates that the image content of the manual exposure frame differs slightly from that of the reference frame, and the electronic device determines that the manual exposure frame is available for the current photo, i.e., no manual exposure frame is available for the current photo. The preset number can be set according to actual needs. For example, the preset number is 7.

[0234] If the image content of the manual exposure frame differs significantly from that of the reference frame, the fusion of the manual exposure frame and the reference frame will result in a poor final image quality. This embodiment of the present application uses the number of frames between the reference frame and the manual exposure frame of the current photo to rationally determine whether there are available manual exposure frames for the current photo. This allows for reuse of manual exposure frames while ensuring the quality of the photo, thereby increasing the speed of photo capture.

[0235] Optionally, when predicting whether there is an available manual exposure frame for the next photo, that is, determining whether the manual exposure frame used for the current photo is available for the next photo, the electronic device can predict the time of the next photo based on the photo taking time and the photo interval time of the current photo; if the time of the next photo or the time interval between the timestamp of the reference frame of the next photo and the timestamp of the manual exposure frame of the current photo is greater than a preset time threshold, it is determined that the manual exposure frame of the current photo is not available for the next photo; if the time interval between the timestamp of the reference frame of the next photo and the timestamp of the manual exposure frame of the current photo is less than or equal to the preset time threshold, it is determined that the manual exposure frame of the current photo is available for the next photo, that is, there is no available manual exposure frame for the next photo.

[0236] The preset time threshold can be set according to actual needs.

[0237] The photo interval can be a predicted value. Optionally, a shortest photo interval is determined by a preset number of photo intervals, and this shortest photo interval is determined as the photo interval. For example, the photo interval is 300ms.

[0238] It should be noted that in the scenarios shown in Figures 10A and 10B, frame multiplexing can be achieved using either the first or second solution. When using the first solution, frames are selected from the ZSL queue when taking a photo; when using the second solution, frames are selected from both the ZSL queue and the historical frame queue.

[0239] The following introduces the photographing speed before and after using frame multiplexing and setting the manual exposure frame in advance, in combination with the photographing speed schematic diagram provided by the embodiment of the present application shown in Figure 11A and the photographing speed schematic diagram provided by the present application shown in Figure 11B.

[0240] 11A shows the photographing speed when manual exposure frames are not multiplexed and manual exposure frames are not set in advance, and FIG. 11B shows the photographing speed when manual exposure frames are multiplexed and manual exposure frames are set in advance.

[0241] Figures 11A and 11B illustrate HDR photography, with the image sensor using staggered output. Four images are selected for fusion during each capture, including two normal exposure images, one short exposure image, and one long exposure image. The image sensor outputs frames at 30 fps.

[0242] As shown in FIG. 11A , the image sensor outputs 30 frames of images in 1 second, namely, frames 100 to 129 .

[0243] After receiving the trigger for the first photo, the electronic device sends the manual exposure parameters for that photo to the image sensor. The image sensor outputs frame 107 based on the manual exposure parameters. During frame selection for the first photo, frames 100, 101, 102, and 107 are selected as the frames for the first photo. Frame 100 is the base frame for the first photo, and frame 107 is the manually exposed frame for the first photo.

[0244] After receiving the trigger for the second photo, the electronic device sends the manual exposure parameters for that photo to the image sensor. The image sensor outputs frame 113 based on the manual exposure parameters. During frame selection for the second photo, frames 108, 109, 110, and 113 are selected as the frames for the second photo. Frame 108 is the reference frame for the second photo, and frame 113 is the manually exposed frame for the second photo.

[0245] After receiving the trigger for the third photo, the electronic device sends the manual exposure parameters for that photo to the image sensor. The image sensor outputs frame 119 based on the manual exposure parameters. During frame selection for the third photo, frames 114, 115, 116, and 119 are selected as the frames for the third photo. Frame 114 is the reference frame for the third photo, and frame 119 is the manually exposed frame for the third photo.

[0246] After receiving the trigger for the fourth photo, the electronic device sends the manual exposure parameters for that photo to the image sensor. The image sensor outputs frame 125 based on the manual exposure parameters. During frame selection for the fourth photo, frames 120, 121, 122, and 125 are selected as the frames for the third photo. Frame 120 is the reference frame for the fourth photo, and frame 125 is the manually exposed frame for the fourth photo.

[0247] As shown in FIG11B , the image sensor outputs 30 frames of images in 1 second, namely, frames 100 to 129 .

[0248] After receiving the triggering operation for the first photo, the electronic device determines that no manual exposure frames are available for this photo. It then sends the manual exposure parameters for this photo to the image sensor. The image sensor outputs frame 107 based on the manual exposure parameters. During frame selection for the first photo, frames 100, 101, 102, and 107 are selected as the frames for the first photo. Frame 100 is the base frame for the first photo, and frame 107 is the manual exposure frame for the first photo.

[0249] After receiving the triggering operation for the second photo, the electronic device determines that the manual exposure frame from the first photo is available for the second photo. In other words, a manual exposure frame is available for the current photo. It then uses multiplexed frame 107 as the manual exposure frame for the current photo. When selecting frames for the second photo, frames 106, 107, 108, and 109 are selected as the frames for the second photo. Frame 108 serves as the reference frame for the second photo, and multiplexed frame 107 serves as the manual exposure frame for the second photo.

[0250] In addition, the electronic device predicts that the frame 107 will be available for the next photo, that is, there will be an available manual exposure frame for the next photo, and the manual exposure frame for the next photo is not set in advance.

[0251] After receiving the triggering operation for the third photo, the electronic device determines that the manually exposed frame from the first photo is available for the current photo. Therefore, multiplexed frame 107 is used as the manually exposed frame for the current photo. When selecting frames for the third photo, frames 107, 110, 111, and 112 are selected as the frames for the third photo. Frame 112 serves as the reference frame for the third photo, and multiplexed frame 107 is used as the manually exposed frame for the second photo.

[0252] Furthermore, the electronic device predicts that frame 107 is unavailable for the next photo, i.e., there is no available manual exposure frame for the next photo, and thus sets a manual exposure frame for the next photo in advance. Specifically, the electronic device sends manual exposure parameters for the next photo to the image sensor, and the image sensor outputs frame 116 according to the manual exposure parameters and stores frame 116.

[0253] After receiving the triggering operation for the fourth photo, the electronic device determines that the manual exposure frame set for the third photo is available for the current photo. Therefore, it selects frame 116 as the manual exposure frame for the current photo. When selecting frames for the fourth photo, it selects frames 113, 114, 115, and 116 as the frames for the fourth photo. Frame 115 serves as the reference frame for the fourth photo.

[0254] In addition, the electronic device predicts that the frame 116 is not available for the next photo, that is, there is no available manual exposure frame for the next photo, and thus does not set the manual exposure frame for the next photo in advance.

[0255] After receiving the trigger for the fifth photo, the electronic device determines that the manual exposure frame for the fourth photo is available for this photo. Therefore, frame 116 is reused as the manual exposure frame for this photo. When selecting frames for the fifth photo, frames 115, 116, 117, and 118 are selected as the frames for this photo. Frame 118 serves as the reference frame for this photo.

[0256] In addition, the electronic device predicts that the frame 116 will be available for the next photo, that is, there will be an available manual exposure frame for the next photo, and the manual exposure frame for the next photo is not set in advance.

[0257] After receiving the triggering operation for the sixth photo, the electronic device determines that the manual exposure frame for the fifth photo is available for the current photo. Therefore, frame 116 is reused as the manual exposure frame for the current photo. When selecting frames for the sixth photo, frames 116, 119, 120, and 121 are selected as the frames for the sixth photo. Frame 121 is the reference frame for the sixth photo.

[0258] Furthermore, the electronic device predicts that frame 116 will not be available for the next photo, i.e., there is no available manual exposure frame for the next photo, and thus sets a manual exposure frame for the next photo in advance. Specifically, the electronic device sends manual exposure parameters for the next photo to the image sensor, and the image sensor outputs frame 125 according to the manual exposure parameters and stores frame 125.

[0259] After receiving the triggering operation for the seventh photo, the electronic device determines that the manual exposure frame set for the sixth photo is available for the current photo. Therefore, it selects frame 125 as the manual exposure frame for the current photo. When selecting frames for the seventh photo, it selects frames 122, 123, 124, and 125 as the frames for the seventh photo. Frame 114 serves as the reference frame for the seventh photo.

[0260] In addition, the electronic device predicts that frame 125 is not available for the next photo, that is, there is no available manual exposure frame for the next photo, and thus does not set the manual exposure frame for the next photo in advance.

[0261] After receiving the triggering operation for the eighth photo, the electronic device determines that a manual exposure frame for the eighth photo is available for the current shot. Therefore, frame 125 is reused as the manual exposure frame for the current shot. When selecting frames for the eighth photo, frames 124, 125, 126, and 127 are selected as the frames for the eighth photo. Frame 127 serves as the reference frame for the fifth photo.

[0262] Comparing Figures 11A and 11B, we can see that before frame multiplexing and pre-setting manual exposure frames, a maximum of four images can be captured in one second, or a capture speed of four frames per second. After frame multiplexing and pre-setting manual exposure frames, a maximum of eight images can be captured in one second, or a capture speed of eight frames per second. This improves HDR capture speed without sacrificing image quality.

[0263] The photo processing solution provided in the embodiment of the present application can be applied to an electronic device, which can be, for example, a mobile phone or a tablet computer. The type of electronic device is not limited here.

[0264] For example, referring to the structural diagram of electronic device 200 shown in FIG12 , electronic device 200 may include but is not limited to: a processor 210 , a memory 220 , a sensor module 230 , a camera 240 , and a display screen 250 , etc. The sensor module 230 may include but is not limited to: a touch sensor 231 .

[0265] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. For example, when the electronic device 200 is a mobile phone, the electronic device 200 may further include: a universal serial bus (USB) interface, a charging management module, a power management module, a battery, an antenna, a mobile communication module, a wireless communication module, an audio module, a speaker, a receiver, a microphone, a headphone jack, a button, a motor, an indicator, a subscriber identification module (SIM) card interface, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, an ambient light sensor, and a bone conduction sensor, etc.

[0266] The processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (AP), a graphics processing unit (GPU), an image signal processor, a controller, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0267] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0268] In some embodiments, the processor 210 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and the like.

[0269] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 210 may include multiple I2C bus lines. The processor 210 may be coupled to the touch sensor 231, the camera 240, and the like via different I2C bus interfaces. For example, the processor 210 may be coupled to the touch sensor 231 via the I2C interface, enabling communication between the processor 210 and the touch sensor 231 via the I2C bus interface, thereby enabling touch functionality of the electronic device 200.

[0270] The MIPI interface can be used to connect the processor 210 to peripheral devices such as the display 250 and the camera 240. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 210 and the camera 240 communicate via the CSI interface to implement the camera function of the electronic device 200. The processor 210 and the display 250 communicate via the DSI interface to implement the display function of the electronic device 200.

[0271] The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 210 to the camera 240, the display 250, the sensor module 230, etc. The GPIO interface can also be configured as an I2C interface or a MIPI interface, etc.

[0272] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0273] Electronic device 200 implements display functionality through a GPU, display screen 250, and an application processor. The GPU is a microprocessor for image processing that connects display screen 250 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or modify display information.

[0274] Display screen 250 is used to display images and videos, etc. Display screen 250 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 200 may include one or N display screens 250, where N is a positive integer greater than 1.

[0275] The electronic device 200 can implement a shooting function through an ISP, a camera 240, a video codec, a GPU, a display screen 250, and an application processor.

[0276] The ISP processes data fed back by camera 240. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 240.

[0277] The camera 240 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 200 may include 1 or N cameras 240, where N is a positive integer greater than 1.

[0278] The digital signal processor is used to process digital image signals.

[0279] Video codecs are used to compress or decompress digital video. Electronic device 200 may support one or more video codecs. This allows electronic device 200 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0280] The memory 220 can be used to store computer executable program codes, which include instructions. The memory 220 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 200 (such as audio data, a phone book, etc.), etc. In addition, the memory 220 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 210 executes various functional applications and data processing of the electronic device 200 by running instructions stored in the internal memory 220 and / or instructions stored in a memory provided in the processor.

[0281] Touch sensor 231, also known as a "touch control device," can be mounted on display screen 250. Together, touch sensor 231 and display screen 250 form a touch screen, also known as a "touch screen." Touch sensor 231 detects touch operations applied to or near the touch sensor. The touch sensor can communicate the detected touch operations to an application processor to determine the type of touch event. Display screen 250 can also provide visual output related to the touch operations.

[0282] After introducing the possible hardware architecture of the electronic device 200 according to the embodiment of the present application, the following describes the subsequent content by taking the electronic device 200 as an example.

[0283] Referring to FIG. 13 , which shows a schematic flow chart of a photograph processing method provided in an embodiment of the present application, the photograph processing method may include the following steps:

[0284] Step S1301: The electronic device 200 displays a photo preview interface.

[0285] Step S1302: The electronic device 200 selects a first photographic frame in response to a first photographing operation on the photographing preview interface, and sends a first manual exposure parameter to the image sensor. The first photographic frame includes at least one first image and at least one first manual exposure frame, and the exposure amount of the first manual exposure frame is different from that of the first image. The first manual exposure parameter is used to instruct the image sensor to perform exposure processing according to the first manual exposure parameter to obtain a second manual exposure frame, and the second manual exposure frame is cached in the cache space.

[0286] For example, based on the photo shooting scene shown in FIG1 , the first photo shooting operation may be an operation in which the user presses the photo shooting button 13. Of course, the first photo shooting operation may also be a voice operation input by the user in the photo shooting preview interface 12. The first photo shooting operation is used to trigger a photo shooting.

[0287] The first manual exposure frame has an exposure level different from that of the first image. Exemplarily, the first image is a normal exposure image, and the first manual exposure frame includes a long exposure image and a short exposure image.

[0288] The first manual exposure parameter is used to set a manual exposure frame for the next photo. That is, upon receiving the first photo operation, electronic device 200 not only selects the photo frame corresponding to the first photo operation but also sets a manual exposure frame for the next photo operation. The second manual exposure frame is the manual exposure frame for the next photo operation.

[0289] In some optional embodiments, the electronic device 200 enters the photo preview phase when displaying the photo preview interface. In the photo preview phase, not only is a photo preview image generated, but also an image frame (e.g., a RAW image) corresponding to the photo preview image is cached in the ZSL queue. At this time, after receiving the first photo frame, the electronic device 200 can select the first photo frame from the ZSL queue, or select the first photo frame from the ZSL queue and the historical frame queue. The second manual exposure frame can be cached in the ZSL queue. The first manual exposure frame and the first image can be cached in the ZSL queue or stored in other storage spaces.

[0290] The cache space may be a ZSL queue or other storage space, which is not limited here.

[0291] The first manual exposure frame may be a reused manual exposure frame of the previous photograph, or may be a manual exposure frame obtained by setting and issuing manual exposure parameters for the current photograph after receiving the first photograph operation.

[0292] Step S1303: The electronic device 200 generates a first photographic image according to the first photographic frame.

[0293] Exemplarily, the electronic device 200 may perform a photographing algorithm on the first photographing frame to obtain a first photographing image.

[0294] Optionally, after selecting the first frame for photographing, the electronic device 200 may not immediately execute all photographic algorithm processing on the first frame during the photographing process. Instead, the electronic device 200 may first persistently store the first frame for photographing (for example, in a file system), wait for an appropriate time to read the first frame from the storage medium, execute the remaining photographic algorithm processing, and obtain the first photographed image. This can reduce the amount of computation during the photographing process, reduce the probability of the electronic device experiencing freezes due to excessive load and system pressure, and further improve the photographing speed.

[0295] The appropriate time may be, for example: the electronic device 200 exits the photo-taking process, such as exiting and closing the camera application, or the camera application running in the background; is in an idle state, such as turning off the screen or keeping the screen off; viewing, sharing or editing images.

[0296] Step S1304: The electronic device 200 selects a second photographing frame in response to a second photographing operation on the photographing preview interface. The second photographing frame includes at least one second image and at least one second manual exposure frame selected from the cache space. The exposure amount of the second manual exposure frame is different from that of the second image. The second photographing operation is performed after the first photographing operation.

[0297] Exemplarily, the second image is a normal exposure image, and the second manual exposure frame includes a long exposure image and a short exposure image. The electronic device 200 may select at least one second image from the ZSL queue, or the ZSL queue and the history frame queue.

[0298] The second photo-taking operation is used to trigger photo-taking. For example, the second photo-taking operation is an operation of the user pressing the photo-taking button 13 or a voice input operation performed by the user in the photo-taking preview interface 12 .

[0299] After receiving the second photo-taking operation, the electronic device 200 can directly obtain the second manual exposure frame from the cache space as the manual exposure frame for the current photo, because the manual exposure frame for the next photo was set in advance during the first photo-taking operation. There is no need to set the manual exposure frame for the current photo, and the effective time of the manual exposure frame is hidden, thereby improving the photo-taking speed.

[0300] Step S1305: The electronic device 200 generates a second photographic image based on the second photographic frame.

[0301] Optionally, when the electronic device 200 selects the second photographic frame, it may not immediately execute all photographic algorithm processing on the second photographic frame during the photographing process, but instead wait for the right time to read the first photographic frame from the storage medium, perform the remaining photographic algorithm processing, and obtain the first photographic image.

[0302] As can be seen from the above, the embodiment of the present application improves the shooting speed by setting and issuing manual exposure parameters for the next photo in advance after receiving the photo operation, that is, setting the manual exposure frame for the next photo in advance, hiding the time when the manual exposure parameters take effect.

[0303] In some optional embodiments, the electronic device 200 may set a manual exposure frame for the next photo in advance each time the photo is taken.

[0304] Of course, in other optional embodiments, the electronic device 200 may also determine whether to set a manual exposure frame for the next photo by judging whether the manual exposure frame of the current photo is available for the next photo.

[0305] Specifically, before sending the first manual exposure parameter to the image sensor, the electronic device 200 may also determine whether the first manual exposure frame is available for the next photo. If the first manual exposure frame is not available for the next photo, the electronic device 200 sends the first manual exposure parameter to the image sensor to instruct the image sensor to output the manual exposure frame required for the next photo (i.e., the second manual exposure frame) according to the first manual exposure parameter.

[0306] If the first manual exposure frame is unavailable for the next photo, no manual exposure frame is set for the next photo, i.e., the first manual exposure parameter is not sent. In this case, upon receiving the second photo operation, electronic device 200 can select the first manual exposure frame and at least one second image as the second photo frame in response to the second photo operation. In other words, the manual exposure frame from the previous photo is reused.

[0307] In the embodiment of the present application, the electronic device 200 first determines whether the first manual exposure frame used in the current photo is available for the next photo, and then determines whether to set a manual exposure frame for the next photo. This allows the same manual exposure frame to be reused between two photo operations, achieving manual exposure frame reuse. Furthermore, the number of manual exposure parameter updates can be reduced, further improving photo capture speed. In some optional embodiments, when determining whether the first manual exposure frame is available for the next photo, the electronic device 200 can predict the next photo capture time based on the photo capture time and photo capture interval of the first photo capture operation. If the time interval between the next photo capture time or the timestamp of the next photo capture's reference frame and the timestamp of the first manual exposure frame is greater than a preset time threshold, the electronic device 200 determines that the first manual exposure frame is unavailable for the next photo capture. If the time interval between the next photo capture time or the timestamp of the next photo capture's reference frame and the timestamp of the first manual exposure frame is less than or equal to the preset time threshold, the electronic device 200 determines that the first manual exposure frame is available for the next photo capture. This ensures photo quality while achieving manual exposure frame reuse and improving photo capture speed. The preset time threshold can be set based on actual needs and is not limited herein.

[0308] In some optional embodiments, the electronic device 200 may, in response to the second photographing operation, send a second manual exposure parameter to the image sensor to instruct the image sensor to perform exposure processing according to the second manual exposure parameter, thereby obtaining a third manually exposed frame, which is then cached in the cache space. The third manually exposed frame is the manually exposed frame for the next photographing operation, i.e., the manually exposed frame is set in advance for the next photographing operation.

[0309] Alternatively, in response to the second photographing operation, if the electronic device 200 determines that the second manual exposure frame is not available for the next photographing operation, the electronic device 200 may send a second manual exposure parameter to the image sensor to instruct the image sensor to perform exposure processing according to the second manual exposure parameter to obtain a third manual exposure frame, and cache the third manual exposure frame in the cache space. If the second manual exposure frame is available for the next photographing operation, the second manual exposure parameter is not sent to the image sensor.

[0310] In some optional embodiments, part of the first image and part of the second image are the same image. This allows frame multiplexing and improves the speed of photographing. For example, as shown in FIG7 , part of the first image may be frame n-1 of the previous photograph, and part of the second image may be frame n-1 of the next photograph. In addition, another part of the first image may be frames n and n-3 of the previous photograph, and another part of the second image may be frames n+1 and n+2 of the next photograph. The first manual exposure frame and the second manual exposure frame are frame n-2. In this way, frame n-2 (i.e., the manual exposure frame) is multiplexed between the two photographs, and frame n-1 (the non-manual exposure frame) is also multiplexed.

[0311] In some optional embodiments, when displaying the photo preview interface, the electronic device 200 may also cache images captured by the image sensor and corresponding to the photo preview interface in a cache space. In this case, the first image is an image cached in the cache space before the first photo operation, the first manual exposure frame is an image cached in the cache space before the first photo operation or before the second photo operation, and the second image is an image cached in the cache space before the second photo operation. For example, the cache space is a ZSL queue, and the image corresponding to the photo preview image is cached in the ZSL queue.

[0312] In some optional embodiments, the first photographing frame includes a cached image in the cache space, or includes a cached image in the cache space and a historical cached image, where the historical cached image is an image cached in the cache space before the first photographing operation, but not cached in the cache space during the first photographing operation and persistently stored in the storage medium; the second image is a cached image in the cache space, or an image cached in the cache space before the second photographing operation, but not cached in the cache space during the second photographing operation and persistently stored in the storage medium.

[0313] Exemplarily, the cache space may be a ZSL queue, and the storage medium may be a hard disk, etc. The historical cached image may be the aforementioned photographic frame written to the file system.

[0314] In some optional embodiments, when the electronic device 200 is selecting the first frame for photographing, if there is a target cache image in the cache space, or there is first target information in the historical frame storage space, the target cache image or the image corresponding to the first target information is selected as the first manual exposure frame, and the historical frame storage space is used to store the information of the photographic frames selected for historical photographing. The photographic frames selected for historical photographing are persistently stored in the storage medium, and the target cache image and the image corresponding to the first target information are manual exposure frames available for the current photographing; at least one first image is selected from the historical frame storage space and / or the cache space; for the image selected from the cache space, the selected image is deleted from the cache space, and the selected image is persistently stored in the storage medium, and the information of the selected image is stored in the historical storage space, and the selected image includes the first manual exposure frame and / or the first image.

[0315] Exemplarily, the historical frame storage space may refer to the historical frame queue described above. The information about the frames used for photographing may refer to metadata about the frames used for photographing. The cache space may be a ZSL queue. The target cache image or the image corresponding to the first target information is a manually exposed frame available for the current photograph. Determining whether a manually exposed frame is available for the current photograph may be determined by the number of frames between the reference frame and the manually exposed frame. For example, if an image within seven frames of the reference frame used for the current photograph exists in the cache space or the historical frame storage space, the image is the target cache image, or the information corresponding to the image is the first target information.

[0316] In some optional embodiments, if the electronic device 200 determines that the target cache image does not exist in the cache space and the first target information does not exist in the historical frame storage space, the electronic device 200 sends a fourth manual exposure parameter to the image sensor to instruct the image sensor to perform exposure processing according to the fourth manual exposure parameter to obtain a manually exposed frame, and selects the manually exposed frame as the first manually exposed frame. That is, if there is no reusable manually exposed frame, the electronic device 200 can send the manual exposure parameter for the current photo after receiving the current photo operation to obtain the manual exposed frame required for the current photo.

[0317] In some optional embodiments, the number of frames between the manual exposure frame available for the second photo and the reference frame of the first photo operation is less than a preset number, for example, 7. Alternatively, the time interval between the timestamp of the manual exposure frame available for the second photo and the timestamp of the reference frame of the first photo operation is less than a preset threshold.

[0318] In some optional embodiments, when selecting the second frame for photographing, the electronic device 200 may select a second manual exposure frame from the cache space, delete the second manual exposure frame in the cache space, persistently store the second manual exposure frame in a storage medium, and store information of the second manual exposure frame in a historical storage space; select a second image from the cache space, delete the second image in the cache space, persistently store the second image in a storage medium, and store information of the second image in a historical storage space; and / or select second target information from the historical storage space, and the image corresponding to the second target information is the second image.

[0319] In some optional embodiments, when selecting a first frame for photographing, the electronic device 200 determines that a target cached image exists in the cache space, selects the target cached image as the first manual exposure frame, where the target cached image is a manual exposure frame available for photographing the current time; selects at least one first image from the cache space; and maintains the first manual exposure frame and the first image cached in the cache space until they are removed by the storage mechanism of the cache space. The storage medium may be a first-in, first-out mechanism of a ZSL queue.

[0320] Accordingly, when selecting the second frame for photographing, the electronic device 200 can select the second manual exposure frame and the second image from the cache space, and maintain the second manual exposure frame and the second image cached in the cache space, waiting to be removed by the storage mechanism of the cache space.

[0321] In some optional embodiments, when the electronic device 200 is generating a first photographic image based on the first photographic frame, if an operation of viewing a thumbnail is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the first photographic frame is obtained from the storage medium, and the first photographic frame is processed by a photographic algorithm to obtain a first photographic image.

[0322] Accordingly, when the electronic device 200 is generating a second photographic image based on the second photographic frame, if an operation of viewing thumbnails is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the second photographic frame is obtained from the storage medium, and the second photographic frame is processed by a photographic algorithm to obtain a second photographic image.

[0323] In some optional embodiments, the preset trigger conditions include at least one of the following: an operation of opening an image access application, an operation of viewing an image, an operation of sharing an image, an operation of editing an image, the system being in idle state, the screen being off or stopped, the running memory being higher than a first threshold, the device temperature being lower than a first temperature threshold, and the processor load being lower than a second threshold.

[0324] Referring to FIG. 14 , a schematic flow chart of a photograph processing method provided in an embodiment of the present application is shown. The photograph processing method may include the following steps:

[0325] Step S1401: The electronic device 200 caches multiple frames of images captured by the image sensor in a ZSL queue in a photo preview mode.

[0326] Step S1402: In response to the first photographing operation, the electronic device 200 selects a first photographing frame from the ZSL queue, persistently stores the first photographing frame to a storage medium, stores metadata of the first photographing frame in a historical frame queue, and deletes the first photographing frame from the ZSL queue.

[0327] Step S1403: The electronic device 200 generates a first photographic image according to the first photographic frame in the storage medium.

[0328] Step S1404: In response to the second photographing operation, the electronic device 200 selects a target image from the ZSL queue, selects target metadata from the metadata of the first photographing frame in the historical frame queue, persistently stores the target image to a storage medium, and deletes the target image in the cache queue.

[0329] Step S1405: The electronic device 200 generates a second photographic image based on the second photographic frame in the storage medium, wherein the second photographic frame includes the target image and the photographic frame corresponding to the target metadata. A portion of the image of the first photographic frame is identical to a portion of the image of the second photographic frame.

[0330] As can be seen from the above, the implementation of this application to achieve frame multiplexing of frames for taking pictures through historical frame queues and storage media can not only improve the shooting speed, but also is not limited by the length of the ZSL queue, and can ensure that memory is saved while ensuring the frame multiplexing rate.

[0331] In some optional embodiments, when the electronic device 200 is generating a first photographic image based on the first photographic frame, if an operation of viewing a thumbnail is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the first photographic frame is obtained from the storage medium, and the first photographic frame is processed by a photographic algorithm to obtain a first photographic image.

[0332] Accordingly, when the electronic device 200 is generating a second photographic image based on the second photographic frame, if an operation of viewing thumbnails is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the second photographic frame is obtained from the storage medium, and the second photographic frame is processed by a photographic algorithm to obtain a second photographic image.

[0333] In some optional embodiments, the preset trigger conditions include at least one of the following: opening an image access application, viewing an image, sharing an image, editing an image, the system being idle, the screen being off or off, the running memory being above a first threshold, the device temperature being below a first temperature threshold, and the processor load being below a second threshold. In this way, when a user needs to view, edit, or share image data, the photo algorithm processing can be performed promptly, allowing the user to view the captured image in a timely manner. Furthermore, the photo algorithm processing can be automatically triggered when the system status meets the conditions, i.e., the running memory, device temperature, and processor load meet the requirements, further alleviating issues such as device lag and overheating.

[0334] In some optional embodiments, the electronic device 200 may, after receiving the first photographing operation, not only select the first photographing frame, but also set the manual exposure parameters, and send the manual exposure parameters to the image sensor to instruct the image sensor to output the manual exposure frame used for the next photographing according to the manual exposure parameters, and cache it in the ZSL queue. In this way, after receiving the next photographing operation, that is, the second photographing operation, the electronic device can select the manual exposure frame available for the current photographing and images of other exposure amounts (such as normal exposure images) from the ZSL queue. It is no longer necessary to set the manual exposure parameters for the current photographing after receiving the second photographing operation and wait for the manual exposure parameters for the current photographing to take effect. The time between the manual exposure parameter setting and the manual exposure parameter taking effect is hidden, thereby further improving the HDR photographing speed.

[0335] In some optional embodiments, the electronic device 200 may set manual exposure parameters for the next photo in advance and send the manual exposure parameters each time a photo is taken.

[0336] In some optional embodiments, after selecting a frame for taking a photo, the electronic device 200 may also determine whether the manual exposure frame used for the current photo is available for the next photo, before setting and issuing the manual exposure parameters for the next photo; if the manual exposure frame used for the current photo is available for the next photo, there is no need to set and issue the manual exposure parameters for the next photo in advance, so that the manual exposure frame used for the current photo can be reused for the next photo, that is, manual exposure frame reuse; if the manual exposure frame used for the current photo is not available for the next photo, the manual exposure parameters for the next photo are set and issued in advance.

[0337] In some optional embodiments, during the process of selecting frames for taking photos, the electronic device 200 may first determine whether there is a manual exposure frame available for the current photo in the ZSL queue and / or the historical frame queue, that is, determine whether there is a reusable manual exposure frame; if there is a reusable manual exposure frame, then select the manual exposure frame as the manual exposure frame for the current photo. After selecting the frame for the current photo, further predict whether the manual exposure frame used for the current photo will be available for the next photo; if there is no manual exposure frame available for the current photo in the ZSL queue and the historical frame queue, it is necessary to set and send manual exposure parameters to instruct the image sensor to output the manual exposure frame according to the manual exposure parameters, and cache the manual exposure frame in the ZSL queue; after the image sensor outputs the manual exposure frame, select the manual exposure frame and other images from the ZSL queue to obtain the frame for the current photo.

[0338] In some optional embodiments, the electronic device 200 may, in the process of determining whether the manual exposure frame used for the current photo is available for the next photo, predict the time of the next photo based on the photo time and the photo interval of the current photo operation; if the time of the next photo or the time stamp of the base frame of the next photo and the time stamp of the manual exposure frame used for the current photo are greater than a preset time threshold, it is determined that the manual exposure frame used for the current photo is not available for the next photo; if the time of the next photo or the time stamp of the base frame of the next photo and the time stamp of the manual exposure frame used for the current photo are less than or equal to the preset time threshold, it is determined that the manual exposure frame used for the current photo is available for the next photo.

[0339] In some optional embodiments, when the electronic device determines whether there is a manual exposure frame available for the current photo in the ZSL queue and / or the historical frame queue, it can be determined based on the number of frames between the manual exposure frame and the reference frame of the current photo. If a manual exposure frame exists and the number of frames between the manual exposure frame and the reference frame of the current photo is less than or equal to a preset number of frames (e.g., 7 frames), the manual exposure frame is considered available for the current photo; conversely, if the number of frames is greater than the preset number of frames, the manual exposure frame is considered unavailable for the current photo. Alternatively, the determination can be made based on the time interval between the timestamp of the manual exposure frame and the timestamp of the reference frame of the current photo.

[0340] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0341] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0342] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0343] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0344] In the embodiments provided in this application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0345] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0346] The electronic device provided in the embodiments of the present application may include a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a method as described in any one of the above method embodiments is implemented.

[0347] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0348] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0349] The present application also provides a chip system, comprising a processor coupled to a memory, and executing a computer program stored in the memory to implement the methods described in the above method embodiments. The chip system can be a single chip or a chip module composed of multiple chips.

[0350] In the above embodiments, the descriptions of each embodiment have different emphases. For portions not described or documented in detail in a particular embodiment, reference should be made to the relevant descriptions of other embodiments. It should be understood that the sequence numbers of the steps in the above embodiments do not imply a sequential order of execution. The execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation of the embodiments of this application. Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used solely to distinguish descriptions and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features designated as "first," "second," "third," or "fourth" may explicitly or implicitly include one or more of such features. Furthermore, it should be understood that "at least one" in the embodiments of this application includes one or more, where "more" means greater than or equal to two. In the embodiments of this application, "and / or" is simply a description of an association between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. In addition, the character “ / ” in this article generally indicates that the previous and next related objects are in an “or” relationship.

[0351] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0352] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A photo processing method, characterized in that: Applied to electronic equipment, the method includes: Display the photo preview interface; In response to a first photographing operation on the photographing preview interface, a first photographing frame is selected, and a first manual exposure parameter is sent to the image sensor, where the first photographing frame includes at least one first image and at least one first manually exposed frame, and the first manually exposed frame and the first image have different exposure amounts; the first manual exposure parameter is used to instruct the image sensor to perform exposure processing according to the first manual exposure parameter to obtain a second manually exposed frame, and the second manually exposed frame is cached in a cache space; generating a first photographic image according to the first photographic frame; In response to a second photographing operation on the photographing preview interface, selecting a second photographing frame, the second photographing frame including at least one second image and at least one second manual exposure frame selected from the cache space, the second manual exposure frame and the second image having a different exposure amount; the second photographing operation being performed after the first photographing operation; A second photographic image is generated based on the second photographic frame.

2. The method according to claim 1, characterized in that Before sending the first manual exposure parameter to the image sensor, the method further includes: determining whether the first manual exposure frame is available for next photo taking; If the first manual exposure frame is not available for the next photo taking, the step of sending the first manual exposure parameter to the image sensor is entered; If the first manual exposure frame is available for the next photographing operation, then in response to the second photographing operation, the first manual exposure frame and at least one second image are selected as frames for the second photographing operation.

3. The method according to claim 2, characterized in that The determining whether the first manual exposure frame is available for next photo taking includes: predicting a next photo-taking time based on the photo-taking time and photo-taking interval of the first photo-taking operation; If the time interval between the time of the next photograph or the timestamp of the reference frame of the next photograph and the timestamp of the first manual exposure frame is greater than a preset time threshold, determining that the first manual exposure frame is unavailable for the next photograph, and the timestamp of the reference frame of the next photograph is determined based on the time of the next photograph; If the time interval between the time of the next photograph or the timestamp of the reference frame of the next photograph and the timestamp of the first manual exposure frame is less than or equal to the preset time threshold, it is determined that the first manual exposure frame is available for the next photograph.

4. The method according to claim 1 or 2, characterized in that The method further comprises: In response to the second photographing operation, sending a second manual exposure parameter to the image sensor to instruct the image sensor to perform exposure processing according to the second manual exposure parameter to obtain a third manual exposure frame, and buffering the third manual exposure frame in the buffer space; Alternatively, in response to the second photo-taking operation, if it is determined that the second manual exposure frame is not available for the next photo-taking, a second manual exposure parameter is sent to the image sensor to instruct the image sensor to perform exposure processing according to the second manual exposure parameter to obtain a third manual exposure frame, and the third manual exposure frame is cached in the cache space.

5. The method according to any one of claims 1 to 4, characterized in that Part of the first image and part of the second image are the same image.

6. The method according to claim 5, characterized in that The method further comprises: When the photo preview interface is displayed, the image corresponding to the photo preview interface and captured by the image sensor is cached in the cache space; The first image is an image cached in the cache space before the first photographing operation, and the first manual exposure frame is an image cached in the cache space before the first photographing operation or before the second photographing operation; The second image is an image cached in the cache space before the second photographing operation.

7. The method according to claim 6, characterized in that The cache space is an image cache queue for zero-delay photography.

8. The method according to claim 6 or 7, characterized in that The first photographing frame includes a cached image in the cache space, or includes a cached image in the cache space and a historical cached image, wherein the historical cached image is an image that was cached in the cache space before the first photographing operation but was not cached in the cache space during the first photographing operation and is persistently stored in a storage medium; The second image is a cached image in the cache space, or an image cached in the cache space before the second photographing operation but not cached in the cache space during the second photographing operation and persistently stored in the storage medium.

9. The method according to claim 8, characterized in that The selecting of the first photographing frame includes: If a target cache image exists in the cache space, or first target information exists in the historical frame storage space, the target cache image or the image corresponding to the first target information is selected as the first manual exposure frame. The historical frame storage space is used to store information of photographic frames selected for historical photographs. The photographic frames selected for historical photographs are persistently stored in the storage medium. The target cache image and the image corresponding to the first target information are manual exposure frames available for the current photograph. Selecting at least one first image from the historical frame storage space and / or the cache space; For the image selected from the cache space, the selected image is deleted from the cache space, and the selected image is persistently stored in the storage medium, and the information of the selected image is stored in the historical storage space, where the selected image includes the first manual exposure frame and / or the first image.

10. The method according to claim 9, characterized in that The method further comprises: If the target cache image does not exist in the cache space and the first target information does not exist in the historical frame storage space, a fourth manual exposure parameter is sent to the image sensor to instruct the image sensor to perform exposure processing according to the fourth manual exposure parameter, obtain a manual exposure frame, and select the manual exposure frame as the first manual exposure frame.

11. The method according to claim 9, characterized in that The number of frames between the manual exposure frame available for the current photo and the reference frame of the first photo operation is less than a preset number, or the time interval between the timestamp of the manual exposure frame available for the current photo and the timestamp of the reference frame of the first photo operation is less than a preset threshold.

12. The method according to claim 9, characterized in that The selecting of the second frame for photographing includes: Selecting the second manual exposure frame from the cache space, deleting the second manual exposure frame in the cache space, persistently storing the second manual exposure frame in the storage medium, and storing information of the second manual exposure frame in the history storage space; Select the second image from the cache space, delete the second image in the cache space, persistently store the second image in the storage medium, and store the information of the second image in the historical storage space; and / or select second target information from the historical storage space, and the image corresponding to the second target information is the second image.

13. The method according to claim 8, characterized in that The selecting of the first photographing frame includes: If a target cache image exists in the cache space, the target cache image is selected as the first manual exposure frame, where the target cache image is a manual exposure frame available for the current photo shooting; Selecting at least one of the first images from the cache space; Maintaining the first manual exposure frame and the first image to continue to be cached in the cache space, waiting to be removed by a storage mechanism of the cache space; The selecting of the second frame for photographing includes: The second manual exposure frame and the second image are selected from the cache space, and the second manual exposure frame and the second image are kept cached in the cache space, waiting to be removed by a storage mechanism of the cache space.

14. The method according to claim 1, wherein Generating a first photographic image according to the first photographic frame includes: If a thumbnail viewing operation is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the first photographing frame is obtained from the storage medium, and the first photographing frame is processed by a photographing algorithm to obtain the first photographed image; Generating a second photographic image according to the second photographic frame includes: If an operation of viewing thumbnails is detected, or the camera application is running in the background or closed, or the camera application is running in the background or closed and a preset trigger condition is detected, the second photographic frame is obtained from the storage medium, and the second photographic frame is processed by a photographic algorithm to obtain the second photographic image.

15. The method according to claim 14, characterized in that The preset trigger conditions include at least one of the following: opening an image access application, viewing an image, sharing an image, editing an image, the system being in idle state, the screen being off or inactive, the running memory being higher than a first threshold, the device temperature being lower than a first temperature threshold, and the processor load being lower than a second threshold.

16. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 15 when executing the computer program.

17. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images