Photographing method and electronic device

By caching and replacement mechanism based on preview image score values ​​in photography technology, the problem of capture delay caused by human reaction lag and shutter delay is solved, the matching degree and memory utilization efficiency of manual capture images are improved, and more efficient shooting is achieved Experience.

WO2024179039A9PCT designated stage expired Publication Date: 2024-12-05HONOR DEVICE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2023/131280
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-02-27
Filing Date
2023-11-13
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

In the existing photography technology, due to the delay in human reaction and the delay in camera shutter, it is difficult for users to accurately capture exciting scenes. The existing solutions require a large number of preview images, occupying a large amount of memory space, and cannot effectively improve the manual capture of images and user expected images. match.

Method used

By determining the score value of each frame of preview image in the capture mode, the candidate images falling into the observation time window are cached based on the score value, and when the photo operation is detected, the image to be compared is determined from the cache, based on the The image and preview images within the preset duration are determined to manually capture images to realize image replacement and cache, saving memory space.

Benefits of technology

Improve the matching degree between manual capture images and user expected images, save memory space, expand the frame selection range, and enhance the shooting experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023131280_05122024_PF_FP_ABST
    Figure CN2023131280_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present application is suitable for the technical field of terminals, and provides a photographing method and an electronic device. By means of determining a corresponding candidate image from among each group of preview images that sequentially fall into an observation time window, each time after a candidate image of one group of preview images is determined, caching the currently determined candidate image into one of preset first cache regions, when all the first cache regions are occupied, using the currently determined candidate image to replace candidate images in all the first cache regions having the earliest cache time, when a photographing operation is detected, determining an image to be compared from among candidate images currently cached in the first cache regions and an observation cache region, and determining a manual snapshot image corresponding to the photographing operation from among the image to be compared and all the preview images within a preset duration, not only can the matching degree between the manual snapshot image and an image expected by a user be improved, but a memory space can also be saved on.
Need to check novelty before this filing date? Find Prior Art

Description

Photographing method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on February 27, 2023, with application number 202310212391.2 and application name “Photographing method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a photographing method and electronic device. Background Art

[0003] In daily life, people often take photos to record some wonderful scenes (such as the scene of jumping to the highest point or the scene of looking back and smiling, etc.). However, due to factors such as slow human reaction and camera shutter delay, the scene actually captured by the user is often delayed after the wonderful scene the user sees. In order to help users accurately capture wonderful scenes, a common solution is to cache all preview images within a certain time range before and after the user presses the shutter, and then select several more exciting preview images from the cached preview images for the user to choose. However, this solution requires caching all preview images within a certain time range before and after the user presses the shutter, which occupies a large amount of memory space.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a photographing method and an electronic device, which can not only improve the matching degree between the manually captured image finally output and the user's expected image, but also save memory space.

[0006] In a first aspect, an embodiment of the present application provides a photographing method, comprising:

[0007] In snapshot mode, determine the score value of each preview image frame;

[0008] Based on the score value of each preview image frame, determining candidate images for each group of preview images from each group of preview images that sequentially fall within the observation time window over time;

[0009] Each time a group of candidate images for preview images is determined, the currently determined candidate image is cached in one of one or more preset first cache areas. When all the first cache areas are occupied, the currently determined candidate image is fixed in a preset observation cache area. After the candidate images for the next group of preview images are determined, the currently determined candidate image is used to replace the candidate image with the earliest cache time in all the first cache areas.

[0010] When a photographing operation is detected, in response to the photographing operation, determining an image to be compared for the photographing operation from candidate images currently cached in all first buffers and the observation buffer;

[0011] Based on the score value of the image to be compared and the score value of each frame of the preview image within a preset time period after the photo operation occurs, the manually captured image corresponding to the photo operation is determined from the image to be compared and all preview images within the preset time period.

[0012] The candidate images currently cached in all the first buffer areas and the observation buffer areas refer to the candidate images that have been cached in all the first buffer areas and the observation buffer areas when the photographing operation occurs.

[0013] In an optional implementation, the electronic device may determine, as the image to be compared, the candidate image with the highest score among all the candidate images cached in the first buffer area and the observation buffer area at the time when the photographing operation occurs.

[0014] The photographing method provided in the embodiment of the present application, by setting an observation time window and one or more preset first buffer areas, only caches the candidate images of each group of preview images that fall within the observation time window, and when all the first buffer areas are dedicated, replaces the candidate image with the earliest cache time in the first buffer area with the most recently determined candidate image. In this way, there is no need to set up a large number of first buffer areas, and the storage capacity of each first buffer area only needs to accommodate one candidate image, thereby saving memory space; in addition, since the final manually captured image is determined from the images to be compared cached before the moment the photographing operation occurs and all preview images within a preset time length after the moment the photographing operation occurs, the frame selection range of the manually captured image finally outputted can cover both a period of time in the historical direction of the moment the photographing operation occurs and a period of time in the future direction of the moment the photographing operation occurs, that is, the frame selection range of the manually captured image is expanded, thereby improving the matching degree between the manually captured image and the user's expected image.

[0015] It can not only help users capture the most exciting photos within a period of time before and after the photo operation, improve the matching degree between manually captured images and users' expected images, but also save memory space.

[0016] In an optional implementation of the first aspect, determining, from each group of preview images that sequentially fall within the observation time window over time, candidate images for each group of preview images includes:

[0017] Preview images in each group of preview images whose scores are greater than a preset score threshold are respectively determined as candidate images for each group of preview images.

[0018] According to the photographing method provided in the embodiment of the present application, by determining the preview images in each group of preview images whose score values ​​are greater than the preset score threshold as candidate images for each group of preview images, the score values ​​of the manually captured images finally output by the electronic device can be greater than the preset score threshold, thereby improving the wonderfulness of the manually captured images.

[0019] In an optional implementation of the first aspect, determining the manually captured image corresponding to the photographing operation from the image to be compared and all preview images within the preset time period based on the score value of the image to be compared and the score value of each preview image frame within a preset time period after the photographing operation occurs includes:

[0020] If no other photographing operation is detected again within the preset time length, the image with the highest score among the image to be compared and all preview images within the preset time length is determined as the manually captured image corresponding to the photographing operation.

[0021] According to the photographing method provided in the embodiment of the present application, when there are no other photographing operations within a preset time period in the future direction of the moment when the photographing operation occurs, by determining the image to be compared corresponding to the photographing operation and the image with the highest score among all preview images within the preset time period as the manually captured image corresponding to the photographing operation, the manually captured image finally outputted can be the most exciting image in a period of time before and after the moment when the photographing operation occurs, thereby improving the excitement of the manually captured image finally outputted.

[0022] In an optional implementation of the first aspect, the determining, based on the score of the image to be compared and the score of each preview image frame within a preset time period after the photographing operation occurs, the manually captured image corresponding to the photographing operation from the image to be compared and all preview images within the preset time period further includes:

[0023] If another photographing operation is detected again within the preset time period, the image with the highest score among the image to be compared and all preview images in the first time period is determined as the manually captured image corresponding to the photographing operation;

[0024] The starting moment of the first time period is the moment when the photographing operation occurs, and the ending moment of the first time period is the moment when the other photographing operation is detected again.

[0025] In an optional implementation of the first aspect, the method further includes:

[0026] If another photographing operation is detected again within the preset time period, determining the image to be compared corresponding to the other photographing operation based on the score values ​​of all preview images within the second time period;

[0027] The starting moment of the second time period is the moment when the other photographing operation is detected again, and the sum of the duration of the second time period and the duration of the first time period is the second duration.

[0028] According to the photographing method provided in an embodiment of the present application, when there are other photographing operations within a preset time length in the future direction of the moment when the photographing operation occurs, the image to be compared corresponding to the photographing operation and the image with the highest score among all preview images in the first time period are determined as the manually captured image corresponding to the photographing operation, and the images to be compared corresponding to the other photographing operations are determined based on the score values ​​of all preview images in the second time period. Since the starting time of the first time period is the moment when the photographing operation occurs, the end time of the first time period is the moment when the other photographing operation is detected again, the starting time of the second time period is the moment when the other photographing operation is detected again, and the sum of the duration of the second time period and the duration of the first time period is the second duration, it is possible to avoid duplication of the frame selection range between two adjacent photographing operations, so that multiple consecutive manual snapshots will not capture completely repeated images, thereby improving the user's photographing experience.

[0029] In an optional implementation of the first aspect, after determining the image to be compared for the photographing operation from all candidate images currently cached in the first buffer and the observation buffer, and before determining the manually captured image corresponding to the photographing operation from the images to be compared and all preview images within the preset time period, the method further includes:

[0030] A target thumbnail is generated based on the image to be compared corresponding to the photographing operation, and the target thumbnail is displayed in a thumbnail display area of ​​a photographing preview interface.

[0031] The photographing method provided by the embodiment of the present application can avoid affecting the user's manual capture speed, thereby improving the user's photographing experience.

[0032] In an optional implementation of the first aspect, after determining the manually captured image corresponding to the photographing operation from the image to be compared and all preview images within the preset time period, the method further includes:

[0033] An actual thumbnail is generated based on the manually captured image corresponding to the photographing operation, and the target thumbnail is replaced with the actual thumbnail.

[0034] According to the photographing method provided in the embodiment of the present application, by replacing the target thumbnail generated at the moment the photographing operation occurs with the actual thumbnail corresponding to the manually captured image, the thumbnail display area can correctly display the thumbnail corresponding to the final manually captured image, thereby not affecting the user's manual capturing speed, and can correctly display the thumbnail corresponding to the final manually captured image.

[0035] In a second aspect, an embodiment of the present application provides an electronic device comprising: one or more processors; one or more memories; the one or more memories storing one or more computer executable programs, the one or more computer executable programs comprising instructions, which, when executed by the one or more processors, enable the electronic device to perform each step of the photographing method as described in any implementation of the first aspect above.

[0036] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer-executable program. When the computer-executable program is called by a computer, the computer executes each step of the photographing method described in any implementation of the first aspect above.

[0037] In a fourth aspect, an embodiment of the present application provides a computer executable program product. When the computer executable program product is run on an electronic device, the electronic device executes each step of the photographing method described in any implementation of the first aspect above.

[0038] In a fifth aspect, embodiments of the present application provide a chip system, comprising a processor coupled to a memory, the processor executing a computer-executable program stored in the memory to implement the steps of the photographing method described in any of the implementations of the first aspect. The chip system can be a single chip or a chip module composed of multiple chips.

[0039] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a schematic diagram of a process of photographing a jumping action according to an embodiment of the present application;

[0041] FIG2 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0042] FIG3 is a schematic diagram of a software architecture of an electronic device according to an embodiment of the present application;

[0043] 4A to 4C are schematic diagrams illustrating a principle for determining a score value of a preview image in different types of photographing scenarios according to an embodiment of the present application;

[0044] 5A to 5E are schematic diagrams of some GUIs that may be involved in implementing a photographing method provided in an embodiment of the present application;

[0045] 6A to 6G are schematic diagrams of the photographing logic of a camera application in a snapshot mode according to an embodiment of the present application;

[0046] FIG7 is a schematic flow chart of a photographing method provided in an embodiment of the application. DETAILED DESCRIPTION

[0047] It should be noted that the terms used in the specific implementation methods of the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two, "at least one" and "one or more" mean one, two or more than two.

[0048] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Therefore, the definition of "first" and "second" features may explicitly or implicitly include one or more of the features.

[0049] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0050] In daily life, people often take photos to record some wonderful scenes (such as the scene of jumping to the highest point or the scene of looking back and smiling, etc.). Whether the wonderful scene can be captured well depends mainly on the following factors:

[0051] (1) Photographing moment: the choice of photographing moment is related to whether one can capture the moment when a wonderful scene occurs;

[0052] (2) The shutter speed of a photo is related to whether the wonderful scene can be frozen and whether the subject can be photographed clearly;

[0053] (3) The focus of the photo is related to whether the wonderful scene can be focused.

[0054] Regarding the timing of a photo, the following situation often occurs during the photography process: When a user wants to capture a wonderful scene, they typically press the shutter button immediately upon seeing it. However, there is often a certain delay between the time they see the wonderful scene and the time they actually press the shutter button. This delay is known as human reaction time and is typically between 100 and 200 milliseconds. As a result, the scene they actually capture may not be the intended wonderful scene, but rather another scene that appears later than the intended one. Furthermore, some cameras may have shutter lag, causing the scene they ultimately capture to be even later than the intended one. Therefore, if users try taking photos multiple times but still fail to capture the wonderful scene, they often try to predict the moment before the wonderful scene occurs, hoping to maximize the chance that the shutter button is pressed at the exact moment the wonderful scene occurs, thereby capturing the wonderful scene. However, since user predictions are not always accurate, the scene they actually capture may appear earlier than the intended one.

[0055] For example, please refer to Figure 1, which is a schematic diagram of the process of photographing a jumping action provided in an embodiment of the present application. The multiple frames of images shown in Figure 1 (i.e., images P1-1 to P1-7) can correspond to the complete process of a jumping action. When capturing a jumping action, as the jumping action proceeds (i.e., over time), the multiple frames of images will appear in sequence in the camera's photo preview interface. For jumping actions, the wonderful scene people usually want to capture is the scene when the jumper jumps to the highest point and / or the scene when the jumper smiles the best (e.g., the scene shown in image P1-4 in Figure 1). However, as mentioned above, assuming that the user sees the wonderful scene they want to capture at time T0 and immediately presses the shutter, due to the lag in the hand-to-eye reaction, the time when the user actually presses the shutter may be time T1. Taking into account the shutter delay of the camera, the scene that is finally captured may be the scene shown in image P1-6. Assuming that the user presses the shutter at time T2 before the jumping scene occurs based on their own prejudgment when shooting a jumping action next time, the scene that is actually captured may be the scene shown in image P1-3.

[0056] A common solution to the above problem is the zero shutter delay solution. By adjusting the physical shutter delay to zero, the zero shutter delay solution can ensure that the scene captured by the camera is likely the scene at the moment the user presses the shutter button. This means that the zero shutter delay solution can reduce or eliminate the delay in capturing photos caused by shutter delay, but it cannot reduce or eliminate the delay in capturing photos caused by delayed human reaction. In other words, the zero shutter delay solution still requires the user to press the shutter button in advance to capture the wonderful scene they see, which places high demands on the user's photography skills. Furthermore, many scenes in reality are unpredictable, occur suddenly, and are fleeting, such as the scene of looking back and smiling. In such scenes, the user cannot make any predictions, that is, they cannot predict when the wonderful scene will occur, so high demands are placed on the user's reaction speed and attention. Therefore, whether the zero shutter delay solution can capture wonderful scenes depends mainly on whether the user's hands and eyes can be synchronized, and it still cannot effectively solve the problem of photo delay from a technical perspective.

[0057] To mitigate the delay in capturing photos caused by human reaction lag, a common photography solution is to cache all preview images within a certain time range before and after the user presses the shutter button, and then select a few of the more interesting preview images from the cache for the user to choose from. However, because this photography solution requires caching all preview images within a certain time range before and after the user presses the shutter button, and due to memory limitations, this certain time range is usually limited to a small range (for example, a total of 0.2 seconds before and after the user presses the shutter button), the range of selectable images is narrow, which reduces the match between the actual captured image and the user's expected image.

[0058] In view of this, an embodiment of the present application provides a photographing method. In a snapshot mode, a score value for each preview image frame is determined. Based on the score value for each preview image frame, a corresponding candidate image is determined from each group of preview images that sequentially fall within an observation time window over time. After each candidate image for a group of preview images is determined, the currently determined candidate image is cached in one of one or more preset first buffer areas. When all first buffer areas are occupied, the currently determined candidate image is fixed in the preset observation buffer area. After the candidate images for the next group of preview images are determined, the candidate image with the earliest cache time in all first buffer areas is replaced with the currently determined candidate image. When a photographing operation is detected, an image to be compared is determined from the candidate images currently cached in the first buffer area and the observation buffer area. Based on the score value of the image to be compared and the score value of each preview image frame within a preset time period after the photographing operation occurs, a manually captured image corresponding to the photographing operation is determined from the image to be compared and all preview images within the preset time period. This method can not only improve the matching degree between the manually captured image and the user's expected image, but also save memory space.

[0059] The photographing method provided in the embodiments of the present application can be applied to electronic devices, that is, the execution subject of the photographing method provided in the embodiments of the present application can be an electronic device. Exemplary electronic devices include cameras, mobile phones, tablet computers, wearable devices, vehicle-mounted devices (such as driving recorders), augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other electronic devices.

[0060] For example, please refer to FIG2 , which is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

[0061] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, and a bone conduction sensor 180M, etc.

[0062] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0063] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0064] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0065] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface.

[0066] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0067] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0068] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0069] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0070] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0071] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0072] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0073] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0074] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0075] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0076] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0077] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0078] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0079] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0080] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0081] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0082] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0083] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0084] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0085] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0086] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0087] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0088] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0089] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0090] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0091] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0092] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0093] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0094] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0095] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0096] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0097] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0098] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0099] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0100] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0101] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0102] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0103] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0104] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0105] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0106] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0107] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0108] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0109] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0110] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0111] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0112] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0113] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0114] It should be understood that the structure shown in FIG2 does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0115] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. This embodiment of the present application uses the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100. Please refer to FIG3 , which is a block diagram of the software structure of an electronic device according to an embodiment of the present application.

[0116] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0117] The application layer can include a series of application packages.

[0118] As shown in FIG3 , the application package may include applications such as camera, gallery, map, navigation, WLAN, Bluetooth, video, and short message.

[0119] The camera application may include a score determination unit and a cache control unit.

[0120] The score determination unit can be used to determine the score value of each preview image frame in the camera application's photo preview interface based on a preset score determination strategy. It should be noted that the specific process of determining the score value of the preview image based on the score determination strategy will be described in subsequent embodiments and will not be described in detail here.

[0121] The cache control unit is used to determine whether to cache the preview image or output the cached preview image as a captured image based on the score value of each frame of the preview image in the capture mode of the camera application.

[0122] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0123] As shown in FIG3 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0124] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0125] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0126] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0127] The phone manager is used to provide communication functions for electronic devices, such as call status management (including answering, hanging up, etc.).

[0128] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0129] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0130] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0131] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0132] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0133] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0134] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0135] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0136] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0137] A 2D graphics engine is a drawing engine for 2D drawings.

[0138] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0139] The following describes the software and hardware workflow of the electronic device 100 in conjunction with a photo-taking scenario.

[0140] When the touch sensor 180K of the electronic device 100 (as shown in FIG2 ) receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, a timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is a control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer, and captures a still image or video through the camera 193 (as shown in FIG2 ).

[0141] The photographing method provided in the embodiments of the present application can be applied in the snapshot mode of a camera application. The snapshot mode can be one of the photographing modes of a camera application. In addition to the snapshot mode, the photographing modes of a camera application can also include a normal mode and an artificial intelligence (AI) mode. In addition to the photographing mode, a camera application can also include a video recording mode or other modes. The embodiments of the present application do not specifically limit the working modes of the camera application.

[0142] In some embodiments, the snapshot mode may include a manual snapshot function.

[0143] The manual capture function can be used to assist the user in selecting the most exciting preview image that meets preset requirements from the preview images that appear between the first moment and the second moment when a user's photo-taking operation is detected, as the image manually captured by the user.

[0144] The photo-taking operation may refer to the action of a user pressing a photo-taking button. For example, the photo-taking button may include a physical button, such as a shutter button on a camera, or a combination of a volume button and a power button on a mobile phone. The photo-taking button may also include a virtual button, such as a photo-taking control in a mobile phone's photo preview interface.

[0145] The moment when the user presses the photo button may be the moment when the photo is taken.

[0146] The first moment may be earlier than the moment when the photographing operation occurs, and the second moment may be later than the moment when the photographing operation occurs.

[0147] In specific applications, the first duration between the first moment and the moment the photo-taking operation occurs can be set based on experience, and the second duration between the second moment and the moment the photo-taking operation occurs can also be set based on experience. For example, considering that non-static actions (such as jumping actions) typically last about 1 second (s), and taking into account factors such as user reaction lag and / or shutter delay, the sum of the first and second durations can be set to 1.5s. Furthermore, since users typically take photos with a purpose, they generally do not wait several seconds after seeing a scene they want to capture before pressing the shutter. Therefore, the second duration can be shorter than the first duration. For example, the first duration can be 1s and the second duration can be 0.5s. That is, the manual capture function can help users select the most exciting preview image that meets preset requirements from preview images that appear between 1s in the historical direction and 0.5s in the future direction at the moment the photo-taking operation occurs, as the image manually captured by the user.

[0148] The preview image that meets the preset requirements may refer to a preview image whose score is greater than a preset score threshold. The most exciting preview image that meets the preset requirements may refer to a preview image with the highest score among the preview images that meet the preset requirements.

[0149] The score of a preview image can be used to indicate the image's quality. A higher score indicates a higher quality and closer to the definition of a quality image. A lower score indicates a lower quality and further from the definition of a quality image.

[0150] The preset score threshold can be set based on actual conditions and is not particularly limited herein. For example, if the score of the preview image is calculated based on a 100-point scale, the preset score threshold may be, for example, 90 points; if the score of the preview image is calculated based on a 10-point scale, the preset score threshold may be, for example, 9 points.

[0151] In the embodiment of the present application, the score value of the preview image can be determined by the electronic device based on a preset score determination strategy. For example, the preset score determination strategy may include:

[0152] Determining a weight value of the preview image in a semantic dimension and a weight value of the preview image in a shutter delay dimension;

[0153] The score value of the preview image is comprehensively determined based on the weight value of the preview image in the semantic dimension and the weight value of the preview image in the shutter delay dimension.

[0154] The semantic dimension may be used to represent a dimension related to the content of the preview image.

[0155] Exemplarily, the semantic dimension may include multiple different sub-dimensions, such as expression dimension, action dimension, clarity dimension, composition dimension, and subject category dimension.

[0156] In the expression dimension, the clearer and better the expression of the subject (e.g., a person) in the preview image (e.g., the presence of a smile, laughter, etc.), the greater the weight of the preview image in the expression dimension; otherwise, the smaller the weight.

[0157] In the action dimension, the more similar the subject's action in the preview image is to the preset action, the greater the weight of the preview image in the action dimension; conversely, the weight is smaller. For example, the preset actions may include exciting moves from various sports, such as a badminton smash, a basketball shot, a jump to the highest point, or a Frisbee throw. In specific applications, for example, skeleton point detection can be performed on the subject in the preview image to determine the similarity between the subject's action and the preset action.

[0158] In the clarity dimension, the higher the clarity of the preview image, the greater the weight value of the preview image in the clarity dimension; otherwise, the smaller the weight value.

[0159] In the composition dimension, the better the composition of the preview image, the greater the weight value of the preview image in the composition dimension; otherwise, the smaller the weight value.

[0160] In the subject category dimension, the weight of a preview image containing a preset subject may be greater than the weight of a preview image not containing the preset subject. For example, the preset subject may include a person and / or a pet (e.g., a cat or dog).

[0161] The shutter delay dimension may be used to represent a dimension related to the moment when the preview image appears in the photo preview interface.

[0162] Under the shutter delay dimension, the closer the preview image appears in the photo preview interface to the preset reaction time for the photo operation, the greater the preview image's weight under the shutter delay dimension. The preset reaction time for the photo operation can be a third time period before the photo operation occurs. The third time period is typically a human reaction time, for example, 100ms to 200ms.

[0163] In an optional implementation, the method for comprehensively determining the score value of the preview image based on the weight value of the preview image in the semantic dimension and the weight value of the preview image in the shutter delay dimension can be, for example: the sum of the weight value of the preview image in the semantic dimension and the weight value of the preview image in the shutter delay dimension is determined as the score value of the preview image.

[0164] The following describes the principles for determining the scores of preview images in different types of photo shooting scenarios. For example, please refer to Figures 4A to 4C, which are schematic diagrams of the principles for determining the scores of preview images in different types of photo shooting scenarios provided by an embodiment of the present application. For ease of understanding, the present embodiment of the present application categorizes photo shooting scenarios into continuous action scenarios, momentary action scenarios, and undefined action scenarios.

[0165] Among them, the continuous action scene can be used to represent a photo-taking scene in which the preview image that meets the preset requirements in the semantic dimension includes multiple frames of preview images that are continuous in the time domain.

[0166] For example, the preview image meeting the preset requirements in the semantic dimension may mean that the weight value of the preview image in the semantic dimension is greater than a preset weight threshold. Optionally, the weight value of the preview image in the semantic dimension may be the sum of the weight values ​​of the preview image in each sub-dimension of the semantic dimension. It should be noted that the method for determining the preset weight threshold can be similar to the method for determining the preset score threshold, and the method for determining the preset weight threshold is not further described here.

[0167] For example, the shooting scene shown on the left side of Figure 4A is a typical continuous action scene. In the shooting scene, since the series of actions from the moment the athlete jumps with the ball (i.e., time t1) to the moment of shooting (i.e., time t3) can generally be considered as exciting actions, therefore, assuming that three continuous shooting actions in the temporal domain occur between time t1 and time t2 as shown in Figure 4A, and the occurrence times of the three continuous shooting actions in the temporal domain are time t1, time t2, and time t3 respectively, then, for the shooting scene, the preview images that meet the preset requirements in the semantic dimension determined based on the preset score determination strategy can include the preview image at time t1, the preview image at time t2, and the preview image at time t3.

[0168] For example, assuming that the weight values ​​of the preview image at time t1, the preview image at time t2, and the preview image at time t3 in the semantic dimension correspond to points C1, C2, and C3 in the time-weight (TOQ) coordinate system shown on the right side of Figure 4A, respectively, and the weight values ​​of the preview image at time t1, the preview image at time t2, and the preview image at time t3 in the shutter delay dimension correspond to points H1, H2, and H3 in the TOQ coordinate system, respectively, then the score value of the preview image at time t1 can be the sum of the weight value corresponding to point C1 and the weight value corresponding to point H1, the score value of the preview image at time t2 can be the sum of the weight value corresponding to point C2 and the weight value corresponding to point H2, and the score value of the preview image at time t3 can be the sum of the weight value corresponding to point C3 and the weight value corresponding to point H3.

[0169] The origin O in the TOQ coordinate system is the moment the photo capture operation occurs. The interval between time t0 and the origin O is typically equivalent to human reaction time, for example, 100 to 200 ms. It is understood that the preview image at time t0 typically has the highest weight under the shutter delay dimension; the further away from time t0 the preview image appears in the photo preview interface, the smaller its weight under the shutter delay dimension.

[0170] The instantaneous action scene can be used to represent a photographing scene in which the preview image that meets the preset requirements in the semantic dimension includes one or more frames of preview images that are discrete (ie, discontinuous) in the time domain.

[0171] For example, the Frisbee throwing scene shown on the left side of Figure 4B is a typical instantaneous action scene. In a Frisbee throwing scene, since only the action at the moment of charging (i.e., time t4) and the action at the moment of throwing the Frisbee (i.e., time t5) are generally considered to be exciting actions, for this Frisbee throwing scene, the preview images that meet the preset requirements in the semantic dimension determined based on the score determination strategy include the preview image at time t4 and the preview image at time t5, and time t4 and time t5 are generally not continuous.

[0172] For example, assuming that the weight values ​​of the preview image at time t4 and the preview image at time t5 in the semantic dimension correspond to points C4 and C5 in the TOQ coordinate system shown on the right side of Figure 4B, respectively, and the weight values ​​of the preview image at time t4 and the preview image at time t5 in the shutter delay dimension correspond to points H4 and H5 in the TOQ coordinate system, respectively, then the score value of the preview image at time t4 can be the sum of the weight value corresponding to point C4 and the weight value corresponding to point H4, and the score value of the preview image at time t5 can be the sum of the weight value corresponding to point C5 and the weight value corresponding to point H5.

[0173] The undefined category scene can be used to represent other photography scenes except the continuous action scene and the instantaneous action scene.

[0174] As shown in Figure 4C, the scoring value determination strategy for preview images in undefined category scenarios can be similar to the scoring value determination strategy for preview images in continuous action scenarios or instantaneous action scenarios. For example, both can be the sum of the weight value of the preview image in the semantic dimension and the weight value in the shutter delay dimension.

[0175] It should be noted that since the process of determining the weight value of the preview image in each dimension is not the focus of protection of this application, the embodiments of this application will not describe it in detail.

[0176] In other embodiments, the snapshot mode may further include an automatic snapshot function.

[0177] Optionally, the automatic capture function can be used to automatically output a preview image that meets preset requirements when a preview image that meets preset requirements is identified, so as to help the user automatically capture more exciting images that the user may not have noticed.

[0178] Optionally, for the same or similar photo scenes, the automatic capture function can be used to output only one preview image that meets preset requirements to reduce the duplication of automatically captured photos. This not only saves memory space of the electronic device, but also eliminates the need for users to subsequently screen multiple similar photos, thereby improving the user's photo-taking experience.

[0179] In an optional implementation, the automatic snapshot function can be an optional function of the camera application, that is, the electronic device can turn the automatic snapshot function on or off based on the user's usage needs. It is understandable that when the automatic snapshot function is turned off, the electronic device does not execute the photo-taking process corresponding to the automatic snapshot function after entering the snapshot mode, and only executes the photo-taking process corresponding to the manual snapshot function; when the automatic snapshot function is turned on, the electronic device executes both the photo-taking process corresponding to the automatic snapshot function and the photo-taking process corresponding to the manual snapshot function after entering the snapshot mode.

[0180] The following describes some graphical user interfaces (GUIs) that may be involved in implementing the photographing method provided in an embodiment of the present application, using a mobile phone as an example. Figures 5A to 5E are schematic diagrams of some GUIs that may be involved in implementing a photographing method provided in an embodiment of the present application.

[0181] For example, when a user needs to take a photo, the user can input a click operation on the camera application icon 511 in the mobile phone desktop 51 as shown in Figure 5A. When the mobile phone detects the click operation on the camera application icon 511, it can display the photo preview interface 52 as shown in Figure 5B in response to the click operation on the camera application icon 511.

[0182] For example, as shown in FIG5B , in some embodiments, the photo preview interface 52 may include a camera settings control 521. When a user wishes to enable or disable the automatic snapshot function, the user may input a click operation on the camera settings control 521 in the photo preview interface 52. When the mobile phone detects the click operation on the camera settings control 521, it may display the camera settings interface 53 shown in FIG5C in response to the click operation on the camera settings control 521.

[0183] For example, as shown in FIG5C , the camera settings interface 53 may include a first switch control 531 for controlling the automatic snapshot function on or off. A user may turn the automatic snapshot function on or off by operating the first switch control 531. For example, the first switch control 531 may include an on state and an off state. When the first switch control 531 is in the off state, the automatic snapshot function is off. In this case, if the user wishes to enable the automatic snapshot function, they may input a click operation on the first switch control 531 in the camera settings interface 53. In response to the click operation on the first switch control 531, the mobile phone may turn the first switch control 531 on. When the first switch control 531 is in the on state, the automatic snapshot function is on. In this case, if the user wishes to disable the automatic snapshot function, they may input a click operation on the first switch control 531 in the camera settings interface 53. In response to the click operation on the first switch control 531, the mobile phone may turn the first switch control 531 off.

[0184] For example, as shown in FIG5D or FIG5E , in other embodiments, the photo preview interface 52 may further include a snapshot switch control 522 for controlling the on or off of the snapshot mode. The user may turn the snapshot mode on or off by operating the snapshot switch control 522. The snapshot switch control 522 may, for example, include an on state and an off state. When the snapshot switch control 522 is in the off state as shown in FIG5D , it indicates that the camera application is not currently in the snapshot mode. In this case, if the user needs to turn on the snapshot mode, the user may input a click operation on the snapshot switch control 522 in the photo preview interface 52 as shown in FIG5D ; the mobile phone may respond to the user's click operation on the snapshot switch control 522 in the photo preview interface 52 as shown in FIG5D by setting the snapshot switch control 522 to the on state as shown in FIG5E . When the snapshot switch control 522 is in the on state, it indicates that the camera application is currently in the snapshot mode. In this case, the mobile phone can execute the photo-taking method provided in the embodiment of the present application; if the user needs to turn off the snapshot mode, the user can input a click operation on the snapshot switch control 522 in the photo preview interface 52 as shown in Figure 5E; the mobile phone can respond to the user's click operation on the snapshot switch control 522 in the photo preview interface 52 as shown in Figure 5E, and set the snapshot switch control 522 to the off state as shown in Figure 5D, thereby turning off the snapshot mode.

[0185] It should be noted that when both the snapshot mode and the automatic snapshot function are turned on, the camera app can implement both the automatic and manual snapshot functions. When the snapshot mode is turned on and the automatic snapshot function is turned off, the camera app can only implement the manual snapshot function.

[0186] Optionally, as shown in FIG5D or FIG5E , the photo preview interface 52 may further include a photo button 523. The photo button 523 may be used as a control button for manual photo taking (e.g., manual snapshot), for the user to input a photo operation. For example, the user may input a photo operation by clicking the photo button 523. Of course, in specific applications, the user may also input a photo operation in other ways. For example, the user may input a photo operation by pressing the volume button and the power button at the same time. For another example, the user may input a photo operation by operating a wireless control device connected to the mobile phone. The embodiment of the present application does not specifically limit the input method of the photo operation. When the mobile phone detects a photo operation, it may take a photo in response to the photo operation.

[0187] In some embodiments, to intuitively indicate the camera application's current capture mode, the capture button 523 may include multiple different display states, each corresponding to a different capture mode of the camera application. For example, the capture button 523 may include at least a first display state, as shown in FIG. 5D , and a second display state, as shown in FIG. 5E . The second display state may correspond to the camera application's snapshot mode. That is, when the camera application's snapshot mode is on, the phone may place the capture button 523 in the second display state to indicate to the user that the camera application is currently in snapshot mode. Thus, when the user sees or clicks the capture button 523, they can intuitively know that the camera application is currently in snapshot mode. When the camera application's snapshot mode is off, the phone may place the capture button 523 in the first display state or another display state to indicate to the user that the camera application is not currently in snapshot mode, or to indicate that the camera application is currently in a capture mode other than snapshot mode.

[0188] In other embodiments, to intuitively indicate the on / off status of the automatic snapshot function in the snapshot mode, as shown in FIG5E , when the automatic snapshot function in the snapshot mode is on, the photo preview interface 52 may further include a description 525 indicating that the automatic snapshot function is on. The description 525 may be, for example, a textual description such as "Wonderful" snapshot is on; or the description 525 may be in other forms (e.g., an identifier). The embodiment of the present application does not specifically limit the form of the description 525.

[0189] In some other embodiments, when other photo modes of the camera application (such as AI mode) will affect the snapshot mode (for example, will affect the photo quality of the snapshot mode), the mobile phone can automatically turn off the other photo modes while turning on the snapshot mode to avoid the other photo modes of the camera application affecting the photo quality of the snapshot mode.

[0190] For example, as shown in FIG5D or FIG5E , the photo preview interface 52 may further include an AI switch control 524 for controlling the AI ​​mode to be turned on or off. The AI ​​switch control 524 may, for example, include an on state and an off state. When the AI ​​switch control 524 is in the on state as shown in FIG5D , it indicates that the camera is currently in AI mode. When the snapshot mode of the camera application is turned on, the mobile phone may automatically set the AI ​​switch control 524 to the off state as shown in FIG5E . Of course, in specific applications, the user can also manually turn on or off the AI ​​mode of the camera application by operating the AI ​​switch control 524.

[0191] The following describes the camera application's photo-taking logic in snapshot mode. Please refer to Figures 6A to 6G, which are schematic diagrams of the camera application's photo-taking logic in snapshot mode according to an embodiment of the present application.

[0192] Each rectangular box containing a number from left to right in the first row of FIG6A can be considered as a preview image that appears sequentially over time in the photo preview interface 52 shown in FIG5E . The number in each rectangular box can be used to represent the frame number of the preview image corresponding to that rectangular box, and the frame number of the preview image can be used to indicate the order in which the preview image appears in the photo preview interface 52. For example, the frame number of the preview image that appears earlier in the photo preview interface 52 can be lower than the frame number of the preview image that appears later in the photo preview interface 52. Preview images with negative frame numbers can be used to represent preview images that appeared in the photo preview interface 52 before the photo operation occurred at time Ts, while preview images with positive frame numbers can be used to represent preview images that appeared in the photo preview interface 52 after the photo operation occurred at time Ts.

[0193] Since the manual snapshot function requires selecting the most exciting preview image that meets preset requirements from the preview images that appeared within a first time period before the photo operation time Ts (e.g., within 1 second in the historical direction of the photo operation time Ts) and within a second time period after the photo operation time Ts (e.g., within 0.5 seconds in the future direction of the photo operation time Ts), as the image manually captured by the user, in actual applications, the mobile phone cannot predict when the user's photo operation will occur. Therefore, in theory, it is necessary to cache all preview images that appear in the photo preview interface 52 so that when the user's photo operation is detected, the most exciting preview image that meets the preset requirements can be selected from the cached preview images. However, caching all preview images that appear in the photo preview interface 52 will occupy a large amount of memory space.

[0194] Based on this, in order to save memory space, the embodiment of the present application does not cache all preview images appearing in the photo preview interface 52 when implementing the manual snapshot function, but selectively caches a portion of the preview images that meet the preset requirements. This not only expands the selectable range of images actually captured, but also saves memory space.

[0195] Specifically, an observation time window may be predefined as a basic observation unit for preview image caching, and the observation time window may be a time window of fixed duration starting from any moment.

[0196] In addition, an image buffer corresponding to the manual snapshot function may be predefined in the memory space of the mobile phone. The image buffer corresponding to the manual snapshot function may include an observation buffer and at least one first buffer.

[0197] The observation buffer can be used to cache each set of preview images that fall into the observation time window over time. For example, the observation buffer can cache images in an iterative manner (i.e., an in-and-out caching method), that is, as time passes, the preview images that are first cached in the observation buffer are released first.

[0198] The first buffer area can be used to cache images that meet preset requirements from each set of preview images that fall within the observation time window. It is understood that since the number of first buffer areas is limited, and the preview images that fall within the observation time window are constantly updated, the first buffer areas can employ a replacement method for image caching. That is, if all first buffer areas are occupied, a preview image in one of the first buffer areas can be replaced with the most recently determined preview image that meets the preset requirements. The replaced preview image can be the preview image with the oldest cache time among all first buffer areas.

[0199] Exemplarily, the duration of the observation time window and the number of image cache areas corresponding to the manual snapshot function can be set in accordance with the following principles: both meeting the requirements of the manual snapshot function for the frame selection range and minimizing the memory space occupied by the cached images. The requirements of the manual snapshot function for the frame selection range can be, for example, that the preview images cached in the image cache area corresponding to the manual snapshot function must at least cover all preview images within the first duration in the historical direction of the moment when the photo operation occurred. It should be noted that the "cover" here can be understood as "taking into account" and cannot be understood as "caching".

[0200] For example, assuming that the first duration is 1s, and the duration of the observation time window determined based on the above principle is 1 / 3s, then the number of image cache areas corresponding to the manual snapshot function can be 1 / (1 / 3) = 3, which can include one observation cache area and two first cache areas. In this way, the preview images cached in the image cache area corresponding to the manual snapshot function can cover all preview images within 1s before the photo operation occurs, and the memory space occupied by the image cache area is also the smallest.

[0201] Exemplarily, the storage capacity of the first buffer area can be determined based on the size of each preview image frame, the image caching mode of the camera application, and the image output format of the camera application. Assuming that the size of each preview image frame is a1 megabyte (MB), the image caching mode of the camera application is a n-frame raw image synthesis mode, and the image output format of the camera application is a binning format, then the storage capacity of the first buffer area can be n*a1*10 / 8 MB; assuming that the image output format of the camera application is a 12-bit intra-sense dual convert gain (IDCG) format, then the storage capacity of the first buffer area can be n*a1*12 / 8 MB; assuming that the image output format of the camera application is a 14-bit IDCG format, then the storage capacity of the first buffer area can be n*a1*14 / 8 MB.

[0202] Exemplarily, the storage capacity of the observation buffer can be determined based on the size of each preview image frame, the duration of the observation time window, the frame rate of the camera, the image caching method of the camera application, and the image output format of the camera application. For example, in conjunction with the above example, assuming that the duration of the observation time window is Δt, the frame rate of the camera application is f, the image caching method of the camera application is n-frame raw image synthesis, and the image output format of the camera application is binning, then the storage capacity of the observation buffer can be (Δt / f)*a1*10 / 8 megabytes; assuming that the image output format of the camera application is 12-bit IDCG format, then the storage capacity of the observation buffer can be (Δt / f)*a1*12 / 8 megabytes; assuming that the image output format of the camera application is 14-bit IDCG format, then the storage capacity of the observation buffer can be (Δt / f)*a1*14 / 8 megabytes.

[0203] Based on this, when implementing the manual snapshot function, the phone can determine the score of each preview image frame that appears in the photo preview interface over time based on a preset score determination strategy. Based on the score of each preview image frame, the phone can also determine candidate images for each group of preview images from each group of preview images that fall within the observation time window over time. It should be noted that each group of preview images does not have the same preview image as another group of preview images.

[0204] For example, as shown in FIG6B , assuming that at the first moment, preview image No. -30 begins to fall within the observation time window, then at the second moment after Δt seconds, preview images No. -30 to -21 will all fall within the observation time window, and at this time preview images No. -30 to -21 can be regarded as a group of preview images; at the next moment after the second moment, preview image No. -20 will fall within the observation time window, and preview image No. -30 will be squeezed out of the observation time window. At this time, preview images No. -29 to -20 fall within the observation time window, but since there is a preview image that is the same as preview images No. -30 to -21 among preview images No. -29 to -20, preview images No. -29 to -20 cannot be regarded as a group of preview images; only at the third moment after Δt seconds from the second moment, when preview images No. -30 to -21 are all squeezed out of the observation time window and preview images No. -20 to -11 all fall within the observation time window, can preview images No. -20 to -11 be regarded as another group of preview images.

[0205] In some embodiments, the candidate images for each group of preview images may include the preview image with the highest score in the group of preview images. In other embodiments, when the image caching mode of the camera application is an n-frame raw image synthesis mode, the candidate images for each group of preview images may include n consecutive preview images including the preview image with the highest score in the group of preview images. That is, in addition to including the preview image with the highest score in the group of preview images, the candidate images for each group of preview images may also include n-1 other preview images adjacent to the preview image with the highest score.

[0206] Exemplarily, the other n-1 preview images adjacent to the preview image with the highest score may be the n-1 consecutive preview images that appear after the preview image with the highest score and are adjacent to the preview image with the highest score. For example, assuming that the preview image with the highest score in the group of preview images from -30 to -21 is preview image -22, then the candidate images in the group of preview images may include preview image -22, preview image -21, preview image -20, and preview image -19.

[0207] After the mobile phone determines each set of candidate images for a preview image group, it may cache the currently determined candidate image in one of the one or more first cache areas. Optionally, if all first cache areas are occupied, the mobile phone may first cache the currently determined candidate image in the observation cache area. When the mobile phone determines the candidate images for the next set of preview images, it may replace the candidate images with the oldest cache time in all first cache areas with the candidate images cached in the observation cache area, and then cache the candidate images for the next set of preview images in the observation cache area.

[0208] For example, in conjunction with FIG6A , assuming that the mobile phone caches candidate images of the preview image group No. -30 to No. -21 (i.e., preview images No. -22 to No. -19) in one of the first buffer areas of the memory space at the second moment, and caches candidate images of the preview image group No. -20 to No. -11 (i.e., preview images No. -13 to No. -10) in another first buffer area of ​​the memory space at the third moment, then, after the mobile phone determines the candidate images of the preview image group No. -10 to No. -1 (i.e., preview images No. -7 to No. -4) at the fourth moment (i.e., the moment after Δt seconds from the third moment), it can First, candidate images for the preview image group of -10 to -1 (i.e., preview images -7 to -4) are fixedly cached in the observation cache. When the phone determines candidate images for the preview image group of 0 to 9 at the fifth moment (i.e., the moment after Δt seconds from the fourth moment), the phone replaces one of the candidate images cached in the first cache at the second moment (i.e., preview images -22 to -19) with the candidate image determined at the fourth moment (i.e., preview images -7 to -4), and the candidate image for the preview image group of 0 to 9 determined at the fifth moment is fixedly cached in the observation cache. This ensures that the image cache corresponding to the manual snapshot function can always cache preview images that meet the preset requirements within the first time period in the historical direction of the moment the photo was taken.

[0209] On this basis, when the user inputs a photo operation at a certain moment, the mobile phone can select a candidate image from all the candidate images currently cached in the first buffer area and the observation buffer area (i.e., at the moment the photo operation occurs) as the image to be compared. Exemplarily, the mobile phone can determine the candidate image with the highest score among all the candidate images currently cached in the first buffer area and the observation buffer area as the image to be compared. For example, in combination with the above example, assuming that the user inputs a photo operation at the fifth moment, the score of preview image No. -22 is 95, the score of preview image No. -13 is 92, and the score of preview image No. -7 is 80, then the mobile phone can determine preview images No. -13 to No. -10 as the images to be compared.

[0210] Thereafter, as time passes, the user can compare the scores of each preview image frame within the second time period after the photo capture operation with the score of the image to be compared, and determine the image with the highest score among the image to be compared and all preview images within the second time period as the captured image, and output the captured image. It should be noted that since the images to be compared may include n preview images, the score of the image to be compared may refer to the score of the preview image with the highest score among the n preview images included in the images to be compared.

[0211] For example, if the images to be compared include preview images No. -13 to No. -10, then when the user inputs the photo operation at time Ts, the mobile phone can compare the score value of preview image No. -13 with the score values ​​of preview images No. 0 to No. 9 respectively. If the score values ​​of preview images No. 0 to No. 9 are all smaller than the score value of preview image No. -13, then the mobile phone can synthesize preview images No. -13 to No. -10 and use them as manually captured images.

[0212] As for the automatic snapshot function, since it is independent of the user's camera operation, when the automatic snapshot function is enabled, the phone can compare the score of each preview image frame with a preset score threshold and cache preview images with scores greater than the preset score threshold in the image cache area corresponding to the automatic snapshot function. In a specific application, if the phone uses n-frame raw image synthesis for image caching, the phone can cache the other n-1 preview images adjacent to the preview image with a score greater than the preset grouping threshold.

[0213] The image cache area corresponding to the automatic capture function can use a replacement method to cache images. That is, when the image cache area corresponding to the automatic capture function is occupied, when there is a preview image with a higher score value in the same photo scene, the mobile phone can replace the preview image currently cached in the image cache area corresponding to the automatic capture function with the preview image with a higher score value.

[0214] In addition, when the mobile phone detects that the photo scene has changed, it can output the preview image currently cached in the image cache area corresponding to the automatic capture function at the moment the photo scene changes. In this way, for each different photo scene, the mobile phone can automatically capture the most exciting image in that photo scene (that is, the preview image with the highest score).

[0215] For example, situations where the photographing scene changes may include but is not limited to the following:

[0216] (1) The subject being photographed changes, such as a person entering or leaving the scene, or a pet entering or leaving the scene;

[0217] (2) The subject's movements change significantly, for example, the person in the photo changes from a stationary state to a running or jumping state;

[0218] (3) The background changes significantly, such as from indoor to outdoor.

[0219] Since in actual applications, the mobile phone cannot predict when the photo operation will occur, nor can it predict the interval between photo operations, the manual capture logic of the camera application under different conditions is described below in conjunction with Figures 6C to 6G.

[0220] As shown in Figure 6C, when no photo capture operation is detected, manual capture is an uncertain event. Therefore, over time, the phone needs to continuously update the candidate images in the first buffer and the observation buffer. In addition, when the automatic capture function is enabled, the phone also needs to continuously update the preview image in the image buffer corresponding to the automatic capture function.

[0221] As shown in Figure 6D, when the phone detects a photo operation, the uncertainty surrounding manual capture disappears. At this point, the phone can retain the highest-scoring candidate images in all first and observation buffers, and release all candidate images in the image buffer corresponding to the manual capture function, except for the highest-scoring candidate image. The score of a candidate image can refer to the score of the highest-scoring preview image among the n preview images included in the candidate image set. For example, assuming that at the time of a photo operation, among preview images -22 to -19 cached in one first buffer, preview image -22 has a score of 95, among preview images -13 to -10 cached in another first buffer, preview image -13 has a score of 92, and among preview images -7 to -4 fixed in the observation buffer, preview image -7 has a score of 85, the phone can retain preview images -13 to -10 and release preview images -13 to -10 and preview images -7 to -4.

[0222] As shown in Figure 6E, after the mobile phone detects a photo operation, as time passes, the preview images in the future direction of the photo operation will fall within the observation time window. At this time, the mobile phone can compare the scores of each preview image frame within the second time period in the future direction of the photo operation with the scores of the candidate images currently retained in the first buffer. If a preview image with a higher score than the retained candidate image exists within the second time period in the future direction of the photo operation, the mobile phone can replace the candidate image currently retained in the first buffer with the preview image with the higher score within the second time period in the future direction of the photo operation. For example, assuming that the score of preview image No. 5 in the future direction of the photo operation is higher than the score of preview image No. -22 currently retained in the first buffer, the mobile phone can replace preview images No. -22 to -19 currently retained in the first buffer with preview images No. 5 to 8.

[0223] As shown in Figure 6F, if the mobile phone detects other photo operations again after the photo operation occurs at the time Ts, and the time interval between the time Ts' of the other photo operations and the time Ts is less than the sum of the first time length and the second time length, then the frame selection range of the other photo operations in the historical direction cannot completely cover all preview images within the first time length. In this case, in order to avoid duplication of the frame selection range between two adjacent photo operations, the mobile phone can select the candidate image with the highest score from the candidate images cached in the first buffer area and the observation buffer area, which are between the time Ts of the photo operation and the time Ts' of the other photo operations, as the image to be compared corresponding to the other photo operations.

[0224] For example, assuming that the first buffer area and the observation buffer area have been cached when other photo operations occur, the candidate images between the time Ts when the photo operation occurs and the time Ts' when the other photo operations occur include preview images No. 12 to No. 15 and preview images No. 26 to No. 29, and the score value of preview image No. 12 is greater than the score value of preview image No. 26, then the mobile phone can determine preview images No. 12 to No. 15 as the images to be compared corresponding to other photo operations.

[0225] When time advances toward the future direction of the time when other photo operations occur, the mobile phone can compare the score value of each frame of the preview image within the second time length in the future direction of the time when other photo operations occur, with the score value of the image to be compared corresponding to the other photo operations, and take the image to be compared corresponding to the other photo operations and all preview images within the second time length in the future direction of the time when other photo operations occur, as the captured image corresponding to the other photo operations.

[0226] In an extreme case of the scenario shown in Figure 6F, as shown in Figure 6G, if the time interval between the above-mentioned other photographic operation occurrence time Ts' and the photographic operation occurrence time Ts is less than the second time length, then the mobile phone only compares the score values ​​of the preview image frames between the photographic operation occurrence time Ts and the other photographic operation occurrence time Ts' with the score values ​​of the images to be compared corresponding to the photographic operation, thereby determining the image with the highest score from the preview image frames between the photographic operation occurrence time Ts and the other photographic operation occurrence time Ts' and the images to be compared corresponding to the photographic operation as the manually captured image corresponding to the photographic operation occurrence time Ts.

[0227] It is understandable that, as shown in FIG5E , each time the user enters a photo operation in the photo preview interface 52, the mobile phone will generate a photo thumbnail and display the photo thumbnail in the thumbnail display area 526 of the photo preview interface 52. The mobile phone will not allow the user to enter a photo operation again until the photo thumbnail of the current photo operation is displayed in the thumbnail display area 526. Based on the photo logic of the camera application in the snapshot mode provided in the embodiment of the present application, after the user enters the photo operation, the mobile phone also needs to observe the preview image that appears within a second time period in the future direction of the time when the photo operation occurs. Therefore, in an optional implementation, after the user enters the photo operation, in order not to affect the speed of the user's continuous snapshots, the mobile phone can also generate a target thumbnail based on the image to be compared corresponding to the determined photo operation when the photo operation is detected, and display the target thumbnail in the thumbnail display area of ​​the photo preview interface.

[0228] Based on this, after the second time period after the moment when the photographing operation occurs, the mobile phone can generate an actual thumbnail based on the captured image corresponding to the photographing operation finally determined, and replace the target thumbnail with the actual thumbnail.

[0229] The following describes the specific process of the photographing method provided by the embodiment of the present application in conjunction with the above embodiment. The photographing method provided by the embodiment of the present application can be applied to static photographing scenes (such as night scenes, street scenes, or landscapes, etc.), and can also be applied to dynamic photographing scenes (such as sports scenes, water splashing scenes, parabolic scenes, human-pet interaction scenes, and fleeting scenes, etc.). The embodiment of the present application does not specifically limit the photographing scenes to which the photographing method is applied.

[0230] Please refer to Figure 7, which is a schematic flow chart of a photographing method provided in an embodiment of the present application. As shown in Figure 7, the photographing method may include S71 to S75, which are described in detail as follows:

[0231] S71 : In snapshot mode, determine a score value of each frame of preview image.

[0232] The method for determining the score value of each preview image frame may refer to the relevant description in the aforementioned embodiment, which will not be described in detail here.

[0233] S72 , based on the score value of each frame of preview image, determine candidate images for each group of preview images from each group of preview images that sequentially fall within the observation time window over time.

[0234] In an optional implementation, the electronic device may determine, in each group of preview images, preview images with scores greater than a preset score threshold as candidate images for each group of preview images.

[0235] For details about the preset score threshold, please refer to the relevant description in the above embodiments, which will not be described in detail here.

[0236] S73, each time a group of candidate images for preview images is determined, the currently determined candidate image is cached in one of the preset one or more first cache areas. When all the first cache areas are occupied, the currently determined candidate image is fixed in the preset observation cache area, and after the candidate images for the next group of preview images are determined, the candidate image with the earliest cache time in all the first cache areas is replaced with the currently determined candidate image.

[0237] It should be noted that, for the specific implementation process of S73, reference may be made to the relevant description in the embodiment corresponding to FIG6A , which will not be described in detail here.

[0238] S74 , when a photographing operation is detected, in response to the photographing operation, determining an image to be compared corresponding to the photographing operation from all candidate images currently cached in the first buffer area and the observation buffer area.

[0239] The candidate images currently cached in all the first buffer areas and the observation buffer areas refer to the candidate images that have been cached in all the first buffer areas and the observation buffer areas when the photographing operation occurs.

[0240] In an optional implementation, the electronic device may determine, as the image to be compared, the candidate image with the highest score among all the candidate images cached in the first buffer area and the observation buffer area at the time when the photographing operation occurs.

[0241] S75, based on the score value of the image to be compared and the score value of each frame of the preview image within the preset time period after the photo operation occurs, determine the manually captured image corresponding to the photo operation from the image to be compared and all preview images within the preset time period.

[0242] The preset duration may be the second duration in the aforementioned embodiment.

[0243] In one optional implementation, if the electronic device does not detect another photographing operation within a preset time period after the photographing operation occurs, the electronic device may determine the image with the highest score among the image to be compared and all preview images within the preset time period as the manually captured image corresponding to the photographing operation. It should be noted that the specific implementation process of this implementation can be referred to the relevant description of the embodiment corresponding to FIG6A , and will not be elaborated on here.

[0244] In another optional implementation, if the electronic device detects another photo operation again within a preset time period after the photo operation occurs, the electronic device can determine the image with the highest score among the image to be compared and all preview images in the first time period as the manually captured image corresponding to the photo operation. The starting time of the first time period can be the time when the photo operation occurs, and the ending time of the first time period can be the time when the other photo operation is detected again (i.e., the time when the other photo operation occurs). It should be noted that for the specific implementation process of this implementation, reference can be made to the relevant description in the embodiment corresponding to Figure 6G, which will not be repeated here.

[0245] In another optional implementation, if the electronic device detects another photo-taking operation again within a preset time period after the photo-taking operation occurs, the electronic device may further determine the images to be compared corresponding to the other photo-taking operations based on the scores of all preview images within a second time period. The start time of the second time period may be the moment when the electronic device detects another photo-taking operation again within the preset time period, and the sum of the duration of the second time period and the duration of the first time period may be the second time period. It should be noted that for the specific implementation process of this implementation, reference may also be made to the relevant description in the embodiment corresponding to FIG. 6G , which will not be described in detail here.

[0246] In other embodiments of the present application, after determining the image to be compared corresponding to the photo operation, the electronic device can generate a target thumbnail based on the image to be compared corresponding to the photo operation, and display the target thumbnail in the thumbnail display area of ​​the photo preview interface. This can avoid affecting the user's manual capture speed and improve the user's photo experience.

[0247] In some other embodiments of the present application, after determining the manually captured image corresponding to the photo operation, the electronic device may also generate an actual thumbnail based on the manually captured image corresponding to the photo operation, and replace the target thumbnail with the actual thumbnail. By replacing the target thumbnail generated at the time the photo operation occurred with the actual thumbnail corresponding to the manually captured image, the thumbnail display area can correctly display the thumbnail corresponding to the final manually captured image, thereby not affecting the user's manual capture speed, and correctly displaying the thumbnail corresponding to the final manually captured image.

[0248] Based on the same technical concept, an embodiment of the present application also provides an electronic device, which may include: a display; one or more processors; one or more memories; the one or more memories store one or more computer programs, and the one or more computer programs include instructions. When the instructions are executed by the above-mentioned one or more processors, the electronic device executes one or more steps in any one of the processes shown in Figure 7.

[0249] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program. When the computer-executable program is called by the computer, the computer executes one or more steps in any one of the processes shown in Figure 7.

[0250] Based on the same technical concept, an embodiment of the present application further provides a chip system, including a processor coupled to a memory, wherein the processor executes a computer-executable program stored in the memory to implement one or more steps in any of the processes shown in FIG7 . The chip system can be a single chip or a chip module composed of multiple chips.

[0251] Based on the same technical concept, an embodiment of the present application also provides a computer executable program product. When the computer executable program product runs on an electronic device, it enables the electronic device to execute one or more steps in any one of the processes shown in Figure 7.

[0252] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described or recorded in detail in a particular embodiment, please refer to the relevant descriptions of other embodiments. It should be understood that the order of the sequence numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0253] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0254] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0255] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for taking photos, It is characterized in that include: In snapshot mode, determine the score value of each frame of preview image; Based on the score value of each frame of the preview image, from each group of preview images that sequentially fall into the observation time window over time, respectively determine the candidate images of each group of preview images; Each time a group of candidate images for preview images is determined, the currently determined candidate image is cached in one of the one or more preset first cache areas, and when all the first cache areas are occupied, the currently determined candidate image is fixed in the preset observation cache area, and after the candidate images for the next group of preview images are determined, the candidate image with the earliest cache time in all the first cache areas is replaced by the currently determined candidate image; When a photographing operation is detected, in response to the photographing operation, determining an image to be compared for the photographing operation from candidate images currently cached in all first buffers and the observation buffer; Based on the score value of the image to be compared and the score value of each frame of the preview image within a preset time after the photo operation occurs, the manually captured image corresponding to the photo operation is determined from the image to be compared and all preview images within the preset time.

2. The photographing method according to claim 1, It is characterized in that The step of determining candidate images of each group of preview images from each group of preview images that sequentially fall into the observation time window over time comprises: Preview images in each group of preview images whose score values ​​are greater than a preset score threshold are respectively determined as candidate images of each group of preview images.

3. The photographing method according to claim 1 or 2, It is characterized in that The step of determining the manually captured image corresponding to the photographing operation from the image to be compared and all the preview images within the preset time period based on the score value of the image to be compared and the score value of each frame of the preview image within the preset time period after the photographing operation occurs includes: If no other photographing operation is detected again within the preset time length, the image with the highest score among the image to be compared and all preview images within the preset time length is determined as the manually captured image corresponding to the photographing operation.

4. The photographing method according to claim 3, It is characterized in that The step of determining the manually captured image corresponding to the photographing operation from the image to be compared and all the preview images within the preset time period based on the score value of the image to be compared and the score value of each frame of the preview image within the preset time period after the photographing operation occurs, further includes: If another photographing operation is detected again within the preset time period, the image with the highest score among the image to be compared and all preview images in the first time period is determined as the manually captured image corresponding to the photographing operation; The starting time of the first time period is the time when the photographing operation occurs, and the ending time of the first time period is the time when the other photographing operation is detected again.

5. The photographing method according to claim 4, It is characterized in that Also includes: If another photographing operation is detected again within the preset time period, determining the image to be compared corresponding to the other photographing operation based on the score values ​​of all preview images within the second time period; The starting moment of the second time period is the moment when the other photographing operation is detected again, and the sum of the duration of the second time period and the duration of the first time period is the second duration.

6. The photographing method according to any one of claims 1 to 5, It is characterized in that After determining the image to be compared for the photographing operation from all candidate images currently cached in the first buffer area and the observation buffer area, and before determining the manually captured image corresponding to the photographing operation from the image to be compared and all preview images within the preset time period, the method further includes: A target thumbnail is generated based on the image to be compared corresponding to the photographing operation, and the target thumbnail is displayed in a thumbnail display area of ​​a photographing preview interface.

7. The photographing method according to claim 6, It is characterized in that After determining the manually captured image corresponding to the photographing operation from the image to be compared and all preview images within the preset time period, the method further includes: An actual thumbnail is generated based on the manually captured image corresponding to the photographing operation, and the target thumbnail is replaced with the actual thumbnail.

8. An electronic device, It is characterized in that include: one or more processors; one or more memories; The one or more memories store one or more computer executable programs, and the one or more computer executable programs include instructions. When the instructions are executed by the one or more processors, the electronic device executes each step in the photographing method as described in any one of claims 1-7.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer-executable program, and when the computer-executable program is called by a computer, the computer executes each step in the photographing method according to any one of claims 1 to 7.

10. A chip system, It is characterized in that It includes a processor, the processor is coupled to a memory, the memory is used to store computer program instructions, when the processor executes the computer program instructions, the chip system implements each step in the photographing method as described in any one of claims 1-7.