Photo processing method and electronic equipment

Abstract

The invention provides a photo processing method and electronic equipment, relates to the technical field of image processing, and can meet the requirement of a user for reselecting wonderful frames. The method comprises the following steps: in response to a shooting instruction, the electronic equipment stores a first photo comprising a first wonderful frame, and caches N frames of first images; wherein the first wonderful frame is recognized by the electronic equipment through an AI snapshot function, and the N frames of first images are collected by the electronic equipment within a preset time period before and after a user inputs a shooting instruction; in response to a first operation of a user on the first photo, the electronic equipment displays N first thumbnails in one-to-one correspondence with the N frames of first images; and in response to a selection operation of the user on a first target thumbnail in the N first thumbnails, the electronic equipment stores a second photo including a second wonderful frame, and the second wonderful frame is a first image corresponding to the first target thumbnail selected by the user.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a photo processing method and electronic device. Background Art

[0002] With the continuous advancement of terminal technology, users have increasingly higher demands for the convenience of using electronic devices such as mobile phones and tablets. For example, users are increasingly demanding the convenience of taking photos with electronic devices. Currently, when taking photos, users may want to capture some wonderful moments, such as a moment when an athlete is running.

[0003] During the shooting process, due to problems such as user reaction delay and transmission delay of electronic devices, it is often difficult for users to capture truly wonderful moments, which greatly reduces the user's shooting experience. Summary of the Invention

[0004] The present application provides a photo processing method and electronic device, which can meet the user's needs of reselecting wonderful frames.

[0005] This application adopts the following technical solutions:

[0006] In a first aspect, a photo processing method is provided, the method comprising: in response to a shooting instruction, an electronic device saves a first photo and caches N frames of first images; wherein the first photo includes a first wonderful frame, and the first wonderful frame is recognized by the electronic device using an AI capture function; the N frames of first images are collected by the electronic device within a preset time period before and after the user inputs the shooting instruction; N ≥ 3, and N is a positive integer.

[0007] In response to a user's first operation on a first photo, the electronic device displays N first thumbnails, each corresponding to the N first image frames. In response to the user's selection of a first target thumbnail from the N first thumbnails, the electronic device saves a second photo, wherein the second photo includes a second highlight frame, which is the first image corresponding to the first target thumbnail selected by the user.

[0008] Based on the first aspect, the electronic device also caches N frames of first images when saving the first photo; on this basis, when the user wants to reselect the wonderful frame, the electronic device can respond to the user's first operation on the first photo and display N first thumbnails corresponding to the N frames of first images one by one; then, in response to the user's selection operation of the first target thumbnail among the N first thumbnails, the electronic device saves the second photo including the second wonderful frame, and the second wonderful frame is the first image corresponding to the first target thumbnail selected by the user, thereby realizing the user's need to reselect the wonderful frame.

[0009] In a possible implementation of the first aspect, the first photo and the second photo are single photos; or, the first photo and the second photo are covers of a live photo with a duration of x seconds, and the number of frames of the live photo is i≥3, where i is a positive integer and x is a positive number.

[0010] In a possible implementation of the first aspect, in response to a user's selection operation of a first target thumbnail among N first thumbnails, the electronic device saves a second photo, including: in response to the user's selection operation of a first target thumbnail among N first thumbnails, the electronic device uses the first image corresponding to the first target thumbnail as a reference frame, fuses the M frames of first images with the reference frame, generates a second photo, and saves the second photo; M=N-1, and M is a positive integer.

[0011] The M frames of first images are images in the N frames of first images excluding the reference frame.

[0012] In this embodiment, the electronic device uses the first image corresponding to the first target thumbnail as a reference frame, merges the M-frame first image with the reference frame, and generates a second photo, thereby solving the problem of poor image quality of the second photo generated after the user reselects the wonderful frame.

[0013] In a possible implementation of the first aspect, the electronic device fuses M frames of first images with a reference frame to generate a second photo, including: the electronic device weightedly processes the pixel value of each pixel in the reference frame and the pixel value of each pixel in each frame of the M frames of first images to generate the second photo.

[0014] In a possible implementation of the first aspect, the electronic device performs weighted processing on the pixel value of each pixel in the reference frame and the pixel value of each pixel in each first image in the M frames to generate a second photo, including: the electronic device performs weighted processing on the pixel value of each pixel in the reference frame and the pixel value of each pixel in each first image in the M frames to generate a first target image; the electronic device calls a preset image algorithm to process the first target image to generate a second photo; wherein, the image quality of the second photo is higher than the image quality of the first photo.

[0015] In this embodiment, the electronic device first performs weighted processing on the pixel value of each pixel in the reference frame and the pixel value of each pixel in each first image of the M frames to generate a first target image, and then calls a preset image algorithm to process the first target image to generate a second photo, thereby further improving the image quality of the second photo.

[0016] In a possible implementation of the first aspect, the electronic device caches a second image of a first resolution; the second photo corresponds to the second resolution; the method also includes: the electronic device receives a second operation of the user on the second photo, the second operation is used to set the resolution of the second photo to a third resolution; the third resolution is smaller than the first resolution and greater than the second resolution.

[0017] In response to the second operation, the electronic device merges the second image with the first resolution and the second photo with the second resolution, and saves a third photo with a third resolution.

[0018] In this embodiment, since the electronic device also caches the second image of the first resolution, when the electronic device receives the user's request to set the resolution of the second photo to the third resolution, the electronic device can merge the second image of the first resolution and the second photo of the second resolution and save the third photo of the third resolution, thereby meeting the user's demand for adjusting the resolution of the photo.

[0019] In a possible implementation of the first aspect, the electronic device fuses the second image of the first resolution and the second photo of the second resolution to save the third photo of the third resolution, including: the electronic device downsampling the second image of the first resolution to generate the second image of the third resolution; the electronic device upsampling the second photo of the second resolution to generate the second photo of the third resolution; and the electronic device fuses the second image of the third resolution with the second photo of the third resolution to generate the third photo of the third resolution.

[0020] In a second aspect, a photo processing method is provided for use in an electronic device. The method includes: in response to a capture instruction, the electronic device saves a first photo and caches N frames of the first image and a second image of a first resolution; wherein the first photo includes a first highlight frame, which is recognized by the electronic device using an AI capture function; the N frames of the first image are captured by the electronic device within a preset time period before and after the user inputs the capture instruction; N ≥ 3, and N is a positive integer.

[0021] In response to the user's first operation on the first photo, the electronic device displays N first thumbnails corresponding one-to-one to the N frames of first images; in response to the user's selection operation on the first target thumbnail among the N first thumbnails, and the second operation on the first target thumbnail, the electronic device saves the second photo.

[0022] Among them, the first target thumbnail corresponds to the second resolution, and the second operation is used to set the resolution of the first target thumbnail to the third resolution, which is smaller than the first resolution and larger than the second resolution; the second photo corresponds to the third resolution, and the second photo includes a second wonderful frame, which is the first image corresponding to the first target thumbnail selected by the user.

[0023] In this embodiment, because the electronic device caches N frames of the first image and a second image at the first resolution when saving the first photo, the electronic device can then display N first thumbnails in response to a user's first operation on the first photo. Furthermore, in response to the user's selection of a first target thumbnail from the N first thumbnails and a second operation on the first target thumbnail, the electronic device can obtain a second photo at a third resolution, thereby simultaneously satisfying the user's needs for selecting a highlight frame and adjusting the resolution.

[0024] In a possible implementation of the second aspect, in response to a user's selection operation on a first target thumbnail among N first thumbnails, and a second operation on the first target thumbnail, the electronic device saves a second photo, including: in response to the user's selection operation on the first target thumbnail among N first thumbnails, and the second operation on the first target thumbnail, the electronic device uses the first image corresponding to the first target thumbnail as a reference frame, fuses the M frames of first images with the reference frame, and generates a first target image; the first target image corresponds to the second resolution.

[0025] The electronic device fuses the first target image of the second resolution with the second image of the first resolution to save the second photo of the third resolution; wherein the M frames of first image are images of the N frames of first image excluding the reference frame.

[0026] In a possible implementation of the second aspect, the electronic device fuses the first target image of the second resolution with the second image of the first resolution and saves the second photo of the third resolution, including: the electronic device downsampling the second image of the first resolution to generate the second image of the third resolution; the electronic device upsampling the first target image of the second resolution to generate the first target image of the third resolution; and the electronic device fuses the first target image of the third resolution with the second image of the third resolution and saves the second photo of the third resolution.

[0027] In a third aspect, a photo processing method is provided for use in an electronic device. The method includes: in response to a capture instruction, the electronic device saves a first photo and caches a second image at a first resolution; the first photo corresponds to a second resolution; and in response to a second user operation on the first photo, the electronic device saves the second photo at a third resolution. The second operation is used to set the resolution of the first photo to the third resolution; the third resolution is greater than the second resolution and less than the first resolution.

[0028] In a possible implementation of the third aspect, in response to the user's second operation on the first photo, the electronic device saves the second photo at a third resolution, including: in response to the user's second operation on the first photo, the electronic device merges the second image of the first resolution with the second photo of the second resolution, and saves the third photo at the third resolution.

[0029] In a possible implementation of the third aspect, the electronic device fuses the second image of the first resolution with the second photo of the second resolution and saves the third photo of the third resolution, including: the electronic device downsampling the second image of the first resolution to generate the second image of the third resolution; the electronic device upsampling the first photo of the second resolution to generate the first photo of the third resolution; and the electronic device fuses the second image of the third resolution with the first photo of the third resolution and saves the second photo of the third resolution.

[0030] In a fourth aspect, an electronic device is provided, wherein the electronic device has the function of implementing any one of the above-mentioned aspects 1 to 3. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0031] In a fifth aspect, an electronic device is provided, comprising: a memory and one or more processors; computer program code is stored in the memory, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device executes any one of the methods described in the first to third aspects above.

[0032] In a sixth aspect, a chip system is provided, which includes: at least one processor and an interface, the interface being used to receive instructions and transmit them to at least one processor; at least one processor executes the instructions so that the electronic device executes any one of the methods described in any one of the first to third aspects above.

[0033] In a seventh aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the computer-readable storage medium is run on a computer, the computer can execute any one of the methods in the first to third aspects above.

[0034] In an eighth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in any one of the first to third aspects above.

[0035] Among them, the technical effects brought about by any implementation method in the above-mentioned fourth to eighth aspects can refer to the technical effects brought about by different implementation methods in the first to third aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a framing interface provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of an interface for enabling the AI ​​snapshot function provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of another interface for enabling the AI ​​snapshot function provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of automatically identifying wonderful frames provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of a software framework of an electronic device provided in an embodiment of the present application;

[0042] Figure 7 A schematic diagram of an interface of a photo processing method provided in an embodiment of the present application;

[0043] Figure 8 A schematic diagram of a process for photo processing provided in an embodiment of the present application;

[0044] Figure 9 A flowchart of another photo processing method provided in an embodiment of the present application;

[0045] Figure 10 A schematic diagram of an interface of another photo processing method provided in an embodiment of the present application;

[0046] Figure 11 A flowchart of another photo processing method provided in an embodiment of the present application;

[0047] Figure 12 A flowchart of another photo processing method provided in an embodiment of the present application;

[0048] Figure 13 A schematic structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following describes the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the present application, unless otherwise specified, " / " represents "or." For example, A / B can represent A or B. "And / or" in this document is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist at the same time, or B exists alone.

[0050] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" are only used for descriptive purposes and do not limit the quantity and execution order, nor can they be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. And words such as "first" and "second" do not necessarily limit them to be different. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "at least one" means one or more, and "a plurality of" means two or more.

[0051] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0052] In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, the technical terms involved in the present application are explained below.

[0053] A highlight moment refers to a moment within a period of time when the subject's state and / or action is at its best. State and action apply to still-life and non-still-life scenes, respectively. In still-life scenes, a highlight moment occurs when the subject is relatively still in physical space, and the camera image of the still-life object is clear, jitter-free, and noise-free. In non-still-life scenes, a highlight moment occurs when the subject's action is at its best in physical space, and the camera image of the non-still-life object is clear, jitter-free, and noise-free.

[0054] For example, when photographing historical sites, a highlight moment is when the building's image is complete, clear, and smooth, without jitter or noise. When photographing a hurdler, a highlight moment is when the athlete jumps and crosses the hurdle, and the image is clear, without jitter or noise. When photographing a close-up portrait, a highlight moment is when the person opens their eyes and smiles, and the image is clear, without jitter or noise.

[0055] Among them, the image frames captured by the camera at the wonderful moment are called wonderful frames, and the photos output after the wonderful frames are processed are called wonderful photos.

[0056] It should be noted that a wonderful frame may also be referred to as a wonderful frame image or a wonderful image frame, and that the terms "wonderful frame" and "wonderful frame image" are merely examples of descriptions, and the present application does not impose any limitations on the description of a wonderful frame. For example, the wonderful frame may also be referred to as an excellent frame image, a preset frame image, a key frame, or the like.

[0057] Live Photos: They can record and dynamically present image and sound information within a preset time period (such as 1.5 seconds) before and after the user takes a photo. They have the media formats of static images and dynamic video clips. Live Photos can contain multiple image frames. After the Live Photo is taken, the electronic device will automatically select an image frame according to preset rules (such as selecting the image frame with the highest clarity among multiple image frames) and set the image frame after image processing as the default cover of the Live Photo. Of course, users can also select other image frames in the Live Photo as the cover photo of the Live Photo, but its clarity is lower than the default cover. Accordingly, users can also perform the same editing operations on Live Photos as ordinary photos.

[0058] An image frame is the smallest unit that makes up a video (or Live Photo, single photo, etc.). A video can consist of several image frames, and a Live Photo can also consist of several image frames. The difference between a video and a Live Photo is that a video is longer than a Live Photo. A single photo consists of one image frame.

[0059] Pixel value: It is the value assigned by the computer when the original image is digitized. It represents the average brightness information of a small square in the original image, or the average reflection (transmission) density information of the small square.

[0060] The following describes the scenario of this application:

[0061] The solution of the embodiment of the present application can be applied to the scene of capturing wonderful frames. For example, manual capture mode and automatic capture mode can be used to capture wonderful frames. Among them, the manual capture mode means that during the photo or video recording process, the user can manually take a photo at the same time to obtain the desired picture (i.e., manually capture a wonderful frame).

[0062] The automatic capture mode means that the electronic device can capture wonderful frames through the artificial intelligence (AI) capture function. For example, the AI ​​capture function is an intelligent capture function provided by the camera application, which is different from the manual capture method and can capture without manual operation of the user. The AI ​​capture function can capture wonderful frames, for example, the wonderful frame of a successful shot, the wonderful frame of jumping to the highest point, the wonderful frame when the look back action is completed, and the wonderful frame of specific photographed objects such as rainbows / sunsets. The embodiments of the present application do not make specific limitations on this.

[0063] The above automatic capture method can be applied to scenarios where a camera application is used in electronic devices such as mobile phones and tablets (the following embodiments take mobile phones as an example), and in such scenarios, wonderful frames can be captured.

[0064] For example, when capturing wonderful frames using automatic capture mode, you can display them on your phone. Figure 1 During the process of the viewfinder interface 101 shown (ie before taking a photo, or before recording a video), wonderful frames are determined from the preview image and stored.

[0065] Another example is that when the automatic capture mode is used to capture the wonderful frame, the mobile phone can display Figure 1 During the process of the viewfinder interface 102 shown (ie, during video recording), wonderful frames are determined from the preview image and stored.

[0066] The following describes the process of automatic capture by mobile phone:

[0067] In some embodiments, the application settings interface of the mobile phone's camera application includes a control a. The mobile phone can receive a user operation a on the control a (such as a click, long press, slide, etc., and the click operation is used as an example below). In response to the user operation a on the control a, the mobile phone can enable the AI ​​snapshot function.

[0068] For example, the phone may display Figure 2 The viewfinder interface 201 before shooting is shown. The interface 201 includes a setting button 202. In response to the user clicking the setting button 202, the mobile phone can display Figure 2 The interface 203 shown is the application setting interface of the camera application, and the interface 203 includes the "Smart Photo" option. In response to the user clicking the "Smart Photo" option in the interface 203, the mobile phone can display Figure 2Interface 204 is shown. Interface 204 is the setting interface of "Smart Photo" (also belongs to the application setting interface of the camera application). Interface 204 includes "Auto Snap" setting item 205. "Auto Snap" setting item 205 includes switch 206. The state of switch 206 in interface 204 is off, which is control a. In response to the user clicking on switch 206 in interface 204, the mobile phone can turn on the AI ​​snapshot function. For example, in response to the user clicking on switch 206 in interface 204, the mobile phone can display Figure 2 The interface 207 shown also includes a switch 206 , and the state of the switch 206 in the interface 207 is on.

[0069] In this embodiment, after the AI ​​capture function is turned on, in response to a preset event that triggers the display of the framing interface, the mobile phone can start AI capture to capture wonderful frames.

[0070] For example, after turning on the AI ​​snapshot function, the electronic device can display Figure 2 In response to the user's return operation on interface 207, the mobile phone may display Figure 2 In response to the user's return operation on interface 203, the mobile phone may display Figure 2 The viewfinder interface 201 shown in FIG. At this point, the mobile phone starts AI snapshot. That is, the preset event is the event in which the user performs a return operation on the interface 203.

[0071] For example, after the AI ​​snapshot function is turned on, in response to the user clicking the application icon of the camera application on the mobile phone desktop, the mobile phone can display the viewfinder interface before taking a photo (such as Figure 1 The viewfinder interface 101 shown in the figure) At this time, the mobile phone starts AI snapshot. That is, the preset event is the event that the user clicks the application icon of the camera application on the mobile phone desktop.

[0072] In other embodiments, the viewfinder interface of the mobile phone's camera application includes a preset control b for the user to enable the AI ​​snapshot function. The mobile phone can receive a user operation b on control b (such as a click, long press, or slide, etc., with a click operation being used as an example below). In response to the user operation b on control b, the mobile phone can enable the AI ​​snapshot function and simultaneously start AI snapshot to capture a wonderful frame.

[0073] Control b is Figure 3 For example, the icon 302 in the viewfinder interface 301 shown in FIG. Figure 3 The framing interface 301 before taking a photo is shown. The interface 301 includes an icon 302. The mobile phone can receive a user's click operation on the icon 302. In response to the user's click operation on the icon 302, the mobile phone can turn on the AI ​​snapshot function and start AI snapshot at the same time.

[0074] refer to Figure 2 and Figure 3 , the user can operate the controls of the AI ​​snapshot function (such as control a and control b) through the same path, and then turn off the AI ​​snapshot function through the above controls. The process can refer to the above embodiment and will not be repeated here. By implementing the above method, the user can turn on / off the AI ​​snapshot function at any time according to their own needs. In this way, when the user needs to automatically capture wonderful photos of wonderful moments, the mobile phone can provide the above service; when the user does not need to automatically capture, the mobile phone can turn off the AI ​​snapshot function, which saves computing costs, reduces power consumption, and improves the user experience.

[0075] In some embodiments, while the mobile phone is displaying the viewfinder interface, the mobile phone's camera application will continuously request new images from the camera. Accordingly, the camera will continuously capture images and report them to the mobile phone's camera application, so that the mobile phone's camera application displays the images in the viewfinder interface. It should be understood that the images displayed by the mobile phone's camera application in the viewfinder interface can be referred to as preview images.

[0076] For example, during the preview process, the mobile phone can identify each captured preview image in real time to determine whether the frame is a highlight frame. After identifying a highlight frame, the mobile phone can automatically perform image processing on the highlight frame and save the highlight photo in the mobile phone's photo album application (or gallery application). It should be understood that in this embodiment, the highlight photo output by the mobile phone is a single photo.

[0077] As another example, the mobile phone receives a user's shooting instruction during the preview process. In response to the user's shooting instruction, the mobile phone obtains multiple frames of preview images within a preset time period before and after the user inputs the shooting instruction. For example, if the preset time period is 1.5 seconds, after receiving the shooting instruction input by the user, the mobile phone obtains multiple frames of preview images within 1.5 seconds before and after the time node of "receiving the shooting instruction input by the user" is centered. Then, for the multiple frames of preview images, the mobile phone can identify each frame of the preview image and determine whether the multiple frames of preview images include a highlight frame. After identifying the highlight frame, the mobile phone generates a live photo based on the multiple frames of preview images, performs image processing on the highlight frame, sets the processed highlight frame as the default cover of the live photo, and then saves the live photo in the album application (or gallery application) of the mobile phone.

[0078] The Live Photos stored on the phone include a JPG-formatted photo and a video. The JPG-formatted photo serves as the default cover for the Live Photo. In practice, the image quality (such as clarity and resolution) of the JPG-formatted photo is higher than that of the video. This is because when generating a Live Photo, the phone applies an image enhancement algorithm to the selected image frame, resulting in a JPG-formatted photo with higher image quality. Other image frames are simply processed and then edited into a video to reduce computing power and device power consumption.

[0079] In some embodiments, the automatic capture mode can enable the mobile phone to automatically identify wonderful moments and take wonderful photos without the user having to perform the shooting operation, which not only simplifies the user operation but also avoids missing wonderful moments. Furthermore, in order to improve the user experience and avoid outputting too many and redundant photos, the mobile phone can identify detailed shooting scenes and set an upper limit for the wonderful photos of each scene. When the number of wonderful photos of a scene reaches the upper limit of the scene, the mobile phone will no longer automatically capture the scene. Alternatively, the mobile phone can set an automatic capture upper limit, and when the number of captured photos reaches the upper limit, the mobile phone will no longer automatically capture.

[0080] The following describes the process of mobile phone recognition of wonderful frames:

[0081] As an example of the present application, a preset algorithm corresponding to the automatic capture of a mobile phone is used to identify wonderful frames. For example, the preset algorithm corresponding to the automatic capture is used to automatically take pictures when it recognizes wonderful moments such as looking back, jumping, running, and cats and dogs.

[0082] For example, Figure 4 As shown, a mobile phone captures an athlete performing a long jump. With automatic capture enabled, the phone's viewfinder displays the athlete's motion status captured at different moments during the long jump. For example, the phone may sequentially display the preview image of the first frame, the second frame, the third frame, and so on, to the kth frame. Simultaneously, using a pre-configured algorithm for automatic capture, the phone identifies each preview frame and identifies the highlight frames.

[0083] For example, in long jump, the flat state of the character after taking off can be called the best completion state of long jump, that is, Figure 4 After identifying the preview images from the first frame to the kth frame, the mobile phone can determine that the preview image from the third frame is a wonderful frame.

[0084] In this embodiment, the preset algorithm for automatic capture can define optimal states for different scenarios. For example, in a long jump, the flat-bed position of the character after taking off can be considered the optimal completion state for the long jump. Another example is that in a jump, the state of the character reaching the highest point after leaving the ground can be considered the optimal jump state, and so on. There are no restrictions.

[0085] As another example of this application, a preset model corresponding to automatic capture by a mobile phone is used to identify highlight frames. Exemplarily, the preset model corresponding to automatic capture is used to output an aesthetic score for each preview image; wherein the aesthetic score is used to indicate the highlight level of the preview image.

[0086] Taking the example of a mobile phone capturing the actions of a person, for example, a preset model corresponding to automatic capture is used to capture the person's actions, such as jumping, looking back, throwing objects, playing table tennis, playing badminton, running, splashing water, playing baseball, playing rugby, etc., and then determines the completion of the action in any preview image frame based on the optimal completion point of each action, and then outputs an aesthetic score for the preview image. Preview images with higher action completion levels have higher corresponding aesthetic scores. After identifying each preview image frame, the mobile phone can determine the preview image with the highest aesthetic score as a highlight frame.

[0087] Through the above-mentioned automatic capture method, the mobile phone can automatically identify wonderful frames and capture wonderful photos. However, if the wonderful photos captured by the mobile phone do not meet the user's needs, that is, the wonderful photos automatically captured by the mobile phone are not the wonderful photos the user wants, the user's shooting experience will be reduced. Based on this, an embodiment of the present application provides a photo processing method. In this method, after automatically capturing a wonderful photo, the mobile phone can reselect a wonderful frame and generate a wonderful photo based on the reselected wonderful frame.

[0088] For example, the method can be applied to smart phones, personal digital assistants (PDAs), tablet computers, laptop computers, video cameras, video recorders, cameras, smart watches, smart wristbands, cellular phones, car computers, and other electronic devices with camera applications. The embodiments of the present application do not impose any particular restrictions on the specific form of the electronic device.

[0089] refer to Figure 5 , is a hardware structure diagram of an electronic device provided in an embodiment of the present application. Figure 5As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a positioning module 181, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0090] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0091] The processor 110 may include one or more processing units, for example, the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0092] In some embodiments, the processor 110 can complete the related processing of automatic capture, as well as the related processing of reselecting wonderful frames and regenerating wonderful photos.

[0093] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0094] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0095] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include one or more filters, switches, power amplifiers, low noise amplifiers (LAN), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be placed in the processor 110. In some embodiments, at least some of the functions of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0096] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate one or more communication processing modules. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0097] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0098] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0099] The electronic device 100 can implement a shooting function through an ISP, camera 193, video codec, GPU, display 194, and application processor. The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature. In some embodiments, the ISP can be set in the camera 193.

[0100] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0101] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0102] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 110 by running the instructions stored in the internal memory 121. For example, the processor 110 can display different content on the display screen 194 in response to the operation for unfolding the display screen 194 by executing the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0103] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0104] The buttons 190 include a power button and a volume button. The buttons 190 can be mechanical buttons or touch buttons. The electronic device 100 can receive button inputs and generate key signal inputs related to the user settings and function controls of the electronic device 100. The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts or for touch vibration feedback. The indicator 192 can be an indicator light that can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0105] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture or a cloud architecture. In the embodiment of the present application, the layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0106] refer to Figure 6 , is a software structure diagram of an electronic device 100 provided in an embodiment of the present application. Figure 6As shown, a layered architecture can divide software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the layered architecture can be divided into four layers: from top to bottom, the application layer (referred to as the application layer), the application framework layer (referred to as the framework layer), the hardware abstraction layer (HAL) layer, and the kernel layer (also known as the driver layer).

[0107] The application layer may include a series of application packages, such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message applications.

[0108] In the embodiment of this application, Figure 6 As shown, the application layer may include a system camera application (also referred to as a camera application). The camera application may provide functions such as taking photos, recording videos, and AI snapshots.

[0109] The camera application can be used to Figure 1 The preview image reported by the bottom layer is displayed in the viewfinder interface 101 and the viewfinder interface 102 in the electronic device. The electronic device may include one or more cameras, each of which can be used to capture the preview image.

[0110] The framework layer provides the application programming interface (API) and programming framework for the application layer. The framework layer includes some predefined functions. Figure 6 As shown, the framework layer can provide camera services.

[0111] The camera service can be used to transmit requests from the application layer to the lower layer (such as the HAL layer), such as requests to end shooting, switch cameras, and adjust focus. The camera service can also be used to process the preview frame data reported by the lower layer (such as the HAL layer) and upload it to the upper layer application.

[0112] The HAL layer is an abstract structure between the kernel layer and the Android runtime. The HAL layer can be a package of hardware drivers, providing a unified interface for upper-layer applications to call.

[0113] like Figure 6As shown, the HAL layer includes the camera hardware abstraction layer (HAL). The camera HAL provides a standard HAL interface definition language (HAL interface definition language) to upper layers (such as the camera service) for call, maintaining normal communication with the upper layer. Furthermore, the camera HAL controls the kernel layer through a standard HAL interface, such as HAL 3.0, and obtains preview frame data reported by the kernel layer, which is then reported to the upper layer (such as the camera service).

[0114] The camera HAL is the core software framework for camera devices. It can include a variety of image processing algorithms, including face recognition, skin beautification, age recognition, fill-light, sharpening and high-definition processing, background blur, high dynamic range (HDR), and multi-frame fusion processing. These algorithms are not restricted.

[0115] The kernel layer includes a camera driver, which is used to start the camera device and transmit image data collected by the camera device to the upper layer (such as the HAL layer). The camera device may include one or more cameras, each of which includes a camera lens and an image sensor. The ISP can be provided separately from the camera device (such as a camera). In some embodiments, the ISP can be provided in the camera device (such as a camera).

[0116] The ISP and camera components are the primary devices for capturing video or images. Light signals reflected from the viewing environment are converted into electrical signals by the camera lens and irradiated on the image sensor. These signals are then processed by the ISP and transmitted as raw parameter streams by the camera driver to upper layers. Furthermore, the camera driver receives notifications from upper layers (such as those instructing to turn a camera on or off). Based on these notifications, it sends a function processing parameter stream to the camera component to turn the corresponding camera on or off.

[0117] The photo processing method provided in the embodiment of the present application can be applied to electronic devices having the above-mentioned hardware structure and software structure.

[0118] Specifically, the photo processing method provided in the embodiments of this application includes the following three embodiments: Embodiment 1: Selecting a Highlight Frame. Embodiment 2: Adjusting the Resolution. Embodiment 3: Selecting a Highlight Frame and Adjusting the Resolution. The following uses a mobile phone as an example electronic device to illustrate each of the above three embodiments.

[0119] Embodiment 1: Selecting a wonderful frame.

[0120] In some embodiments, the mobile phone displays a framing interface (such as the framing interface 101 and the framing interface 102 described above), which can be used to display a preview image captured by the mobile phone. Exemplarily, in response to a shooting instruction, the mobile phone saves the first photo and caches N frames of the first image. The first photo includes a first highlight frame, which is recognized by the mobile phone using the above-mentioned AI capture function; the N frames of the first image are captured by the mobile phone within a preset time period before and after the user inputs the shooting instruction; N ≥ 3, and N is a positive integer.

[0121] It should be noted that, for the specific implementation method of the mobile phone using the AI ​​capture function to automatically identify the first wonderful frame, please refer to the description in the above embodiment, which will not be repeated here.

[0122] For example, the first photo can be stored in an access area corresponding to the photo album application (or gallery application) of the mobile phone. For example, the N frames of the first image can be cached in the internal memory 121 of the mobile phone. When the internal memory 121 includes a read-only memory (ROM), the N frames of the first image can be cached in the memory space of the ROM.

[0123] Optionally, the N frames of first images cached by the mobile phone can be original RAW domain images, that is, after the mobile phone collects N frames of first images within a preset time period before and after the user inputs the shooting command, the N frames of first images are directly cached in the storage space of the mobile phone.

[0124] Alternatively, the N frames of first images cached by the mobile phone can be RGB domain images or YUV domain images, that is, after the mobile phone collects N frames of RAW domain images within a preset time period before and after the user inputs the shooting command, the N frames of RAW domain images are converted into RGB domain images or YUV domain images to obtain N frames of first images, and the N frames of first images are cached in the storage space of the mobile phone.

[0125] Optionally, the mobile phone may compress the N frames of first image and cache them into the storage space of the mobile phone, thereby preventing a large amount of image data from occupying the memory space of the mobile phone.

[0126] In some embodiments, after the mobile phone saves the first photo in the album application, the mobile phone can view the saved first photo in response to the user's operation of launching the album application; or edit the first photo, such as cropping, rotating, collecting, sending, etc., without limitation. For example, the mobile phone can also respond to the user's operation of reselecting a highlight frame for the first photo in the album application. For example, the mobile phone displays Figure 7 Interface 401 is shown, and interface 401 includes a first photo 402.

[0127] For example, the interface 401 further includes an edit button 403. In response to the user clicking the edit button 403, the mobile phone displays Figure 7 The interface 404 shown includes a "Select Highlight Frame" option 405. In response to the user clicking on the "Select Highlight Frame" option 405, the mobile phone displays Figure 7 The interface 406 shown includes N first thumbnails. The N first thumbnails correspond one to one with the N frames of first images cached by the mobile phone. For example, the N frames of first images cached by the mobile phone include: the first image of the first frame, the first image of the second frame, the first image of the third frame, ..., the first image of the Nth frame; the N first thumbnails displayed on the interface 406 include: thumbnails Figure 1 , abbreviation Figure 2 , abbreviation Figure 3 , ..., thumbnail N. Then, the first image of the first frame and the thumbnail Figure 1 Correspondingly, the first image of the second frame and the thumbnail Figure 2 Correspondingly, the first image of the third frame and the thumbnail Figure 3 Correspondingly, ... the first image of the Nth frame corresponds to thumbnail N.

[0128] It should be noted that the one-to-one correspondence between the N first thumbnails and the N first images cached in the mobile phone means that the first thumbnails have the same image content as the corresponding first images.

[0129] For example, Figure 7 In the interface 406 shown, in response to the user sliding operation on the first thumbnail in the interface 406 (such as sliding to the left or sliding to the right), the mobile phone can scroll and display each first thumbnail in the N first thumbnails. Figure 1 , abbreviation Figure 2 , abbreviation Figure 3 , ..., thumbnail N. Based on this, in response to the user's thumbnail N in the first thumbnail Figure 1 (or the first target thumbnail) selection operation, the phone displays Figure 7 The interface 407 shown in FIG. Figure 1 Then, in response to the user's confirmation operation, the mobile phone generates a second photo. The second photo includes a second wonderful frame, which is the thumbnail selected by the user. Figure 1 The corresponding first image.

[0130] Optionally, the first photo and the second photo are single photos. On this basis, the first photo is generated by the mobile phone based on the first wonderful frame, that is, the mobile phone uses the AI ​​capture function to capture the first wonderful frame and then generates the first photo. Correspondingly, the second photo is generated by the mobile phone based on the second wonderful frame, that is, the user selects the above thumbnail Figure 1 (i.e., the second wonderful frame selected by the user), the mobile phone generates a second photo.

[0131] Optionally, the first photo and the second photo are the covers of a live photo with a duration of x seconds, and the number of frames of the live photo is i≥3, i is a positive integer, and x is a positive number. On this basis, the first photo is the cover of the live photo generated by the mobile phone based on the first wonderful frame, that is, after the mobile phone uses the AI ​​capture function to capture the first wonderful frame, the mobile phone processes the first wonderful frame and sets it as the cover of the live photo. Correspondingly, the second photo is the cover of the live photo generated by the mobile phone based on the second wonderful frame, that is, the user selects the above thumbnail Figure 1 After the mobile phone processes the corresponding first image (ie, the second wonderful frame selected by the user), it sets it as the cover of the live photo.

[0132] In summary, using the solution of the embodiments of the present application, upon receiving a capture command input by a user, a mobile phone can capture N frames of first images within a preset time period before and after the user inputs the capture command, and cache these N frames of first images in the mobile phone. Later, when the user wishes to select a highlight frame, the mobile phone can display N first thumbnails corresponding to the N frames of first images for the user to select, and generate a second photo based on the highlight frame selected by the user. This allows the user to obtain a true image of a highlight moment, satisfying the user's need to select a highlight frame and improving the user experience.

[0133] In some embodiments, as Figure 8 As shown, when the mobile phone saves a first photo (such as a photo in JPG format), it also caches N frames of first images. Then, in response to the user's editing operation on the first photo (such as selecting a highlight frame), the mobile phone processes the N frames of first images through the ISP to generate N first thumbnails. Furthermore, in response to the user's selection operation of the first target thumbnail, the mobile phone uses the first image corresponding to the first target thumbnail as a reference frame, merges the M frames of first images with the reference frame, and generates a second photo. Among them, the M frames of first images are the images of the N frames of first images excluding the reference frame, M = N-1, and M is a positive integer.

[0134] For example, in combination with the above Figure 7 As shown, if the first target thumbnail selected by the user is a thumbnail Figure 1 , the phone will zoom in Figure 1 The corresponding first image (i.e., the first image of the first frame) is used as a reference image, and the M frames of first images (i.e., the first image of the second frame, the first image of the third frame, ..., the first image of the Nth frame) are fused with the reference frame (the first image of the first frame) to generate a second photo.

[0135] In some embodiments, as Figure 6As shown, after the camera application of the application layer receives a user editing operation (such as the operation of selecting a highlight frame), the camera application of the application layer sends the user's operation instruction (i.e., the instruction to select a highlight frame) to the camera HAL of the HAL layer. After receiving the operation instruction, the camera HAL of the HAL layer reads N frames of first images cached by the mobile phone and processes the N frames of first images to generate N first thumbnails. Then, the camera HAL of the HAL layer reports the N first thumbnails to the camera application of the application layer, so that the camera application displays the N first thumbnails.

[0136] Then, the camera application of the application layer receives the user's thumbnail of the first target (such as thumbnail Figure 1 ) after the selection operation, the corresponding message (that is, the user selects the thumbnail Figure 1 After receiving the corresponding message, the camera HAL will send the thumbnail selected by the user to the camera HAL. Figure 1 The corresponding first image is used as a reference frame, and then the M frames of first images are fused with the reference frame to generate a second image.

[0137] Optionally, the camera HAL of the HAL layer fuses the M frames of the first image with the reference frame to generate a first target image. Furthermore, the camera HAL invokes a preset image processing algorithm (such as the aforementioned multiple image processing algorithms) corresponding to the HAL layer to process the first target image and generate a second photo.

[0138] For example, the preset image processing algorithm may include noise reduction (NR) processing, which can remove noise points in the image, eliminate blur bands, enhance color saturation, and improve realism. Therefore, the image quality of the second photo generated after the noise reduction processing is higher than that of the first photo.

[0139] In some embodiments, the fusion of the M frames of first image and the reference frame involves the mobile phone obtaining a pixel value for each pixel in the reference frame and a pixel value for each pixel in each of the M frames of first image. The mobile phone then performs weighted processing on the pixel value for each pixel in the reference frame and the pixel value for each pixel in each of the M frames of first image.

[0140] For example, Figure 9 As shown, assuming that the reference frame is the first image of the first frame, and the M first images include the second first image, the third first image, ..., the Nth first image, the mobile phone weights the pixel value of each pixel in the first image of the first frame with the pixel value of each pixel in the second first image, the pixel value of each pixel in the third first image, ..., the pixel value of each pixel in the Nth first image to obtain the second image.

[0141] In summary, through the above method, in response to the user selecting a highlight frame, the mobile phone uses the user-selected highlight frame (such as the first image corresponding to the first target thumbnail described above) as a reference frame and fuses it with the other first images (i.e., the M frames of first images) to obtain a second photo, thereby resolving the issue of poor image quality of the highlight frame reselected by the user. Furthermore, because the second photo is processed by the mobile phone using a preset image processing algorithm, the image quality of the second photo can be made higher than that of the first photo. This not only satisfies the user's need to select a highlight frame, but also further improves the image quality, thereby enhancing the user experience.

[0142] Embodiment 2: Adjusting the resolution.

[0143] In some embodiments, when the mobile phone stores the first photo, it also caches a second image of the first resolution. The first photo corresponds to the second resolution, and the first resolution is greater than the second resolution.

[0144] For an example of a mobile phone saving the first photo, please refer to the description in the above embodiment, which will not be repeated here.

[0145] It should be noted that in related art, the arrangement of multiple pixels included in an image sensor is generally a Bayer array. In a Bayer array, a pixel unit includes one red pixel (R), two green pixels (G), and one blue pixel (B). A pixel comprises four sub-pixels arranged in a 2×2 matrix. This arrangement results in a larger number of pixels in an image sensor, resulting in a higher native resolution.

[0146] However, due to the different image output modes of image sensors, even when outputting images from these image sensors, the resolution of the output images may be lower than the native resolution of the image sensor. For example, if the native resolution of an image sensor is 50MP (megapixels), but the phone's camera app defaults to taking photos at 12MP (megapixels), the resolution of the photos taken by the phone will be 12MP.

[0147] In summary, it is understandable that the image sensor is capable of outputting high-resolution images, but because the default resolution of the phone's camera application is low, the resolution of photos taken with the phone is lower than the native resolution of the image sensor. Based on this, embodiments of the present application can also adjust the resolution of photos taken by the phone in response to user operations.

[0148] For example, the phone displays Figure 10The interface 501 shown includes a first photo 502. The interface 501 also includes an edit button 503. In response to the user clicking the edit button 503, the mobile phone displays Figure 10 The interface 504 shown includes an "adjust resolution" option 505. In response to the user clicking on the "adjust resolution" option 505, the mobile phone displays Figure 10 The interface 506 shown includes a "resolution input box" 507. The user can enter the resolution that he wants to adjust in the "resolution input box 507". For example, in response to the user entering the third resolution (such as 30MP) in the "resolution input box 507", the mobile phone displays Figure 10 The interface 508 shown includes a second photo 509 , and the resolution of the second photo 509 is 30 MP.

[0149] It should be noted that Figure 10 The resolution in which the user can enter the corresponding value is used as an example. Of course, the mobile phone can also display multiple resolution value options for the user to choose from, which is subject to the actual setting. The embodiment of the present application is not limited to this.

[0150] Optional, such as Figure 11 As shown, when a mobile phone saves a first photo (e.g., a photo in JPG format), it also caches a second image at a first resolution. Then, in response to a user editing operation on the first photo (e.g., adjusting the resolution), the mobile phone reads the first image at the first resolution, then fuses the second image at the first resolution with the first photo at the second resolution, and processes the fused image using a preset image algorithm to generate a second photo at a third resolution.

[0151] Alternatively, assuming the first resolution is 50MP, the second resolution is 12MP, and the user-adjusted resolution (i.e., the third resolution) is 30MP, the phone downsamples the second image at the first resolution to generate a second image at the third resolution, i.e., downsampling the second image to 30MP. The phone upsamples the first photo at the second resolution to generate a first photo at the third resolution, i.e., upsampling the first photo to 30MP.

[0152] Then, the mobile phone fuses the second image of the third resolution with the first photo of the third resolution to generate a second photo of the third resolution.

[0153] For example, the mobile phone downsamples the second image at the first resolution to generate a third-resolution second image, i.e., a 30MP second image. The mobile phone then filters the 30MP second image to obtain a high-frequency image and a low-frequency image of the second image. The high-frequency image refers to areas in the second image with large brightness (or grayscale) variations, while the low-frequency image refers to areas in the second image with small brightness (or grayscale) variations.

[0154] For example, a mobile phone can fuse a 30MP high-frequency image with a 30MP first photo to generate a 30MP second photo. This can make the generated second photo richer in details and higher in image quality.

[0155] To sum up, by adopting the solution of the embodiment of the present application, the mobile phone can provide the user with an interface for adjusting the resolution. In response to the user's operation of adjusting the resolution, the mobile phone can generate a second photo with a third resolution based on the current photo (i.e., the first photo) and the cached second image, which not only improves the resolution of the photo and makes the photo clearer, but also improves the user experience.

[0156] Embodiment 3: Select a wonderful frame and adjust the resolution.

[0157] In some embodiments, in response to a user's selection operation, after the mobile phone selects a highlight frame, it generates a second photo in the manner of Example 1, and the second photo corresponds to the second resolution. Furthermore, in response to a second user operation on the second photo (i.e., setting the resolution of the first photo to the third resolution), the mobile phone generates a third photo at the third resolution based on the second image at the first resolution and the second photo at the second resolution.

[0158] It is understood that the second image at the first resolution is cached by the mobile phone after receiving the shooting instruction. For an example of the second image at the first resolution, please refer to the description of the above embodiment 2, and will not be repeated here. In addition, for examples of the first resolution, the second resolution, and the third resolution, please refer to the description of the above embodiment 2, and will not be repeated here.

[0159] Exemplarily, in response to the user's second operation on the second photo, the mobile phone fuses the second image of the first resolution and the second photo of the second resolution to generate a third photo of the third resolution.

[0160] For example, the mobile phone downsamples the second image at the first resolution to generate a second image at the third resolution. The mobile phone upsamples the second photo at the second resolution to generate a second photo at the third resolution. Furthermore, the mobile phone fuses the second image at the third resolution with the second photo at the third resolution to generate a third photo at the third resolution.

[0161] For example, if the first resolution is 50MP, the second resolution is 12MP, and the third resolution is 30MP, the phone will downsample the 50MP second image to 30MP and upsample the 12MP second photo to 30MP. The phone will then fuse the 30MP second image with the 30MP second photo to generate a 30MP third photo.

[0162] The specific implementation process of the mobile phone fusing the 30MP second image with the 30MP second photo may refer to the description in the above-mentioned embodiment 2, which will not be repeated here.

[0163] In other embodiments, after saving the first photo, the mobile phone caches N frames of the first image and a second image at the first resolution. Based on this, in response to a first user action on the first photo, the mobile phone displays N first thumbnails. In response to a user selection of a first target thumbnail from the N first thumbnails, and a second action on the first target thumbnail, the mobile phone generates the second photo. In other words, in this embodiment, the user can simultaneously select a highlight frame and adjust the resolution.

[0164] The first target thumbnail corresponds to the second resolution, and the second photo corresponds to the third resolution.

[0165] For example, Figure 12 As shown, after a mobile phone receives a first photo (e.g., a JPG-formatted photo), it caches N frames of the first image and a second image at a first resolution. Then, in response to a user editing operation on the first photo (e.g., selecting a highlight frame), the mobile phone processes the N frames of the first image via the ISP to generate N first thumbnails. Furthermore, in response to a user selecting a first target thumbnail and a second operation on the first target thumbnail, the mobile phone uses the first image corresponding to the first target thumbnail as a reference frame and fuses the M frames of the first image with the reference frame to generate the first target image.

[0166] It is understood that in this embodiment, the first target thumbnail corresponds to the second resolution, and the second operation is used to set the resolution of the first target thumbnail to the third resolution. The mobile phone uses the first image corresponding to the first target thumbnail as a reference frame, and then fuses the M frames of the first image with the reference frame to generate the first target image with the second resolution.

[0167] Then, the mobile phone fuses the first target image of the second resolution with the second image of the first resolution to generate a second photo of the third resolution.

[0168] For example, the mobile phone downsamples the second image at the first resolution to generate a second image at the third resolution; then upsamples the first target image at the second resolution to generate a first target image at the third resolution. The mobile phone then fuses the first target image at the third resolution with the second image at the third resolution to generate a second photo at the third resolution.

[0169] Among them, for the specific implementation method of the mobile phone generating the first target image and generating the second photo with the third resolution, reference can be made to the examples in the above-mentioned embodiment 1 and embodiment 2, and no further details will be given here.

[0170] In summary, using the solutions of the embodiments of the present application, a mobile phone can generate a second photo in response to a user selecting a highlight frame. After generating the second photo, a third photo at a third resolution can be generated in response to the user adjusting the resolution. Alternatively, the mobile phone can also generate a second photo at a third resolution in response to the user selecting a highlight frame and adjusting the resolution. In other words, the mobile phone can provide users with multiple implementation methods, simultaneously meeting the user's needs for selecting a highlight frame and adjusting the resolution, greatly improving the user experience.

[0171] It should be noted that the contents recorded in each embodiment of the present application can explain the technical solutions in other embodiments of the present application, and the technical features recorded in each embodiment can also be applied in other embodiments, and the technical features in other embodiments can be combined to form a new solution. This application only exemplifies several embodiments for illustration, and does not mean that this application is limited to this.

[0172] The present application provides an electronic device that may include a camera, a memory, and one or more processors; the memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs the various functions or steps in the above embodiments. The structure of the electronic device can refer to the above Figure 5 The structure of the electronic device 100 is shown.

[0173] The present application also provides a chip system for use in electronic devices. Figure 13 As shown, the chip system 1100 includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 can be Figure 5 The processor 110 shown is, on this basis, the interface circuit 1102 can be, for example, an interface circuit between the processor 1101 and an external memory; or an interface circuit between the processor 1101 and an internal memory.

[0174] The processor 1101 and the interface circuit 1102 can be interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices (such as the memory of the electronic device 100). For another example, the interface circuit 1102 can be used to send signals to other devices (such as the processor 1101). Exemplarily, the interface circuit 1102 can read instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the electronic device can perform the various functions or steps performed by the mobile phone in the above embodiment. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0175] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the various functions or steps executed by the mobile phone in the above method embodiment.

[0176] The embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the functions or steps executed by the mobile phone in the above method embodiment.

[0177] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules according to the system, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, the functional units in various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0181] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0182] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A photo processing method, characterized in that: Applied to electronic equipment; the method comprises: In response to a capture instruction, the electronic device saves a first photo and caches N frames of first images; wherein the first photo includes a first highlight frame, which is recognized by the electronic device using an AI capture function; the N frames of first images are captured by the electronic device within a preset time period before and after the user inputs the capture instruction; N ≥ 3, and N is a positive integer; In response to a first operation of the user on the first photo, the electronic device displays N first thumbnails; the N first thumbnails correspond one-to-one to the N frames of the first image; In response to the user's selection operation of the first target thumbnail among the N first thumbnails, the electronic device saves a second photo; wherein the second photo includes a second highlight frame, which is the first image corresponding to the first target thumbnail selected by the user.

2. The method according to claim 1, characterized in that The first photo and the second photo are a single photo; or, The first photo and the second photo are covers of a live photo with a duration of x seconds. The number of frames of the live photo is i≥3, where i is a positive integer and x is a positive number.

3. The method according to claim 1 or 2, characterized in that The electronic device saving the second photo in response to the user selecting the first target thumbnail among the N first thumbnails includes: In response to a user selecting a first target thumbnail from the N first thumbnails, the electronic device uses a first image corresponding to the first target thumbnail as a reference frame, fuses M frames of first images with the reference frame to generate a second photo, and saves the second photo; M=N-1, where M is a positive integer; The M frames of first images are images in the N frames of first images excluding the reference frame.

4. The method according to claim 3, characterized in that The step of fusing the M frames of first images with the reference frame to generate the second photo includes: The electronic device performs weighted processing on the pixel value of each pixel in the reference frame and the pixel value of each pixel in each of the M frames of first images to generate the second photo.

5. The method according to claim 4, characterized in that The electronic device performs weighted processing on the pixel value of each pixel in the reference frame and the pixel value of each pixel in each of the M frames of first images to generate the second photo, including: The electronic device performs weighted processing on the pixel value of each pixel in the reference frame and the pixel value of each pixel in each of the M frames of first images to generate a first target image; The electronic device calls a preset image algorithm to process the first target image to generate the second photo; wherein the image quality of the second photo is higher than the image quality of the first photo.

6. The method according to any one of claims 1 to 5, characterized in that The electronic device caches a second image with a first resolution; the second photo corresponds to the second resolution; and the method further includes: The electronic device receives a second operation of the user on the second photo; the second operation is used to set the resolution of the second photo to a third resolution; the third resolution is smaller than the first resolution and larger than the second resolution; In response to the second operation, the electronic device merges the second image with the first resolution and the second photo with the second resolution, and saves the third photo with the third resolution.

7. The method according to claim 6, characterized in that The electronic device fuses the second image at the first resolution and the second photo at the second resolution to save the third photo at the third resolution, including: The electronic device performs downsampling processing on the second image with the first resolution to generate the second image with the third resolution; The electronic device performs upsampling processing on the second photo with the second resolution to generate the second photo with the third resolution; The electronic device fuses the second image of the third resolution with the second photo of the third resolution to generate a third photo of the third resolution.

8. A photo processing method, characterized in that: Applied to electronic equipment; the method comprises: In response to a capture instruction, the electronic device saves a first photo and caches N frames of the first image and a second image of the first resolution; wherein the first photo includes a first highlight frame, which is recognized by the electronic device using an AI snapshot function; the N frames of the first image are captured by the electronic device within a preset time period before and after the user inputs the capture instruction; N ≥ 3, and N is a positive integer; In response to a first operation of the user on the first photo, the electronic device displays N first thumbnails; the N first thumbnails correspond one-to-one to the N frames of the first image; In response to a user's selection operation of a first target thumbnail among the N first thumbnails, and a second operation on the first target thumbnail, the electronic device saves a second photo; wherein, the first target thumbnail corresponds to a second resolution, and the second operation is used to set the resolution of the first target thumbnail to a third resolution; the third resolution is smaller than the first resolution and larger than the second resolution; the second photo corresponds to the third resolution; the second photo includes a second wonderful frame, and the second wonderful frame is the first image corresponding to the first target thumbnail selected by the user.

9. The method according to claim 8, characterized in that The electronic device saving the second photo in response to a user selecting a first target thumbnail from the N first thumbnails and a second operation on the first target thumbnail, comprising: In response to a user selecting a first target thumbnail from the N first thumbnails and a second operation on the first target thumbnail, the electronic device uses a first image corresponding to the first target thumbnail as a reference frame, fuses the M frames of first images with the reference frame, and generates a first target image; the first target image corresponds to the second resolution; The electronic device fuses the first target image with the second resolution and the second image with the first resolution, and saves the second photo with the third resolution; The M frames of first images are images in the N frames of first images excluding the reference frame.

10. The method according to claim 9, characterized in that The electronic device fuses the first target image at the second resolution with the second image at the first resolution and saves the second photo at the third resolution, including: The electronic device performs downsampling processing on the second image with the first resolution to generate the second image with the third resolution; The electronic device performs upsampling processing on the first target image with the second resolution to generate the first target image with the third resolution; The electronic device fuses the first target image with the third resolution and the second image with the third resolution, and saves the second photo with the third resolution.

11. An electronic device, characterized in that: include: memory and one or more processors, cameras; The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 7, or executes the method according to any one of claims 8 to 10.

12. A chip system, characterized in that: include: at least one processor and an interface; The interface is used to receive instructions and transmit them to the at least one processor; The at least one processor executes the instructions so that the electronic device performs the method according to any one of claims 1 to 7, or performs the method according to any one of claims 8 to 10.

13. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the method according to any one of claims 1 to 7, or to execute the method according to any one of claims 8 to 10.

14. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 10.

Images