Image snapshot method, electronic equipment and storage medium

Abstract

The invention provides an image snapshot method, electronic equipment and a storage medium, and relates to the technical field of image processing. After a manual snapshot (wonderful snapshot) function is started, the electronic equipment continuously and automatically collects multiple frames of images and analyzes local optimal frames in the images, and after a user triggers photographing, the electronic equipment continuously collects the multiple frames of images and analyzes the local optimal frames in the images. And then the electronic device determines an optimal frame in the local optimal frame determined before photographing and the local optimal frame determined after photographing as a wonderful snapshot photo. As the finally output wonderful snapshot photo is the optimal frame in a period of time from before shooting to after shooting, according to the scheme of the invention, in a snapshot scene, no matter whether the response moment during shooting is delayed or advanced, the optimal image frame in the shooting process can be snapshot, the optimal moment can be recorded, and the user requirements can be met to the greatest extent.

Description

Image capture method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on February 27, 2023, with application number 202310212397.X and application name “Image capture method, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image processing technology, and in particular to an image capture method, electronic equipment, and storage medium. Background Art

[0003] Mobile phones and other electronic devices can capture photos and videos. When users want to "capture" a wonderful moment with their phones, they trigger the photo by clicking the capture button. However, due to response and action delays, the resulting photo may not be the ideal photo of the moment.

[0004] Currently, one possible way to capture photos of a moment is to first shoot a video and then take a screenshot of a specific frame from the video to save as a photo. This process is cumbersome, impacting the user experience. Furthermore, the quality of the photos captured from the video is very low, further degrading the user experience.

[0005] Summary of the Invention

[0006] The present application provides an image capture method and electronic device. In a capture scenario, regardless of whether the reaction time when taking a photo is delayed or advanced, the optimal image frame in the photo-taking process can be captured, the best moment can be recorded, and user needs can be met to the greatest extent.

[0007] To achieve the above objectives, this application adopts the following technical solutions:

[0008] In a first aspect, the present application provides an image capture method, the method comprising:

[0009] In a photographing mode of the camera, a preview interface of the photographing mode is displayed, wherein a first icon is displayed in the preview interface, and the first icon is in a first state;

[0010] In response to a user operation on the first icon, starting a first function and switching the first icon from the first state to a second state;

[0011] In response to the user operating the capture button at the first moment, the following steps are performed:

[0012] Determine a first image from a plurality of image frames within a first time period before the first moment and a plurality of image frames within a second time period after the first moment;

[0013] generating a first photo based on the first image;

[0014] The first photo is stored in a gallery.

[0015] Through the image capture method provided by the embodiment of the present application, after the first function is turned on, when the user triggers the photo taking, the electronic device will select a frame of image from multiple frames of image within a time period from a certain moment before the photo is taken to a certain moment after the photo is taken, as a wonderful snapshot photo. For example, when the user presses the shooting button (shutter), one frame of image is selected from multiple image frames collected within 1 second in the historical direction before the photo is taken and multiple image frames collected within 0.5 seconds in the future direction after the photo is taken, as a wonderful snapshot photo. Since the wonderful snapshot photo that is finally output is a frame of image selected within a period from before the photo is taken to after the photo is taken, such as an image frame that meets the user's needs obtained by screening under certain conditions, through the solution of the present application, in the snapshot scenario, regardless of whether the reaction moment when taking the photo is delayed or advanced, the optimal image frame in the photo taking process can be captured, the best moment can be recorded, and the user's needs can be met to the greatest extent.

[0016] It should be noted that, in actual implementation, the above-mentioned first function can be called a manual capture function or a wonderful capture function, which can be determined according to actual usage requirements and is not limited in the embodiments of this application.

[0017] It should also be noted that the multiple frames of images in the first time period before the first moment and the multiple frames of images in the second time period after the first moment may include a frame of image at the first moment.

[0018] The image capture method provided in the embodiment of the present application includes two methods: automatic capture and manual capture.

[0019] For manual snapshots, an AI algorithm is added to assist with frame selection within a certain period before and after the user takes the shot. This implementation of an AI semantic-based frame selection solution helps users more easily capture the moment in typical snapshot scenarios. For example, after the user presses the shutter button, the optimal frame is selected from multiple image frames within 1 second in the past direction and 0.5 seconds in the future direction as the perfect snapshot.

[0020] Automatic snapshots use AI algorithms to perceive image content and automatically capture photos within predefined scenes. For example, during a preview, the system captures preview images, compares them frame by frame, selects the optimal frame, and caches it. Once a scene change occurs, the system automatically triggers a photo capture, resulting in an automatically captured photo.

[0021] In some possible implementations, the first image is an image with the highest comprehensive score among multiple frames of images in a first time period before the first moment and multiple frames of images in a second time period after the first moment; wherein the comprehensive score includes an image quality score, a portrait score, and an action score.

[0022] In some possible implementations, after the first function is turned on, the method further includes: continuously capturing preview images to obtain a first preview image frame stream; determining a local optimal image for every M preview image frames in the first preview image frame stream, and inputting the local optimal image determined each time into the first cache queue in sequence, where the local optimal image is the image with the highest comprehensive score among the M preview image frames.

[0023] The first cache queue is a first-in-first-out queue, and the first cache queue caches the most recently determined S frames of local optimal images and the most recently acquired N frames of images, where N is less than or equal to M.

[0024] The total number of frames of the multiple frames of images in the first time period is S×M+N.

[0025] In some possible implementations, the first time period is 1 second. For example, assuming a frame rate of 30 frames per second, the total number of frames of the multi-frame image in the first time period is 30 frames, S is 2, and M and N are both 10.

[0026] In some possible implementations, the first time period is less than 1 second. For example, assuming a frame rate of 30 frames per second, the total number of frames of the multiple frames in the first time period is less than 30 frames, S is 2, M is 10, and N is a positive integer less than 10.

[0027] In the solution of this application, if all 1 second of content is cached, then the electronic device needs to cache at least 1s×30fps=30 frames of images. This application evaluates the local optimum and caches the local optimal frame of the most recently captured image and the most recently captured N frames of image, so only a small number of frames need to be dynamically cached. For example, this application only needs to cache 1+1+10=12 frames at most to achieve historical direction backtracking. If multi-frame synthetic photography is required, then only 4+4+10=18 frames need to be cached at most. Therefore, the solution of this application can save a lot of memory and reduce power consumption.

[0028] In the present application, after the first function is turned on, the electronic device automatically captures multiple frames of images and analyzes the local optimal frames in the images. After the user triggers the photo, the electronic device continues to capture multiple frames of images and analyzes the local optimal frames in the images. Then, the electronic device determines an optimal frame from the local optimal frames determined before the photo is taken and the local optimal frames determined after the photo is taken, as the wonderful snapshot. Since the wonderful snapshot photo that is finally output is the optimal frame in a period of time from before to after the photo is taken, the present application allows, in a snapshot scenario, regardless of whether the reaction time when taking the photo is delayed or advanced, to capture the optimal image frame during the photo shooting process, record the best moment, and meet user needs to the greatest extent.

[0029] In some possible implementations, before determining the first image with the highest comprehensive score, the method further includes: determining the second image with the highest comprehensive score among the S-frame local optimal image and the N-frame image; and fixing the second image in the first cache queue.

[0030] In some possible implementations, before determining the first image with the highest comprehensive score, the method further includes: continuously acquiring T-frame preview images within the second time period after the first moment; inputting the T-frame preview images into the first cache queue in sequence, and replacing images in the first cache queue except the second image with the T-frame preview images.

[0031] The total number of frames of the multiple frames of images in the second time period is T.

[0032] In some possible implementations, the second time period is 0.5 seconds. For example, assuming that the frame rate is 30 frames per second, the total number of frames of the multiple frames in the second time period is 15 frames.

[0033] In the present application, when the user presses the shutter, the electronic device can select frames within 1s in its historical direction and 0.5s in the future direction. If all 1.5 seconds of content are cached, the electronic device needs to cache at least 1.5s×30fps=45 frames of images. The present application evaluates the local optimum and caches the local optimal frames before taking the photo and the images collected after taking the photo, so only a small number of frames need to be dynamically cached. As mentioned above, the present application only needs to cache 12 frames in total. If multiple frames need to be synthesized for taking photos, then only 18 frames need to be cached in total. Therefore, the present application can save a lot of memory and reduce power consumption.

[0034] In some possible implementations, determining the first image with the highest comprehensive score includes: comparing the comprehensive score of the second image with the comprehensive score of each preview image in the T-frame preview images, and determining the image with the highest comprehensive score as the first image.

[0035] In some possible implementations, generating a first photograph based on the first image includes: fusing the first image with a third image to obtain the first photograph, where the third image includes at least one image captured immediately after the first image and / or at least one image captured immediately before the first image.

[0036] In some possible implementations, the method further includes: when the first icon is in the second state, receiving a user operation on the first icon; in response to the user operation on the first icon, turning off the first function and switching the first icon from the second state to the first state.

[0037] In some possible implementations, the photo mode further includes a second function, a switch button for the second function is displayed in the photo mode settings interface, and the switch button for the second function is set to an off state by default. When the switch button for the second function is off, the second function is disabled.

[0038] It should be noted that, in actual implementation, the above-mentioned second function can be called an automatic snapshot function, which can be determined according to actual usage requirements and is not limited in the embodiments of this application.

[0039] In some possible implementations, the method further includes: when the second function is turned on, continuously capturing preview images to obtain a second preview image frame stream; identifying the current shooting scene as the first scene based on the second preview image frame stream; comparing each preview image frame in the second preview image frame stream frame by frame to determine the fourth image with the highest comprehensive score; when the current shooting scene is identified as a change to the second scene based on the second preview image frame stream, generating a second photo based on the fourth image; storing the second photo in the gallery to complete an automatic capture.

[0040] With the above method, when the second function is enabled, during the preview process, preview images are captured, compared frame by frame, the optimal frame is selected, and the optimal frame is cached. Once a scene transition occurs, a photo is automatically triggered to obtain an automatically captured photo. Because the automatically captured photo ultimately output is the optimal frame within a period of time before the transition, the present application solution can automatically capture the optimal image frame in the current scene, recording the best moment and meeting user needs to the greatest extent possible.

[0041] In some possible implementations, after storing the second photo in the gallery, the method further includes: in response to a user operation, jumping from the preview interface to the gallery interface; displaying the fourth image and a second icon in the gallery interface, the second icon being used to indicate that the fourth image is an image obtained by automatic capture.

[0042] In some possible implementations, generating a second photo based on the fourth image includes: fusing the fourth image with a fifth image to obtain the second photo; wherein the fifth image includes at least one image captured immediately after the fourth image, and / or at least one image captured immediately before the fourth image.

[0043] In some possible implementations, the method also includes: when the first icon is in the first state, receiving a user operation on the first icon; in response to the user operation on the first icon, a pop-up box displays first information, and the first information is used to prompt whether to turn on the second function; in response to the user confirming the operation to turn on the second function, turning on the first function and the second function, and switching the first icon from the first state to the second state, and switching the switch button of the second function from the off state to the on state.

[0044] In some possible implementations, the method further includes: turning on or off the second function in response to a user operation in a photo setting interface.

[0045] In some possible implementations, when the second function is turned on, the method further includes: if the first function is turned off, disabling the second function; if the first function is turned back on, lifting the disabling of the second function.

[0046] In some possible implementations, the method further includes: when the second function is enabled, after completing one automatic snapshot, waiting for a first preset time period before starting the next automatic snapshot.

[0047] In some possible implementations, the method further includes: when both the first function and the second function are turned on, the execution priority of the first function is higher than the execution priority of the second function, and the execution of the second function is stopped during the execution of the first function.

[0048] In some possible implementations, the comprehensive score is a weighted average of the image quality score, the portrait score, and the action score, wherein the image quality score, the portrait score, and the action score are respectively preset with corresponding weighting coefficients.

[0049] In some possible implementations, the influencing factors of the image algorithm used vary for different scenes. For example, for scenes with people, the influencing factors of the image algorithm may include image clarity, facial expression, facial composition, facial angle, and whether the eyes are open or closed. For another example, for scenes with action, the influencing factors of the image algorithm may include action evaluation, image clarity, facial expression, facial composition, and facial angle.

[0050] In some embodiments, for each scene, a weighted calculation can be performed based on the influencing factors of the image algorithm to obtain a comprehensive image score. For example, for a human scene, the weighted coefficients corresponding to image clarity, facial expression, facial composition, facial angle, and eye opening and closing are 0.2, 0.5, 0.1, 0.1, and 0.2, respectively. For another example, for an action scene, the weighted coefficients corresponding to action evaluation, image clarity, facial expression, facial composition, and facial angle are 0.4, 0.2, 0.2, 0.1, and 0.1, respectively.

[0051] The method also includes: identifying the current scene as a first scene type based on image features of the preview image; wherein, for different scene types, the image quality score, the portrait score, and the action score are respectively preset with different score weighting coefficients; searching for the score weighting coefficient corresponding to the first scene type; and performing a weighted average operation on the image quality score, the portrait score, and the action score according to the score weighting coefficient corresponding to the first scene type to obtain the comprehensive score.

[0052] For example, through the above-mentioned scheme of the present application, for action scenes, a comprehensive score is obtained after weighted average operation, and the image with the highest comprehensive score is selected, that is, the frame of image with the largest action amplitude (for example, the highest upward jump) and the best smile is selected as the captured photo.

[0053] In some possible implementations, the method further includes: when the AI ​​beauty shot function is turned on and the first function is turned off, collecting a third preview image frame stream; if the brightness of the third preview image frame stream is greater than a brightness threshold, and the third preview image frame stream has a preset subject, and the action score of the preset subject is greater than the action score threshold, then displaying first recommendation information, and the first recommendation information is used to recommend to the user to turn on the first function.

[0054] In some possible implementations, after displaying the first recommendation information, the method further includes: in response to the user confirming an operation to turn on the first function, turning off the AI ​​beauty shooting function and turning on the first function.

[0055] It can be understood that if the brightness of the preview image is greater than the brightness threshold, it can be ensured that after entering the wonderful capture state, the electronic device can obtain better motion exposure reduction effect and good image quality when capturing, ensuring that the captured photos have good display effects.

[0056] In a second aspect, the present application provides an image capture method, comprising:

[0057] In a photographing mode of the camera, a preview interface of the photographing mode is displayed, wherein a first icon is displayed in the preview interface, and the first icon is in a first state;

[0058] In response to a user operating a capture button at a first moment, saving a first image frame captured at the first moment to a gallery;

[0059] In response to a user operation on the first icon, starting a first function and switching the first icon from the first state to a second state;

[0060] Displaying a second image frame captured in real time at a second moment in the preview interface;

[0061] In response to the user operating the shooting button at a third moment, the second image frame is saved in the gallery, wherein the second moment is earlier than the third moment.

[0062] In conjunction with the method of the second aspect, in one possible implementation, the method further includes:

[0063] In response to the user operating the shooting button at a fourth moment, the third image frame is saved in the gallery, wherein a fifth moment at which the third image frame is captured is later than the fourth moment.

[0064] According to the image capture method of the above-mentioned embodiment of the present application, when the first icon in the preview interface of the photo mode is turned off, the electronic device can save the corresponding image frame in real time in response to the user's photo operation; when the first icon is turned on, the electronic device can capture an image frame before or after the operation in response to the user's photo operation, so that the user can quickly turn on the capture function in dynamic scenes and turn off the capture function in non-dynamic scenes to obtain better photo effects, so as to adapt to different shooting scenes.

[0065] In conjunction with the method of the second aspect, in one possible implementation, the method further includes:

[0066] When the first setting item of the camera is turned on, the electronic device automatically saves the fourth image frame to the gallery when detecting that the real-time image frame displayed on the preview interface transitions, wherein the time of capturing the fourth image frame is earlier than the time when the transition occurs.

[0067] According to the image capture method of the above-mentioned embodiment of the present application, it is possible to automatically capture a frame of images within a transition and save them to the gallery without user intervention, thereby assisting the user in capturing the image.

[0068] In a third aspect, the present application provides an image capture method, comprising:

[0069] In a photographing mode of the camera, a preview interface of the photographing mode is displayed, wherein a first icon is displayed in the preview interface, and the first icon is in a first state;

[0070] In response to a user operation on the first icon, starting a first function and switching the first icon from the first state to a second state;

[0071] In response to the user operating the capture button at the first moment, the following steps are performed:

[0072] Determine a first image among multiple frames of images in a time period between a first moment and a third moment; generate a first photo based on the first image; and store the first photo in a gallery; wherein the third moment is later than the first moment.

[0073] Through the image capture method provided by the embodiment of the present application, after the first function is turned on, after the user triggers the photo taking, the electronic device determines an optimal frame from the one-frame image acquired when taking the photo and the multiple frames of image acquired after taking the photo, as a wonderful snapshot. For example, when the user presses the shooting button (shutter), the optimal frame is selected from the one-frame image acquired when taking the photo and the multiple image frames within 0.5s in the future direction after taking the photo, as a wonderful snapshot. Since the wonderful snapshot photo that is finally output is the optimal frame in a period of time from the moment of taking the photo to after taking the photo, through the solution of the present application, in the snapshot scenario, even if the reaction time when taking the photo is in advance, the optimal image frame in the photo taking process can be captured, the best moment can be recorded, and the user needs can be met to the greatest extent.

[0074] In conjunction with the method of the third aspect, in one possible implementation, the method further includes:

[0075] In response to the user operating the capture button at the first moment, the following steps are performed:

[0076] Determine a first image among multiple frames of images in a time period between a second moment and a first moment; generate a first photo based on the first image; and store the first photo in a gallery; wherein the second moment is earlier than the first moment.

[0077] Through the image capture method provided by the embodiment of the present application, after the first function is turned on, the electronic device continuously and automatically collects multiple frames of images and analyzes the local optimal frames in the images. After the user triggers the photo, the electronic device determines an optimal frame from the one-frame image acquired when taking the photo and the multiple-frame image acquired before taking the photo, as a wonderful snapshot. For example, when the user presses the shooting button (shutter), the optimal frame is selected from the multiple image frames within 1 second in the historical direction before taking the photo and the one-frame image acquired when taking the photo, as a wonderful snapshot. Since the wonderful snapshot photo that is finally output is the optimal frame in a period of time from before taking the photo to the moment of taking the photo, through the solution of the present application, in the snapshot scenario, even if the reaction time when taking the photo is delayed, the optimal image frame in the photo process can be captured, the best moment can be recorded, and the user needs can be met to the greatest extent.

[0078] In a fourth aspect, the present application provides an image capture method, including: continuously capturing preview images to obtain a second preview image frame stream; identifying the current shooting scene as a first scene based on the second preview image frame stream; comparing each preview image frame in the second preview image frame stream frame by frame to determine a fourth image with the highest comprehensive score, wherein the comprehensive score includes an image quality score, a portrait score, and an action score; when the current shooting scene is identified as a second scene based on the second preview image frame stream, generating a second photo based on the fourth image; storing the second photo in a gallery to complete an automatic capture.

[0079] With the image capture method provided in the embodiment of the present application, when the second function is enabled, during the preview process, preview images are captured, compared frame by frame, the optimal frame is selected, and the optimal frame is cached. Once a scene transition occurs, a photo is automatically triggered to obtain an automatically captured photo. Because the automatically captured photo ultimately output is the optimal frame within a period of time before the transition, the present application solution can automatically capture the optimal image frame in the current scene, record the best moment, and meet user needs to the greatest extent possible.

[0080] In some possible implementations, the method further includes: after completing one automatic snapshot, waiting for a first preset time period before starting the next automatic snapshot.

[0081] In some possible implementations, generating a second photo based on the fourth image includes: fusing the fourth image with a fifth image to obtain the second photo; wherein the fifth image includes at least one image captured immediately after the fourth image, and / or at least one image captured immediately before the fourth image.

[0082] In some possible implementations, after storing the second photo in the gallery, the method further includes: in response to a user operation, jumping from the preview interface to the gallery interface; displaying the fourth image and a second icon in the gallery interface, the second icon being used to indicate that the fourth image is an image obtained by automatic capture.

[0083] In some possible implementations, before continuously capturing the preview image to obtain the second preview image frame stream, the method further includes: enabling the second function in response to a user operation.

[0084] In a fifth aspect, the present application provides an image capture device, comprising a unit for executing the method described in the first aspect. The device may be configured to execute the method described in the first aspect. For a detailed description of the units in the device, please refer to the description of the first aspect above and will not be repeated here for the sake of brevity.

[0085] The method described in the first aspect above can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions, such as an image acquisition module or unit, a processing module or unit, etc.

[0086] In a sixth aspect, the present application provides an electronic device comprising a processor, a computer program or instructions stored in the processor and a memory, wherein the processor is configured to execute the computer program or instructions so that the method in the first aspect is executed.

[0087] In a seventh aspect, the present application provides a computer-readable storage medium having stored thereon a computer program (also referred to as instructions or code) for implementing the method in the first aspect. For example, when the computer program is executed by a computer, the computer can perform the method in the first aspect.

[0088] In an eighth aspect, the present application provides a chip comprising a processor. The processor is configured to read and execute a computer program stored in a memory to perform the method of the first aspect and any possible implementation thereof. Optionally, the chip further comprises a memory, the memory being connected to the processor via a circuit or wire.

[0089] In a ninth aspect, the present application provides a chip system comprising a processor. The processor is configured to read and execute a computer program stored in a memory to perform the method of the first aspect and any possible implementation thereof. Optionally, the chip system further comprises a memory, the memory being connected to the processor via a circuit or wire.

[0090] In a tenth aspect, the present application provides a computer program product, which includes a computer program (also referred to as instructions or codes). When the computer program is executed by an electronic device, the electronic device implements the method in the first aspect.

[0091] It can be understood that the beneficial effects of the second to tenth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] FIG1 is a schematic diagram of a solution for overcoming the problem of delayed response when capturing an image in the related art;

[0093] FIG2 is a schematic diagram of an image capture method provided in an embodiment of the present application;

[0094] FIG3 is a schematic diagram of the structure of an electronic device used in the image capture method provided in an embodiment of the present application;

[0095] FIG4 is a schematic diagram of the architecture of a software system used in the image capture method provided in an embodiment of the present application;

[0096] FIG5A is a first schematic diagram of an interface for applying the image capture method provided in an embodiment of the present application;

[0097] FIG5B is a second schematic diagram of an interface for applying the image capture method provided in an embodiment of the present application;

[0098] FIG5C is a second schematic diagram of an interface for applying the image capture method provided in an embodiment of the present application;

[0099] FIG6 is a third schematic diagram of an interface for applying the image capture method provided in an embodiment of the present application;

[0100] FIG7 is a fourth schematic diagram of an interface for applying the image capture method provided in an embodiment of the present application;

[0101] FIG8 is a fifth schematic diagram of an interface for applying the image capture method provided in an embodiment of the present application;

[0102] FIG9 is a sixth schematic diagram of an interface for applying the image capture method provided in an embodiment of the present application;

[0103] FIG10 is a schematic diagram of an algorithm of a manual capture method in an image capture method according to an embodiment of the present application;

[0104] FIG11 is a second algorithm diagram of the manual capture method in the image capture method provided in an embodiment of the present application;

[0105] FIG12 is a schematic diagram of an algorithm for an automatic image capture method in an embodiment of the present application;

[0106] FIG13 is a first schematic diagram of a combination of a manual capture mode and an automatic capture mode in the image capture method provided in an embodiment of the present application;

[0107] FIG14 is a second schematic diagram of a combination of manual capture and automatic capture in the image capture method provided in an embodiment of the present application;

[0108] FIG15 is a schematic diagram 1 of a step-by-step implementation of a manual capture method in an image capture method provided in an embodiment of the present application;

[0109] FIG16 is a second schematic diagram of a step-by-step implementation of a manual capture method in an image capture method provided in an embodiment of the present application;

[0110] FIG17 is a third schematic diagram of a step-by-step implementation of a manual capture method in an image capture method provided in an embodiment of the present application;

[0111] FIG18 is a fourth schematic diagram of a step-by-step implementation of a manual capture method in an image capture method provided in an embodiment of the present application;

[0112] FIG19 is a flowchart diagram 1 of the image capture method provided in an embodiment of the present application;

[0113] FIG20 is a second flow chart of the image capture method provided in an embodiment of the present application;

[0114] FIG21 is a schematic diagram of scene recognition, shot moment detection, and wonderful moment recognition in the image capture method provided in an embodiment of the present application;

[0115] FIG22 is a flow chart of an image capture method according to an embodiment of the present application;

[0116] FIG23 is a schematic diagram of a flow chart of an image capture method provided in an embodiment of the present application;

[0117] FIG24 is a schematic diagram of an interface for applying an image capture method provided in an embodiment of the present application;

[0118] FIG25 is a flow chart of another image capture method provided in an embodiment of the present application;

[0119] FIG26 is a schematic diagram of an interface for applying another image capture method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0120] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0121] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0122] In the specification and claims herein, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. In the description of the embodiments of this application, unless otherwise specified, the meaning of "plurality" refers to two or more. For example, "multiple processing units" refers to two or more processing units, etc.; "multiple components" refers to two or more components, etc.

[0123] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0124] Mobile phones and other electronic devices can capture photos and videos. When users want to "capture" a wonderful moment with their phone, they trigger the photo by clicking the capture button. However, due to the response lag during the "capture" process, the resulting photo may not be the ideal photo of the moment. This response lag can be caused by human factors (such as response delay and motion delay) or by the phone itself (such as camera response delay).

[0125] Currently, one possible way to capture photos of a moment is to first shoot a video and then take a screenshot of a specific frame from the video to save as a photo. This process is cumbersome, impacting the user experience. Furthermore, the quality of the photos captured from the video is low, which also degrades the user experience.

[0126] In the related art, in order to solve the problem of missing wonderful moments due to delayed reaction when capturing images, a corresponding solution has been proposed. As shown in Figure 1, when the user clicks the shooting button (denoted as T1), an image frame at a certain moment before T1 (denoted as T2) is collected and used as the captured photo. For example, the difference between moment T2 and moment T1 is a preset value, such as 50 milliseconds (ms). In this way, by selecting a frame 50ms ahead, the image frame of the moment before the photo is taken is obtained, and the "0" delay is physically achieved to avoid the problem of delayed reaction. However, the photos captured in this way may not be clear, or the face angle may not be good, or the facial expression may not be ideal, so they may not be ideal photos of a certain moment and cannot meet user needs.

[0127] Based on this, an embodiment of the present application provides an image capture method and electronic device, which improves the user experience by improving the underlying system of the mobile phone. For example, as shown in Figure 2, in this application, the artificial intelligence (AI) capture frame selection interval can be set to the period from 1 second (s) before taking the photo to 0.5s after taking the photo. Among them, the time period between 1s before taking the photo and the moment of taking the photo can be called the historical time period, also known as the first time period, recorded as [-1s, 0s]. The time period between the moment of taking the photo and 0.5s after taking the photo can be called the future time period, also known as the second time period, recorded as [0s, 0.5s]. Accordingly, the AI ​​capture frame selection interval is expressed as [-1s, 0.5s]. When triggering the photo, the optimal frame can be intelligently selected within the AI ​​capture frame selection interval.

[0128] Among them, the method of intelligently selecting the optimal frame may include: selecting a frame that meets the preset image conditions from multiple frames of images within the AI ​​capture frame selection interval as the local optimal frame, or selecting multiple frames of images that meet the preset image conditions from multiple frames of images within the AI ​​capture frame selection interval, fusing them as the local optimal frame, and then outputting the local optimal frame.

[0129] The preset image condition can be any of the following:

[0130] (1) The image quality score is greater than or equal to the preset image quality threshold;

[0131] (2) The portrait score is greater than or equal to the preset portrait threshold;

[0132] (3) the action score is greater than or equal to the preset action threshold;

[0133] (4) The weighted average of the image quality score, portrait score, and action score (i.e., the comprehensive score) is greater than or equal to the preset overall threshold.

[0134] Among them, the image quality score can be determined based on the image resolution, whether the image is blurry, etc.

[0135] The portrait score can be determined based on the facial expression of the person in the image. For example, if the facial expression of the person in the image matches a preset expression, the portrait score can be determined based on the score corresponding to the preset expression. For example, the preset expressions may include smiling, laughing, blinking, pouting, etc.

[0136] The action score can be determined based on the action of the character in the image. For example, when the action of the character in the image matches a preset action, the action score can be determined based on the score corresponding to the preset action. For example, the preset actions may include jumping upwards, running, hurdling, shooting, and so on.

[0137] It should be noted that the above preset image conditions are illustrative examples. It is understood that in actual implementation, the preset image conditions may also be other possible situations. For example, the preset image condition may be: the weighted average of the image quality score and the portrait score is greater than or equal to a preset threshold. Specifically, the preset image conditions can be set according to actual usage requirements and are not limited in the embodiments of this application.

[0138] This application solution provides manual capture mode and automatic capture mode.

[0139] Manual capture mode: When the camera is turned on and the "Great Capture" function is enabled, if the user clicks the capture button, the electronic device can determine the local optimal frame within the period from 1 second before to 0.5 seconds after the photo is taken according to the first frame selection strategy provided in this application, and generate a photo based on the local optimal frame. The photo generated is also called a manually captured photo. The first frame selection strategy will be described in detail below.

[0140] Automatic capture mode: When the camera is turned on, the "Great Capture" function is turned on, and the "Automatic Capture" function is turned on, the electronic device can collect multiple frames of preview images in real time and determine the local optimal frame in the multiple frames of preview images according to the second frame selection strategy provided in this application. When the electronic device detects a scene change (i.e., a transition), it can generate a photo based on the local optimal frame. The photo generated is also called an automatically captured photo. The second frame selection strategy will be described in detail below.

[0141] Therefore, the present application can achieve the following technical effects: in a snapshot scenario, regardless of whether the reaction time when taking a photo is delayed or advanced, the optimal image frame in the photo-taking process can be captured, the best moment can be recorded, and user needs can be met to the greatest extent.

[0142] The image capture method provided in the embodiments of the present application can be applied to electronic devices with shooting functions. Electronic devices include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. The terminal devices can be mobile phones, smart TVs, wearable devices, tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of the present application do not limit the specific technologies and specific device forms adopted by the electronic devices.

[0143] 3 , which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, etc.

[0144] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0145] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. For example, the processor 110 is used to execute the image capture method in the embodiment of the present application.

[0146] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0147] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0148] The external memory 120 generally refers to an external memory. In the embodiment of the present application, the external memory refers to a memory other than the memory of the electronic device and the cache of the processor, and the memory is generally a non-volatile memory.

[0149] Internal memory 121, also referred to as "memory," can be used to store computer-executable program code, including instructions. Internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (e.g., sound playback, image playback, etc.).

[0150] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be an organic light-emitting diode (OLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0151] The electronic device 100 also includes various sensors that can convert various physical signals into electrical signals. For example, the pressure sensor 180A is used to sense and convert pressure signals into electrical signals. The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall effect sensor. The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc. The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. The bone conduction sensor 180M can obtain a vibration signal.

[0152] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0153] For example, in an embodiment of the present application, the touch sensor 180K can detect a user's click operation on an application icon, and pass the detected click operation to the application processor, determine that the click operation is used to start or run the application, and then execute the running operation of the application.

[0154] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0155] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0156] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0157] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0158] The above is a specific description of the embodiments of the present application using the electronic device 100 as an example. It should be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. The electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0159] The electronic device provided in the embodiments of the present application can be user equipment (UE), for example, it can be a mobile terminal (such as a user's mobile phone), a tablet computer, a desktop computer, a laptop computer, a handheld computer, a netbook, a personal digital assistant (PDA), etc.

[0160] Furthermore, operating systems run on the above components, such as the iOS operating system developed by Apple, the Android open-source operating system developed by Google, and the Windows operating system developed by Microsoft. Application programs can be installed and run on these operating systems.

[0161] The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0162] FIG4 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0163] A layered architecture divides the system into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: from top to bottom, the application layer (applications), the application framework layer (application framework), the hardware abstraction layer (HAL), the kernel layer (kernel), and the hardware layer.

[0164] The application layer may include a series of application packages. In the embodiment of the present application, the application package may include a camera, a gallery, etc.

[0165] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, where the camera access interface may include a highlight capture service and an automatic capture service. The camera access interface is used to provide an application programming interface and programming framework for camera applications.

[0166] The hardware abstraction layer is an interface layer located between the application framework layer and the kernel layer, providing a virtual hardware platform for the operating system. In the embodiment of the present application, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0167] The camera hardware abstraction layer (HAL) can provide virtual hardware for camera devices such as the front and rear cameras. The camera algorithm library can include a wonderful capture algorithm and an automatic capture algorithm. In other words, the camera algorithm library contains the operating code and data for implementing the image capture method provided in the embodiments of this application.

[0168] The kernel layer is the layer between hardware and software. It includes various hardware drivers. These drivers include camera device drivers, digital signal processor drivers, and image processor drivers. The camera device driver drives the camera sensor to capture images and the image signal processor to pre-process them. The digital signal processor driver drives the digital signal processor to process images. The image processor driver drives the graphics processor to process images.

[0169] The following summarizes the image capture method in the embodiment of the present application in combination with the above hardware structure and system structure:

[0170] Step 1: The electronic device 100 starts a shooting function and obtains a preview image frame stream.

[0171] This step 1 is performed continuously. In response to the user's operation on the camera application icon (such as a click operation), the camera application calls the camera access interface of the application framework layer to start the camera application, and then sends an instruction to start the camera application by calling the camera device (such as the rear camera) in the camera hardware abstraction layer. The camera hardware abstraction layer sends the instruction to the camera device driver of the kernel layer. The camera device driver can start the sensor of the rear camera, collect the preview image light signal through the sensor, and transmit the preview image light signal to the image signal processor for preprocessing to obtain a preview image frame stream (at least 2 frames of preview images constitute an image sequence), and then transmit the preview image frame stream to the camera hardware abstraction layer through the camera device driver.

[0172] Step 2: The electronic device 100 obtains a processing flow according to the preview image frame flow.

[0173] This step 2 is continuous. The camera hardware abstraction layer can send the preview image frame stream to the camera algorithm library. Based on the support of the digital signal processor and image processor, the camera algorithm library can first downsample the original stream to obtain a low-resolution processed stream.

[0174] Step 3: The electronic device 100 evaluates the image frames in the processing flow and determines a local optimal frame.

[0175] This step 3 is ongoing. Leveraging the support of the digital signal processor and image processor, the camera algorithm library can utilize image quality detection algorithms, face detection algorithms, smile detection algorithms, and eye closure detection algorithms to detect the image quality, image content, and motion of each image frame. It then determines the image quality score, face score, motion score, and overall score for each frame, ultimately identifying the best image frame, or the locally optimal frame.

[0176] Step 4: The electronic device 100 displays the captured wonderful photos.

[0177] The camera algorithm library can send the captured wonderful photos to the camera hardware abstraction layer, which can then display them.

[0178] It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0179] The execution subject of the image capture method provided in the embodiment of the present application can be the above-mentioned electronic device (such as a mobile phone), or it can be a functional module and / or functional entity in the electronic device that can implement the image capture method, and the present application solution can be implemented by hardware and / or software. The specific implementation can be determined according to actual use requirements and is not limited by the embodiment of the present application. The following uses an electronic device as an example and combines the accompanying drawings to illustrate the image capture method provided in the embodiment of the present application.

[0180] In order to better understand the embodiment of the present application, the following first briefly describes the "wonderful capture" function and the "automatic capture" function provided in the embodiment of the present application in conjunction with the mobile phone interface diagram:

[0181] When "Wonderful Snap" is in the off state, if the user triggers a photo, the electronic device will complete the photo in a conventional photo taking manner, that is, it will capture a frame of image at the moment the user triggers the photo, and generate a photo based on the frame of image. Figure 5A shows an interface diagram of generating a photo when the user triggers the photo taking when "Wonderful Snap" is in the off state. As shown in (b) of Figure 5A, the "Wonderful Snap" function is not turned on, and the user clicks the shooting button 10 to trigger the photo taking. As shown in (c) of Figure 5A, in response to the user operation, the electronic device captures a frame of image at the moment the user triggers the photo taking, generates a photo based on the frame of image, and displays a thumbnail 11 of the photo. (a) and (d) of Figure 5A respectively show the camera preview interface before and after taking the photo. It should be noted that the electronic device generates a photo based on a frame of image at the shooting moment (the first moment), and it does not select frames from multiple frames of image before and after taking the photo.

[0182] Figure 5B shows a schematic diagram of the interface for turning on "Wonderful Snapshot" and capturing an image in an embodiment of the present application. As shown in (a) in Figure 5B, after the mobile phone turns on the camera, it enters the photo mode and displays the photo preview interface. A switch icon 13 for the "Wonderful Snapshot" function provided by an embodiment of the present application is added to the photo preview interface or the photo function bar. In the initial case, the "Wonderful Snapshot" function can be turned off by default, and the switch icon 13 is unselected by default. In order to recommend this function to customers, the electronic device can introduce the "Wonderful Snapshot" function in a pop-up window for the switch icon 13. For example, when turning on "Wonderful Snapshot", when shooting scenes of people smiling, jumping, running, and cats, dogs, etc., the intelligent high-speed shutter helps you capture wonderful moments.

[0183] When "Wonderful Snap" is on, if the user triggers shooting, the electronic device can "capture" the wonderful moment and provide the user with photos of the wonderful moment. As shown in (b) in Figure 5B, the user clicks the switch icon 13 to trigger the "Wonderful Snap" function. As shown in (c) in Figure 5B, in response to the user operation, the electronic device updates the switch icon 13 from an unselected state (such as the icon color is light) to a selected state (such as the icon color is dark), and the electronic device accordingly turns on the "Wonderful Snap" function and displays a prompt message that "Wonderful Snap" is turned on in the photo preview interface. In this case, if the user triggers shooting, the electronic device can "capture" the wonderful moment and provide the user with photos of the wonderful moment.

[0184] As shown in (d) of Figure 5B, when the "Great Snap" function is turned on, the user clicks the shooting button 10 to trigger the taking of a photo. As shown in (e) of Figure 5B, in response to the user's operation, the electronic device "captures" the wonderful moment, provides the user with a photo of the wonderful moment, and displays a thumbnail 11 of the photo. It should be noted that the electronic device selects a frame of image before the shooting moment according to the frame selection strategy provided in the embodiment of the present application to generate a photo. If the user needs to view the original image of the photo, the user can click on the thumbnail 11 of the photo; in response to the user's operation, the electronic device switches from the current photo preview interface to the gallery interface, and the original image of the photo is displayed in the gallery interface for the user to view.

[0185] Figure 5C shows another interface schematic diagram of the "Great Snap" function being turned on and image capture in an embodiment of the present application. Figure 5C (a) shows the camera preview interface within a preset time period (such as within 1 second) before taking a photo, during which the electronic device is collecting images. As shown in Figure 5C (b), when the "Great Snap" function is turned on, the user clicks the shooting button 10 to trigger the photo capture. Figure 5C (c) shows the camera preview interface within a preset time period (such as within 0.5 seconds) after the photo is triggered. During this period, the electronic device is still collecting images and no photo has been generated. As shown in Figure 5C (d), in response to the user operation, the electronic device selects a frame from the multiple frames of images collected before the photo is triggered and the multiple frames of images collected after the photo is triggered, generates a photo, and displays a thumbnail 11 of the photo, thereby "capturing" the wonderful moment and providing the user with a photo of the wonderful moment. It should be noted that the electronic device selects a frame of image after the shooting moment according to the frame selection strategy provided in the embodiment of the present application to generate a photo.

[0186] It should be noted that when the "Great Capture" function is switched from off to on, the electronic device will correspondingly change certain function settings or parameter settings of the camera.

[0187] In some embodiments, when the "Highlight Snap" function is switched from off to on, the electronic device adjusts shutter parameters and increases shutter speed to "capture" the wonderful moment and provide the user with photos of the wonderful moment.

[0188] In some embodiments, as shown in (b) of FIG. 5B , when the “Highlight Snap” function is turned off, the camera supports a zoom range 14 of [0.5x, 10x], where x represents a multiple, and for example, 0.5x, 1x, 5x, and 10x can be displayed in the user interface. As shown in (c) of FIG. 5B , when the “Highlight Snap” function is turned on, the camera supports a zoom range 15 of [0.5x, 5x], and for example, 0.5x, 1x, and 5x can be displayed in the user interface. In other words, when the “Highlight Snap” function is turned on, the camera supports a maximum telephoto zoom of 5x, which is more conducive to “capturing” wonderful moments and providing users with photos of wonderful moments.

[0189] In some embodiments, when the "Highlight Snap" function is off, the camera's "AI Beauty Snap" function is enabled by default, as shown in FIG5B(b), where the "AI Beauty Snap" function icon is in an on state 16. When the "Highlight Snap" function is on, the camera's "AI Beauty Snap" function is disabled, as shown in FIG5B(c), where the "AI Beauty Snap" function icon is in an off state 17. In other words, when the "Highlight Snap" function is enabled, the electronic device automatically disables the "AI Beauty Snap" function, making it easier to "capture" wonderful moments and provide users with photos of those moments.

[0190] In some embodiments, when the "Highlight Snap" function is off, the camera's "flash" function can be turned on or off by the user, as shown in FIG5B (b), where the "flash" function icon is in a configurable state 18. When the "Highlight Snap" function is on, the camera's "flash" function is disabled, as shown in FIG5B (c), where the "flash" function icon is in a disabled state 19. In other words, when the "Highlight Snap" function is on, the electronic device disables the "flash" function, which is more conducive to "capturing" wonderful moments and providing users with photos of wonderful moments.

[0191] In an embodiment of the present application, the electronic device prompts the user with relevant status information of the "Great Snap" function in various ways. As shown in (a) in Figure 6, when the "Great Snap" function is turned on, the electronic device updates the switch icon 11 of the "Great Snap" function to a selected state, and displays a prompt message 20 that "Great Snap" is turned on and hides the prompt message 20 after a period of time, and displays a prompt entry 21 for "Great Snap". After the user clicks on the prompt entry 21, as shown in (b) in Figure 6, the electronic device pops up a prompt box 22: "Great Snap", when shooting scenes of people smiling, jumping, running, and cats, dogs, etc., the smart high-speed shutter helps you capture wonderful moments. Through various prompt methods, it is easier for users to understand the "Great Snap" function and status changes.

[0192] In an embodiment of the present application, as shown in (c) in Figure 6, when the "Great Snap" function is turned on, if the user clicks the switch icon 11 of the "Great Snap" function, then as shown in (d) in Figure 6, the electronic device can turn off the "Great Snap" function, and update the switch icon 11 of the "Great Snap" function to an unselected state, and display a prompt message 23 that "Great Snap" is turned off and hide the prompt message 23 after a period of time, and hide the prompt entry 21 of "Great Snap".

[0193] It should be noted that when the "Great Snap" function is turned on and then off, the electronic device can automatically reset the zoom range supported by the camera, automatically turn on the "AI Beauty Shot" function, and unblock the "Flash" function.

[0194] As mentioned above, when the "Great Snap" function is turned on, the user can trigger the "Great Snap" by clicking the photo button to obtain a great photo. In addition, this application also adds an "Auto Snap" function. When the "Auto Snap" function is turned on, the electronic device can automatically perform "Great Snap" and obtain a great photo without the user having to click the photo button. The following is an introduction to the "Auto Snap" function.

[0195] In some embodiments, the "auto-snap" function is turned off by default. The present application provides the following two ways to turn on the "auto-snap" function.

[0196] Method 1: The electronic device prompts the user through a pop-up reminder whether to turn on the "automatic snapshot" function.

[0197] In some embodiments, after the user turns on the "Great Snap" function for the first time and then turns off the "Great Snap" function, when the user manually triggers the "Great Snap" function again, the electronic device can pop up a window to remind the user whether to turn on the "Auto Snap" function.

[0198] For example, as shown in (a) of FIG7 , when the “Great Snap” function is turned off, after the user clicks the switch icon 11 of the “Great Snap” function again, as shown in (b) of FIG7 , the electronic device displays a pop-up window: Do you want to turn on the “Auto Snap” function? Automatically take photos when intelligently identifying people smiling, jumping, running, and scenes such as cats and dogs. After turning it on, it can be turned off in “Settings” > “Smart Photo”. A cancel control and an on control are displayed. After the user clicks the switch icon 11 to turn on the control, as shown in (c) of FIG7 , the electronic device no longer displays a pop-up window and switches back to the photo preview interface. In this case, both the “Great Snap” function and the “Auto Snap” function have been turned on, so the electronic device can not only “automatically capture” wonderful photos, but also “capture” wonderful photos in response to the user’s triggering of the photo operation.

[0199] Method 2: The user triggers "Auto Snap" in "Settings" > "Smart Photo" > "Auto Snap", and the electronic device responds to the user operation and turns on the "Auto Snap" function.

[0200] For example, as shown in (a) of FIG8 , after a user clicks the photo setting icon 24 in the photo function bar, the electronic device switches to the photo setting interface, as shown in (b) of FIG8 , which includes a smart photo setting option. After the user clicks the smart photo setting option, as shown in (c) of FIG8 , the electronic device switches from the photo setting interface to the smart photo setting interface, which includes an automatic snapshot setting option. Assuming that the switch for the automatic snapshot setting option is off, after the user slides a finger in the direction of the arrow on the switch icon for the automatic snapshot setting option, as shown in (d) of FIG8 , the electronic device switches the switch for the automatic snapshot setting option to on and turns on the "auto snapshot" function. After the "auto snapshot" function is turned on, as shown in (e) and (f) of FIG8 , the electronic device can intelligently recognize scenes such as people smiling, jumping, running, and cats and dogs and automatically take photos.

[0201] It should be noted that when the "Great Snap" function is enabled, the electronic device can respond to user operations to enable the "Auto Snap" function. Of course, the electronic device can also respond to user operations to disable the "Auto Snap" function. Specifically, when the "Auto Snap" function is enabled, the user can trigger the "Auto Snap" function to be disabled in "Settings" > "Smart Photo" > "Auto Snap".

[0202] In some embodiments, when the "Auto Snap" function is turned on, if the "Highlight Snap" function is turned off, the "Auto Snap" function will be automatically disabled; if the "Highlight Snap" function is turned on again, the "Auto Snap" function will be automatically undisabled.

[0203] In some embodiments, when the "auto-snap" function is turned on, if the "wonderful capture" function is turned on, the electronic device can not only intelligently identify scenes such as people smiling, jumping, running, and cats and dogs and automatically take pictures, that is, "auto-snap", but can also respond to the user's triggering of the photo operation and "capture" wonderful photos.

[0204] As described above, the embodiment of the present application not only adds a switch for the "wonderful capture" function in the photo mode interface, but also adds a switch for the "automatic capture" function in the photo setting interface, making it convenient for users to set it according to actual usage needs.

[0205] In an embodiment of the present application, the electronic device also adds corresponding marks to the photos captured "automatically" so that the user can know that the electronic device has "automatically captured" wonderful photos for the user to view and manage.

[0206] For example, FIG9 (a) shows a photo interface of a gallery, which displays thumbnails of all photos in the gallery, including thumbnails of photos captured automatically, thumbnails of photos captured by the user when the user triggers the "Great Snap" function, and thumbnails of photos captured when the user triggers the function. For example, the electronic device may superimpose an automatic capture mark 26 on the thumbnail 25 of the photo captured automatically, making it easier for the user to identify the photo captured automatically.

[0207] As another example, FIG9 (b) shows the album interface of the gallery, in which the album interface displays thumbnails of all the photo collections in the gallery. When the user clicks on the thumbnail of all the photo collections, the electronic device switches to the thumbnail interface of all the photos, which displays thumbnails of wonderful photos obtained by "automatically capturing", thumbnails of wonderful photos obtained by the user triggering "wonderful capturing", and thumbnails of photos obtained by the user triggering shooting. Similar to FIG9 (a), the electronic device can superimpose an automatic capture mark 26 on the thumbnail 25 of the photo obtained by "automatically capturing", so that the user can easily identify the photos obtained by "automatically capturing".

[0208] After the user clicks on the photo thumbnail 25 obtained by "auto-snap" as shown in (a) or (c) of Figure 9, the electronic device displays the photo 27 obtained by "auto-snap", as shown in (d) of Figure 9, and displays an auto-snap mark 26 in a preset area. After the user clicks on the auto-snap mark 26, as shown in (e) of Figure 9, the electronic device displays a pop-up window introducing the "auto-snap" function. For example, when the camera turns on "auto-snap", it automatically takes photos when it intelligently recognizes scenes such as people smiling, jumping, running, and cats and dogs.

[0209] The above describes the settings and application process of the "Highlight Snap" function and the "Automatic Snap" function provided in the embodiment of this application on the mobile phone interface. The following, combined with the accompanying drawings, details the frame selection strategy and snapshot principle of the "Highlight Snap" function and the frame selection strategy and snapshot principle of the "Automatic Snap" function provided in the embodiment of this application.

[0210] (1) Frame selection strategy and capture principle of the “Excellent Capture” function (corresponding to manual capture)

[0211] When the "Great Snap" function is turned on, the electronic device can respond to the user's trigger to take a photo, and the intelligent high-speed shutter can capture the wonderful moments of people smiling, jumping, running, and scenes such as cats and dogs.

[0212] First, it should be noted that in the embodiments of the present application, when the camera is turned on and the "Great Snap" function is enabled, before the electronic device receives a user-triggered photo capture operation, the camera of the electronic device will continuously capture multiple preview image frames, and accordingly, a preview image frame stream can be obtained. For the captured preview image frame stream, the electronic device can periodically analyze a preset number of frames, determine the local optimal frame, and store it in a cache queue.

[0213] For ease of description, we assume a frame rate of 30 fps, meaning the camera can capture 30 frames of image every second. FPS stands for frames per second, meaning the number of frames per second.

[0214] FIG10 is a schematic diagram showing a wonderful snapshot process (manual triggering of taking photos is required) in an embodiment of the present application.

[0215] In an embodiment of the present application, for example, referring to FIG10 , the frame selection interval for a "wonderful capture" may be [-1s, 0.5s]. That is, the present application solution can select the optimal frame for a "wonderful capture" from multiple frames of images within the historical time period (a total of 1s) before the manual photo (the wonderful capture) and the future time period (a total of 0.5s) after the manual photo is taken.

[0216] The historical time period contains a total of 30 frames, or possibly less than 30 frames. The following description uses 30 frames as an example. The future time period contains a total of 15 frames. In other words, the present application solution can select the optimal frame for a "wonderful capture" from the 30 frames before and the 15 frames after the manual capture.

[0217] (a) in Figure 10 shows the frame selection process of the "wonderful snapshot" of the solution of the present application. Referring to (a) in Figure 10, before taking a picture, preview images are continuously captured to obtain a preview image frame stream. For example, for the preview image frame stream, a local optimal frame is periodically determined for every 10 frames of image, and input into the cache queue in sequence. Due to the limitation of memory size, only a limited number (for example, 3) of the most recently determined local optimal frames are retained in the cache queue for later determination of the optimal frame of the historical time period. For the sake of convenience, the following is an example of alternately retaining the three most recently determined local optimal frames in the cache queue.

[0218] To better illustrate that the frame selection process of the "Great Snap" feature of the present application is different from the framing method of conventional photo taking, Figure 10(b) shows the framing method of conventional photo taking. Referring to Figure 10(b), when "Great Snap" is off, if the user triggers a photo, the electronic device will complete the photo taking using the conventional photo taking method, that is, it will capture a frame of image at the moment the user triggers the photo taking, and generate a photo based on the frame of image.

[0219] In some embodiments, each local optimal frame and multiple nearby frames can be input into a cache queue together, so that it is convenient to later combine the multiple nearby frames to perform image fusion on the local optimal frame to obtain a local optimal frame with higher image quality.

[0220] Referring again to FIG10 , at the moment of triggering a wonderful snapshot, the electronic device determines the optimal frame for the previous time period and the optimal frame for the future time period. Then, by comparing the optimal frame for the previous time period with the optimal frame for the future time period, the optimal frame is determined. Based on this optimal frame, the wonderful snapshot is generated and output, and the photo is stored in the album library.

[0221] It should be noted that the historical time period is 1 second long and the future time period is 0.5 seconds long as an example for illustrative explanation here. It can be understood that in actual implementation, the specific length of the historical time period and the future time period can be determined according to actual usage requirements, and the embodiments of this application are not limited thereto.

[0222] It should also be noted that Figure 10 is illustrated using a total of 30 frames in the historical time period as an example. It is understandable that since the moment of triggering the photo-taking is random and the frame rate may also be other parameters, the historical time period may be less than 30 frames and the historical time period may be less than 1 second.

[0223] The above describes the overall idea of ​​the frame selection strategy of the "wonderful capture" function provided in the embodiment of the present application in combination with Figure 10. The specific implementation method of the frame selection strategy of the "wonderful capture" function is described in detail below in combination with the accompanying drawings.

[0224] FIG11 shows a schematic diagram of determining the optimal frame of a historical time period in an embodiment of the present application.

[0225] As shown in Figure 11 (a), the historical time period is 1 second, and the preview image frame stream includes the most recently acquired 30 frames. The local best moment (LBM) is determined for every 10 frames. Accordingly, the cache queue stores the first LBM, the second LBM, and the most recent 10 frames, where the most recent 10 frames include the third LBM.

[0226] As time goes by, the latest 10 frames of image data are input into the cache queue, causing the cache queue's storage content to be updated. According to the first-in-first-out rule, the first LBM stored in the cache queue is discarded, and the second LBM stored in the cache queue is updated to the first LBM.

[0227] As shown in (b) in Figure 11, the historical time period is 1 second, the preview image frame stream includes the 30 most recently captured frames, the last second LBM in the cache queue is updated to the first LBM this time, the last third LBM is updated to the second LBM this time, and the latest 10 frames of images are updated and cached, among which the latest 10 frames of images include the third LBM this time.

[0228] As described above, as time goes by, the 10 most recently acquired frames of images are sequentially input into the cache queue, so that the storage content of the cache queue is updated alternately according to the first-in-first-out rule.

[0229] When the user triggers a photo, the latest captured image may be less than 10 frames, as shown in (c) in Figure 11, and the historical time period is less than 1 second. Assuming that the preview image frame stream includes the 24 most recently captured frames, the last second LBM in the cache queue is updated to the first LBM this time, and the last third LBM is updated to the second LBM this time, and the latest 4 frames of images are updated and cached, which may not include the third LBM.

[0230] Through the image capture method provided by the embodiment of the present application, after the wonderful capture function is turned on, the electronic device continuously and automatically captures multiple frames of images and analyzes the local optimal frames in the images, and caches the local optimal frames of the most recently captured images. After the user triggers the photo, the electronic device continues to capture multiple frames of images and analyzes the local optimal frames in the images. Then, the electronic device determines an optimal frame from the local optimal frames determined before the photo is taken and the local optimal frames determined after the photo is taken, as the wonderful captured photo. Since the wonderful captured photo that is finally output is the optimal frame in the time period from before to after the photo is taken, through the solution of the present application, in the capture scenario, regardless of whether the reaction time when taking the photo is delayed or advanced, the optimal image frame in the photo process can be captured, the best moment can be recorded, and user needs can be met to the greatest extent.

[0231] (2) Frame selection strategy and capture principle of the “auto-capture” function (corresponding to auto-capture)

[0232] After the "auto snapshot" function is turned on, electronic devices can intelligently identify people smiling, jumping, running, as well as cats, dogs and other scenes, and automatically take photos without the user having to click the photo button.

[0233] First of all, it should be noted that in an embodiment of the present application, when the camera is turned on, the "wonderful capture" function is turned on, and the "automatic capture" function is turned on, the camera of the electronic device will continue to capture multiple frames of preview images, and a preview image frame stream can be obtained accordingly. For the preview image frame stream captured, the electronic device can compare frame by frame, determine a local optimal frame in the current scene, and store the local optimal frame in a cache queue. When the electronic device detects a scene change (i.e., a transition), it can generate an automatically captured photo based on the local optimal frame.

[0234] Figure 12 shows a schematic diagram of the automatic capture process in an embodiment of the present application. As shown in Figure 12, the electronic device continuously captures preview images and determines the shooting scene (referred to as scene 1) based on the objects in the preview image, and compares the images in the preview image frame stream frame by frame to determine the local optimal frame of scene 1. When the electronic device recognizes scene 2 (i.e., transition), the electronic device generates and outputs an automatically captured photo based on the local optimal frame of scene 1, and stores the photo in the album gallery.

[0235] The above describes the frame selection strategy and capture principle of manual capture, and the frame selection strategy and capture principle of automatic capture. The following describes the possible combination implementation scenarios of manual capture and automatic capture.

[0236] In some embodiments, when the "wonderful capture" function is turned on and the "automatic capture" function is turned on, the order of execution of "automatic capture" and "manual capture" is not limited. It is determined according to actual usage requirements and is not limited in the embodiments of this application.

[0237] For example, as shown in (a) of FIG13 , the electronic device may first perform “automatic capture” and then perform “manual capture”.

[0238] As another example, as shown in (b) of FIG13 , the electronic device may also first perform “manual capture” and then perform “automatic capture”.

[0239] It should be noted that in the scenario where "manual snapshot" is triggered, the execution priority of "manual snapshot" is higher than the execution priority of "automatic snapshot", that is, "manual snapshot" is executed first, and "automatic snapshot" is prohibited during the process of "manual snapshot", and "automatic snapshot" is resumed after "manual snapshot" is completed. Therefore, the execution of "automatic snapshot" will not affect or interfere with "manual snapshot". The corresponding solution strategy is shown below in combination with the accompanying drawings.

[0240] For example, as shown in (a) in FIG14 , after the electronic device receives the user's triggering operation to take a photo, it executes the processes of manual capture frame selection, thumbnail reporting, and large image callback, during which automatic capture is not allowed.

[0241] As another example, as shown in (b) in Figure 14, the time interval between automatic frame capture and manual frame capture should be greater than or equal to a preset first interval threshold, for example, the first interval threshold can be 2s. For example, when the time interval between automatic frame capture and manual frame capture is 3s, then due to the long distance, the automatic capture process and the manual capture process do not affect each other. Among them, manual capture will report thumbnails and large images to the Android application package (APK). Automatic capture reports large images to APK, but does not report thumbnails. For another example, when the time interval between automatic frame capture and manual frame capture is 1s, then due to the close distance, when manual capture is triggered, automatic capture will be canceled and the manual capture process will be executed first.

[0242] In an embodiment of the present application, the time interval between two consecutive automatic snapshots should be greater than or equal to a preset second interval threshold, for example, the second interval threshold can be 3 minutes. This can avoid frequent automatic snapshots, reduce load, and reduce power consumption. For example, as shown in (c) in Figure 14, a second automatic snapshot is not allowed within 3 minutes after an automatic snapshot. For example, in the process of shooting a user skipping rope, the user skipping rope is a repetitive action, and the captured preview image is also repetitive. Therefore, it is necessary to wait for a period of time before taking the next automatic snapshot to avoid frequent automatic snapshots.

[0243] It should be noted that since manual capture involves frame selection before taking a picture (such as 1 second) and frame selection after taking a picture (such as 0.5 seconds), and due to memory limitations and the uncertainty of the arrival of the manual capture time, this application optimizes the storage of frames in the entire frame selection time window. For ease of understanding, the following describes the process of manual capture frame selection and caching in detail by state.

[0244] Manual snapshot (state 0): image caching and frame selection process before taking a photo

[0245] Figure 15 shows a schematic diagram of the image buffering and frame selection process before a manual snapshot. As shown in Figure 15, within the 1-second window before the snapshot (i.e., the historical time period described above), the preview image frame stream focuses on the most recently acquired 30 frames, determining the LBM for each 10-frame period. For ease of understanding, the image captured at the time of the snapshot is labeled 0, the images acquired before the snapshot are labeled with negative numbers, and the images acquired after the snapshot are labeled with positive numbers.

[0246] In an embodiment of the present application, the electronic device may determine the LBM from multiple frames of image according to a preset frame selection principle. For example, the preset frame selection principle may be: the weighted average of the image quality score, the portrait score, and the action score (i.e., the comprehensive score) is greater than or equal to a preset overall threshold.

[0247] Exemplarily, as shown in FIG15 , the image frames are analyzed and compared by using a preset frame selection principle, and it is determined that the optimal local frame (first LBM) among the 10 frames within [-1s, -2 / 3s] is the -22th frame, the optimal local frame (second LBM) among the 10 frames within [-2 / 3s, -1 / 3s] is the -13th frame, and the optimal local frame (third LBM) among the 10 frames within [-1 / 3s, 0s] is the -7th frame.

[0248] Among them, the cache queue stores 4 frames of images (-22, -21, -20, -19) that have been filtered, another 4 frames of images (-13, -12, -11, -10) and the most recently collected 10 frames of images (-9, -8, -7, -6, -6, -4, -3, -2, -1, 0).

[0249] It should be noted that in an embodiment of the present application, after the user presses the shutter, the electronic device can select frames within 1s in its historical direction and 0.5s in the future direction. If all 1.5s of content is cached, then the electronic device needs to cache at least 1.5s×30fps=45 frames of images. By evaluating the local optimal LBM, the present application only needs to dynamically cache a small number of frames. As mentioned above, only 4+4+10=18 frames need to be cached to achieve backtracking in the historical direction. If multi-frame synthesis is not required for taking pictures, only 1+1+10=12 frames need to be cached. Therefore, the present application solution can save a lot of memory and reduce power consumption. The following is an illustrative explanation using the example of 18 frames of dynamic cache.

[0250] In addition, as shown in FIG15 , the cache queue also stores images for automatic capture, such as (-60, -59, -58, -57), wherein the -60th frame is the local optimal frame in the transition.

[0251] Manual snapshot (state 1): image caching and frame selection process when taking a photo

[0252] Figure 16 shows a schematic diagram of the image caching and frame selection process for manual snapshots at the moment of capture. As shown in Figure 16, upon receiving a user click on the capture button, the electronic device makes a decision based on all cached frames from the historical time period, determining the locally optimal frame LBM (-22) for that historical time period. The cache queue (-22, -21, -20, -19) is temporarily fixed, while the remaining positions are occupied by frames from the future time period, which are sequentially input. The cache queue not only caches LBMs from the historical time period but also caches frames from the future time period, for a total of 18 cached frames.

[0253] Manual capture (state 2): future direction image caching and frame selection process

[0254] Figure 17 shows a schematic diagram of the image caching and frame selection process after a manual snapshot. As shown in Figure 17, within the 0.5s time window after the snapshot is taken (i.e., the future time period described above), the preview image frame stream focuses on the 15 most recently captured frames, which are sequentially input into the cache queue. In some embodiments, each time a frame is input into the cache queue, the electronic device can compare the input frame with the LBM of the historical time period. If the LBM of the historical time period is better than the input frame, the LBM of the historical time period remains fixed. If the input frame is better than the LBM of the historical time period, the input frame is determined as the new local optimal frame and temporarily fixed, while the remaining position is occupied by frames from the future time period that are input sequentially. The newly input frame is then compared with the new local optimal frame, and so on, ultimately determining the optimal frame within the time window of 1s before the snapshot and the time window of 0.5s after the snapshot.

[0255] As shown in FIG17 , it is finally determined that the optimal frame within the time window of 1 s before taking the photo and the time window of 0.5 s after taking the photo is the second frame. In this case, the manually captured photo is generated by performing image fusion on the (2nd, 3rd, 4th, 5th) frames.

[0256] Manual snapshot (state 3): Continue to capture for a short period of time

[0257] FIG18 is a schematic diagram showing the process of continuing manual snapshots within a short period of time after a manual snapshot. As shown in FIG18 , in some cases, two manual snapshots may occur within a short period of time because the user continuously clicks the capture button within a short period of time. The previous manual snapshot can be selected based on approximately 45 frames of preview images at [-30, 15], and the current manual snapshot can be selected based on approximately 20 frames of preview images at [16, 35]. This is because the time interval between two consecutive manual snapshots is short, and the time window of this manual snapshot in the historical direction is actually less than 1 second. For example, as shown in FIG18 , the number of frames collected in the historical direction for this manual snapshot is 5. Among them, the time window of this manual snapshot in the future direction is not affected and remains 0.5 seconds. For example, as shown in FIG18 , the number of frames collected in the future direction for this manual snapshot is 15.

[0258] It should be noted that for scenes where manual capture continues within a short period of time, the historical direction selection and caching methods, as well as the future direction frame selection and caching methods, are similar to the processing described above for states 0 to 2. Specifically, in the historical direction, the preview image frame stream focuses on the five most recently captured frames (16, 17, 18, 19, 20), and the LBM for the historical time period is determined to be frame 17. Accordingly, the four frames (17, 18, 19, 20) are fixed in the cache queue. In the future direction, the captured images (21, 22, …, 35) are sequentially input into the cache queue. Each time a frame is input into the cache queue, the electronic device compares the input frame with the LBM for the historical time period. If the LBM for the historical time period is better than the input frame, the LBM for the historical time period remains fixed. If the input frame is better than the LBM for the historical time period, the input frame is determined as the new local optimal frame and is temporarily fixed. The remaining positions are occupied by frames from the future time period that are input sequentially. Then, the newly input frame image is compared with the new local optimal frame, and so on, and finally the optimal frame among about 20 preview frames [16, 35] is determined.

[0259] As shown in FIG18 , the optimal frame is finally determined to be the 22nd frame. In this case, a manually captured photo is generated by performing image fusion on the (22nd, 23rd, 24th, and 25th) frames.

[0260] The above describes the frame selection strategy and capture principle of the image capture method provided in the embodiment of the present application. Based on the above content, the implementation process of the image capture method provided in the embodiment of the present application is introduced below.

[0261] The image capture method provided in the embodiment of the present application includes two methods: automatic capture and manual capture. For automatic capture, during the preview process, the preview image is collected, compared frame by frame, the optimal frame is selected, and the optimal frame is cached. Once a scene change occurs, the photo is automatically triggered to obtain an automatically captured photo. For manual capture, when the user presses the shooting button (shutter), the optimal frame is selected from multiple image frames within 1s in the historical direction and 0.5s in the future direction of the photo as a wonderful captured photo.

[0262] Example 1

[0263] Figure 19 is a schematic diagram of a manual capture process of an image capture method provided in an embodiment of the present application. Referring to Figure 19 , the method includes the following steps S101-S105.

[0264] S101. In a photographing mode of a camera, a preview interface of the photographing mode is displayed, wherein a first icon of a manual snapshot function is displayed in the preview interface, and the first icon is in a first state.

[0265] S102: In response to a user operation on the first icon, enable a manual snapshot function and switch the first icon from a first state to a second state.

[0266] S103. In response to the user operating the shooting button at the first moment, determine the first image with the highest comprehensive score among the multiple frames of images in the first time period before the first moment and the multiple frames of images in the second time period after the first moment, where the comprehensive score includes the image quality score, the portrait score and the action score.

[0267] For example, when the user presses the shooting button (shutter), the best frame is selected from multiple image frames within 1 s in the historical direction and 0.5 s in the future direction of the photo as a wonderful snapshot.

[0268] In some embodiments, the "Highlight Snap" function is run during photo preview, performing real-time analysis and comparison of multiple preview frames to determine a comprehensive score for each frame, i.e., a "Highlight Score." The frame with the highest "Highlight Score" (i.e., the optimal frame) is then selected. In other embodiments, real-time analysis and comparison of multiple preview frames is performed to determine a "Highlight Ranking" for each frame based on the comprehensive score. The frame with the highest "Highlight Ranking" is then selected.

[0269] S104: Generate a first photo based on the first image.

[0270] S105: Store the first photo in the gallery.

[0271] Through the image capture method provided by the embodiment of the present application, after the wonderful capture function is turned on, the electronic device continuously and automatically captures multiple frames of images and analyzes the local optimal frames in the images, and caches the local optimal frames of the most recently captured images. After the user triggers the photo, the electronic device continues to capture multiple frames of images and analyzes the local optimal frames in the images. Then, the electronic device determines an optimal frame from the local optimal frames determined before the photo is taken and the local optimal frames determined after the photo is taken, as the wonderful captured photo. Since the wonderful captured photo that is finally output is the optimal frame in the time period from before to after the photo is taken, through the solution of the present application, in the capture scenario, regardless of whether the reaction time when taking the photo is delayed or advanced, the optimal image frame in the photo process can be captured, the best moment can be recorded, and user needs can be met to the greatest extent.

[0272] Example 2

[0273] Figure 20 is a schematic diagram of the automatic capture process of the image capture method provided in an embodiment of the present application. Referring to Figure 20 , the method includes the following steps S201-S205.

[0274] S201: When the automatic snapshot function is turned on, continuously capture preview images to obtain a second preview image frame stream.

[0275] S202: Identify, based on the second preview image frame stream, that the current shooting scene is a first scene.

[0276] S203 : Compare each preview image in the second preview image frame stream frame by frame to determine a fourth image with the highest comprehensive score, where the comprehensive score includes an image quality score, a portrait score, and a motion score.

[0277] S204: When it is recognized according to the second preview image frame stream that the current shooting scene changes to a second scene, generate a second photo according to the fourth image.

[0278] S205: Store the second photo in the gallery, completing an automatic capture.

[0279] Through the image capture method provided in the embodiment of the present application, when the automatic capture function is turned on, during the preview process, the preview image is captured, compared frame by frame, the optimal frame is selected, and the optimal frame is cached. Once a scene change occurs, the photo is automatically triggered to obtain an automatically captured photo.

[0280] It should be noted that the electronic device may execute steps S101-S105 individually; may also execute steps S201-S205 individually; or may execute steps S101-S105 first and then execute steps S201-S205.

[0281] Figure 21 shows a schematic diagram of the relationship between scene recognition, split-screen moment detection, and wonderful moment recognition in the image capture method provided by an embodiment of the present application. As shown in Figure 21, the image capture method includes three parts:

[0282] 1) Scene recognition: Obtain image features. Through image feature analysis, the scene content can be determined and the current shooting scene can be identified.

[0283] For example, the shooting scene may be a person scene, a pet scene, a landscape scene, an action scene, etc. It should be noted that the classification of the shooting scene can be determined according to actual use requirements and is not limited in the embodiments of the present application.

[0284] 2) Detection of split-screen moments: Determine the timing of automatic capture through various methods such as transition detection and human body detection.

[0285] For example, assuming the current scene is a person scene, the electronic device continuously captures multiple preview frames and performs frame-by-frame image analysis and comparison to determine the local optimal frame in the person scene and cache this local optimal frame. When the electronic device detects a transition from a person scene to a pet scene, the electronic device automatically captures the image, generating and outputting an automatically captured photo based on the cached local optimal frame in the person scene.

[0286] For another example, assuming the current scene is a human scene and the subject is Person 1, the electronic device continuously captures multiple preview frames and performs frame-by-frame image analysis and comparison to determine the local optimal frame in the human scene and cache this local optimal frame. When the electronic device detects that the subject of the human scene changes from Person 1 to Person 2, the electronic device automatically captures the image, generating and outputting an automatically captured photo containing Person 1 based on the cached local optimal frame in the human scene.

[0287] 3) Wonderful moment recognition: Perform image analysis based on a preset image algorithm to identify pictures of wonderful moments.

[0288] For example, as shown in FIG21 , the influencing factors of the image algorithm include at least one of the following: image clarity, facial expression, action evaluation, facial composition, facial angle, overall composition, symmetry, eye opening and closing, etc. Each influencing factor corresponds to a score value. The greater the degree of influence, the greater the score value. The electronic device can perform image analysis based on the various influencing factors in the image algorithm, calculate an overall image score (also known as a wonderful score), and determine the photo with the highest overall image score as a wonderful moment.

[0289] It should be noted that the influencing factors of the image algorithm are described here for illustrative purposes only. It is understandable that in actual implementation, the influencing factors of the image algorithm may also be other possible situations, such as whether the image is shaking, whether the object in the image is centered, etc.

[0290] In some embodiments, for different scenes, the influencing factors of the image algorithm used are different.

[0291] For example, for scenes with people, the factors affecting the image algorithm may include image clarity, facial expression, facial composition, facial angle, and whether the eyes are open or closed. For another example, for scenes with actions, the factors affecting the image algorithm may include action evaluation, image clarity, facial expression, facial composition, and facial angle.

[0292] In some embodiments, for each scene, a weighted calculation can be performed based on the influencing factors of the image algorithm to obtain a comprehensive image score.

[0293] For example, for human scenes, the weighting coefficients for image clarity, facial expression, facial composition, facial angle, and eye opening and closing are 0.2, 0.5, 0.1, 0.1, and 0.2, respectively. For another example, for action scenes, the weighting coefficients for action evaluation, image clarity, facial expression, facial composition, and facial angle are 0.4, 0.2, 0.2, 0.1, and 0.1, respectively.

[0294] It should be noted that the weighting coefficient values ​​are listed above by way of example. It can be understood that the weighting coefficient values ​​of different factors in different scenarios can be determined based on actual usage requirements, and the embodiments of this application do not limit them.

[0295] Based on the content shown in Figure 21, Figure 22 shows an exemplary flow chart of the image capture method provided by an embodiment of the present application. Among them, the collected preview image is input into the cache queue, and after image analysis, the local optimal frame LBM is determined. Automatic capture is triggered at the time of transition, or manual capture is triggered when the user clicks the shutter. Image processing is then performed, that is, image fusion is performed based on LBM, a photo is generated, and then the photo is output. It should be noted that for automatic capture, during the preview process, the preview image is collected, compared frame by frame, the optimal frame is selected, and the optimal frame is cached. Once a scene change occurs, the photo is automatically triggered to obtain an automatically captured photo. For manual capture, after the user presses the shooting button (shutter), the optimal frame is selected from multiple image frames within 1s in the historical direction of the photo and 0.5s in the future direction as a wonderful captured photo.

[0296] In some embodiments, the electronic device supports the user to manually trigger the "wonderful capture" function. Figure 23 shows a flow chart of the electronic device turning on the wonderful capture function.

[0297] S301, the electronic device turns on the AI ​​beauty shooting function of the camera.

[0298] For example, after the camera is turned on, it enters the AI ​​beauty shooting state of the photo shooting mode by default. In the AI ​​beauty shooting state, the switch icon of the "wonderful capture" function is displayed in the photo shooting function bar, and the switch icon of the "wonderful capture" function is in the off state.

[0299] S302: The electronic device receives a click operation by the user on a switch icon of a wonderful capture function.

[0300] S303, in response to the click operation, the electronic device turns off the AI ​​beauty shot function and turns on the wonderful snapshot function.

[0301] For example, as shown in (a) of Figure 24, when the "AI Beauty Shot" function is turned on and the "Wonderful Snapshot" function is turned off, the user can click the switch icon of the "Wonderful Snapshot" function to trigger the electronic device to turn on the "Wonderful Snapshot" function. As shown in (b) of Figure 24, in response to the user clicking the switch icon of the "Wonderful Snapshot" function, the electronic device turns on the "Wonderful Snapshot" function and turns off the "AI Beauty Shot" function. Thus, after the user triggers the "Wonderful Snapshot" function, the electronic device switches from the AI ​​Beauty Shot state to the Wonderful Snapshot state.

[0302] In other embodiments, the electronic device may also recommend to the user to enable the "Highlight Snap" function when it determines that preset conditions are met. The preset conditions include: the brightness of the preview image is greater than a brightness threshold, there is a preset subject (e.g., a person or animal) in the preview image, and the motion score of the preset subject is greater than a motion score threshold.

[0303] FIG25 shows a flowchart of recommending activation of the wonderful capture function when the electronic device is in the AI ​​beauty capture state.

[0304] S401: The electronic device has enabled the AI ​​beauty shooting function of the camera.

[0305] When the AI ​​Beauty Shot function is turned on, that is, when the camera is in the AI ​​Beauty Shot state, the electronic device will capture a preview image and determine the brightness of the preview image.

[0306] S402: The electronic device determines whether the brightness of the preview image is greater than a brightness threshold.

[0307] If the brightness of the preview image is greater than the brightness threshold, then after entering the wonderful capture state, the electronic device can achieve good motion exposure reduction and good image quality when capturing the captured photo, ensuring that the captured photo has a good display effect. The value of the brightness threshold is determined according to actual usage requirements and is not limited in this embodiment of the application.

[0308] If the brightness of the preview image is greater than the brightness threshold, the following S402 is executed. If the brightness of the preview image is less than or equal to the brightness threshold, it indicates that the shooting environment is dim and the wonderful capture function is not recommended, and the AI ​​beauty shot function is maintained.

[0309] S403: The electronic device determines whether there is a preset subject in the preview image, and whether the action score of the preset subject is greater than a motion score threshold.

[0310] For example, the preset subject may be a person, and the action score of the preset subject may refer to the score given to the person when jumping upwards, running, hurdling, shooting a basketball, or shooting a goal.

[0311] As another example, the preset subject may be an animal such as a cat or a dog, and the action score of the preset subject may refer to the score given to the cat or dog when it jumps upward or runs.

[0312] In the embodiment of the present application, based on the current preview perception algorithm capabilities, based on the reuse algorithm and the premise of simplifying the judgment criteria as much as possible, the design scheme provided by the present application is as follows:

[0313] When a human body appears in the preview image and the human body area is greater than the human body area threshold, and the action score of the human body in the preview image is greater than the action score threshold, the electronic device recommends entering the wonderful capture state.

[0314] When a pet (cat or dog) appears in the preview image and the pet area is greater than the pet area threshold, and the action score of the pet in the preview image is greater than the action score threshold, the electronic device recommends entering the wonderful capture state.

[0315] If the preview image contains a preset subject and its motion score is greater than the motion score threshold, the electronic device proceeds to step S404 below. If the preview image does not contain a preset subject, or if the preview image contains a preset subject and its motion score is less than or equal to the motion score threshold, the electronic device continues to use the AI ​​Beauty Shot function.

[0316] S404, it is recommended that the electronic device enable the wonderful capture function.

[0317] S405 , in response to the user's operation of determining to turn on the wonderful capture function, the electronic device turns off the AI ​​beauty capture function and turns on the wonderful capture function.

[0318] When the camera is in AI beauty shooting state, if the preview image meets the preset conditions, the electronic device will recommend the user to use "wonderful capture" and enter the wonderful capture state after the user clicks to confirm.

[0319] Exemplarily, as shown in (a) of Figure 26, when the "AI Meipai" function is turned on and the "Wonderful Snap" function is turned off, the electronic device detects whether the preview image meets the preset conditions. As shown in (b) of Figure 26, when the electronic device detects that the preview image meets the preset conditions, the electronic device pops up a prompt box near the switch icon of the wonderful snapshot, recommending to the user to turn on the "Wonderful Snap" function. As shown in (c) of Figure 26, if the user confirms to turn on the "Wonderful Snap" function, the user can click the switch icon of the wonderful snapshot. As shown in (d) of Figure 26, when the electronic device receives the user's click operation, the electronic device turns on the "Wonderful Snap" function and turns off the "AI Meipai" function. Thus, the electronic device switches from the AI ​​Meipai state to the wonderful snapshot state.

[0320] It should be noted that the prompt box can remain displayed for a preset time period (such as 5 seconds). If no user operation is detected within the preset time period, the electronic device hides the prompt box.

[0321] In addition, to avoid jumps and conflicts, Wonderful Snapshot and AI Beauty Snapshot are mutually exclusive within a period of time (such as 5 seconds) after the recommendation of Wonderful Snapshot starts. That is, after Wonderful Snapshot is turned on, AI Beauty Snapshot will automatically be turned off.

[0322] In some embodiments, if the focal length of the current scene (eg, 10x) exceeds a certain zoom range (eg, [0.5x, 5x]), the electronic device will not recommend a "wonderful shot."

[0323] It should be noted that, in the embodiments of the present application, "greater than" can be replaced by "greater than or equal to", "less than or equal to" can be replaced by "less than", or "greater than or equal to" can be replaced by "greater than", and "less than" can be replaced by "less than or equal to".

[0324] The various embodiments described herein may be independent solutions or may be combined according to internal logic, and all of these solutions fall within the scope of protection of this application.

[0325] The above mainly describes the solution provided by the embodiment of the present application from the perspective of method steps. It is understandable that, in order to achieve the above functions, the electronic device implementing the method includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of protection of this application.

[0326] The present application also provides a chip, which is coupled to a memory and is used to read and execute computer programs or instructions stored in the memory to perform the methods in the above embodiments.

[0327] The present application also provides an electronic device, which includes a chip, and the chip is used to read and execute computer programs or instructions stored in a memory, so that the methods in each embodiment are executed.

[0328] This embodiment further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image capture method in the above-mentioned embodiment.

[0329] This embodiment further provides a computer program product, wherein the computer-readable storage medium stores program code. When the computer program product runs on a computer, the computer executes the above-mentioned related steps to implement the image capture method in the above-mentioned embodiment.

[0330] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the image capture method in the above-mentioned method embodiments.

[0331] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0332] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0333] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image capture method, characterized in that: include: In a camera mode, displaying a preview interface of the camera mode, wherein a first icon is displayed in the preview interface, and the first icon is in a first state; In response to a user operation on the first icon, starting a first function and switching the first icon from the first state to a second state; In response to the user operating the capture button at the first moment, the following steps are performed: Determine a first image from a plurality of image frames in a first time period before the first moment and a plurality of image frames in a second time period after the first moment; generating a first photo according to the first image; The first photo is stored in a gallery.

2. The method according to claim 1, characterized in that The first image is an image with the highest comprehensive score among multiple frame images in a first time period before the first moment and multiple frame images in a second time period after the first moment; wherein the comprehensive score includes an image quality score, a portrait score and an action score.

3. The method according to claim 1 or 2, characterized in that: After enabling the first function, the method further includes: Continuously collecting preview images to obtain a first preview image frame stream; For each of the M preview image frames in the first preview image frame stream, determine a local optimal image, and sequentially input the local optimal image determined each time into the first cache queue, wherein the local optimal image is an image with the highest comprehensive score among the M preview image frames; The first cache queue is a first-in-first-out queue, and the first cache queue caches the most recently determined S-frame local optimal images and the most recently acquired N-frame images, where N is less than or equal to M; The total number of frames of the multiple frames of images in the first time period is S×M+N.

4. The method according to claim 3, characterized in that Before determining the first image with the highest comprehensive score, the method further includes: Determine the second image with the highest comprehensive score among the S frames of local optimal images and the N frames of images; The second image is fixed in the first cache queue.

5. The method according to claim 4, characterized in that Before determining the first image with the highest comprehensive score, the method further includes: Continuously acquiring T frames of preview images within the second time period after the first moment; Inputting the T-frame preview images into the first cache queue in sequence, and replacing the images in the first cache queue except the second image with the T-frame preview images; The total number of frames of the multiple frames of images in the second time period is T.

6. The method according to claim 5, characterized in that The determining of the first image with the highest comprehensive score comprises: The comprehensive score of the second image is compared with the comprehensive score of each preview image in the T frame preview images, and the image with the highest comprehensive score is determined as the first image.

7. The method according to any one of claims 1 to 6, characterized in that The step of generating a first photo according to the first image includes: Performing image fusion on the first image and the third image to obtain the first photo; The third image includes at least one image captured immediately after the first image and / or at least one image captured immediately before the first image.

8. The method according to any one of claims 1 to 7, characterized in that The first time period is 1 second or less than 1 second, and the second time period is 0.5 seconds.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: When the first icon is in the second state, receiving an operation of the first icon by a user; In response to a user operation on the first icon, the first function is turned off, and the first icon is switched from the second state to the first state.

10. The method according to any one of claims 1 to 9, characterized in that The photographing mode further includes a second function, and a switch button of the second function is displayed in the setting interface of the photographing mode, and the switch button of the second function is set to an off state by default; When the switch button of the second function is in the off state, the second function is in the off state.

11. The method according to claim 10, characterized in that The method further comprises: When the second function is turned on, the preview image is continuously captured to obtain a second preview image frame stream; identifying, according to the second preview image frame stream, that the current shooting scene is a first scene; Comparing each preview image in the second preview image frame stream frame by frame to determine a fourth image with the highest comprehensive score; When it is recognized according to the second preview image frame stream that the current shooting scene changes to a second scene, generating a second photo according to the fourth image; The second photo is stored in the gallery.

12. The method according to claim 11, characterized in that After storing the second photo in the gallery, the method further includes: In response to a user operation, jumping from the preview interface to the gallery interface; The fourth image and a second icon are displayed in the gallery interface, and the second icon is used to indicate that the fourth image is an image obtained by automatic capture.

13. The method according to claim 11 or 12, characterized in that: The step of generating a second photo according to the fourth image includes: fusing the fourth image with the fifth image to obtain the second photo; The fifth image includes at least one image acquired immediately after the fourth image and / or at least one image acquired immediately before the fourth image.

14. The method according to any one of claims 9 to 13, characterized in that The method further comprises: When the first icon is in the first state, receiving an operation of the first icon by a user; In response to the user's operation on the first icon, a pop-up box displays first information, where the first information is used to prompt whether to enable the second function; In response to the user confirming the operation of turning on the second function, the first function and the second function are turned on at the same time, the first icon is switched from the first state to the second state, and the switch button of the second function is switched from the off state to the on state.

15. The method according to any one of claims 10 to 14, characterized in that The method further comprises: In response to the user's operation in the photo setting interface, the second function is turned on or off.

16. The method according to any one of claims 10 to 15, characterized in that When the second function is enabled, the method further includes: If the first function is turned off, the second function is disabled; If the first function is restarted, the disabling of the second function is released.

17. The method according to any one of claims 12 to 16, characterized in that The method further comprises: When the second function is turned on, after completing one automatic snapshot, a first preset time interval is maintained before starting the next automatic snapshot.

18. The method according to any one of claims 10 to 17, characterized in that The method further comprises: When both the first function and the second function are turned on, the execution priority of the first function is higher than the execution priority of the second function, and the execution of the second function is stopped during the execution of the first function.

19. The method according to claim 2 or 12, characterized in that: The comprehensive score is a weighted average of the image quality score, the portrait score, and the action score; The image quality score, the portrait score and the action score are respectively preset with corresponding weighting coefficients.

20. The method according to claim 2 or 12, characterized in that The method further comprises: According to the image features of the preview image, the current scene is identified as a first scene type; wherein, for different scene types, the image quality score, the portrait score, and the action score are respectively preset with different score weighting coefficients; Finding a score weighting coefficient corresponding to the first scene type; According to the score weighting coefficient corresponding to the first scene type, a weighted average operation is performed on the image quality score, the portrait score and the action score to obtain the comprehensive score.

21. The method according to any one of claims 1 to 20, characterized in that The method further comprises: When the AI ​​beauty shooting function is turned on and the first function is turned off, collecting a third preview image frame stream; If the brightness of the third preview image frame stream is greater than the brightness threshold, and the third preview image frame stream has a preset subject, and the action score of the preset subject is greater than the action score threshold, then the first recommendation information is displayed, and the first recommendation information is used to recommend to the user to enable the first function.

22. The method according to claim 21, characterized in that After displaying the first recommendation information, the method further includes: In response to the user confirming the operation of turning on the first function, turning off the AI ​​beauty shooting function and turning on the first function.

23. An image capture method, characterized in that: include: Continuously collecting preview images to obtain a second preview image frame stream; identifying, according to the second preview image frame stream, that the current shooting scene is a first scene; Comparing each preview image in the second preview image frame stream frame by frame to determine a fourth image with the highest comprehensive score, wherein the comprehensive score includes an image quality score, a portrait score, and an action score; When it is recognized according to the second preview image frame stream that the current shooting scene changes to a second scene, generating a second photo according to the fourth image; The second photo is stored in the gallery, completing an automatic capture.

24. The method according to claim 23, characterized in that The method further comprises: After completing one automatic capture, wait for a first preset time and then start the next automatic capture.

25. The method according to claim 23 or 24, characterized in that The step of generating a second photo according to the fourth image includes: fusing the fourth image with the fifth image to obtain the second photo; The fifth image includes at least one image acquired immediately after the fourth image and / or at least one image acquired immediately before the fourth image.

26. The method according to any one of claims 23 to 25, characterized in that After storing the second photo in the gallery, the method further includes: In response to a user operation, jumping from the preview interface to the gallery interface; The fourth image and a second icon are displayed in the gallery interface, and the second icon is used to indicate that the fourth image is an image obtained by automatic capture.

27. The method according to any one of claims 23 to 26, characterized in that Before continuously acquiring the preview image to obtain the second preview image frame stream, the method further includes: In response to the user's operation, the second function is activated.

28. An electronic device, characterized in that: The electronic device comprises a processor, a memory and a computer program stored in the memory, wherein the processor is configured to execute the computer program so that the electronic device implements the method according to any one of claims 1 to 22 or the method according to any one of claims 23 to 27.

29. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 22, or the method according to any one of claims 23 to 27.

30. A computer program product, characterized in that The computer program product comprises a computer program code, and when the computer program code is executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 22, or the method according to any one of claims 23 to 27.

Images