Photographing method and electronic device

Abstract

Provided in the present application are a photographing method and an electronic device. A reference image and a portrait of a user can be superimposed and displayed in a photographing preview interface of the electronic device, the reference image may be an image of an incidental scene that has been captured, and the user may make adaptive adjustments on the basis of a combined effect of the portrait of the user and the reference image that are displayed in the photographing preview interface, thereby capturing a satisfactory combined photo. The photographing process is not affected by factors such as time and space, such that the user has more sufficient adjustment time and better photographing conditions. By means of the photographing method and the electronic device, the efficiency of capturing a combined photo of a user and an incidental scene is higher, the quality of the combined photo is higher, and the photographing experience of the user is better.

Description

Shooting method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on February 8, 2024, with application number 202410176953.7 and invention name “Photographing method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal device software, and in particular, to a shooting method and electronic device. Background Art

[0003] Natural scenes such as meteors, lightning, and icebergs breaking off ice shelves are sporadic and typically disappear within a short period of time, making them difficult to capture and preserve with electronic devices like mobile phones and cameras. This makes it even more difficult for users to capture these sporadic natural scenes simultaneously. On the one hand, users struggle to determine the timing of the natural scene, making it difficult to capture the scene in time. Furthermore, this uncertainty makes it difficult for users to maintain their facial expressions, posture, or movements for extended periods of time.

[0004] How to improve the efficiency of electronic devices in coordinating with users in sporadic scenarios is an issue worth considering.

[0005] Summary of the Invention

[0006] The present application provides a shooting method, in which an electronic device can display on a shooting preview interface a superimposed effect of a shooting object in an image acquired in real time by a camera and an image of an incidental scene that has been saved on the electronic device. The user can adaptively adjust actions, postures, or expressions according to the superimposed effect, and the shooting with the incidental scene can be unrestricted by time or space. The shooting method is conducive to improving the efficiency of the electronic device in shooting incidental scenes with the user.

[0007] In a first aspect, a shooting method is provided, which is applied to an electronic device, including: displaying a shooting preview interface, the shooting preview interface including a reference image and an image of a photographed object, the reference image and the image of the photographed object being displayed superimposed, the reference image being stored locally in the electronic device, and the image of the photographed object including a portion of an image acquired in real time by a camera of the electronic device; in response to a shooting operation, saving a target image, the target image including the image displayed on the shooting preview interface.

[0008] In some scenarios, the shooting preview interface can also be called the user interface of the camera application.

[0009] In a possible implementation, the photographed subject may be a photographed user.

[0010] In a possible implementation, all or part of the image elements of the reference image may serve as the background of the target image, and the photographed object may serve as the foreground of the target image.

[0011] In a possible implementation, the reference image may be an image of an occasional scene, and the reference image may be stored in a storage medium of the electronic device.

[0012] The electronic device can simultaneously display a reference image and an image of the subject being photographed on a shooting preview interface. The user can adjust the subject being photographed according to the content displayed on the shooting preview interface. For example, if the subject being photographed is the user, the user can adjust actions, postures, or expressions according to the content displayed on the shooting preview interface to better achieve the effect of coordinating with the reference image. The implementation of this shooting method does not need to rely on a specific time, place, or event. The implementation of this technical solution is conducive to improving the success rate or efficiency of electronic devices in coordinating shooting with users in occasional scenes, reducing the waste of storage resources and energy of electronic devices due to invalid shooting, and improving the user's shooting experience.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the reference image is displayed with a first transparency, and a value of the first transparency is less than or equal to a first threshold.

[0014] In one possible implementation, the electronic device may also display the reference image through a floating window or a split-screen window on the shooting preview interface, or the electronic device may also display the superimposed display effect of the reference image and the photographed object through a floating window or a split-screen window on the shooting preview interface.

[0015] It should be noted that the value of the first transparency is less than or equal to the first threshold, which can also be understood as the reference image being displayed in a manner that tends to be visible on the shooting preview interface.

[0016] Compared with the floating window or split-screen window display method, displaying the reference image on the shooting preview interface through transparency has a larger area of ​​the reference image displayed on the shooting preview interface, which is conducive to more clearly showing the superimposed display effect of the reference image and the photographed object, and is conducive to improving the success rate or efficiency of electronic devices in shooting occasional scenes with users, and is conducive to improving the display effect of the target image, which is conducive to enhancing the user's shooting experience.

[0017] In combination with the first aspect, in certain implementations of the first aspect, before saving the target image in response to a shooting operation, the method further includes: determining the shooting parameters of the target image based on the shooting parameters of the reference image, the shooting parameters including one or more of the following: metering mode, sensitivity, aperture, focal length, shutter or exposure compensation.

[0018] In a possible implementation, the target image and the reference image may be captured using the same camera of the electronic device.

[0019] In a possible implementation, a difference between a shooting parameter of the target image and a shooting parameter of the reference image is less than or equal to a first threshold.

[0020] Using similar shooting parameters to shoot the reference image and the target image is conducive to making the superposition effect of the photographed object and the reference image more coordinated, and is conducive to improving the shooting quality of the target image.

[0021] In combination with the first aspect, in certain implementations of the first aspect, before saving the target image in response to a shooting operation, the method also includes: displaying first shooting prompt information, the first shooting prompt information being determined based on shooting parameters of the reference image, the shooting parameters including one or more of the following: metering mode, sensitivity, aperture, focal length, shutter, or exposure compensation.

[0022] In combination with the first aspect, in certain implementations of the first aspect, before saving the target image in response to a shooting operation, the method also includes: displaying second shooting prompt information, the second shooting prompt information being determined based on scene information of the reference image, the scene information including one or more of the following: light intensity, light angle, or light color.

[0023] When a user is shooting a target image, the electronic device can provide shooting prompts based on the shooting information of the reference image to guide the user to take a better reshoot. The implementation of this technical solution is conducive to improving the effect of the electronic device's reshoot, reducing the waste of storage resources and energy of the electronic device due to invalid shooting, and improving the user's shooting experience.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the reference image includes the target object and the motion path of the target object, the motion path of the target object is determined based on multiple first images, the first image includes the target object, and the multiple first images are used to indicate the position of the target object at different times.

[0025] In some scenarios, the reference image may be an image of an incidental scene, such as a meteor falling from the sky, lightning, an iceberg breaking off an ice shelf, or a dolphin jumping out of the water. In these incidental scenes, the target object may be a meteor, lightning, an iceberg, or a dolphin.

[0026] In a possible implementation, different first images correspond to different shooting times, and the positions of the target objects in different first images are different. In other words, multiple first images can be used to record the position information of the target object at different times.

[0027] In some scenarios, the first image may also be referred to as an image corresponding to the photo stream of the electronic device.

[0028] It should be noted that the first image can also be understood as the data / information of the first image. In other words, the motion path of the target object is determined based on multiple first images. It can also be understood as: the motion path of the target object is determined based on the correspondence between the position and time of the target object contained in multiple first images.

[0029] When a reference image includes an object's motion path, generating the reference image by capturing multiple first images of the target object at different times facilitates synthesizing a clearer reference image and improving the harmony between the reference image and the photographed object. Furthermore, generating a reference image including the motion path by processing multiple first images by the electronic device improves the success rate of reference image generation.

[0030] In combination with the first aspect, in certain implementations of the first aspect, before displaying the shooting preview interface, the shooting method also includes: shooting multiple first images in response to a target event; wherein the target event includes one or more of the following: the brightness change value of the first image is greater than or equal to the brightness threshold; the moving distance of the target object is greater than or equal to the distance threshold; or, the user's photo-taking operation.

[0031] In a possible implementation, the brightness threshold and the distance threshold may be set by a user, or the brightness threshold and the distance threshold may be determined by the electronic device according to a shooting scene.

[0032] Electronic devices can set multiple trigger conditions for automatic shooting, and when the trigger conditions are met, the electronic device automatically executes the shooting operation. The implementation of this technical solution helps to reduce the difficulty of electronic devices in capturing occasional scenes and helps to reduce the waste of electronic devices' energy or storage resources caused by long periods of ineffective shooting.

[0033] In combination with the first aspect, in certain implementations of the first aspect, before capturing multiple first images in response to a target event, the method further includes: caching multiple second images, where the second images are images included in a preview stream of the electronic device, and the second images include the target object.

[0034] In combination with the first aspect, in some implementations of the first aspect, the first image is an image included in a photo stream of the electronic device, and a resolution of the second image is smaller than a resolution of the first image.

[0035] In some scenarios, the second image is used to determine whether the target event has occurred. A resolution of the second image smaller than that of the first image is beneficial for improving the efficiency of the electronic device in real-time displaying the light signal acquired by the camera on the shooting preview interface.

[0036] It should be noted that the electronic device caching multiple second images can be understood as the data of the second images being stored in the memory of the electronic device rather than on a storage medium such as a disk. One possible scenario is that the data of the second images cached by the electronic device is data that cannot be viewed by the user who took the target image, but can be used by the electronic device.

[0037] In this technical solution, the image data corresponding to the preview stream is used to determine whether an automatic shooting operation needs to be performed. On the one hand, this helps to reduce the ineffective occupation of the storage resources of the electronic device during the shooting process. On the other hand, since the data volume of the image corresponding to the preview stream is small, the electronic device can process this type of data more quickly, which helps to trigger the automatic shooting operation more timely.

[0038] In combination with the first aspect, in some implementations of the first aspect, the second image is further used to determine a background of the reference image.

[0039] In some scenarios, the second image used to determine whether the target event occurs may also be referred to as online preview stream data.

[0040] The prior preview stream data is obtained before the photo-taking operation is triggered. Therefore, the prior preview stream data generally does not include the motion path of the target object. Therefore, the prior preview stream data is more suitable for obtaining the background data of the target image. In other words, using the prior preview stream data can obtain a purer background of the target image.

[0041] In combination with the first aspect, in certain implementations of the first aspect, before displaying the shooting preview interface, the method further includes: determining a third image from the second image in response to a user selection; and generating the reference image based on the first image and the third image.

[0042] In combination with the first aspect, in certain implementations of the first aspect, the motion path of the target object is determined based on multiple first images and multiple second images.

[0043] The image data corresponding to the preview stream also contains information about the correspondence between the position and time of the target object. Using the first image and the second image to determine the motion path of the target object is conducive to obtaining a more accurate and clearer motion path of the target object, which is conducive to improving the shooting quality of the reference image.

[0044] In combination with the first aspect, in some implementations of the first aspect, the first image is used to determine a mask, and the mask is used to segment the target object from the first image.

[0045] It should be noted that the segmentation here can also be understood as separation, extraction, etc. In addition, the mask can also be used to segment the target object from the second image.

[0046] The resolution of the images corresponding to the photo stream is higher, and the resolution of the mask used to determine the segmentation of the moving object is higher, which is beneficial to improving the segmentation effect of the mask on the first image and improving the shooting quality of the reference image.

[0047] In combination with the first aspect, in some implementations of the first aspect, the mask is determined based on a deep learning network trained through self-supervision using adversarial loss.

[0048] The mask determined by the deep learning network based on adversarial loss self-supervised training is more accurate. Using these images to determine the mask for segmenting moving objects is beneficial to improving the segmentation effect of the mask for the first image and improving the shooting quality of the reference image.

[0049] In combination with the first aspect, in certain implementations of the first aspect, before saving the target image in response to a shooting operation, the method also includes: adjusting the display mode of the motion path in response to an operation acting on the motion path, the display mode including one or more of the following: color, position, or depth of field.

[0050] In the process of completing the capture of the target image, the electronic device can provide a variety of adjustable options for the user to adjust so that the user can obtain a satisfactory re-shooting effect. The implementation of this technical solution is conducive to improving the electronic device's ability to coordinate with the user and occasional scenes, and improving the utilization efficiency of the electronic device's storage resources and energy.

[0051] For the relevant explanations and descriptions of the beneficial effects in the following technical solutions, please refer to the relevant content in the first aspect in the previous text, and will not be repeated below.

[0052] In a second aspect, a shooting method is provided, which is applied to an electronic device, including: caching a second image, where the second image is an image included in a preview stream of the electronic device, and the second image includes a target object; in response to a target event, shooting a first image, where the first image includes the target object; and generating a reference image based on the first image and the second image, where the reference image includes the target object and a motion path of the target object, where the motion path of the target object is determined based on the first image and / or the second image.

[0053] In combination with the second aspect, in certain implementations of the second aspect, the target event includes one or more of the following: a brightness change value of the first image is greater than or equal to a brightness threshold; a moving distance of the target object is greater than or equal to a distance threshold; or a user's photo-taking operation.

[0054] In combination with the second aspect, in some implementations of the second aspect, generating a reference image based on the first image and the second image includes: determining a background of the reference image according to the second image.

[0055] In combination with the second aspect, in certain implementations of the second aspect, generating a reference image based on the first image and the second image includes: determining a third image from the second image in response to a user selection; and generating the reference image based on the first image and the third image.

[0056] In combination with the second aspect, in certain implementations of the second aspect, a reference image is generated based on the first image and the third image, including: determining a mask for segmenting the target object based on the first image and a deep learning network trained based on adversarial loss self-supervision; determining multiple fourth images based on the mask and the first image and the third image, or determining multiple fourth images based on the mask and the third image, the fourth images being used to indicate the position of the target object at different times; and determining the motion path of the target object based on the multiple fourth images.

[0057] According to a third aspect, a shooting device is provided, which includes a processing module, and the processing module is used to: display a shooting preview interface, the shooting preview interface includes a reference image and an image of a photographed object, the reference image and the image of the photographed object are superimposed and displayed, the reference image is stored locally in the electronic device, and the image of the photographed object includes a part of an image acquired in real time by the camera of the electronic device; in response to a shooting operation, save a target image, and the target image includes the image displayed on the shooting preview interface.

[0058] In combination with the third aspect, in certain implementations of the third aspect, the processing module is specifically configured to: display the reference image with a first transparency, where a value of the first transparency is less than or equal to a first threshold.

[0059] In combination with the third aspect, in certain implementations of the third aspect, before saving the target image in response to a shooting operation, the processing module is also used to: determine the shooting parameters of the target image based on the shooting parameters of the reference image, and the shooting parameters include one or more of the following: metering mode, sensitivity, aperture, focal length, shutter or exposure compensation.

[0060] In combination with the third aspect, in certain implementations of the third aspect, the processing module is also used to: in response to a shooting operation, before saving the target image, display a first shooting prompt information, the first shooting prompt information being determined based on the shooting parameters of the reference image, and the shooting parameters including one or more of the following: metering mode, sensitivity, aperture, focal length, shutter or exposure compensation.

[0061] In combination with the third aspect, in certain implementations of the third aspect, the processing module is also used to: display second shooting prompt information, the second shooting prompt information is determined based on the scene information of the reference image, and the scene information includes one or more of the following: light intensity, light angle or light color.

[0062] In combination with the third aspect, in certain implementations of the third aspect, the reference image includes the target object and the motion path of the target object, the motion path of the target object is determined based on multiple first images, the first image includes the target object, and the multiple first images are used to indicate the position of the target object at different times.

[0063] In combination with the third aspect, in certain implementations of the third aspect, the processing module is further used to: before displaying the shooting preview interface, in response to a target event, capture multiple first images; wherein the target event includes one or more of the following: the brightness change value of the first image is greater than or equal to the brightness threshold; the moving distance of the target object is greater than or equal to the distance threshold; or, the user's photo-taking operation.

[0064] In combination with the third aspect, in certain implementations of the third aspect, the processing module is also used to: cache multiple second images before taking multiple first images in response to a target event, where the second images are images included in the preview stream of the electronic device, and the second images include the target object.

[0065] In combination with the third aspect, in certain implementations of the third aspect, the first image is an image included in a photo stream of the electronic device, and a resolution of the second image is smaller than a resolution of the first image.

[0066] In combination with the third aspect, in certain implementations of the third aspect, the second image is further used to determine the background of the reference image.

[0067] In combination with the third aspect, in certain implementations of the third aspect, the processing module is further used to: before displaying the shooting preview interface, in response to a user selection, determine a third image from the second image; and generate the reference image based on the first image and the third image.

[0068] In combination with the third aspect, in certain implementations of the third aspect, the motion path of the target object is determined based on multiple first images and multiple second images.

[0069] In combination with the third aspect, in some implementations of the third aspect, the first image is used to determine a mask, and the mask is used to segment the target object from the first image.

[0070] In combination with the third aspect, in certain implementations of the third aspect, the mask is determined based on a deep learning network trained through adversarial loss self-supervision.

[0071] In combination with the third aspect, in certain implementations of the third aspect, the processing module is also used to: before saving the target image in response to a shooting operation, adjust the display mode of the motion path in response to an operation acting on the motion path, and the display mode includes one or more of the following: color, position or depth of field.

[0072] In a fourth aspect, a shooting device is provided, which includes a processing module, wherein the processing module is used to: cache a second image, where the second image is an image included in a preview stream of the electronic device, and the second image includes a target object; in response to a target event, shoot a first image, where the first image includes the target object; generate a reference image based on the first image and the second image, where the reference image includes the target object and a motion path of the target object, and the motion path of the target object is determined based on the first image and / or the second image.

[0073] In combination with the fourth aspect, in certain implementations of the fourth aspect, the target event includes one or more of the following: a brightness change value of the first image is greater than or equal to a brightness threshold; a moving distance of the target object is greater than or equal to a distance threshold; or a user's photo-taking operation.

[0074] In combination with the fourth aspect, in certain implementations of the fourth aspect, the processing module is specifically used to: determine the background of the reference image based on the second image.

[0075] In combination with the fourth aspect, in certain implementations of the fourth aspect, the processing module is specifically used to: determine a third image from the second image in response to a user selection; and generate a reference image based on the first image and the third image.

[0076] In combination with the fourth aspect, in certain implementations of the fourth aspect, the processing module is specifically used to: determine a mask for segmenting the target object based on the first image and a deep learning network trained based on adversarial loss self-supervision; determine multiple fourth images based on the mask and the first image and the third image, or determine multiple fourth images based on the mask and the third image, the fourth images being used to indicate the position of the target object at different times; and determine the motion path of the target object based on the multiple fourth images.

[0077] In a fifth aspect, an electronic device is provided, which includes a processor and a memory, the memory being used to store program instructions, and the processor being used to: display a shooting preview interface, the shooting preview interface including a reference image and an image of a photographed object, the reference image and the image of the photographed object being displayed superimposed, the reference image being stored locally in the electronic device, and the image of the photographed object including a portion of an image acquired in real time by a camera of the electronic device; in response to a shooting operation, saving a target image, the target image including the image displayed on the shooting preview interface.

[0078] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is specifically configured to: display the reference image with a first transparency, where a value of the first transparency is less than or equal to a first threshold.

[0079] In combination with the fifth aspect, in certain implementations of the fifth aspect, before saving the target image in response to a shooting operation, the processor is also used to: determine the shooting parameters of the target image based on the shooting parameters of the reference image, where the shooting parameters include one or more of the following: metering mode, sensitivity, aperture, focal length, shutter or exposure compensation.

[0080] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: display first shooting prompt information before saving the target image in response to a shooting operation, the first shooting prompt information being determined based on the shooting parameters of the reference image, and the shooting parameters including one or more of the following: metering mode, sensitivity, aperture, focal length, shutter, or exposure compensation.

[0081] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: display second shooting prompt information, the second shooting prompt information is determined based on the scene information of the reference image, and the scene information includes one or more of the following: light intensity, light angle, or light color.

[0082] In combination with the fifth aspect, in certain implementations of the fifth aspect, the reference image includes the target object and the motion path of the target object, the motion path of the target object is determined based on multiple first images, the first image includes the target object, and the multiple first images are used to indicate the position of the target object at different times.

[0083] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: before displaying the shooting preview interface, in response to a target event, capture multiple first images; wherein the target event includes one or more of the following: a brightness change value of the first image is greater than or equal to a brightness threshold; a moving distance of the target object is greater than or equal to a distance threshold; or, a user's photo-taking operation.

[0084] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: cache multiple second images before capturing multiple first images in response to a target event, where the second images are images included in a preview stream of the electronic device, and the second images include the target object.

[0085] In combination with the fifth aspect, in certain implementations of the fifth aspect, the second image is also used to determine the background of the reference image.

[0086] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: before displaying the shooting preview interface, in response to a user selection, determine a third image from the second image; and generate the reference image based on the first image and the third image.

[0087] In combination with the fifth aspect, in certain implementations of the fifth aspect, the motion path of the target object is determined based on multiple first images and multiple second images.

[0088] In combination with the fifth aspect, in certain implementations of the fifth aspect, the first image is used to determine a mask, and the mask is used to segment the target object from the first image.

[0089] In combination with the fifth aspect, in certain implementations of the fifth aspect, the mask is determined based on a deep learning network trained based on adversarial loss self-supervision.

[0090] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: before saving the target image in response to a shooting operation, adjust the display mode of the motion path in response to an operation acting on the motion path, and the display mode includes one or more of the following: color, position, or depth of field.

[0091] In a sixth aspect, an electronic device is provided, comprising a processor and a memory, the memory being used to store program instructions, the processor being used to: cache a second image, the second image being an image included in a preview stream of the electronic device, the second image including a target object; in response to a target event, capture a first image, the first image including a target object; and generate a reference image based on the first image and the second image, the reference image including the target object and a motion path of the target object, the motion path of the target object being determined based on the first image and / or the second image.

[0092] In combination with the sixth aspect, in certain implementations of the sixth aspect, the target event includes one or more of the following: a brightness change value of the first image is greater than or equal to a brightness threshold; a moving distance of the target object is greater than or equal to a distance threshold; or a user's photo-taking operation.

[0093] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically used to: determine the background of the reference image based on the second image.

[0094] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically used to: determine a third image from the second image in response to a user selection; and generate a reference image based on the first image and the third image.

[0095] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically used to: determine a mask for segmenting the target object based on the first image and a deep learning network trained based on adversarial loss self-supervision; determine multiple fourth images based on the mask and the first image and the third image, or determine multiple fourth images based on the mask and the third image, the fourth images being used to indicate the position of the target object at different times; determine the motion path of the target object based on the multiple fourth images.

[0096] In the seventh aspect, a computer program product is provided, which includes computer program code, and when the computer program code is run on a computer, the method in the first aspect and any possible implementation thereof is executed, or the method in the second aspect and any possible implementation thereof is executed.

[0097] In an eighth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the method in the first aspect and any possible implementation thereof is executed, or the method in the second aspect and any possible implementation thereof is executed.

[0098] In the ninth aspect, a chip is provided, comprising a processor for reading instructions stored in a memory, wherein when the processor executes the instructions, the chip implements the method in the first aspect and any possible implementation thereof, or the chip implements the method in the second aspect and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] FIG1 is a schematic diagram of a hardware architecture of an electronic device provided in an embodiment of the present application.

[0100] FIG2 is a schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application.

[0101] 3 to 8 are schematic diagrams of some user interfaces including retake function entrances provided in embodiments of the present application.

[0102] 9 and 10 are schematic diagrams of a user interface for selecting a reference image provided in an embodiment of the present application.

[0103] 11 and 12 are schematic diagrams of a shooting method provided in an embodiment of the present application.

[0104] 13 to 15 are schematic diagrams of another shooting method provided in an embodiment of the present application.

[0105] 16 and 17 are schematic diagrams of another shooting method provided in an embodiment of the present application.

[0106] Figures 18 and 19 are schematic diagrams of another shooting method provided in an embodiment of the present application.

[0107] FIG20 is a schematic diagram of a photo taken of a user and a reference picture provided in an embodiment of the present application.

[0108] 21 and 22 are schematic diagrams of a user interface for camera settings according to an embodiment of the present application.

[0109] FIG23 is a schematic diagram of a composite image provided in an embodiment of the present application.

[0110] FIG24 is a schematic diagram of a background image of the composite image in FIG23 .

[0111] Figure 25 is a schematic diagram of images contained in multiple photo streams provided in an embodiment of the present application.

[0112] FIG26 is a schematic diagram of the segmentation result of the moving object in the image in FIG25 .

[0113] FIG. 27 is a schematic diagram of a synthesis result of the motion path of the moving object in FIG. 26 .

[0114] Figure 28 is a schematic diagram of a user interface for generating a composite image provided in an embodiment of the present application.

[0115] FIG29 is a schematic diagram of another user interface for generating a composite image provided in an embodiment of the present application.

[0116] Figure 30 is a schematic diagram of a shooting device provided in an embodiment of the present application.

[0117] Figure 31 is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0118] The technical solution in this application will be described below with reference to the accompanying drawings.

[0119] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0120] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0121] 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0122] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0123] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0124] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0125] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0126] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0127] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0128] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0129] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0130] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0131] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0132] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0133] Electronic device 100 can implement a camera function using an ISP, camera 193, a video codec, a GPU, a display 194, and an application processor. The ISP processes data fed back by camera 193. Camera 193 is used to capture still images or video. The digital signal processor processes digital signals, and can process not only digital image signals but also other digital signals. The video codec compresses or decompresses digital video.

[0134] NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn.

[0135] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

[0136] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0137] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0138] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0139] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts and can also be used for touch vibration feedback.

[0140] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0141] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0142] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0143] Figure 2 is a software structure diagram of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application (app) layer, the application framework layer, the Android runtime (Android runtime) and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0144] As shown in FIG2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0145] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0146] As shown in FIG2 , the application framework layer may include a window manager, an activity manager, a package manager, a resource manager, a view system, a telephony manager, a notification manager, and the like.

[0147] Resource Manager, also known as Resource Management Service (RMS), provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0148] A window manager, also known as a window management service (WMS), manages windowed programs. It can determine the display size, determine whether a status bar is present, lock the screen, and take screenshots.

[0149] The activity manager, also known as the activity manager service (AMS), manages all application processes in the system.

[0150] The package manager, also known as the package management service (PMS), is responsible for functions such as application installation and uninstallation, component query and matching, and permission management.

[0151] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0152] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0153] The system library can include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0154] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0155] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0156] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0157] A 2D graphics engine is a drawing engine for 2D drawings.

[0158] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0159] Before formally introducing the embodiments of the present application, some professional terms that may be used in the following content are first explained and illustrated.

[0160] The preview stream displays the continuous images captured by the camera on the phone screen. It is primarily used by users to compose, focus, and adjust shooting parameters. The images contained in the preview stream are typically low-resolution, unprocessed raw data. To ensure smoothness and speed during the preview process, they are quickly compressed using a hardware encoder and displayed on the screen. Color correction and optimization may also be performed based on screen characteristics to ensure optimal viewing experience.

[0161] The photo stream refers to the high-resolution image data captured by the camera and saved to the phone's storage media after the photo button is pressed. The image data contained in the photo stream is generally of higher resolution. In some cases, this image data is processed by the image signal processor (ISP) for white balance adjustment, exposure compensation, noise reduction, and color correction. The resolution of images in the capture stream is generally higher than that of images in the preview stream, resulting in output photos with richer detail, more accurate colors, and higher clarity.

[0162] A mask, also known as a layer mask, is used to link to a layer in order to hide part of it from the image. A mask can be regarded as a cover that can be used to hide or display part of an image, video or layer to protect or isolate it.

[0163] Optical flow is a concept used in motion detection of objects in the field of view. It describes the motion of an observed object, surface, or edge caused by its motion relative to the observer. Optical flow methods are very useful in pattern recognition, computer vision, and other image processing fields. They can be used for motion detection, object segmentation, collision time and object expansion calculation, motion-compensated coding, and stereoscopic measurement of object surfaces and edges.

[0164] The Random Sampling Consensus (RANSAC) algorithm uses an iterative approach to estimate the parameters of a mathematical model from a set of observed data that may contain outliers. RANSAC is a non-deterministic algorithm, meaning it produces a reasonable result with a certain probability, and more iterations increase this probability.

[0165] Given that the timing of the occurrence of incidental scenes is difficult to capture and generally the time they appear is short, it is difficult for electronic devices to achieve the co-photography of users with incidental scenes. The present application provides a shooting method, in which the electronic device can simultaneously display the saved photos of incidental scenes and the image of the user captured by the camera of the electronic device on a shooting preview interface. By referring to the shooting preview interface, the user can change the posture to adjust the effect of co-photographing with the incidental scene to complete the co-photographing with the incidental scene. The shooting method is not restricted by time and space, and improves the success rate of the user taking photos with the incidental scene.

[0166] Figures 3 to 8 are schematic diagrams of some user interfaces for enabling the above-mentioned shooting method provided in an embodiment of the present application. For the sake of convenience of description, the function of the above-mentioned shooting method is referred to as the "reshooting" function of the electronic device below. It should be noted that this name is only exemplary. In some scenarios, the function may also have other names, such as "post-shooting" or "additional shooting", etc., and the present application does not impose any restrictions on this.

[0167] Figure 3 shows a user interface 10 (hereinafter referred to as the first user interface 10) of a camera application of an electronic device. In some examples, the first user interface 10 may be provided with an entrance for enabling the aforementioned reshoot function. In response to the user selecting the entrance to the reshoot function, the electronic device may display a user interface related to the reshoot function.

[0168] For example, the first user interface 10 may display a camera function option control 11, which may be used to display identification information (e.g., icons and / or text) of one or more functions supported by the electronic device's camera, such as identification information for a selfie function, identification information for a photography function, identification information for a portrait shooting function, and identification information 12 for a retake function. The camera function option control 11 may also be used to select the camera function corresponding to the aforementioned identification information. For example, in response to a user selecting identification information 12 for the retake function displayed in the camera function option control 11, the electronic device may display a user interface related to the retake function.

[0169] In some scenarios, the first user interface 10 may also be referred to as a shooting preview interface of the electronic device. When a user uses the shooting function of the electronic device, the electronic device may display an image captured by the camera of the electronic device on the first user interface 10 .

[0170] Figure 4 shows a user interface 15A of a desktop application of an electronic device (hereinafter referred to as a second user interface 15A). In some scenarios, the second user interface 15A may display an application icon 13 of a camera application. In some examples, in response to a user's operation on the application icon 13 of the camera application, the electronic device may display one or more function portals of the camera application on the second user interface 15A. In response to a user's operation of selecting a retake function portal of the camera application, the electronic device may display a user interface related to the retake function.

[0171] For example, in response to a user long pressing or swiping up the application icon 13 of the camera application, the electronic device may display a quick function option control 14 of the camera application in the second user interface 15A (e.g., near the application icon 13 of the camera application). The quick function option control 14 may display identification information (e.g., icons and / or text) of one or more functions supported by the camera application, such as identification information of a selfie function, identification information of a photography function, identification information of a portrait shooting function, and identification information 16 of a retake function. In response to a user selecting the identification information 16 of the retake function displayed in the quick function option control 14, the electronic device may display a user interface related to the retake function.

[0172] Figure 5 shows a user interface 15B of another desktop application of the electronic device (hereinafter referred to as the third user interface 15B). In some scenarios, the third user interface 15B may display a camera function folder control 17. The camera function folder control 17 may include or display icons of one or more functions of the camera application, such as an icon for a selfie function, an icon for a photography function, an icon for a macro function, and an icon 18 for a retake function, etc. In response to the user selecting the icon 18 for the retake function, the electronic device may display a user interface related to the retake function.

[0173] FIG6 shows a user interface 20 (hereinafter referred to as the fourth user interface 20) of a gallery application of an electronic device. The fourth user interface 20 can be used to display one or more pictures, photos, videos, or images stored in the electronic device. In some scenarios, the fourth user interface 20 can display photos taken by the user. The fourth user interface 20 also displays a photo function option control 21 corresponding to these photos. The photo function option control 21 can be used to display identification information (such as icons and / or text) of one or more functions supported by the photo. For example, identification information of the "share" function, identification information of the "edit" function, identification information of the "delete" function, and identification information 22 of the "retake" function, etc.

[0174] In some examples, in response to the user selecting the identification information 22 of the "reshoot" function, the electronic device can display a user interface related to the reshoot function, which can display the picture displayed on the fourth user interface 20, or the electronic device can use the picture displayed on the fourth user interface 20 as a reference picture to enable the reshoot function.

[0175] It should be noted that in some scenarios, the above-mentioned reference pictures may also be called reference images, reference photos or reference graphics, etc. This application does not distinguish between these words with similar meanings.

[0176] In some examples, some images stored by the electronic device can support the retake function. For example, images whose shooting scene information and shooting parameter information can be determined can support the retake function. Similarly, for example, images whose shooting parameter values ​​are within the adjustment range of the shooting parameter values ​​supported by the camera of the electronic device can support the retake function. This part will be further explained in detail below and will not be expanded here.

[0177] As shown in the schematic diagrams on the left and right sides of Figure 7, respectively, a foldable electronic device in a folded state and an unfolded state. The foldable electronic device can be folded along the horizontal direction of the device. In some scenarios, this type of foldable electronic device can be called a horizontally foldable electronic device.

[0178] In some scenarios, the folded electronic device on the left side of FIG. 7 may display the user interface 20 of the aforementioned gallery application (i.e., the fourth user interface 20). Alternatively, the display screen of the folded electronic device may display pictures or photos stored in the electronic device. In some examples, the pictures or photos may be pictures or photos that support a retake function.

[0179] In some examples, in response to a user switching the electronic device from a folded state to an unfolded state shown on the right side of Figure 7, or in response to the electronic device detecting that the current user (the user operating the electronic device) appears in the picture captured by the rear camera of the electronic device, the electronic device can determine the user's intention to perform a reshoot operation, and then, the electronic device can display a user interface related to the reshoot function, which can display the picture displayed on the fourth user interface 20, or the electronic device can use the picture displayed on the fourth user interface 20 as a reference picture to enable the reshoot function.

[0180] Figure 8 illustrates another foldable electronic device in its unfolded and folded states. Unlike the horizontally foldable electronic device in Figure 7 , the foldable electronic device in Figure 8 can be folded in the longitudinal direction. In some scenarios, this type of foldable electronic device may be referred to as a longitudinally foldable electronic device. The schematic diagram on the left side of Figure 8 shows the electronic device in its unfolded state, and the schematic diagram on the right side shows the electronic device in its folded state.

[0181] Similar to the horizontally foldable electronic device in FIG7 , the unfolded electronic device on the left side in FIG8 can display the user interface 20 of the aforementioned gallery application (i.e., the fourth user interface 20). In some examples, the pictures or photos displayed on the fourth user interface 20 can be pictures or photos that support the retake function.

[0182] In some examples, in response to a user switching the electronic device from an unfolded state to a folded state shown on the right side of Figure 8, or in response to the electronic device detecting that the current user (the user operating the electronic device) appears in the picture captured by the front camera of the electronic device, the electronic device can determine the user's intention to perform a reshoot operation, and then, the electronic device can display a user interface related to the reshoot function, which can display the picture displayed on the fourth user interface 20, or the electronic device can use the picture displayed on the fourth user interface 20 as a reference picture to enable the reshoot function.

[0183] The electronic device can be provided with multiple entrances for activating the reshoot function, and the user can activate the reshoot function from different function entrances. The setting of multiple reshoot function entrances is conducive to improving the efficiency of activating the reshoot function of the electronic device, and is conducive to improving the efficiency of the electronic device in synchronizing the user with occasional scenes, and is conducive to improving the user's shooting experience.

[0184] In some scenarios, the aforementioned embodiments provide a user interface for the retake function entry. For example, the first user interface 10, the second user interface 15A, and the third user interface 15B can all be referred to as entry interfaces or first-level interfaces. In other words, the aforementioned first user interface 10, the second user interface 15A, and the third user interface 15B can all be considered as one of multiple first-level interfaces. In response to the user's operation of enabling the retake function on the first-level interface, the electronic device can display a fifth user interface 25 as shown in FIG8. The fifth user interface 25 can be considered as a related interface of the retake function described above. The fifth user interface 25 can also be considered as one of the lower-level interfaces (secondary interfaces) of the multiple first-level interfaces. The user can select a picture from the second-level interface to be used as a reference picture for the retake operation. In other words, the second-level interface can be used by the user to select a reference picture to be used in the retake process.

[0185] 9 , one or more pictures may be displayed on the fifth user interface 25 , and these pictures may include pictures taken by the user, pictures or pictures received by the user from other devices or networks, and the like.

[0186] In some examples, the fifth user interface 25 may display a first type of picture, which may include information about the shooting scene, such as the lighting intensity, lighting angle, and light color of the shooting scene.

[0187] In some examples, the fifth user interface 25 may display a second type of picture, which may include information on shooting parameters, such as metering mode, ISO sensitivity, aperture, focal length, shutter or exposure compensation, etc.

[0188] In one possible implementation, the second category of pictures may specifically include category A pictures, category B pictures, and category C pictures, wherein the values ​​of the shooting parameters of category A pictures are within the adjustment range of the shooting parameters of the first camera of the electronic device, the values ​​of the shooting parameters of category B pictures are within the adjustment range of the shooting parameters of the second camera of the electronic device, and the values ​​of the shooting parameters of category C pictures are not within the adjustment range of the shooting parameters of the aforementioned first camera or second camera.

[0189] In some scenarios, the first camera may also be referred to as the rear camera of the electronic device, and the second camera may also be referred to as the front camera of the electronic device. In one possible implementation, the first camera has a wider adjustment range for shooting parameters, making it applicable to more shooting scenarios, or in other words, having stronger shooting capabilities.

[0190] It should be noted that the number of the first camera and the second camera can be multiple.

[0191] In some examples, a third type of picture may also be displayed on the fifth user interface 25 , and the third type of picture may include both the above-mentioned shooting scene information and shooting parameter information.

[0192] In FIG. 9 , the multiple pictures on the fifth user interface 25 may be sorted in the order of the time of shooting.

[0193] In some examples, FIG10 shows a schematic diagram of another secondary interface, or referred to as the sixth user interface 27. The sixth user interface 27 is similar to the fifth user interface 25 and is also used to display one or more pictures. These pictures may include pictures taken by the user, pictures or pictures received by the user from other devices or networks, etc. These pictures may include one or more of the first category pictures, the second category pictures, and the third category pictures mentioned above.

[0194] Different from the order of shooting time in FIG. 9 , the pictures in the sixth user interface 27 in FIG. 10 may be sorted in descending order of priority.

[0195] Here, the priority of the picture can be determined based on the information contained in the picture. For pictures that contain more information that is beneficial to the reshoot function, their priority is higher and they can be arranged at a front position on the sixth user interface 27. For pictures that contain less information that is beneficial to the reshoot function, their priority is lower and they can be arranged at a back position on the sixth user interface 27.

[0196] For example, among the aforementioned first-category pictures, second-category pictures, and third-category pictures, the third-category pictures that include both shooting scene information and shooting parameter information may have a higher priority and may be ranked higher.

[0197] For example, among the aforementioned Class A pictures, Class B pictures and Class C pictures, Class A pictures corresponding to the first camera can be sorted first, Class B pictures corresponding to the second camera can be sorted last, and Class C pictures can be sorted after Class A pictures and Class B pictures.

[0198] The electronic device can display multiple saved pictures based on whether the pictures are suitable for reshooting or the feasibility of the pictures being suitable for reshooting. The implementation of this technical solution is conducive to enabling users to select suitable pictures more quickly, improving the quality of the reshot photos, and enhancing the user's shooting experience.

[0199] The description of the secondary interface in Figures 9 and 10 is only exemplary. The secondary interface can also be displayed in other forms. The sorting of multiple pictures in the secondary interface can also be sorted in an order other than chronological order and priority order, for example, geographical location order, etc. This application does not impose any restrictions on this.

[0200] In some examples, the secondary interfaces represented by the fifth user interface 25 and the sixth user interface 27 may further display a first prompt message 26 , which may be used to prompt the user to select a retake photo to be taken.

[0201] In some examples, a "Cancel" control 28 may also be displayed on the secondary interface shown by the fifth user interface 25 and the sixth user interface 27. In response to the operation acting on the "Cancel" control 28, the electronic device may no longer display the fifth user interface 25 or the sixth user interface 27, but redisplay the primary interface such as the first user interface 10, etc.

[0202] For the case where the reshoot function is turned on by the reshoot function entrance in the fourth user interface 20 in Figures 7, 8 and 9, the electronic device can directly use the picture displayed in the fourth user interface 20 as a reference picture for reshooting. In other words, for the case where the reshoot function is turned on by the reshoot function entrance in the fourth user interface 20, the electronic device may no longer display the above-mentioned fifth user interface 25 or sixth user interface 27.

[0203] In some examples, in response to a user selecting a reference image on the aforementioned secondary interface, the electronic device may display the reference image in the shooting preview interface. In other words, in response to a user selecting a reference image on the aforementioned secondary interface 25, the electronic device may display the user interface of the camera application, i.e., the aforementioned first user interface 10, which may also display the reference image selected by the user.

[0204] In a possible implementation, the electronic device may display the reference picture on the first user interface 10 in a weakened display manner.

[0205] As shown in Figure 11, in some examples, the electronic device can display a reference image on the first user interface 10 in a low-transparency manner. Figures 11 to 20 are illustrated using the aforementioned horizontally foldable electronic device as an example. It should be noted that the electronic device can also be an ordinary non-foldable electronic device or a vertically foldable electronic device, and this application does not impose any restrictions on this.

[0206] Here, low transparency may mean that the transparency α of the reference image displayed on the first user interface 10 is greater than or equal to a first threshold, so that the posture, action, or expression of the user currently being photographed can be more prominently displayed on the first user interface 10. In other words, the transparency α of the reference image on the first user interface 10 should be set to a value that does not affect the normal display of the image captured by the camera of the electronic device on the first user interface 10.

[0207] For example, as shown in FIG12 , when a photographing user uses an electronic device to photograph a subject, the first user interface 10 can overlay the image of the subject on top of the aforementioned reference image. Here, the image of the subject is part of the image information captured in real time by the electronic device's camera. In other words, when the retake function is enabled, the electronic device can extract the subject's image (e.g., the subject's image) from the image captured by the camera and overlay the reference image and the subject's image on the shooting preview interface.

[0208] It should be noted that the subject can be a person, an animal, a plant, etc. In the case where the subject is a person, the number of people can be one or more. The embodiment of the present application does not limit the specific content of the subject. In the following examples, the subject is taken as an example of a user being photographed.

[0209] In some examples, in response to a user turning on a reshoot function, the electronic device can extract the subject being photographed based on the image captured by the camera. If the image captured by the camera includes a portrait, the portrait can be extracted first and the portrait can be superimposed with the reference image and displayed on the shooting preview interface.

[0210] According to the result of superimposing the photographed user and the reference image on the first user interface 10 , the photographing user can provide adjustment suggestions to the photographed user, such as moving away from or closer to the electronic device, moving left or right, etc.

[0211] As shown in Figure 13, in some examples, the electronic device can display the reference image in a floating window manner on the first user interface 10. In other words, the electronic device can display a first floating window 31 on the first user interface 10, and the first floating window 31 can display the aforementioned reference image.

[0212] Exemplarily, the first floating window 31 can be an active window, or in other words, in response to the user's operation of dragging or moving the first floating window 31, the electronic device can adjust the position of the first floating window 31 displayed on the first user interface 10 according to the user's operation.

[0213] Exemplarily, the size of the first floating window 31 can be adjusted, or in other words, in response to the user stretching or squeezing the first floating window 31, the electronic device can expand the display area of ​​the floating window according to the user's stretching operation, and the electronic device can reduce the display area of ​​the first floating window 31 according to the user's squeezing operation.

[0214] Exemplarily, the first floating window 31 can be switched with the viewfinder displayed by the first user interface 10, or in other words, in response to the user's operation of switching the first floating window 31 and the viewfinder (such as tapping, long pressing, double-clicking the floating window 31, etc.), the electronic device can display the content displayed in the first floating window 31 in Figure 14 in full screen, and display the content displayed in the viewfinder in Figure 13 in the second floating window 32.

[0215] For example, as shown in Figure 15 , when a photographing user uses an electronic device to photograph a subject, the electronic device may overlay a reference image and the subject's image in a first floating window 31, while the subject's image may continue to be displayed in the viewfinder. Referring to Figure 14 , in response to an operation to switch between the contents displayed in the first floating window 31 and the viewfinder, the electronic device may display the subject's image in a second floating window 32, displaying the reference image and the subject's image as a full-screen window.

[0216] According to the result of superimposing the photographed user and the reference image on the first user interface 10 , the photographing user can provide adjustment suggestions to the photographed user, such as moving away from or closer to the electronic device, moving left or right, etc.

[0217] As shown in Figure 16, in some examples, the electronic device can display the reference image in a split-screen window manner on the first user interface 10. For example, the electronic device can display the aforementioned reference image in the left area 33 of the first user interface 10, and display the content captured by the electronic device's camera (i.e., the content in the viewfinder) in the right area 34 of the first user interface 10.

[0218] For example, as shown in FIG17 , when the photographing user uses the electronic device to photograph the photographed user, the left area 33 of the first user interface 10 may overlay the image of the photographed user on the basis of displaying the aforementioned reference picture.

[0219] It should be understood that the above-mentioned contents displayed in the left area 33 and the right area 34 of the split-screen window are merely exemplary, and the electronic device may also exchange the display contents of the left and right display areas, or in other words, the electronic device may also display the image in the electronic device viewfinder in the left area 33 and display the reference picture in the right area 34.

[0220] According to the result of superimposing the photographed user and the reference image on the first user interface 10 , the photographing user can provide adjustment suggestions to the photographed user, such as moving away from or closer to the electronic device, moving left or right, etc.

[0221] The electronic device can display the superposition results of the reference image and the photographed user in real time on the user shooting interface. The shooting user can guide the photographed user to adjust or change the shooting image (action, posture or expression, etc.) according to the superimposed display results. The implementation of this technical solution is conducive to improving the success rate of users taking photos with occasional scenes, reducing the waste of storage resources and energy of electronic devices due to invalid shooting, and improving the user's shooting experience.

[0222] It should be noted that the different methods of displaying reference images on the above-mentioned first user interface 10 are not limited to the type of electronic device. In other words, for foldable electronic devices or general electronic devices, they can all display reference images and shooting viewfinders on the above-mentioned first user interface 10 at the same time, and this application does not impose any restrictions on this.

[0223] Taking the reference picture in FIG11 as an example, which is displayed on the first user interface 10 in a low-transparency manner, in response to determining the shooting operation, the electronic device can apply the target shooting information to take a picture of the user being photographed, that is, generate a supplementary photo.

[0224] For example, the operation of determining to take a photo may be an operation of clicking a photo button on the first user interface 10, or it may be a voice message issued by the user, such as "take a photo", etc. This application does not impose any restrictions on this.

[0225] In some scenarios, the shooting information contained in the reference image may also be referred to as reference shooting information, and the target shooting information may refer to the shooting information corresponding to the reference shooting information. For example, the reference shooting information may include the shooting scene information and / or shooting parameter information mentioned above. The shooting scene information may include one or more of the following: light intensity, light angle, and light color, and the shooting parameter information may include one or more of the following: metering method, ISO sensitivity, aperture, focal length, shutter speed, or exposure compensation.

[0226] In some examples, the target shooting information includes the same shooting scene information and shooting parameter information as the reference shooting information. For example, the target shooting information includes the same lighting angle as the reference shooting information, and the target shooting information includes the same aperture size as the reference shooting information.

[0227] In some examples, the deviation between the shooting scene information contained in the above-mentioned target shooting information and the shooting scene information contained in the reference shooting information is less than a preset threshold, and the deviation between the shooting parameter information contained in the above-mentioned target shooting information and the shooting parameter information contained in the reference shooting information is less than a preset threshold.

[0228] In some examples, the electronic device may also display one or more shooting prompt information while displaying the reference picture on the first user interface 10 . The shooting prompt information may be determined based on the aforementioned reference shooting information.

[0229] For example, the reference shooting information may include scene information of the reference image, such as lighting intensity, lighting angle, and light color. Referring to FIG12 , the electronic device may display information such as the lighting angle of the reference image on the first user interface 10. This information may be referred to as scene prompt information 111. Before performing a photo-taking operation, the lighting angle of the selected image is obtained to be 15°, and the reference aperture is F1.4. The scene prompt information 111 displayed on the first user interface 10 indicates the lighting angle is 15° and the reference aperture is F1.4.

[0230] For example, the reference shooting information may include parameter information for shooting the reference image, such as the metering method, ISO sensitivity, aperture, focal length, shutter speed, or exposure compensation. Referring to FIG12 , the electronic device may display information such as the aperture of the reference image on the first user interface 10 . This information may be referred to as parameter prompt information 112 . Before taking a photo, the user may adjust the aperture for shooting based on the parameter prompt information 112 displayed on the first user interface 10 .

[0231] In some examples, the shooting prompt information displayed by the electronic device on the first user interface 10 can be determined based on the shooting information of the reference image and the current shooting information. The current shooting information can be understood as the shooting scene information and / or shooting parameter information corresponding to the image of the photographed object currently displayed on the first user interface 10.

[0232] For example, the reference shooting information may include the lighting angle of the scene in the reference image. If the lighting angle indicated in the current shooting information deviates too much from the lighting angle of the scene in the reference image, the electronic device may display a shooting prompt message on the first user interface 10, such as, "The lighting angle deviates too much. The reshoot result may not be ideal. Please try adjusting the shooting angle or shooting position."

[0233] For example, the reference shooting information may include the focal length in the shooting parameters of the reference image. If the focal length indicated by the current shooting information deviates too much from the focal length in the shooting parameters of the reference image, the electronic device may display a shooting prompt message on the first user interface 10, such as, "The focal length deviation is too large. The reshoot result may not be ideal. Please try adjusting the camera's focus."

[0234] When the user is taking a retake, the electronic device can provide shooting prompts based on the shooting information of the reference picture to guide the user to take the retake better. The implementation of this technical solution is conducive to improving the effect of the electronic device's retake, reducing the waste of storage resources and energy of the electronic device due to invalid shooting, and improving the user's shooting experience.

[0235] In one possible implementation, the position of the image of the photographed user displayed on the first user interface 10 on the interface is determined based on the image captured by the camera of the electronic device. The position of the image of the photographed user on the first user interface 10 on the interface can be adjusted by adjusting the relative position of the photographed user and the electronic device, the shooting posture or the shooting angle, etc.

[0236] In some examples, the photographing user can adjust the relative position and / or depth of field relationship between a picture element in the reference picture and the image of the photographed user. In other words, in response to an operation on a picture element in the reference picture, the electronic device can adjust the position and / or depth of field relationship between the picture element and the image of the photographed user.

[0237] Taking FIG. 18 as an example, in response to the operation of clicking and moving the picture element 171 in the reference picture, the electronic device may adjust the position of the picture element 171 on the first user interface 10 according to the user's operation.

[0238] Taking FIG19 as an example, in response to a long press on a picture element 171 in a reference picture, the electronic device may display an operation control 181, which may display one or more operation options related to the picture element 171, such as pinning, pinning, brightening, dimming, hiding, etc. For example, in response to the user selecting the pin option, the electronic device may display the picture element 171 at the top layer of the multi-layer interface elements on the first user interface 10; in response to the user selecting the brightening option, the electronic device may increase the brightness of the picture element 171 on the first user interface 10.

[0239] The user can directly operate the picture elements in the reference picture on the first user interface 10, adjust the relative relationship between the picture elements in the reference picture and the image of the photographed object, enrich the shooting effect during the reshoot process, and help improve the user's shooting experience.

[0240] Before completing the reshoot operation and generating the reshoot photo, the electronic device can provide a variety of adjustable options for the user to adjust so that the user can obtain a satisfactory reshoot effect. The implementation of this technical solution is conducive to improving the electronic device's ability to coordinate with the user and occasional scenes, and improving the utilization efficiency of the electronic device's storage resources and energy.

[0241] Figure 21 exemplarily provides a schematic diagram of a photo generated by an electronic device of a user and a reference picture. For occasional scenes, using the shooting method provided by this application, the user's photo taking with the occasional scene can be free from restrictions such as time and geographical location, and the user's shooting experience is better.

[0242] The above examples focus on how to achieve the method of taking photos of users and occasional scenes. It should be noted that the above shooting method is not only applicable to the photo shooting of users and occasional scenes, but also to the photo shooting of users and non-occasional scenes. This application does not impose any restrictions on this.

[0243] The following examples further illustrate the method for capturing sporadic scenes provided by the embodiments of the present application. Sporadic scenes refer to scenes where the target scene is unpredictable or difficult to predict, or where it is predictable but difficult to capture, or where the capture process consumes a lot of energy or resources. For example, scenes of a meteor gliding across the sky, lightning, an iceberg sliding off an ice shelf, or a whale leaping out of the water.

[0244] In some examples, the electronic device can determine whether a capture condition is met based on the preview stream. If the capture condition is met, the electronic device performs a capture operation. In some scenarios, the ability of the electronic device to automatically capture a photo when the capture condition is met can also be referred to as a "conditional capture" function. In other words, for an electronic device with the "conditional capture" function enabled, the electronic device will perform a capture operation if it determines based on the preview stream that a trigger condition is met.

[0245] The preview stream may refer to the image frame captured in real time by the camera of the electronic device. Compared with the photo stream, the resolution of the image contained in the preview stream is smaller, and the electronic device is more efficient in processing the data of the preview stream. Determining whether the preset shooting conditions are met based on the preview stream is beneficial to reducing the memory resources occupied by the electronic device during the shooting process. The electronic device is more efficient in processing the image data of the preview stream, and the electronic device can determine whether shooting is needed more promptly, which is beneficial to improving the success rate of shooting occasional scenes.

[0246] The shooting condition can be understood as a trigger condition that triggers the electronic device to perform automatic shooting. In other words, in response to the shooting condition being met, the electronic device saves the image captured by the camera.

[0247] In some examples, the shooting condition may mean that a change in brightness of an image included in the preview stream is greater than or equal to a preset brightness threshold σ.

[0248] Taking the shooting of a lightning scene as an example, at time T0, the brightness of the image corresponding to the preview stream is L0. At time T0+Δt, the brightness of the image corresponding to the preview stream is L1. At time T0+2Δt, the brightness of the image corresponding to the preview stream is L2. At time T0+3Δt, the brightness of the image corresponding to the preview stream is L3.

[0249] Exemplarily, when the brightness difference between brightness L1 and brightness L0 is less than the preset brightness threshold σ, the electronic device can determine that the shooting condition is not met at time T0+Δt and does not perform the photo-taking operation; when the brightness difference between brightness L2 and brightness L1 is equal to the preset brightness threshold σ, the electronic device can determine that the shooting condition is met at time T0+2Δt and performs the photo-taking operation; when the brightness difference between brightness L3 and brightness L2 is greater than the preset brightness threshold σ, the electronic device can determine that the shooting condition is met at time T0+3Δt and performs the photo-taking operation.

[0250] In some examples, the shooting condition may mean that the moving distance of the movable object in the image included in the preview stream is greater than or equal to a preset distance threshold τ.

[0251] Taking the scene of a meteor falling from the sky as an example, at time T0, object A does not appear in the image corresponding to the preview stream. At time T0+Δt, object A appears in the image corresponding to the preview stream and is located at point P1(x1,y1) on the screen. At time T0+2Δt, object A in the image corresponding to the preview stream is located at point P2(x2,y2) on the screen. At time T0+3Δt, object A in the image corresponding to the preview stream is located at point P3(x3,y3) on the screen.

[0252] Exemplarily, when the maximum distance between point P1 and the adjacent screen edge is less than the preset distance threshold τ, the electronic device can determine that the shooting condition is not met at time T0+Δt and does not perform the photo-taking operation; when the distance between point P2 and point P1 is less than the preset distance threshold τ, the electronic device can determine that the shooting condition is not met at time T0+2Δt and does not perform the photo-taking operation; when the distance between point P3 and point P2 is greater than or equal to the preset distance threshold τ, the electronic device can determine that the shooting condition is met at time T0+3Δt and performs the photo-taking operation.

[0253] In some examples, the shooting conditions may be preset by the electronic device, or the shooting conditions may be set by the user.

[0254] Exemplarily, as shown in FIG21 , a user interface 35 (hereinafter referred to as the seventh user interface 35 ) including setting controls for shooting conditions provided in an embodiment of the present application is provided. The seventh user interface 35 may also be referred to as a camera setting user interface 35. Exemplarily, referring to FIG3 , in response to clicking the “Settings” button on the first user interface 10 , the electronic device may display the seventh user interface 35 .

[0255] The seventh user interface 35 displays a number of function controls related to photographing and filming, such as a "photo ratio" setting function control, a "voice-activated photographing" setting function control, a "timer shooting" setting function control, and a "conditional photographing" setting function control 36. The "conditional photographing" setting function control 36 can be used to turn the conditional photographing function on or off.

[0256] In one possible implementation, when a user turns on the conditional shooting function, in order to improve the shooting effect of the electronic device when this function is enabled, the electronic device may display a prompt message on the seventh user interface 35. This prompt message may be used to indicate the applicable scenarios of the conditional shooting function, for example: "The conditional shooting function has been enabled. This function is suitable for capturing occasional scenes such as falling meteors and whales jumping out of the water."

[0257] In some examples, in response to selecting the "conditional photography" setting function control 36, the electronic device can display an eighth user interface 38 as shown in Figure 22. The eighth user interface 38 can be used to set the different trigger conditions of the above-mentioned conditional photography function to be turned on or off. The eighth user interface 38 can also be used to set the specific setting method of the trigger conditions of the above-mentioned conditional photography function.

[0258] For example, the eighth user interface 38 may include a "brightness change condition" setting control 37 and a "distance change condition" setting control 39. When the switch buttons of the "brightness change condition" setting control 37 and the "distance change condition" setting control 39 are both in the off state, the "conditional photo shooting" setting function control 36 in FIG. 21 may be displayed as being in the off state; when the switch buttons of the "brightness change condition" setting control 37 and / or the "distance change condition" setting control 39 are in the on state, the "conditional photo shooting" setting function control 36 in FIG. 21 may be displayed as being in the on state.

[0259] In some examples, when the switch button of the "brightness change condition" setting control 37 is in the off state, in response to the operation of clicking the switch button of the "brightness change condition" setting control 37, the electronic device can display the "brightness change condition automatic setting" control 371A as shown in Figure 22, and the "brightness change condition automatic setting" control 371A can be in the enabled state. In other words, when the brightness change condition is enabled as the trigger condition of the conditional shooting function, the electronic device can set a default value for the brightness change threshold (i.e., the aforementioned brightness threshold σ) for triggering automatic shooting due to brightness change. The default value of the brightness threshold σ can be determined according to the shooting scene, or in other words, the default value of the brightness threshold σ is set for a preset scene (such as the various occasional scenes described above).

[0260] In some examples, when the "brightness change condition automatic setting" control 371A is in the turned-on state, in response to the user's closing operation, the electronic device may display the "brightness change condition manual setting" control 371B as shown in Figure 22 on the eighth user interface 38. Exemplarily, the "brightness change condition manual setting" control 371B may include a slider control. In response to the user selecting different positions on the slider control (such as sliding selection or tapping selection, etc.), the electronic device may determine the brightness change threshold that triggers automatic shooting based on the position selected by the user. In one possible implementation, the slider control of the "brightness change condition manual setting" control 371B may have a default value or a recommended value, and the value of the default value may be the same as the default value of the "brightness change condition automatic setting" control 371A.

[0261] In some examples, when the switch button of the "distance change condition" setting control 39 is in the off state, in response to clicking the switch button of the "distance change condition" setting control 39, the electronic device can display the "distance change condition automatic setting" control 391A as shown in Figure 22, and the "distance change condition automatic setting" control 391A can be in the enabled state. In other words, when the distance change condition is enabled as the trigger condition of the conditional shooting function, the electronic device can set a default value for the brightness change threshold (i.e., the aforementioned distance threshold τ) for triggering automatic shooting due to distance change. The default value of the distance threshold τ can be determined according to the shooting scene, or in other words, the default value of the distance threshold τ is set for a preset scene (such as the various occasional scenes described above).

[0262] In some examples, when the "distance change condition automatic setting" control 391A is in the turned-on state, in response to the user's closing operation, the electronic device may display a "distance change condition manual setting" control 391B as shown in Figure 21 on the eighth user interface 38. Exemplarily, the "distance change condition manual setting" control 391B may include a slider control. In response to the user selecting different positions on the slider control (such as sliding selection or tapping selection, etc.), the electronic device may determine the distance change threshold for triggering automatic shooting based on the position selected by the user. In one possible implementation, the slider control of the "distance change condition manual setting" control 391B may have a default value or a recommended value, and the value of the default value may be the same as the default value of the "distance change condition automatic setting" control 391A.

[0263] In some examples, the "brightness change condition" setting control 36 and the "distance change condition" setting control 39 shown in Figure 22 can both be in an enabled state, and then, when the brightness threshold σ and / or the distance threshold τ are met, the electronic device can trigger automatic photography.

[0264] Before and for a period of time after a photo-taking operation is triggered, the electronic device may continuously obtain a preview stream. After the photo-taking operation is triggered, the electronic device may obtain data from the photo-taking stream and save the data from the photo-taking stream on a storage medium of the electronic device. In one possible implementation, the electronic device may combine the preview stream and the photo-taking stream to generate a composite image. The composite image may include the motion path of a moving object in the scene to be captured. For example, the composite image may include the motion path of a meteor falling across the sky. For another example, the composite image may include the motion path of a whale leaping out of the water and reentering the water.

[0265] Here, the composite image generated by the electronic device in combination with the preview stream and the photo stream can be used for the reshoot function of the electronic device in the aforementioned embodiment, but it should be noted that the implementation of the above-mentioned reshoot function does not depend on the composite image, or in other words, photos taken by the electronic device, pictures obtained through the Internet, etc. can also be applied to the above-mentioned reshoot function, and this application does not impose any restrictions on this.

[0266] The electronic device's camera can continuously capture light from the scene and convert it into electrical signals. After analog-to-digital conversion, the electrical signals can generate digital signals. These digital signals can be displayed in the form of images, which can be understood as the images included in the preview stream. Before triggering a photo capture operation, the electronic device can continuously capture these images and determine whether the capture conditions are met based on them.

[0267] It should be noted that the image frames included in the preview stream may be cached in the memory of the electronic device in the form of bits. In some scenarios, the preview stream or the image frames included in the preview stream may also be referred to as preview stream data.

[0268] In some examples, the digital signal or preview stream image data acquired before triggering the capture operation can be cached in the electronic device's memory. In some scenarios, this data can also be referred to as a prior preview stream. As previously mentioned, the prior preview stream can be used to determine whether capture conditions are met. In some examples, the prior preview stream can also be used in the process of generating a composite image.

[0269] In some examples, when the shooting conditions are met, the electronic device can perform a photo-taking operation. Specifically, the electronic device can save the image data of the photo stream on the storage medium of the electronic device. Compared with the preview stream, the image contained in the photo stream has a higher resolution and contains more image information, which is conducive to generating clearer and higher-resolution photos.

[0270] In some examples, if the shooting conditions are met, the electronic device can continue to cache the image data of the preview stream. In some scenarios, this portion of data can also be called a subsequent preview stream. In some examples, the subsequent preview stream can be used in the process of generating a composite image.

[0271] For example, when a camera triggers a photo capture, the electronic device can automatically capture multiple photos. Alternatively, the electronic device can generate multiple photos based on the image data in the photo stream. Each of these photos only captures the meteor's position at a specific moment in its fall and doesn't reflect the meteor's entire path. For ease of explanation, images that capture the meteor's entire path are referred to as composite images.

[0272] Figure 23 provides an exemplary schematic diagram of a composite image. This composite image can be considered to be composed of background data and foreground data. Background data can refer to image data that remains unchanged or substantially unchanged throughout the meteor's fall, such as the data corresponding to the mountain peak in Figure 23. In some scenarios, background data can also be referred to as data corresponding to static objects in the composite image. Foreground data can refer to data corresponding to objects whose lighting information changes throughout the meteor's fall, such as the data corresponding to the meteor's path in Figure 23.

[0273] In some examples, the electronic device may generate the aforementioned composite image based on the aforementioned previous preview stream, the subsequent preview stream, and the photo stream.

[0274] Exemplarily, the electronic device may determine background data of a composite image based on a previous preview stream. FIG24 exemplarily provides a background of a composite image generated using the above method.

[0275] The background data for the composite image can be data about static objects in the composite image. This data refers to image data that remains unchanged or largely unchanged throughout the meteor's trajectory. The previous preview stream is captured before the capture operation is triggered, so it generally does not include the meteor's trajectory path. Therefore, the previous preview stream is more suitable for obtaining background data for the composite image. In other words, using the previous preview stream can produce a purer composite image background.

[0276] In some examples, the electronic device may determine foreground data of the composite image based on the photo stream data.

[0277] For example, the electronic device may determine a mask for segmenting the foreground image in the image data of the photo stream based on the subsequent preview stream, and then use the mask to segment and extract the image of the meteor-related portion in the photo stream.

[0278] In one possible implementation, the electronic device predicts the above-mentioned mask based on a deep learning network trained through adversarial loss self-supervision. The input data of the deep learning network can be part or all of the photo stream data, and the output of the deep learning network can be the above-mentioned mask.

[0279] FIG25 exemplarily provides some photos included in the photo stream data. After segmenting and extracting these photos using masks, images corresponding to the foreground data shown in FIG26 can be obtained.

[0280] In some examples, the electronic device can determine the motion path of the meteor throughout the entire process of its fall in the composite image based on the first preview stream and the second preview stream. In other words, the electronic device can determine the corresponding relationship between the position and time of the meteor in the composite image based on the first preview stream and the second preview stream.

[0281] For example, the electronic device can use the mask predicted by the deep learning network to segment the foreground in the preview stream (including the preceding and following preview streams). Using methods such as optical flow, the image information of multiple meteors obtained at different positions at different times can be processed to obtain the complete motion path of the meteor in the preview stream, for example, the first path.

[0282] The first path information includes not only the meteor's location information but also the time information or timestamp corresponding to each meteor's location. In the above scheme, the foreground data image obtained by using a mask to segment the photos in the photo stream also contains time information. By combining and connecting multiple foreground data in the photo stream according to their time correspondence, the meteor's path determined by the photo stream can be obtained, which is called the second path.

[0283] For example, the preview stream may include seven second images: images Pr1, Pr2, Pr3, Pr4, Pr5, Pr6, and Pr7. Image Pr0 is recorded at time f0, when no meteor has appeared. Image Pr0 can be used as the background for generating a composite image, similar to the image in Figure 24 above. Image Pr1 is recorded at time f1, and the electronic device determines that the shooting conditions are met based on image Pr1. In other words, image Pr1 is recorded before the photo stream is generated. Images Pr2 through Pr7 are recorded at times t1, t2, t3, t4, and t5, respectively. The photo stream may include two first images, as shown in Figure 25: images Po1 and Po2, recorded at times t1 and t3, respectively. Table 1 below shows the correspondence between the images in the preview stream and the images in the photo stream.

[0284] Table 1

[0285] As described above, the electronic device can use the mask to extract the foreground parts of the image Po1 and the image Po2 contained in the photo stream, and obtain the two foreground images in Figure 26, namely, the image Pso1 and the image Pso2.

[0286] Exemplarily, the electronic device may synthesize the motion path of the meteor, ie, the second path, based on the image Pso1 and the image Pso2.

[0287] One possible situation is that the second path synthesized by using the images Pso1 and Pso2 is continuous or uninterrupted. In this case, the electronic device may use the second path as the foreground portion of the synthesized image.

[0288] One possible scenario is that the second path synthesized using images Pso1 and Pso2 is discontinuous. In this case, the electronic device can further segment the image data in the preview stream to generate more images similar to images Pso1 and Pso2. For example, images Pr2, Pr4, Pr6, and Pr7 in the preview stream can be segmented to generate images Psr2, Psr4, Psr6, and Psr7. These four images correspond to times f1, t2, t4, and t5, respectively. The data for these four times is not included in the still stream. Based on this, the electronic device can synthesize the meteor's motion path using the two images segmented from the still stream (images Pso1 and Pso2) and the four images segmented from the preview stream (images Psr2, Psr4, Psr6, and Psr7). This is called a third path, for example. Compared to the second path described above, this third path contains more information about the meteor's location and is therefore more complete.

[0289] In some examples, the acquisition of the photo stream may not rely on the electronic device's automatic photo capture, or the photos included in the photo stream may be saved locally on the electronic device in response to a user's photo capture operation. For example, the first images in the photo stream, image Po1 and image Po2, may also be captured by the electronic device in response to a user's manual capture operation.

[0290] For example, in response to the user pressing a photo button, the electronic device may save one or more photos, which may also be used to generate the aforementioned composite image.

[0291] Similarly, the electronic device can generate the motion path of the shooting star based on the photos taken manually by the user (as opposed to the photos taken automatically by the electronic device). The electronic device can also generate the motion path of the shooting star by combining the preview stream and the photos taken manually by the user.

[0292] FIG27 exemplarily provides a schematic diagram of a complete meteor motion path obtained using the above method. The schematic diagram of the meteor motion path can be the aforementioned second path or the third path.

[0293] In some examples, the electronic device may combine the complete meteor motion path with the background image to obtain a composite image.

[0294] For example, the electronic device can extract feature points from the image, match the descriptors of these feature points, eliminate errors according to the RANSAC algorithm, and then use homography to align the background with the complete path of the meteor's motion to obtain the composite image shown in Figure 23 above.

[0295] In the above solution, the electronic device can combine the preview stream and the photo stream to jointly generate a composite image, which is beneficial to improving the quality of the generated composite image and improving the success rate of the electronic device in shooting occasional scenes.

[0296] In some examples, the number of composite images determined using the preview stream and the photo stream can be multiple, and these images can be saved to the storage medium of the electronic device for user selection. In one possible implementation, the reference image mentioned above can be one of the multiple composite images.

[0297] In some examples, the process of generating a composite photo can be performed in the background, or in other words, when the photo stream and / or preview stream of the electronic device is capable of generating a composite photo, the electronic device can automatically generate the aforementioned composite photo.

[0298] Exemplarily, as shown in FIG28 , in response to the event of successfully generating a composite photo, during the process of the user using the electronic device to take pictures, the electronic device may display a prompt message 281 on the current user interface, or the electronic device may display a prompt message 281 on the first user interface 10. The prompt message 281 may be used to prompt that a composite photo containing a complete path has been successfully generated.

[0299] For example, as shown in FIG28 , in response to an event of successfully generating a composite photo, the electronic device may also display a display control 282 on the first user interface 10 , which may be used to display the generated composite photo.

[0300] In some examples, the user may select photos in the photo stream for generating the composite image, or in other words, the composite image may be generated based on the photos in the photo stream selected by the user.

[0301] Exemplarily, after the electronic device completes automatic shooting and obtains photos from multiple photo streams, it can display the ninth user interface 40 as shown in Figure 29. The ninth user interface 40 can display photos from multiple photo streams saved after the automatic shooting operation of the electronic device is triggered. The ninth user interface 40 can also display a recommended composite image 42, which can be automatically generated by the electronic device based on part or all of the photos in the aforementioned multiple photo streams. Exemplarily, this part of the photos can be called composite materials, and these composite materials can be arranged in chronological order according to the shooting time. In a possible implementation, these composite materials used to generate the recommended composite image 42 can be highlighted in the ninth user interface 40, such as highlighting or adding a mark (such as a check mark, etc.) to this part of the photos.

[0302] Taking the example of a shooting scene of a meteor falling across the sky, the recommended composite image 42 may include the meteor's path (foreground portion) during its descent. The recommended composite image 42 may also include background portions such as mountain peaks. In other words, the recommended composite image 42 here can be considered the aforementioned composite image.

[0303] In some examples, in response to the user's operation of reselecting the composite material on the ninth user interface 40 , the electronic device may display the regenerated composite image on the ninth user interface 40 .

[0304] Exemplarily, a “regenerate” control 42 may also be displayed on the ninth user interface 40. In response to the user reselecting the synthetic material b and clicking the “regenerate” control 42, the electronic device may display the regenerated synthetic image on the ninth user interface 40.

[0305] In one possible implementation, when the number of synthetic materials selected by the user is insufficient to generate a synthetic image, the electronic device may display a prompt message on the ninth user interface 40, for example, "The number of photos synthesized by the user is too small, please select at least 6 photos", etc. This prompt message can be used to prompt the user to increase the number of synthetic materials.

[0306] In some examples, a "manual adjustment" control 44 may also be displayed on the ninth user interface 40. In response to the user selecting the "manual adjustment" control 44, the electronic device may display an image adjustment control. In response to the operation of the image adjustment control, the electronic device may adjust the composite image displayed on the ninth user interface 40, for example, color, brightness, contrast or saturation.

[0307] For example, in response to a user selecting the "manual adjustment" control 44, the electronic device may display a tenth user interface, which may include the aforementioned image adjustment control and the composite image in the ninth user interface 40. The image adjustment control may display one or more image adjustment function icons, such as a color adjustment function icon, a brightness adjustment function icon, a contrast adjustment function icon, or a saturation adjustment function icon. In response to a user selecting an image adjustment function icon, the electronic device may adjust the composite image accordingly.

[0308] In a possible implementation, the image adjustment control may further include a functional control for adjusting the depth of field, size, position, color, etc. of the motion path of the object in the composite image.

[0309] The method embodiment of the present application is described in detail above in conjunction with Figures 1 to 29. The device embodiment of the present application is described below in conjunction with Figures 30 and 31. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment. Therefore, for parts not described in detail, reference can be made to the above method embodiment.

[0310] Figure 30 shows a camera device 3000 provided in an embodiment of the present application. This device 3000 may have the functions of the electronic device in the above method embodiments and may be used to execute the steps performed by the functions of the electronic device in the above method embodiments. This function may be implemented in hardware, or in software or hardware executing corresponding software implementations. The hardware or software may include one or more modules corresponding to the above functions.

[0311] In a possible implementation, the photographing device 3000 may include an acquisition module 3010 and a processing module 3020 , and the acquisition module 3010 and the processing module 3020 are coupled to each other.

[0312] The acquisition module 3010 can be used to support the electronic device in acquiring user input, such as the aforementioned operation of the first thread or the second thread acquiring an object in the shared memory space.

[0313] The processing module 3020 is used to support the electronic device in executing the processing actions in the above method embodiments, such as performing unchanged processing operations on shared objects.

[0314] Optionally, the storage and shooting device 3000 may further include a storage unit 3030 for storing program codes and data of the shooting device 3000 .

[0315] Figure 31 shows an electronic device 3100 provided in an embodiment of the present application. As shown in the figure, the electronic device 3100 includes: at least one processor 3110 and a transceiver 3120. The processor 3110 is coupled to a memory and is configured to execute instructions stored in the memory to control the transceiver 3120 to send and / or receive signals.

[0316] Optionally, the electronic device 3100 further includes a memory 3130 for storing instructions.

[0317] In some embodiments, the processor 31110 and the memory 3130 may be combined into a processing device, and the processor 3110 is configured to execute program codes stored in the memory 3130 to implement the above functions. In specific implementations, the memory 3130 may also be integrated into the processor 3110 or independent of the processor 3110.

[0318] In some embodiments, the transceiver 3120 may include a receiver (or receiver) and a transmitter (or transmitter).

[0319] The transceiver 3120 may further include an antenna, and the number of antennas may be one or more. The transceiver 3120 may be a communication interface or an interface circuit.

[0320] When the electronic device 3100 is a chip, the chip includes a transceiver module and a processing module. The transceiver module may be an input / output circuit or a communication interface; and the processing module may be a processor, microprocessor, or integrated circuit integrated on the chip.

[0321] This embodiment further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the shooting method in the above-mentioned embodiment.

[0322] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the shooting method in the above-mentioned embodiment.

[0323] In addition, embodiments of the present application further provide a device, which may be a chip, component, or module, and may include a processor and a memory connected thereto. The memory is configured to store computer-executable instructions. When the device is in operation, the processor executes the computer-executable instructions stored in the memory, causing the chip to perform the shooting methods described in the aforementioned method embodiments.

[0324] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0325] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0326] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0327] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0328] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0329] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0330] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0331] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A shooting method, applied to an electronic device, characterized in that: include: Displaying a shooting preview interface, the shooting preview interface including a reference image and an image of a photographed object, the reference image and the image of the photographed object being displayed superimposed, the reference image being stored locally on the electronic device, and the image of the photographed object including a portion of an image captured in real time by a camera of the electronic device; In response to a shooting operation, a target image is saved, where the target image includes the image displayed on the shooting preview interface.

2. The shooting method according to claim 1, wherein: The displaying of the shooting preview interface includes: The reference image is displayed with a first transparency, where a value of the first transparency is less than or equal to a first threshold.

3. The shooting method according to claim 1 or 2, characterized in that: Before saving the target image in response to the shooting operation, the method further includes: The shooting parameters of the target image are determined according to the shooting parameters of the reference image, where the shooting parameters include one or more of the following: metering mode, sensitivity, aperture, focal length, shutter speed, or exposure compensation.

4. The shooting method according to any one of claims 1 to 3, characterized in that: Before saving the target image in response to the shooting operation, the method further includes: First shooting prompt information is displayed, where the first shooting prompt information is determined according to shooting parameters of the reference image, where the shooting parameters include one or more of the following: metering mode, sensitivity, aperture, focal length, shutter, or exposure compensation.

5. The shooting method according to any one of claims 1 to 4, characterized in that: Before saving the target image in response to the shooting operation, the method further includes: Second shooting prompt information is displayed, where the second shooting prompt information is determined according to scene information of the reference image, where the scene information includes one or more of the following: light intensity, light angle, or light color.

6. The shooting method according to any one of claims 1 to 5, characterized in that: The reference image includes a target object and a motion path of the target object. The motion path of the target object is determined based on a plurality of first images. The first images include the target object. The first images are used to indicate positions of the target object at different times.

7. The shooting method according to claim 6, characterized in that: Before displaying the shooting preview interface, the shooting method further includes: In response to a target event, a plurality of the first images are captured; the target event includes one or more of the following: A brightness change value of the first image is greater than or equal to a brightness threshold; The moving distance of the target object is greater than or equal to a distance threshold; Or, the user's photo taking operation.

8. The shooting method according to claim 7, characterized in that: Before taking a plurality of the first images in response to the target event, the method further includes: A plurality of second images are cached, where the second images are images included in the preview stream of the electronic device, and the second images include the target object.

9. The shooting method according to claim 8, characterized in that: The first image is an image included in a photo stream of the electronic device, and a resolution of the second image is smaller than a resolution of the first image.

10. The shooting method according to claim 8 or 9, characterized in that: The second image is also used to determine the background of the reference image.

11. The shooting method according to any one of claims 8 to 10, characterized in that: Before displaying the shooting preview interface, the method further includes: In response to a user selection, determining a third image from the second image; The reference image is generated based on the first image and the third image.

12. The shooting method according to any one of claims 8 to 11, characterized in that: The motion path of the target object is determined based on a plurality of the first images and a plurality of the second images.

13. The shooting method according to any one of claims 6 to 12, characterized in that: The first image is used to determine a mask, and the mask is used to segment the target object from the first image.

14. The shooting method according to claim 13, wherein: The mask is determined by a deep learning network trained through self-supervision based on an adversarial loss.

15. The shooting method according to any one of claims 6 to 14, characterized in that: Before saving the target image in response to the shooting operation, the method further includes: In response to an operation acting on the motion path, a display manner of the motion path is adjusted, where the display manner includes one or more of the following: color, position, or depth of field.

16. A shooting method, applied to an electronic device, characterized in that: include: caching a second image, where the second image is an image included in a preview stream of the electronic device, and the second image includes a target object; In response to a target event, capturing a first image, where the first image includes the target object; A reference image is generated based on the first image and the second image, wherein the reference image includes a target object and a motion path of the target object, and the motion path of the target object is determined according to the first image and / or the second image.

17. The shooting method according to claim 16, characterized in that: The target event includes one or more of the following: A brightness change value of the first image is greater than or equal to a brightness threshold; The moving distance of the target object is greater than or equal to a distance threshold; Or, the user's photo taking operation.

18. The shooting method according to claim 16 or 17, characterized in that: The generating a reference image based on the first image and the second image includes: A background of the reference image is determined based on the second image.

19. The shooting method according to any one of claims 16 to 18, characterized in that: The generating a reference image based on the first image and the second image includes: In response to a user selection, determining a third image from the second image; The reference image is generated based on the first image and the third image.

20. The shooting method according to claim 19, characterized in that: The generating a reference image based on the first image and the third image includes: Determining a mask for segmenting the target object based on the first image and a deep learning network trained by self-supervision based on adversarial loss; determining a plurality of fourth images based on the mask, the first image, and the third image, or determining a plurality of fourth images based on the mask and the third image, the fourth images being used to indicate positions of the target object at different times; A motion path of the target object is determined according to the plurality of fourth images.

21. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method according to any one of claims 1 to 15 or claims 16 to 20.

22. A photographing device, characterized in that: Comprising modules for implementing the method of any one of claims 1 to 15 or claims 16 to 20.

23. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 15 or claims 16 to 20 is implemented.

24. A chip, characterized in that: The chip comprises a processor and a memory, wherein the processor is used to read instructions stored in the memory. When the processor executes the instructions, the chip implements the method according to any one of claims 1 to 15 or claims 16 to 20.

Images