Photographing method and related apparatus

Abstract

The application discloses a photographing method and related device. After an electronic device starts a "photo combining" function for a selected picture containing an object A, a camera application is started, and indication information of the picture is displayed in a preview interface, and an image captured by a camera of the electronic device is displayed at the same time. Then, the electronic device can receive a photographing instruction input by a user, and fit the picture and the image captured by the camera into a synthesized image containing the object A and an object B at the same time. The method can realize high-quality combination of the object A and the object B without the aid of other devices or other personnel.

Description

[0001] This application is a divisional application. The original application has the application number 202211037891.9 and the original application date is August 26, 2022. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to shooting methods and related devices. Background Technology

[0003] When traveling in a group, there is often a need for group photos. Here are some methods for taking group photos:

[0004] 1. Ask a passerby to take a group photo. This method not only requires bothering others, but the photo quality may also be mediocre due to the passerby's lack of knowledge about the camera equipment.

[0005] 2. Use a selfie stick, tripod, or place the shooting device directly on the ground to take photos. However, selfie sticks increase the user's load, make it impossible to control the shooting distance, and result in lower image quality when using the front-facing camera. Tripods also increase the load, do not provide real-time preview, and pose a risk of damaging the shooting device. Placing the shooting device on the ground does not allow for real-time preview and makes it impossible to control the shooting distance.

[0006] 3. Take photos of multiple people separately and then use photo editing software to create a group photo. This method of creating a group photo is time-consuming, does not support real-time preview of the group photo, and the overall effect is mediocre. Summary of the Invention

[0007] This application provides a shooting method and related equipment that can achieve high-quality group photos of multiple subjects without the aid of other equipment or personnel.

[0008] In a first aspect, embodiments of this application provide a shooting method applied to an electronic device. The method may include: the electronic device launching a gallery application and displaying a first user interface, the first user interface displaying a first image, the first image containing a first object; the electronic device receiving a first operation on the first image; the electronic device launching a camera application and displaying a second user interface, the second user interface displaying a preview frame, the preview frame displaying an image captured by the electronic device's camera, and indication information of the first image; the electronic device receiving a second operation for taking a picture; the electronic device storing a second image, the second image being obtained by fitting the image captured by the camera and the first image, the image captured by the camera containing a second object, and the second image containing both the first object and the second object.

[0009] By implementing the method provided in the first aspect, electronic devices can access the camera application through a gallery application and take a group photo of the first and second subjects without the aid of other equipment or personnel. During the photo-taking process, the relative positions of the first and second subjects can be previewed and adjusted in real time. The shooting distance and the camera used can also be flexibly adjusted to achieve high-quality group photo results. Furthermore, this method is easy to operate, quick, convenient, and supports continuous and large-scale group photo taking.

[0010] In conjunction with the first aspect, in some implementations, the first object or the second object is any one of the following: a person, an animal, a landmark, a building, or a doll.

[0011] In conjunction with the first aspect, in some implementations, the number of either the first object or the second object can be multiple.

[0012] In conjunction with the first aspect, in some embodiments, the indicator information of the first image displayed by the electronic device in the preview frame can be a first image with transparency or blurring. In this way, the user can see both the first image through the preview frame and, through the blurred or transparent first image, the image captured by the camera displayed by the electronic device, achieving the effect of real-time previewing the relative positions of objects in the first image and objects in the camera-captured image.

[0013] In conjunction with the first aspect, in some implementations, after receiving the second operation, the electronic device can fit the second image in any of the following ways:

[0014] Method 1: Fit the region where the second object is located in the image captured by the camera to the first image to obtain the second image.

[0015] In specific implementation, the electronic device identifies a second object in the image captured by the camera; determines a main region containing the second object in the image captured by the camera, the main region being the region determined based on the second object; determines a co-location region in the first image, the part outside the co-location region in the first image, and the part outside the main region in the image captured by the camera have a correlation greater than a second value; fits the main region to the co-location region in the first image to obtain the second image.

[0016] Method 2 involves fitting the region containing the first object in the first image to the image captured by the camera to obtain the second image.

[0017] The electronic device identifies a first object in a first image; determines a main region containing the first object in the first image, the main region being a region determined based on the first object; determines a co-location region in the image captured by the camera, the part outside the co-location region in the image captured by the camera, and the part outside the main region in the first image having a correlation greater than a second value; fits the main region to the co-location region in the image captured by the camera to obtain a second image.

[0018] In method 1 or method 2 above, the main region can be a region obtained by extending outward by a first value distance from the corresponding object (such as the first object or the second object), or a region with a similar or identical color or pixel count to the corresponding object, extended outwards. The first value can be preset by the electronic device. The first value can be related to the camera model configured in the electronic device.

[0019] Determining the main region based on the corresponding object (such as the first object or the second object) can ensure the integrity of the objects in the main region while avoiding excessive expansion of the range, thus ensuring the effect of subsequent fitting.

[0020] In either method 1 or method 2 described above, the position of the corresponding area in the first image is the same as the position of the main area in the image captured by the camera. This allows the user to adjust the image in real-time based on the content in the preview frame, directly affecting the final fit and achieving the user's goal of adjusting the group photo effect in real time.

[0021] In either method 1 or method 2 above, the size and shape of the co-location area in the first image and the size and shape of the main area in the image captured by the camera may be the same or different.

[0022] In conjunction with the first aspect, in some embodiments, the second image is the highest quality frame among the composite images obtained by fitting multiple frames captured by the camera to the first image; the multiple frames captured by the camera include images captured by the camera at the moment the electronic device receives the second operation and at moments prior to that moment, and the number of multiple frames is preset by the electronic device. This ensures a high-quality composite photo effect.

[0023] In conjunction with the first aspect, in some embodiments, the second image is a composite image obtained by fitting the image captured by the camera at the moment the second operation is received with the first image.

[0024] In conjunction with the first aspect, in some embodiments, before the electronic device receives a first operation on the first image, the method further includes: the electronic device receiving a user operation applied to the first control; the electronic device displaying a second control in the first user interface; and the first operation including a user operation applied to the second control.

[0025] In conjunction with the first aspect, in some embodiments, after the electronic device stores the second image, it can receive a third operation to view the second image and then display it. Essentially, after taking a photo of the first and second objects together, the user can also view the photo on the electronic device.

[0026] Secondly, embodiments of this application provide a shooting method applied to an electronic device. The method may include: the electronic device launching a gallery application and displaying a first user interface, the first user interface displaying a first image, the first image containing a first object; the electronic device receiving a first operation on the first image; the electronic device launching a camera application and displaying a second user interface, the second user interface displaying a preview box, the preview box displaying a composite image, the composite image being obtained by fitting an image captured by the electronic device's camera and the first image, the image captured by the camera containing a second object; the electronic device receiving a second operation for taking a picture; the electronic device storing a second image, the second image being the composite image displayed in the preview box when the second operation is received, and the second image containing both the first and second objects.

[0027] Implementing the method provided in the second aspect, electronic devices can access the camera application through a gallery app, displaying the group photo effect to the user in real-time within a preview window. This allows the user to more intuitively experience the group photo effect and complete the photo of the first and second subjects without the aid of other equipment or personnel. Furthermore, during the photo-taking process, the relative positions between the first and second subjects can be previewed and adjusted in real-time, and the shooting distance and the camera used can be flexibly adjusted to achieve high-quality group photo results. Moreover, this method is easy to operate, quick, convenient, and supports continuous and large-scale group photo shooting.

[0028] In conjunction with the second aspect, in some implementations, the first object or the second object is any one of the following: a person, an animal, a landmark, a building, or a doll.

[0029] In conjunction with the second aspect, in some implementations, the number of either the first object or the second object can be multiple.

[0030] In conjunction with the second aspect, in some implementations, the electronic device can fit the composite image displayed in the preview frame in any of the following ways:

[0031] Method 1: Fit the region where the second object is located in the image captured by the camera to the first image to obtain the second image.

[0032] Method 2 involves fitting the region containing the first object in the first image to the image captured by the camera to obtain the second image.

[0033] For details on the specific implementation and technical effects of methods 1 and 2, please refer to the generation method of the second image in the first aspect, which will not be repeated here.

[0034] In conjunction with the second aspect, in some embodiments, a first control is displayed in the first user interface. Before the electronic device receives the first operation on the first image, it can also receive a user operation applied to the first control and display a second control in the first user interface; the first operation includes a user operation applied to the second control.

[0035] In conjunction with the second aspect, in some embodiments, after the electronic device stores the second image, it can receive a third operation to view the second image and then display it. Essentially, after taking a photo of the first and second objects together, the user can also view the photo on the electronic device.

[0036] Thirdly, embodiments of this application provide a shooting method applied to an electronic device. The method may include: the electronic device displaying a second user interface, the second user interface displaying a preview box and a third control, the preview box displaying an image captured by the electronic device's camera; the electronic device detecting a fourth operation applied to the third control; the electronic device displaying one or more images; the electronic device detecting a fifth operation applied to a first image among the one or more images, the first image containing a first object; the electronic device displaying the image captured by the electronic device's camera and indication information of the first image in the preview box of the second user interface; the electronic device receiving a second operation for taking a picture; the electronic device storing a second image, the second image being obtained by fitting the image captured by the camera and the first image, the image captured by the camera containing a second object, and the second image containing both the first object and the second object.

[0037] Implementing the method provided in the third aspect, the electronic device can activate the group photo function through a preview interface, and can complete a group photo of the first and second subjects without the aid of other equipment or personnel. Furthermore, during the photo-taking process, the relative positions between the first and second subjects can be previewed and adjusted in real time. The shooting distance and the camera used can also be flexibly adjusted to achieve high-quality group photo results. Moreover, this method is easy to operate, quick, convenient, and supports continuous and large-scale group photo shooting.

[0038] In conjunction with the third point, the second user interface (i.e., the preview interface) can be provided by the camera app or by other apps calling the camera interface. These other apps could be social media apps, shopping apps, etc. Essentially, the duet function can be launched through the camera app or other apps.

[0039] The indications for the first object, the second object, and the first image in the third aspect can all be referenced from the relevant descriptions in the first aspect.

[0040] In conjunction with the third aspect, in some implementations, the method and technical effect of the electronic device fitting the second image can be referred to the relevant description in the first aspect.

[0041] In conjunction with the third aspect, in some embodiments, the second image is the highest quality frame among the composite images obtained by fitting multiple frames captured by the camera to the first image; the multiple frames captured by the camera include images captured by the camera at the moment the electronic device receives the second operation and at moments prior to that moment, and the number of multiple frames is preset by the electronic device. This ensures a high-quality composite photo effect.

[0042] In conjunction with the third aspect, in some embodiments, the second image is a composite image obtained by fitting the image captured by the camera at the moment the second operation is received with the first image.

[0043] In conjunction with the third aspect, in some embodiments, after the electronic device stores the second image, it can receive a third operation to view the second image and then display it. Essentially, after taking a photo of the first and second objects together, the user can also view the photo on the electronic device.

[0044] Fourthly, embodiments of this application provide a shooting method applied to an electronic device. The method may include: the electronic device displaying a second user interface, the second user interface displaying a preview box and a third control, the preview box displaying an image captured by the electronic device's camera; the electronic device detecting a fourth operation applied to the third control; the electronic device displaying one or more images; the electronic device detecting a fifth operation applied to a first image among the one or more images, the first image containing a first object; the electronic device displaying a composite image in the preview box of the second user interface, the composite image being obtained by fitting the image captured by the electronic device's camera and the first image, the image captured by the camera containing a second object; the electronic device receiving a second operation for taking a picture; and the electronic device storing a second image, the second image being the composite image displayed in the preview box when the second operation is received, and the second image containing both the first and second objects.

[0045] Implementing the method provided in the fourth aspect, electronic devices can initiate the group photo function through a preview interface. The preview window displays the group photo effect to the user in real time, allowing for a more intuitive understanding of the result. This enables the creation of a group photo between the first and second subjects without the aid of other equipment or personnel. Furthermore, during the photo-taking process, the relative positions of the first and second subjects can be previewed and adjusted in real time. The shooting distance and the camera used can also be flexibly adjusted to achieve high-quality group photo results. Moreover, this method is easy to operate, quick, convenient, and supports continuous and large-scale group photo shooting.

[0046] In conjunction with the fourth aspect, in some implementations, the second user interface (i.e., the preview interface) can be provided by the camera application or by other applications calling the camera interface. These other applications could be, for example, social media applications, shopping applications, etc. Essentially, the duet function can be initiated through the camera application or other applications.

[0047] In conjunction with the fourth aspect, in some implementations, the first or second object is any one of the following: a person, an animal, a landmark, a building, or a doll.

[0048] In conjunction with the fourth aspect, in some implementations, the number of either the first object or the second object can be multiple.

[0049] In conjunction with the fourth aspect, in some implementations, the electronic device can fit the composite image displayed in the preview frame in any of the following ways:

[0050] Method 1: Fit the region where the second object is located in the image captured by the camera to the first image to obtain the second image.

[0051] Method 2 involves fitting the region containing the first object in the first image to the image captured by the camera to obtain the second image.

[0052] For details on the specific implementation and technical effects of methods 1 and 2, please refer to the generation method of the second image in the first aspect, which will not be repeated here.

[0053] In conjunction with the fourth aspect, in some embodiments, after the electronic device stores the second image, it can receive a third operation to view the second image and display it. Essentially, after taking a photo of the first and second objects together, the user can also view the photo on the electronic device.

[0054] Fifthly, embodiments of this application provide an electronic device, including: a memory and one or more processors; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and one or more processors call the computer instructions to cause the electronic device to perform a method as performed by the electronic device in any of the first, second, third, or fourth aspects.

[0055] In a sixth aspect, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform a method as described in any of the first, second, third, or fourth aspects.

[0056] In a seventh aspect, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform a method as implemented by any of the electronic devices described in the first, second, third, or fourth aspects.

[0057] Eighthly, embodiments of this application provide a chip system including at least one processor for implementing a method as performed by an electronic device as described in the first, second, third, or fourth aspects.

[0058] By implementing the technical solution provided in this application, an electronic device can take a group photo of a first object and a second object without the aid of other equipment or personnel. Furthermore, during the photo-taking process, the relative positions between the first and second objects can be previewed and adjusted in real time. The shooting distance and the camera used can also be flexibly adjusted to achieve a high-quality group photo effect. Moreover, this method is easy to operate, quick, convenient, and supports continuous and large-scale group photo shooting. Attached Figure Description

[0059] Figure 1A A schematic diagram of the hardware structure of the electronic device 100 provided in the embodiments of this application;

[0060] Figure 1B A schematic diagram of the software structure of an electronic device 100 provided in an embodiment of this application;

[0061] Figure 1C A schematic diagram of another software structure of the electronic device 100 provided in an embodiment of this application;

[0062] Figures 2A-2H This application provides an electronic device 100 with a set of user interfaces for taking group photos.

[0063] Figures 3A-3FThis is another set of user interfaces for taking group photos provided by the electronic device 100 in the embodiments of this application;

[0064] Figures 4A-4B The user interface provided by the gallery application after the electronic device 100 in this application saves the fitted image;

[0065] Figure 5 A flowchart illustrating the shooting method provided in the embodiments of this application;

[0066] Figures 6A-6B Examples of several image processing procedures provided for embodiments of this application. Detailed Implementation

[0067] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0068] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0069] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0070] The following embodiments of this application provide a shooting method and related apparatus. In this shooting method, an electronic device can activate a "combined shot" function for a selected image containing object A, then launch a camera application, displaying the image with transparency or blur in the preview interface provided by the camera application, and simultaneously displaying the image captured by the electronic device's camera. After the image captured by the camera includes object B, the electronic device can receive a photo-taking command input by the user, and then fit the image with the image captured by the camera to generate a composite image that simultaneously contains object A and object B.

[0071] The objects A or B mentioned above may include, but are not limited to: people, animals, landmarks, buildings, dolls, etc. There may be multiple objects A or objects B.

[0072] The above-described shooting method can achieve a group photo of subject A and subject B. This method allows for taking a group photo without the aid of other equipment or personnel, and during the process, the relative positions of subject B and subject A can be previewed and adjusted in real time. The shooting distance and the camera used can also be flexibly adjusted to achieve high-quality group photo results. Furthermore, this method is easy to operate, quick, convenient, and supports continuous and large-scale group photo shooting.

[0073] The above shooting method can be applied to scenarios where group photos are needed, including but not limited to: multiple group photos of the same person at the same location; group photos of new people and people from old photos, and group photos of multiple people at the same location at different times.

[0074] In some implementations, the electronic device can extract the region containing object B from an image captured by a camera and fit that region onto an image containing object A, while the rest of the image remains unchanged, to generate a composite image that simultaneously contains object A and object B. Image fitting allows object B to be smoothly and naturally integrated into the corresponding region of the image containing object A, resulting in a natural, high-quality composite image. In other implementations, the electronic device can also extract the region containing object A from an image and fit that region onto an image captured by a camera. For a detailed description of the specific method for generating the composite image, please refer to the following method embodiments.

[0075] In some implementations, the electronic device can select N frames of images captured by the camera up to and including that moment, starting from the moment the photo-taking command is received. Each of these N frames is then fitted to an image containing object A to obtain an N-frame composite image. The highest quality frame is then selected from these N composite images and stored as the final group photo. This method allows for the selection of the optimal frame from multiple composite images, achieving a high-quality group photo effect. Furthermore, this method also considers the time delay between the user seeing the preview of the group photo and the input of the photo-taking command, thus avoiding using the composite image corresponding to the moment the photo-taking command is received as the final group photo effect.

[0076] In some implementations, after the electronic device launches the camera application, it can directly fit the image captured by the camera with the selected image containing object A, and display the composite image directly in the preview interface provided by the camera application.

[0077] In this embodiment, the "combined image" function is provided by an electronic device. This function allows the electronic device to display an image containing object A with transparency or blurring in the preview interface provided by the camera application, while simultaneously displaying an image captured by the electronic device's camera. Upon receiving a user-inputted photo-taking command, the electronic device fits the image to the image captured by the camera, generating a composite image that simultaneously contains object A and object B. Alternatively, the "combined image" function allows the electronic device to directly fit the image captured by the camera to the image containing object A, and directly display the composite image in the preview interface provided by the camera application.

[0078] The term "co-op" is merely a name used in this embodiment, and its meaning has been described in this embodiment. Its name does not constitute any limitation on this embodiment. For example, the "co-op" function can also be referred to as other names, such as "group photo" function, multi-person group photo, multi-person co-op, etc.

[0079] The electronic device provided in the embodiments of this application will be introduced first below.

[0080] The electronic devices provided in this application embodiment are equipped with cameras, and may include, but are not limited to, smartphones, tablets, dedicated cameras (such as SLR cameras and point-and-shoot cameras), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices (such as smartwatches and smart glasses), and personal digital assistants (PDAs). Exemplary embodiments of the electronic devices include, but are not limited to, devices equipped with cameras. Portable electronic devices running Linux or other operating systems. These devices can also be other portable electronic devices, such as laptops.

[0081] Figure 1A A schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application is shown.

[0082] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc.

[0083] The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0084] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0085] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0086] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0087] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0088] In some embodiments, the controller or processor 110, such as the GPU, can be used to process the image containing object A by making it transparent or blurred, and then overlay it with the image captured by the camera 193 to synthesize a preview image displayed in the viewfinder.

[0089] In some embodiments, the processor 110, such as the controller or GPU, can be used to fit the image captured by the camera to the image containing object A to generate a preview image displayed in the viewfinder, so that the group photo effect can be presented to the user. For a detailed description of the fitting operation performed by the processor 110, please refer to the following method embodiments.

[0090] For ease of use, electronic devices typically shoot in handheld mode, which usually results in shaky footage. In some embodiments, the controller or processor 110, such as a GPU, can also be used to perform image stabilization on the image captured by the camera in a collaborative shooting scenario, and then overlay or fit the image with the image containing object A to provide a preview image for display on the screen 194.

[0091] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0092] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0093] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0094] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0095] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0096] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0097] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0098] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be manufactured using organic light-emitting diodes (OLEDs), active-matrix organic light-emitting diodes (AMOLEDs), flexible light-emitting diodes (FLEDs), miniled, microled, micro-OLEDs, quantum dot light-emitting diodes (QLEDs), etc. In some embodiments, the electronic device may include one or N displays 194, where N is a positive integer greater than 1.

[0099] In some implementations, in a collaborative shooting scenario, the display screen 194 can display a preview interface provided by the camera application, and display an image containing object A with transparency or blurring in the preview interface, while simultaneously displaying the image captured by the camera 193 of the electronic device.

[0100] In other embodiments, in a group photo scenario, the display screen 194 can display a composite image obtained by the processor 110 fitting the image captured by the camera and the image containing object A, so that the group photo effect can be presented to the user.

[0101] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0102] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin texture. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 193.

[0103] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0104] Electronic device 100 may have multiple cameras 193, such as front-facing cameras and rear-facing cameras. The number of front-facing cameras and rear-facing cameras may be multiple. The types of cameras 193 may include, but are not limited to, wide-angle cameras, ultra-wide-angle cameras, and telephoto cameras.

[0105] The hardware configuration and physical location of the camera 193 may vary, therefore, the size, range, content or clarity of the images captured by different cameras may differ.

[0106] The output image size of a camera (e.g., 193) can vary or be the same. The output image size refers to the length and width of the image captured by the camera. Both the length and width of the image can be measured in pixels. The output image size can also be called image size, image dimensions, pixel dimensions, or image resolution. Common camera aspect ratios include 4:3, 16:9, or 3:2, etc. The aspect ratio refers to the approximate ratio of the number of pixels in the length and width of the image captured by the camera.

[0107] Camera 193 can correspond to the same focal length or different focal lengths. The larger the focal length of a camera, the smaller its field of view (FOV). The field of view refers to the range of angles that an optical system can image.

[0108] Camera 193 can be mounted on both sides of the electronic device. A camera that is on the same plane as the display screen 194 of the electronic device can be called a front-facing camera, and a camera that is on the same plane as the back cover of the electronic device can be called a rear-facing camera. The front-facing camera can be used to capture images of the photographer facing the display screen 194, and the rear-facing camera can be used to capture images of the subject (such as people, landscapes, etc.) facing the photographer.

[0109] In some embodiments, camera 193 can be used to acquire depth data. For example, camera 193 may have a time-of-flight (TOF) 3D sensing module or a structured light 3D sensing module for acquiring depth information. The camera used to acquire depth data can be a front-facing camera or a rear-facing camera.

[0110] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0111] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0112] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0113] In some implementations, in a combined photography scenario, the processor 110 (e.g., a controller or GPU) can fit the image captured by the camera to an image containing object A, and store the fitting result in memory. The specific implementation of the fitting operation performed by the processor 110 can be found in the detailed description of the subsequent method embodiments. In this way, the resulting composite image simultaneously contains object A and object B captured by the camera, achieving a combined photo of multiple objects.

[0114] In some implementations, the processor 110 (e.g., a controller or GPU) can fit the N frames of images captured by the camera 193 at the moment the electronic device 100 receives the photo-taking command to the image containing object A, and then select the frame with the best quality from the N frames of composite images and store it in the memory as the composite image result.

[0115] In some implementations, in a combined shooting scenario, the processor 110 (e.g., a controller or GPU) can fit multiple frames of images captured by the camera to an image containing object A, and store the fitting results in memory. For example, by fitting multiple frames of images captured by the camera and an image containing object A into multiple images, the video encoder in the processor 110 can encode the multiple composite images to generate a video file. In this way, each frame of the video file can contain object A and the object in the newly captured image from the camera.

[0116] Internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). Internal memory 121 is used to store the fitting results of processor 110.

[0117] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and double data rate synchronous dynamic random access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM). Non-volatile memory can include disk storage devices and flash memory.

[0118] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0119] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0120] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0121] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0122] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0123] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0124] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered mobile operating system as an example to exemplify the software structure of electronic device 100.

[0125] Figure 1B This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0126] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, a mobile operating system is divided into four layers, from top to bottom: the application layer, the application framework / core service layer, the underlying libraries and runtime, and the kernel layer.

[0127] The application layer can include a series of application packages.

[0128] like Figure 1B As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0129] The program framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The program framework layer includes some predefined functions.

[0130] like Figure 1B As shown, the program framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0131] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0132] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0133] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0134] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0135] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0136] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0137] Runtime can refer to all the code libraries, frameworks, etc., required for a program to run. For example, for the C language, the runtime includes a series of function libraries required for C programs to run. For the Java language, in addition to the core libraries, the runtime also includes the virtual machine required for Java programs to run. The aforementioned core libraries can include the functionalities that the Java language needs to call.

[0138] The underlying library can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0139] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0140] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0141] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0142] A 2D graphics engine is a graphics engine for 2D drawing.

[0143] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0144] The following example, using a scene of capturing a photograph, illustrates the workflow of the electronic device's software and hardware.

[0145] When touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking a touch click as an example, where the corresponding control is the camera application icon, the camera application calls the application framework layer's interface to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through camera 193.

[0146] Figure 1C This is another software structure block diagram of the electronic device 100 provided in the embodiments of this application.

[0147] like Figure 1CAs shown, the electronic device 100 includes: a gallery module, a camera module, a display module, a recognition module, and a synthesis module. Among them:

[0148] The image library module is used to store multiple images, including one image containing object A. The image library module can also provide an entry point to enable the "combination" function and can be used to store the fitting results of electronic device 100.

[0149] The camera module is responsible for capturing images.

[0150] The display module is used to display the image with transparency or blurring, and simultaneously display the image captured by the camera module. Alternatively, the display module is used to display a composite image derived from the image captured by the camera module and the image containing object A.

[0151] The recognition module is used to identify the area where object B is located in an image captured by the imaging module, as well as the background area other than object B. Alternatively, the recognition module is used to identify the area where object A is located in an image containing object A, as well as the background area other than the area where object A is located.

[0152] The compositing module is used to fit the region containing object B in the image captured by the camera module to the image containing object A. Alternatively, the compositing module is used to fit the region containing object A in the image containing object A to the image captured by the camera module.

[0153] The user interface provided in the embodiments of this application is described below.

[0154] Figures 2A-2H An exemplary illustration shows a set of user interfaces for taking group photos provided by an electronic device 100 in an embodiment of this application.

[0155] Figure 2A An exemplary user interface 21 for displaying installed applications is shown on an electronic device 100.

[0156] The user interface 21 displays: a status bar 201, a calendar and time indicator 202, a weather indicator 203, a page indicator 204, a tray icon with icons for commonly used applications 205, and other application icons. Among them:

[0157] The status bar 201 may include: one or more signal strength indicators for mobile communication signals (also known as cellular signals), Bluetooth indicators, one or more signal strength indicators for Wi-Fi signals, battery status indicators, time indicators, etc.

[0158] The calendar and time indicator 202 are used to indicate the calendar and current time. The weather indicator 203 is used to indicate the weather.

[0159] Page indicator 204 can be used to indicate which page the user is currently viewing the application icon on. In this embodiment, application icons can be distributed across multiple pages, and users can swipe left and right to browse application icons on different pages.

[0160] The tray 205 can display icons for commonly used applications, such as: phone icon, SMS icon, camera icon 205a, contacts icon, etc.

[0161] Other application icons may include icons for video applications, online conferencing applications, calendar applications, game applications, map applications, settings applications, gallery applications, etc.

[0162] In this embodiment, the camera app is an application on the electronic device 100 used to access the camera and capture images. The name of this application is not limited in this embodiment. The camera app in this application also provides a "co-op" function. The camera app can be a system application provided by the electronic device 100 or a third-party application.

[0163] The gallery is an image management app on electronic devices such as smartphones and tablets 100, also known as a "photo album." This embodiment does not limit the name of the application. The app allows users to perform various operations on images stored on the electronic device 100, such as browsing, editing, deleting, and selecting. In other words, the "gallery" manages images. In other cases, the app may also allow users to perform the aforementioned operations on images stored on a cloud server. It is understood that in this embodiment, the images may be captured by the electronic device 100 using the camera 193, obtained from other applications, or downloaded from a webpage. This embodiment of the gallery also provides an entry point for a "duet" function. The gallery can be a system application provided by the electronic device 100 or a third-party application.

[0164] Beyond this, the electronic device 100 can also install more applications, and the icons of these applications can be displayed on the screen. For example, the electronic device 100 can also install shopping applications, ticket booking applications, and so on. Shopping applications can be used to purchase items online. Ticket booking applications can be used to order various tickets, such as airline tickets, train tickets, ship tickets, and so on.

[0165] The names of the above applications are merely terms used in the embodiments of this application, and their meanings have been recorded in this embodiment. Their names do not constitute any limitation on this embodiment.

[0166] Not limited to this, Figure 2A The user interface 21 shown may also include a navigation bar, a sidebar, etc. In some embodiments, Figure 2A The user interface 21 shown in the example can be referred to as the home screen.

[0167] Figure 2B An example is shown of the user interface 22 provided by the gallery app of the electronic device 100.

[0168] Figure 2B It can be detected by electronic device 100 acting on Figure 2A The user interface displayed in response to a user action (such as a click or touch) on icon 206a in the China Library application.

[0169] like Figure 2B As shown, the user interface 22 may include: a status bar, an application title bar, and an image area 207. Wherein:

[0170] For reference, please refer to the status bar. Figure 2A The status bar 201 in the user interface 21 shown here will not be described in detail here.

[0171] The application title bar may include a back button and a current page indicator. The back button is an app-level back button, which can be used to return to the previous menu level. Those skilled in the art will understand that the logical previous level of a page is fixed and determined during application design. The current page indicator can be used to indicate the current page, such as the text information "Gallery," but it is not limited to text information; the current page indicator can also be an icon.

[0172] Image area 207 may display one or more images, such as image 207a. When the electronic device detects an up or down swipe operation in image area 207, in response to the swipe operation, the electronic device can update the image displayed in image area 207 so that the user can browse the images. That is, the user can swipe up or down in image area 207 to browse more images. Not limited to up or down swipe operations, the user can also swipe left or right in image area 207 to browse more images. The images displayed in image area 207 may be thumbnails. At this time, the original image corresponding to the image displayed in image area 207 may be stored on the electronic device or on a cloud server. Unless otherwise stated, the images referred to in the following embodiments may be stored on the electronic device or on a cloud server.

[0173] like Figure 2B As shown, the electronic device 100 can detect user actions (such as clicks, touches, etc.) on image 207a in the user interface 22. In response to this action, the electronic device 100 can display, as shown... Figure 2C The user interface shown is 23.

[0174] like Figure 2C As shown, the electronic device 100 can display the original image corresponding to image 207a in a larger area on the user interface 23. The user interface 23 displays image 208, which can be the original image of image 207a. In some embodiments, image 208 can be a photo taken by the electronic device 100 at a location (e.g., location 1), or it can be obtained by the electronic device 100 from the network. Image 208 contains a person object and a background area. The person object can be referred to as object A, and image 208 can be referred to as an image containing object A. The background area shows the location of object A, for example... Figure 2C The background area of ​​image 208 shows coconut trees, seagulls, a hillside, and the sun. The user interface 23 may also display: a share control, a favorite control, an edit control, a delete control, and more controls 209. The share control, favorite control, edit control, and delete control are used to receive user operations, and the electronic device 100 can respond to these user operations by performing operations such as sharing, favorite, editing, and deleting on image 208. More controls 209 can be used to receive user operations (such as click operations, touch operations, etc.), and the electronic device 100 can respond to these user operations by displaying... Figure 2D The function list shown is 210.

[0175] like Figure 2D As shown, the function list 210 displays multiple function controls, such as controls for copying image 208 to the album, moving it to the album, casting it to the screen, printing it, etc. Among the multiple function controls displayed in the function list 210 is a control 210a for enabling the "dual-image" function for image 208. This control 210a is used to receive user operations (such as click operations, touch operations, etc.), and the electronic device 100 can respond to the user operation and start the "dual-image" function for image 208.

[0176] Through the above Figures 2B-2D For example, a user can click on an image to view it, and then activate the "dual photo" feature for that image through the entry point provided by the user interface that displays the image.

[0177] Not limited to clicking control 210a in a user interface displaying an image, electronic device 100 can also initiate a "combination" function for images in response to other user actions. For example, electronic device 100 can also... Figure 2B Upon receiving a long-press operation on the displayed image 207a, the "dual-photo" function for that image is activated. Alternatively, the electronic device 100 can also... Figure 2B Upon receiving a circle gesture on the displayed image 207a, the "dual-image" function for that image is activated. Alternatively, the electronic device 100 can also... Figure 2C Upon receiving a long press operation on the image 208 shown, the "dual-image" function for that image is activated. Alternatively, the electronic device 100 can also be controlled by voice commands. This application does not limit the specific implementation of the operation that triggers the electronic device 100 to activate the "dual-image" function for the image.

[0178] The "co-op" feature for images is not limited to images in the "Gallery". Other scenarios in which users can trigger the "co-op" feature for images include when users view images on the internet, when users trigger the "co-op" feature for images in applications such as file browsers, and so on.

[0179] Figures 2E-2H An example is shown of a set of user interfaces provided by the camera application after the electronic device 100 activates the "dual capture" function for image 208.

[0180] After electronic device 100 activates the "combination shot" function for image 208, it can automatically launch the camera application.

[0181] Figure 2E The user interface 24 displayed after launching the camera application on electronic device 100.

[0182] Figure 2E The exemplary user interface 24 can be detected by the electronic device 100 and act upon, for example... Figure 2D The user interface 24 is displayed after a user action (such as a click, touch, etc.) on the control 210a shown. The user interface 24 can be the user interface for the default shooting mode of the camera application, which can be used by the user to take photos using the default rear camera.

[0183] like Figure 2E As shown, the user interface 24 may include: area 211, shooting mode list 212, control 213, control 214 and control 215.

[0184] in:

[0185] Area 211 can be called preview frame 211 or viewfinder 211. Preview frame 211 can be used to display an image 208 with a certain degree of transparency or blurring.

[0186] After the camera 193 of the electronic device is activated and captures an image, the preview box 211 can also display the image captured in real time by the camera 193. The electronic device 100 can refresh the displayed content in real time so that the user can preview the image currently captured by the camera 193. The electronic device 100 can overlay an image 208 with a certain degree of transparency or blurring on the image captured in real time by the camera 193. Because the image 208 has a certain degree of transparency and / or blurring, the user can see not only the image 208 in the preview box 211, but also the image captured in real time by the camera 193 below the image 208.

[0187] In some implementations, after the electronic device 100 activates the "dual-shot" function and launches the camera application, it can initially display only an image 208 with a certain degree of transparency or blurring in the preview frame 211, such as... Figure 2E As shown, then the camera 193 is activated and the real-time captured image is also displayed in the preview box 211, as shown. Figure 2F As shown.

[0188] In other embodiments, after the electronic device 100 activates the "dual-image" function and launches the camera application, it can directly start the camera 193 to capture images. At this time, the electronic device 100 can simultaneously display the image 208 with a certain degree of transparency or blurring and the image captured in real time by the camera 193 in the preview frame 211. That is to say, the electronic device 100 receives... Figure 2E After a user operation is performed on control 210a, it can be displayed directly. Figure 2F The user interface shown does not need to be displayed. Figure 2E The user interface shown.

[0189] The shooting mode list 212 can display one or more shooting mode options. These options may include: photo mode 212A, video mode 212B, portrait mode 212C, panorama mode 212D, and more options 212E. These options can be displayed as text information on the interface, such as "Photo," "Video," "Portrait," "Panorama," and "More." Alternatively, they can be displayed as icons or other interactive elements (IE).

[0190] Control 213 can be used to receive user operations that trigger shooting (taking a photo or recording a video). The electronic device can detect user operations performed on control 213 (such as a click operation on control 213). In response to this operation, the electronic device 100 can save the image captured by camera 193 and perform a fitting operation with image 208 to generate an image in the gallery. When the user switches to video recording mode, control 213 can be changed to the corresponding video recording control. The electronic device can detect user operations performed on the video recording control. In response to this operation, the electronic device 100 can save multiple frames of images captured by camera 193 and perform fitting operations with image 208 respectively to generate a video in the gallery. In addition, the electronic device 100 can also display a thumbnail of the saved image in control 214. This image can be generated by the electronic device 100 after the "co-shoot" function is enabled, or it can be generated when the "co-shoot" function is not enabled. That is to say, the user can click control 213 or the video recording control to trigger shooting. The control 213 or the video recording control can be a button or other form of control. In this application, control 213 can be referred to as a photo capture control. The photo capture control and the video recording control can be collectively referred to as the shooting control.

[0191] Control 215 can be used to receive user actions that trigger the camera to flip. Electronic device 100 can detect user actions (such as a click on control 215) applied to control 215, and in response to such actions, electronic device 100 can flip the camera, for example, switch the rear camera to the front camera.

[0192] The electronic device 100 can detect user operations applied to the shooting mode options, which can be used to select a shooting mode. In response to this operation, the electronic device 100 can activate the user-selected shooting mode. Specifically, when the user operation is applied to more shooting mode options 212E, the electronic device 100 can further display more other shooting mode options, such as a slow-motion shooting mode option, etc., showcasing richer video recording functions to the user. (Not limited to...) Figure 2E As shown, the shooting mode list 212 may not display more shooting mode options 212E. Users can browse other shooting mode options by swiping left / right in the shooting mode list 212.

[0193] like Figure 2F As shown, the preview box 211 displays both a lighter-colored image and a darker-colored image. The lighter-colored image is a picture 208 with a certain degree of transparency or blurring, while the darker-colored image is an image captured in real time by the camera 193.

[0194] In the preview frame 211, the position of the image 208, which has a certain degree of transparency or is blurred, remains unchanged. Meanwhile, when the user takes a picture using the electronic device 100, they can change the framing of the currently used camera in the preview frame by moving the electronic device. That is, the user can change the framing of the camera 193 by moving the electronic device, thereby changing the image captured in real-time by the camera 193 displayed in the preview frame 211. For example, the user can move the electronic device left, right, forward, or backward, or even walk while holding the electronic device. In some other embodiments, the user can also adjust the focal length of the currently used camera (e.g., electronic focus), or change the currently used camera (e.g., replacing the rear camera with a front camera), thereby changing the image captured in real-time by the camera displayed in the preview frame 211. In some other embodiments, the position of the person, animal, or other object captured by the camera 193 can also be changed, thereby adjusting the framing in the preview frame 211.

[0195] The preview box 211 can show the user the relative positions of various objects in the image 208 and the real-time image captured by the camera 193. Based on the content presented in the preview box 211, the user can adjust the real-time image captured by the camera to ensure that object B is included in the image and that object B is positioned relative to object A in image 208 in a way that satisfies the user. The object that needs to be photographed with object A is called object B.

[0196] like Figure 2F As shown, image 208 contains object A and a background area, where the background area indicates the location of object A, for example... Figure 2F The background area of ​​image 208 shown in preview box 211 indicates that object A is located in a location with coconut trees, seagulls, a hillside, and the sun.

[0197] In some implementations, control 213 can only be activated when the position indicated by the background area of ​​the real-time image captured by the camera displayed in preview box 211 and the position indicated by image 208 (e.g., but not limited to the position indicated by the background area of ​​image 208) are correlated. In other cases, even if control 213 receives a user operation, electronic device 100 will not execute a subsequent response. After control 213 is activated, the user can click control 213 when the image captured by the camera contains object B and object B is in a satisfactory relative position to object A in image 208, thereby triggering electronic device 100 to save the image captured by camera 193 and perform a fitting operation with image 208. That is, only when the position indicated by image 208 and the position indicated by the real-time image captured by camera 193 of electronic device 100 are correlated can the user trigger electronic device 100 to capture an image and perform a subsequent fitting operation.

[0198] In some other embodiments, control 213 can be kept active as long as the user allows the image captured by the camera to contain object B and the object B to be in a position relative to object A in image 208 that satisfies the user. The user can then click control 213 to trigger the electronic device 100 to save the image captured by camera 193 and fit it to image 208.

[0199] refer to Figure 2F ,exist Figure 2F The image captured by camera 193 contains a person object, which can be object B. Due to the user's... Figure 2F The user was not satisfied with the relative positions of objects B and A, so the camera 193 in the preview frame 211 was adjusted to align objects B and A as shown in the image. Figure 2G The relative position shown is satisfactory.

[0200] like Figure 2G As shown, the electronic device 100 can detect user operations (such as clicks, touches, etc.) applied to the control 213. In response to this operation, the electronic device 100 can save the image captured by the camera 193 and perform a fitting operation with the image 208 to generate an image from the image library. The specific implementation of this fitting operation can be found in the detailed description in the subsequent method embodiments. Furthermore, refer to... Figure 2H The electronic device 100 can also display a thumbnail of the image generated after the fitting operation in the control 214 for the user to click and view.

[0201] In some implementations, when the electronic device 100 fits the image captured by the camera 193 to the image 208, the fitting may fail, and a corresponding composite image cannot be generated. For example, when the background area of ​​the image captured by the camera 193 has a low correlation with the image 208, the electronic device 100 will not be able to successfully perform the fitting operation.

[0202] After the electronic device 100 takes a picture and performs a fitting operation in response to a user operation on the control 213, the electronic device 100 can keep the "co-shoot" function enabled, allowing the user to take multiple pictures and achieve the purpose of multiple group photos. In some other embodiments, after the electronic device takes a picture and performs a fitting operation in response to a user operation on the control 213, it can also turn off the "co-shoot" function and enter the normal shooting mode.

[0203] Not limited to Figures 2E-2G In addition to the photo-taking mode shown, users can also select video recording mode or other modes. The electronic device 100 can also take photos in video recording mode or other modes to achieve the purpose of taking group photos in video or other modes.

[0204] Figures 3A-3F Another set of user interfaces for taking group photos is provided by the electronic device 100 in an embodiment of this application, which is exemplarily shown.

[0205] Figures 3A-3E and Figures 2A-2G The difference lies in the different entry points for the "group photo" function in the two photo-taking methods.

[0206] Figure 3A The user interface 31 shown is Figure 2A The user interface 21 shown is the same as described above. Figure 2A And related descriptions.

[0207] Figure 3B An example is shown of the user interface 32 provided by the camera application of the electronic device 100.

[0208] Figure 3B It can be detected by electronic device 100 acting on Figure 3A Following a user action (such as a click or touch) to the camera app icon 205a, the user interface 32 is launched and displayed in response to that user action. The user interface 32 can be the default shooting mode interface for the camera app, which can be used by the user to take photos using the default rear camera.

[0209] like Figure 3B As shown, the user interface 32 may include: area 301, shooting mode list 302, control 303, control 304 and control 305, and control 306 for activating the "duet" function. Among them:

[0210] Area 301 can be referred to as preview frame 301 or viewfinder 301. Preview frame 301 can be used to display images captured in real time by camera 193 of electronic device 100. Electronic device 100 can refresh the displayed content in real time so that the user can preview the image currently captured by camera 193.

[0211] The shooting mode list 302 may display one or more shooting mode options. These one or more shooting options may include: photo mode option 302A, video mode option 302B, portrait mode option 302C, panorama mode option 302D, and more options 302E.

[0212] Control 303 can be used to receive user operations that trigger shooting (taking a photo or recording a video). The electronic device can detect user operations performed on control 303 (such as a click operation on control 303), and in response to the operation, the electronic device 100 can save the image displayed in preview frame 211. When the user switches to video recording mode, control 303 can be changed to the corresponding video recording control, and the electronic device can detect user operations performed on the video recording control, and in response to the operation, the electronic device 100 can save the image displayed in preview frame 211. In addition, the electronic device 100 can also display a thumbnail of the saved image in control 304. That is to say, the user can click control 303 or video recording control to trigger shooting. Control 303 or video recording control can be a button or other form of control. In this application, control 303 can be referred to as a photo-taking control. The photo-taking control and video recording control can be collectively referred to as shooting controls.

[0213] Control 305 can be used to receive user actions that trigger the camera to rotate. Electronic device 100 can detect user actions (such as a click on control 305) acting on control 305, and in response to the action, electronic device 100 can rotate the camera, for example, switching the rear camera to the front camera.

[0214] The electronic device 100 can detect user operations on the shooting mode option, which can be used to select a shooting mode. In response to the operation, the electronic device 100 can activate the shooting mode selected by the user.

[0215] Control 306 can be used to receive user operations (such as click operations, touch operations, etc.), and electronic device 100 can respond to the user operation by displaying, for example, in user interface 33. Figure 3CThe image area 307 is shown. Image area 307 may display one or more images, such as image 307a. When the electronic device detects an up or down swipe operation in image area 307, in response to the swipe operation, the electronic device can update the image displayed in image area 307 so that the user can browse the images. That is, the user can swipe up or down in image area 307 to browse more images. Not limited to up or down swipes, the user can also swipe left or right in image area 307 to browse more images. The images displayed in image area 307 can be thumbnails. In this case, the original image corresponding to the image displayed in image area 307 can be stored on the electronic device or on a cloud server.

[0216] Image area 307 may also display a prompt message 307b. Prompt message 307b is used to prompt the user to select one of the images displayed in image area 307 as a group photo.

[0217] The electronic device 100 can detect user operations (such as clicks, touches, etc.) applied to the image 307a, and in response to the operation, activate the "combination" function for the original image corresponding to the image 307a, and display it. Figure 3D The user interface shown is 34.

[0218] like Figure 3D As shown, after activating the "Dual Concert" function, the electronic device 100 can change the display format of the control 306 (e.g., it can change its display color, change its display shape, or so on). Figure 3D (e.g., changing hollow circles to solid circles) to indicate to the user that the current electronic device 100 has the "synchronization" function enabled.

[0219] exist Figure 3D In the user interface 34 shown, after the electronic device 100 activates the "dual-image" function, the preview box 301 displays an image 208 with a certain degree of transparency or blurring, as well as an image captured in real time by the camera 193. Image 208 can be... Figure 3C The image shown is the original image corresponding to image 307a. The electronic device 100 can refresh the displayed content in real time so that the user can preview the image currently captured by the camera 193. The electronic device 100 can overlay an image 208 with a certain degree of transparency or blurring onto the image captured in real time by the camera 193. Because image 208 has a certain degree of transparency and / or blurring, the user can see not only image 208 in the preview frame 211, but also the image captured in real time by the camera 193 below image 208.

[0220] Figure 3DThe functions of the preview box 301, the shooting mode list 302, and the controls 304-305 in the user interface 34 can all be found in the following references. Figure 2F The function of the corresponding control in the middle. It can be considered that... Figure 3D and Figure 2G Apart from the difference in control 306, all other areas and controls are the same.

[0221] refer to Figure 3D ,exist Figure 3D The image captured by camera 193 contains a person object, which can be object B. Due to the user's... Figure 3D The user was not satisfied with the relative positions of objects B and A, so the user adjusted the framing of the camera 193 in the preview frame 301, thereby adjusting objects B and A in the preview frame 301 to be as shown. Figure 3E The relative position shown is satisfactory.

[0222] like Figure 3E As shown, the electronic device 100 can detect user operations (such as clicks, touches, etc.) applied to the control 303. In response to this operation, the electronic device 100 can save the image captured by the camera 193 and perform a fitting operation between it and the original image corresponding to image 307a, thereby generating an image from the image library. The specific implementation of this fitting operation can be found in the detailed description in the subsequent method embodiments. Furthermore, refer to... Figure 3F The electronic device 100 can also display a thumbnail of the image generated after the fitting operation in the control 304 for the user to click and view.

[0223] In some implementations, when the electronic device 100 fits the image captured by the camera 193 to the original image corresponding to image 307a, the fitting may fail, and the corresponding composite image cannot be generated. For example, when the background area of ​​the image captured by the camera 193 has a low correlation with the original image corresponding to image 307a, the electronic device 100 will not be able to successfully perform the fitting operation.

[0224] Figures 4A-4B An exemplary illustration shows the user interface provided by the gallery application after the electronic device 100 saves the fitted image in an embodiment of this application.

[0225] Figure 4A It can be detected by electronic device 100 acting on Figure 2A or Figure 3A The user interface displayed in response to a user action (such as a click or touch) on icon 206a in the China Library application. Figure 4A The user interface shown and Figure 2B The user interface 22 shown is similar; please refer to the relevant description. For example... Figure 4AAs shown, a new image 207b has been added to the image area 207 shown in the user interface 41. Image 207b can be a thumbnail or the original image of the fitted image saved by the electronic device 100.

[0226] like Figure 4A As shown, after detecting a user operation (such as a click or touch) applied to image 207b, electronic device 100 can respond to the operation by... Figure 4B The user interface shown displays the original image corresponding to image 207b, such as image 401, in a larger area. In some other embodiments, Figure 4B The user interface 43 shown can also be an electronic device 100 responding to... Figure 2H The control 214 shown, or, in Figure 3F The display is based on user interaction on control 304.

[0227] In some implementations... Figure 4A The image shown is 207b, or... Figure 4B The displayed image 401 may also include a "Double Photo" label, indicating that the image was taken by the electronic device 100 after the "Double Photo" function was enabled. This embodiment of the application does not limit the implementation form of the "Double Photo" label; for example, it can be text, an icon, etc.

[0228] Based on the electronic device 100 and the user interface implemented on the electronic device 100 described above, the shooting method provided in the embodiments of this application is described below.

[0229] refer to Figure 5 , Figure 5 The process of this shooting method is illustrated as an example. Figure 5 The shooting method shown is illustrated using a photographic scene as an example.

[0230] like Figure 5 As shown, the shooting method may include the following steps:

[0231] S101, electronic device 100 detects a first operation to activate the "combination" function for the first image.

[0232] S102, in response to the first operation, the electronic device 100 activates the "combination" function for the first image, the first image containing object A.

[0233] Specifically, the first image can be a picture taken by the camera 193 stored in the electronic device 100, a picture downloaded by the electronic device 100 from the network, or a picture shared by another device. The first image can be a photo taken a long time ago or a photo taken recently by the electronic device 100. The first image contains an object that the user wants to take a picture with, which can be referred to as object A. There can be one or more objects A. Object A can include, but is not limited to, people, animals, landmarks, buildings, dolls, etc. For example, the first image can also be an old photo taken by a relative many years ago, etc.

[0234] For example, the first image could be, for instance, Figure 2C The original image corresponding to image 207a shown is Figure 2C Image 208 shown Figure 3C The original image corresponding to image 307a shown, or, Figure 3D Image 301 is shown. Object A can be the person in the image above.

[0235] The "dual-image" function is provided by electronic device 100. This function allows electronic device 100 to display a first image with transparency or blur in the preview interface provided by the camera application, while simultaneously displaying the image captured by the camera 193 of electronic device 100. When electronic device 100 receives a photo-taking command input by the user, it fits the first image and the image captured by the camera to generate a composite image. After the "dual-image" function is enabled, the electronic device will perform various subsequent operations.

[0236] In some implementations, reference is made to the foregoing. Figures 2A-2D And related descriptions, the first operation to enable the "duet" function for the first image could be, for example, in... Figure 2C The user action performed on control 210a in the function list 210 displayed on the electronic device 100 is the result of an input on the user interface 23 of the illustrated image 208 after an operation is performed on control 209. Control 209 may also be referred to as the first control, and control 210a may also be referred to as the second control.

[0237] In some implementations, the first operation for activating the "dual-image" function for the first image may, for example, be that the electronic device 100... Figure 2B The long press operation received on the displayed image 207a, or the electronic device 100 in Figure 2B The circle-drawing gesture received on the displayed image 207a, or the electronic device 100 in Figure 2C The long press operation received on image 208 shown.

[0238] In this embodiment of the application, the user interface provided by the gallery application for displaying the first image can be referred to as the first user interface, for example... Figure 2C , Figure 2D All of these can be referred to as the first user interface.

[0239] In some implementations, reference is made to the foregoing. Figures 3A-3C And related descriptions, the first operation used to activate the "duet" function for the first image may be, for example, the electronic device 100 receiving a signal acting on the first image. Figure 3B The user operation on control 306 is then applied to image 307a in image area 307 displayed on electronic device 100. Control 306 can also be referred to as a third control, the user operation on control 306 can also be referred to as a fourth operation, and the user operation on image 307a can also be referred to as a fifth operation.

[0240] in, Figure 3B The user interface shown can be provided not only by the camera app but also by other applications on the electronic device (such as social media apps, shopping apps, etc.) that call the camera interface. Essentially, other applications on the electronic device can call the camera interface to enable the "duet" function. The user interface used to enable the "duet" function, whether provided by the camera app or by other applications calling the camera interface, is as follows: Figure 3B , Figure 3C The user interface shown can be referred to as the second user interface.

[0241] Not limited to this, the first operation for activating the "duet" function for the first image can also be a voice control command, an air gesture command, a pressing operation on a physical button, etc. This application does not limit the specific implementation of the first operation.

[0242] S103, electronic device 100 launches camera application and displays second user interface, the second user interface displaying a preview box, the preview box displaying a first image with transparency and / or blur, and an image captured by camera 193.

[0243] After the electronic device 100 activates the "dual-shot" function for the first image, it can automatically launch the camera application. The camera application can be a system application installed on the electronic device 100 or a third-party application.

[0244] Specifically, after the electronic device 100 launches the camera application, it can display a second user interface provided by the camera application. This second user interface displays a preview frame showing a first image with transparency and / or blurring, as well as an image captured in real time by the camera 193. Here, the second user interface can also be referred to as a preview interface. In some embodiments, the electronic device 100 can overlay the first image with transparency and / or blurring onto the image captured in real time by the camera 193.

[0245] If the first image has transparency, the user can see both the first image and other content displayed on the electronic device 100 that is covered by it. The transparency of the first image can be preset, adjusted by the user according to actual needs, or adjusted according to the ambient light of the current electronic device 100 environment, or according to the brightness of the image captured by the camera 193, etc.

[0246] If the first image is blurred, the electronic device 100 can first identify the outline or content of the first image, and then depict its outline or content with dotted lines, or blur its content, to obtain the blurred first image. In this case, the user can see both the first image and other content displayed by the electronic device 100 that is covered by the blurred first image.

[0247] The image captured in real time by camera 193 and displayed in the preview box of the second user interface is refreshed in real time. When the user moves the electronic device 100 or changes the posture of the electronic device 100, changes the camera being used, or the object currently being photographed by camera 193, such as a person or animal, changes its location, the image captured by camera 193 will change in real time, and the changed image will be displayed in the preview box in real time.

[0248] Furthermore, the image captured in real-time by camera 193 and displayed in the preview frame can also be obtained by processing the original image captured by camera 193. For example, electronic device 100 can perform image stabilization processing on the original image captured by camera 193 before sending it to the display screen for display in the preview frame. As another example, if the user adjusts the focus of camera 193, electronic device 100 will crop the original image captured by camera 193 and send the cropped result to the display screen for display in the preview frame. As yet another example, the user can select the current shooting mode, and in different modes, electronic device 100 can perform different processing on the original image captured by camera 193, such as blurring the background area in portrait mode, and then send the cropped result to the display screen for display in the preview frame.

[0249] For example, refer to Figures 2F-2G The second user interface could be, for example, Figures 2F-2G The user interface 22 shown can have a preview box, for example, Figures 2F-2G The preview box 211 in the middle. Figures 2F-2G The lighter-colored image 208 displayed in the preview box can be the first image with transparency, while the darker-colored image is the image captured in real time by the camera 193.

[0250] For example, refer to Figures 3D-3E The second user interface could be, for example, Figures 3D-3E The user interface 34 shown can have a preview box, for example, Figures 3D-3E Preview box 301 in the middle. Figures 3D-3E The lighter-colored image 208 displayed in the preview box can be the first image with transparency, while the darker-colored image is the image captured in real time by the camera 193.

[0251] As can be seen, the preview box in the second user interface can show the user the relative positions of various objects in the first image and the image captured in real time by the camera 193. The user can adjust the image captured in real time displayed in the preview box according to the content presented, so that the relative positions of the objects in the image captured by the camera and object A in the first image meet the user's actual needs. The ways in which the user adjusts the image captured in real time displayed in the preview box may include, but are not limited to: the user moving the electronic device 100 or changing the posture of the electronic device 100, the user changing the camera used by the electronic device 100, or the user instructing the object currently being photographed by the camera 193, such as a person or animal, to change its position, etc. For example, refer to... Figures 2F-2G Or refer to Figures 3D-3E Users can adjust the relative positions of the human figures in the images captured by camera 193 and object A (i.e., the human figures in the image) in the first image.

[0252] In this embodiment, besides displaying a first image with transparency or blur in the second user interface provided by the camera application to indicate the first image, the electronic device 100 can also display the lines of objects contained in the first image, the location of objects in the first image, a blurred first image, etc., to show the first image in the preview box, thereby allowing the user to know the relative position of objects in the first image and objects in the image captured in real time by the camera 193. The information used to indicate the relative position of objects in the first image and objects in the image captured in real time by the camera 193 can be referred to as the indication information of the first image. For example, the indication information of the first image may include the first image with transparency or blur displayed in the preview box, the lines of objects contained in the first image, the location of objects in the first image, a blurred first image, etc., and this embodiment does not limit this.

[0253] S104, electronic device 100 receives the second operation.

[0254] S105, in response to the second operation, the electronic device 100 fits the first image and the image captured by the camera 193 to generate a composite image.

[0255] The second operation can be a user operation that triggers the electronic device 100 to take a picture or record a video. A user operation that triggers the electronic device 100 to take a picture can also be called a photo-taking command. A user operation that triggers the electronic device 100 to record a video can also be called a video-recording command.

[0256] For example, refer to Figures 2F-2G The second operation can be, for example, a user operation applied to the camera control 213, or a user operation applied to the recording control after the user selects the recording mode. (See reference) Figures 3D-3E The second operation can also be a user operation applied to the photo control 303, or a user operation applied to the recording control after the user selects the recording mode.

[0257] In some implementations, the electronic device 100 will only respond to the second operation if the correlation between the location indicated by the image captured by the camera 193 displayed in the preview box of the second user interface and the location indicated by the first image is greater than a first value. Here, the correlation between images refers to the degree of similarity and matching between the images in their relative positions. If the correlation between two images is high, it indicates that the two images contain similar content, and the two images may have been captured from the same location and from similar perspectives. The first value can be preset and is not limited here. For example, refer to the preceding text... Figures 2E-2GAccording to the relevant description, the control 213 can only be activated when the position indicated by the background area of ​​the real-time image captured by the camera displayed in the preview box 211 and the position indicated by the image 208 (e.g., but not limited to the position indicated by the background area of ​​the image 208) are related. In other cases, even if the control 213 receives a user operation, the electronic device 100 will not execute a subsequent response.

[0258] In other embodiments, after the electronic device 100 displays the second user interface, it will immediately respond to the second operation if it receives it. For example, refer to the preceding text. Figures 2E-2G According to the relevant description, control 213 can be kept active. As long as the user positions the object in the image captured by the camera in a position that satisfies the user and the object A in image 208, the user can click control 213 to trigger the electronic device 100 to respond to the operation and perform subsequent steps.

[0259] The following will combine Figures 6A-6B The illustrated image processing steps demonstrate how electronic device 100 fits the first image and the image captured by camera 193 to generate a composite image. This process may specifically include the following steps:

[0260] Step 1. Identify the object in the image captured by the camera, which can be called object B.

[0261] Specifically, the electronic device 100 can identify relatively distinctive objects such as people, animals, landmarks, buildings, and dolls in the images captured by the camera and designate them as object B. In some other embodiments, the electronic device 100 can also identify the object closest to the camera as object B based on the distance between each object in the image captured by the camera and the camera itself. In other embodiments, the user can also select an object as object B from the image captured by the camera displayed in the preview box. For example, the user can click on the location of the desired object in the preview box, and the electronic device 100 can automatically circle the person or scene at that location, without the user needing to manually draw the outline of the person or object to select object B.

[0262] In some implementations, the electronic device 100 can identify object B contained in an image captured by a camera using an image recognition algorithm. The image recognition algorithm could be, for example, an artificial intelligence (AI) algorithm.

[0263] In this embodiment of the application, the number of objects B identified by the electronic device 100 can be one or more, and there is no limitation here.

[0264] For example, refer to Figures 6A-6B , Figures 6A-6B Each of the following (a) figures shows a frame of image captured by the camera of electronic device 100. Object B in this image, as shown in the figure, is the male figure in the image, as identified by electronic device 100. The electronic device captures... Figure 6B The focal length used in (a) is greater than that used for image acquisition. Figure 6A The focal length used in (a) is shown in the figure.

[0265] Step 2. In the image captured by the camera, determine the main region containing object B, using object B as the reference.

[0266] In some implementations, the main region can be the area containing object B, which is obtained by extending a distance M around object B in the image captured by the camera.

[0267] M can be an empirical value obtained through testing. In some implementations, the M value can be the same for the same electronic device, and the M value can be different for different electronic devices, or the M value corresponding to different cameras on the same electronic device can be different. This ensures that the images captured by different electronic devices or different cameras can all contain a main region of object B.

[0268] Setting the M value using empirical values ​​ensures that the electronic device finds an accurate main region, guaranteeing the integrity of object B within that region while avoiding excessive expansion of the range, thus ensuring the effectiveness of subsequent fitting.

[0269] In some implementations, M can be a single value or a set of values. When M is a set of values, the values ​​in the set can be different. That is, the electronic device 100 can extend different distances around the object B from different locations to obtain a main area containing the object B.

[0270] For example, refer to Figures 6A-6B In (b), the area enclosed by the dashed line is a possible main region containing object B. Figures 6A-6B The size and location of the content contained in the images captured by each of the cameras can be different.

[0271] In other implementations, the main region may be an area in the image captured by the camera that extends outward from object B and has a similar or the same color or pixels as object B.

[0272] Step 3. Perform a relevance search on the main region in the first image to find co-locations with a relevance greater than the second value.

[0273] Specifically, performing a relevance search for the main region in the first image means first identifying the background region in the image captured by the camera, excluding the main region. Then, identifying the regions in the first image whose relevance to the background region is greater than a second value and / or the region with the highest relevance, the remaining parts of the first image excluding these background regions are considered co-location regions. In other words, the relevance between the remaining parts of the first image excluding the co-location regions and the remaining parts of the image captured by the camera excluding the main region is greater than the second value. The second value can be preset, and this embodiment does not limit its implementation.

[0274] This application does not limit the size, shape, or position of the corresponding region in the first image. The position of the corresponding region in the first image is the same as the position of the main region containing object B in the image captured by the camera. This ensures that the relative positions of object A and object B presented to the user by the electronic device 100 in the preview frame are the same as the relative positions of object A and object B in the fitted composite image, allowing the user to adjust the photo effect in real time based on the content in the preview frame. In some embodiments, the size and shape of the corresponding region in the first image and the size and shape of the main region containing object B in the image captured by the camera can be the same or different.

[0275] For example, refer to Figure 6A In (b) and (c), the co-located areas in the first image and the main area in the image captured by the camera are the same in size, shape and position.

[0276] For example, refer to Figure 6B In (b) and (c), the co-location region in the first image and the main region in the image captured by the camera have the same shape and position. The size of the co-location region in the first image is larger than the size of the main region in the image captured by the camera.

[0277] In step 3, if the electronic device 100 can find a corresponding area in the first image that corresponds to the main area, it indicates that the geographical location of the first image is the same as or similar to the geographical location of the image captured by the electronic device 100. This achieves a group photo taken at the same or similar location. Furthermore, since the location is the same or similar, the lighting and background in the first image are similar to those in the image captured by the camera. This greatly improves the fitting effect in the subsequent step 4, resulting in a better group photo effect. This distinguishes the shooting method provided in this embodiment from simple image cutout group photos; this method can bring a more natural group photo effect.

[0278] In some implementations, after the electronic device 100 displays the second user interface, if it immediately responds to the second operation upon receiving it, step 3 may fail due to the uncertainty of the image captured by the current camera; that is, the electronic device 100 may be unable to find the corresponding co-location area in the first image. In this case, the electronic device 100 can output a prompt message to inform the user that the current compositing has failed.

[0279] Step 4. Fit the main region to the corresponding region of the first image, keeping the rest of the first image unchanged, to generate a composite image.

[0280] Specifically, the electronic device can zoom in or out on the main region containing object B captured by the camera, then merge this main region into the corresponding region of the first image, and perform a smooth transition at the merging boundary. In other words, the electronic device replaces the corresponding region in the first image with the zoomed-out or original main region, and performs a smooth transition at the boundary between the main region and the first image to make the final composite image look natural.

[0281] For example, refer to Figure 6A If the co-location region and the main region are the same size, the electronic device 100 replaces the co-location region in (c) with the main region and performs transitional smoothing on the boundary to generate a composite image as shown in (d).

[0282] For example, refer to Figure 6B The electronic device 100 enlarges the main region, replaces the co-located region in (c) with the enlarged main region, and performs transitional smoothing on the boundary to generate a composite image as shown in (d).

[0283] By following steps 1-4 above, object B can be smoothly and naturally integrated into the first image containing object A, generating a composite object that simultaneously contains object A and object B, thus achieving a natural and high-quality composite photo effect.

[0284] When the electronic device 100 executes S104-S105 in the photo-taking mode, it can fit the image captured at the moment the photo-taking command is received with the first image to generate a composite image. In other embodiments, the electronic device 100 can save the moment the photo-taking command is received and the N frames of images previously captured by the camera 193 in real time, and fit each of the N frames of images with the first image to generate an N-frame composite image.

[0285] When the electronic device 100 executes S104-S105 in recording mode, it can fit the multiple frames of images captured by the camera 193 between the moment the recording instruction is received and the moment the recording instruction is received with the first image to generate a multi-frame composite image.

[0286] The above process displays the relative positions of objects A and B in the group photo to the user through overlay in the preview interface. The fitting operation between the first image and the image captured by the camera is only started after the first operation is received. This method can save the power consumption and processing power of electronic devices.

[0287] S106, Electronic device 100 saves the composite image.

[0288] If the electronic device 100 executes S104-S105 in photo mode, and only fits the image captured at the moment the photo-taking command is received with the first image to generate a composite image, the electronic device 100 can save the generated composite image in image format to the memory. If the electronic device fits the first image with the N frames of images captured by the camera 193 at the moment the photo-taking command is received, respectively, to generate N composite images, the electronic device 100 can select the frame with the best quality from the N composite images and save it in image format to the memory. Here, the electronic device 100 can select the frame with the best quality from the N composite images through artificial intelligence learning. The quality of the composite image can be evaluated by, but is not limited to, any one or more of the following parameters: peak signal-to-noise ratio (PSNR) and mean square error (MSE), mean absolute error (MAE) and signal-to-noise ratio (SNR).

[0289] In other embodiments, if the electronic device 100 executes S104-S105 in the photo-taking mode, the electronic device will fit the first image with the N frames of images captured by the camera 193 at the moment the photo-taking command is received, and generate N frames of composite images. Then, the user can select a satisfactory composite image from the N frames of composite images and store it in the memory, and delete the remaining composite images.

[0290] In the above embodiments, the composite image (i.e., composite picture) maintained by the electronic device 100 in shooting mode by executing S104-S106 can also be referred to as the second picture.

[0291] If the electronic device 100 executes S104-S105 in recording mode, it can store the multi-frame composite image generated by fitting the multi-frame images captured by the camera 193 of the recording device with the first image in a video format into the memory, that is, save the multi-frame composite image as a video file.

[0292] After the electronic device 100 saves the composite image, it can view the composite image and perform operations such as editing, moving, and sharing it. The operation of viewing the composite image can also be referred to as a third operation.

[0293] For example, refer to Figures 4A-4B The electronic device 100 generates a composite image in photo mode. Figure 4B The image library application shown displays a thumbnail 207b of the composite image in image area 207. After receiving a user action applied to the thumbnail 207b, the electronic device 100 can display, as shown... Figure 4B The original image shown in the thumbnail 207b is the composite image generated by the electronic device 100.

[0294] In some implementations, after generating the composite image, the electronic device 100 can use the composite object as the first image and execute the shooting method described in S101-S106 one or more times, thus achieving the purpose of multiple group photos. For example, the same user can first use their own photo as the first image, and then take multiple group photos with themselves in the first image, achieving multiple group photos of the same person in the same location.

[0295] In some implementations, if the first image is an old photo taken by a relative many years ago, then through the shooting methods described in S101-S106 above, the user can take a cross-time and space group photo with their relatives at the same location at different times.

[0296] In steps 1-4 of S105 described above, the electronic device 100 specifically fits the region where object B is located in the image captured by the camera 193 to the first image, thereby obtaining a composite image. Not limited thereto, in other embodiments of this application, the electronic device 100 may also fit the region where object A is located in the first image to the image captured by the camera 193, thereby obtaining a composite image. Specifically, this embodiment may include the following steps:

[0297] Step 1. Identify the object in the first image, which can be called object A;

[0298] Step 2. In the first image, determine the main region containing object A, using object A as the reference.

[0299] Step 3. Perform a correlation search on the main region in the image captured by the camera to find the co-location region with a correlation greater than the second value;

[0300] Step 4. Fit the main region to the co-location region of the image captured by the camera, while keeping the rest of the image captured by the camera unchanged, to generate a composite image.

[0301] The specific implementation of each of the above steps can be found in the implementation of each step in S105 above, and will not be repeated here.

[0302] Through the above implementation method, the area where object A is located in the first image is fitted to the image captured by camera 193 to obtain a composite image. If the first image is an old photo, the object in the old photo can be extracted and integrated into the newly captured image to realize the renovation of the old photo.

[0303] In the above embodiments of this application, object A in the first image can also be referred to as the first object, and object B in the image captured by the camera can also be referred to as the second object.

[0304] In the above Figure 5 In the shooting method shown, the electronic device 100 first overlays the first image and the image captured by the camera in the preview interface, and then generates a composite image after receiving the first operation. Not limited to this, in other embodiments of this application, the electronic device 100 can also directly perform the operation of fitting and generating a composite image after starting the camera application, and directly display the composite image in the preview interface. Specifically, after executing S101-S103 of the above method, the electronic device 100 starts the camera application, captures images in real time through the camera 193, and fits each frame of the real-time captured image by the camera 193 with the first image to obtain a multi-frame composite image. These multi-frame composite images are then displayed in chronological order in the preview box of the preview interface (i.e., the second user interface mentioned above). If a photo-taking command is received, the composite image displayed in the preview box is saved as an image; if a video recording command is received, the multi-frame composite image displayed in the preview box during video recording is saved as a video. This real-time compositing and real-time display of composite images requires the electronic device 100 to have strong processing capabilities. This method allows users to see the group photo effect in real time in the preview box, giving them a more intuitive feel for the effect.

[0305] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0306] This application also provides an electronic device, which may include a memory and a processor. The memory may be used to store a computer program; the processor may be used to invoke the computer program in the memory, causing the electronic device to execute the method of electronic device 100 in any of the above embodiments.

[0307] This application also provides a chip system including at least one processor for implementing the functions involved in the electronic device 100 in any of the above embodiments.

[0308] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0309] The chip system can consist of chips or include chips and other discrete components.

[0310] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0311] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0312] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0313] This application also provides a computer program product comprising: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the method executed on the electronic device 100 side in any of the above embodiments.

[0314] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed on the electronic device 100 side in any of the above embodiments.

[0315] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0316] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0317] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0318] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. A shooting method, characterized in that, The method is applied to an electronic device, and the method includes: The user interface of the camera application is displayed, and the user interface of the camera application displays a preview box, which displays an image captured by the camera of the electronic device, as well as a first image with transparency and / or a first image with blurring, the first image containing a first object; If the correlation between the location indicated by the image captured by the camera and the location indicated by the first image is greater than a first value, the shooting control in the user interface is activated; In response to an operation on the shooting control, the electronic device fits a second image based on the image captured by the camera and the first image, wherein the image captured by the camera contains a second object and the second image contains the first object and the second object; The electronic device stores the second image.

2. The method according to claim 1, characterized in that, After the electronic device receives the operation to take a picture, but before storing the second image, the method further includes: The electronic device identifies the second object in the image captured by the camera; In the image captured by the camera, a main region containing the second object is determined, wherein the main region is a region determined with the second object as a reference. In the first image, a co-location region is determined, and the part outside the co-location region in the first image, and the part outside the main region in the image captured by the camera have a correlation greater than a second value. The main region is fitted to the co-located region in the first image to obtain the second image.

3. The method according to claim 1, characterized in that, After the electronic device receives the operation to take a picture, but before storing the second image, the method further includes: The electronic device identifies the first object in the first image; In the first image, a main region containing the first object is determined, wherein the main region is a region determined with the first object as a reference. In the image captured by the camera, a co-location region is determined, and the part of the image captured by the camera outside the co-location region, and the part of the first image outside the main region, have a greater correlation than a second value; The main region is fitted to the co-location region in the image captured by the camera to obtain the second image.

4. The method according to claim 2, characterized in that, The position of the co-location region in the first image is the same as the position of the main region in the image captured by the camera.

5. The method according to claim 3, characterized in that, The position of the co-location region in the image captured by the camera is the same as the position of the main region in the first image.

6. The method according to claim 1, characterized in that, The second image is the highest quality frame among the composite images obtained by fitting multiple frames captured by the camera to the first image; The multi-frame images captured by the camera include images captured by the camera at the moment when the electronic device receives the operation for taking a picture and at moments prior to that moment, and the number of the multi-frame images is preset by the electronic device.

7. The method according to any one of claims 1-3 and 6, characterized in that, Before displaying the camera application's user interface, the method further includes: The electronic device receives a user operation applied to the first control; In response to a user action applied to the first control, the electronic device displays a second control; The electronic device receives a user operation applied to the second control; Displaying the user interface of the camera application specifically includes: displaying the user interface of the camera application in response to a user operation on the second control.

8. The method according to any one of claims 1-3 and 6, characterized in that, After the electronic device stores the second image, the method further includes: The electronic device receives an operation to view the second image; The electronic device displays the second image.

9. The method according to any one of claims 1-8, characterized in that, Before the electronic device acquires the first image, the method further includes: The electronic device launches a gallery application and displays a first user interface, in which the first image is displayed.

10. The method according to any one of claims 1-9, characterized in that, The first image includes one of the following: an image captured by the camera of the electronic device, an image downloaded by the electronic device from the network, or an image shared by another device received by the electronic device.

11. A shooting method, characterized in that, The method is applied to an electronic device, and the method includes: The camera application's interface is displayed, and the camera application's user interface displays a preview box; If the correlation between the location indicated by the image captured by the camera of the electronic device and the location indicated by the first image is greater than a first value, a composite image is displayed in the preview box. The composite image is obtained by fitting the image captured by the camera of the electronic device and the first image, wherein the first image contains a first object and the image captured by the camera contains a second object. The electronic device receives an operation to take a picture; The electronic device stores a second image, which is a composite image displayed in the preview box when the operation for taking a picture is received, and the second image includes the first object and the second object.

12. The method according to claim 11, characterized in that, Before the electronic device displays the composite image in the preview frame, the method further includes: The electronic device identifies the second object in the image captured by the camera; In the image captured by the camera, a main region containing the second object is determined, wherein the main region is a region determined with the second object as a reference. In the first image, a co-location region is determined, and the part outside the co-location region in the first image, and the part outside the main region in the image captured by the camera have a correlation greater than a second value. The main region is fitted to the co-located region in the first image to obtain the synthesized image.

13. The method according to claim 11, characterized in that, Before the electronic device displays the composite image in the preview frame, the method further includes: The electronic device identifies the first object in the first image; In the first image, a main region containing the first object is determined, wherein the main region is a region determined with the first object as a reference. In the image captured by the camera, a co-location region is determined, and the part of the image captured by the camera outside the co-location region, and the part of the first image outside the main region, have a greater correlation than a second value; The main region is fitted to the co-located region in the image captured by the camera to obtain the synthesized image.

14. The method according to claim 12, characterized in that, The position of the co-location region in the first image is the same as the position of the main region in the image captured by the camera.

15. The method according to claim 13, characterized in that, The position of the co-location region in the image captured by the camera is the same as the position of the main region in the first image.

16. The method according to any one of claims 11-15, characterized in that, The first image includes one of the following: an image captured by the camera of the electronic device, an image downloaded by the electronic device from the network, or an image shared by another device received by the electronic device.

17. An electronic device, characterized in that, The electronic device includes: a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-16.

18. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-16.

19. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-16.

Images