A photographing method, an electronic device, and a computer readable storage medium

Abstract

The application provides a photographing method, an electronic device and a computer readable storage medium, and relates to the technical field of terminals. The method is applied to an electronic device, and the method comprises the following steps: acquiring a first image comprising multiple faces collected by a camera of the electronic device focusing on a first focusing position; controlling the camera to focus on a second focusing position based on the position of a first face in the first image, the second focusing position corresponding to the position of the first face in the first image, and the distance between the first face and the camera being between the face closest to the camera and the face farthest from the camera; and acquiring a second image in response to a shutter pressing operation. Through the faces in the first image, the camera can be controlled to focus on a face between the face closest to the camera and the face farthest from the camera, thereby ensuring the definition of the faces in the photographed second image, and the problem of unclear faces in a photograph taken in a multiple-person scene can be solved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a shooting method, electronic device and computer-readable storage medium. Background Technology

[0002] With the development of technology, smart terminal devices, such as mobile phones and tablets, have become indispensable in people's lives. To meet people's needs to record moments of daily life through photos, most smartphones have a camera function.

[0003] However, in scenarios where multiple people are taking photos, the distance between each person and the camera is not uniform, and the focus point of a smartphone camera can generally only be fixed at one position. Therefore, there are cases where the distance between the position of some faces and the focus point is relatively far, resulting in the problem of unclear faces in the photo. Summary of the Invention

[0004] In view of this, this application provides a shooting method, an electronic device, and a computer-readable storage medium that can solve the problem of unclear faces in photos taken in multi-person scenes.

[0005] In a first aspect, this application provides a shooting method applied to an electronic device. The method includes: acquiring a first image captured by a camera of the electronic device focusing at a first focus position; the first image including multiple faces; based on the position of the first face in the first image, controlling the camera to focus at a second focus position, the second focus position corresponding to the position of the first face in the first image, the distance between the first face and the camera being between the face closest to the camera and the face farthest from the camera; and acquiring a second image in response to a shutter press operation; the second image being captured by the camera focusing at the second focus position.

[0006] In some examples, in group photos, staggered positioning is used to ensure everyone appears in the camera's frame. After the electronic device's camera focuses, the faces in the first image may be blurry due to the distance between the faces and the focus point. Therefore, the electronic device can determine a suitable second focus position based on the information in the first image, so that the camera focuses between the face furthest from the camera and the face closest to the camera, thus ensuring that each face is sharp in the second image captured when the shutter is pressed.

[0007] In this process, electronic devices typically select a specific object during focusing. For example, in portrait scenes, the face is usually chosen as the focus object; in landscape scenes, objects with clear outlines, such as trees, flowers, or rocks, are typically selected. Therefore, when determining the second focus position, the electronic device can determine the second focus position based on the first face in the first image that is between the face closest to the camera and the face furthest from the camera. This ensures that after the camera focuses based on the second focus position, it can capture a second image that includes a clearer face.

[0008] In one possible implementation of the first aspect, the method further includes: processing the second image to obtain a photograph.

[0009] By processing a second image captured by the camera of an electronic device at a second focus position, a photo that can be viewed by the user can be obtained. In some examples, the second image is the original image, which is then processed by the electronic device to become a user-viewable image.

[0010] In one possible implementation of the first aspect, the first focus position corresponds to the face closest to the camera. For example, if the electronic device can detect the distance between the camera and the face, it can directly select the face closest to the camera as the first focus position. Alternatively, if the electronic device cannot detect the distance between the camera and the face, it can determine the face closest to the camera based on the size of the face region; the face region corresponding to the closest face is the largest, and vice versa.

[0011] In another possible implementation, the first focus position can also correspond to the face furthest from the camera.

[0012] In one possible implementation of the first aspect, the method further includes: detecting multiple second faces in the first image and calculating the sharpness of each second face; and determining a first face from the multiple second faces based on the sharpness of each second face.

[0013] When a first image is captured at the first focus position, it can be determined that faces near the first focus position have the highest sharpness. Therefore, the positions of other faces can be inferred based on their sharpness. If the first focus position corresponds to a face closest to the camera, the electronic device can assume that the sharper the facial region, the closer the face is to the camera. Therefore, the first face can be identified based on its sharpness, and a second focus position can be determined based on this first face.

[0014] In one possible implementation of the first aspect, determining the first face from a plurality of second faces based on the sharpness of each second face includes:

[0015] Among multiple second faces, the second face whose clarity meets the first condition is regarded as the first face. The first condition includes at least one of the following conditions: the clarity is located in the first interval; the clarity is the middle value among the multiple clarity values ​​corresponding to the multiple second faces; the clarity is the clarity other than the first clarity value and the second clarity value among the multiple clarity values ​​corresponding to the multiple second faces.

[0016] When a first image is captured at the first focus position, the electronic device can assume that the clearer the facial region corresponding to the face, the closer the distance between the face and the camera. Therefore, based on the clarity of the face, the first face can be identified, and the second focus position can be determined based on the first face. This first condition ensures that the distance between the first face identified by the electronic device and the camera is between the face closest to the camera and the face farthest from the camera, thereby ensuring that the camera can capture a second image with high clarity.

[0017] In one possible implementation of the first aspect, processing the second image includes: processing the second image according to an image processing network model; the image processing network model is trained based on multiple third images and multiple fourth images, each third image includes multiple faces, and the clarity of the third face in each third image is greater than a first preset threshold, the clarity of the faces other than the third face in the third image is less than or equal to the first preset threshold, and the distance between the third face and the camera is between the face closest to the camera and the face farthest from the camera; each fourth image includes the same faces as the third images, and the clarity of each face in the fourth image is greater than the second preset threshold.

[0018] In one possible implementation of the first aspect, the third image is obtained by image degradation of the fourth image.

[0019] In the fourth image, the clarity of each face is relatively high. The third image, obtained after image degradation processing based on the fourth image, has similar characteristics to the second image, with the faces corresponding to the focus positions being the clearest.

[0020] Image degradation refers to a decrease in image quality, which generally includes blurring, distortion, and noise. The third image, obtained by processing the fourth image through image degradation, can simulate the second image captured by an electronic device at the second focus position. Since each face in the fourth image has high clarity, an image processing network model can be trained to repair blurred faces in the image, thus ensuring image clarity.

[0021] In one possible implementation of the first aspect, processing the second image includes: sharpening the face in the second image.

[0022] Sharpening the face in the second image can make the details in the face stand out more, thus making the face image clearer.

[0023] In one possible implementation of the first aspect, sharpening a face in the second image includes: determining a fifth face in the second image, wherein the sharpness of the fifth face is less than a third preset threshold; and sharpening the fifth face.

[0024] In one possible implementation of the first aspect, controlling the camera to focus on the second focus position includes: searching for the second focus position in a stored mapping relationship, the mapping relationship including a position in an image and a focus position corresponding to the position in the corresponding image; the position in the image includes the position of the first face; and controlling the camera's focus point to switch from the first focus position to the second focus position.

[0025] In one possible implementation of the first aspect, the camera is an autofocus (AF) camera.

[0026] In one possible implementation of the first aspect, the camera is a front-facing camera.

[0027] Secondly, this application provides an electronic device, which includes a display screen, a memory, and one or more processors; the display screen, the memory, and the processors are coupled; the display screen is used to display images generated by the processor, and the memory is used to store computer program code, which includes computer instructions; when the processor executes the computer instructions, the electronic device performs the following: acquiring a first image captured by the camera of the electronic device focusing to a first focus position; the first image includes multiple faces; based on the position of the first face in the first image, controlling the camera to focus to a second focus position, the second focus position corresponding to the position of the first face in the first image, the distance between the first face and the camera being between the face closest to the camera and the face farthest from the camera; and in response to a shutter press operation, acquiring a second image; the second image is acquired by the camera focusing to the second focus position.

[0028] When the processor executes computer instructions, it also causes the electronic device to perform: processing a second image to obtain a photograph.

[0029] When the processor executes computer instructions, it also causes the electronic device to: detect multiple second faces in a first image and calculate the sharpness of each second face; and determine a first face from the multiple second faces based on the sharpness of each second face.

[0030] When the processor executes computer instructions, it also causes the electronic device to perform the following: among a plurality of second faces, the second face whose clarity meets a first condition is used as the first face, the first condition including at least one of the following conditions: the clarity is located in a first interval; the clarity is the clarity value that is in the middle among the multiple clarity values ​​corresponding to the plurality of second faces; the clarity is the clarity other than the first clarity value and the second clarity value among the multiple clarity values ​​corresponding to the plurality of second faces.

[0031] When the processor executes computer instructions, it also causes the electronic device to perform the following: processing a second image according to an image processing network model; the image processing network model is trained based on multiple third images and multiple fourth images, each third image includes multiple faces, and the clarity of the third face in each third image is greater than a first preset threshold, the clarity of the faces other than the third face in the third image is less than or equal to the first preset threshold, and the distance between the third face and the camera is between the face closest to the camera and the face farthest from the camera; each fourth image includes the same faces as the third images, and the clarity of each face in the fourth image is greater than a second preset threshold.

[0032] When the processor executes computer instructions, it also causes the electronic device to perform: sharpening the face in the second image.

[0033] When the processor executes computer instructions, it also causes the electronic device to: determine a fifth face in the second image, wherein the clarity of the fifth face is less than a third preset threshold; and sharpen the fifth face.

[0034] When the processor executes computer instructions, it also causes the electronic device to: search for a second focus position in a stored mapping relationship, the mapping relationship including the position in the image and the focus position corresponding to the position in the corresponding image; the position in the image includes the position of the first face; and control the camera's focus point to switch from the first focus position to the second focus position.

[0035] Thirdly, this application provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the second and third aspects above, and any possible design of the above.

[0036] Fourthly, this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the methods described in the second and third aspects above, and any possible design of the above.

[0037] Fifthly, this application provides an apparatus included in an electronic device, which has the function of implementing the behavior of the electronic device in any of the above aspects and possible implementations. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes at least one module or unit corresponding to the above function. For example, an allocation module or unit, a scanning module or unit, a recycling module or unit, a moving module or unit, and a storage module or unit, etc.

[0038] Sixthly, embodiments of this application provide a chip system including a processor and potentially a memory, for implementing any of the methods provided in the third to fifth aspects above. The chip system may be composed of chips or may include chips and other discrete devices.

[0039] Understandably, the electronic device described in the second aspect and any possible design of the above-described device, the computer-readable storage medium described in the third aspect, and the computer program product described in the fourth aspect are all used to perform the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0040] Figure 1 A schematic diagram of a group photo scene provided in an embodiment of this application;

[0041] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0042] Figure 3 A schematic diagram of the hardware and software structure of an electronic device provided in an embodiment of this application;

[0043] Figure 4 A schematic flowchart illustrating a shooting method provided in an embodiment of this application;

[0044] Figure 5 A schematic diagram illustrating a shooting method provided in an embodiment of this application;

[0045] Figure 6 A schematic diagram illustrating the range of image resolution provided in an embodiment of this application;

[0046] Figure 7 This is a schematic diagram of data stream transmission for a shooting method provided in an embodiment of this application. Detailed Implementation

[0047] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships may exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0048] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0049] Before introducing the embodiments of this application, the technologies involved in the embodiments of this application will be described in detail.

[0050] 1. Camera, lens

[0051] A camera consists of a lens and an image sensor. The lens is a lens structure composed of several lenses, and its main function is to focus the reflected light from the subject onto the image sensor of the camera. Then, the image sensor converts the light signal into an electrical signal, thus obtaining a picture including the subject.

[0052] 2. Focus

[0053] The focusing process determines the sharpness of the subject in a photograph. The principle of focusing is to adjust the distance between the lens and the image sensor in the camera, so that the image plane of the subject coincides with the image plane of the image sensor, thus achieving a sharp image.

[0054] Generally, the image of the subject is sharpest when it is located at the focus point.

[0055] 3. Depth of field

[0056] The range from a point in front of the focal point to a point behind it, within which objects in the scene can form a clear image, is called the depth of field.

[0057] In group photos, each person stands in a different position, meaning that the distance between each person and the camera is different. Figure 1 As shown, in the scenario where user A is taking a photo with users B and C, user A is closest to the camera, user B is next, and user C is furthest from the camera.

[0058] Because the camera's autofocus process automatically focuses on the face closest to the lens, when user A presses the shutter button to take a picture, they will get a result like... Figure 1 The photo shown. (It is available in...) Figure 1 The photos displayed on the phone screen show that the image of user A, who is closest to the camera, is clear, while the images of users B and C are not clear.

[0059] To address the issue of blurry portraits in group photos, this application provides a shooting method. An electronic device can analyze a preview photo captured by the camera before taking the picture to determine the camera's focus position, thereby reducing the blurriness of portraits farther from the focus point. Then, when the user presses the shutter, the electronic device acquires a photo corresponding to the focus point and further refines it to obtain a photo containing multiple clear portraits. This allows for the capture of photos with multiple clear portraits in group photos.

[0060] To better understand the embodiments of this application, the electronic devices provided in the embodiments of this application are first introduced. The shooting method provided in the embodiments of this application can be applied to electronic devices. For example, the electronic device may specifically be a smartphone, tablet computer, smart screen, laptop computer, wearable device (such as a smartwatch), ultra-mobile personal computer (UMPC), personal digital assistant (PDA), artificial intelligence device, or other electronic devices with shooting functions. The embodiments of this application do not limit the specific type of electronic device or the operating system installed.

[0061] The following section introduces the hardware structure of electronic devices.

[0062] like Figure 2The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0063] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0064] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0065] The processor 110 includes a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0066] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0067] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0068] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0069] The charging management module 140 receives charging input from a charger, which can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0070] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc. In some embodiments, the power management module 141 and the charging management module 140 may also be housed in the same device.

[0071] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices through wireless communication technology.

[0072] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0073] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation.

[0074] The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be housed in the same device.

[0075] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity, Wi-Fi), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), and other wireless communication technologies.

[0076] The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0077] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0078] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0079] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0080] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0081] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0082] The lens in camera 193 is essentially a convex lens, composed of several lenses.

[0083] The camera 193 also includes a focusing motor, which can move the entire lens to control the image distance and adjust the image sharpness.

[0084] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0085] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0086] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, image processing, facial recognition, speech recognition, and text understanding.

[0087] Electronic device 100 can implement audio functions such as music playback and recording through audio module 170, speaker, receiver, microphone, headphone jack, and application processor.

[0088] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc. Electronic devices can collect various data through the sensor module 180.

[0089] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types.

[0090] It is understood that the interface connection relationships between the modules in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer modules than those provided in the above embodiments, and the modules may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0091] The software system of the aforementioned electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses a layered architecture as an example to exemplify the software structure of the electronic device 100.

[0092] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, Figure 3 This is a hardware and software structure block diagram of an electronic device 100 according to an embodiment of the present invention. The hardware and software structure of the electronic device includes an application layer, an application framework layer, a hardware abstraction layer, a kernel layer, and a hardware layer.

[0093] It should be noted that the solution of this application can also be implemented in other operating systems as long as the functions implemented by each functional module are similar to those in the embodiments of this application.

[0094] The application layer can include a series of application packages. For example... Figure 3As shown, the application package can include applications such as camera, gallery, and calling.

[0095] Camera apps offer the ability to take photos or videos. You can view the photos or videos taken by the camera app through the gallery.

[0096] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 3 As shown, the application framework layer may include camera services.

[0097] Camera services provide a camera service interface for camera applications. For example, camera applications can use camera services to set shooting parameters, start / stop previews, and take photos. Camera services isolate the application layer from the underlying implementation, facilitating the development and portability of both the application and the underlying hardware. For instance, when electronic devices have camera software developed by third-party software vendors installed, the camera service can also be used to implement shooting functions.

[0098] A hardware abstraction layer is used to abstract hardware. For example, a hardware abstraction layer may include a camera abstraction layer and other hardware device abstraction layers. The camera abstraction layer may include camera devices and camera algorithm libraries.

[0099] The camera device provides an interface for calling the camera driver to serve the camera service.

[0100] The camera algorithm library may include algorithmic instructions such as camera algorithms and image algorithms. The camera algorithm library can also execute the steps in the shooting method provided in the embodiments of this application. For example, the camera algorithm library can acquire a preview photo captured by the camera and determine the camera's focus position based on the sharpness of the preview photo. Then, when the user presses the shutter button, the camera algorithm library can process the photo obtained when the shutter button is pressed to obtain a photo including multiple sharp portraits.

[0101] The kernel layer is the layer between hardware and software. The kernel layer contains at least the camera driver.

[0102] The camera driver can drive the photosensitive element to start converting light signals into electrical signals. The camera driver can also drive the focusing motor to move the lens, thereby changing the position of the focus point.

[0103] For details on the various hardware modules in the hardware layer, please refer to the previous text. Figure 2 The relevant explanations will not be repeated here.

[0104] In the software architecture of the aforementioned electronic devices, the application layer needs to call services provided by the application framework layer, the application framework layer needs to call functions provided by the core system library layer, and the core system library layer needs to call services provided by the operating system kernel layer. All these components work together to provide users with a complete electronic device experience.

[0105] The shooting method provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0106] like Figure 4 As shown, the shooting method provided in this application embodiment includes the following steps:

[0107] Step S401: In response to the camera application launching, acquire a preview image.

[0108] Once the camera application is launched, the electronic device's camera begins capturing a preview image (i.e., the aforementioned "first image") and displays the captured preview image on the electronic device's screen in real time for the user to preview.

[0109] In some examples, in response to the camera app launching and the camera app being in selfie mode, the electronic device acquires a preview image via the front-facing camera.

[0110] In some embodiments, when acquiring a preview image, the electronic device may calculate the sharpness of the image captured by the current camera, and if the sharpness of the image captured by the current camera is greater than a preset sharpness threshold, the image captured by the current camera may be used as a preview image.

[0111] In some examples, electronic devices can also determine the sharpness of the image captured by the camera based on jitter information from the gyroscope sensor. For instance, the lower the jitter, the sharper the image. Therefore, when the jitter is less than a preset jitter threshold, the image captured by the camera is used as a preview image.

[0112] Step S402: Determine the focus position based on the sharpness of all faces in the preview image.

[0113] In some embodiments, in related technologies, since electronic devices typically only activate the front-facing camera when the user is taking a selfie, and fixed-focus (FF) cameras are less expensive, the front-facing camera of smartphones in related technologies is an FF camera. However, FF cameras cannot capture clear images when photographing moving objects. Therefore, to meet the user's need to photograph moving objects using the front-facing camera, the shooting method provided in this application embodiment is applied to an electronic device equipped with an autofocus (AF) camera. An electronic device equipped with an autofocus camera can quickly achieve focus when photographing moving objects, thereby ensuring that the photos taken by the electronic device include clear images of the moving objects.

[0114] Understandably, the moving object mentioned in the above embodiments can be an object that moves relative to the electronic device. For example, with the ground as the frame of reference, if the face does not move but the phone shakes, the face is also considered a moving object.

[0115] Because during the autofocus process of an AF camera, it will automatically focus on the face closest to the lens (i.e., the aforementioned "first focus position"), such as Figure 5 As shown, since user A is closest to the camera, the preview stream captured by the electronic device during the selfie process shows that user A's face area is the largest. Therefore, the camera will focus on user A's face, and at this time, the face area 1 corresponding to user A in the preview image is the clearest. User B is closer to the camera than user C, therefore, the face area 2 corresponding to user B in the preview image is clearer than the face area 3 corresponding to user C.

[0116] In some embodiments, the electronic device may first determine the position and sharpness of multiple faces (i.e., the aforementioned "first face") in the preview image, and sort all the faces according to their sharpness to obtain a face sequence. Then, the electronic device may determine the focus position based on the position of the face at a preset location within the face sequence.

[0117] In some examples, such as Figure 5 The electronic device can sort all faces from highest to lowest resolution in the group photo preview image shown, resulting in the following face sequence: User A, User B, User C. The electronic device can use the face with the middle resolution (the resolution value is in the middle) as the face to determine the focus position (i.e., the aforementioned "second face").

[0118] Given that the preset position is the center, the electronic device can determine that user B is in the center of the face sequence. Therefore, the electronic device can determine the focus position (i.e., the aforementioned "second focus position") based on user B's face position. Specifically, since the image is a two-dimensional plane, user B's face position is a coordinate position in the image. The electronic device stores a correspondence (mapping relationship) between focus positions and coordinate positions in the image. Therefore, after knowing the coordinate position in the image, the electronic device can find the focus position based on the correspondence. For example, the correspondence includes: the image coordinate position (x1, y1) corresponds to focus position code1, the image coordinate position (x2, y2) corresponds to focus position code3, and the image coordinate position (x3, y3) corresponds to focus position code3. Therefore, if user B's coordinate position is (x3, y3) in the example above, the electronic device can determine the focus position as code3.

[0119] In the example above, after determining the focus position based on user B's face position, it can be guaranteed that user B's face is the clearest in the photo. For example... Figure 6 As shown, the sharpness of a user's face is highest when it is at the focus point, and the farther away from the focus point, the blurrier the image becomes. Therefore, by setting user B's face as the focus point, the blurriness of users A and C can be kept to a minimum. Conversely, when the focus point is automatically focused on user A's face, user B's blurriness is relatively low, while user C's blurriness is relatively high, with the blurriness of B and C increasing sequentially. Thus, through the above embodiment, the blurriness of portraits far from the focus point can be reduced, ensuring that the sharpness of all portraits remains at a high level.

[0120] In some embodiments, the face with a sharpness value that falls within a preset interval (i.e., the aforementioned "first interval") in the sequence of sharpness values ​​of all faces can be selected. For example, when the sharpness is sorted from largest to smallest, and the preset interval is 45% to 55%, if the preview image is a group photo of four people, namely users D, E, F, and G, the electronic device can obtain the following face sequence by sorting all faces from largest to smallest sharpness: user D, user E, user F, and user G.

[0121] The facial sharpness of user E ranks at 2 / 4 = 50% in the sequence. Therefore, the facial sharpness of user E is within the preset range. Using the same method, the facial sharpness rankings of other users are not within the preset range. Therefore, the electronic device can select the face of user E as the face at the focus position.

[0122] If there are multiple faces with a certain level of clarity within a preset range, for example, if the electronic device determines that the clarity of both user H's face and user I's face are within the preset range, the electronic device can choose to determine the focus position based on the face position of user H, or the electronic device can choose to determine the focus position based on the face position of user I.

[0123] In some embodiments, the sharpness is a sharpness other than the first sharpness and the second sharpness among multiple sharpness values ​​corresponding to multiple second faces. Specifically, when the electronic device is focused to the position closest to the camera (the first focus position), the first sharpness can be the highest sharpness value among the multiple second faces, and the second sharpness can be the lowest sharpness value among the multiple second faces. Optionally, the first sharpness can be the lowest sharpness value among the multiple second faces, and the second sharpness can be the highest sharpness value among the multiple second faces. The first sharpness and the second sharpness are the highest and lowest sharpness values ​​among the second faces, respectively. The specific values ​​of the first and second sharpness can be selected according to different applications, and this application embodiment does not limit this.

[0124] In some embodiments, the electronic device may use the Sobel operator, the Tenengrad gradient method, the Laplacian operator, and the sum of modulus of gray difference (SMD) method to calculate facial sharpness. For specific calculation procedures, please refer to relevant technologies; they will not be elaborated upon here.

[0125] In some embodiments, the electronic device can detect the position of a face in a preview image using face detection technology, and obtain, for example... Figure 5 The image shows the face region. The electronic device can then calculate the sharpness of the image within the face region as the face sharpness.

[0126] In some embodiments, where the electronic device can measure the distance between a face and the camera, the electronic device can select, from all faces, the face whose focus is between the face closest to the camera and the face farthest from the camera.

[0127] In some embodiments, the electronic device can also determine the distance between a face and the camera by the size of the face area; for example, the closer the face is to the camera, the larger its area.

[0128] Based on the above embodiments, when the electronic device detects the position of a face in the preview image using face detection technology, it can also detect the number of faces.

[0129] In one scenario, when there is only one face in the preview image, the electronic device can automatically focus on that face. In this case, there is no problem with the face being blurry. Therefore, the electronic device can skip step S403 and directly execute steps S404 and S405.

[0130] In another scenario, if the electronic device detects that there is more than one face in the preview image using face detection technology, i.e., the number of faces is greater than one, the autofocus will focus on one of the faces, and the other faces may be blurry. In other words, there is a problem of blurred faces. Therefore, the electronic device can calculate the sharpness of all faces in the preview image and determine the focus position. After that, it continues to execute the following step S403.

[0131] Step S403: Control the camera to focus at the focus position.

[0132] As in the example above, once the electronic device determines the focus position, it can control the camera lens to move to the corresponding position, thus achieving focus at the focus position.

[0133] In some embodiments, such as Figure 5 As shown, after controlling the camera to focus, it can be seen that face region 2 changed from blurry to clear, while face region 1 changed from clear to blurry. Compared to focusing on face region 2 instead of face region 1, the blurriness of face region 3 is reduced.

[0134] Step S404: In response to the shutter being pressed, acquire the captured image (i.e., the aforementioned "second image").

[0135] Optionally, the electronic device may also perform the following step S405:

[0136] Step S405: Input the captured image into the image processing network to obtain a photo.

[0137] In some embodiments, after controlling the camera to focus at the focus position, the electronic device, in response to the user pressing the shutter button, can acquire a captured image obtained by the photosensitive element converting light signals into electrical signals. This captured image is a raw image, also known as a digital negative; therefore, the capturing device can be input into an image processing network, and after processing by the image processing network, a photograph is obtained.

[0138] The training data for the image processing network can be the original image, or the processed image obtained after the original image has undergone processes such as bad pixel correction, noise reduction, color interpolation, automatic white balance, and color correction. Thus, after a captured image is input into the image processing network, the processed photo can be obtained.

[0139] In some embodiments, when the captured image is an original image, after the electronic device performs step S404, the captured image can also be input to an image processor of the related technology to obtain a photo that can be viewed by the user. The image processor of the related technology can process the original image and perform image processing such as bad pixel correction, noise reduction, color interpolation, automatic white balance, and color correction.

[0140] In some embodiments, according to the above example, when the focus point is on user B's face, and the faces of user A and user C are both within the depth of field of this focus point location, the resulting image, while not the sharpest, is within the range of sharpness visible to the human eye. Therefore, the images of users A, B, and C can be considered sharp. In this case, an image including sharp faces can be directly obtained. However, when the focus point is on user B's face, and the faces of user A and user C are not within the depth of field of this focus point location, the resulting image is blurry to the human eye.

[0141] Therefore, in order to ensure that the image processing network can process the original image and obtain a clear output image, a group photo of multiple people where every face is clear (i.e., the aforementioned "fourth image", "a group photo of multiple people where every face is clear" means "the clarity of each face in the fourth image is greater than the second preset threshold") and a group photo of multiple people where the faces closest to and farthest from the lens are the most blurred, and the faces at a moderate distance from the lens are the clearest (i.e., the aforementioned "third image", "a group photo of multiple people where the faces closest to and farthest from the lens are the most blurred, and the faces at a moderate distance from the lens are the clearest" means "the clarity of the third face in each third image is greater than the first preset threshold, the clarity of the faces other than the third face in the third image is less than or equal to the first preset threshold, and the distance between the third face and the camera is between the face closest to the camera and the face farthest from the camera") can be used as training data to train the image processing network.

[0142] Among them, a group photo where every face is clear can be taken with an SLR camera, and all faces are within the depth of field; therefore, all faces in this photo can be considered clear. The faces closest to and furthest from the lens are the most blurry, while the faces at a moderate distance from the lens are the clearest (an example from the third image). This can be obtained by degrading a group photo where every face is clear (an example from the fourth image).

[0143] Image degradation refers to reducing the quality of an original image through various means, resulting in a low-quality image. Forms of image degradation generally include blurring, distortion, and the presence of noise. Therefore, image degradation can be achieved by increasing the image's blurriness or adding noise. There are numerous methods for image degradation processing in related technologies, which will not be listed here.

[0144] In some embodiments, the electronic device can also sharpen a group photo captured by a DSLR camera, in which all the portraits are within the depth of field, to highlight the details in the group photo, further improve the clarity of the group photo, and ensure the clarity of the output image of the trained image processing network.

[0145] The sharpening methods can include unsharp masking, Laplacian sharpening, etc. For details on these methods, please refer to relevant technical documentation; they will not be elaborated upon here.

[0146] In other embodiments, when faces are not clear in the images captured by the SLR camera, the electronic device can first process the captured images to obtain images in which every face is clear (i.e., the aforementioned "fourth image"). Then, the clear images are subjected to image degradation processing to obtain images in which the faces closest to and farthest from the lens are the most blurry, and the faces at a moderate distance from the lens are the clearest (i.e., the aforementioned "third image").

[0147] In this way, after the captured image is processed by the image processing network, the blurry faces that may still exist in the image can be restored, resulting in a clearer photo.

[0148] In some embodiments, the image processing network may be pre-trained before the electronic device leaves the factory, and the image processing network may be stored in the electronic device before the electronic device leaves the factory. Optionally, the electronic device may update the image processing network through system updates.

[0149] In some embodiments, the electronic device may also perform post-processing on the captured image obtained in step S404, for example, sharpening the captured image to make the details in the captured image obtained in step S404 more prominent, thereby obtaining a clearer effect.

[0150] In some examples, the electronic device can first determine the sharpness of each face in the captured image, and then perform sharpening processing on the images of faces with a sharpness lower than a preset face sharpness threshold (i.e., the aforementioned "fifth face") to improve the sharpness of the face.

[0151] In other examples, after the electronic device first determines the sharpness of each face, it can perform different degrees of sharpening based on the sharpness of each face. The relationship between the degree of sharpening and the sharpness of the face is inversely proportional; that is, the higher the image sharpness, the lower the degree of sharpening, and vice versa.

[0152] In some embodiments, the electronic device can also perform post-processing on the photo obtained in step S405. For specific post-processing, please refer to the above embodiments, which will not be repeated here.

[0153] Using the shooting method described above, the electronic device can determine a suitable focus position in the scene of a group photo, thereby reducing the blurriness of each face image. Then, based on the image processing network, the image captured after the shutter is pressed can be processed to obtain a group photo in which each face is clear.

[0154] The following is a schematic diagram of data stream transmission for a shooting method provided in an embodiment of this application.

[0155] like Figure 7 As shown, the system architecture of an electronic device includes an application layer, a framework layer, a hardware abstraction layer, a kernel layer, and a hardware layer. For details on the modules included in each layer and the specific function of each module, please refer to the above. Figure 3 The relevant details will not be repeated here.

[0156] When a user taps the camera application installed on an electronic device, the device responds by launching the camera application. Within the camera application, if the user selects selfie mode, the application sends a command to the camera algorithm library via the camera service, enabling the library to begin acquiring preview images.

[0157] When the camera application is in selfie mode, the front-facing camera's photosensitive element begins to capture images. After simple processing by the image signal processor, the image captured in real time is sent to the camera application via the camera driver, camera device, and camera service. The camera application then displays the received image on the electronic device's display interface in real time, providing the user with a preview.

[0158] The camera algorithm library can acquire images from the camera device to provide a preview to the user and determine the preview image from them.

[0159] The camera algorithm library can determine the focus position based on the clarity of all faces in the preview image and send a focus command to the camera driver so that the camera driver can drive the focus motor to move the lens to the position corresponding to the focus command (i.e., the aforementioned "second focus position"), thereby controlling the camera to focus to the focus position determined by the camera algorithm library.

[0160] When a user clicks the shutter button displayed on an electronic device, the camera algorithm library can detect this action via the camera service. In response, the library uses an image signal processor and camera driver to acquire the image captured by the sensor. It then processes this image using a pre-defined image processing network to produce a clear photograph. Finally, the camera algorithm library sends the clear photograph to a photo library via the camera service.

[0161] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform the various functions or steps described in the method embodiments.

[0162] This application also provides a computer program product, including a computer program that, when run on an electronic device, causes the electronic device to perform the various functions or steps described in the above method embodiments.

[0163] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0165] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0166] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0167] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A photographing method characterized by comprising: The application is applied to an electronic device, comprising: obtaining a first image collected by a camera of the electronic device focusing on a first focus position; detecting a plurality of second human faces in the first image and calculating a sharpness of each second human face; determining a first human face from the plurality of second human faces according to the sharpness of each second human face, the first human face satisfying at least one of the following conditions: the sharpness is in a first interval, the sharpness is a median value among a plurality of sharpnesses corresponding to the plurality of second human faces, and the sharpness is a value other than a first sharpness and a second sharpness among the plurality of sharpnesses corresponding to the plurality of second human faces; controlling the camera to focus on a second focus position based on a position of the first human face in the first image, the second focus position corresponding to the position of the first human face in the first image, and a distance between the first human face and the camera being between a closest human face to the camera and a farthest human face to the camera; obtaining a second image in response to a shutter pressing operation, the second image being collected by the camera focusing on the second focus position.

2. The method of claim 1, wherein, The first focus position corresponds to the closest human face to the camera.

3. The method of claim 2, wherein, The first sharpness is a lowest value among the plurality of sharpnesses corresponding to the plurality of second human faces, and the second sharpness is a highest value among the plurality of sharpnesses corresponding to the plurality of second human faces.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: processing the second image to obtain a photo.

5. The method of claim 4, wherein, The processing of the second image comprises: processing the second image according to an image processing network model; the image processing network model is trained based on a plurality of third images and a plurality of fourth images, each third image including a plurality of human faces, a third human face in each third image having a sharpness greater than a first preset threshold, and a human face other than the third human face in the third image having a sharpness less than or equal to the first preset threshold, the third human face being between the closest human face to the camera and the farthest human face to the camera, and each fourth image including the same human faces as the third image, each human face in the fourth image having a sharpness greater than a second preset threshold.

6. The method of claim 5, wherein, The third image is obtained by image degradation of the fourth image.

7. The method according to claim 5 or 6, characterized in that, The processing of the second image comprises: sharpening the human faces in the second image.

8. The method of claim 7, wherein, The sharpening of the human faces in the second image comprises: determining a fifth human face in the second image, the fifth human face having a sharpness less than a third preset threshold; sharpening the fifth human face.

9. The method of any one of claims 1-3, 5-6, 8, wherein, The controlling of the camera to focus on the second focus position comprises: searching for the second focus position in a stored mapping relationship, the mapping relationship including a position in an image and a focus position corresponding to the position in the corresponding image, and the position in the image including the position of the first human face; controlling a focus point of the camera to switch from the first focus position to the second focus position.

10. The method of any one of claims 1-3, 5-6, 8, wherein, The camera is an auto-focus (AF) camera.

11. The method of any one of claims 1-3, 5-6, 8, wherein, The camera is a front-facing camera.

12. An electronic device, comprising: The electronic device includes a display screen, a memory, and one or more processors; the display screen, the memory, and the processors are coupled; the display screen is configured to display images generated by the processors, the memory is configured to store computer program code, the computer program code includes computer instructions; when the processors execute the computer instructions, the electronic device performs the method of any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, Computer instructions are included, when the computer instructions run on an electronic device, the electronic device performs the method of any one of claims 1-11.

Images