Shooting method and device, electronic equipment, readable storage medium and chip

Abstract

The invention relates to the technical field of electronic equipment, provides a shooting method and device, electronic equipment, a readable storage medium and a chip, and can solve the problem that in the prior art, if face shielding occurs in a shooting scene, a user cannot find the face shielding in time, and the shooting effect of a photo is affected. The method is applied to the electronic equipment and comprises the steps that a preview image is displayed in the shooting process, and the preview image comprises at least one shooting object; when the first shooting object in the preview image is shielded, displaying an image when the first shooting object is not shielded; and / or when the second shooting object in the preview image does not completely enter the shooting range, displaying prompt information which is used for indicating that the second shooting object does not completely enter the shooting range.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to a photographing method, device, electronic equipment, readable storage medium and chip. Background Art

[0002] At present, electronic devices such as mobile phones, tablet computers, and cameras all have shooting functions and can record various life scenes, such as taking group photos. Usually, there are a large number of subjects in a group photo shooting scene, so the subjects need to stand in the front and back rows in an alternating manner during the shooting process. At this time, the photos taken by users through electronic devices are prone to the situation where the subjects in the front row of the shooting scene block or half-block the faces of the subjects in the back row. Users need to carefully check whether there is any face occlusion. When there is face occlusion, if the user fails to discover it in time, it will affect the shooting effect of the photo. Summary of the invention

[0003] The present application provides a shooting method, device, electronic device, readable storage medium and chip, which solve the problem in the prior art that if a face is blocked in the shooting scene and the user fails to discover it in time, the shooting effect of the photo will be affected.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, a shooting method is provided, which is applied to an electronic device, the method comprising: displaying a preview image during a shooting process, the preview image including at least one shooting object; when a first shooting object is blocked in the preview image, displaying an image when the first shooting object is not blocked; and / or, when a second shooting object does not completely enter a shooting range in the preview image, displaying a prompt message, the prompt message being used to indicate that the second shooting object has not completely entered the shooting range.

[0006] In the technical solution provided in the present application, the electronic device can display an image of the first photographed object when it is not obstructed when there is an obstructed first photographed object in the preview image. After learning the image of the first photographed object when it is not obstructed, the photographer can determine the specific information of the first photographed object based on the image, and then make targeted adjustments, thereby improving the shooting efficiency and shooting effect of the photo. In addition, the electronic device can also display prompt information when the photographed object in the preview image does not completely enter the shooting range. The photographer can adaptively adjust the position of the photographed object or the shooting angle of the electronic device according to the intelligent prompt, thereby improving the shooting effect of the photo and enhancing the user experience.

[0007] In this embodiment, the object being blocked includes the key points of the object being partially blocked (half blocked) and the key points of the object being completely blocked (full blocked). The blocking also includes being blocked by an obstruction or being blocked by other objects.

[0008] In some embodiments, when a first subject is obscured in a preview image, an image when the first subject is not obscured is displayed, including: when the first subject is obscured in the preview image, determining position information of the first subject; displaying prompt information for indicating that the subject is obscured according to the position information; and in response to a first control operation, displaying an image when the first subject is not obscured.

[0009] The first control operation may be any one of a click, a double click, and a long press. The electronic device responding to the first control operation includes: the electronic device responding to a user's click operation on a prompt information indicating that the photographed object is blocked; or the electronic device responding to a user's double click operation at any position of the preview interface, etc.

[0010] In some embodiments, the method further includes: in response to a second control operation, exiting displaying the image when the first photographed object is not blocked.

[0011] The second control operation may be any one of a click, a double click, and a long press. The electronic device responding to the second control operation includes: the electronic device responding to a click operation of the user on the image when the first photographed object is not blocked; or the electronic device responding to a double click operation of the user at any position of the preview interface, etc.

[0012] In some embodiments, when the first subject is obscured in the preview image, an image of the first subject when it is not obscured is displayed, including: when the first subject is obscured by an obstructing object in the preview image, the image of the first subject when it is not obscured and the image of the obstructing object are displayed.

[0013] In other embodiments, when the first photographic subject is obscured in the preview image, an image of the first photographic subject when it is not obscured is displayed, and also includes: when the first photographic subject is obscured by an obstructing object in the preview image, only the image of the first photographic subject when it is not obscured is displayed, and the image of the obstructing object is not displayed.

[0014] In some embodiments, after displaying a preview image during the shooting process, when the first subject is blocked in the preview image, an image when the first subject is not blocked is displayed; or, when the second subject has not completely entered the shooting range in the preview image, before displaying the prompt information, the method also includes: switching the shooting mode to a group photo shooting mode.

[0015] In one implementation, switching the shooting mode to the group photo shooting mode includes: in response to a user operation, switching the shooting mode to the group photo shooting mode.

[0016] In another implementation, switching the shooting mode to the group photo shooting mode includes: when it is recognized that the preview image includes at least two shooting subjects, switching the shooting mode to the group photo shooting mode.

[0017] When users use electronic devices to take group photos, it is easy for the faces of the subjects to be blocked or the subjects to not completely enter the shooting range. Therefore, the electronic device can determine the first subject and / or the second subject in the preview image when the shooting mode is the group photo shooting mode, thereby improving the shooting effect of the photo and enhancing the user experience.

[0018] In some embodiments, when the subject in the preview image is a person, the first subject is determined according to the following method: key point detection is performed on the subject in the preview image to determine the key points and key point information corresponding to each subject; and the subject whose key points are not fully displayed in the preview image is determined as the first subject that is semi-occluded in the preview image.

[0019] In some embodiments, the method further includes: extracting a feature vector of a photographed object that is not semi-occluded in the preview image, the feature vector including a face feature vector or a body feature vector; performing similarity matching between the feature vector of the photographed object in the feature library and the feature vector of the photographed object that is not semi-occluded in the preview image; marking the photographed object in the feature library whose similarity is greater than or equal to a preset threshold as a successfully matched photographed object, and determining the photographed object in the feature library whose similarity is less than a preset threshold as the first photographed object that is fully occluded in the preview image, wherein the feature library includes identification information of all photographed objects that appear in the shooting process, as well as the feature vector, key point information and close-up face image of each photographed object, and the identification information is used to uniquely indicate the corresponding photographed object.

[0020] In this embodiment, the key point information includes the position information of the key point, the key point category, the key point confidence, etc.

[0021] It should be noted that the close-up face image in the feature library is continuously updated as the preview image changes and the recognition process of the first photographed object. Therefore, the feature library records the latest close-up face image of the photographed object. The feature library can be a face feature library or a body feature library. The feature library generated by the electronic device during the shooting process will be cleared each time the shooting function is restarted, and will be regenerated when the shooting function is turned on again.

[0022] In other embodiments, when the subject in the preview image is a person, the first subject is determined in the following manner: extracting a feature vector of the subject in the preview image; performing similarity matching between the feature vector of the subject in the preview image and the feature vector of the subject in a feature library, wherein the feature library is generated based on the preview stream during the shooting process, and the feature library includes identification information of all subjects appearing in the shooting process, as well as the feature vector, key point information and close-up image of the face of each subject, and the identification information is used to uniquely indicate the corresponding subject; marking the subjects in the feature library whose similarity is greater than or equal to a preset threshold as successfully matched subjects, and determining the subjects in the feature library whose similarity is less than the preset threshold as the first subjects that are obstructed in the preview image.

[0023] In some embodiments, the method further includes: determining the number of successfully matched subjects in the feature library; determining unsuccessfully matched subjects in the preview image when the number of successfully matched subjects is less than the number of subjects in the feature vector extracted from the preview image; generating identification information of unsuccessfully matched subjects in the preview image; adding the identification information of unsuccessfully matched subjects in the preview image, as well as the feature vector, key point information and close-up face image of the unsuccessfully matched subjects to the feature library, and marking the unsuccessfully matched subjects as successfully matched in the feature library.

[0024] The electronic device uses the method for determining the first photographic object provided by this embodiment. When a photographic object is not displayed in the preview image (including facial key points not being displayed or body key points not being displayed) or is out of the preview image, the photographic object may be blocked by other photographic objects, or may appear in the previous frame of the preview image but not in the next frame of the preview image. The method can identify the photographic object.

[0025] In some embodiments, when the second photographic object in the preview image has not completely entered the photographing range, prompt information is displayed, including: determining an edge preset area of ​​the preview image; detecting key points and key point information of the photographic object in the edge preset area; determining the photographic object whose key points are not fully displayed as the second photographic object; determining the position information of the second photographic object; and displaying prompt information based on the position information.

[0026] In some embodiments, the key points include at least one of the facial key points and the body key points of the photographed subject.

[0027] In a second aspect, a shooting device is provided, which is applied to an electronic device, and the device includes: a first display module, which is used to display a preview image during the shooting process, wherein the preview image includes at least one shooting object; a second display module, which is used to display an image of the first shooting object when the first shooting object is blocked in the preview image; and / or, when the second shooting object does not completely enter the shooting range in the preview image, display a prompt message, wherein the prompt message is used to indicate that the second shooting object has not completely entered the shooting range.

[0028] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method shown in the first aspect when executing the computer program.

[0029] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method shown in the first aspect is implemented.

[0030] In a fifth aspect, a chip is provided, the chip comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method shown in the first aspect is implemented.

[0031] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1A A schematic diagram of displaying a preview image during shooting provided by an embodiment of the present application;

[0033] Figure 1B A schematic diagram of displaying a preview image during shooting provided by another embodiment of the present application;

[0034] Figure 2 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application;

[0035] Figure 3 A schematic diagram of a shooting method when a shooting object is blocked provided by an embodiment of the present application;

[0036] Figure 4 A schematic diagram of a shooting interface provided by an embodiment of the present application;

[0037] Figure 5A A schematic diagram of a switching process of a shooting mode provided in an embodiment of the present application;

[0038] Figure 5B A schematic diagram of a switching process of a shooting mode provided in another embodiment of the present application;

[0039] Figure 6 A schematic diagram of a scene in which a photographed object is blocked by an obstruction provided in an embodiment of the present application;

[0040] Fig. 7A A schematic diagram of a scenario for displaying prompt information provided by an embodiment of the present application;

[0041] Figure 7B A schematic diagram of a scenario for displaying prompt information provided by another embodiment of the present application;

[0042] Fig. 8A A schematic diagram of a process in which an electronic device according to an embodiment of the present application displays an image of a first photographed object that is not blocked;

[0043] Figure 8B A schematic diagram of a process in which an electronic device displays an image of a first photographed object when the first photographed object is not blocked, provided by another embodiment of the present application;

[0044] Figure 8C Schematic diagram of a process of an electronic device displaying an image of a first photographed object when it is not blocked, provided in another embodiment of the present application

[0045] Fig. 9 A schematic diagram of a process in which an electronic device according to an embodiment of the present application exits displaying an image when a first photographed object is not blocked;

[0046] Fig.10 A schematic flowchart of determining a first photographed object with face occlusion provided by an embodiment of the present application;

[0047] Fig.11 A schematic flowchart of determining a first photographed object with face occlusion provided by another embodiment of the present application;

[0048] Fig.12 A schematic diagram of a photographing method provided by an embodiment of the present application when a photographing object is not completely within a photographing range;

[0049] Fig.13 A schematic diagram of a scene in which a prompt message is displayed when a second photographing object has not completely entered the photographing range provided by an embodiment of the present application;

[0050] Fig.14 A schematic diagram of the structure of a photographing device provided in an embodiment of the present application;

[0051] Fig.15 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solution provided in the embodiments of the present application is described below with reference to the accompanying drawings.

[0053] It should be understood that in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0054] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0055] At present, electronic devices such as mobile phones, tablet computers, and cameras all have shooting functions, and can record various life scenes, such as recording the growth status of children at different stages, shooting commemorative videos, and taking group photos, etc. Among them, when electronic devices take group photos, due to the influence of factors such as shooting angle, ambient light, and the standing position of the subjects, the faces of some subjects are easily blocked to varying degrees in the shooting screen of the electronic device, resulting in the subjects' faces not being fully displayed in the captured pictures, affecting the shooting effect.

[0056] In order to obtain a better shooting effect, when taking a group photo, when the electronic device detects that there is a face occlusion in the preview image, it can display a prompt message at the corresponding position of the preview image to remind the photographer that there is an occlusion at that position. The photographer adjusts the position of the occluded person according to the prompt message, and then retakes the photo to obtain a picture with no face occlusion. For example, see Figure 1A As shown in , when the face of subject A is partially blocked by subject B, the electronic device can display a prompt box at the location of the face of subject A to prompt the photographer that the face of subject A is blocked. The photographer adjusts the position of subject A according to the prompt information and then reshoots, thereby obtaining a picture with an unblocked face.

[0057] In some embodiments, when the electronic device determines whether there is face occlusion in the preview image, it performs face key point detection on the current frame preview image by running a face detection algorithm and a face key point detection algorithm. If the face key point information of a subject in the current frame preview image is not fully displayed, it is determined that the subject has face occlusion, and a prompt message is displayed at the location of the subject. However, the electronic device can only display the prompt message when it recognizes that the face is partially occluded in this way. When there is full face occlusion in the preview image, for example, see Figure 1A As shown in , when subject D located in the front row of subject C completely blocks the face of subject C, the face cannot be identified and confirmed by this method.

[0058] In addition, when users use electronic devices to take group photos, there is usually a situation where the subjects at the edge of the preview image are not completely within the shooting range, that is, only a part of the people at both sides of the picture are displayed in the picture, and the rest are not displayed in the picture. Figure 1B As shown in the figure, the subjects E and F located at the edge of the preview image are not completely within the shooting range of the electronic device, resulting in partial body cutoff of the subjects by the edge of the frame. As for the situation where the subjects are not completely within the shooting range of the electronic device, there is currently no relevant technology that can intelligently identify and give prompt information.

[0059] To this end, an embodiment of the present application provides a shooting method, through which an electronic device can provide an intelligent prompt when a certain shooting object in a preview image is blocked or not completely within the shooting range during shooting. The photographer can adaptively adjust the position of the shooting object according to the intelligent prompt, thereby improving the shooting effect of the photo and enhancing the user experience.

[0060] The shooting method provided in the embodiment of the present application can be applied to electronic devices including cameras and display screens, such as mobile phones, tablet computers, cameras, video cameras, wearable devices (e.g., watches, bracelets, helmets, headphones, etc.), vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), smart home devices (e.g., smart TVs, smart speakers, smart cameras, etc.), and other electronic devices. It is understandable that the embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0061] In one possible implementation, the camera of the electronic device may be a rear camera. In another possible implementation, the camera of the electronic device may be a front camera. The embodiment of the present application does not impose any restriction on the position of the camera.

[0062] Figure 2 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0063] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0064] The processor 210 may include one or more processing units, for example: the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. Among them, the controller can be the nerve center and command center of the electronic device. The controller can generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0065] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0066] The charging management module 240 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 may receive charging input from a wired charger through the USB interface 230. In some wireless charging embodiments, the charging management module 240 may receive wireless charging input through a wireless charging coil of the electronic device. While the charging management module 240 is charging the battery 242, it may also power the electronic device through the power management module 241.

[0067] The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240, and supplies power to the processor 210, the internal memory 221, the external memory, the display screen 294, the camera 293, and the wireless communication module 260. The power management module 241 can also be used to monitor parameters such as battery capacity, battery cycle number, and battery health status (leakage, impedance).

[0068] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor and baseband processor.

[0069] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0070] The mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 250 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0071] In some embodiments, at least some functional modules of the mobile communication module 250 may be disposed in the processor 210. In some embodiments, at least some functional modules of the mobile communication module 250 and at least some modules of the processor 210 may be disposed in the same device.

[0072] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio playback device (not limited to a speaker 270A, a receiver 270B, etc.), or displays an image or video through a display screen 294. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 210 and be set in the same device as the mobile communication module 250 or other functional modules.

[0073] The wireless communication module 260 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to electronic devices. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 210. The wireless communication module 260 can also receive the signal to be sent from the processor 210, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0074] The camera 293 is used to capture static images or videos. The object generates an optical image through the camera 293 and projects it onto the sensor module 280. The sensor module 280 converts the optical signal into an electrical signal, and then transmits the electrical signal to the processor 210 to convert it into a digital image signal. For example, when taking a photo, the shutter is opened, and the light is transmitted to the sensor module 280 through the lens. The optical signal is converted into an electrical signal, and the sensor module 280 transmits the electrical signal to the processor 210 for processing and converting it into an image visible to the naked eye. The processor 210 can also perform algorithm optimization on the noise, brightness, and skin color of the image. The processor 210 can also optimize the exposure, color temperature and other parameters of the shooting scene.

[0075] The camera 293 collects the video image to form preview image data. The processor 210 preprocesses the preview image data to obtain a preview stream. The preprocessing may include filtering and noise reduction, target enhancement, image segmentation and other processing.

[0076] In some embodiments, the electronic device may include 1 or N cameras 293 , where N is a positive integer greater than 1.

[0077] The display screen 294 is used to display images, videos, etc., such as various types of device management interfaces in the embodiments of the present application. The display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 294, where N is a positive integer greater than 1.

[0078] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0079] The internal memory 221 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 210 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 221. The internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.).

[0080] In addition, the internal memory 221 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0081] The electronic device can implement audio functions through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone interface 270D, and the application processor.

[0082] The audio module 270 is used to convert digital audio signals into analog audio signals for output, and is also used to convert analog audio inputs into digital audio signals. The audio module 270 can also be used to encode and decode audio signals. In some embodiments, the audio module 270 can be arranged in the processor 210, or some functional modules of the audio module 270 can be arranged in the processor 210.

[0083] The speaker 270A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device can listen to music or listen to hands-free calls through the speaker 270A. For example, the speaker can play the comparison analysis results provided in the embodiment of the present application.

[0084] The receiver 270B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or voice message, the voice can be received by placing the receiver 270B close to the human ear.

[0085] Microphone 270C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 270C to input the sound signal into the microphone 270C. The electronic device can be provided with at least one microphone 270C. In other embodiments, the electronic device can be provided with two microphones 270C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 270C to realize the collection of sound signals, noise reduction, identification of sound sources, realization of directional recording function, etc.

[0086] In some embodiments, the electronic device may receive an ultrasonic signal sent by another electronic device through the microphone 270C, and identify the frequency and reception strength of the ultrasonic signal through the processor 210 .

[0087] The earphone interface 270D is used to connect a wired earphone and can be a USB interface 230 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0088] The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0089] The key 290 includes a power key, a volume key, etc. The key 290 can be a mechanical key or a touch key. The electronic device can receive key input and generate key signal input related to user settings and function control of the electronic device.

[0090] Motor 291 can generate vibration prompts. Motor 291 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Motor 291 can also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 294.

[0091] Indicator 292 may be an indicator light, which may be used to indicate the charging status of the electronic device, changes in power level, messages, missed calls, notifications, etc.

[0092] The SIM card interface 295 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device by inserting the SIM card interface 295 or removing the SIM card from the SIM card interface 295. The electronic device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 295 at the same time. The types of the multiple cards can be the same or different.

[0093] The following combination Figure 2 The electronic device shown exemplarily illustrates the shooting method provided in the embodiment of the present application through two aspects: (a) the shooting object is blocked, and (b) the shooting object does not completely enter the shooting range.

[0094] (1) The subject is obscured

[0095] During the shooting process, the electronic device usually displays a preview image, which includes at least one shooting object. In this embodiment, the electronic device can identify the preview image, and when there is an obstructed shooting object in the preview image, the electronic device displays the image of the shooting object when it is not obstructed on the preview interface. The details are as follows.

[0096] Figure 3A schematic diagram of a method for photographing when a photographing object is blocked provided in an embodiment of the present application, see Figure 3 As shown, the method includes the following steps S301 to S306.

[0097] S301, the electronic device displays a preview image during shooting, where the preview image includes at least one shooting object.

[0098] The photographed object may be a person, or a static object, animal or plant, etc.

[0099] In some embodiments, the electronic device enters shooting mode in response to user operation and displays a preview image. For example, in a terminal device with shooting function such as a mobile phone or tablet computer, when a user clicks the camera application icon in the terminal device, or presses a shortcut key for starting the camera application (such as long pressing the power button, or double-clicking the volume button, etc.), the terminal device enters shooting mode in response to the user operation and displays a preview image during shooting.

[0100] In other embodiments, the electronic device enters the shooting mode immediately after being powered on and displays the preview image. For example, for a dedicated shooting device such as a camera or a camcorder, after the user presses the power button of the shooting device, the shooting device enters the shooting mode in response to the user operation, and then displays the preview image during the shooting process.

[0101] For example, in response to a user clicking on a camera application icon in a mobile phone, the electronic device displays the following Figure 4 The shooting interface shown. Figure 4 As shown, the shooting interface includes a first display area located in the middle part, and a second display area and a third display area located on both sides of the first display area. The first display area is used to display the preview image, and the image zoom adjustment option is also included in the display area; the second display area is used to display the operation options for adjusting the preview image parameters, such as exposure, brightness and color contrast; the third display area is used to display the operation options of the shooting type, such as taking pictures, recording videos, switching between the front and rear cameras, and shooting mode.

[0102] S302: The electronic device switches the shooting mode to a group photo shooting mode.

[0103] The electronic device includes multiple shooting modes, such as portrait mode, panorama mode, night scene mode, and group photo shooting mode, etc. The electronic device can switch the shooting mode to the group photo shooting mode in response to a user operation, or automatically switch the shooting mode to the group photo shooting mode through scene recognition.

[0104] In one implementation, the electronic device switches the shooting mode to the group photo shooting mode in response to the user's control operation on the shooting interface. The control operation may be that the user clicks on the group photo operation option in the third display area, see Figure 5A or double-click or slide the capture interface, see Figure 5B as shown in .

[0105] In another implementation, the electronic device recognizes the preview image in the shooting interface through a scene recognition algorithm, and switches the shooting mode to the group photo shooting mode when it is detected that the preview image includes facial key points of at least two shooting subjects.

[0106] S303: The electronic device identifies a first photographed object that is blocked in the preview image.

[0107] The first photographed object may be any photographed object in the preview image, and the first photographed object may also be referred to as an obscured object.

[0108] In this embodiment, the first photographed object being blocked includes the first photographed object being partially blocked or the first photographed object being fully blocked. The first photographed object being partially blocked means that the key points of the first photographed object are partially blocked, that is, the key points of the first photographed object are partially displayed in the preview image. For example, see Figure 1A As shown in , the subject B located in the front row of the preview image blocks the subject A located in the back row of the preview image, resulting in the facial key points of subject A being partially displayed in the preview image. At this time, subject A is the first subject. The first subject being fully blocked means that all the key points of the first subject are blocked, that is, the key points of the first subject are not displayed in the preview image. For example, see Figure 1A As shown in , the subject D located in the front row of the preview image completely blocks the subject C located in the back row of the preview image, resulting in that the key points of the face of the subject C are not displayed in the preview image. At this time, the subject C is the first subject.

[0109] In some embodiments, when the subject in the preview image is a person, the key point may be a face key point or a body key point of the first subject. Among them, the face key points are used to describe the facial features present in the image. The face key points include two eyes, the tip of the nose, and the two corners of the mouth. The face key points being blocked include the two corners of the mouth and the tip of the nose being blocked, or the two eyes, the tip of the nose, and the two corners of the mouth being blocked. The body key points are used to describe the body features present in the image. For example, the body key points may specifically be 17 joints, such as nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles.

[0110] In this embodiment, the electronic device can determine the facial key points of each photographed object through a facial key point detection algorithm, and determine the facial key point information of the photographed object based on the facial key points, wherein the facial key point information includes the position information of the facial key points, the facial key point category, and the facial key point confidence, etc. The electronic device can also determine the human body key points of each photographed object through a key point confidence detection algorithm and a part affinity fields (PAFs) detection algorithm, and determine the human body key point information of the photographed object based on the human body key points, wherein the human body key point information includes the position information of the human body key points, the human body key point category, and the human body key point confidence, etc.

[0111] In some other embodiments, when the photographed object in the preview image is a static object, the key point may be a boundary contour point of the static object.

[0112] In addition, the first photographed subject being blocked also includes the first photographed subject being blocked by other photographed subjects or blocked by blocking objects, for example, in a group photo scene, the photographed subjects in the front row block the photographed subjects in the back row; wherein the blocking object refers to an object other than the photographed subject within the shooting range. For example, see Figure 6 As shown in , the leaves in the shooting range block one eye and the tip of the nose of the shooting subject G, causing the key points of the face of the shooting subject G to be displayed in the preview image. At this time, the shooting subject G is the first shooting subject. Or objects such as the user's fingers block the camera, thereby blocking the shooting subject.

[0113] S304: The electronic device displays prompt information, where the prompt information is used to indicate that the first photographed object is blocked.

[0114] For example, the prompt information may be text information, such as "face blocked" (the electronic device may display this when the face of the first photographed subject is partially blocked), or "face disappeared" (the electronic device may display this when the face of the first photographed subject is completely blocked), etc. Fig. 7A As shown in (a) and (b) of FIG. 1 ; it may also be a prompt box, and text information such as "click to view" is displayed around the prompt box to guide the photographer to view the blocked first shooting object, see Figure 7B as shown in .

[0115] In some embodiments, when the electronic device determines that the first photographed subject is blocked, it determines the position information of the first photographed subject in the preview image, and displays prompt information according to the position information. For example, the electronic device determines the position of the face of the first photographed subject, and then displays prompt information in a preset area. The preset area can be a rectangular area including the position of the face of the first photographed subject, see Figure 7BIt should be understood that when the electronic device displays the preview image, the preview image of the current frame is constantly changing with the picture information of the captured scene. Therefore, the position information of the first captured object in the preview image can be determined according to the current position information or historical position information of the first captured object in the preview image.

[0116] S305: In response to the first control operation, the electronic device displays an image of the first photographed object when it is not blocked.

[0117] The first control operation may be any one of a click, a double click and a long press.

[0118] In some embodiments, when the electronic device determines that the first photographed subject is partially obscured by the obstructing object, in response to the user's first control operation on the prompt information, an image of the first photographed subject when it is not obscured and an image of the obstructing object are displayed, see Fig. 8A or, only the image of the first photographed object when it is not blocked is displayed, and the image of the blocked object is not displayed. The blocked object may be any other photographed object in the preview image except the first photographed object.

[0119] In some other embodiments, when the electronic device determines that the first photographed subject is completely blocked by the blocking object, in response to the user's first control operation on the prompt information, an image of the first photographed subject when it is not blocked and an image of the blocking object are displayed, see Figure 8B as shown in .

[0120] In some further embodiments, when the electronic device determines that the first photographed subject is blocked by an obstruction (including partial blocking and full blocking), in response to the user's first control operation on the prompt information, an image of the first photographed subject when it is not blocked is displayed, see Figure 8C as shown in .

[0121] Optionally, when the electronic device identifies that there is an obstructed first photographed object in the preview image, it may not need to execute step S304 and step S305, that is, it may not display prompt information, but directly display the image when the first photographed object is not obstructed.

[0122] S306: In response to the second control operation, the electronic device exits displaying the image when the first photographed object is not blocked.

[0123] The second control operation may be any one of a click, a double click and a long press.

[0124] Exemplarily, the electronic device responds to a double-click operation of the user on any position in the shooting interface, or responds to a click operation of the user on the image when the first shooting object is not blocked, and exits displaying the image when the first shooting object is not blocked, that is, redisplays the preview image, and the display process is shown in FIG. Fig. 9 as shown in .

[0125] It should be noted that when the electronic device recognizes that there are both semi-obstructed and fully-obstructed subjects in the preview image, it displays indication information at both the positions of the semi-obstructed subjects and the fully-obstructed subjects.

[0126] This embodiment provides a display method for an electronic device when a photographic object is blocked. The electronic device can display intelligent prompt information at a corresponding position when the first photographic object is blocked in the preview image, and display an image of the first photographic object when it is not blocked in response to a user's control operation on the indication information, so that the photographer can determine the specific information of the first photographic object based on the displayed image, and then make targeted adjustments, thereby improving the efficiency and effect of photographing. The method in this embodiment can solve the problem of being unable to provide an occlusion prompt when the photographic object is completely blocked during the shooting process.

[0127] The following takes the shooting scene of taking a group photo, the shooting subjects are people, and the recognition of key points of faces as an example to explain two implementation methods of the electronic device identifying the presence of an obscured first shooting subject in the preview image in the above step S303.

[0128] Example 1

[0129] Fig.10 A schematic flowchart of determining a first photographed subject with face occlusion provided by an embodiment of the present application, wherein the occlusion includes half face occlusion and full face occlusion. Fig.10 As shown, the method includes the following steps S1001 to S1007.

[0130] S1001: The electronic device obtains a first preview image.

[0131] The first preview image refers to any preview image frame during the shooting process, which may be the first preview image frame displayed by the electronic device after entering the shooting mode, which may also be called the initial preview image, or any preview image frame displayed by the electronic device during the shooting process.

[0132] S1002: The electronic device performs face detection on the first preview image to generate a face frame for each photographed object.

[0133] The electronic device can detect the face of each subject appearing in the first preview image through a face detection algorithm, and then generate a face frame corresponding to each face, wherein the number of face frames is the number of subjects in the first preview image.

[0134] Exemplarily, after the electronic device performs face detection on the subjects in the first preview image using a face detection algorithm, it determines that there are facial features of 10 people in the first preview image, and then outputs face frames corresponding to the facial features of the 10 people, and determines the number of subjects in the first preview image to be 10 people.

[0135] S1003, the electronic device performs facial key point detection on the face frame to determine facial key point information corresponding to each photographed object.

[0136] Among them, the facial key points refer to the facial features that can be used to indicate the facial features that have appeared in the shooting preview interface, and the facial key points can be used to judge the integrity of the face. For example, the facial key points can specifically refer to 5 facial key points, including two eyes, the tip of the nose, and the two corners of the mouth. The facial key point information includes the facial key points of each photographed object, the location information of the facial key points, the facial key point category, and the facial key point confidence.

[0137] In some embodiments, the electronic device performs facial key point detection on the face frame of each subject in the first preview image through a key point detection algorithm to determine the facial key points of each subject, and determines the facial key point information corresponding to each subject based on the facial key points.

[0138] It should be noted that, when the first preview image is an initial preview image, step S1003 also includes establishing a facial feature library, which is used to record the identification information of each photographed object during the shooting process, as well as the facial feature vectors, facial key point information, and facial close-up images of each photographed object, wherein the identification information is used to uniquely indicate the corresponding photographed object, for example, the identification information can be the identity number of the photographed object, such as identity identifier 1 (IDentity, ID), which indicates photographed object 1, ID2 indicates photographed object 2, and so on, to generate unique identification information corresponding to each photographed object. When the first preview image is not an initial preview image, at this time, the facial feature library already exists and records the identification information of each photographed object during the shooting process, as well as the facial feature vectors, facial key point information, and facial close-up images of each photographed object. Among them, the position information of the facial key points in the facial key point information of the photographed subject is used for the electronic device to display indication information according to the position information of the facial key points when determining that a certain photographed subject is the first photographed subject; the close-up image of the face of the photographed subject is used for the electronic device to display the close-up image of the face, that is, the image of the first photographed subject before being blocked, in response to the user's control operation on the indication information corresponding to the photographed subject when determining that a certain photographed subject is the first photographed subject.

[0139] In addition, it should be noted that the facial feature library generated by the electronic device during the shooting process will be cleared each time the shooting function is restarted, and will be regenerated when the shooting function is turned on again.

[0140] S1004: The electronic device determines the photographed subject whose facial key points are not fully displayed in the first preview image as the first photographed subject whose face is partially obscured in the first preview image.

[0141] In some embodiments, the electronic device performs facial key point detection on the face frame of each subject in the first preview image, and when it is recognized that a subject in the first preview image has only some facial key points, the subject is determined as the first subject. For example, if the electronic device only recognizes the key point information of the two eyes and the tip of the nose of subject 1 through the facial key point detection algorithm, but fails to recognize the two corners of the mouth of subject 1, subject 1 is determined as the first subject. At this time, only the two eyes and the tip of the nose of subject 1 are displayed in the preview image, but the two corners of the mouth of subject 1 are not displayed, that is, the two corners of the mouth of subject 1 are half-occluded.

[0142] S1005: The electronic device extracts facial feature vectors of the photographed subject that is not partially obscured in the first preview image.

[0143] For example, Fig. 7ATaking the preview image shown in the figure as the first preview image as an example, the electronic device has identified that there is a half-occluded first shooting object in the preview image. Then, the facial feature vectors of other shooting objects in the preview image are extracted.

[0144] The facial feature vector refers to the facial features used to identify whether two faces are of the same person. When the similarity between the facial feature vector of a certain photographed subject and the facial feature vector of another photographed subject is greater than or equal to a preset threshold, it indicates that the similarity match is successful, that is, the two photographed subjects are the same person.

[0145] S1006, the electronic device performs similarity matching between the facial feature vector of the photographed subject in the facial feature library and the facial feature vector of the photographed subject that is not partially obscured in the first preview image.

[0146] The electronic device judges the photographed objects under all identification information IDs in the face feature library by matching the face feature vector similarity.

[0147] S1007, the electronic device marks the photographed objects whose similarity in the face feature library is greater than or equal to the preset threshold as the photographed objects that have been successfully matched, and determines the photographed objects whose similarity in the face feature library is less than the preset threshold as the first photographed objects that are fully occluded in the first preview image.

[0148] When the electronic device successfully matches the facial feature vector of a certain photographed object in the facial feature library with the facial feature vector of a certain photographed object in the first preview image without half-occlusion of the face, that is, the feature similarity is greater than or equal to the preset threshold, the facial feature vector corresponding to the photographed object is added or updated in the facial feature library, and the facial key point information and facial close-up image of the photographed object are updated, and the photographed object in the first preview image is marked as successfully matched. When the electronic device fails to compare the facial feature vector of a certain photographed object in the facial feature library with the facial feature vector of the photographed object that is not currently half-occluded, that is, the feature similarity is less than the preset threshold, the facial feature library marks the photographed object in the first preview image as failed to match, indicating that the facial information of the photographed object is not displayed in the current frame preview image, that is, it is fully occluded by the face. The electronic device determines the photographed object as the first photographed object with full face occlusion. When the photographed objects under all identification information IDs in the facial feature library are in the successfully matched state, it indicates that there is no photographed object with full face occlusion in the current frame preview image.

[0149] Optionally, when the electronic device compares the facial features of each subject in the facial feature library with the facial features of the subject whose face is not currently partially occluded for feature similarity, the electronic device simultaneously records the number of subjects that have been successfully matched in the facial feature library, and then compares the number of subjects that have been successfully matched in the facial feature library with the number of subjects whose face is not currently partially occluded in the first preview image. If the number of subjects that have been successfully matched is less than the number of subjects whose face is not currently partially occluded, it indicates that other subjects have been newly added to the first preview image. At this time, the electronic device adds identification information of the other subject in the facial feature library, saves the facial feature vector, facial key point information, and facial close-up image of the other subject, and marks the other subject in the first preview image as successfully matched.

[0150] It should be noted that if the first preview image is an initial preview image, then when the initial preview image does not contain half-occluded subjects and the initially generated face feature library is compared for feature similarity, all subjects that do not contain half-occluded subjects are in a successful matching state. If the first preview image is a non-initial preview image, it will be compared for feature similarity with the face feature vectors of the subjects in the historical face feature library to determine whether there are fully-occluded subjects in the non-initial preview image. The historical face feature library is a face feature library that is compared and continuously updated based on preview images at different times.

[0151] In this embodiment, the electronic device can identify the subjects whose faces are partially or fully obscured in the current preview interface when taking a group photo through a combination of face detection, face key point detection, feature extraction, feature comparison algorithms, etc., thereby determining the first subject with obscured face.

[0152] In addition, the electronic device uses the method for determining the first photographic object provided by the present embodiment. When a photographic object is not displayed in the preview image (including facial key points not being displayed or body key points not being displayed) or is out of the preview image, the photographic object may be blocked by other photographic objects, or may appear in the previous frame of the preview image but not in the next frame of the preview image. The method can identify the photographic object.

[0153] Example 2

[0154] Fig.11 A schematic flowchart of determining a first photographed subject with face occlusion provided by another embodiment of the present application, wherein the occlusion includes half face occlusion and full face occlusion. Fig.11 As shown, the method includes the following steps S1101 to S1105.

[0155] S1101, the electronic device obtains a preview stream during shooting.

[0156] It should be understood that, under normal circumstances, when a photographer uses an electronic device to shoot, he can adjust the shooting angle of the electronic device and / or the position or state of the shooting object according to the content of the picture displayed in the preview image, so that the shooting object can have a better display effect in the preview image, thereby improving the shooting quality of the photo. For example, when taking a group photo, according to the shooting angle and the visual effect of the film, the shooting object with a taller height can be arranged in the middle of the shooting preview image, and the shooting object with a shorter height can be arranged on both sides of the edge of the preview image, and then fine adjustments can be made according to the display effect in the shooting preview image, such as adjusting the brightness and color saturation of the shooting picture in the shooting preview image of the electronic device according to the brightness and color saturation of the external ambient light. In this process, the electronic device is in the process of displaying a shooting preview image including the shooting object. When the screen display effect in the shooting preview interface meets the user's requirements, the user starts the shooting action and completes the photo shooting process.

[0157] Therefore, from the display of the shooting preview interface to the completion of shooting, the electronic device can continuously collect the preview stream during the shooting process through the camera, wherein the preview stream refers to the real-time preview image data of the shot scene obtained by the electronic device through the camera.

[0158] S1102, the electronic device generates a facial feature library of the photographed object according to the preview stream.

[0159] The electronic device detects facial key points on the preview stream and establishes a facial feature library, which includes identification information of all shooting objects appearing during the shooting process, as well as facial feature vectors, facial key point information and facial close-up images of each shooting object. The identification information is used to uniquely indicate the corresponding shooting object.

[0160] S1103, the electronic device extracts a facial feature vector of the subject in the preview image.

[0161] S1104, the electronic device performs similarity matching between the facial feature vector of the photographed object in the preview image and the facial feature vector of the photographed object in the facial feature library.

[0162] S1105, the electronic device marks the photographed objects whose similarity in the face feature library is greater than or equal to a preset threshold as the photographed objects that have been successfully matched, and determines the photographed objects whose similarity in the face feature library is less than the preset threshold as the first photographed objects that are occluded in the preview image.

[0163] In this embodiment, when the electronic device extracts the facial feature vector of the photographed object in the preview image, it extracts the facial feature vectors of all photographed objects (including semi-occluded photographed objects) displayed in the preview image, and compares the extracted facial feature vectors with the facial feature vectors of the photographed objects in the facial feature library. In this way, when determining the fully-occluded photographed objects, it is also possible to determine the semi-occluded photographed objects.

[0164] (ii) The subject is not completely within the shooting range

[0165] During the shooting process, the electronic device usually displays a preview image, which includes at least one shooting object. In this embodiment, the electronic device can identify the preview image, and when there is a shooting object that is not completely within the shooting range in the preview image, the electronic device displays a prompt message on the preview interface. The details are as follows.

[0166] Fig.12 This is a schematic diagram of a shooting method provided by an embodiment of the present application when the shooting object is not completely within the shooting range, see Fig.12 As shown, the method includes the following steps S1201 to S1206.

[0167] S1201: The electronic device displays a preview image during shooting, where the preview image includes at least one shooting object.

[0168] S1202: The electronic device determines a preset edge area of ​​the preview image.

[0169] It should be understood that when a user uses an electronic device to take a group photo, the subjects in the shooting scene usually have the situation that the subjects in the front row block the body parts of the subjects in the back row, but do not block the heads, that is, the faces, of the subjects in the back row. In this case, since the key points of the human body of the subjects except the head can be considered to be within the shooting range. As for the shooting effect, as long as there is no problem of the subjects in the front row blocking the faces of the subjects in the back row. Therefore, in this embodiment, the electronic device only needs to determine whether there is a second subject that has not completely entered the shooting range in the edge preset areas above and below or on the left and right sides of the preview image. Therefore, the electronic device first needs to determine the edge preset areas of the preview image.

[0170] The edge preset area may be set according to actual conditions, such as the number of people in the photographed subject or the photographing scene, and this embodiment does not impose any specific limitation thereto.

[0171] S1203, the electronic device identifies the key points and information of the key points of the human body of the photographed object in the preset edge area.

[0172] The electronic device can identify the human body key points of each photographed object in the edge preset area based on the key point confidence and the partial affinity field detection algorithm, and determine the human body key point information of the photographed object based on the human body key points. The human body key point information includes the position information of the human body key points, the human body key point category, and the human body key point confidence, etc.

[0173] S1204: The electronic device determines the photographed subject whose key points of the human body are not fully displayed as the second photographed subject.

[0174] Incomplete recognition of key points of the human body in the preset edge area of ​​the preview image indicates that there is a shooting object that has not completely entered the shooting range (also known as a person crossing the boundary) in the preset edge area, and the electronic device determines the shooting object as the second shooting object.

[0175] S1205: The electronic device determines position information of a second photographed object according to the key point information of the human body.

[0176] Exemplarily, the electronic device may determine the position information of the human body key points in the human body key point information as the position information of the second photographed object.

[0177] S1206, the electronic device displays prompt information according to the location information.

[0178] The prompt information is used to indicate that the second photographed object has not completely entered the photographing range. The prompt information may be a prompt box, or a human frame may be highlighted.

[0179] For example, see Fig.13 As shown in , the electronic device first determines a preset area 1 and a preset area 2 located on both sides of the edge of the preview image according to a preset size, and then determines position information 1 of the second photographic subject 1 when it is detected that the second photographic subject 1 is located in the preset area 1 and its key human body points such as the head, wrist, shoulder and neck have not completely entered the shooting range, and displays prompt information according to the position information 1, for example, a prompt box is displayed at the position of the second photographic subject 1, and the prompt box selects the range where the photographic subject 1 is located; when the electronic device detects that the second photographic subject 2 is located in the edge preset area 2 and its key human body points such as the wrist, shoulder and elbow have not completely entered the shooting range, it determines position information 2 of the second photographic subject 2 and displays prompt information according to the position information 2.

[0180] Through the method provided in the embodiment of the present application, the electronic device detects key points of the human body of the subject in the shooting scene to determine whether the person has crossed the boundary, thereby assisting the photographer to better complete the shooting work.

[0181] In summary, in the shooting method provided in the embodiment of the present application, the electronic device can not only identify the occlusion of the shooting object, but also identify the situation where the shooting object has not completely entered the shooting range. Then, when a shooting object in the shooting preview image is blocked, the image of the shooting object when it is not blocked is displayed; or when a shooting object in the shooting preview image has not completely entered the shooting range, a prompt message is displayed, and the prompt message is used to indicate that the shooting object has not completely entered the shooting range. After obtaining the image or prompt message when the shooting object is not blocked, the photographer can adaptively adjust the position of the shooting object according to the image or prompt message, thereby improving the shooting effect of the photo and enhancing the user experience.

[0182] It should be noted that the shooting method provided in the embodiment of the present application is not only applicable to application scenarios in which users use electronic devices to take group photos, but can also be applied to other application scenarios. For example, when the electronic device is shooting a person, the person's face is blocked by an obstruction, or part of the person is cut off by the shooting picture.

[0183] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0184] The present application also provides a photographing device for use in electronic equipment, see Fig.14 As shown, the device includes a first display module 1401 and a second display module 1402. The functions of each module are specifically as follows.

[0185] The first display module 1401 is used to display a preview image during the shooting process, where the preview image includes at least one shooting object.

[0186] The second display module 1402 is used to display an image of the first photographic object when the first photographic object is blocked in the preview image; and / or, when the second photographic object does not completely enter the shooting range in the preview image, display a prompt message, wherein the prompt message is used to indicate that the second photographic object has not completely entered the shooting range.

[0187] An embodiment of the present application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the shooting method shown in the above embodiments is implemented.

[0188] The present application also provides a chip, see Fig.15 As shown, the chip includes a processor and a memory, in which a computer program is stored. When the computer program is executed by the processor, the shooting method in the above embodiments is implemented.

[0189] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the shooting method provided in the above embodiments is implemented.

[0190] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by an electronic device, the electronic device implements the shooting methods provided in the above embodiments.

[0191] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0192] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct memory bus random access memory (DR RAM).

[0193] In the embodiments provided in the present application, the division of each framework or module is only a logical function division. There may be other division methods in actual implementation. For example, multiple frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0194] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0195] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0196] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0197] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A shooting method, characterized in that: Applied to electronic equipment, the method comprises: Displaying a preview image during the shooting process, wherein the preview image includes at least one shooting object; When the first photographed object is blocked in the preview image, an image when the first photographed object is not blocked is displayed; and / or, When the second photographing object in the preview image does not completely enter the photographing range, prompt information is displayed, where the prompt information is used to indicate that the second photographing object does not completely enter the photographing range.

2. The method according to claim 1, characterized in that When the first photographed object is blocked in the preview image, displaying an image when the first photographed object is not blocked includes: When the first photographed object is blocked in the preview image, determining position information of the first photographed object; Displaying prompt information for prompting that the photographed object is blocked according to the position information; In response to the first control operation, an image when the first photographic object is not blocked is displayed.

3. The method according to claim 2, characterized in that The method further comprises: In response to the second control operation, the display of the image when the first photographing object is not blocked is exited.

4. The method according to claim 3, characterized in that When the first photographed object is blocked in the preview image, displaying an image when the first photographed object is not blocked includes: When a first photographic subject is blocked by a blocking object in the preview image, an image of the first photographic subject when it is not blocked and an image of the blocking object are displayed.

5. The method according to any one of claims 1 to 4, characterized in that: After displaying the preview image during the shooting process, when the first shooting object is blocked in the preview image, displaying an image when the first shooting object is not blocked; or when the second shooting object in the preview image does not completely enter the shooting range, before displaying the prompt information, the method further includes: Switch the shooting mode to group photo shooting mode.

6. The method according to claim 5, characterized in that The step of switching the shooting mode to the group photo shooting mode includes: In response to a user operation, switching the shooting mode to a group photo shooting mode; or, When it is recognized that the preview image includes at least two photographing subjects, the photographing mode is switched to a group photo photographing mode.

7. The method according to any one of claims 1 to 6, characterized in that: When the subject in the preview image is a person, the first subject is determined according to the following method: Performing key point detection on the photographed objects in the preview image to determine the key points and key point information corresponding to each of the photographed objects; The photographed object whose key points are not completely displayed in the preview image is determined as the first photographed object that is partially blocked in the preview image.

8. The method according to claim 7, characterized in that The method further comprises: Extracting feature vectors of photographed objects that are not partially obscured in the preview image; Performing similarity matching between a feature vector of a photographed object in a feature library and a feature vector of a photographed object that is not partially occluded in the preview image; The photographed objects whose similarity in the feature library is greater than or equal to a preset threshold are marked as successfully matched photographed objects, and the photographed objects whose similarity in the feature library is less than the preset threshold are determined as the first photographed objects that are fully occluded in the preview image, wherein the feature library includes identification information of all photographed objects that appear in the shooting process, as well as feature vectors, key point information and close-up facial images of each photographed object, and the identification information is used to uniquely indicate the corresponding photographed object.

9. The method according to any one of claims 1 to 6, characterized in that: When the subject in the preview image is a person, the first subject is determined according to the following method: Extracting a feature vector of a photographed object in the preview image; Performing similarity matching on a feature vector of a photographed object in the preview image and a feature vector of a photographed object in the feature library, wherein the feature library is generated according to a preview stream during the photographing process, and the feature library includes identification information of all photographed objects appearing during the photographing process, as well as a feature vector, key point information, and a close-up face image of each photographed object, and the identification information is used to uniquely indicate a corresponding photographed object; The photographed objects in the feature library whose similarity is greater than or equal to a preset threshold are marked as successfully matched photographed objects, and the photographed objects in the feature library whose similarity is less than the preset threshold are determined as the first photographed objects that are blocked in the preview image.

10. The method according to claim 8 or 9, characterized in that: The method further comprises: Determine the number of successfully matched photographed objects in the feature library; When the number of successfully matched photographed objects is less than the number of photographed objects of the extracted feature vectors in the preview image, determining the unsuccessfully matched photographed objects in the preview image; Generating identification information of the photographed object that has not been successfully matched in the preview image; The identification information of the unsuccessfully matched photographed object in the preview image, as well as the feature vector, key point information and close-up face image of the unsuccessfully matched photographed object are added to the feature library, and the unsuccessfully matched photographed object is marked as a successfully matched state in the feature library.

11. The method according to any one of claims 8 to 10, characterized in that: The feature vector includes a face feature vector or a body feature vector.

12. The method according to any one of claims 1 to 11, characterized in that: When the second photographing object in the preview image has not completely entered the photographing range, displaying prompt information includes: Determine a preset edge area of ​​the preview image; Detect key points and key point information of the photographed object in a preset edge area; Determine the photographed object whose key points are not fully displayed as the second photographed object; Determining position information of the second photographed object; The prompt information is displayed according to the location information.

13. The method according to any one of claims 7 to 12, characterized in that: The key points include at least one of the face key points and the body key points of the photographed object.

14. A photographing device, characterized in that: Applied to electronic equipment, the device comprises: A first display module, used for displaying a preview image during shooting, wherein the preview image includes at least one shooting object; A second display module is configured to display an image of the first photographed object when the first photographed object is blocked in the preview image; and / or, When the second photographing object in the preview image does not completely enter the photographing range, prompt information is displayed, where the prompt information is used to indicate that the second photographing object does not completely enter the photographing range.

15. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 13 when executing the computer program.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

17. A chip, characterized in that: The chip includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 13 is implemented.

Images