Photography method and electronic device

Abstract

The present application provides a photography method. The photography method is applied to an electronic device. In the method, the electronic device can first determine in a real-time preview picture a subject that a user wants to photograph; then, on the basis of the position of the subject, the electronic device determines a recommended composition comprising the subject, and helps a user to photograph through guidance to obtain a photo with the same composition as the recommended composition. Thus, with the assistance of electronic devices, users can take high-quality photos meeting their photography expectations, thereby improving the photography experience of the users.

Description

Shooting method and electronic equipment Technical Field

[0001] The embodiments of the present invention relate to the field of terminal technology, and in particular to a shooting method and an electronic device. Background Art

[0002] With the development of electronic technology, more and more people are accustomed to recording their lives by taking photos.

[0003] When taking photos with electronic devices, ordinary users, lacking photography knowledge, may end up with photos that lack aesthetic appeal and produce subpar results due to compositional errors. This problem is particularly acute in the age of social media, where users hope to capture aesthetically pleasing photos with simple operations to attract attention and connect with others. Therefore, how electronic devices can help users capture high-quality photos is a pressing issue that needs to be addressed. Summary of the Invention

[0004] The embodiments of the present invention provide a photographing method and an electronic device. By implementing the method, a user can take photos that meet his / her shooting expectations and are of high quality with the assistance of the electronic device.

[0005] In a first aspect, an embodiment of the present invention provides a shooting method, which is applied to an electronic device, and the method includes: displaying a first real-time preview screen captured by a first camera. In response to a user operation received on a first area of ​​the first real-time preview screen, determining an object located in the first area as a shooting subject. Determining a recommended composition based on the position of the shooting subject, the recommended composition includes the shooting subject. Displaying a guidance prompt, the guidance prompt is used to guide the user to adjust the shooting direction so that the center of the first real-time preview screen coincides with the center of the recommended composition or the distance is less than a preset value. When the center of the first real-time preview screen coincides with the center of the recommended composition or the distance is less than a preset value, displaying a second real-time preview screen captured by the second camera, the field of view of the second real-time preview screen being the same as the field of view of the recommended composition.

[0006] After the user determines the subject, the electronic device can then determine a recommended composition based on the subject, allowing the electronic device to quickly eliminate composition options that do not include the subject, thereby more quickly narrowing down the range of recommended compositions. This can speed up the electronic device's determination of the recommended composition.

[0007] In addition, the recommended composition determined based on the above strategy may be closer to the user's shooting expectations, preventing the problem that the recommended composition determined by the electronic device does not include the subject the user wants to shoot, thereby enhancing the user experience during the shooting process.

[0008] In combination with the first aspect, in some embodiments, the electronic device includes a first camera and a second camera, and the focal length of the first camera is smaller than the focal length of the second camera.

[0009] Since a smaller focal length increases the camera's field of view, this method maximizes the capture range of the first real-time preview image captured by the first camera, allowing the electronic device to choose from a wider range of recommended compositions. This allows the electronic device to determine a more aesthetically pleasing recommended composition that better meets the user's shooting expectations.

[0010] In combination with the first aspect, in some embodiments, determining the recommended composition specifically includes: determining the recommended composition based on the first real-time preview screen; or determining the recommended composition based on a third real-time preview screen obtained by a third camera, where the field of view of the third real-time preview screen is larger than the field of view of the first real-time preview screen.

[0011] In conjunction with the first aspect, in some embodiments, determining a recommended composition specifically includes: stitching together multiple first real-time preview images taken at different times to generate a stitched image, wherein the stitched image has a field of view that is larger than the field of view of the first real-time preview image. Determining the recommended composition based on the stitched image.

[0012] Both of the above implementations can expand the recommended composition selection range of the electronic device as much as possible without the user noticing, so that the recommended composition determined by the electronic device may be more aesthetically pleasing and more in line with the user's shooting expectations.

[0013] In combination with the first aspect, in some embodiments, the method further includes: displaying a composition prompt, where the composition prompt is used to prompt the user to recommend a composition rule that complies with the composition.

[0014] If a photo doesn't conform to composition principles, it will lack aesthetic appeal and produce poor results. Therefore, displaying composition prompts allows users to understand the composition principles the electronic device has determined and guided the user's photo to adhere to, thereby increasing their perception and trust in the electronic device's assistance with photo taking.

[0015] In conjunction with the first aspect, in some embodiments, the guidance prompt includes: a center positioning marker and a reference positioning marker, wherein the center positioning marker is located at the center of the first real-time preview image. The center positioning marker is always located at the center of the first real-time preview image, and the reference positioning marker is located at the center of the recommended composition. When the user adjusts the shooting orientation, the position of the reference positioning marker, located at the center of the reference composition area, will continuously change.

[0016] In addition, the guidance prompt may further include a thumbnail of the second real-time preview image; and / or a thumbnail of the recommended composition.

[0017] In conjunction with the first aspect, in some embodiments, the user operation includes: a click operation on the first area, a long press operation, or a smear operation, wherein the smear operation is a back-and-forth sliding operation on the screen surface of the electronic device.

[0018] In a second aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a sensor. The memory is used to store a computer program, and the processor is used to call the computer program so that the electronic device executes the method described in the first aspect.

[0019] In a third aspect, an embodiment of the present invention provides a computer program product comprising instructions. When the computer program product is run on an electronic device, the electronic device executes the method according to the first aspect.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, causes the electronic device to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1A is a schematic diagram of an electronic device determining a recommended composition according to an embodiment of the present invention;

[0022] FIG1B is a schematic diagram of the content that a user actually wants to capture during a capture process according to an embodiment of the present invention;

[0023] FIG2A is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present invention;

[0024] FIG2B is a software structure block diagram of an electronic device 100 provided in an embodiment of the present invention;

[0025] 3A to 3F are user interfaces involved in determining a photographing subject by the electronic device 100 according to an embodiment of the present invention;

[0026] 4A to 4E are user interfaces of the electronic device 100 for guiding a user to take photos according to an embodiment of the present invention;

[0027] FIG5 is a flow chart of a photographing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0029] In a shooting scenario, an electronic device can turn on its camera and display a live preview. This live preview is a real-time image captured by the electronic device's camera. Users can use the live preview as a reference to adjust shooting angles and parameters, allowing them to better understand the shooting process and ultimately achieve a satisfactory photo.

[0030] Currently, the electronic device can determine multiple recommended compositions based on the real-time preview image. After the user makes a selection from the multiple recommended compositions, the electronic device can guide the user to operate the electronic device so that the user can take a photo that is the same as or similar to the selected recommended composition.

[0031] However, when users are shooting in crowded environments with complex backgrounds, electronic devices may not be able to accurately identify the subject based on the real-time preview image. Furthermore, different users may have very personalized shooting intentions when using electronic devices to shoot, and different people may want to capture completely different subjects when facing the same scene. For these two reasons, the recommended composition determined by the electronic device based on the real-time preview image may not meet the user's actual shooting expectations.

[0032] For example, as shown in FIG1A , the live preview image captured by the electronic device's camera contains four subjects: (1) a person with hands on hips, (2) a person raising their hands, (3) a pet dog, and (4) a cheering person. Based on this live preview image, the electronic device determines three recommended compositions: Recommended Composition A, which includes (1) the person with hands on hips, (2) the person raising their hands, and (3) the pet dog; Recommended Composition B, which includes (1) the person with hands on hips and (2) the person raising their hands; and Recommended Composition C, which includes only (3) the pet dog. Because (4) the cheering person is located at the edge of the live preview image, the electronic device 100 may not have recognized this subject, and therefore, none of the recommended compositions it determines includes (4) the cheering person. Furthermore, because there are many combinations of subjects in the live preview image, the three recommended compositions determined by the electronic device do not cover all combinations. If a user actually wants to take a photo of only (2) the person raising their hands and (3) the pet dog, or (4) the cheering person, or (5) both (3) the pet dog and (4) the cheering person, none of the three recommended compositions determined by the electronic device will meet their needs. Therefore, users still cannot take satisfactory photos with the assistance of electronic devices. In addition, incorrect composition recommendations may confuse or frustrate users, thereby reducing the user experience during the shooting process.

[0033] FIG. 2A shows a schematic structural diagram of the electronic device 100 .

[0034] The electronic device 100 in the embodiment of the present invention can be implemented as a mobile phone, a tablet computer, etc. The embodiment of the present application does not limit the specific type of the electronic device 100.

[0035] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0036] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0037] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0038] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0039] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0040] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0041] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0042] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0043] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0044] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0045] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0046] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0047] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0048] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0049] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0050] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniLED, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device can include one or N display screens 194, where N is a positive integer greater than one.

[0051] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0052] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0053] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. The focal lengths of the N cameras included in the electronic device 100 may be different to support users using different cameras to take photos with different fields of view in different shooting scenarios. In some embodiments, when N cameras 193 are included, the electronic device 100 may also open multiple cameras 193 at the same time and process the images captured by each camera 193 separately.

[0054] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0055] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0056] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0057] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0058] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0059] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0060] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0061] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0062] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0063] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0064] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0065] FIG2B is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0066] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: from top to bottom, the application layer, the application framework layer, the Android runtime, the system library layer, the hardware abstraction layer (HAL), and the kernel layer.

[0067] The application layer can include a series of application packages.

[0068] As shown in FIG2B , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0069] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0070] As shown in FIG2B , the application framework layer may include a window manager, a content provider, a telephony manager, a resource manager, a notification manager, a view system, a camera manager, and the like.

[0071] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0072] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0073] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0074] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0075] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0076] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0077] The camera manager includes a series of camera-related interfaces, such as the Camera2 API, which allows developers to access and control camera-related data.

[0078] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0079] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0080] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0081] The system library layer can include multiple functional modules, such as a surface manager, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), and a media library.

[0082] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0083] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0084] A 2D graphics engine is a drawing engine for 2D drawings.

[0085] The HAL provides a standardized interface for abstracting hardware access, supporting communication between the kernel layer and other layers above the HAL, eliminating the need for upper-layer applications and services to communicate directly with the hardware. The HAL can contain multiple modules, such as a camera module, an audio module, a Bluetooth module, a sensor module, and a display module.

[0086] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0087] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0088] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a still image or video through the camera 193.

[0089] The following describes the process by which electronic device 100 acquires and displays a real-time preview image, based on the software architecture of electronic device 100 shown in FIG2B : The camera driver in the kernel layer can receive the raw preview image captured and transmitted by the camera. The raw preview image can be transmitted as a video stream. The camera driver can send the raw preview image to the camera module in the HAL. The camera module can perform format conversion on the raw preview image, enabling the converted image to be understood and processed by other layers above the HAL. The camera module in the HAL can then send the processed image to the system library. Relevant modules in the system library can further process the image. For example, the 3D graphics processing library can render complex graphics based on the received data, while the 2D graphics engine can perform graphics processing on the image. The system library can then send the real-time preview image, resulting from this further processing, to the camera manager in the application framework layer. The camera application in the application layer can obtain and display the real-time preview image through the interface provided by the camera manager.

[0090] In the embodiment of the present application, the process of determining the subject and determining the recommended composition including the subject can be performed by the camera application in the application layer. This process will be described in detail in the subsequent embodiments of the present application and will not be described in detail here.

[0091] The following is an exemplary description of the user interface involved when the electronic device 100 takes a photo.

[0092] 3A to 3F exemplarily illustrate user interfaces involved when the electronic device 100 determines a subject to be photographed.

[0093] Figure 3A illustrates an exemplary user interface 310, which may be a shooting interface provided by the camera application of electronic device 100. As shown in Figure 3A, user interface 310 may include a viewfinder 311, a function switch 312, a focus adjustment control 313, and a capture button 314. The viewfinder 311 displays a first real-time preview image captured by the first camera. The function switch 312 triggers the electronic device 100 to enable or disable the "Smart Photography Tutor" function. The focus adjustment control 313 triggers the camera to adjust its focus, allowing the user to capture photos with different fields of view. The capture button 314 triggers the electronic device 100 to capture a photo. When the "Smart Photography Tutor" function is enabled, the electronic device 100 may first determine the subject the user currently wants to capture, determine a recommended composition based on the subject, and then guide the user to adjust the shooting position and capture the photo, thereby obtaining a photo that closely matches the recommended composition. In other implementations, "Smart Photography Tutor" may also be referred to by other names, such as "Photography Assistant" or "Photography Assist." The specific name of this function is not limited in this embodiment of the present application.

[0094] When the electronic device 100 does not enable the "Smart Photography Tutor" function, it detects an input operation (such as a touch operation) by the user on the function switch 312. In response to the above input operation, the electronic device 100 can enable the "Smart Photography Tutor" function and display the user interface 320 as shown in Figure 3B. The user interface 320 includes a viewfinder 321 and prompt information 322. Among them, the viewfinder 321 is used to display the first real-time preview screen obtained by the first camera. The prompt information 322 is used to prompt the user to select a shooting subject from multiple objects displayed in the first real-time preview screen. The prompt information 322 can be implemented as text, pictures, sound content, etc. For example, the prompt information 322 can be implemented as the text "Click to select the subject you want to shoot". The embodiment of the present application does not limit the specific form of the prompt information 322.

[0095] The user can select the subject to be photographed by clicking. For example, as shown in FIG3C , a click operation of the user on the first area of ​​the first real-time preview image displayed in the viewfinder 321 is detected. In response to the click operation, the electronic device 100 can determine the pet dog near the area as the subject to be photographed.

[0096] After determining the subject, as shown in Figure 3D, the electronic device 100 can display a dotted frame that closely follows the outline of the subject. In some implementations, the user can fine-tune the range of the subject's outline. For example, upon receiving an inward or outward drag operation on the edge of the dotted frame, the electronic device 100 can adjust the range of the subject's outline to more accurately define the subject. In addition, the electronic device 100 can also display a Done button 323. When the user has selected all the subjects they want to photograph, they can trigger the electronic device 100 to end the subject selection process by clicking the Done button 323.

[0097] The user can select multiple shooting subjects. As shown in Figure 3E, a click operation of the user on the second area of ​​the first real-time preview screen displayed by the viewfinder 321 is detected. In response to the click operation, the electronic device 100 will perform subject recognition with the second area as the center. Since a little man with his hand raised is displayed near the second area, the electronic device 100 can also determine the little man with his hand raised as the shooting subject and highlight the shooting subject. Afterwards, an input operation (such as a touch operation) of the user on the completion button 323 is detected. In response to the input operation, the electronic device 100 can end the shooting subject selection process.

[0098] After determining the subject, the electronic device 100 can determine a recommended composition based on the first real-time preview image. Since the subjects previously determined by the electronic device 100 are the pet dog and the person raising their hands, the recommended composition determined by the electronic device 100 also includes these two subjects. The process of determining the recommended composition by the electronic device 100 will be described in detail in subsequent embodiments of this application and will not be described in detail here.

[0099] After determining the recommended composition, the electronic device 100 may guide the user to assist the user in adjusting the shooting orientation and taking a photo that is the same as or similar to the recommended composition.

[0100] 4A to 4E exemplarily illustrate user interfaces of the electronic device 100 for guiding a user to take a photo.

[0101] Figure 4A illustrates an exemplary user interface 410. As shown in Figure 4A , user interface 410 may include a framing frame 411, a center positioning mark 412A, a reference positioning mark 412B, prompt information 413, and a preview frame 414A. Framing frame 411 displays a first real-time preview image captured by the first camera. Center positioning mark 412A and reference positioning mark 412B guide the user in adjusting the shooting orientation. Center positioning mark 412A is located at the center of the framing frame. Reference positioning mark 412B is located at the center of the recommended composition area. Prompt information 412 prompts the user to adjust the shooting orientation. Preview frame 414A may display the image captured by the second camera, thereby better demonstrating the shooting effect to the user and assisting the user in adjusting the shooting orientation. In some implementations, as shown in Figure 4B , electronic device 100 may display preview frame 414B instead of preview frame 414A. Preview frame 414B may display the recommended composition determined by electronic device 100.

[0102] When the user adjusts the shooting direction, the center positioning mark 412A will always be located at the center of the first real-time preview screen. As for the reference positioning mark 412B, due to the change in the shooting direction, the first real-time preview screen obtained by the camera of the electronic device 100 will change accordingly, causing the position of the area where the recommended composition is located in the first real-time preview screen to change. Therefore, when the user adjusts the shooting direction, the position of the reference positioning mark 412B located at the center of the area where the reference composition is located will continue to change. When the center positioning mark 412A and the reference positioning mark 412B coincide or the distance between them is less than a preset value, the electronic device 100 can display the alignment mark 415 in the user interface 410 as shown in Figure 4C.

[0103] Afterwards, as shown in FIG4D , electronic device 100 may display user interface 420 through automatic device zoom or manual user focus adjustment. Preview box 421 within user interface 420 displays a second real-time preview image captured by the second camera. The field of view of the second real-time preview image is the same as that of the recommended composition, and the content in the second real-time preview image can be the same or similar to that in the recommended composition.

[0104] Afterwards, as shown in FIG4E , since the second real-time preview image conforms to the triangular composition principle, the composition prompt displayed by the electronic device 100 may be implemented as the text “triangular composition” and triangular graphic lines.

[0105] FIG5 is a flow chart of a shooting method provided by an embodiment of the present invention. The method is applied to an electronic device 100. As shown in FIG5 , the method includes:

[0106] Step S101: In response to a received first user operation, a first real-time preview image acquired by a first camera is displayed.

[0107] In some implementations, the first user operation may be a user operation that triggers the electronic device 100 to run a camera application. In other implementations, the first user operation may also be a user operation that triggers the electronic device 100 to call the camera to take a photo while the electronic device 100 is running other applications. The specific form of the first user operation is not limited in this embodiment of the application.

[0108] In response to the received first user operation, the electronic device 100 may display a first real-time preview image captured by the first camera. The first real-time preview image may serve as an important reference for the user to adjust the shooting angle and shooting parameters.

[0109] In some implementations, if the electronic device 100 includes multiple cameras, and the focal lengths of the cameras are different, the first camera may be the camera with the smallest focal length and the largest field of view in the electronic device 100 .

[0110] Step S102: Determine a shooting subject in response to the received second user operation.

[0111] In some implementations, the electronic device 100 may first receive a second user operation. The second user operation is an input operation by the user directed to a certain area in the first real-time preview screen. The input operation can be implemented as a click operation, a long press operation, a smear operation, etc., acting on a certain area in the first real-time preview screen. The smear operation is a back-and-forth sliding operation on the screen surface of the electronic device 100. The specific form of this input operation is not limited in this embodiment of the application.

[0112] In response to the received second user operation, the electronic device 100 may perform object recognition on the first real-time preview image and determine one or more objects in the first real-time preview image. The electronic device 100 may then determine the object located in the area of ​​effect of the second user operation as the subject of the photograph.

[0113] For example, upon receiving a user click operation on the first area of ​​the first live preview image displayed in the viewfinder 321 shown in FIG3C , the electronic device 100 may perform object recognition and identify four objects. Among the four objects, since the pet dog is located in the first area, the electronic device 100 may identify the pet dog as the subject of the photograph.

[0114] In other implementations, before executing step S102, the electronic device 100 may first perform object recognition on the first real-time preview image to determine one or more objects. Thereafter, in response to receiving the second user operation, the electronic device 100 may determine the object closest to the center of the area affected by the second user operation as the subject of the photograph.

[0115] In addition, in some implementations, the electronic device 100 can turn on the front camera and obtain a real-time preview image of the front camera. The electronic device 100 can track the viewpoint of the real-time preview image obtained by the front camera and determine the object located in the user's gaze area in the first real-time preview image as the shooting subject. Since the user usually pays attention to the status of the subject they want to shoot on the screen during the shooting process, determining the shooting subject in this way not only saves the user from additional actions during the shooting process and improves the user's shooting experience, but also the determined shooting subject may be more in line with the user's shooting expectations.

[0116] In some implementations, the electronic device 100 can highlight the subject. This can be done, for example, by displaying a dotted frame closely following the outline of the subject as shown in FIG3D or FIG3F , or by displaying the content in the area where the subject is located in a contrasting color.

[0117] Considering that the user may want to take photos of multiple different subjects at the same time, in some implementations, the electronic device 100 may perform step S102 multiple times to determine multiple different shooting subjects.

[0118] Step S103: Determine a recommended composition based on the real-time preview image, where the recommended composition includes the subject.

[0119] In some implementations, the electronic device 100 may input the first real-time preview image into a pre-trained aesthetic evaluation model. The aesthetic evaluation model may first determine multiple compositions based on the first real-time preview image, each of which may include a subject. The electronic device 100 may then assign aesthetic scores to each of the multiple compositions and determine the composition with the highest aesthetic score as the recommended composition. In other implementations, the electronic device 100 may display the multiple compositions with the highest aesthetic scores and then determine a composition selected by the user as the recommended composition.

[0120] In some implementations, if the electronic device 100 includes multiple cameras and the electronic device 100 can turn on multiple cameras at the same time. Then, the electronic device 100 can turn on a third camera while displaying the first real-time preview image obtained by the first camera, and the focal length of the third camera is smaller than the focal length of the first camera. The electronic device 100 can obtain a third real-time preview image through the third camera and determine a recommended composition based on the third real-time preview image. Due to the difference in focal length, the field of view of the third real-time preview image is larger than the field of view of the first real-time preview image. Therefore, compared with the first real-time preview image, determining the recommended composition based on the third real-time preview image can enable the electronic device 100 to have a larger range of recommended composition selections. In this way, the recommended composition determined by the electronic device 100 may be more aesthetically pleasing and more in line with the user's shooting expectations.

[0121] In other implementations, the electronic device 100 can stitch together multiple first real-time preview images captured at different times into a single stitched image. The stitched image has a field of view greater than that of a single first real-time preview image. The electronic device 100 can then determine a recommended composition based on the stitched image. Through this strategy, the electronic device 100 can expand the range of options when determining a recommended composition, making the recommended composition it determines more aesthetically pleasing and more in line with the user's shooting expectations.

[0122] In addition, the electronic device 100 may also determine the recommended composition in other ways. The embodiment of the present application does not specifically limit the way in which the electronic device 100 determines the recommended composition.

[0123] Since the recommended composition determined by the electronic device 100 needs to include the subject, the electronic device 100 can quickly eliminate composition options that do not include the subject during the process of determining the recommended composition, thereby more quickly narrowing down the selection range of the recommended composition. This can speed up the speed at which the electronic device 100 determines the recommended composition. In addition, the electronic device 100 first determines the subject and then determines the recommended composition based on the real-time preview image. This can make the recommended composition closer to the user's shooting expectations, prevent the problem of the recommended composition determined by the electronic device 100 not including the subject the user wants to shoot, and enhance the user experience during the shooting process.

[0124] Step S104: displaying a guidance prompt, which is used to guide the user to adjust the shooting orientation so that the center of the first real-time preview image coincides with the center of the recommended composition or the distance therebetween is less than a preset value.

[0125] The electronic device 100 can guide the user to adjust the shooting orientation by displaying a guidance prompt so that the center of the first real-time preview screen coincides with the center of the recommended composition or the distance therebetween is less than a preset value. The guidance prompt can be implemented in various forms. In some implementations, the guidance prompt may include a center positioning mark and a reference positioning mark. The center positioning mark is located at the center of the first real-time preview screen. The reference positioning mark is located at the center of the area where the recommended composition is located. During the process of adjusting the shooting orientation, the position of the center positioning mark remains fixed, while the position of the reference positioning mark changes continuously. For example, the center positioning mark can be implemented as the center positioning mark 412A in the user interface 410 shown in Figure 4A, and the reference positioning mark can be implemented as the reference positioning mark 412B in the user interface 410 shown in Figure 4A.

[0126] In some implementations, the guidance prompt displayed by the electronic device 100 may include a thumbnail of a second real-time preview screen captured by the second camera, wherein the field of view of the second real-time preview screen is the same as the field of view of the recommended composition. In other implementations, the guidance prompt displayed by the electronic device 100 may include a thumbnail of a recommended composition previously determined by the electronic device 100. By displaying the second real-time preview screen or a thumbnail of the recommended composition, the electronic device 100 can better assist the user in adjusting the shooting direction and show the user the shooting effect, thereby improving the user's experience in adjusting the shooting direction.

[0127] In other implementations, the guidance prompt displayed by the electronic device 100 may further include prompt text. For example, the prompt text may be implemented as prompt information 413 in the user interface 410 shown in FIG4A .

[0128] In addition, the guidance prompt can also be implemented in other forms, such as an indicating arrow, etc. The embodiment of the present application does not limit the specific form of the guidance prompt.

[0129] As the user adjusts the shooting orientation, the position of the recommended composition in the first real-time preview image changes, causing the position of the reference positioning mark located at the center of the recommended composition to also change. Therefore, the electronic device 100 needs to continuously update the position of the reference positioning mark so that the user can correctly adjust the shooting orientation based on the position of the reference positioning mark.

[0130] In some implementations, the electronic device 100 can obtain the moving direction and moving distance when the user adjusts the shooting direction based on the data measured by the inertial measurement unit (IMU). Afterwards, the electronic device 100 can determine the position of the reference positioning mark in the first real-time preview screen based on the above-mentioned moving direction and moving distance. Using the above-mentioned strategy, compared to searching for the position of the reference positioning mark in the entire first real-time preview screen, the speed at which the electronic device 100 updates the position of the reference positioning mark can be accelerated, thereby improving the user's experience of adjusting the shooting direction under the guidance of the electronic device 100.

[0131] In some implementations, when the center of the first live preview image coincides with the center of the recommended composition or the distance therebetween is less than a preset value, the electronic device 100 may display an alignment mark. For example, the alignment mark may be implemented as an alignment mark 415 in the user interface 410 as shown in FIG4B .

[0132] Step S105 : Displaying a second real-time preview image acquired by the second camera. The field of view of the second real-time preview image is the same as the field of view of the recommended composition.

[0133] When the center of the real-time preview screen coincides with the center of the recommended composition or the distance between them is less than a preset value, the electronic device 100 can display a second real-time preview screen captured by the second camera through automatic zoom of the device or manual focus adjustment by the user. The field of view of the second real-time preview screen is the same as the field of view of the recommended composition. The second camera can be the same as or different from the first camera. For example, the second real-time preview screen can be the screen displayed in the preview box 421 in the user interface 420 shown in Figure 4D. At this time, the content in the second real-time preview screen can be the same as or similar to the content in the recommended composition.

[0134] (Optional) Step S106: Display composition prompts.

[0135] The composition hint is used to indicate which composition rule the recommended composition complies with in the composition principle. For example, the composition hint can be implemented as the text "Triangle Composition" and a triangle graphic line in the user interface 420 shown in FIG4E .

[0136] In addition, the composition prompt can also be implemented in other forms, such as: displaying the composition prompt in a pop-up window; adding a composition prompt watermark to the photo and / or adding a composition information tag to the photo after the user adjusts the shooting orientation according to the guidance of the electronic device 100 and takes the photo. The specific form of the composition prompt is not limited in the embodiments of the present application.

[0137] As mentioned above, if the photographed photo does not conform to the composition principle, the photo will lack beauty and have poor effect. Therefore, by displaying the composition prompt, the user's perception and credibility of the electronic device 100 assisting in shooting can be improved.

Claims

1. A shooting method, characterized in that: The method is applied to an electronic device, and includes: Displaying a first real-time preview image acquired by the first camera; In response to a received user operation performed on a first area of ​​the first real-time preview image, determining an object located in the first area as a photographic subject; determining a recommended composition according to the position of the photographic subject, wherein the recommended composition includes the photographic subject; Displaying a guidance prompt, wherein the guidance prompt is used to guide the user to adjust the shooting orientation so that the center of the first real-time preview image coincides with the center of the recommended composition or the distance between them is less than a preset value; When the center of the first real-time preview image coincides with the center of the recommended composition or the distance therebetween is less than a preset value, a second real-time preview image acquired by the second camera is displayed, and the field of view of the second real-time preview image is the same as that of the recommended composition.

2. The method according to claim 1, characterized in that The electronic device includes the first camera and a second camera, and the focal length of the first camera is smaller than the focal length of the second camera.

3. The method according to claim 1, characterized in that The determining of the recommended composition specifically includes: determining the recommended composition according to the first real-time preview image; Alternatively, the recommended composition is determined based on a third real-time preview image acquired by a third camera, and the field of view of the third real-time preview image is larger than the field of view of the first real-time preview image.

4. The method according to claim 1 or 2, characterized in that The determining of the recommended composition specifically includes: splicing a plurality of the first real-time preview images at different times to obtain a spliced ​​image, wherein the field of view of the spliced ​​image is larger than the field of view of the first real-time preview image; A recommended composition is determined according to the spliced ​​image.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: A composition prompt is displayed, where the composition prompt is used to prompt the user to which composition rules the recommended composition complies with.

6. The method according to any one of claims 1 to 5, characterized in that The guidance prompts include: a center positioning mark and a reference positioning mark, wherein the center positioning mark is located at the center of the first real-time preview image and the reference positioning mark is located at the center of the recommended composition; and / or, a thumbnail of the second real-time preview image; and / or, a thumbnail of the recommended composition.

7. The method according to claims 1-6, characterized in that The user operations include: A click operation, a long press operation, or a smear operation is performed on the first area.

8. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a sensor. The memory is used to store a computer program, and the processor is used to call the computer program, so that the electronic device executes the method according to any one of claims 1 to 7.

9. A computer program product comprising instructions, characterized in that When the computer program product is run on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 7.

Images