A photographing method and a terminal device

Abstract

The application discloses a photographing method and a terminal device, and relates to the technical field of terminals. When the terminal device identifies a photographing scene requiring a background blurring effect, multiple images are collected by gradually reducing the aperture, and the multiple images are superimposed to generate a photo with the blurring effect. The first image is collected by using a full-aperture camera to obtain the blurring effect. When the remaining multiple images are collected, the aperture of the camera is gradually reduced, the burrs of the out-of-focus dispersion circle in the image collected by using the full-aperture camera are weakened, and the blurring effect is optimized. Moreover, only when the scene of generating the photo containing the blurring effect that can be perceived by a user is identified, the method of collecting multiple images by gradually reducing the aperture is used to generate the photo, so that the additional power consumption caused by frequent adjustment of the aperture and collection of multiple images in scenes not requiring optimization of the blurring effect is avoided, and the photographing performance of the terminal device is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method for taking pictures and a terminal device. Background Technology

[0002] As the image size of mobile imaging devices (such as mobile phones) gradually increases, the demand for background blur also becomes stronger. For example, mobile phone photography has large aperture modes and portrait modes, which can capture images with blurred backgrounds. However, due to the insufficient processing precision of aspherical lenses in mobile imaging devices, the background blur is not uniform, resulting in jagged edges in the blurred background and onion ring-like shapes in the bokeh effect, lacking a smooth transition. Summary of the Invention

[0003] This application provides a photography method and terminal device that can optimize the background blur effect, making the blurred background more uniform and reducing jagged edges.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] Firstly, a method for taking photos is provided, applied to a terminal device including a camera. The method includes: the terminal device displaying a photo preview interface; receiving a photo-taking operation from a user on the preview interface; the terminal device sequentially acquiring multiple frames of images; and superimposing the multiple frames of images to generate a photo. During the acquisition of multiple frames of images, the aperture of the camera gradually decreases, resulting in a background blur effect in the generated photo.

[0006] The process of gradually reducing the aperture can be regarded as a modulation of a given line shape applied to an image including the bokeh effect. By capturing multiple frames of images by gradually reducing the aperture and then stacking them, the jagged edges of the out-of-focus blur circle in the image can be reduced, thus optimizing the bokeh effect.

[0007] In conjunction with the first aspect, in one possible implementation, during the acquisition of multiple frames of images, the aperture of the camera gradually decreases, which includes: the aperture size of the camera acquiring the nth frame of the image is one-nth of the aperture size of the camera acquiring the 1st frame of the image.

[0008] In this method, the camera aperture is reduced more uniformly, which makes the light spot in the blurred area spread more evenly and the blurring effect softer.

[0009] In conjunction with the first aspect, in one possible implementation, the terminal device acquires the aperture size of the camera during the photo preview; the camera captures the first frame image using the aperture size during the photo preview. That is, the camera uses the aperture size during preview as the maximum aperture for taking the photo, in order to generate a bokeh effect. Optionally, the aperture size of the camera during photo preview is the camera's fully open aperture.

[0010] In conjunction with the first aspect, in one possible implementation, the product of the camera's aperture size and exposure time remains constant during the acquisition of multiple frames of images.

[0011] In this method, the aperture is gradually reduced and the exposure time is gradually increased. The amount of light entering the lens is consistent for each frame of the image, and the brightness of each frame of the image is consistent. When multiple frames of images are superimposed, no additional image brightness alignment processing is required.

[0012] In conjunction with the first aspect, in one possible implementation, during the acquisition of multiple frames of images, the camera acquires each frame of images with the same exposure time.

[0013] This not only optimizes the background blur effect but also allows for the capture and restoration of high dynamic range scenes.

[0014] In conjunction with the first aspect, in one possible implementation, before sequentially acquiring multiple frames of images, the terminal device also acquires the camera's focal length, full-aperture size, and acceptable blur circle diameter; and acquires the focusing distance; and determines the generated photo, including a background blur effect, based on the camera's focal length, full-aperture size, acceptable blur circle diameter, and focusing distance.

[0015] In this method, a photo is generated only when the scene containing a blur effect that can be perceived by the user is identified. The method of gradually reducing the aperture and acquiring multiple frames is used to generate the photo, avoiding the extra power consumption caused by frequently adjusting the aperture and acquiring multiple images in scenes where the blur effect does not need to be optimized, thereby improving the performance of the terminal device in taking pictures.

[0016] In one possible implementation, determining that the generated photo includes a background blur effect based on the camera's focal length, maximum aperture size, acceptable blur circle diameter, and focusing distance includes: calculating a first depth of field based on the camera's focal length, maximum aperture size, acceptable blur circle diameter, and focusing distance; and determining that the generated photo includes a background blur effect if the first depth of field is less than or equal to a preset value.

[0017] Among these factors, aperture value, focus distance, and acceptable blur circle diameter are positively correlated with depth of field; focal length is negatively correlated with depth of field. A first depth of field is calculated based on aperture value, focus distance, focal length, and acceptable blur circle diameter. When the first depth of field is less than or equal to a preset value, it indicates that the bokeh effect is perceptible to the user. In other words, the optimization step is only performed when the generated photo includes a user-perceptible bokeh effect. This preset value can be set according to actual conditions, for example, based on the analysis of a large number of photos, to make the judgment more accurate.

[0018] In one possible implementation, calculating the first depth of field based on the camera's focal length, maximum aperture size, acceptable blur circle diameter, and focusing distance includes:

[0019]

[0020] Where DOF is the first depth of field, M is the camera's maximum aperture value, C is the camera's acceptable blur circle diameter, D is the focusing distance, and EFL is the camera's focal length.

[0021] During the photo preview stage, the terminal device configures a corresponding camera based on the shooting scene. The parameters of this camera are pre-programmed information. By obtaining the camera configured during the photo preview stage, the terminal device can acquire the camera's parameters, including its focal length, aperture size (optionally, the widest aperture), and acceptable circle of blur diameter. Optionally, the terminal device can use any one of contrast autofocus, phase autofocus, or laser autofocus for focusing, and obtain the focusing distance after focusing is completed.

[0022] In conjunction with the first aspect, in one possible implementation, if it is determined, based on the camera's focal length, full-aperture size, acceptable blur circle diameter, and focus distance, that the generated photograph does not include a background blur effect, the camera acquires the image and generates the photograph at full aperture.

[0023] In this method, in scenarios where the photo does not include a blur effect, the terminal device can use conventional methods to capture one or more frames of images and generate a photo.

[0024] Secondly, a terminal device is provided, which has the function of implementing the method described in the first aspect. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described function.

[0025] Thirdly, a terminal device is provided, comprising: a processor and a memory; the memory is used to store computer execution instructions, and when the terminal device is running, the processor executes the computer execution instructions stored in the memory to cause the terminal device to perform the method as described in any one of the first aspects above.

[0026] Fourthly, a terminal device is provided, comprising: a processor; the processor being coupled to a memory, and after reading instructions from the memory, executing, according to the instructions, the method as described in any one of the first aspects above.

[0027] Fifthly, a computer-readable storage medium is provided that stores instructions which, when executed on a computer, cause the computer to perform the method described in any one of the first aspects.

[0028] In a sixth aspect, a computer program product containing instructions is provided, which, when run on a computer, enables the computer to perform the method described in any one of the first aspects above.

[0029] In a seventh aspect, an apparatus (e.g., a chip system) is provided, comprising a processor for supporting a terminal device in implementing the functions described in the first aspect above. In one possible design, the apparatus further comprises a memory for storing program instructions and data necessary for the terminal device. When the apparatus is a chip system, it may be composed of chips or may include chips and other discrete devices.

[0030] The technical effects of any of the design methods in aspects two through seven can be found in the technical effects of different design methods in aspect one, and will not be repeated here. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the hardware structure of a terminal device provided in an embodiment of this application;

[0032] Figure 2 A schematic diagram illustrating the scenario to which the photographing method provided in this application is applicable;

[0033] Figure 3 A flowchart illustrating a photographing method provided in an embodiment of this application;

[0034] Figure 4 This is a schematic diagram illustrating pre-programmed information in a photographing method provided in an embodiment of this application.

[0035] Figure 5 A schematic diagram of a method for obtaining the focus distance in a photographing method provided in an embodiment of this application;

[0036] Figure 6 A schematic diagram illustrating the correspondence between depth of field and focusing distance in the photographing method provided in this application embodiment;

[0037] Figure 7 A scenario example illustrating the photographing method provided in this application embodiment. Figure 1 ;

[0038] Figure 8 A scenario example illustrating the photographing method provided in this application embodiment. Figure 2 ;

[0039] Figure 9 This application provides a schematic diagram of the software architecture of a terminal device.

[0040] Figure 10This is a schematic diagram of a module interaction for a photographing method provided in an embodiment of this application;

[0041] Figure 11 This is a schematic diagram of the structural composition of a terminal device provided in an embodiment of this application. Detailed Implementation

[0042] In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0043] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0044] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0045] The photo-taking method provided in this application can be applied to terminal devices (also known as mobile imaging devices) that include a camera module. These terminal devices may include mobile phones, tablets, laptops, personal computers (PCs), ultra-mobile personal computers (UMPCs), handheld computers, netbooks, personal digital assistants (PDAs), wearable devices (e.g., smartwatches, smart bracelets), in-vehicle devices, virtual reality devices, etc., and this application does not impose any limitations on these.

[0046] Figure 1 This is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of this application. Figure 1 As shown, the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a camera 190, and a display screen 191, etc. The sensor module 180 may include a temperature sensor, an ambient light sensor, a touch sensor, a depth sensor, etc.

[0047] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0048] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0049] The controller can serve as the central nervous system and command center of the terminal device 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0050] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0051] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0052] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0053] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of terminal device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0054] The charging management module 140 receives charging input from a charger, which can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also power the terminal device via the power management module 141. The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and powers the processor 110, internal memory 121, external memory, display screen 191, camera 190, and wireless communication module 160, etc.

[0055] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0056] Terminal device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0057] Terminal device 100 implements display functions through a GPU, display screen 191, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 191 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0058] Display screen 191 is used to display images, videos, etc. Display screen 191 is also referred to as screen 191. Display screen 191 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal device 100 may include one or N displays 191, where N is a positive integer greater than 1.

[0059] Terminal device 100 can perform shooting functions through ISP, camera 190, video codec, GPU, display 191 and application processor.

[0060] A camera 190, also known as a camera module, is used to capture still images or videos. An object passes through a lens, generating an optical image that is projected onto a photosensitive element. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Image Signal Processor) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for further processing. The DSP converts the digital image signal into standard image signals in formats such as RGB and YUV.

[0061] The ISP (Image Signal Processor) is used to process data fed back from the camera 190. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 190.

[0062] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0063] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0064] In some embodiments, the terminal device 100 may include one or N cameras 190, where N is a positive integer greater than 1. For example, the terminal device 100 may include a main camera, a wide-angle camera, a telephoto camera, etc.

[0065] In this embodiment, the camera module includes a variable aperture component, which can optionally be a physical aperture or a virtual aperture. The variable aperture component can adjust the aperture size of the camera module; the aperture is usually represented by the letter F, for example, F2.8 represents an aperture value of 2.8. The aperture value and aperture size are inversely proportional; the smaller the aperture value, the larger the aperture. The aperture size affects the amount of light entering the camera module per unit time; the larger the aperture, the more light reaches the photosensitive element from the lens per unit time, i.e., the greater the amount of light entering per unit time, and the brighter the image. For example, common aperture values ​​are as follows: F1, F1.4, F2, F2.8, F4, F5.6, F8, F11, F16, F22, F32, F44, F64, with the amount of light entering differing by a factor of two between every two adjacent aperture values. For example, adjusting the aperture from F4 to F2.8 doubles the amount of light entering the camera; adjusting the aperture from F2.8 to F2 also doubles the amount of light entering the camera. Optionally, the variable aperture assembly can steplessly adjust the aperture size, meaning the aperture size does not have to be one of the common aperture values ​​listed above.

[0066] When the terminal device 100 takes a picture, the lens of the camera 190 is adjusted so that the focus is on the subject. The range in which the foreground and background remain sharp after the lens is focused is the depth of field (DOF). The depth of field determines which parts of the photograph are sharp and which parts are blurred; a shallow depth of field results in a small area of ​​sharpness and a noticeable background blur effect; a large depth of field results in a large area of ​​sharpness and a less noticeable background blur effect.

[0067] Factors affecting depth of field include the lens aperture, focal length, and focusing distance during shooting.

[0068] When the aperture value (F-number) decreases, the angle at which light passes through the lens widens, and light outside the focal plane cannot be accurately focused, resulting in a blurred background. A larger aperture (smaller aperture value) results in a shallower depth of field; a smaller aperture (larger aperture value) results in a greater depth of field. When the camera has a variable aperture, it supports multiple aperture sizes; typically, the camera uses its maximum aperture to create an image with a bokeh effect; the maximum aperture is the widest aperture.

[0069] Lenses with longer focal lengths have a shallower depth of field, while lenses with shorter focal lengths have a greater depth of field. For example, telephoto lenses have a shallower depth of field, resulting in a stronger blurring effect between the background and foreground, while wide-angle lenses have a deeper depth of field and a weaker blurring effect.

[0070] The closer the subject is to the camera, the shallower the depth of field; the farther the subject is from the camera, the deeper the depth of field.

[0071] Each time the terminal device 100 takes a picture, it can configure the corresponding camera according to the shooting scene; for example, when shooting in "portrait mode", a telephoto lens is configured; when shooting in "large aperture mode", a large aperture lens is configured. For example, such as... Figure 2 As shown, the desktop interface 101 of the mobile phone 100 includes a "camera" icon; after receiving a user's click operation on the "camera" icon, the mobile phone 100 displays the user interface of the camera application. For example, if the user selects "portrait," the mobile phone 100 displays a preview interface 102 for portrait mode. In one implementation, in portrait mode, the mobile phone 100 captures images through a telephoto lens. Exemplarily, the preview interface 102 includes a "shutter" control 103, and after receiving a user's click operation on the "shutter" control 103, the mobile phone 100 generates a photo 104.

[0072] Terminal device 100 can be configured with a corresponding camera according to the shooting scene to achieve the shooting of photos with a bokeh effect. For example Figure 2 The image shows a portrait mode. Outside the blurred area (depth of field), the blurry points formed when light cannot be accurately focused are called blur circles, also known as circles of confusion. The larger the circle of confusion, the more pronounced the background blur effect. The size and shape of the circle of confusion depend on the shape of the aperture and the way light travels.

[0073] Currently, due to insufficient lens processing precision and aberration balance, the out-of-focus blur circle will have a burr-like cross-section, resulting in poor bokeh.

[0074] For example, Figure 3 A flowchart illustrating a photographing method provided in an embodiment of this application is shown.

[0075] S301. After the camera application is launched, the terminal device displays the photo preview interface.

[0076] After the camera application is launched, it enters the photo preview stage. During this stage, the device configures the appropriate camera based on the shooting scene; for example, a telephoto lens is configured for "Portrait Mode," and a wide-angle lens for "Still Life Mode." Optionally, if the device only has one camera, no camera configuration is required. Alternatively, if the camera includes a variable aperture, the device can also configure the aperture size used for that scene; for example, if the telephoto lens has a variable aperture, the maximum aperture (full opening) of the telephoto lens can be used in "Portrait Mode."

[0077] S302, Received the user's request to take a photo.

[0078] For example, refer to Figure 2 The user can take a photo by clicking the "shutter" control 103.

[0079] S303. The terminal device determines whether the generated photo includes a bokeh effect. If it determines that the generated photo includes a bokeh effect, proceed to S304; if it determines that the generated photo does not include a bokeh effect, proceed to S305.

[0080] In some embodiments, the terminal device determines whether the generated photo includes a bokeh effect based on the focal length, aperture size, focusing distance, and acceptable blur circle diameter of the camera corresponding to the current scene.

[0081] A larger aperture (smaller aperture value) allows light to pass through the lens at a wider angle, preventing light outside the focal plane from being accurately focused and resulting in a blurred background; in other words, a larger aperture (smaller aperture value) corresponds to a shallower depth of field, and a smaller aperture (smaller aperture value) corresponds to a deeper depth of field.

[0082] The longer the focal length, the shallower the depth of field; the shorter the focal length, the deeper the depth of field.

[0083] The closer the focusing distance, the shallower the depth of field; the farther the focusing distance, the deeper the depth of field.

[0084] A smaller acceptable blur circle diameter results in a shallower depth of field; a larger acceptable blur circle diameter results in a deeper depth of field. In one example, the acceptable blur circle diameter is twice the size of a sensor sampling pixel in the camera.

[0085] That is, aperture value, focusing distance, and acceptable blur circle diameter are positively correlated with depth of field; focal length is negatively correlated with depth of field.

[0086] In one implementation, the terminal device calculates the depth of field based on the focal length, aperture, focusing distance, and acceptable blur circle diameter of the camera corresponding to the current scene, and determines whether the generated photo includes a bokeh effect based on the depth of field.

[0087] In one example, the depth of field can be calculated according to the following formula (1).

[0088]

[0089] Where DOF represents depth of field, M represents aperture value, C represents acceptable blur circle diameter, D represents focusing distance, and EFL represents focal length.

[0090] If the depth of field is less than or equal to the preset value (Z0), the generated photo is determined to include a bokeh effect; if the depth of field is greater than the preset value (Z0), the generated photo is determined to not include a bokeh effect.

[0091] During the photo preview stage, the terminal device configures the corresponding camera according to the shooting scene. The parameters of this camera are pre-programmed information, for example, such as... Figure 4 As shown, the camera parameters include the camera focal length, the aperture value corresponding to the scene (e.g., the maximum aperture of the camera in "Portrait Mode"), the acceptable blur circle diameter, etc.

[0092] Once the terminal device obtains the camera configured during the photo preview stage, it can acquire the camera's parameters, including the camera's focal length, aperture size (optional, full aperture size), and acceptable blur circle diameter.

[0093] In one implementation, the terminal device obtains the focus distance when it receives a user's action to take a photo. Optionally, the terminal device can use any one of contrast autofocus, phase autofocus, or laser autofocus for focusing, and obtain the focus distance after focusing is completed. For example, as shown... Figure 5 As shown, when the terminal device receives a user's photo-taking operation, if contrast focusing is used, the distance between the subject and the camera is determined by the focus detection to obtain the focusing distance; if phase focusing is used, the distance between the subject and the camera is calculated based on the phase difference to obtain the focusing distance; if laser focusing is used, the distance between the subject and the camera is directly measured by laser to obtain the focusing distance.

[0094] Understandably, since the camera parameters are fixed, the depth of field varies depending on the focusing distance.

[0095] For example, Figure 6 A schematic diagram illustrating the relationship between depth of field and focusing distance is shown. For example... Figure 6 As shown, depth of field is positively correlated with focusing distance; the greater the focusing distance, the greater the depth of field. Given a preset value Z0, when the depth of field is less than or equal to Z0, the captured image includes a bokeh effect; when the depth of field is greater than Z0, the captured image does not include a bokeh effect.

[0096] In one implementation, a large number of photos taken by different cameras on the terminal device can be analyzed to determine the depth-of-field threshold between photos that include and do not have a user-perceptible bokeh effect, and this threshold is set as Z0. This ensures that when the depth of field is less than or equal to Z0, the captured photos include a user-perceptible bokeh effect, and when the depth of field is greater than Z0, the captured photos do not. In other words, bokeh effect optimization is only performed if the captured photos include a user-perceptible bokeh effect, with user experience serving as the trigger for optimization processing.

[0097] S304. The terminal device continuously acquires n frames of images by gradually reducing the aperture, and then superimposes the n frames to generate a single photograph, which includes a bokeh effect.

[0098] The aperture of the terminal device gradually decreases from the first frame to the nth frame. For example, given an initial aperture size M, the aperture size for capturing the first frame is M*k, the aperture size for capturing the second frame is M*k / 2, ..., and the aperture size for capturing the nth frame is M*k / n. For example, M*k is the full aperture, k is 4, and n is 8; that is, the first frame is captured with the full aperture, the second frame is captured with half the full aperture, the third frame is captured with 1 / 3 of the full aperture, ..., and the eighth frame is captured with 1 / 8 of the full aperture.

[0099] The process of gradually reducing the aperture can be regarded as a modulation of a given linearity applied to the widest aperture. By capturing multiple frames of images by gradually reducing the aperture and then stacking them, the blurring effect of the out-of-focus circle in the image captured at the widest aperture can be reduced, thus optimizing the bokeh effect.

[0100] For example, such as Figure 7 As shown, when the aperture is wide open, the blurred area exhibits an onion-ring shape, the point spread function shows significant fluctuations, and bokeh is noticeable. The composite image obtained by gradually decreasing the aperture produces a Gaussian-like composite line. The process of continuously acquiring multiple frames by gradually decreasing the aperture and then superimposing these frames essentially modulates the composite line of the image acquired at wide open aperture. In the superimposed image, the blurred area shows uniform diffusion, without the onion-ring shape; the point spread function exhibits uniform fluctuations, and bokeh is reduced.

[0101] In some embodiments, the terminal device acquires shooting parameters during the photo preview; these shooting parameters may include aperture size, exposure time, etc. The terminal device continuously acquires n frames of images based on the shooting parameters during preview, and then superimposes these n frames to generate a single photograph. For example, refer to... Figure 2The user can take a photo by clicking the "shutter" control 103. Upon receiving the user's click on the "shutter" control 103, the mobile phone 100 obtains the shooting parameters from the preview. The mobile phone 100 then continuously captures n frames of images based on the preview parameters, generating a photo 104.

[0102] The shooting parameters when the terminal device takes a photo can be determined based on the shooting parameters during preview.

[0103] (I) Aperture Size

[0104] The aperture size during preview is used as the aperture size for capturing the first frame of the image. The aperture size gradually decreases as the terminal device captures the first to the nth frame. For example, the aperture size for capturing the nth frame is: aperture size during preview / n. Optionally, if the aperture size during preview is the camera's fully open aperture, then the aperture size for capturing the nth frame is: fully open aperture / n.

[0105] Capturing the first frame with the aperture wide open yields an image with a bokeh effect. Capturing multiple frames by gradually reducing the aperture and then superimposing them on the first frame reduces the blurring effect of the out-of-focus circles in the image captured with the aperture wide open, thus optimizing the bokeh effect.

[0106] (II) Exposure Time

[0107] In one implementation, during the process of the terminal device acquiring images from the first frame to the nth frame, the aperture gradually decreases and the exposure time gradually increases, so that the amount of light entering the camera is the same for each frame from the first frame to the nth frame.

[0108] For example, if the aperture size multiplied by the exposure time of the first frame is used as a baseline, then the exposure time for the nth frame is: baseline value / aperture size of the nth frame, where the aperture size of the nth frame is the widest aperture / n. In this example, if the aperture size of the nth frame is 1 / n of the aperture size of the first frame, then the exposure time for the nth frame is n times the exposure time of the first frame.

[0109] In this way, the aperture gradually decreases and the exposure time gradually increases, so the amount of light entering the lens is consistent for each frame, and the brightness of each frame is consistent. When multiple frames are superimposed, no additional image brightness alignment processing is required.

[0110] For example, Figure 8 A schematic diagram illustrating a scenario example of the photographing method provided in this application is shown. For example... Figure 8As shown, after the terminal device receives a user's request to take a picture, it captures multiple frames of images. During this process, the aperture gradually decreases while the exposure time gradually increases. For example, when capturing the first frame, the aperture size is the widest possible aperture (widest aperture * 8 / 8), and the exposure time is EIT / 8, where EIT is the base exposure integration time. When capturing the second frame, the aperture size is the widest possible aperture * 6 / 8, and the exposure time is EIT / 6; when capturing the third frame, the aperture size is the widest possible aperture * 4 / 8, and the exposure time is EIT / 4; when capturing the fourth frame, the aperture size is the widest possible aperture * 2 / 8, and the exposure time is EIT / 2; and when capturing the fifth frame, the aperture size is the widest possible aperture * 1 / 8, and the exposure time is EIT / 1. The value of aperture size * exposure time remains a constant baseline. This ensures that the brightness of each frame in the captured images is consistent.

[0111] In another implementation, during the process of the terminal device acquiring images from the first frame to the nth frame, the aperture gradually decreases while the exposure time of each frame remains unchanged.

[0112] For example, if the aperture size multiplied by the exposure time of the first frame is used as a baseline value, then the aperture size multiplied by the exposure time of the nth frame is: baseline value / n, where the aperture size of the nth frame is the widest aperture / n. Thus, the light ratio (light intake ratio) between the nth frame and the first frame is 1:n, and the amount of light entering the lens when capturing the nth frame is 1 / n of the amount of light entering the lens when capturing the first frame.

[0113] This not only optimizes the background blur effect but also allows for the capture and restoration of high dynamic range scenes.

[0114] After acquiring n frames of images, the terminal device overlays these n frames to generate a single photograph. In one implementation, the terminal device performs automatic frame alignment on the n frames, ensuring that the main subjects in each frame are aligned before overlaying, resulting in a clearer main subject in the photograph.

[0115] S305. The terminal device captures an image and generates a photo, which does not include a blurring effect.

[0116] The terminal device can capture one or more frames of images using conventional methods to generate a photograph. In one implementation, the terminal device uses a fixed aperture size when capturing images, for example, capturing images using the maximum aperture.

[0117] The photography method provided in this application, when identifying a scene requiring background blur, progressively reduces the aperture to capture multiple frames of images and then superimposes these frames to generate a photo with a blurred effect. Specifically, the first frame is captured using the camera's full aperture to achieve the blur effect; as the remaining frames are captured, the aperture is progressively reduced, weakening the blurring effect caused by the out-of-focus circles in the fully open aperture image. Furthermore, this method of progressively reducing the aperture to capture multiple frames is only used when the scene in which the generated photo contains a blur effect perceptible to the user. This avoids the extra power consumption caused by frequently adjusting the aperture and capturing multiple images in scenes where blur optimization is not required, thus improving the photography performance of the terminal device.

[0118] In this embodiment, the terminal device described above is a terminal device capable of running an operating system and installing applications. Optionally, the operating system running on the terminal device may be... system, system, Systems, etc.

[0119] In some embodiments, the software system of the terminal device may adopt a layered architecture, an event-driven architecture, a microkernel architecture, or a cloud architecture. This application embodiment uses a layered architecture. Taking the system as an example, the software structure of the terminal device 100 is illustrated.

[0120] Figure 9 The software architecture diagram of the terminal device provided in the embodiments of this application is shown.

[0121] Understandably, a layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, The system can include an application (App) layer, an application framework (FWK) layer, a hardware abstraction layer (HAL), and a kernel layer. For example... Figure 9 As shown, The system may also include the Android runtime and system libraries.

[0122] The application layer can include a series of application packages. For example... Figure 9 As shown, the application package may include applications such as a camera app, gallery, calendar, call, music, and text messaging. The camera app is used for taking photos and videos.

[0123] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes predefined functions. It provides programming services to the application layer through the API interface. Figure 9 As shown, the application framework layer includes a camera service, a media framework, and a media codec service. The camera service manages and controls the camera on the terminal device, for example, turning the camera on or off; controlling the camera's on / off state based on requests from the camera application; controlling camera parameters such as exposure time and focal length; and acquiring camera preview data to provide real-time preview functionality. The media framework manages multimedia resources such as photos, images, videos, and audio. The media codec service manages the encoding and decoding of multimedia resources such as photos, images, videos, and audio.

[0124] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0125] The core library consists of two parts: one part contains the functionalities that Java needs to call, and the other part contains the core Android library.

[0126] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0127] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0128] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0129] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0130] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0131] A 2D graphics engine is a graphics engine for 2D drawing.

[0132] The Hardware Access Layer (HAL) is used to abstract hardware, encapsulate kernel-level drivers, and provide interfaces to higher layers. For example, ... Figure 9 As shown, the hardware abstraction layer includes camera HAL, display HAL, audio HAL, etc.

[0133] The kernel layer provides low-level drivers for various hardware components of terminal devices. For example, such as... Figure 9 As shown, the kernel layer can include camera drivers, display drivers, audio drivers, etc.

[0134] For example, in combination Figure 9 The module in Figure 10 This illustration shows a module interaction diagram of the photographing method provided in an embodiment of this application.

[0135] like Figure 10 As shown, after the camera application receives the user's action to take a photo, it sends a 3A request to the camera service. The 3A includes auto exposure (AE), auto focus (AF), and auto white balance (AWB).

[0136] The camera service sends a 3A request to the camera HAL, along with aperture and exposure time information. In one implementation, the aperture information includes the maximum aperture value. The camera HAL obtains the maximum aperture value based on the aperture information and, based on a preset value n (the number of frames captured), obtains the aperture value corresponding to each frame. In another implementation, the aperture information includes a sequence of aperture values ​​corresponding to n frames. In one example, the exposure time information includes a sequence of exposure times corresponding to each of the n frames.

[0137] After receiving the 3A request, the camera HAL sends the 3A command to the camera driver.

[0138] The camera driver reads aperture and exposure time information, executes the 3A command, and calls the camera to capture images from the first frame to the nth frame. After capturing each frame, the camera sends one frame to the camera HAL.

[0139] The camera's HAL performs automatic frame alignment on n frames, overlays these n frames, and generates a single photograph. The camera HAL then sends this photograph to the camera application.

[0140] During the process of the camera capturing the first image to the nth frame, the aperture is gradually reduced. After superimposing these n frames, the blurring effect of the out-of-focus circle in the image captured with the aperture fully open can be reduced, thus optimizing the bokeh effect.

[0141] It is understood that the terminal device provided in this application embodiment includes hardware structures and / or software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0142] This application embodiment can divide the terminal device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0143] In one example, please refer to Figure 11 The diagram illustrates a possible structural schematic of the terminal device involved in the above embodiments. The terminal device 1100 includes: a processing unit 1110, a storage unit 1120, an image acquisition unit 1130, and a display unit 1140.

[0144] The processing unit 1110 is used to control and manage the actions of the terminal device 1100. The storage unit 1120 is used to store the program code and data of the terminal device 1100. The image acquisition unit 1130 is used to acquire images. The display unit 1140 is used to display the interface of the terminal device 1100; for example, displaying a photo preview interface, displaying photos, etc.

[0145] Of course, the unit modules in the aforementioned terminal device 1100 include, but are not limited to, the aforementioned processing unit 1110, storage unit 1120, image acquisition unit 1130, and display unit 1140.

[0146] Optionally, the terminal device 1100 may also include an audio unit, a communication unit, etc. The audio unit is used for acquiring and playing audio. The communication unit is used to support communication between the terminal device 1100 and other devices.

[0147] The processing unit 1110 can be a processor or controller, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The storage unit 1120 can be a memory. The image acquisition unit 1130 can be a camera, etc. The display unit 1140 can be a display screen, etc. The audio unit can include a microphone, a speaker, etc. The communication unit can include a mobile communication unit and / or a wireless communication unit.

[0148] For example, processing unit 1110 is a processor (such as...) Figure 1 The processor 110 shown can be a memory (such as a storage unit 1120). Figure 1 The internal memory 121 shown, and the image acquisition unit 1130 can be a camera (such as...) Figure 1 The camera 190 shown can be a display screen (e.g., a screen with a display unit 1140). Figure 1 The display screen 191 shown is an audio unit that can be an audio module (such as...). Figure 1 The audio module 170 shown. The communication unit may include a mobile communication unit (such as...). Figure 1 The mobile communication module 150 and wireless communication unit shown are shown. Figure 1 The wireless communication module 160 shown in this application embodiment can be the terminal device 1100. Figure 1 The terminal device 100 shown. The aforementioned processor, memory, display screen, camera, audio module, mobile communication unit, and wireless communication unit can be connected together, for example, via a bus.

[0149] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of a terminal device). As another example, the interface circuit can be used to send signals to other devices (e.g., the processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the terminal device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0150] This application also provides a computer-readable storage medium including computer instructions that, when executed on the terminal device, cause the terminal device to perform various functions or steps performed by the terminal device in the above method embodiments.

[0151] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps performed by the mobile phone in the above method embodiments.

[0152] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0156] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for taking photos, characterized in that, Applied to a terminal device, the terminal device including a camera, the method includes: Displays the camera preview interface; Receives the user's photo-taking action on the photo preview interface; Multiple frames of images are acquired sequentially; wherein, during the acquisition of the multiple frames of images, the aperture of the camera is gradually reduced. The multiple frames of images are superimposed to generate a photograph; the photograph includes a background blur effect.

2. The method according to claim 1, characterized in that, The process of gradually reducing the aperture of the camera during the acquisition of the multiple frames of images includes: The aperture size of the camera capturing the nth frame image is 1 / n of the aperture size of the camera capturing the 1st frame image.

3. The method according to claim 2, characterized in that, The method further includes: Obtain the aperture size of the camera during photo preview; The camera captures the first frame image at the aperture size used in the photo preview.

4. The method according to claim 3, characterized in that, The aperture size of the camera during the photo preview is the camera's full-open aperture size.

5. The method according to any one of claims 1-4, characterized in that, During the acquisition of the multiple frames of images, the product of the camera's aperture size and exposure time remains constant.

6. The method according to any one of claims 1-4, characterized in that, During the acquisition of the multiple frames of images, the camera takes the same exposure time for each frame.

7. The method according to any one of claims 1-6, characterized in that, Before sequentially acquiring multiple frames of images, the method further includes: Obtain the focal length, maximum aperture size, and acceptable blur circle diameter of the camera; Obtain the focus distance; The generated photo, including a background blur effect, is determined based on the camera's focal length, maximum aperture, acceptable blur circle diameter, and focus distance.

8. The method according to claim 7, characterized in that, The generated photo, determined based on the camera's focal length, maximum aperture, acceptable blur circle diameter, and focusing distance, includes a background blur effect including: The first depth of field is calculated based on the focal length, full aperture, and acceptable blur circle diameter of the camera, as well as the focusing distance. If the first depth of field is less than or equal to a preset value, the generated photo is determined to include a background blur effect.

9. The method according to claim 8, characterized in that, The calculation of the first depth of field based on the camera's focal length, maximum aperture, acceptable blur circle diameter, and focusing distance includes: Wherein, DOF is the first depth of field, M is the maximum aperture value of the camera, C is the acceptable blur circle diameter of the camera, D is the focusing distance, and EFL is the focal length of the camera.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: If the generated photo does not include a background blur effect, based on the camera's focal length, full-aperture size, acceptable blur circle diameter, and focus distance, the camera acquires the image and generates the photo at full aperture.

11. A terminal device, characterized in that, include: A processor, memory, and a camera; wherein the camera is used to acquire images; The memory stores one or more computer programs, the one or more computer programs including instructions that, when executed by the processor, cause the terminal device to perform the method as described in any one of claims 1-10.

12. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-10.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-10.

Images