Focus processing method and electronic device

Abstract

An embodiment of the present application provides a focusing processing method and an electronic device, relating to the field of terminal technologies. The method includes: in response to a first operation for triggering shooting, controlling a flash of the terminal device to perform a first lighting operation. Obtaining a first distance based on the first lighting operation of the flash, where the first distance is the distance between the shooting object and the terminal device. Adjusting shooting parameters according to the first distance, where the shooting parameters include the lens position. Performing image acquisition based on the adjusted shooting parameters to obtain a first image. In this way, in the case where the terminal device is not equipped with a TOF device, the focusing problem in a specific scenario can be solved by combining the PDAF technology with the flash.

Description

Technical Field

[0001] This application relates to the field of terminal technologies, and in particular, to a focusing processing method and an electronic device. Background Art

[0002] With the development of focusing technologies, the PDAF technology has emerged. Applying the PDAF technology to smart phones can achieve fast and accurate focusing. However, in specific scenarios such as backlighting and dark backgrounds, simply using the PDAF technology to achieve focusing still faces challenges.

[0003] To solve the problem of difficult focusing in specific scenarios, in the prior art, a TOF device is used. The TOF device can calculate the distance between an object and a sensor by emitting light pulses. Therefore, combining the TOF device with the PDAF technology can solve the problem of difficult focusing in specific scenarios.

[0004] However, when the terminal device is not equipped with a TOF device, it is difficult to accurately focus on the object to be photographed in specific scenarios. Summary of the Invention

[0005] Embodiments of this application provide a focusing processing method and an electronic device, which are applied to the field of terminal technologies. The technical solution of this application can combine existing devices of the terminal device to solve the problem of difficult focusing in specific scenarios when the terminal device is not equipped with a TOF device.

[0006] In a first aspect, an embodiment of this application proposes a focusing processing method. The method includes:

[0007] In response to a first operation that triggers shooting, control a flash of the terminal device to perform a first lighting operation;

[0008] Obtain a first distance based on the first lighting operation of the flash, where the first distance is the distance between the object to be photographed and the terminal device;

[0009] Adjust shooting parameters according to the first distance, where the shooting parameters include the lens position;

[0010] Perform image acquisition based on the adjusted shooting parameters to obtain a first image.

[0011] In this implementation, by controlling the flash of the terminal device to perform the first lighting operation, the light emitted by the first lighting operation of the flash can be used to obtain the first distance, the position of the lens can be adjusted according to the first distance, and after adjusting the shooting parameters and then performing image acquisition, an image with accurate focusing can be obtained.

[0012] In a possible implementation, obtaining the first distance based on the first lighting operation of the flash includes:

[0013] Based on the first lighting operation of the flash, obtain the first moment when the flash is lit; and,

[0014] Obtain the second moment when the light of the flash returns to the terminal device, where the light of the flash is reflected by the subject and then returns to the terminal device;

[0015] Determine the first duration according to the first moment and the second moment;

[0016] Obtain the first distance according to the first duration and the speed corresponding to the light.

[0017] In this implementation manner, by recording the first moment when the flash starts the first lighting operation and obtaining the second moment when the light of the flash returns to the terminal device, it is convenient to calculate the time difference and improve the efficiency of the terminal device in calculating the focus position of the lens.

[0018] In a possible implementation manner, obtaining the second moment when the light of the flash returns to the terminal device includes:

[0019] After the first moment, based on the first sensor for detecting exposure in the terminal device, detect whether the exposure parameter changes;

[0020] When the exposure parameter changes, obtain the second moment, and the second moment is the moment when the exposure parameter change is detected.

[0021] In this implementation manner, by obtaining the second moment when the exposure parameter changes, the moment when the light emitted by the flash returns can be determined. At the same time, it is operable to record the moment when AE changes.

[0022] In a possible implementation manner, adjusting the shooting parameters according to the first distance includes:

[0023] Determine the target value of the shooting parameters according to the first distance;

[0024] Control the flash to perform the second lighting operation, and during the process of performing the second lighting operation, adjust the shooting parameters based on the target value of the shooting parameters;

[0025] Perform image acquisition based on the adjusted shooting parameters to obtain the first image.

[0026] In this implementation manner, through the calculated first distance, the distance between the lens and the imaging sensor can be calculated, and then the shooting parameters can be adjusted to improve the accuracy of the focusing result.

[0027] In a possible implementation manner, after performing the first lighting operation, the method further includes:

[0028] Obtain first brightness information, where the first brightness information is the brightness information detected by a preset sensor in the terminal device.

[0029] In this implementation manner, by detecting the brightness information through the preset sensor, the ambient brightness can be detected, providing a reference for adjusting the appropriate brightness exposure of the flash, and thus the exposure parameters can be adjusted quickly.

[0030] In a possible implementation manner, the shooting parameters further include exposure parameters and white balance parameters;

[0031] Determine the target values of the shooting parameters according to the first distance, including:

[0032] Determine the target value of the lens position according to the first distance and the focal length of the lens in the terminal device;

[0033] Determine the convergence values of the exposure parameters and the convergence values of the white balance parameters according to the first brightness information;

[0034] Determine the target power corresponding to the second lighting operation according to the first distance, and the first distance is in a direct proportional relationship with the target power;

[0035] Predict the second brightness information of the flash when performing the second lighting operation according to the target power, and adjust the convergence values of the exposure parameters and the convergence values of the white balance parameters based on the second brightness information to obtain the target values of the exposure parameters and the target values of the white balance parameters.

[0036] In this implementation manner, by determining the convergence values of the exposure parameters and the convergence values of the white balance parameters through the first brightness information, stable exposure parameters can be provided when taking pictures. By predicting the second brightness information of the flash when performing the second lighting operation, it can ensure that sufficient light sources are provided in case of insufficient light, ensuring that the shooting object is fully illuminated, thereby improving the brightness and clarity of the photo. By predicting the target values of the exposure parameters and the target values of the white balance parameters through the second brightness information, the quality of the images taken by the flash can be improved.

[0037] In a possible implementation manner, controlling the flash to perform the second lighting operation includes:

[0038] Control the flash to perform the second lighting operation according to the target power.

[0039] In this implementation manner, by the flash performing the second lighting operation based on the target power, not only can the energy consumption of the main flash startup supplied by the terminal device be optimized, but also it can ensure that the taken images are softer and there will be no overexposure or underexposure.

[0040] In a possible implementation, during the execution of the second lighting operation, the shooting parameters are adjusted based on the target values of the shooting parameters, including:

[0041] During the execution of the second lighting operation, when it is determined that the brightness of the flash reaches the preset peak, each shooting parameter is adjusted to its corresponding target value.

[0042] In this implementation, by adjusting each shooting parameter to its corresponding target value when the brightness of the flash reaches the preset peak, it can be ensured that when the exposure conditions meet the requirements, both the exposure and color balance of the captured image are optimal.

[0043] In a possible implementation, the method further includes:

[0044] When the shooting mode triggered by the first operation is the stroboscopic mode, start commands are sent to the first camera and the second camera of the terminal device respectively, and the field of view angle of the first camera is greater than that of the second camera;

[0045] When the shooting object enters the field of view range of the first camera, multiple marked images are collected in sequence based on the first camera, and during the process of collecting the marked images, the flash frequency of the flash is dynamically determined;

[0046] When the shooting object enters the field of view range of the second camera, the flash is controlled to continuously flash based on the dynamically determined flash frequency, and during the process of the continuous flashing of the flash, multiple original images are collected in sequence based on the second camera;

[0047] A stroboscopic image is generated according to the multiple original images.

[0048] In this implementation, by starting the stroboscopic photography mode to obtain the states of the shooting object at different moments, the visual effect of the image can be improved. A wider viewing angle can be captured by the first camera, which can better show the movement trajectory of the shooting object. The details of the shooting object can be better captured by the second camera.

[0049] In a possible implementation, during the process of collecting the marked images, dynamically determining the flash frequency of the flash includes:

[0050] During the process of collecting the marked images, according to the first marked image and the second marked image, the real-time moving speed of the shooting object is determined, the first marked image is the original image closest to the collection moment, and the second marked image is the marked image adjacent to the first marked image;

[0051] According to the real-time moving speed of the shooting object, the flash frequency of the flash is dynamically determined, and the flash frequency is in a direct proportional relationship with the real-time moving speed.

[0052] In this implementation manner, by marking the image to determine the stroboscopic power of the flash, the motion trajectory of the captured object can be recorded completely and clearly. By dynamically determining the flash frequency of the flash, reasonable exposure of the captured object can be achieved, and the quality of the output image can be improved.

[0053] In a possible implementation manner, the method further includes:

[0054] When the duration range of the second camera does not include the captured object, control the flash to stop flashing.

[0055] In this implementation manner, by determining whether the duration range of the second camera includes the captured object, the loss of the terminal device can be saved, and the waste of resources caused by still turning on the flash when the main camera can no longer capture the captured object can be avoided.

[0056] In a possible implementation manner, according to multiple original images, a stroboscopic image is generated, including:

[0057] For any one of the multiple original images, perform object extraction on the captured object included in the original image to generate an intermediate image including the captured object;

[0058] Perform fusion processing on the intermediate images corresponding to the multiple original images respectively to obtain a stroboscopic image.

[0059] In this implementation manner, by operating on the original image to extract the area including the captured object, the unnecessary loss generated by the terminal device during the image fusion operation can be reduced. At the same time, by extracting the area including the captured object, the interference of the background content in the original image on the image processing can be reduced.

[0060] In a possible implementation manner, the method further includes:

[0061] Display a real-time rendering image in the first area of the shooting interface, and the real-time rendering image is an intermediate result image generated during the process of sequentially performing fusion processing on multiple intermediate images.

[0062] In this implementation manner, by displaying a real-time rendering image in the first area of the shooting interface, it can help the user view the shooting effect in real time and improve the user-friendliness.

[0063] In a second aspect, an embodiment of the present application provides a focus processing device, which may be an electronic device, or a chip or a chip system inside the electronic device. The focus processing device may include a display unit and a processing unit.

[0064] When the focus processing device is an electronic device, the display unit may be a display screen. The display unit is used to perform the display step so that the electronic device implements a focus processing method described in the first aspect or any possible implementation manner of the first aspect.

[0065] When the focus processing device is an electronic device, the processing unit may be a processor. The focus processing device may further include a storage unit, and the storage unit may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements a focus processing method described in the first aspect or any possible implementation manner of the first aspect.

[0066] When the focus processing device is a chip or a chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements a focus processing method described in the first aspect or any possible implementation manner of the first aspect. The storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or may be a storage unit outside the chip in the electronic device (for example, a read-only memory, a random access memory, etc.).

[0067] Exemplarily, the display unit is used for screen display processing of a terminal device. Exemplarily, the screen acts as a viewfinder to provide real-time preview, etc.

[0068] The processing unit is used to control the flash to perform a flash operation, and can determine the distance between the terminal device and the shooting object according to the flash operation of the flash, and then perform focus processing according to the distance. Further, for example, a series of image processing may also be performed on the original image acquired by the camera. Exemplarily, the image processing may include operations such as denoising, demosaicing, color correction, and sharpening. And it may also include operations of automatic exposure, automatic focus, and automatic white balance algorithms, and may also perform format conversion on the processed image data. In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory. The memory is used to store code instructions, and the processor is used to run the code instructions to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0069] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. A computer program or instructions are stored in the computer-readable storage medium. When the computer program or instructions are run on a computer, the computer is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0070] Fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, it causes the computer to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0071] Sixth aspect, the present application provides a chip or a chip system. The chip or the chip system includes at least one processor and a communication interface. The communication interface and the at least one processor are interconnected by a line. The at least one processor is configured to run a computer program or instruction to execute the method described in the first aspect or any possible implementation manner of the first aspect. Among them, the communication interface in the chip can be an input / output interface, a pin, a circuit, etc.

[0072] In a possible implementation, the chip or the chip system described above in the present application further includes at least one memory, and instructions are stored in the at least one memory. The memory can be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0073] It should be understood that the second aspect to the sixth aspect of the present application correspond to the technical solutions of the first aspect of the present application. The beneficial effects obtained by each aspect and the corresponding feasible implementation manners are similar and will not be elaborated herein. Description of the Drawings

[0074] Figure 1 Schematic diagram of different focusing positions provided by an embodiment of the present application;

[0075] Figure 2 Schematic diagram of an image sensor provided by an embodiment of the present application;

[0076] Figure 3 Schematic diagram of the principle of a focusing pixel provided by an embodiment of the present application;

[0077] Figure 4 Schematic diagram of an application scenario of a focusing technology provided by an embodiment of the present application;

[0078] Figure 5 Schematic diagram of the technical principle of a TOF device provided by an embodiment of the present application;

[0079] Figure 6 Schematic diagram of the hardware structure of a terminal device provided by an embodiment of the present application;

[0080] Figure 7 Schematic diagram of the software structure of a terminal device provided by an embodiment of the present application;

[0081] Figure 8Schematic flowchart of the flash-assisted focusing solution provided by the embodiments of the present application;

[0082] Figure 9 Schematic diagram of the measures taken for focusing failure provided by the embodiments of the present application;

[0083] Figure 10 Schematic diagram of the pre-flash of the flash provided by the embodiments of the present application;

[0084] Figure 11 Schematic diagram of the flash-assisted ranging provided by the embodiments of the present application;

[0085] Figure 12 Schematic flowchart of the process of capturing images in a dark environment provided by the embodiments of the present application;

[0086] Figure 13 Schematic diagram of the light intensity distribution provided by the embodiments of the present application;

[0087] Figure 14 Schematic diagram of the attenuation function of the flash brightness provided by the embodiments of the present application;

[0088] Figure 15 Schematic diagram of the solution timing provided by the embodiments of the present application;

[0089] Figure 16 Schematic flowchart of the process of stroboscopic photography provided by the embodiments of the present application;

[0090] Figure 17 Schematic diagram of enabling the stroboscopic mode provided by the embodiments of the present application;

[0091] Figure 18 Schematic diagram of the viewfinder provided by the embodiments of the present application;

[0092] Figure 19 Schematic diagram of the wide-angle camera capturing an image provided by the embodiments of the present application;

[0093] Figure 20 Schematic diagram of the marked images at different times provided by the embodiments of the present application;

[0094] Figure 21 Schematic diagram of the normal camera capturing an image provided by the embodiments of the present application;

[0095] Figure 22 Schematic diagram of extracting the region containing the subject provided by the embodiments of the present application;

[0096] Figure 23 Schematic diagram of the image fusion processing provided by the embodiments of the present application;

[0097] Figure 24Schematic diagram of real-time rendered image provided by an embodiment of this application;

[0098] Figure 25 Schematic diagram of a scene with flash stopped provided by an embodiment of this application;

[0099] Figure 26 Schematic flowchart of a focusing processing method provided by an embodiment of this application;

[0100] Figure 27 Schematic flowchart of a stroboscopic photography method provided by an embodiment of this application;

[0101] Figure 28 Schematic diagram of the hardware structure of an electronic device provided by an embodiment of this application. Detailed implementation manners

[0102] To facilitate a clear description of the technical solutions of the embodiments of this application, the following briefly introduces some terms and technologies involved in the embodiments of this application:

[0103] 1. Focusing: In the field of optics, focusing refers to the process of adjusting the focal position of an optical system to form a clear image of light from an object on a photosensitive element. When an object is within the effective focal length range of a lens, by adjusting the distance between the lens and the photosensitive surface, the light reflected or emitted by the object can be accurately converged to form an image on the photosensitive surface.

[0104] If this distance is adjusted properly, the image formed on the photosensitive surface will be very clear, and it can be called in a focused state; conversely, if the distance is inappropriate, the image will appear blurred.

[0105] 2. CAM module: A camera module (Camera Module, CAM) is a component that integrates an optical lens, an image sensor, an image signal processor (Image Signal Processor, ISP), and other related electronic components and circuits. Its main function is to achieve the shooting and acquisition of images or videos.

[0106] 3. PDAF: Phase detection auto focus (PDAF) is an autofocus technology used in cameras and smartphones. It determines the focus position by comparing the phase differences of light entering from different parts of the lens.

[0107] Specifically, the PDAF technology sets dedicated focus pixels on the sensor. These pixels are not directly used for imaging but are specifically designed to capture light information and measure the phase difference between two beams of light passing through different regions of the lens. If the phases of the two beams of light are consistent, it indicates that the object is in the quasi-focused state; if there is a phase difference, the direction and distance that the lens needs to move are determined by calculating this difference to achieve precise focusing.

[0108] 4. TOF: In the field of optical measurement and imaging, Time of Flight (TOF) is a technical means for calculating the distance between the object to be measured and the camera. The specific operation method is as follows: measure the time required for the light to be emitted from the emitter, hit the object to be measured, and then be reflected back and received by the receiver, and complete the distance calculation based on this time.

[0109] 5. AE: Auto Exposure (AE) is an important function of imaging devices such as cameras and mobile phone cameras. Its purpose is to automatically adjust exposure parameters such as aperture size, shutter speed, and ISO according to the light conditions of the shooting scene to obtain an appropriate exposure amount, ensuring that the brightness of the captured image is moderate, neither overexposed (the image is too bright, resulting in loss of details) nor underexposed (the image is too dark to see the content clearly).

[0110] 6. AF: Auto Focus (AF) is one of the key functions of imaging devices such as cameras and mobile phone cameras. It can automatically and quickly focus the shooting object clearly on the image sensor, which enables users to conveniently obtain clear images or videos without having to manually and precisely adjust the lens focusing position.

[0111] 7. AWB: Auto White Balance (AWB) is a function of imaging devices such as cameras and mobile phone cameras. Its main function is to automatically adjust the color balance of the image under different lighting conditions, so that white objects still appear close to real white in the captured image, thereby ensuring that the color of the entire image looks natural and accurate, and avoiding color distortion of the image caused by light color deviation.

[0112] 8. Optical flow method: The optical flow method is a computer vision technology for analyzing the motion of objects in an image sequence. Its basic idea is to describe the motion of an object by calculating the motion speed vector (including magnitude and direction) of pixels in the image.

[0113] In the embodiments of the present application, the motion speed of the shooting object in the image can be determined according to the images collected at adjacent times.

[0114] 9. Other terms: In the embodiments of the present application, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and roles. For example, the first chip and the second chip are only used to distinguish different chips, and do not limit their sequence. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different.

[0115] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0116] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (s) or plural item (s). For example, at least one (item) of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c can be single or multiple.

[0117] 10. Electronic device: The electronic device according to the embodiments of the present application may include a handheld device, a vehicle-mounted device, etc. having a focusing and shooting function. For example, some electronic devices are: mobile phone, tablet computer, handheld computer, laptop computer, mobile internet device (MID), wearable device, virtual reality (VR) device, augmented reality (AR) device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, cellular phone, cordless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, personal digital assistant (PDA), handheld device with wireless communication function, computing device or other processing devices connected to a wireless modem, vehicle-mounted device, wearable device, terminal device in a 5G network or terminal device in a future evolved public land mobile network (PLMN), etc. The embodiments of the present application are not limited thereto.

[0118] By way of example and not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. A wearable device may also be referred to as a wearable intelligent device, which is a general term for devices developed by applying wearable technologies to the intelligent design of daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is either directly worn on the body or integrated into the user's clothing or accessories. A wearable device is not just a hardware device, but also realizes powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable intelligent devices include those with complete functions and large sizes that can realize complete or partial functions without relying on a smart phone, such as smart watches or smart glasses, etc., and those that only focus on a certain type of application function and need to cooperate with other devices such as smart phones, such as various smart bracelets and smart jewelry for monitoring physical signs.

[0119] In addition, in the embodiments of the present application, the electronic device may also be a terminal device in an Internet of Things (IoT) system. The IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-object interconnection.

[0120] The electronic device in the embodiments of the present application may also be referred to as: terminal device, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile terminal, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, etc.

[0121] In the embodiments of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement service processing through processes. For example, Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system, etc. The application layer contains applications such as browsers, address books, word processing software, and instant messaging software.

[0122] To better understand the technical solution of the present application, the related technologies involved in the present application will be further introduced in detail below.

[0123] With the development of terminal devices, the photographing effect of mobile phone cameras is getting better and better. The improvement of the photographing effect is inseparable from the development of focusing technology. Nowadays, in terminal devices, the PDAF technology has been widely used, mainly because it can provide fast and accurate focusing.

[0124] The following will be combined with Figures 1 to 3 to introduce the technical principle of PDAF to understand how the PDAF technology realizes focusing. Figure 1 Schematic diagrams of different focusing positions provided for the embodiments of the application, Figure 2 Schematic diagram of the imaging sensor provided for the embodiments of the present application, Figure 3 Schematic diagram of the principle of the focusing pixel provided for the embodiments of the present application.

[0125] It is understandable that the principle of the human eye to achieve focus is similar to that of the PDAF technology to achieve focus. Therefore, the focusing principle of the PDAF technology can be understood by understanding the focusing principle of the human eye.

[0126] When the two eyes of a person achieve focus, since the two eyes of a person are similar to two independent "imaging systems", and there is a certain distance between the positions of the two eyes, the object images seen by the left and right eyes at the same moment are different, forming different visual information. Then, the brain will receive and process the visual information from both eyes, and judge the distance and spatial position of the object by analyzing these differences, and then achieve focus, so that the image of the object accurately falls on the retina.

[0127] Then, it can be understood that when the PDAF technology achieves focus, it is necessary to reserve some masked pixel points on the photosensitive element. These pixel points are divided into two parts, left and right. One part only allows the light on the left to pass through, and the other part only allows the light on the right to pass through. At the same time, it can be understood that the masked pixel points in the left and right parts are similar to the left and right eyes of a person. By using these two groups of pixel points to image the photographed object respectively, two slightly different images are obtained. Then, the phase difference between these two images is compared, and the offset value of the focus is calculated according to the phase difference information, so as to judge the direction and distance that the lens needs to move to achieve accurate focus.

[0128] To further understand the meaning of focus, the following will be combined with Figure 1 for introduction.

[0129] In one implementation, as shown in (a) of Figure 1 assume that there is an object 101, a lens 102, and an imaging sensor 103. Among them, the lens 102 and the imaging sensor 103 belong to the components of the camera. When the camera captures the object 101, the light 104 reflected by the object 101 enters through the upper end of the lens 102, and the light 105 reflected by the object 101 enters through the lower end of the lens 102. After that, the two beams of light converge in front of the imaging sensor 103 and converge to the focal point 106, but the focal point 106 is not on the imaging sensor 103. It can be understood that at this time, the focus fails and the focal point is not on the imaging sensor.

[0130] Or, as shown in Figure 1As shown in (b) therein, it is assumed that there are an object 101, a lens 102, and an imaging sensor 103. Among them, the lens 102 and the imaging sensor 103 are components of a camera. When the camera captures the object 101, the light ray 104 reflected by the object 101 enters through the upper end of the lens 102, and the light ray 105 reflected by the object 101 enters through the lower end of the lens 102. After that, the two light rays converge after the imaging sensor 103 and converge at the focal point 106, but the focal point 106 is not on the imaging sensor 103. It can be understood that at this time, the focusing fails and the focal point is not on the imaging sensor.

[0131] In another implementation, as Figure 1 shown in (c) therein, it is assumed that there are an object 101, a lens 102, and an imaging sensor 103. Among them, the lens 102 and the imaging sensor 103 are components of a camera. When the camera captures the object 101, the light ray 104 reflected by the object 101 enters through the upper end of the lens 102, and the light ray 105 reflected by the object 101 enters through the lower end of the lens 102. After that, the two light rays converge on the imaging sensor 103 and converge at the focal point 106, and the focal point 106 is on the imaging sensor 103. It can be understood that at this time, the focusing is successful and the focal point is on the imaging sensor.

[0132] It should be noted that the light ray can enter through the upper end or the lower end of the lens, or can enter through the left end or the right end of the lens. The embodiments of the present application do not limit this. As long as the two light rays enter from the symmetric positions of the lens, it can be used to analyze whether the focusing can be determined to be successful.

[0133] In addition, according to Figure 1 shown in (a) therein, it can be found that the focal point 106 is located in front of the imaging sensor 103, which can be understood as before focusing. At this time, the light ray received by the upper end of the imaging sensor 103 is the light ray 105, and the light ray received by the lower end of the imaging sensor 103 is the light ray 104. And according to Figure 1 shown in (b) therein, it can be found that the focal point 106 is located behind the imaging sensor 103, which can be understood as after focusing. At this time, the light ray received by the upper end of the imaging sensor 103 is the light ray 104, and the light ray received by the lower end of the imaging sensor 103 is the light ray 105.

[0134] Then it can be understood that if the imaging sensor can distinguish whether the light ray comes from the upper half or the lower half of the lens, it can be determined whether the current focusing failure situation belongs to before focusing or after focusing.

[0135] In order to distinguish whether the light ray comes from the upper half or the lower half of the lens, in the prior art, a sensor with phase detection pixels covering the left half or the right half is provided. The following will be introduced in combination with Figure 2 this.

[0136] As Figure 2 shown, assume there is a sensor 201, where the sensor 201 contains many sub - sensors, each sub - sensor corresponding to a pixel. These pixels are usually arranged in pairs, and each pair of pixels receives light from different parts of the lens. The sub - sensors are divided into three categories: unobstructed phase - detection pixels 202, phase - detection pixels 203 with the right half blocked, and phase - detection pixels 204 with the left half blocked. These three types of sensors together constitute the sensor 201 to receive light from the lens. It can be understood that the phase - detection pixels 203 with the right half blocked can only receive light from the left, and can also be called left - pixel sensors; the phase - detection pixels 204 with the left half blocked can only receive light from the right, and can also be called right - pixel sensors.

[0137] Then, regarding the technical principles of the left - pixel sensor and the right - pixel sensor, it can be understood in combination with Figure 3 this.

[0138] As Figure 3 shown in (a) of [reference], in the technical implementation of the right - pixel sensor, assume there is a lens 301, an occlusion area 302, an imaging sensor 303, an optical axis 304, light rays 305 in the upper half of the optical axis, and light rays 306 in the lower half of the optical axis. Then, after the light rays 305 and other light rays in the upper half of the optical axis pass through the lens 301, they propagate to the upper half of the imaging sensor 303; after the light rays 306 and other light rays in the lower half of the optical axis pass through the lens 301, due to the existence of the occlusion area 302, they cannot propagate to the lower half of the imaging sensor 303. Then it can be understood that the right - pixel sensor can only receive the light from the upper half of the lens.

[0139] As Figure 3 shown in (b) of [reference], in the technical implementation of the left - pixel sensor, assume there is a lens 301, an occlusion area 302, an imaging sensor 303, an optical axis 304, light rays 305 in the upper half of the optical axis, and light rays 306 in the lower half of the optical axis. Then, after the light rays 305 and other light rays in the upper half of the optical axis pass through the lens 301, due to the existence of the occlusion area 302, they cannot propagate to the upper half of the imaging sensor 303; after the light rays 306 and other light rays in the lower half of the optical axis pass through the lens 301, they propagate to the lower half of the imaging sensor 303. Then it can be understood that the left - pixel sensor can only receive the light from the lower half of the lens.

[0140] Based on the above content, the technical principles of the left - pixel sensor and the right - pixel sensor can be understood. Then it can be understood that after the light passes through different pixel sensors, the light will form different phases, and then the phase difference can be calculated based on different phases to achieve focusing.

[0141] However, although PDAF technology can complete the focusing task in many cases, there are still some limitations. Exemplarily, in a low-light environment, PDAF technology may not be able to accurately identify the phase difference, resulting in inaccurate focusing. It can be understood that when the light is insufficient, the focusing pixels on the specially set sensor of PDAF receive less light, so it is impossible to measure the phase difference between the two beams of light passing through different regions of the lens, and thus it is difficult to perform accurate phase difference calculation, and accurate focusing cannot be achieved.

[0142] The following combines Figure 4 to introduce the application scenarios of PDAF. Figure 4 It is a schematic diagram of the application scenario of the focusing technology provided by the embodiment of the present application.

[0143] As Figure 4 shown, assume there is a terminal device 401. As Figure 4 shown in (a), the interface can be understood as the desktop interface of the terminal device 401. This interface may include: controls for the desktop such as time, date, weather, etc., and controls for applications such as theme, phone, camera, etc.

[0144] In one implementation, in response to a user's trigger operation on the control 402 of the camera application, the terminal device displays an interface as Figure 4 shown in (b). This interface can be understood as the camera application's photo-taking interface. It can be understood that the photo-taking interface of the terminal device displays the scene captured by the current camera.

[0145] Then it can be understood that the scene corresponding to the current screen 403 has sufficient light. Therefore, it is easy to focus successfully, and the manifestation of successful focusing can be that the person 404 in the output image is clear.

[0146] In another implementation, in response to a user's trigger operation on the control 402 of the camera application, the terminal device displays an interface as Figure 4 shown in (c). This interface can be understood as the camera application's photo-taking interface. It can be understood that the photo-taking interface of the terminal device displays the scene captured by the current camera.

[0147] Then it can be understood that the scene corresponding to the current screen 405 has relatively dim light. Therefore, it is easy to focus fail, and the manifestation of focus failure can be that the person 406 in the output image is blurred.

[0148] It should be noted that there can be many scenarios where PDAF has difficulty in focusing. For example, when shooting a flat or less textured area, since phase detection depends on the distance change between pixels, and in a flat area, this change is small, resulting in an unclear phase difference, PDAF technology may have difficulty in accurately focusing.

[0149] For another example, in a backlit scene, strong light may cause overexposure of the light received by the phase detection pixels, which may interfere with the phase difference calculation of the PDAF technology and result in inaccurate focusing.

[0150] Therefore, it is necessary to determine according to the specific scene whether the PDAF technology can focus successfully, and the embodiments of the present application do not limit this.

[0151] In the prior art, a TOF device is used to solve the problem of difficult focusing in specific scenes. The TOF device can calculate the distance between an object and a sensor by emitting laser light. The following will introduce the principle of the TOF device in combination with Figure 5 the principle of the TOF device will be introduced. Figure 5 FIG. is a schematic diagram of the technical principle of the TOF device provided by the embodiments of the present application.

[0152] As Figure 5 shown, assume there is an object 5001 and a TOF device 5002. Among them, the TOF device 5002 includes a laser emitter 5003, a laser detector 5004, etc.

[0153] When the TOF device 5002 works, the laser emitter 5003 emits laser light, and the emitted laser light can be understood as light ray 5006. When the light ray 5006 reaches the surface of the object 5001, reflection will occur, and the reflected laser light can be understood as light ray 5007. When the light ray 5007 returns to the TOF device 5002, the laser detector 5004 can detect the returned light ray.

[0154] After the laser emitter 5003 completes the operation of emitting laser light and the laser detector 5004 completes the detection of the reflected laser light, the TOF device will start the ranging function to complete the calculation of the distance between the terminal device and the shooting object. According to the calculated distance between the terminal device and the shooting object, it can assist the PDAF technology to complete the focusing operation.

[0155] Then it can be understood that for a terminal device equipped with a TOF device, the TOF technology can be used for auxiliary focusing. However, for a terminal device not equipped with a TOF device, it is impossible to use the TOF technology for auxiliary focusing. Then, for the specific scenes described above, such as in a dim light environment, when shooting a flat area, and in a backlit shooting environment, there may be a problem that it is impossible to accurately focus on the shooting object, and thus a clear image cannot be obtained.

[0156] In view of the problems introduced above, the present application proposes a technical concept: Since flashlights are configured in most terminal devices, for example, the flashlight can be used for assisted focusing. For example, the PDAF technology can be combined with the flashlight, and the distance between the photographed object and the terminal device can be calculated by recording the time from when the flashlight emits light to when it returns, thereby solving the problem of accurate focusing when the terminal device is not equipped with a TOF device.

[0157] The focusing processing method of the embodiments of the present application can be executed by an electronic device equipped with a camera, or by a chip, a chip system, or a processor that supports the electronic device to implement the focusing processing method, or by a logic module or software that can implement all or part of the functions of the electronic device. The present application does not make specific limitations on this. Hereinafter, taking the electronic device as the execution subject as an example, the focusing processing method of the embodiments of the present application will be described in detail.

[0158] Among them, the electronic device can be, for example, a terminal device. First, hereinafter, in combination with Figure 6 and Figure 7 a simple introduction to the terminal device will be given.

[0159] Exemplarily, Figure 6 is a schematic diagram of the hardware structure of a terminal device provided by an embodiment of the present application.

[0160] Figure 6 is a schematic diagram of the structure of the terminal device provided by an embodiment of the present application. The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a mobile communication module 150, a wireless communication module 160, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a gyroscope sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, an image sensor, etc.

[0161] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The components shown may be implemented by hardware, software, or a combination of software and hardware.

[0162] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. In the embodiments of the present application, the processor 110 may, for example, control the flash to perform a flash operation, and may determine the distance between the terminal device and the photographed object according to the flash operation of the flash, and then perform focus processing according to the distance. Further, the processor 110 may, for example, also perform a series of image processing on the raw image collected by the camera. Exemplarily, the image processing may include operations such as denoising, demosaicing, color correction, and sharpening. And it may also include operations of automatic exposure, autofocus, and automatic white balance algorithms, and may also perform format conversion on the processed image data, etc.

[0163] The terminal device realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0164] The terminal device may realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0165] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera sensor. The optical signal is converted into an electrical signal, and the camera sensor transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP may be disposed in the camera 193.

[0166] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0167] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device. In some embodiments, the angular velocity of the terminal device around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes. In an embodiment of the present application, the gyroscope sensor can detect the tilt angle of the terminal device when taking an image.

[0168] The distance sensor 180F is used to measure the distance. The terminal device can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0169] Image sensor 180J is used to detect changes in AE parameters. In the embodiment of the present application, when the light emitted by the flash reaches the surface of the photographed object, reflection occurs. The moment when the reflected light returns to the terminal device can be recorded by the moment when the image sensor detects the change in the AE value.

[0170] The software system of the terminal device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device.

[0171] For example, Figure 7 A schematic diagram of the software structure of a terminal device provided in an embodiment of the present application.

[0172] like Figure 7As shown, the layered architecture divides software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system may include an application layer, an application framework layer, an Android runtime, system libraries, a hardware abstraction layer (HAL), and a kernel layer. It should be noted that the embodiments of this application take the Android system as an example. In other operating systems (such as HarmonyOS, iOS system, etc.), as long as the functions implemented by each functional module are similar to those of the embodiments of this application, the solution of this application can also be implemented.

[0173] Among them, the application layer may include a series of application packages.

[0174] Such as Figure 7 As shown, the application packages may include applications such as camera, gallery, phone, map, phone, music, settings, email, video, and social. Of course, the application layer may also include other application packages, such as third-party applications such as payment applications, shopping applications, bank applications, and social applications, which are not limited in this application. In the embodiments of this application, for example, the user can trigger a shooting operation through the camera application, and the image generated after shooting can be viewed in the gallery application.

[0175] Among them, the application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0176] Such as Figure 7 As shown, the application framework layer may include a multimedia framework, a window manager, a content provider, a resource manager, a view system, etc.

[0177] In this embodiment, the multimedia framework can, for example, receive the shooting instructions sent by the camera application to perform operations such as starting the camera and starting the shooting preview. In addition, the multimedia framework can also call the camera HAL in the HAL layer to perform subsequent shooting operations.

[0178] Among them, the system libraries may include multiple functional modules. For example: surface manager, 3D graphics processing library (such as: OpenGL ES), 2D graphics engine (such as: SGL), memory manager, etc.

[0179] Among them, the HAL layer is an encapsulation of the Linux kernel driver, providing interfaces upward and shielding the implementation details of the lower-level hardware. The HAL layer may include a camera HAL, software code libraries, etc.

[0180] In this embodiment, the camera HAL is responsible for interacting with the actual hardware device (such as a camera). By providing a standardized interface, the HAL enables the Android framework layer to operate the device through the HAL layer without the need to understand the hardware details. For example, the camera HAL can communicate with the camera driver to control tasks such as starting, configuring, and data acquisition of the camera. Further, the camera HAL can also send a start instruction to the flash driver in the kernel layer to control the flash to light up, thereby performing the main flash and pre-flash operations described in this application. Additionally, the camera HAL can drive the underlying lens to perform focus processing based on the object distance determined based on the flash.

[0181] The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0182] In this embodiment, the camera driver is responsible for driving the underlying camera hardware for image acquisition, and the flash driver is responsible for driving the underlying flash hardware for lighting processing.

[0183] Next, in combination with the accompanying drawings, the technical solutions of the embodiments of this application and how the technical solutions of the embodiments of this application solve the above technical problems will be described in detail. These several specific embodiments below can be implemented independently or in combination with each other. For the same or similar concepts or processes, they may not be repeated in some embodiments.

[0184] Based on the above introduction, the technical solutions provided by this application will be introduced in detail below in combination with specific drawings. Figure 8 It is a schematic flowchart of the flash-assisted focusing solution provided by the embodiment of this application. Figure 9 It is a schematic diagram of the measures taken for focusing failure provided by the embodiment of this application. Figure 10 It is a schematic diagram of starting the pre-flash of the flash provided by the embodiment of this application. Figure 11 It is a schematic diagram of flash-assisted ranging provided by the embodiment of this application.

[0185] As Figure 8As shown, a solution for using a flash to assist in focusing provided by an embodiment of the present application is introduced. It can be understood that by using the pre-flash of the flash to emit light, when the light reaches the surface of the photographed object, the light will be reflected by the photographed object and then return to the lens, and then the sensor in the lens module will detect that the exposure parameters have changed. Therefore, the time difference between the moment when the flash turns on the pre-flash and the moment when the sensor detects the change in exposure parameters can be used to calculate the time difference between the emission and return of the light, and based on the time difference and the speed of light, the distance between the photographed object and the terminal device can be calculated. Based on the distance between the photographed object and the terminal device, the position that the lens needs to adjust can be calculated, and thus accurate focusing can be achieved.

[0186] The following will be described in detail in conjunction with Figure 8 and with reference to Figure 8 , the method includes:

[0187] S801. Detect a first operation.

[0188] In an embodiment of the present application, the first operation can be understood as an operation to trigger shooting. The following will be described in conjunction with Figure 9 in (a) to understand the first operation.

[0189] As shown in (a) of Figure 9 , it is assumed that when the terminal device detects that the user opens the camera application, it can display an interface as shown in (a) of Figure 9 . The interface may include: a shooting control 901, a thumbnail, a preview screen 902, a control for switching the camera, a shooting mode selection item, etc.

[0190] Exemplarily, the user can trigger shooting by clicking the shooting control 901. Therefore, the click operation on the shooting control 901 can be understood as the first operation introduced above. And there can be various specific implementations of the first operation to trigger shooting. In addition to clicking the shooting control, the first operation can also be, for example, a pressing operation on the volume key output by the terminal device, or the first operation can also be an operation on the shooting instruction input to the voice assistant.

[0191] S802. Determine whether the focusing is successful. If so, execute S808; if not, execute S803.

[0192] The technical solution of this application is based on the focusing result. In some cases, for example, in the well-lit environment introduced above, if successful focusing can be achieved based on the PDAF technology, the photographing process can be executed based on the focusing result of the PDAF technology. In other cases, for example, in the environment with low light and flat areas introduced above, it may be impossible to achieve successful focusing based on the PDAF technology. In this case, the auxiliary focusing can be achieved by means of a flash according to the technical solution of this application. Therefore, after triggering the shooting in response to the first operation input by the user, for example, it can be first determined whether the focusing based on the PDAF technology is successful.

[0193] Thus, different subsequent processes are performed according to the focusing result. On the one hand, the probability of successful focusing can be increased. On the other hand, the energy consumption of the terminal device can be reduced, and the resource waste caused by still turning on the flash to perform subsequent processing when successful focusing can be achieved based on the PDAF technology can be avoided.

[0194] When determining whether the focusing is successful, in one implementation, for example, a difference value threshold of a phase detection (PD) point can be set according to factors such as the lens characteristics of the terminal device and the shooting scene. This threshold is the boundary for determining whether the focusing is successful. For example, when the difference value of the phase detection point exceeds this threshold, it can be considered that the focusing fails; when the difference value of the phase detection point is less than this threshold, it can be considered that the focusing is successful.

[0195] It can be understood that when the difference value of the phase detection point is less than this threshold, it means that the light phase differences obtained from each detection point are relatively close to the phase difference value in the ideal focusing state. This enables the camera to more accurately determine the distance between the photographed object and the terminal device, so that the focusing component of the lens can be adjusted more accurately, and the focus can be accurately placed on the photographed object to achieve successful focusing. When the difference value of the phase detection point exceeds the threshold, it indicates that the difference from the phase difference value in the ideal focusing state is relatively large. Therefore, the focusing is likely to fail.

[0196] It should be noted that, in addition to using the difference value of the phase detection point to determine whether the focusing is successful as mentioned above, it can also be determined according to the focusing speed. Specifically, which method is used to determine whether the focusing is successful can be selected according to the actual situation, and the embodiments of this application do not limit this.

[0197] The following combines Figure 9 and Figure 10 to introduce the operation process of performing auxiliary focusing by starting the flash to obtain a clear image when the focusing fails using the PDAF technology in a specific scenario.

[0198] In one implementation, when the focusing fails and the initial state of the flash is the self-start mode, the flash will be automatically activated to implement the function of auxiliary ranging, so as to obtain an image of the shooting scene.

[0199] As Figure 9 shown in (a) of Figure 9 , it is assumed that the terminal device can display an interface as shown in (a) of

[0200] when detecting that the user opens the camera application. This interface can be understood as the shooting interface of the camera application. Among them, the picture 902 of the shooting scene displayed on the shooting interface has a darker background, and the current state of the flash is the self-start mode 903. Figure 9 In response to the triggering operation on the shooting control 901, when the terminal device detects that the current focusing fails, it displays an interface as shown in (b) of

[0201] . In this interface, the flash changes from the self-start mode 903 to the on mode 904. In response to the automatic on mode 904 of the flash, the lens emits light 905, and the picture 906 becomes clear. Figure 9 After that, after a certain period of time, the interface displayed by the terminal device jumps to an interface as shown in (c) of

[0202] . This interface shows the initial interface after shooting, which can be understood as the interface of the scene captured by the camera when the camera application is opened. Among them, the interface may include a thumbnail control 907 and a picture 908 of the shooting scene. It can be understood that the thumbnail control 907 displays a preview of the captured image. At the same time, it can be understood that since the flash has completed the auxiliary focusing operation and returned to the self-start state, therefore, in the current scene, the light in the picture 908 of the shooting scene is dim, but the person becomes clear.

[0203] As Figure 10 shown in (a) of Figure 10 , it is assumed that the terminal device can display an interface as shown in (a) of

[0204] when detecting that the user opens the camera application. This interface can be understood as the shooting interface of the camera application. Among them, the picture 1002 of the shooting scene displayed on the shooting interface has a darker background, and the current state of the flash is the off mode 1003. Figure 10In the interface shown in (b), when the terminal device detects that the current screen is dim, a message prompt box 1004 for failed focusing is popped up. After performing the "Confirm" operation on 1004, the flash will be turned on, and after performing the "Cancel" operation on 1004, the photo taking will be cancelled or the flash turn-on will be rejected.

[0205] In response to the trigger operation for "Confirm" on the message prompt box 1004, the interface shown in Figure 10 (c) is displayed. In this interface, the flash changes from the off mode 1003 to the on mode 1005. In response to the on mode 1005 of the flash, the lens emits light 1006, and the picture 1007 becomes clear.

[0206] After that, after a certain period of time, the display interface of the terminal device jumps to the interface shown in Figure 10 (d). This interface may include a thumbnail control 1008 and a picture 1009 of the shooting scene. Among them, the thumbnail control 1008 displays a preview of the captured image, and in the picture 1009 of the shooting scene, the background is dim but the person is clear.

[0207] It should be noted that when the focusing fails and the flash is in the off state, in addition to the above-described method of popping up a message prompt box to manually start the flash, the flash can also be automatically turned on without the user's confirmation. This method can reduce the user's operations. In practical applications, the specific method of selecting to start the flash can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0208] S803: Start the pre-flash of the flash and record the first moment of start.

[0209] In the case of failed focusing based on the PDAF technology, it is necessary to use the flash to achieve assisted focusing. At this time, for example, the flash can be controlled to perform a pre-flash operation, where the pre-flash refers to a series of flashes emitted by the flash.

[0210] In one implementation, the distance between the device and the object can be determined according to the duration from the emission to the return of the light of the flash. The moment when the pre-flash of the flash is started is also equivalent to the moment when the light is emitted. Therefore, the moment when the pre-flash operation of the flash is started can be recorded as the first moment, and the first moment can also be understood as the starting moment of the light emission.

[0211] Thus, recording the moment when the flash starts the pre-flash can facilitate the calculation of the time difference, and then the efficiency of the terminal device in calculating the focusing position of the lens can be improved.

[0212] In actual operation, recording the moment when the flash changes from the non - turned - on state to the turned - on state can be regarded as the first moment. The specific manifestation of the non - turned - on state can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0213] S804. Obtain the second moment when AE changes.

[0214] In the case of focusing failure, when using the flash to achieve assisted focusing, in addition to the first moment of recording the pre - flash start of the flash introduced in S803 above, it is also necessary to record the moment when the light emitted by the pre - flash of the flash returns, which can also be regarded as the second moment when AE changes.

[0215] In one implementation, the image sensor detects the change of the AE value in real time. When the light emitted by the pre - flash of the flash returns, the initial value of AE will change. Then, it is necessary to record the moment when the AE value changes as the second moment.

[0216] S805. Determine the first duration according to the first moment and the second moment.

[0217] According to the above content, the first moment is obtained from S803 and the second moment is obtained from S804. Then, according to the difference between the two moments, the time difference between the emission and return of the light emitted by the pre - flash of the flash can be calculated. Then, this time difference can be regarded as the first duration. Thus, the first duration is obtained, laying a foundation for further calculating the distance between the terminal device and the shooting object.

[0218] The following combines Figure 11 to introduce the process of flash - assisted focusing.

[0219] As Figure 11 shown, assume that there is a terminal device 1101 photographing an object 1102. Among them, the terminal device 1101 may include a lens module 1103, and the lens 1103 includes a flash. When the flash is in the turned - on state, record the moment when the flash starts to pre - flash as t1, that is, the first moment mentioned above.

[0220] After the flash is turned on, it will emit uniform light, which can be regarded as light 1104. When the light 1104 reaches the surface of the object 1102, reflection will occur, and the reflected light can be regarded as light 1105. When the light 1105 reaches the lens module 1103, record the moment when AE changes as t2, that is, the second moment mentioned above. Then, the time difference between t1 and t2 is the first duration, which can also be regarded as the time difference between the emission and return of the light.

[0221] S806. Determine the first distance according to the first duration and the speed of light.

[0222] According to the above content, what is recorded in the embodiments of this application is the time difference from the emission to the return of the light. Then it can be understood that when the light travels from the emission until it reaches the surface of the photographed object, it has passed the distance from the terminal device to the photographed object; and when the light travels from the photographed object back to the terminal device, it has passed the distance from the photographed object to the terminal device. The two distances are equal. Therefore, the time difference for the light to travel from the terminal device to the photographed object or the time difference for the light to travel from the photographed object to the terminal device is the same, and both are half of the first duration. At the same time, it can be understood that the distance from the terminal device to the photographed object can be regarded as the first distance.

[0223] Thus, according to the first duration and the speed of light, the first distance can be calculated, and based on the first distance, the position that the lens needs to adjust can be calculated.

[0224] It can be understood that the longer the time for the light to travel from the emission to the return, the farther the distance from the terminal device to the photographed object. Then, there is a proportional relationship between the time for the light to travel from the emission to the return and the distance from the terminal device to the photographed object. According to the distance calculation formula, the relationship between the time t for the light to travel from the emission to the return and the distance from the terminal device to the photographed object can satisfy the following formula (1):

[0225] d = c * t / 2 Formula (1)

[0226] Wherein, d is the distance between the terminal device and the photographed object, c is the speed of light, and t is the total time for the light to travel from the emission to the return.

[0227] It should be noted that the above introduction in combination with Formula (1) is one implementation of calculating the distance from the terminal device to the photographed object. The specific calculation method for calculating the distance from the terminal device to the photographed object in this embodiment is not limited, as long as there is a proportional relationship between the time for the light to travel from the emission to the return and the distance from the terminal device to the photographed object.

[0228] It should be noted that when the terminal device photographs an object, the terminal device may have a certain tilt angle, and the tilt angle of the terminal device may cause a certain gap between the determined first distance and the actual distance between the terminal device and the photographed object. Therefore, in one implementation, for example, the determined first distance can also be corrected according to the tilt angle corresponding to the terminal device, so as to improve the accuracy of the determined object distance between the terminal device and the photographed object. Exemplarily, trigonometric function relationships can be calculated according to the tilt angle to achieve the correction of the distance error caused by the tilt angle. And, the tilt angle of the terminal device can be determined, for example, according to the data collected by the gyroscope sensor.

[0229] S807. Adjust the lens position according to the first distance and the focal length of the lens in the terminal device.

[0230] During the image imaging process, in order to achieve clear imaging, the relationships among the object distance, image distance, and focal length are involved. Among them, the object distance refers to the distance from the shooting object to the lens, which can also be understood as the first distance calculated by S806. The image distance refers to the distance from the image to the lens, which can also be understood as the parameter adjusted by the motor for focusing. The focal length is an inherent property of the lens and is usually a fixed value. At the same time, it can be understood that according to the principle of optical imaging, when the object distance, image distance, and focal length satisfy the following formula two, accurate focusing can be achieved:

[0231] 1 / f = 1 / u + 1 / v Formula Two

[0232] Among them, f represents the focal length of the lens, u is the object distance, and v is the image distance.

[0233] It can be understood that according to the basic principle of optical imaging, when the object distance is different, in order to make the object clearly imaged on the imaging sensor, the image distance needs to be changed accordingly. At the same time, it can be understood that when the object distance is determined and the focal length is a fixed value, there is a specific image distance that can make the object imaging the clearest. By adjusting the position of the lens to change the image distance, clear focusing can be achieved.

[0234] Therefore, according to the calculated distance between the object and the terminal device, the distance between the lens and the imaging sensor can be calculated, and then the motor can be adjusted to achieve focusing, improving the accuracy of the focusing result.

[0235] S808. Execute the photographing process.

[0236] When the distance between the lens and the imaging sensor is adjusted to the appropriate position, clear focusing can be achieved, and at this time, the photographing process can be executed.

[0237] It can be understood in combination with Figure 10 (d) therein. When the focusing is successful, a preview image 1008 of the captured image will be output. It can be found that the image in the preview image 1008 is clear.

[0238] In summary, when the camera and the flash are combined, the distance measurement function of the TOF device can be replaced. Specifically, when the focusing fails, the flash is turned on, and the first duration from the emission of the light to its return to the sensor is measured. According to the first duration and the speed of light, the first distance can be calculated. Based on the first distance, the distance between the lens and the sensor can be adjusted. Accordingly, the focusing operation is completed and the photographing process is executed. Using the flash to assist in focusing solves the problem of focusing failure in specific scenarios when the terminal device is not equipped with a TOF device.

[0239] In addition, in the embodiments of the present application, after measuring the distance between the shooting object and the terminal device, the power output by the flash can also be adjusted to obtain an appropriate main flash brightness for image acquisition. The embodiments of the present application will be introduced below with reference to specific drawings.Figure 12 Schematic flowchart of taking images in a dark environment provided by an embodiment of the present application Figure 13 Schematic diagram of light intensity distribution provided by an embodiment of the present application Figure 14 Schematic diagram of the attenuation function of the flash brightness provided by an embodiment of the present application Figure 15 Schematic diagram of the scheme timing provided by an embodiment of the present application. The method includes:

[0240] S1201. Start the pre-flash of the flash.

[0241] In S803, when the focusing fails, the pre-flash of the flash is used to measure the first distance. In addition to the above-introduced content, the pre-flash of the flash can also be used to measure the brightness of the surrounding environment. According to the measured brightness of the surrounding environment, the convergence value of the exposure parameters can be determined. It can be understood that the camera of the terminal device can calculate the appropriate flash intensity and exposure parameters of the main flash by receiving the light reflected by the pre-flash. Thus, the pre-flash of the flash can help the camera more accurately determine the required flash amount under complex light conditions.

[0242] S1202. Obtain the first brightness information detected by the sensor.

[0243] In the embodiment of the present application, the sensor refers to a device used to detect changes in AE parameters. During the process of turning on the camera, the sensor detects the brightness change in real time. When the flash works, the light it emits will instantaneously increase the light intensity of the surrounding environment. Exemplarily, after the photosensitive element in the sensor receives the optical signal, it converts it into an electrical signal. By measuring the change in the electrical signal, the current brightness information, that is, the first brightness information, can be obtained. Thus, the sensor detecting the brightness information can help the processor quickly adjust the exposure parameters.

[0244] S1203. Determine the convergence values of AE and AWB according to the first brightness information.

[0245] Regarding AE, controlling convergence means that the camera finds the appropriate exposure setting. When the flash starts the pre-flash, the light intensity emitted by the flash is relatively stable and known. The sensor can measure this stable brightness information, that is, the first brightness mentioned above. Combining the first brightness information, the processor can accurately calculate the parameters required to achieve the appropriate exposure amount, and then determine the convergence value of AE.

[0246] Therefore, the convergence value of AE can be understood as the value of the exposure parameters that makes the automatic exposure system finally reach stability.

[0247] For AWB, control convergence means that the camera quickly adjusts and achieves natural color balance under different light sources. Since the color temperature of the flash is usually fixed, after the sensor obtains the image information containing the flash light, it can analyze the color deviation of the white area in the image according to the known color temperature of the flash, and then determine the parameters required to correct the color balance to obtain the convergence value of AWB.

[0248] Thus, the first luminance information can be used to determine the convergence values of AE and AWB, which can provide stable exposure parameters when taking pictures.

[0249] S1204. Determine the target power of the main flash according to the first distance.

[0250] Currently, the existing flashes on the market have a fixed power, which is prone to overexposure or underexposure. Moreover, the light of the flash is relatively hard, resulting in a rather rigid photo-taking result. It can be understood that when taking pictures with the flash on, the sharpening is relatively severe, lacking a soft feeling, and the visual experience of the taken pictures is not very comfortable. In the embodiments of the present application, the automatic control of the flash power can be realized through the first distance to achieve a better lighting effect. Thus, the automatic control of the flash power can not only ensure that the terminal device supplies the startup of the main flash with optimal energy consumption, but also ensure that the taken images are softer and there will be no overexposure or underexposure.

[0251] Since the intensity of light decreases with the increase of distance, when the power of the flash can ensure the exposure effect and emit light, when it reaches the surface of the photographed object, the intensity of light will be different due to the different distances between the photographed object and the terminal device. It can be understood that when the distance between the photographed object and the terminal device is closer, the intensity of light reaching the surface of the photographed object is stronger; when the distance between the photographed object and the terminal device is farther, the intensity of light reaching the surface of the photographed object is weaker.

[0252] It can be combined Figure 13 for understanding. As Figure 13 shown, assume there is a terminal device 1301. When the flash pre-flash is turned on, it will emit light as shown in the figure, and the light is indicated by 1302 in the figure. According to the inverse square law in optics, the intensity I of light is inversely proportional to the square of the distance d. Then it can be understood that when the photographed object is at the r position, the intensity of light reaching the photographed object is 1; when the photographed object is at the 2r position, the intensity of light reaching the photographed object is 1 / 4; when the photographed object is at the 3r position, the intensity of light reaching the photographed object is 1 / 9.

[0253] Therefore, in order to ensure that when the light of the flash reaches the surface of the subject, the intensity of the light can be maintained to ensure the exposure effect, it is necessary to adjust the initial power of the flash, which can also be understood as the target power. It can also be understood that when the distance is closer, in order to ensure that the light intensity is the same when the light reaches the surface of the subject to be photographed, then the target power of the flash needs to be smaller; when the distance is farther, the target power of the flash needs to be larger to ensure that the light intensity remains unchanged when it reaches the surface of the subject to be photographed. Thus, regardless of the distance between the subject and the terminal device, when the light reaches the surface of the subject, the light intensity is the same, then a better exposure effect can be ensured, and furthermore, the captured image can be made softer.

[0254] Based on the above content, it can be understood that the target power of the flash is in a proportional relationship with the first distance, and the required light intensity is determined. It can also be understood that the required power of the flash is determined. Then, the relationship between the target power of the flash and the required power can, for example, satisfy the following formula three:

[0255] P = P0 / d 2 Formula Three

[0256] Among them, d represents the first distance, the target power of the flash is P0, and the required power of the flash is P.

[0257] It should be noted that the above-provided calculation of the target power of the flash is a possible implementation method, but there can also be other functional forms to describe the relationship between the target power of the flash and the first distance, as long as it only needs to satisfy the proportional relationship between the target power of the flash and the first distance. Therefore, the functional expression of the target power of the flash and the first distance can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0258] S1205. Predict the second brightness information of the main flash according to the target power.

[0259] The main flash is the flash operation performed during the actual shooting process. The actual power of the main flash is adjusted based on the target power in S1204 to ensure accurate exposure of the subject. The brightness of the main flash is proportional to the output power. The greater the output power, the brighter the light emitted by the flash. Therefore, the brightness information of the main flash, that is, the second brightness information, can be predicted through the target power of the main flash of the flash.

[0260] Exemplarily, for example, a mapping relationship between power and brightness information can be pre-constructed in advance, so as to predict the second brightness information of the main flash according to the target power. Or, a prediction model can also be pre-trained to make a prediction based on the target power according to the prediction model, so as to predict the second brightness information of the main flash.

[0261] S1206. Predict the target values of AE and AWB based on the second brightness information.

[0262] When shooting with a flash, the combination of ambient light and flash light affects the final exposure and white balance settings. Therefore, the camera needs to estimate the impact of the flash brightness on the scene before shooting. After estimating the flash impact, the camera adjusts the 3A settings to adapt to the new light conditions. It can be understood that the target values of AE and AWB under the estimated main flash brightness are predicted to prepare for the conditions of shooting the final image. Thus, by predicting the target values of AE and AWB under the estimated main flash brightness and adjusting the exposure parameters and white balance settings, the quality of the images taken with the flash can be improved.

[0263] Exemplarily, for example, a mapping relationship between brightness information and AE, AWB can be pre - constructed, so as to predict the target values of AE and AWB corresponding to the main flash according to the predicted second brightness information. Or, a prediction model can be pre - trained to predict the target values of AE and AWB corresponding to the main flash according to the prediction model. Or, based on the convergence values of AE and AWB determined by the pre - flash, by comparing the changes between the first brightness information and the second brightness information, the target values of AE and AWB corresponding to the main flash can be predicted.

[0264] S1207. Start the main flash of the flash according to the target power.

[0265] After the conditions for shooting the image are ready, the pre - flash of the flash goes out and the main flash of the flash turns on. According to different first distances, the target power of the flash is different, so the power for starting the main flash of the flash is different. Thus, according to the first distance, the target power of the flash is determined, and by adjusting the working power of the flash appropriately according to the target power, the loss of the terminal device can be saved, and the images taken can be made softer, improving the shooting quality.

[0266] S1208. When the brightness of the main flash reaches the preset peak value, adjust the convergence values of AE and AWB to the target values.

[0267] The peak can be understood as the highest value of brightness. The duration of the flash refers to the time interval from when the flash is lit to when it goes out, and this time is usually between a few milliseconds and a few microseconds. As time changes, the brightness of the flash reaches the peak and then decays until it goes out.

[0268] It can be combined with Figure 14 to understand the effect of the flash reaching the peak. As shown in Figure 14 , assuming there is a coordinate system with the abscissa being time T and the ordinate being flash brightness L, the curve 1401 represents the decay function of the flash. As time goes by, the brightness of the flash rises, and then, after reaching the highest point, it starts to decline. From Figure 14As can be seen, when the time is t1, the brightness of the flash is the maximum, which is l3, and can be understood as the peak value. After that, as time goes by, the brightness becomes smaller and smaller until it goes out.

[0269] During the shooting process, after the camera waits for the brightness of the main flash to reach an ideal peak value, according to this brightness condition, it adjusts the convergence values of AE and AWB to the target values to optimize the shooting effect. Thus, issuing the target values after the main flash brightness reaches the peak can ensure that when the exposure conditions meet the requirements, both the exposure and color balance of the captured image are optimal.

[0270] S1209. Capture an image.

[0271] After the target values of AE and AWB are issued, when the flash reaches the peak, the camera captures an image. Thus, the captured image can be properly exposed, have balanced colors, and be clear and of high quality.

[0272] S1210. The main flash of the flash is turned off, and the first image is output.

[0273] After the image is captured, the flash follows the change law of the brightness attenuation function and finally turns off. After a series of processing on the captured image, it can include noise reduction, etc., and finally the imaged image is output, which can be understood as the first image. Thus, turning off the main flash can reduce the loss of the terminal device, and outputting the processed image can meet the needs of users.

[0274] In summary, the combination of pre-flash and main flash used by the flash can significantly improve the effect of flash photography, especially in an environment with changing light or complex reflection conditions. In this way, the terminal device can automatically adjust the target power of the flash, reduce the risk of overexposure or underexposure, and thus obtain a more natural and balanced image. At the same time, taking a picture when the main flash brightness reaches the peak can ensure that the exposure amount of the image meets the requirements.

[0275] Figure 15 Integrating the above two embodiments, according to the timing relationship, the operations completed when the pre-flash is turned on and the operations completed when the main flash is turned on can be obtained.

[0276] Such as Figure 15As shown, at time t1, the terminal device detects a first operation; in response to the first operation, focus is performed based on the PDAF technology. When the focusing fails, a pre-flash of the flash light is started at time t2; after the pre-flash of the flash light is turned on, at time t3, a first distance is determined based on the pre-flash; after obtaining the first distance, at time t4, the target power of the main flash is determined based on the first distance, and the target values of AE / AWB / AF under the estimated main flash brightness are also determined. It can also be understood that the target value of AF under the main flash brightness corresponds to the adjusted lens position introduced in the above embodiments, and the target value of AF is a parameter value used to indicate focusing; after the estimation operation is completed, the pre-flash is extinguished at time t5; then, at time t6, the main flash is turned on based on the target power; at time t7, when the brightness of the main flash reaches the peak, the target values of AE / AWB / AF are sent down, where the target value of AF is used to indicate adjusting the lens position; at time t8, the terminal device captures an image; finally, at time t9, the main flash is turned off.

[0277] Based on the above introduction, the present application can also dynamically adjust the flashing frequency of the flash light according to the speed of the shooting object, and then obtain a stroboscopic image showing the movement trajectory. The embodiments of the present application will be introduced below with reference to specific drawings. Figure 16 It is a schematic diagram of the process of stroboscopic photography provided by an embodiment of the present application. Figure 17 It is a schematic diagram of turning on the stroboscopic mode provided by an embodiment of the present application. Figure 18 It is a schematic diagram of the viewfinder provided by an embodiment of the present application. Figure 19 It is a schematic diagram of the wide-angle camera capturing an image provided by an embodiment of the present application. Figure 20 It is a schematic diagram of the marked image at different times provided by an embodiment of the present application. Figure 21 It is a schematic diagram of the normal camera capturing an image provided by an embodiment of the present application. Figure 22 It is a schematic diagram of extracting the area containing the shooting object provided by an embodiment of the present application. Figure 23 It is a schematic diagram of image fusion processing provided by an embodiment of the present application. Figure 24 It is a schematic diagram of real-time rendering of an image provided by an embodiment of the present application. Figure 25 It is a schematic diagram of the scene of stopping the flash provided by an embodiment of the present application.

[0278] S1601. Start the stroboscopic photography mode.

[0279] Stroboscopic photography refers to using a lamp that can flash multiple times at extremely short time intervals to emit periodic flashes, and performing multiple exposures on a moving object, so as to record the positions and postures of the object at different times on one image. In the embodiments of the present application, the flash light of the terminal device is used to achieve multiple flashes at extremely short time intervals, and then multiple exposures are performed on the shooting object, so that the positions and postures of the shooting object at different times can be obtained.

[0280] Exemplarily, for the movement process of an athlete, a stroboscopic photography mode can be adopted to obtain the states of the athlete at different times, and present the movement states at different times on the same picture, which can be used to analyze whether the movements of the athlete at different times are standard, so as to better guide the athlete's movement. Therefore, it is very necessary to use the stroboscopic photography mode.

[0281] Thus, in the embodiment of the present application, by starting the stroboscopic photography mode to obtain the states of the shooting object at different times, the visual effect of the image can be improved. At the same time, the user can use the image output by the stroboscopic photography to analyze the movement state of the shooting object, thereby improving the quality of production or life.

[0282] Next, in conjunction with Figure 17 an exemplary introduction to the method of starting the stroboscopic photography will be given.

[0283] As Figure 17 shown, it is assumed that the terminal device 1701 can display an interface as shown in (a) in Figure 17 when detecting that the user opens the camera application. The interface may include: a shooting control, a thumbnail, a preview screen 1702, a control for switching the camera, a shooting mode selection item, etc. The shooting mode selection item includes but is not limited to: portrait, photo, video, professional or more controls 1703.

[0284] In response to the user's triggering operation on the more controls 1703, the terminal device displays an interface as shown in (b) in Figure 17 The interface may include: a shooting mode selection item. The shooting mode selection item includes but is not limited to: professional, panoramic, HDR control, time-lapse photography, watermark, stroboscopic photography, high pixel, micro movie or other types of shooting mode selection items.

[0285] It can be understood that the user's triggering operation 1704 on the stroboscopic photography control can start the stroboscopic photography mode.

[0286] It should be noted that the above-described method of starting the stroboscopic photography mode is by clicking on the relevant controls after opening the camera application. In another implementation manner, the stroboscopic photography mode can be used as a desktop component and directly clicked to start. In the actual operation of opening the stroboscopic photography mode, the method of opening the stroboscopic photography mode can be determined according to actual needs, and the embodiment of the present application does not limit this.

[0287] S1602. Send operation instructions to the wide-angle camera and the main camera.

[0288] A wide-angle camera is a camera device that can capture images with a wider viewing angle, and its lens has a shorter focal length. It can be understood that according to the optical imaging formula, the shorter the focal length, the wider the imaging viewing angle. Then it can also be understood that a wider scene can be captured using a wide-angle camera. However, due to optical characteristics, there is usually a certain degree of distortion in the images captured by wide-angle cameras. Therefore, after activating the stroboscopic photography mode, a wider viewing angle can be captured using a wide-angle camera, and the movement trajectory of the subject can be better presented.

[0289] The main camera is the default camera used in most daily shooting scenarios, and its lens has a longer focal length. It can be understood that compared with the wide-angle camera, the scene captured by the main camera is relatively narrower. However, the main camera generally has better focusing ability and can accurately focus on the subject. Therefore, after activating the stroboscopic photography mode, more details of the subject can be better captured using a normal camera.

[0290] Therefore, by using the combination of a wide-angle camera and a main camera, it can be ensured that the output image has more details even at a wider viewing angle, and the movement state of the subject can be better presented.

[0291] The following Figure 18 gives an exemplary introduction to activating the wide-angle camera and the main camera.

[0292] As Figure 18 shown in (a) of , assume there is a shooting scene that includes: a ball 1801 in a moving state and a white cloud 1802.

[0293] In response to the user's trigger operation for stroboscopic photography, the terminal device will activate the wide-angle camera and the main camera.

[0294] When the terminal device activates the wide-angle camera, the preview screen of the terminal device shows an interface as shown in Figure 18 (b) of , and this interface includes: a prompt message 1803 for the optical zoom of the wide-angle camera and a preview screen 1804 of the wide-angle camera.

[0295] Among them, the prompt message 1803 for the optical zoom of the wide-angle camera shows "0.5X", and 0.5X usually indicates that the wide-angle camera is enabled, and the focal length of this lens is 0.5 times that of the main camera. For example, if the main camera focal length is 26mm, then the focal length of the ultra-wide-angle lens is approximately 13mm. According to the optical imaging formula, this means that the wide-angle camera can capture a wider scene. It can be understood in combination with the preview screen 1804 that the complete scene information can be browsed in the current screen.

[0296] When the terminal device activates the main camera, the preview screen of the terminal device shows asFigure 18 The interface shown in (c) therein, which includes: the prompt information 1805 of the original focal length of the main camera and the preview screen 1806 of the main camera.

[0297] Among them, the prompt information 1805 of the original focal length of the main camera shows "1X", and 1X usually indicates that the main camera is enabled, and the focal length of this lens is 1 times the focal length of the main camera. If the focal length of the main camera is 26mm, when in the 1X state, the actual focal length of the lens is 26mm, and the shooting angle of view is the standard angle of view designed by the main camera itself, and the presented picture range is more in line with the normal field of view of the human eye. It can be understood in combination with the preview screen 1806 that in the current picture, some scene information can be browsed, but the scene information is clearer.

[0298] In the embodiment of the present application, the prompt information 1803 of the optical zoom of the wide-angle camera and the prompt information 1805 of the original focal length of the main camera are only popped up when the stroboscopic photography mode is initially turned on, and no further prompts are popped up in the subsequent processing.

[0299] It should be noted that the shooting object in this embodiment is a ball, and the ball is in a moving state, which can also be understood as recording the moving state of the ball, and other background contents are regarded as static states, such as the white cloud 1802. At the same time, it should be noted that in practical applications, other moving objects can be selected as the shooting object, and the embodiment of the present application does not limit this.

[0300] S1603. When the shooting object enters the field of view range of the first camera, collect a marked image through the wide-angle camera.

[0301] After the operation instruction of the wide-angle camera is started, the wide-angle camera needs to collect a marked image. Among them, the marked image can be understood as the image collected by the wide-angle camera when the shooting object enters the field of view range of the wide-angle camera. Since the wide-angle camera has a wider field of view range, therefore, using the wide-angle camera to collect images can capture more moving pictures of the shooting object, and using more moving pictures can calculate the stroboscopic frequency of the flash more accurately.

[0302] The following combines Figure 19 to give an exemplary introduction to the process of collecting marked images using the wide-angle camera.

[0303] As Figure 19As shown in (a) therein, it is assumed that the terminal device has enabled the stroboscopic photography mode. In this mode, it is assumed that there are the framing situations of the viewfinders shown in Example 1, Example 2, and Example 3. Among them, each example includes: the viewfinder 1901 of the wide-angle camera, the viewfinder 1902 of the main camera, and the subject and background content located within the viewfinder. In the current step, the framing situation of the viewfinder corresponding to the wide-angle camera will be described first.

[0304] In Figure 19 Example 1 shown in (a) therein, when the subject moves to position 1903, it enters the first field of view range of the wide-angle camera. At this time, the wide-angle camera will collect the marked image of the subject during movement, such as Figure 19 the image 1 shown in (b) therein.

[0305] In Figure 19 Example 2 shown in (a) therein, when the subject moves to position 1905, it is within the first field of view range of the wide-angle camera. At this time, the wide-angle camera will collect the marked image of the subject during movement, such as Figure 19 the image 2 shown in (b) therein.

[0306] Among them, in Example 2, position 1904 can be understood as the position where the subject was located at the previous moment of position 1905, which is represented by a dotted line and corresponds to position 1903 in Example 1. Then it can be understood that the subject moves from position 1904 to position 1905.

[0307] In Figure 19 Example 3 shown in (a) therein, when the subject moves to position 1908, it is within the first field of view range of the wide-angle camera. At this time, the wide-angle camera will collect the marked image of the subject during movement, such as Figure 19 the image 3 shown in (b) therein.

[0308] Among them, in Example 3, position 1907 can be understood as the position where the subject was located at the previous moment of position 1908, which is represented by a dotted line and corresponds to position 1905 in Example 2, and position 1906 can be understood as the position where the subject was located at the two previous moments of position 1908, which is represented by a dotted line and corresponds to position 1904 in Example 2, and corresponds to position 1903 in Example 1. Then it can be understood that the subject moves from position 1906 to position 1907, and then to position 1908.

[0309] It should be noted that the above three examples shown are the viewfinders and the collected images corresponding to the continuously moving subject. In actual operation, there is no limit to the number of frames of the collected marked images, but it is necessary to satisfy that the marked images are collected based on the continuously moving subject.

[0310] S1604. Determine the stroboscopic frequency of the flash based on the marked image.

[0311] After multiple frames of marked images are captured using a wide-angle camera, image processing algorithms can be used to process the multiple frames of marked images, and the processed image frames can be used to calculate the stroboscopic frequency of the flash.

[0312] It can be understood that in the stroboscopic photography mode, when the subject enters the field of view of the wide-angle camera and does not enter the field of view of the main camera, the wide-angle camera will operate prior to the main camera. Then, the wide-angle camera can pre-capture multiple frames of marked images, and these marked images record the positions of the subject at different times.

[0313] After that, using image processing methods, such as the optical flow method, the motion speed of the subject can be calculated based on the images captured at different times. According to the motion speed of the subject, the stroboscopic frequency of the flash at the next moment can be determined, and the flash is controlled to flash using the stroboscopic frequency of the flash at the next moment, so as to keep the image captured by the main camera clear.

[0314] It can be understood that during the processing of stroboscopic photography, the flash is used to provide exposure for the subject. Then, in order to enable the stroboscopic frequency of the flash to adapt to the motion of the subject, when the motion speed of the subject is fast, the stroboscopic frequency of the flash needs to be increased, and when the motion speed of the subject is slow, the stroboscopic frequency of the flash needs to be decreased. Then, the stroboscopic frequency of the flash is in a proportional relationship with the motion speed of the subject.

[0315] It should be noted that the relationship formula between the motion speed of the subject and the stroboscopic frequency of the flash can be determined according to specific requirements, but it needs to satisfy that the motion speed of the subject and the stroboscopic frequency of the flash are in a proportional relationship, and the specific form of the relationship formula is not limited in the embodiments of the present application.

[0316] The following combines Figure 20 to give an exemplary introduction to the process of determining the stroboscopic frequency of the flash.

[0317] As Figure 20 shown, assume that there are three frames of marked images captured at adjacent times, which can be Image 1, Image 2, and Image 3. Among them, in Image 1, the subject is at position 2001, in Image 2, the subject is at position 2002, and in Image 3, the subject is at position 2003. It can be understood that when the subject is at adjacent times, it moves from position 2001 to position 2002 and from position 2002 to position 2003.

[0318] Using the optical flow method, the speed of the captured object can be calculated as v1 based on Image 1 and Image 2. Based on v1, the stroboscopic frequency f1 can be obtained, and the stroboscopic frequency at the next moment of the flash can be controlled according to the frequency f1 to achieve appropriate exposure of the image captured by the main camera at the next moment.

[0319] Similarly, according to the above logic, the speed of the captured object calculated based on Image 2 and Image 3 is v2. Based on v2, the stroboscopic frequency f2 can be obtained, and the stroboscopic frequency at the next moment of the flash can be controlled according to the frequency f2 to achieve appropriate exposure of the image captured by the main camera at the next moment.

[0320] It should be noted that the image processing method used in the above embodiments is the optical flow method. In actual applications, other image processing algorithms can also be used, and the embodiments of this application do not limit this.

[0321] S1605. Control the continuous flashing of the flash according to the stroboscopic frequency of the flash.

[0322] According to the above calculation method, the stroboscopic frequency of the flash can be calculated, and the continuous flashing of the flash can be controlled according to the frequency of the flash, which can achieve dynamic adjustment of the flash frequency. Thus, by adjusting different flash frequencies according to the different moving speeds of the captured object, reasonable exposure of the captured object can be achieved, and the quality of the output image can be improved.

[0323] S1606. Collect the original image through the main camera.

[0324] When the captured object enters the main camera, the image captured by the main camera can be understood as the original image. Collecting the original image through the main camera can capture the motion details of the captured object at different moments more clearly. Thus, using the main camera to collect the original image can not only ensure the high definition of the collected image but also ensure the quality of the final output image.

[0325] The following combines Figure 21 to give an exemplary introduction to the process of collecting the original image through the main camera.

[0326] As shown in (a) of Figure 21 , assume that the terminal device has enabled the stroboscopic photography mode. In this mode, assume there are the framing situations of the viewfinder shown in Example 1, Example 2, and Example 3. Among them, each example includes: the viewfinder 2101 of the wide-angle camera, the viewfinder 2102 of the main camera, and the captured object and background content located within the viewfinder. The framing situation of the viewfinder corresponding to the main camera is described in the current step.

[0327] In Figure 21In Example 1 illustrated in (a) below, when the subject moves to position 2103, it enters the first field of view of the main camera. At this time, the main camera will capture the original image of the moving subject, such as Figure 21 the image 1 shown in (b) below.

[0328] In Figure 21 Example 2 illustrated in (a) below, when the subject moves to position 2105, it is within the first field of view of the main camera. At this time, the main camera will capture the original image of the moving subject, such as Figure 21 the image 2 shown in (b) below.

[0329] Among them, in Example 2, position 2104 can be understood as the position where the subject was at the previous moment of position 2105, represented by a dashed line, and corresponding to position 2103 in Example 1. Then it can be understood that the subject moves from position 2104 to position 2105.

[0330] In Figure 21 Example 3 illustrated in (a) below, when the subject moves to position 2108, it is within the first field of view of the main camera. At this time, the main camera will capture the original image of the moving subject, such as Figure 21 the image 3 shown in (b) below.

[0331] Among them, in Example 3, position 2107 can be understood as the position where the subject was at the previous moment of position 2108, represented by a dashed line, and corresponding to position 2105 in Example 2, and position 2106 can be understood as the position where the subject was at the two previous moments of position 2108, represented by a dashed line, and corresponding to position 2104 in Example 2, and corresponding to position 2103 in Example 1. Then it can be understood that the subject moves from position 2106 to position 2107, and then to position 2108.

[0332] It should be noted that the above three illustrated examples are the viewfinder frames and the captured images corresponding to the continuous movement of the subject. In actual operation, there is no limit to the number of frames of the captured original images, but it is necessary to ensure that the original images are captured based on the continuous movement of the subject.

[0333] S1607. Extract the region containing the subject from the original image to generate an intermediate image.

[0334] The intermediate image refers to the image generated after extracting the area containing the subject from the original image. Among them, the area of the subject can be extracted through an image processing algorithm, and operations such as background subtraction and edge detection are performed on the subject to generate an image that only includes the area of the subject. Thus, by extracting the area containing the subject from the original image, unnecessary losses generated during the image fusion operation of the terminal device can be reduced. At the same time, extracting the area containing the subject can reduce the interference of the background content in the original image on image processing.

[0335] The following Figure 22 exemplarily introduces the process of extracting the area containing the subject.

[0336] As Figure 22 shown in (a) of, it is assumed that there are original images 1, 2, and 3 collected by the main camera. The three frames of images may include the subject 2201 and background content 2202 unrelated to the subject.

[0337] As Figure 22 shown in (b) of, after extracting the area containing the subject from image 1, image 4 is generated; after extracting the area containing the subject from image 2, image 5 is generated; after extracting the area containing the subject from image 3, image 6 is generated.

[0338] Among them, images 4, 5, and 6 only contain the subject 2201. At this time, images 4, 5, and 6 can all be understood as intermediate images.

[0339] It should be noted that when performing the operation of extracting the area containing the subject, the image processing algorithm used can be determined according to specific circumstances, and the embodiments of the present application do not limit this.

[0340] S1608. Perform image fusion processing on the intermediate image to generate a real-time rendering image.

[0341] When multiple frames of intermediate images are obtained, in order to clearly present the motion state of the subject at different times, it is necessary to perform image fusion processing on the multiple frames of images. The fused image is a real-time rendering image. Among them, using a black frame to perform fusion processing with the intermediate image or using a gray frame to perform fusion processing with the intermediate image can ensure that the focus of the final image is on the subject and is not interfered by the background content.

[0342] It can be understood that a black frame refers to a completely dark frame in an image sequence, and its pixel values are basically zero. During image fusion, the black frame can be used for background subtraction. A gray frame has a certain gray value, and usually its pixel values are relatively evenly distributed. During the image fusion process, the gray frame can be used for brightness calibration and balancing. Thus, by fusing the black frame with the intermediate image, interference from the background can be avoided. At the same time, by fusing the gray frame with the intermediate image, it can be used for brightness calibration and balancing.

[0343] The following will Figure 23 exemplarily introduce the process of image fusion.

[0344] As Figure 23 shown, assume there are intermediate image 1, intermediate image 2, and intermediate image 3. When performing image fusion processing on these three image frames, image 4 can be inserted between image 1 and image 2, and image 5 can be inserted between image 2 and image 3. By fusing image 1, image 2, image 3, image 4, and image 5, image 6 can be obtained. Then it can be understood that image 6 is a real-time rendering image.

[0345] In one implementation, both image 4 and image 5 are black frames, then image 6 is the fusion result image of the intermediate image and the black frame; in another implementation, both image 4 and image 5 are gray frames, then image 6 is the fusion result image of the intermediate image and the gray frame.

[0346] It should be noted that the number of black frames or gray frames inserted between image frames can be determined according to actual needs, and this application does not make any limitations.

[0347] S1609. In the shooting interface, display the real-time rendering image.

[0348] After performing fusion processing on the intermediate image, the real-time rendering image can be displayed in the shooting interface of the terminal device. Thus, setting the real-time rendering image in the shooting interface can help users view the shooting effect in real time, improving user-friendliness.

[0349] The following will Figure 24 exemplarily introduce the effect of displaying the real-time rendering image.

[0350] As Figure 24 shown, assume the terminal device has enabled the stroboscopic photography mode, and the real-time rendering image 2401 will be displayed in the shooting interface of the terminal device. According to the real-time rendering image, the movement trajectory of the shooting object can be observed in real time.

[0351] S1610. Determine whether the field of view of the main camera contains the shooting object. If so, execute S1603; if not, execute S1611.

[0352] When the subject is not within the field of view of the main camera, the main camera can no longer capture the subject. Therefore, the flash needs to stop flashing. Thus, by determining whether the field of view of the main camera contains the subject, the loss of the terminal device can be saved, and the waste of resources caused by still turning on the flash when the main camera can no longer capture the subject can be avoided.

[0353] The following Figure 25 gives an exemplary introduction to the process of determining whether the field of view of the main camera contains the subject.

[0354] As Figure 25 shown, assume that the terminal device has enabled the stroboscopic photography mode. The shooting interface includes the viewfinder 2501 of the main camera, the viewfinder 2502 of the wide-angle camera, the subject, and the image in the viewfinder.

[0355] As Figure 25 shown in (a) therein, when the subject is at position 2503 and this position is within the field of view of the main camera 2501, a real-time rendered image 2504 is presented, and the wide-angle camera continues to collect the marked image and subsequent operations.

[0356] As Figure 25 shown in (b) therein, when the subject is at position 2505 and this position is not within the field of view of the main camera 2501, a real-time rendered image 2506 is presented, and the flash stops flashing. Among them, since the image capture by the main camera has ended, the real-time rendered image 2506 at this time is the last real-time rendered image.

[0357] S1611. The flash stops flashing and a stroboscopic image is output.

[0358] When the flash stops flashing, the subject will no longer be exposed, and there is no need to collect images anymore. At this time, the real-time rendered image when the flash is turned off can be understood as a stroboscopic image. Thus, after turning off the flash, outputting the stroboscopic image instead of collecting images can reduce unnecessary operations and improve the agility of the solution.

[0359] As Figure 25 shown in (b) therein, the real-time rendered image 2506 can be understood as the finally output stroboscopic image.

[0360] In summary, in this embodiment, the real-time motion trajectory of the object to be photographed is obtained by starting the stroboscopic photography mode. The marker image is collected by the wide-angle camera. On the one hand, a wider scene can be photographed, and on the other hand, the stroboscopic frequency at the next moment under the flash can be calculated using the collected marker image to ensure that the original image collected by the main camera is clear; the original image is collected by the main camera, and more accurate focusing on the object to be photographed can be achieved. It can be understood that when photographing the object to be photographed, the flash can be used for auxiliary ranging to obtain the first distance. According to the first distance, the target power of the main flash of the flash is calculated. According to the target power of the flash and the stroboscopic frequency of the flash calculated from the marker image, when the main flash brightness reaches the peak, the collection of the original image is completed. By using the combination of the main camera and the wide-angle camera, the motion states of the object to be photographed at different moments can be captured, and the motion states at different moments are presented in the same image, which can improve the visual effect of the image. Moreover, in the shooting interface, a real-time rendered image is provided, which improves the user-friendliness of the solution of this application.

[0361] Based on the content introduced in the above embodiment, the following further describes the focusing processing method provided by this application in conjunction with Figure 26 the following. Figure 26 FIG. is a schematic flowchart of the focusing processing method provided by an embodiment of this application.

[0362] S2601. In response to a first operation that triggers shooting, control the flash of the terminal device to perform a first lighting operation.

[0363] In this embodiment, the first operation is used to trigger shooting. For example, it can be the operation of the user clicking the shutter introduced in the above embodiment.

[0364] Exemplarily, the specific implementation of responding to the first operation that triggers shooting can be referred to the introduction of S801 in the above embodiment, and details are not described herein again.

[0365] In this embodiment, the first lighting operation is used to obtain the distance between the object to be photographed and the terminal device. For example, it can be the pre-flash of starting the flash introduced in the above embodiment.

[0366] Exemplarily, the specific implementation of controlling the flash of the terminal device to perform the first lighting operation can be referred to the introduction of S803 in the above embodiment, and details are not described herein again.

[0367] S2602. Obtain a first distance based on the first lighting operation of the flash, where the first distance is the distance between the object to be photographed and the terminal device.

[0368] In one implementation, obtaining the first distance based on the first lighting operation of the flash can be performed in the following manner, for example:

[0369] First, based on the first lighting operation of the flash, obtain the first moment when the flash is lit; and,

[0370] Obtain the second moment when the light of the flash returns to the terminal device, where the light of the flash is reflected by the subject and then returns to the terminal device;

[0371] Among them, to obtain the second moment when the light of the flash returns to the terminal device, for example, after the first moment, based on the first sensor for detecting exposure in the terminal device, detect whether the exposure parameter changes. When the exposure parameter changes, obtain the second moment, and the second moment is the moment when the exposure parameter change is detected.

[0372] Exemplarily, for the specific implementation of obtaining the second moment when the light of the flash returns to the terminal device, reference can be made to the introduction of S804 in the above embodiment, which will not be elaborated here.

[0373] Secondly, determine the first duration according to the first moment and the second moment.

[0374] In the embodiment of the present application, the first duration is the time difference between the first moment and the second moment, and for example, it can be the time difference between the light emitted by the pre-flash of the flash from emission to return to the terminal device introduced in the above embodiment. Exemplarily, for the specific implementation of the current step, reference can be made to the introduction of S805 in the above embodiment, which will not be elaborated here.

[0375] Finally, obtain the first distance according to the first duration and the speed corresponding to the light.

[0376] Exemplarily, for the specific implementation of the current step, reference can be made to the introduction of S806 in the above embodiment, which will not be elaborated here.

[0377] After performing the first lighting operation, it is also necessary to obtain the first brightness information, and the first brightness information is the brightness information detected by a preset sensor in the terminal device.

[0378] Exemplarily, for the specific implementation of the current step, reference can be made to the introduction of S1202 in the above embodiment, which will not be elaborated here.

[0379] S2603. Adjust the shooting parameters according to the first distance, and the shooting parameters include the lens position.

[0380] In one implementation, to adjust the shooting parameters according to the first distance, for example, the following method can be adopted:

[0381] First, determine the target value of the shooting parameters according to the first distance.

[0382] Among them, in the process of determining the target value of the shooting parameters, for example, the target value of the lens position can be determined according to the first distance and the focal length of the lens in the terminal device.

[0383] Exemplarily, the specific implementation of the current step can refer to the introduction of S807 in the above embodiment, which will not be elaborated here.

[0384] Determine the convergence value of the exposure parameter and the convergence value of the white balance parameter according to the first brightness information.

[0385] Exemplarily, the specific implementation of the current step can refer to the introduction of S1203 in the above embodiment, which will not be elaborated here.

[0386] Determine the target power corresponding to the second lighting operation according to the first distance, and the first distance is in a proportional relationship with the target power.

[0387] In this embodiment, the second lighting operation is used for exposure when taking pictures, and can be, for example, the main flash of starting the flash lamp introduced in the above embodiment.

[0388] Exemplarily, the specific implementation of the current step can refer to the introduction of S1204 in the above embodiment, which will not be elaborated here.

[0389] Predict the second brightness information of the flash lamp when performing the second lighting operation according to the target power.

[0390] Exemplarily, the specific implementation of the current step can refer to the introduction of S1205 in the above embodiment, which will not be elaborated here.

[0391] Adjust the convergence value of the exposure parameter and the convergence value of the white balance parameter based on the second brightness information to obtain the target value of the exposure parameter and the target value of the white balance parameter.

[0392] Exemplarily, the specific implementation of the current step can refer to the introduction of S1206 in the above embodiment, which will not be elaborated here.

[0393] Secondly, control the flash lamp to perform the second lighting operation, and during the process of performing the second lighting operation, when it is determined that the brightness of the flash lamp reaches the preset peak value, adjust the shooting parameters based on the target value of the shooting parameters. Among them, control the flash lamp to perform the second lighting operation according to the target power.

[0394] Exemplarily, the specific implementation of controlling the flash lamp to perform the second lighting operation can refer to the introduction of S1207 in the above embodiment, which will not be elaborated here. The specific implementation of adjusting the shooting parameters based on the target value of the shooting parameters can refer to the introduction of S1208 in the above embodiment, which will not be elaborated here.

[0395] S2604. Perform image acquisition based on the adjusted shooting parameters to obtain a first image.

[0396] In one implementation, for image acquisition based on the adjusted shooting parameters, for example, the specific implementation of the current step can refer to the introduction of S1209 in the above embodiments and will not be elaborated here.

[0397] After the acquired image is obtained, the main flash of the flash is turned off, and the first image is output. For example, the specific implementation of the current step can refer to the introduction of S1210 in the above embodiments and will not be elaborated here.

[0398] In this embodiment, a focusing processing method is provided, which can solve the focusing problem in a dark environment at low cost. It can be understood that by using the flash to emit light, record the first moment when the flash is started and the second moment when the AE change is detected, calculate the first duration, calculate the first distance between the terminal device and the shooting object according to the first duration and the speed of light, and adjust the lens focus according to the first distance to obtain a first image. The embodiment of the present application utilizes the existing component of the flash to solve the focusing problem in a dark environment and reduces the cost of solving the problem.

[0399] At the same time, using the first distance, dynamically adjust the target power of the main flash of the flash, which can ensure that the obtained image has appropriate exposure, neither overexposed nor underexposed. Moreover, starting the flash according to the target power can avoid unnecessary loss to the terminal device. When the main flash brightness reaches the peak value, adjust the parameters to the target value, which can ensure that when the exposure conditions meet the requirements, the parameters of the image are also optimal.

[0400] Based on the content introduced in the above embodiments, the following will be further described in combination with Figure 27 the method for obtaining a stroboscopic image provided by the present application. Figure 27 It is a schematic flowchart of the stroboscopic photography method provided by the embodiment of the present application.

[0401] S2701. When the shooting mode triggered by the first operation is the stroboscopic mode, send start instructions to the first camera and the second camera of the terminal device respectively. The field of view angle of the first camera is greater than that of the second camera.

[0402] For example, the specific implementation of the first operation triggering the stroboscopic mode can refer to the introduction of S1601 in the above embodiments and will not be elaborated here.

[0403] In the embodiment of the present application, the first camera can be, for example, the wide-angle camera introduced in the above embodiments, and the second camera can be, for example, the main camera introduced in the above embodiments.

[0404] The specific implementation of sending start commands to the first camera and the second camera of the terminal device can refer to the introduction of S1602 in the above embodiments, which will not be elaborated here.

[0405] S2702. When the object to be photographed enters the field of view of the first camera, collect multiple marker images in sequence based on the first camera, and dynamically determine the flash frequency of the flash during the process of collecting the marker images.

[0406] In one implementation, the flash frequency of the flash is dynamically determined by the process of collecting the marker images. For example, the following method can be adopted:

[0407] First, during the process of collecting the marker images, determine the real-time moving speed of the object to be photographed according to the first marker image and the second marker image. The first marker image is the original image closest to the collection time, and the second marker image is the marker image adjacent to the first marker image.

[0408] In the embodiments of the present application, the marker image can be, for example, the image collected by the wide-angle camera introduced in the above embodiments, and the original image can be, for example, the image collected by the main camera introduced in the above embodiments.

[0409] Exemplarily, the specific implementation of collecting the marker images can refer to the introduction of S1603 in the above embodiments, which will not be elaborated here.

[0410] Second, dynamically determine the flash frequency of the flash according to the real-time moving speed of the object to be photographed. The flash frequency is in a proportional relationship with the real-time moving speed.

[0411] Exemplarily, the specific implementation of determining the flash frequency of the flash can refer to the introduction of S1604 in the above embodiments, which will not be elaborated here.

[0412] S2703. When the object to be photographed enters the field of view of the second camera, control the flash to continuously flash based on the dynamically determined flash frequency, and collect multiple original images in sequence based on the second camera during the process of the flash continuously flashing.

[0413] Exemplarily, the specific implementation of controlling the flash to continuously flash based on the dynamically determined flash frequency can refer to the introduction of S1605 in the above embodiments, which will not be elaborated here. The specific implementation of collecting multiple original images in sequence based on the second camera can refer to the introduction of S1606 in the above embodiments, which will not be elaborated here.

[0414] S2704. Generate a stroboscopic image according to the multiple original images.

[0415] In one implementation manner, generating the stroboscopic image can, for example, adopt the following method:

[0416] First, for any one of the multiple original images, object extraction is performed on the photographed object included in the original image to generate an intermediate image including the photographed object.

[0417] Exemplarily, the specific implementation of performing object extraction on the photographed object included in the original image may refer to the introduction of S1607 in the above embodiment, which will not be elaborated here.

[0418] Secondly, fusion processing is performed according to the intermediate images respectively corresponding to the multiple original images to obtain a real-time rendering image.

[0419] Exemplarily, the specific implementation of performing fusion processing on multiple intermediate images to obtain a real-time rendering image may refer to the introduction of S1608 in the above embodiment, which will not be elaborated here.

[0420] Then, the real-time rendering image is displayed in the first area of the shooting interface, and the real-time rendering image is an intermediate result image generated during the process of sequentially performing fusion processing on multiple intermediate images.

[0421] Exemplarily, the specific implementation of displaying the real-time rendering image in the first area of the shooting interface may refer to the introduction of S1609 in the above embodiment, which will not be elaborated here.

[0422] Finally, when the duration range of the second camera does not include the photographed object, the flash is controlled to stop flashing to obtain a stroboscopic image.

[0423] Exemplarily, the specific implementation of determining whether the duration range of the second camera includes the photographed object may refer to the introduction of S1610 in the above embodiment, which will not be elaborated here.

[0424] When it is determined that the duration range of the second camera does not include the photographed object, the flash stops flashing and a stroboscopic image is output. The specific implementation of the current step may refer to the introduction of S1611 in the above embodiment, which will not be elaborated here.

[0425] In this embodiment, when the user selects the stroboscopic photography mode, motion state diagrams of the photographed object at different times can be provided, improving the visual effect of the image. Moreover, a real-time rendering image is displayed on the shooting interface, improving the user-friendliness of the method of the present application. It can be understood that a wide-angle camera is used to obtain a wider field of view, and the main camera is used to obtain higher-definition details. Combining the wide-angle camera with the main camera can ensure that the output stroboscopic image has both details and comprehensively shows the motion trajectory of the photographed object.

[0426] It should be noted that the module names involved in the embodiments of the present application can all be defined as other names, as long as the functions of each module can be realized, and no specific restrictions are imposed on the module names.

[0427] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0428] The focus processing method of the embodiments of the present application has been described above. Next, the device for executing the above method provided by the embodiments of the present application will be described. Those skilled in the art can understand that the method and the device can be combined and cited with each other, and the relevant device provided by the embodiments of the present application can execute the steps in the above focus processing method.

[0429] The focus processing method provided by the embodiments of the present application can be applied to an electronic device with a shooting function. The electronic device includes a terminal device. The specific device form of the terminal device and the like can refer to the above relevant description and will not be elaborated here.

[0430] In one implementation, the embodiments of the present application provide an electronic device Figure 28 is a schematic diagram of the hardware structure of the electronic device provided by the embodiments of the present application.

[0431] As Figure 28 shown, the electronic device 280 includes: a processor 2801 and a memory 2802; the memory 2802 stores computer-executable instructions; the processor 2801 executes the computer-executable instructions stored in the memory 2802, so that the electronic device 280 executes the above method.

[0432] When the memory 2802 is independently provided, the electronic device further includes a bus 2803 for connecting the memory 2802 and the processor 2801.

[0433] The embodiments of the present application provide a chip. The chip includes a processor, and the processor is used to call a computer program in the memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those of the above relevant embodiments and will not be elaborated here.

[0434] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the above method is implemented. The method described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over a computer-readable medium as one or more instructions or code. The computer-readable medium can include a computer storage medium and a communication medium, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0435] In a possible implementation, the computer-readable medium may include RAM, ROM, a compact disc read-only memory (CD-ROM), or other optical disc storage, magnetic disk storage, or other magnetic storage devices, or any other medium targeted to carry or store the required program code in the form of instructions or data structures and accessible by a computer. Moreover, any connection is properly termed a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technology such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology such as infrared, radio, and microwave is included in the definition of the medium. As used herein, disk and disc include optical disc, laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically using lasers. The above combinations should also be included within the scope of the computer-readable medium.

[0436] The embodiments of the present application provide a computer program product. The computer program product includes a computer program. When the computer program is run, the computer is caused to execute the above method.

[0437] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable devices to generate a machine, such that the instructions executed by the processing unit of the computer or other programmable data processing device generate means for implementing the processes Figure 1One process or multiple processes and / or boxes Figure 1 Apparatus for functions specified in one box or multiple boxes.

[0438] In the above specific embodiments, the objectives, technical solutions and beneficial effects of the present invention have been further described in detail. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present invention shall be included in the protection scope of the present invention.

Claims

1. A focusing processing method, characterized in that, Applied to a terminal device, including: In response to a first operation that triggers shooting, controlling a flash of the terminal device to perform a first lighting operation; Obtaining a first distance based on the first lighting operation of the flash, where the first distance is the distance between the shooting object and the terminal device; Adjusting shooting parameters according to the first distance, where the shooting parameters include the lens position; Performing image acquisition based on the adjusted shooting parameters to obtain a first image; When the shooting mode triggered by the first operation is a stroboscopic mode, sending start instructions to a first camera and a second camera of the terminal device respectively, where the field of view angle of the first camera is greater than that of the second camera; When the shooting object enters the field of view range of the first camera, sequentially collecting multiple marked images based on the first camera, and dynamically determining the flash frequency of the flash during the process of collecting the marked images; When the shooting object enters the field of view range of the second camera, controlling the flash to continuously flash based on the dynamically determined flash frequency, and sequentially collecting multiple original images based on the second camera during the process of the flash continuously flashing; Generating a stroboscopic image according to the multiple original images.

2. The method according to claim 1, wherein The obtaining the first distance based on the first lighting operation of the flash includes: Based on the first lighting operation of the flash, obtaining a first moment when the flash is lit; and, Obtaining a second moment when the light of the flash returns to the terminal device, where the light of the flash is reflected by the shooting object and then returns to the terminal device; Determining a first duration according to the first moment and the second moment; Obtaining the first distance according to the first duration and the speed corresponding to the light.

3. The method according to claim 2, wherein The obtaining the second moment when the light of the flash returns to the terminal device includes: After the first moment, based on a first sensor for detecting exposure in the terminal device, detecting whether the exposure parameters change; When the exposure parameters change, obtaining the second moment, where the second moment is the moment when the exposure parameters are detected to change.

4. The method according to any one of claims 1 to 3, characterized in that The adjusting the shooting parameters according to the first distance includes: Determining a target value of the shooting parameters according to the first distance; Controlling the flash to perform a second lighting operation, and adjusting the shooting parameters based on the target value of the shooting parameters during the process of performing the second lighting operation; Performing image acquisition based on the adjusted shooting parameters to obtain a first image.

5. The method according to claim 4, wherein After performing the first lighting operation, the method further includes: Obtaining first brightness information, where the first brightness information is the brightness information detected by a preset sensor in the terminal device.

6. The method according to claim 5, wherein The shooting parameters further include exposure parameters and white balance parameters; The determining the target value of the shooting parameters according to the first distance includes: Determining a target value of the lens position according to the first distance and the focal length of the lens in the terminal device; Determining a convergence value of the exposure parameters and a convergence value of the white balance parameters according to the first brightness information; Determine the target power corresponding to the second lighting operation according to the first distance, where the first distance is in a direct proportional relationship with the target power; Predict the second brightness information of the flash when performing the second lighting operation according to the target power, and adjust the convergence value of the exposure parameter and the convergence value of the white balance parameter based on the second brightness information to obtain the target value of the exposure parameter and the target value of the white balance parameter.

7. The method according to claim 6, wherein The controlling the flash to perform the second lighting operation includes: Control the flash to perform the second lighting operation according to the target power.

8. The method according to claim 7, wherein The adjusting the shooting parameters based on the target values of the shooting parameters during the execution of the second lighting operation includes: During the execution of the second lighting operation, when it is determined that the brightness of the flash reaches a preset peak, adjust each of the shooting parameters to its corresponding target value.

9. The method according to claim 1, wherein The dynamically determining the flash frequency of the flash during the acquisition of the marked image includes: During the acquisition of the marked image, determine the real-time moving speed of the shooting object according to the first marked image and the second marked image, where the first marked image is the original image closest to the acquisition moment, and the second marked image is the marked image adjacent to the first marked image; Dynamically determine the flash frequency of the flash according to the real-time moving speed of the shooting object, where the flash frequency is in a direct proportional relationship with the real-time moving speed.

10. The method according to claim 1, wherein The method further includes: When the duration range of the second camera does not include the shooting object, control the flash to stop flashing.

11. The method according to claim 1, wherein The generating the stroboscopic image according to the multiple original images includes: For any one of the multiple original images, extract the object in the original image to generate an intermediate image including the shooting object; Perform a fusion process on the intermediate images corresponding to the multiple original images respectively to obtain a stroboscopic image.

12. The method according to claim 11, wherein The method further includes: Display a real-time rendered image in a first area of the shooting interface, where the real-time rendered image is an intermediate result image generated during the process of sequentially performing a fusion process on the intermediate images corresponding to the multiple original images.

13. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the electronic device to execute the method according to any one of claims 1 to 12.

14. A chip system, characterized in that, The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions to cause the electronic device to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions, and when the computer instructions run on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes computer program code which, when running on an electronic device, causes the electronic device to execute the method according to any one of claims 1 to 12.

Images