Focusing processing method and electronic equipment

Abstract

The embodiment of the invention provides a focusing processing method and electronic equipment, and relates to the technical field of terminals. The method comprises the following steps: in response to a first operation for triggering shooting, controlling a flash lamp of the terminal equipment to execute a first lightening operation. And obtaining a first distance based on the first lightening operation of the flash lamp, wherein the first distance is the distance between the shooting object and the terminal device. And adjusting shooting parameters according to the first distance, wherein the shooting parameters comprise the lens position. And performing image acquisition based on the adjusted shooting parameters to obtain a first image. Therefore, under the condition that the terminal equipment is not equipped with a TOF device, the focusing problem in a specific scene can be solved by combining the PDAF technology and the flash lamp.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a focusing processing method and an electronic device. Background Art

[0002] With the development of focusing technology, PDAF technology came into being. Applying PDAF technology on smartphones can achieve fast and accurate focusing. However, in certain scenes such as backlighting and dark background, focusing using PDAF technology alone still faces challenges.

[0003] In order to solve the problem of difficulty in focusing in certain scenes, TOF devices are used in the prior art. TOF devices can calculate the distance between the object and the sensor by emitting light pulses. Therefore, the combination of TOF devices and PDAF technology can solve the problem of difficulty in focusing in certain scenes.

[0004] However, when the terminal device is not equipped with a TOF device, it is difficult to accurately focus on the subject in certain scenarios. Summary of the invention

[0005] The embodiment of the present application provides a focusing processing method and electronic device, which are applied to the field of terminal technology. The technical solution of the present application can combine the existing devices of the terminal device when the terminal device is not equipped with a TOF device to solve the problem of focusing difficulty in specific scenes.

[0006] In a first aspect, an embodiment of the present application provides a focusing processing method. The method comprises:

[0007] In response to a first operation of triggering shooting, controlling a flash of a terminal device to perform a first lighting operation;

[0008] Acquire a first distance based on a first lighting operation of the flash, where the first distance is the distance between the photographed object and the terminal device;

[0009] adjusting shooting parameters according to the first distance, wherein the shooting parameters include a lens position;

[0010] Image acquisition is performed based on the adjusted shooting parameters to obtain a first image.

[0011] In this implementation, by controlling the flash of the terminal device to perform a first lighting operation, the light emitted by the first lighting operation of the flash can be used to obtain the first distance, and the position of the lens can be adjusted according to the first distance. In addition, the image can be captured after adjusting the shooting parameters, so that an accurately focused image can be obtained.

[0012] In a possible implementation, acquiring the first distance based on a first lighting operation of the flashlight includes:

[0013] Based on a first lighting operation of the flash, acquiring a first moment of lighting of the flash; and

[0014] Acquire a second moment when the light of the flash lamp returns to the terminal device, wherein the light of the flash lamp returns to the terminal device after being reflected by the photographed object;

[0015] Determine a first duration according to the first moment and the second moment;

[0016] A first distance is obtained according to the first duration and a speed corresponding to the light.

[0017] In this implementation, by recording the first moment when the flash starts the first lighting operation and obtaining the second moment when the flash light returns to the terminal device, the time difference can be easily calculated, thereby improving the efficiency of the terminal device in calculating the lens focus position.

[0018] In a possible implementation, obtaining the second moment when the light of the flash lamp returns to the terminal device includes:

[0019] After the first moment, based on a first sensor for detecting exposure in the terminal device, detecting whether the exposure parameter changes;

[0020] When the exposure parameter changes, a second moment is acquired, where the second moment is the moment when the change in the exposure parameter is detected.

[0021] In this implementation, by acquiring the second moment when the exposure parameter changes, the moment when the light emitted by the flash returns can be determined. At the same time, it is feasible to record the moment when the AE changes.

[0022] In a possible implementation, adjusting the shooting parameters according to the first distance includes:

[0023] Determining a target value of a shooting parameter according to the first distance;

[0024] controlling the flash to perform a second lighting operation, and in the process of performing the second lighting operation, adjusting the shooting parameter based on the target value of the shooting parameter;

[0025] Image acquisition is performed based on the adjusted shooting parameters to obtain a first image.

[0026] In this implementation, the distance between the lens and the imaging sensor can be calculated by calculating the first distance, and then the shooting parameters can be adjusted to improve the accuracy of the focusing result.

[0027] In a possible implementation, after performing the first lighting operation, the method further includes:

[0028] Acquire first brightness information, where the first brightness information is brightness information detected by a preset sensor in the terminal device.

[0029] In this implementation, by detecting brightness information through a preset sensor, the surrounding environment brightness can be detected, providing a reference for the flash to adjust the exposure to a suitable brightness, thereby enabling rapid adjustment of exposure parameters.

[0030] In a possible implementation, the shooting parameters also include exposure parameters and white balance parameters;

[0031] Determining a target value of a shooting parameter according to the first distance includes:

[0032] Determining a target value of a lens position according to the first distance and a focal length of a lens in the terminal device;

[0033] Determining a convergence value of an exposure parameter and a convergence value of a white balance parameter according to the first brightness information;

[0034] Determining a target power corresponding to the second lighting operation according to the first distance, wherein the first distance is in direct proportion to the target power;

[0035] According to the target power, second brightness information of the flash when performing the second lighting operation is predicted, and the convergence value of the exposure parameter and the convergence value of the white balance parameter are adjusted based on the second brightness information to obtain the target value of the exposure parameter and the target value of the white balance parameter.

[0036] In this implementation, the convergence value of the exposure parameter and the convergence value of the white balance parameter are determined through the first brightness information, so that stable exposure parameters can be provided when shooting images. By predicting the second brightness information when the flash performs the second lighting operation, it can be ensured that sufficient light sources are provided when the light is insufficient, and the photographed object is ensured to be fully illuminated, thereby improving the brightness and clarity of the photo. By predicting the target value of the exposure parameter and the target value of the white balance parameter through the second brightness information, the quality of the image shot by the flash can be improved.

[0037] In a possible implementation, controlling the flashlight to perform a second lighting operation includes:

[0038] According to the target power, the flash lamp is controlled to perform a second lighting operation.

[0039] In this implementation, the flash performs the second lighting operation based on the target power, which not only optimizes the energy consumption of the terminal device to supply the main flash to start, but also ensures that the captured image is softer without overexposure or underexposure.

[0040] In a possible implementation, during the process of performing the second lighting operation, adjusting the shooting parameter based on the target value of the shooting parameter includes:

[0041] During the second lighting operation, when it is determined that the brightness of the flash has reached a preset peak value, each shooting parameter is adjusted to a corresponding target value.

[0042] In this implementation, when the brightness of the flash reaches a preset peak value, each shooting parameter is adjusted to its corresponding target value, so that when the exposure conditions meet the requirements, the exposure and color balance of the captured image are optimal.

[0043] In a possible implementation, the method further includes:

[0044] When the shooting mode triggered by the first operation is a stroboscopic mode, a start instruction is sent to a first camera and a second camera of the terminal device respectively, and the field of view of the first camera is greater than the field of view of the second camera;

[0045] When the photographed object enters the field of view of the first camera, a plurality of marked images are sequentially collected based on the first camera, and in the process of collecting the marked images, the flash frequency of the flash is dynamically determined;

[0046] When the photographed object enters the field of view of the second camera, the flash is controlled to flash continuously based on the dynamically determined flash frequency, and during the flash continuously flashing, a plurality of original images are sequentially collected based on the second camera;

[0047] Generate a stroboscopic image based on multiple original images.

[0048] In this implementation, by starting the stroboscopic photography mode to obtain the state of the subject at different times, the visual effect of the image can be improved. The first camera can capture a wider angle of view, which can better show the motion trajectory of the subject. The second camera can better capture the details of the subject.

[0049] In a possible implementation, during the process of acquiring the marker image, dynamically determining the flash frequency of the flash light includes:

[0050] In the process of collecting the marked images, determining the real-time moving speed of the photographed object according to the first marked image and the second marked image, the first marked image being the original image at the most recent collection time, and the second marked image being the marked image adjacent to the first marked image;

[0051] The flash frequency of the flash is dynamically determined according to the real-time moving speed of the subject, and the flash frequency is directly proportional to the real-time moving speed.

[0052] In this implementation, by marking the image to determine the flash power, the motion trajectory of the subject can be completely and clearly recorded. By dynamically determining the flash frequency of the flash, the subject can be properly exposed and the quality of the output image can be improved.

[0053] In a possible implementation, the method further includes:

[0054] When the duration range of the second camera does not include the photographed object, the flash is controlled to stop flashing.

[0055] In this implementation, by determining whether the duration range of the second camera includes the subject, the loss of the terminal device can be saved and the waste of resources caused by turning on the flash when the main camera can no longer capture the subject can be avoided.

[0056] In a possible implementation, generating a stroboscopic image according to a plurality of original images includes:

[0057] For any one of the multiple original images, extract the photographed object contained in the original image to generate an intermediate image containing the photographed object;

[0058] The stroboscopic image is obtained by performing fusion processing on the intermediate images corresponding to the multiple original images.

[0059] In this implementation, by extracting the area containing the photographed object from the original image, unnecessary losses generated by the terminal device when performing image fusion operations can be reduced. At the same time, by extracting the area containing the photographed object, the interference of the background content in the original image on image processing can be reduced.

[0060] In a possible implementation, the method further includes:

[0061] A real-time rendering image is displayed in the first area of ​​the shooting interface. The real-time rendering image is an intermediate result image generated in the process of sequentially fusing multiple intermediate images.

[0062] In this implementation, by displaying the real-time rendered image in the first area of ​​the shooting interface, the user can browse the shooting effect in real time, thereby improving user-friendliness.

[0063] In a second aspect, an embodiment of the present application provides a focus processing device, which may be an electronic device, or a chip or chip system in an electronic device. The focus processing device may include a display unit and a processing unit.

[0064] When the focus processing device is an electronic device, the display unit may be a display screen. The display unit is used to perform the display step so that the electronic device implements a focus processing method described in the first aspect or any possible implementation of the first aspect.

[0065] When the focus processing device is an electronic device, the processing unit may be a processor. The focus processing device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the electronic device to implement a focus processing method described in the first aspect or any possible implementation of the first aspect.

[0066] When the focus processing device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements a focus processing method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit in the chip (e.g., a register, a cache, etc.), or a storage unit in the electronic device located outside the chip (e.g., a read-only memory, a random access memory, etc.).

[0067] Exemplarily, the display unit is used for screen display processing of the terminal device. Exemplarily, the screen acts as a viewfinder, providing real-time preview, etc.

[0068] A processing unit is used to control the flash to perform a flash operation, and can determine the distance between the terminal device and the photographed object according to the flash operation of the flash, and then perform focusing processing according to the distance. Furthermore, for example, a series of image processing can be performed on the original image captured by the camera. Exemplary, image processing can include, for example, operations such as denoising, demosaicing, color correction and sharpening. And it can also include the operations of automatic exposure, automatic focus and automatic white balance algorithms, and the processed image data can also be format converted. In the third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory is used to store code instructions, and the processor is used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0069] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0070] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0071] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0072] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0073] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A schematic diagram of different focus positions provided in an embodiment of the present application;

[0075] Figure 2 A schematic diagram of an imaging sensor provided in an embodiment of the present application;

[0076] Figure 3 A schematic diagram of the principle of a focus pixel provided in an embodiment of the present application;

[0077] Figure 4 A schematic diagram of an application scenario of the focusing technology provided in an embodiment of the present application;

[0078] Figure 5 A schematic diagram of the technical principle of the TOF device provided in the embodiment of the present application;

[0079] Figure 6 A schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application;

[0080] Figure 7 A schematic diagram of a software structure of a terminal device provided in an embodiment of the present application;

[0081] Figure 8A schematic diagram of the process of the flashlight assisted focusing solution provided in an embodiment of the present application;

[0082] Fig. 9 A schematic diagram of measures to be taken in case of focus failure provided in an embodiment of the present application;

[0083] Fig.10 A schematic diagram of starting a pre-flash of a flashlight provided in an embodiment of the present application;

[0084] Fig.11 A schematic diagram of flashlight-assisted ranging provided in an embodiment of the present application;

[0085] Fig.12 A schematic diagram of a process for capturing images in a dark environment provided in an embodiment of the present application;

[0086] Fig.13 A schematic diagram of light intensity distribution provided in an embodiment of the present application;

[0087] Fig.14 A schematic diagram of a flashlight brightness attenuation function provided in an embodiment of the present application;

[0088] Fig.15 A schematic diagram of the timing of the scheme provided in the embodiment of the present application;

[0089] Fig.16 A schematic diagram of the stroboscopic photography process provided in an embodiment of the present application;

[0090] Fig.17 A schematic diagram of turning on the strobe mode provided in an embodiment of the present application;

[0091] Fig.18 A schematic diagram of a viewfinder provided in an embodiment of the present application;

[0092] Fig.19 A schematic diagram of an image captured by a wide-angle camera according to an embodiment of the present application;

[0093] Fig. 20 A schematic diagram of a marked image at different times provided in an embodiment of the present application;

[0094] Fig.21 A schematic diagram of an ordinary camera capturing an image provided in an embodiment of the present application;

[0095] Fig. 22 A schematic diagram of extracting an area containing a photographed object provided in an embodiment of the present application;

[0096] Fig.23 A schematic diagram of image fusion processing provided in an embodiment of the present application;

[0097] Fig.24A schematic diagram of a real-time rendered image provided by an embodiment of the present application;

[0098] Fig.25 A schematic diagram of a flash stopping scenario provided in an embodiment of the present application;

[0099] Fig.26 A flowchart of a focusing method according to an embodiment of the present invention;

[0100] Fig. 27 A schematic diagram of the process of the stroboscopic photography method provided in an embodiment of the present application;

[0101] Fig.28 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0102] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0103] 1. Focus: In the field of optics, focusing refers to the process of adjusting the focal position of the optical system so that the light from the object forms a clear image on the photosensitive element. When the object is within the effective focal length range of the lens, by adjusting the distance between the lens and the photosensitive surface, the light reflected or emitted by the object can be accurately converged on the photosensitive surface to form an image.

[0104] If this distance is adjusted properly, the image formed on the photosensitive surface will be very clear, which can be said to be in a quasi-focus state; conversely, if the distance is not appropriate, the image will appear blurry.

[0105] 2. CAM module: Camera module (CAM) is a component that integrates optical lens, image sensor, image signal processor (ISP) and other related electronic components and circuits. Its main function is to realize the shooting and acquisition of images or videos.

[0106] 3. PDAF: Phase detection auto focus (PDAF) is an autofocus technology used in cameras and smartphones. It determines the focus position by comparing the phase difference of light entering from different parts of the lens.

[0107] Specifically, PDAF technology sets up special focus pixels on the sensor. These pixels are usually not used directly for imaging, but are used to capture light information and measure the phase difference between two beams of light passing through different areas of the lens. If the phases of the two beams of light are consistent, it means that the object is already in focus; if there is a phase difference, the direction and distance that the lens needs to move are determined by calculating this difference to achieve precise focus.

[0108] 4. TOF: In the field of optical measurement and imaging, Time of Flight (TOF) is a technical means to calculate the distance between the measured object and the camera. The specific operation method is to measure the time required for the light to be emitted from the transmitter, reflected back after hitting the measured object and received by the receiver, and the distance is calculated based on this time.

[0109] 5. AE: Auto Exposure (AE) is an important function of imaging devices such as cameras and mobile phone cameras. It aims to automatically adjust exposure parameters such as aperture size, shutter speed and sensitivity according to the lighting conditions of the shooting scene to obtain appropriate exposure and ensure that the brightness of the captured image is moderate, neither overexposed (the image is too bright, resulting in loss of details) nor underexposed (the image is too dark and the content cannot be seen clearly).

[0110] 6. AF: Auto Focus (AF) is one of the key functions of imaging devices such as cameras and mobile phone cameras. It can automatically and quickly focus the subject on the image sensor, allowing users to easily obtain clear images or videos without having to manually adjust the lens focus position.

[0111] 7. AWB: Auto White Balance (AWB) is a function of imaging devices such as cameras and mobile phone cameras. Its main function is to automatically adjust the color balance of the image under different lighting conditions so that white objects still appear close to real white in the image after shooting, thereby ensuring that the color of the entire image looks natural and accurate, and avoiding image color distortion caused by light color deviation.

[0112] 8. Optical flow method: Optical flow method is a computer vision technology used to analyze the motion of objects in image sequences. Its basic idea is to describe the motion of objects by calculating the motion velocity vector (including size and direction) of pixels in the image.

[0113] In the embodiment of the present application, the movement speed of the object in the image can be determined based on images captured at adjacent moments.

[0114] 9. Other terms: In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0115] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0116] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0117] 10. Electronic equipment: The electronic equipment of the embodiments of the present application may include handheld devices, vehicle-mounted devices, etc. with focusing and shooting functions. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (PLANs). The embodiments of the present application do not limit this.

[0118] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0119] In addition, in the embodiments of the present application, the electronic device may also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0120] The electronic device in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0121] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0122] In order to better understand the technical solution of the present application, the relevant technologies involved in the present application are further introduced in detail below.

[0123] With the development of terminal devices, the photo-taking effect of mobile phone cameras is getting better and better. The improvement of photo-taking effect is inseparable from the development of focusing technology. Nowadays, PDAF technology has been widely used in terminal devices, mainly because it can provide fast and accurate focusing.

[0124] Combine the following Figure 1~Figure 3 Introduce the technical principles of PDAF to understand how PDAF technology achieves focusing. Figure 1 A schematic diagram of different focus positions provided for an embodiment of the application, Figure 2 A schematic diagram of an imaging sensor provided in an embodiment of the present application, Figure 3 A schematic diagram of the principle of a focus pixel provided in an embodiment of the present application.

[0125] It can be understood that the principle of human eye focusing is similar to the principle of PDAF technology focusing. So we can understand the focusing principle of PDAF technology by understanding the principle of human eye focusing.

[0126] When a person's eyes are focused, they are like two independent "imaging systems" and there is a certain distance between the two eyes. Therefore, the images of objects seen by the left and right eyes at the same time are different, forming different visual information. Then, the brain receives and processes the visual information from both eyes, and judges the distance and spatial position of the object by analyzing these differences, and then focuses, so that the image of the object falls accurately on the retina.

[0127] Well, it can be understood that when PDAF technology is used to achieve focusing, it is necessary to reserve some shielded pixels on the photosensitive element. These pixels are divided into two parts, one part only allows light from the left side to pass through, and the other part only allows light from the right side to pass through. At the same time, it can be understood that the shielded pixels in the left and right parts are similar to the left and right eyes of a person. Through these two groups of pixels, the photographed object is imaged separately to obtain two slightly different images. Then the phase difference of the two images is compared, and the focus offset value is calculated based on the phase difference information, so as to determine the direction and distance the lens needs to move to achieve accurate focus.

[0128] In order to further understand the meaning of focus, Figure 1 Make an introduction.

[0129] In one implementation, Figure 1 As shown in (a) in FIG. 1 , it is assumed that there are an object 101, a lens 102, and an imaging sensor 103. Among them, the lens 102 and the imaging sensor 103 are components of the camera. When the camera shoots the object 101, the light 104 reflected from the object 101 enters through the upper end of the lens 102, and the light 105 reflected from the object 101 enters through the lower end of the lens 102. Afterwards, the two beams of light converge in front of the imaging sensor 103 and converge to the focus 106, but the focus 106 is not on the imaging sensor 103. It can be understood that the focus fails at this time, and the focus is not on the imaging sensor.

[0130] Or, if Figure 1As shown in (b) in FIG. 1 , it is assumed that there are an object 101, a lens 102, and an imaging sensor 103. Among them, the lens 102 and the imaging sensor 103 are components of the camera. When the camera shoots the object 101, the light 104 reflected from the object 101 enters through the upper end of the lens 102, and the light 105 reflected from the object 101 enters through the lower end of the lens 102. Afterwards, the two beams of light converge on the imaging sensor 103 and converge on the focus 106, but the focus 106 is not on the imaging sensor 103. It can be understood that the focus fails at this time, and the focus is not on the imaging sensor.

[0131] In another implementation, Figure 1 As shown in (c) in FIG. 1 , it is assumed that there are an object 101, a lens 102, and an imaging sensor 103. Among them, the lens 102 and the imaging sensor 103 are components of the camera. When the camera shoots the object 101, the light 104 reflected from the object 101 enters through the upper end of the lens 102, and the light 105 reflected from the object 101 enters through the lower end of the lens 102. Afterwards, the two beams of light converge on the imaging sensor 103 and converge to the focus 106, and the focus 106 is on the imaging sensor 103. It can be understood that the focus is successful at this time, and the focus is on the imaging sensor.

[0132] It should be noted that the light can pass through the upper or lower end of the lens, or through the left or right end of the lens. The embodiment of the present application does not limit this. As long as two beams of light enter the lens from symmetrical positions, it can be used to analyze whether the focus can be determined to be successful.

[0133] In addition, according to Figure 1 As can be seen from (a) in FIG. 1 , the focus 106 is located in front of the imaging sensor 103 and can be understood as being in front of the focus. At this time, the light received by the upper end of the imaging sensor 103 is the light 105 and the light received by the lower end of the imaging sensor 103 is the light 104. Figure 1 As can be seen from (b) in FIG. 1 , the focus 106 is located behind the imaging sensor 103 , which can be understood as post-focus. At this time, the light received by the upper end of the imaging sensor 103 is the light 104 , and the light received by the lower end of the imaging sensor 103 is the light 105 .

[0134] It can be understood that if the imaging sensor can distinguish whether the light comes from the upper or lower part of the lens, it can determine whether the current focus failure is pre-focus or post-focus.

[0135] In order to distinguish whether the light comes from the upper half or the lower half of the lens, a sensor that covers the left half or the right half of the phase detection pixel is provided in the prior art. Figure 2 Make an introduction.

[0136] like Figure 2 As shown, it is assumed that there is a sensor 201, wherein the sensor 201 includes many sub-sensors, each sub-sensor corresponds to a pixel, and these pixels are usually arranged in pairs, and each pair of pixels receives light from different parts of the lens. The sub-sensors are divided into three categories: unshielded phase detection pixels 202, phase detection pixels 203 covering the right half, and phase detection pixels 204 covering the left half. These three types of sensors together constitute the sensor 201 to receive light from the lens. It can be understood that the phase detection pixel 203 covering the right half can only receive light from the left, and can also be called a left pixel sensor; the phase detection pixel 204 covering the left half can only receive light from the right, and can also be called a right pixel sensor.

[0137] Then, for the technical principles of the left pixel sensor and the right pixel sensor, we can combine Figure 3 To understand.

[0138] like Figure 3 As shown in (a) in the technical implementation of the right pixel sensor, it is assumed that there are a lens 301, a shielding area 302, an imaging sensor 303, an optical axis 304, a light 305 in the upper half of the optical axis, and a light 306 in the lower half of the optical axis. Then, after the light 305 and other light in the upper half of the optical axis pass through the lens 301, they propagate to the upper half of the imaging sensor 303; after the light 306 and other light in the lower half of the optical axis pass through the lens 301, they cannot propagate to the lower half of the imaging sensor 303 due to the shielding area 302. It can be understood that the right pixel sensor can only receive the light in the upper half of the lens.

[0139] like Figure 3 As shown in (b), in the technical implementation of the left pixel sensor, it is assumed that there are a lens 301, a shielding area 302, an imaging sensor 303, an optical axis 304, a light 305 in the upper half of the optical axis, and a light 306 in the lower half of the optical axis. Then, after the light 305 and other light in the upper half of the optical axis pass through the lens 301, they cannot propagate to the upper half of the imaging sensor 303 due to the shielding area 302; after the light 306 and other light in the lower half of the optical axis pass through the lens 301, they propagate to the lower half of the imaging sensor 303. It can be understood that the left pixel sensor can only receive the light in the lower half of the lens.

[0140] Based on the above content, we can understand the technical principles of the left pixel sensor and the right pixel sensor. We can also understand that after light passes through different pixel sensors, the light will form different phases. Then we can calculate the phase difference based on the different phases to achieve focusing.

[0141] However, although PDAF technology can complete the focusing task in many cases, it also has some limitations. For example, in a low-light environment, PDAF technology may not be able to accurately identify the phase difference, resulting in inaccurate focusing. It can be understood that when the light is insufficient, the focus pixels on the sensor specially set up by PDAF receive weak light, so it is impossible to measure the phase difference between the two beams of light passing through different areas of the lens, and it is difficult to perform accurate phase difference calculations, and accurate focusing cannot be achieved.

[0142] Combine the following Figure 4 Introduce the application scenarios of PDAF. Figure 4 A schematic diagram of an application scenario of the focusing technology provided in an embodiment of the present application.

[0143] like Figure 4 As shown, it is assumed that there is a terminal device 401, such as Figure 4 The interface shown in (a) can be understood as the desktop interface of the terminal device 401, which may include desktop controls such as time, date, and weather, as well as application controls such as themes, phone, and camera.

[0144] In one implementation, in response to a user triggering operation on the control 402 of the camera application, the terminal device displays the following Figure 4 The interface shown in (b) in FIG. 1 can be understood as the camera interface of the camera application. It can be understood that the camera interface of the terminal device displays the scene captured by the current camera.

[0145] It can be understood that the scene corresponding to the current picture 403 has sufficient light, so it is easy to focus successfully. The manifestation of successful focusing can be that the person 404 in the output image is clear.

[0146] In another implementation, in response to a user triggering operation on the control 402 of the camera application, the terminal device displays the following Figure 4 The interface shown in (c) in FIG. 1 can be understood as a camera interface of a camera application. It can be understood that the camera interface of the terminal device displays the scene currently captured by the camera.

[0147] It can be understood that the scene corresponding to the current picture 405 is dark, so it is easy to fail to focus. The manifestation of focus failure can be that the person 406 in the output image is blurred.

[0148] It should be noted that there are many scenes where PDAF has difficulty focusing. For example, when shooting flat or less textured areas, since phase detection relies on the change in distance between pixels, and in flat areas, this change is small, resulting in an unclear phase difference, PDAF technology may have difficulty focusing accurately.

[0149] For another example, in a backlit scene, strong light may cause the light received by the phase detection pixels to be overexposed, which may interfere with the phase difference calculation of the PDAF technology and cause inaccurate focusing.

[0150] Therefore, whether the PDAF technology can focus successfully needs to be determined according to the specific scenario, and the embodiments of the present application do not limit this.

[0151] In the prior art, TOF devices are used to solve the problem of difficulty in focusing in certain scenes. The TOF device can calculate the distance between the object and the sensor by emitting laser. Figure 5 Introduce the principle of TOF device. Figure 5 A schematic diagram of the technical principle of the TOF device provided in an embodiment of the present application.

[0152] like Figure 5 As shown, it is assumed that there is an object 5001 and a TOF device 5002. The TOF device 5002 includes a laser emitter 5003, a laser detector 5004, and the like.

[0153] When the TOF device 5002 is working, the laser emitter 5003 emits laser light, which can be understood as light 5006. When the light 5006 reaches the surface of the object 5001, it is reflected, and the reflected laser light can be understood as light 5007. When the light 5007 returns to the TOF device 5002, the laser detector 5004 can detect the returned light.

[0154] When the laser emitter 5003 completes the operation of emitting laser light and the laser detector 5004 completes the detection of reflected laser light, the TOF device will start the ranging function to calculate the distance between the terminal device and the photographed object. According to the calculated distance between the terminal device and the photographed object, the PDAF technology can be assisted to complete the focusing operation.

[0155] It is understandable that for terminal devices equipped with TOF devices, TOF technology can be used for auxiliary focusing. However, for terminal devices not equipped with TOF devices, TOF technology cannot be used for auxiliary focusing. Therefore, for the specific scenarios described above, such as dim light, shooting in flat areas, and shooting against the light, it may be impossible to accurately focus on the subject, and thus it is impossible to obtain a clear image.

[0156] In response to the above-mentioned problems, this application proposes a technical concept: since most terminal devices are equipped with a flash, the flash can be used to assist focusing. For example, the PDAF technology can be combined with the flash to calculate the distance between the photographed object and the terminal device by recording the time from the flash light emitting to the return light, thereby solving the problem of accurate focusing when the terminal device is not equipped with a TOF device.

[0157] The focus processing method of the embodiment of the present application can be executed by an electronic device with a camera, or by a chip, a chip system or a processor that supports the electronic device to implement the focus processing method, or by a logic module or software that can implement all or part of the functions of the electronic device, and the present application does not impose specific restrictions on this. The focus processing method of the embodiment of the present application is described in detail below, taking the electronic device as the execution subject as an example.

[0158] The electronic device may be, for example, a terminal device. Figure 6 and Figure 7 Provide a brief introduction to the terminal equipment.

[0159] For example, Figure 6 A schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application.

[0160] Figure 6 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a mobile communication module 150, a wireless communication module 160, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a gyroscope sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, an image sensor, etc.

[0161] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.

[0162] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors. In an embodiment of the present application, the processor 110 may, for example, control a flash to perform a flash operation, and may determine the distance between the terminal device and the photographed object according to the flash operation of the flash, and then perform a focus process according to the distance. Furthermore, the processor 110 may, for example, also perform a series of image processing on the original image acquired by the camera. Exemplarily, image processing may include, for example, operations such as denoising, demosaicing, color correction and sharpening. And it may also include the operations of automatic exposure, automatic focus and automatic white balance algorithms, and the processed image data may also be converted into a format, etc.

[0163] The terminal device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0164] The terminal device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0165] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0166] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0167] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device. In some embodiments, the angular velocity of the terminal device around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes. In an embodiment of the present application, the gyroscope sensor can detect the tilt angle of the terminal device when taking an image.

[0168] The distance sensor 180F is used to measure the distance. The terminal device can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0169] Image sensor 180J is used to detect changes in AE parameters. In the embodiment of the present application, when the light emitted by the flash reaches the surface of the photographed object, reflection occurs. The moment when the reflected light returns to the terminal device can be recorded by the moment when the image sensor detects the change in the AE value.

[0170] The software system of the terminal device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device.

[0171] For example, Figure 7 A schematic diagram of the software structure of a terminal device provided in an embodiment of the present application.

[0172] like Figure 7As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system may include an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, a hardware abstraction layer (HAL) and a kernel layer. It should be noted that the embodiment of the present application is illustrated by taking the Android system as an example. In other operating systems (such as Hongmeng system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those of the embodiment of the present application, the solution of the present application can also be implemented.

[0173] Among them, the application layer can include a series of application packages.

[0174] like Figure 7 As shown, the application package may include applications such as camera, gallery, phone, map, phone, music, settings, mailbox, video, social, etc. Of course, the application layer may also include other application packages, such as third-party applications such as payment applications, shopping applications, banking applications, social applications, etc., which are not limited in this application. In an embodiment of the present application, for example, a user may trigger a shooting operation through a camera application, and the image generated after the shooting is completed may be viewed in the gallery application.

[0175] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0176] like Figure 7 As shown, the application framework layer may include a multimedia framework, a window manager, a content provider, a resource manager, a view system, and the like.

[0177] In this embodiment, the multimedia framework may receive a shooting instruction sent by the camera application to execute operations such as starting to shoot and starting to preview the shooting, etc. Furthermore, the multimedia framework may also call the camera HAL in the HAL layer to execute subsequent shooting operations.

[0178] The system library may include multiple functional modules, such as a surface manager, a 3D graphics processing library (such as OpenGL ES), a 2D graphics engine (such as SGL), a memory manager, etc.

[0179] The HAL layer is a package of the Linux kernel driver, providing an interface to the upper layer and shielding the implementation details of the low-level hardware. The HAL layer can include the camera HAL, software code library, etc.

[0180] In this embodiment, the camera HAL is responsible for interacting with the actual hardware device (such as a camera). By providing a standardized interface, the Android framework layer does not need to understand the hardware details, and only needs to operate the device through the HAL layer. For example, the camera HAL can communicate with the camera driver to control the camera startup, configuration, and data acquisition tasks. Furthermore, the camera HAL can also send a startup instruction to the flash driver in the kernel layer, so that the flash driver controls the flash to light up, thereby performing the main flash and pre-flash operations described in this application. In addition, the camera HAL can also drive the underlying lens to perform focus processing based on the object distance determined based on the flash.

[0181] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0182] In this embodiment, the camera driver is responsible for driving the underlying camera hardware to perform image acquisition, and the flash driver is responsible for driving the underlying flash hardware to perform lighting processing.

[0183] The following, in conjunction with the accompanying drawings, describes in detail the technical solutions of the embodiments of the present application and how the technical solutions of the embodiments of the present application solve the above-mentioned technical problems with specific embodiments. The following specific embodiments can be implemented independently or in combination with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0184] Based on the above introduction, the technical solution provided by this application is described in detail below with reference to specific drawings. Figure 8 A schematic diagram of the process of the flashlight assisted focusing solution provided in an embodiment of the present application, Fig. 9 A schematic diagram of measures to be taken in case of focus failure provided in an embodiment of the present application, Fig.10 A schematic diagram of starting a pre-flash of a flashlight provided in an embodiment of the present application, Fig.11 A schematic diagram of flash-assisted ranging provided in an embodiment of the present application.

[0185] like Figure 8As shown, the scheme of using a flash to assist focusing provided by an embodiment of the present application is introduced. It can be understood that the pre-flash of the flash is used to emit light. When the light reaches the surface of the photographed object, the light will be reflected by the photographed object and then return to the lens. Then the sensor in the lens module will detect the change in exposure parameters. Therefore, the time difference between the emission and return of the light can be calculated by using the moment when the flash turns on the pre-flash and the moment when the sensor detects the change in exposure parameters. The distance between the photographed object and the terminal device can be calculated based on the time difference and the speed of light. The position where the lens needs to be adjusted can be calculated based on the distance between the photographed object and the terminal device, thereby achieving accurate focusing.

[0186] Combine the following Figure 8 For detailed introduction, refer to Figure 8 , the method comprising:

[0187] S801: Detect a first operation.

[0188] In the embodiment of the present application, the first operation can be understood as an operation to trigger shooting. Fig. 9 (a) in the figure provides an understanding of the first operation.

[0189] like Fig. 9 As shown in (a) of FIG. 1 , it is assumed that the terminal device can display the following when detecting that the user opens the camera application: Fig. 9 The interface shown in (a) in FIG. 1 may include: a photo control 901 , a thumbnail, a preview screen 902 , a control for switching cameras, a shooting mode selection item, and the like.

[0190] For example, the user can trigger shooting by clicking the photo control 901, so the click operation on the photo control 901 can be understood as the first operation described above. There are many specific implementations of the first operation to trigger shooting. In addition to clicking the photo control, the first operation can also be, for example, a pressing operation output by a volume key of a terminal device, or the first operation can also be an operation for a shooting instruction input by a voice assistant.

[0191] S802, determine whether the focusing is successful, if so, execute S808, if not, execute S803.

[0192] The technical solution of the present application is based on the focusing result. In some cases, such as the well-lit environment described above, if the PDAF technology can achieve successful focusing, the photo-taking process can be performed based on the focusing result of the PDAF technology. In other cases, such as the dark and flat environments described above, it may be that the PDAF technology cannot achieve successful focusing. In this case, the flash can be used to achieve auxiliary focusing according to the technical solution of the present application. Therefore, after the shooting is triggered in response to the first operation input by the user, for example, it can be first determined whether the focusing based on the PDAF technology is successful.

[0193] Therefore, different subsequent processing is performed according to the focusing results. On the one hand, the probability of successful focusing can be improved, and on the other hand, the energy consumption of the terminal device can be reduced, avoiding the waste of resources caused by turning on the flash to perform subsequent processing when successful focusing can be achieved based on PDAF technology.

[0194] When determining whether the focus is successful, in one implementation, for example, a phase detection (PD) point difference value threshold can be set based on factors such as the lens characteristics of the terminal device and the shooting scene. This threshold is the limit for determining whether the focus is successful. For example, when the difference value of the phase detection point exceeds this threshold, it can be considered that the focus has failed; when the difference value of the phase detection point is less than the threshold, it can be considered that the focus is successful.

[0195] It can be understood that when the difference value of the phase detection point is less than the threshold, it means that the light phase difference obtained from each detection point is closer to the phase difference value in the ideal focus state. This enables the camera to more accurately judge the distance between the subject and the terminal device, so that the focusing component of the lens can be adjusted more accurately, and the focus can be accurately placed on the subject to achieve successful focusing. When the difference value of the phase detection point exceeds the threshold, it means that the difference from the phase difference value in the ideal focus state is large, so focusing is prone to fail.

[0196] It should be noted that, in addition to the above-mentioned method of judging whether the focus is successful by using the difference value of the phase detection point, it can also be judged according to the focus speed. The specific method used to judge whether the focus is successful can be selected according to the actual situation, and the embodiment of the application does not limit this.

[0197] Combine the following Fig. 9 and Fig.10 This section describes how to use the flash to assist in focusing and obtain a clear image when PDAF fails to achieve focus in certain scenarios.

[0198] In one implementation, when focusing fails and the initial state of the flash is the self-starting mode, the flash is automatically started to implement the auxiliary ranging function, thereby acquiring an image of the shooting scene.

[0199] like Fig. 9 As shown in (a) of FIG. 1 , it is assumed that the terminal device can display the following when detecting that the user opens the camera application: Fig. 9 The interface shown in (a) in FIG. 1 can be understood as a camera interface of a camera application. The camera interface displays a picture 902 of a shooting scene, the background of the picture 902 is dark, and the current state of the flash is the self-starting mode 903.

[0200] In response to the trigger operation for the photo control 901, when the terminal device detects that the current focus fails, the following is displayed: Fig. 9 In the interface shown in (b) of FIG. 1 , the flash in the interface changes from the self-starting mode 903 to the opening mode 904. In response to the automatic opening mode 904 of the flash, the lens emits light 905, and the picture 906 becomes clear.

[0201] After a certain period of time, the terminal device display interface jumps to the following Fig. 9 The interface shown in (c) in the figure shows the initial interface after the shooting is completed, which can be understood as the interface of the scene shot by the camera when the camera application is opened, wherein the interface may include a thumbnail control 907 and a picture 908 of the shooting scene. It can be understood that the thumbnail control 907 displays a preview of the shot image. At the same time, it can be understood that since the flash has completed the auxiliary focus operation and returned to the self-starting state, in the current scene, the light in the picture 908 of the shooting scene is dim, but the person becomes clear.

[0202] In another implementation, when focusing fails and the initial state of the flash is off mode, the terminal device will pop up a prompt to start the flash. After starting the flash according to the prompt, the flash can be used to complete the auxiliary focusing operation to obtain an image of the shooting scene.

[0203] like Fig.10 As shown in (a) of FIG. 1 , it is assumed that the terminal device can display the following when detecting that the user opens the camera application: Fig.10 The interface shown in (a) in FIG. 1 can be understood as a camera interface of a camera application. The camera interface displays a picture 1002 of a shooting scene, the background in the picture 1002 is dark, and the current state of the flash is an off mode 1003 .

[0204] In response to the trigger operation on the photo control 1001, the following is displayed: Fig.10In the interface shown in (b), the terminal device detects that the current picture is dark and pops up a focus failure message prompt box 1004. After executing the "confirm" operation 1004, the flash will be turned on, and after executing the "cancel" operation 1004, the photo shooting will be canceled or the flash will be refused to be turned on.

[0205] In response to the trigger operation of "confirm" on the message prompt box 1004, the following is displayed: Fig.10 In the interface shown in (c) of FIG. 1 , the flash in the interface changes from the off mode 1003 to the on mode 1005. In response to the flash turning on mode 1005, the lens emits light 1006, and the picture 1007 becomes clear.

[0206] After a certain period of time, the terminal device display interface jumps to the following Fig.10 The interface shown in (d) in FIG. 1 may include a thumbnail control 1008 and a picture of a shooting scene 1009 . The thumbnail control 1008 displays a preview of the shot image, and the picture of the shooting scene 1009 has a dark background but clear characters.

[0207] It should be noted that when the focus fails and the flash is off, in addition to the above-mentioned pop-up message prompt box to manually start the flash, the flash can also be automatically turned on without user confirmation, which can reduce user operations. In actual applications, the specific method of starting the flash can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0208] S803: Start the pre-flash of the flashlight and record the first moment of starting.

[0209] In the case where focusing based on the PDAF technology fails, auxiliary focusing is required with the help of a flash. At this time, for example, the flash can be controlled to perform a pre-flash operation, wherein the pre-flash refers to a series of flashes emitted by the flash.

[0210] In one implementation, the distance between the device and the object can be determined based on the time it takes for the light of the flash to be emitted and returned. The moment when the pre-flash of the flash is started is equivalent to the moment when the light is emitted, so the moment when the pre-flash operation of the flash is started can be recorded as the first moment, and the first moment can also be understood as the starting moment of the light emission.

[0211] Therefore, recording the moment when the flash starts the pre-flash can facilitate the calculation of the time difference, which can improve the efficiency of the terminal device in calculating the focus position of the lens.

[0212] In actual operation, recording the moment when the flash changes from a non-open state to an open state can be understood as the first moment, and the specific manifestation of the non-open state can be determined according to actual needs, and the embodiment of the present application does not limit this.

[0213] S804: Obtain the second moment when AE changes.

[0214] In the case of focusing failure, when using the flash to assist focusing, in addition to the need to record the first moment when the flash starts the pre-flash as introduced in S803 above, it is also necessary to record the moment when the light emitted by the pre-flash of the flash returns, which can also be understood as the second moment when AE changes.

[0215] In one implementation, the image sensor detects changes in the AE value in real time. When the light emitted by the pre-flash of the flash returns, the initial value of AE will change. Therefore, it is necessary to record the moment when the AE value changes as the second moment.

[0216] S805: Determine a first duration according to the first moment and the second moment.

[0217] According to the above content, the first moment is obtained from S803 and the second moment is obtained from S804, then the time difference from the emission to the return of the light emitted by the pre-flash of the flash can be calculated according to the difference between the two moments, and then this time difference can be understood as the first duration. Thus, the first duration is obtained, which lays the foundation for further calculating the distance between the terminal device and the photographed object.

[0218] Combine the following Fig.11 Describes the process of using flash to assist focusing.

[0219] like Fig.11 As shown, it is assumed that a terminal device 1101 shoots an object 1102, wherein the terminal device 1101 may include a lens module 1103, and the lens 1103 includes a flash. When the flash is turned on, the moment when the flash starts pre-flash is recorded as t1, that is, the first moment mentioned above.

[0220] When the flash is turned on, it emits uniform light, which can be understood as light 1104. When light 1104 reaches the surface of the photographed object 1102, it is reflected, and the reflected light can be understood as light 1105. When light 1105 reaches the lens module 1103, the time when AE changes is recorded as t2, which is the second time mentioned above. Then, the time difference between t1 and t2 is the first duration, which can also be understood as the time difference from the emission to the return of the light.

[0221] S806: Determine a first distance according to the first duration and the speed of light.

[0222] According to the above content, the embodiment of the present application records the time difference from the emission to the return of the light. It can be understood that when the light is emitted until it reaches the surface of the photographed object, it passes the distance from the terminal device to the photographed object; and when the light is from the photographed object to the return terminal device, it passes the distance from the photographed object to the terminal device. The two distances are equal, so the time difference from the terminal device to the photographed object or the time difference from the photographed object to the terminal device is the same, both of which are half of the first duration. At the same time, it can be understood that the distance from the terminal device to the photographed object can be understood as the first distance.

[0223] Therefore, according to the first duration and the speed of light, the first distance can be calculated, and according to the first distance, the position where the lens needs to be adjusted can be calculated.

[0224] It can be understood that the longer the time from the light emission to the return, the farther the distance between the terminal device and the photographed object. Therefore, the time from the light emission to the return is directly proportional to the distance from the terminal device to the photographed object. According to the distance calculation formula, the relationship between the time t from the light emission to the return and the distance from the terminal device to the photographed object can satisfy the following formula 1:

[0225] d=c*t / 2 Formula 1

[0226] Where d is the distance between the terminal device and the object being photographed, c is the speed of light, and t is the total time it takes for light to be emitted and returned.

[0227] It should be noted that the above-mentioned formula 1 introduces an implementation of calculating the distance from the terminal device to the object. This embodiment does not limit the specific calculation method of the distance from the terminal device to the object, as long as the time from the emission to the return of the light is directly proportional to the distance from the terminal device to the object.

[0228] It should be noted that when the terminal device shoots an object, the terminal device may have a certain tilt angle, wherein the tilt angle of the terminal device may cause the determined first distance to be somewhat different from the actual distance between the terminal device and the photographed object. Therefore, in one implementation, for example, the above-determined first distance may also be corrected according to the tilt angle corresponding to the terminal device to improve the accuracy of the determined object distance between the terminal device and the photographed object. Exemplarily, the relationship between trigonometric functions may be calculated according to the tilt angle to correct the distance error caused by the tilt angle. And, the tilt angle of the terminal device may be determined, for example, based on data collected by a gyroscope sensor.

[0229] S807: Adjust the lens position according to the first distance and the focal length of the lens in the terminal device.

[0230] In the image formation process, in order to achieve clear imaging, the relationship between object distance, image distance and focal length is involved. Among them, the object distance refers to the distance from the photographed object to the lens, which can also be understood as the first distance calculated by S806. The image distance refers to the distance from the image to the lens, which can also be understood as the parameter for adjusting the motor focus. Focal length is an inherent property of the lens and is usually a fixed value. At the same time, it can be understood that according to the principle of optical imaging, when the object distance, image distance and focal length satisfy the following formula 2, accurate focus can be achieved:

[0231] 1 / f=1 / u+1 / v Formula 2

[0232] Among them, f is the focal length of the lens, u is the object distance, and v is the image distance.

[0233] It can be understood that according to the basic principle of optical imaging, when the object distance is different, in order to make the object imaged clearly on the imaging sensor, the image distance needs to be changed accordingly. At the same time, it can be understood that when the object distance is determined and the focal length is a fixed value, there is a specific image distance that can make the object imaged most clearly. By adjusting the position of the lens to change the image distance, clear focus can be achieved.

[0234] Therefore, based on the calculated distance between the object and the terminal device, the distance between the lens and the imaging sensor can be calculated, and then the motor can be adjusted to achieve focus, thereby improving the accuracy of the focus result.

[0235] S808, executing the photo taking process.

[0236] When the distance between the lens and the imaging sensor is adjusted to a suitable position, clear focus can be achieved and the photo-taking process can be executed.

[0237] Can be combined Fig.10 As can be understood from (d) in FIG. 1 , when the focusing is successful, a preview image 1008 of the captured image will be output. It can be found that the image in the preview image 1008 is clear.

[0238] In summary, when a camera and a flash are combined, the distance measurement function of the TOF device can be replaced. Specifically, when focusing fails, the flash is turned on to measure the first time length from the emission of light to the return to the sensor. According to the first time length and the speed of light, the first distance can be calculated. According to the first distance, the distance between the lens and the sensor can be adjusted. Based on this, the focusing operation is completed and the photo-taking process is executed. Using a flash to assist focusing solves the problem of focus failure in specific scenarios when the terminal device is not equipped with a TOF device.

[0239] In addition, in the embodiment of the present application, after measuring the distance between the photographed object and the terminal device, the power output by the flash can also be adjusted to obtain a suitable main flash brightness to capture the image. The embodiment of the present application is described below in conjunction with specific drawings. Fig.12 A schematic diagram of a process for capturing images in a dark environment provided in an embodiment of the present application. Fig.13 A schematic diagram of light intensity distribution provided in an embodiment of the present application, Fig.14 A schematic diagram of the attenuation function of the flashlight brightness provided in an embodiment of the present application, Fig.15 A schematic diagram of the timing of the scheme provided in the embodiment of the present application. The method includes:

[0240] S1201, start the pre-flash of the flash.

[0241] In S803, when focusing fails, the pre-flash of the flash is used to measure the first distance. In addition to the above, the pre-flash of the flash can also be used to measure the brightness of the surrounding environment. According to the measured brightness of the surrounding environment, the convergence value of the exposure parameter can be determined. It can be understood that the camera of the terminal device can calculate the appropriate flash intensity and exposure parameters of the main flash by receiving the light reflected by the pre-flash. Therefore, the pre-flash of the flash can help the camera more accurately determine the required flash amount under complex lighting conditions.

[0242] S1202: Acquire first brightness information detected by a sensor.

[0243] In the embodiment of the present application, the sensor refers to a device used to detect changes in AE parameters. In the process of turning on the camera, the sensor detects brightness changes in real time. When the flash is working, the light it emits will instantly increase the light intensity of the surrounding environment. Exemplarily, after receiving the light signal, the light sensitive element in the sensor converts it into an electrical signal. By measuring the change in the electrical signal, the current brightness information can be obtained, which can also be understood as the first brightness information. Therefore, the sensor detects the brightness information, which can help the processor quickly adjust the exposure parameters.

[0244] S1203: Determine a convergence value of AE and AWB according to the first brightness information.

[0245] For AE, controlling convergence means that the camera finds the appropriate exposure setting. When the flash is turned on, the light intensity emitted by the flash is relatively stable and known, and the sensor can measure this stable brightness information, which is the first brightness mentioned above. Combined with the first brightness information, the processor can accurately calculate the parameters required to achieve the appropriate exposure, and then determine the convergence value of AE.

[0246] Therefore, the convergence value of AE can be understood as enabling the automatic exposure system to eventually reach a stable exposure parameter value.

[0247] For AWB, controlling convergence means that the camera quickly adjusts and achieves a natural color balance under different light sources. Since the color temperature of the flash is usually fixed, after the sensor obtains the image information containing the flash light, it can analyze the color deviation of the white area in the image based on the known color temperature of the flash, and then determine the parameters required to correct the color balance and obtain the convergence value of AWB.

[0248] Therefore, the first brightness information can be used to determine the convergence value of AE and AWB, and can provide a stable exposure parameter when shooting an image.

[0249] S1204: Determine a target power of the main flash according to the first distance.

[0250] At present, the existing flash lights on the market are all fixed power, which is prone to overexposure or underexposure, and the light of the flash is relatively hard, resulting in a relatively stiff photo result. It is understandable that when the photos taken after turning on the flash are severely sharpened, lack a soft feeling, and the look of the photos taken is not very comfortable. In an embodiment of the present application, the flash power can be automatically controlled through the first distance to achieve a better lighting effect. Therefore, the automatic control of the flash power can not only ensure that the terminal device supplies the main flash with the optimal energy consumption, but also ensure that the captured image is softer and will not be overexposed or underexposed.

[0251] Since the intensity of light decreases with increasing distance, when the flash power can ensure the exposure effect and emit light, the intensity of light reaching the surface of the photographed object will be different due to the different distances between the photographed object and the terminal device. It can be understood that when the distance between the photographed object and the terminal device is closer, the intensity of light reaching the surface of the photographed object is stronger; when the distance between the photographed object and the terminal device is farther, the intensity of light reaching the surface of the photographed object is weaker.

[0252] Can be combined Fig.13 To understand, such as Fig.13 As shown, assuming that there is a terminal device 1301, when the flash pre-flash is turned on, the light shown in the figure will be emitted, and 1302 is used in the figure to illustrate the light. According to the inverse square law in optics, the intensity I of the light is inversely proportional to the square of the distance d. Then it can be understood that when the subject is at position r, the intensity of the light reaching the subject is 1; when the subject is at position 2r, the intensity of the light reaching the subject is 1 / 4; when the subject is at position 3r, the intensity of the light reaching the subject is 1 / 9.

[0253] Therefore, in order to ensure that the intensity of the light when the flash reaches the surface of the object can maintain the exposure effect, it is necessary to adjust the initial power of the flash, which can also be understood as the target power. It can also be understood that when the distance is closer, in order to ensure that the light intensity is consistent when the light reaches the surface of the object, the target power of the flash needs to be smaller; when the distance is farther, the target power of the flash needs to be larger to ensure that the intensity of the light remains unchanged when it reaches the surface of the object. In this way, no matter how far the object is from the terminal device, the intensity of the light is consistent when it reaches the surface of the object, so that a better exposure effect can be guaranteed, and the captured image can be guaranteed to be softer.

[0254] According to the above content, it can be understood that the target power of the flashlight is in direct proportion to the first distance, and the required light intensity is certain, which can also be understood as the required power of the flashlight is certain. Then, the relationship between the target power of the flashlight and the required power can satisfy the following formula 3, for example:

[0255] P=P0 / d 2 Formula 3

[0256] Wherein, d represents the first distance, the target power of the flash is P0, and the power required by the flash is P.

[0257] It should be noted that the calculation of the target power of the flashlight provided above is a possible implementation method, but there may be other function forms to describe the relationship between the target power of the flashlight and the first distance, and it is only necessary to satisfy that the target power of the flashlight is in direct proportion to the first distance. Therefore, the function expression of the target power of the flashlight and the first distance can be determined according to actual needs, and the embodiment of the present application does not limit this.

[0258] S1205. Predict the second brightness information of the main flash according to the target power.

[0259] The main flash is the flash operation performed during the actual shooting process. The actual power of the main flash is adjusted based on the target power in S1204 to ensure accurate exposure of the subject. The brightness of the main flash is proportional to the output power. The greater the output power, the brighter the light emitted by the flash. Therefore, the brightness information of the main flash can be predicted by the target power of the main flash of the flash, which can also be understood as the second brightness information.

[0260] For example, a mapping relationship between power and brightness information may be pre-constructed, so as to predict the second brightness information of the main flash according to the target power. Alternatively, a prediction model may be pre-trained to predict the second brightness information of the main flash according to the prediction model based on the target power.

[0261] S1206. Predict target values ​​of AE and AWB according to the second brightness information.

[0262] When shooting with flash, the combination of ambient light and flash light affects the final exposure and white balance settings. Therefore, the camera needs to estimate the impact of the flash brightness on the scene before shooting. After estimating the impact of the flash, the camera will adjust the 3A settings to adapt to the new light conditions. It can be understood as estimating the target values ​​of AE and AWB at the main flash level to prepare the conditions for capturing the final image. Therefore, by estimating the target values ​​of AE and AWB at the main flash level and adjusting the exposure parameters and white balance settings, the quality of flash-shot images can be improved.

[0263] For example, a mapping relationship between brightness information and AE and AWB may be pre-constructed, so as to predict the target values ​​of AE and AWB corresponding to the main flash according to the predicted second brightness information. Alternatively, a prediction model may be pre-trained to predict the target values ​​of AE and AWB corresponding to the main flash according to the prediction model. Alternatively, based on the convergence values ​​of AE and AWB determined based on the pre-flash, the changes of the first brightness information and the second brightness information may be compared, so as to predict the target values ​​of AE and AWB corresponding to the main flash.

[0264] S1207: Start the main flash of the flashlight according to the target power.

[0265] When the conditions for shooting an image are ready, the pre-flash of the flash goes out and the main flash of the flash turns on. According to different first distances, the target power of the flash is different, and thus the power of the main flash of the flash is different. Therefore, according to the first distance, the target power of the flash is determined, and the working power of the flash is adjusted appropriately according to the target power, which can save the loss of the terminal device, make the captured image softer, and improve the shooting quality.

[0266] S1208 , when the main brightness reaches a preset peak value, the convergence value of AE and AWB is adjusted to a target value.

[0267] Peak value can be understood as the highest value of brightness. The duration of a flash refers to the time interval from the flash lighting up to the flash going out, which is usually between a few milliseconds and a few microseconds. As time changes, the brightness of the flash will decay after reaching its peak until it goes out.

[0268] Can be combined Fig.14 Understand the effect of flash peaking, such as Fig.14 As shown in FIG. 1 , assuming that there is a coordinate system with the horizontal axis being time T and the vertical axis being the flash brightness L, curve 1401 represents the attenuation function of the flash. As time goes by, the brightness of the flash rises, and then reaches the highest point and then begins to decline. Fig.14It can be seen that when the time is t1, the brightness of the flash is the largest, which is l3, which can be understood as the peak value. After that, as time goes by, the brightness becomes smaller and smaller until it goes out.

[0269] During the shooting process, the camera waits for the main flash brightness to reach an ideal peak value, and then adjusts the convergence value of AE and AWB to the target value according to this brightness condition to optimize the shooting effect. Therefore, sending the target value after the main flash brightness reaches the peak value can ensure that the exposure and color balance of the captured image are optimal when the exposure conditions meet the requirements.

[0270] S1209: Collect images.

[0271] When the target values ​​of AE and AWB are issued, the camera will capture images when the flash reaches its peak value, so that the captured images can be properly exposed, color-balanced, clear and high-quality.

[0272] S1210: The main flash of the flash is turned off, and a first image is output.

[0273] After the image is captured, the flash follows the change law of the brightness attenuation function and is finally turned off. After a series of processing of the captured image, including denoising, the image after imaging is finally output, which can be understood as the first image. Therefore, turning off the main flash can reduce the loss of the terminal device, and outputting the processed image can meet the needs of users.

[0274] In summary, the combination of pre-flash and main flash can significantly improve the effect of flash photography, especially in environments with changing light or complex reflection conditions. In this way, the terminal device can automatically adjust the target power of the flash to reduce the risk of overexposure or underexposure, thereby obtaining a more natural and balanced image. At the same time, taking the picture when the main flash reaches its peak can ensure that the image exposure meets the requirements.

[0275] Fig.15 By integrating the above two embodiments, according to the timing relationship, the operations completed when the pre-flash is turned on and the operations completed when the main flash is turned on can be obtained.

[0276] like Fig.15As shown, at time t1, the terminal device detects the first operation; in response to the first operation, focusing is performed based on the PDAF technology, and when focusing fails, a pre-flash of the flash is started at time t2; after the pre-flash of the flash is turned on, at time t3, the first distance is determined based on the pre-flash; after obtaining the first distance, at time t4, the target power of the main flash is determined based on the first distance, and the target values ​​of AE\AWB\AF under the main brightness are estimated. It can also be understood that the target value of AF under the main brightness corresponds to the adjusted lens position introduced in the above embodiment, and the target value of AF is a parameter value used to indicate focusing; when the estimated operation is completed, the pre-flash is extinguished at time t5; thereafter, at time t6, the main flash is turned on based on the target power; at time t7, when the main brightness reaches a peak, the target values ​​of AE\AWB\AF are issued, wherein the target value of AF is used to indicate the adjustment of the lens position; at time t8, the terminal device captures an image; finally, at time t9, the main flash is turned off.

[0277] Based on the above introduction, the present application can also dynamically adjust the flash frequency of the flash according to the speed of the photographed object, thereby obtaining a stroboscopic image showing the motion trajectory. The following is an introduction to the embodiments of the present application in conjunction with specific drawings. Fig.16 A schematic diagram of the stroboscopic photography process provided in an embodiment of the present application, Fig.17 This is a schematic diagram of turning on the stroboscopic mode provided in an embodiment of the present application. Fig.18 A schematic diagram of a viewfinder provided in an embodiment of the present application, Fig.19 A schematic diagram of an image captured by a wide-angle camera according to an embodiment of the present application, Fig. 20 A schematic diagram of a marked image at different times provided in an embodiment of the present application, Fig.21 A schematic diagram of an ordinary camera capturing an image provided in an embodiment of the present application, Fig. 22 A schematic diagram of extracting a region containing a photographed object provided in an embodiment of the present application, Fig.23 A schematic diagram of image fusion processing provided in an embodiment of the present application, Fig.24 A schematic diagram of a real-time rendered image provided by an embodiment of the present application, Fig.25 A schematic diagram of a flash stopping scenario provided in an embodiment of the present application.

[0278] S1601, start stroboscopic photography mode.

[0279] Stroboscopic photography refers to the use of a lamp that can flash multiple times in a very short time interval to emit periodic flashes, expose a moving object multiple times, and thus record the position and posture of the object at different times in one image. In the embodiment of the present application, the flash of the terminal device is used to flash multiple times in a very short time interval, and then expose the photographed object multiple times, so that the position and posture of the photographed object at different times can be obtained.

[0280] For example, for the athlete's movement process, the stroboscopic photography mode can be used to obtain the athlete's state at different times, and the movement states at different times can be presented on the same picture, which can be used to analyze whether the athlete's movements at different times are standard, so as to better guide the athlete's movement. Therefore, it is very necessary to use the stroboscopic photography mode.

[0281] Therefore, in the embodiment of the present application, by starting the stroboscopic photography mode to obtain the state of the photographed object at different times, the visual effect of the image can be improved. At the same time, the user can use the image output by the stroboscopic photography to analyze the motion state of the photographed object, thereby improving the quality of production or life.

[0282] Combine the following Fig.17 An exemplary method of starting stroboscopic photography is described below.

[0283] like Fig.17 As shown, it is assumed that the terminal device 1701 can display the following when detecting that the user opens the camera application: Fig.17 The interface shown in (a) in the figure may include: a camera control, a thumbnail, a preview screen 1702, a control for switching cameras, a shooting mode selection item, etc. The shooting mode selection item includes but is not limited to: portrait, photo, video, professional or more controls 1703.

[0284] In response to the user's triggering operation on more controls 1703, the terminal device displays the following Fig.17 The interface shown in (b) in the figure may include: shooting mode selection items. The shooting mode selection items include but are not limited to: professional, panoramic, HDR control, time-lapse photography, watermark, stroboscopic photography, high pixel, micro-movie or other types of shooting mode selection items.

[0285] It can be understood that the user's trigger operation 1704 on the stroboscopic photography control can start the stroboscopic photography mode.

[0286] It should be noted that the above-mentioned method of starting the stroboscopic photography mode is to open the camera application and then click the relevant control. In another implementation, the stroboscopic photography mode can be used as a desktop component and can be started by directly clicking. In the actual operation of opening the stroboscopic photography mode, the method of opening the stroboscopic photography mode can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0287] S1602: Issue operation instructions for the wide-angle camera and the main camera.

[0288] A wide-angle camera is a camera device that can capture images with a wider viewing angle, and its lens has a shorter focal length. It can be understood that according to the optical imaging formula, the shorter the focal length, the wider the viewing angle of the image. At the same time, it can be understood that a wider scene can be captured using a wide-angle camera. However, due to optical characteristics, the images captured by a wide-angle camera usually have a certain degree of distortion. Therefore, after starting the stroboscopic photography mode, a wider viewing angle can be captured using a wide-angle camera, which can better show the motion trajectory of the subject.

[0289] The main camera is the default camera used in most daily shooting scenarios, and its lens has a longer focal length. Understandably, the main camera captures a narrower scene than the wide-angle camera. However, the main camera generally has better focusing ability and can accurately focus on the subject. Therefore, after activating the stroboscopic photography mode, the normal camera can better capture the details of the subject.

[0290] Therefore, by using the combination of a wide-angle camera and a main camera, it can be ensured that the output image has more details at a wider viewing angle and can better present the motion state of the subject.

[0291] Combine the following Fig.18 An example is given for starting the wide-angle camera and the main camera.

[0292] like Fig.18 As shown in (a) of FIG. 8 , it is assumed that there is a shooting scene, which includes a moving ball 1801 and white clouds 1802 .

[0293] In response to the user's trigger operation for stroboscopic photography, the terminal device will start the wide-angle camera and the main camera.

[0294] When the terminal device starts the wide-angle camera, the preview screen of the terminal device is displayed as follows: Fig.18 The interface shown in (b) in FIG. 1 includes: a prompt message 1803 of the optical zoom of the wide-angle camera, and a preview screen 1804 of the wide-angle camera.

[0295] Among them, the prompt information 1803 of the optical zoom of the wide-angle camera shows "0.5X", and 0.5X usually means that the wide-angle camera is enabled and the focal length of this lens is 0.5 times the focal length of the main camera. For example, if the focal length of the main camera is 26mm, then the focal length of the ultra-wide-angle lens is about 13mm. According to the optical imaging formula, this means that the wide-angle camera can capture a wider scene. It can be understood in conjunction with the preview screen 1804. In the current screen, you can browse the complete scene information.

[0296] When the terminal device starts the main camera, the preview screen of the terminal device is displayed as follows Fig.18 The interface shown in (c) includes: prompt information 1805 of the original focal length of the main camera and a preview screen 1806 of the main camera.

[0297] Among them, the prompt information 1805 of the original focal length of the main camera shows "1X", and 1X usually means that the main camera is enabled and the focal length of this lens is 1 times the focal length of the main camera. If the focal length of the main camera is 26mm, when it is in the 1X state, the actual focal length of the lens is 26mm, and the shooting angle is the standard angle designed by the main camera itself. The range of the picture presented is more in line with the normal field of view of the human eye. It can be understood in conjunction with the preview screen 1806. In the current screen, some scene information can be browsed, but the scene information is clearer.

[0298] In the embodiment of the present application, the prompt information 1803 of the optical zoom of the wide-angle camera and the prompt information 1805 of the original focal length of the main camera are popped up only when the stroboscopic photography mode is initially turned on, and the prompt will no longer pop up in subsequent processing.

[0299] It should be noted that the object of the shooting in this embodiment is a ball, and the ball is in motion, which can also be understood as recording the motion state of the ball, and other background content is considered to be static, for example, white cloud 1802. It should also be noted that in actual applications, other moving objects can be selected as the object of the shooting, and the embodiment of the present application does not limit this.

[0300] S1603: When the photographed object enters the field of view of the first camera, collect a marked image through the wide-angle camera.

[0301] After the wide-angle camera is started, the wide-angle camera needs to collect a marked image. The marked image can be understood as the image collected by the wide-angle camera when the subject enters the field of view of the wide-angle camera. Since the wide-angle camera has a wider field of view, using the wide-angle camera to collect images can capture more motion pictures of the subject. With more motion pictures, the stroboscopic frequency of the flash can be calculated more accurately.

[0302] Combine the following Fig.19 The process of collecting marked images using a wide-angle camera is introduced by way of example.

[0303] like Fig.19As shown in (a) in the figure, it is assumed that the terminal device turns on the stroboscopic photography mode. In this mode, it is assumed that there are the framing conditions of the framing frames shown in Examples 1, 2, and 3. Each example includes: a framing frame 1901 of the wide-angle camera, a framing frame 1902 of the main camera, and the photographed object and background content located in the framing frame. In the current step, the framing condition of the framing frame corresponding to the wide-angle camera is first described.

[0304] exist Fig.19 In Example 1 shown in (a) of FIG. 1 , when the photographed object moves to position 1903, it enters the first field of view of the wide-angle camera. At this time, the wide-angle camera will collect the marked image of the photographed object when it moves, such as Fig.19 Image 1 shown in (b).

[0305] exist Fig.19 In Example 2 shown in (a) of FIG. 1 , when the photographed object moves to position 1905, it is within the first field of view of the wide-angle camera. At this time, the wide-angle camera will collect the marked image of the photographed object when it moves, such as Fig.19 Image 2 shown in (b).

[0306] In Example 2, position 1904 can be understood as the position of the photographed object at the last moment of position 1905, which is indicated by a dotted line and corresponds to position 1903 in Example 1. It can be understood that the photographed object moves from position 1904 to position 1905.

[0307] exist Fig.19 In Example 3 shown in (a) of FIG. 1 , when the photographed object moves to position 1908, it is within the first field of view of the wide-angle camera. At this time, the wide-angle camera will collect the marked image of the photographed object when it moves, such as Fig.19 Image 3 shown in (b).

[0308] Among them, in Example 3, position 1907 can be understood as the position of the photographed object at the previous moment of position 1908, represented by a dotted line, and corresponds to position 1905 in Example 2, and position 1906 can be understood as the position of the photographed object at the previous two moments of position 1908, represented by a dotted line, and corresponds to position 1904 in Example 2, and corresponds to position 1903 in Example 1. It can be understood that the photographed object moves from position 1906 to position 1907, and then to position 1908.

[0309] It should be noted that the three examples illustrated above are the framing frames and captured images corresponding to the continuous motion of the subject. In actual operation, there is no limit on the number of frames for capturing the marked images, but what needs to be satisfied is that the marked images are captured when the subject is in continuous motion.

[0310] S1604: Determine the stroboscopic frequency of the flashlight according to the marked image.

[0311] After a plurality of frames of marked images are collected by a wide-angle camera, the plurality of frames of marked images can be processed by an image processing algorithm, and the processed image frames can be used to calculate the stroboscopic frequency of the flash.

[0312] It can be understood that in stroboscopic photography mode, when the subject enters the field of view of the wide-angle camera but not the field of view of the main camera, the wide-angle camera will run before the main camera. Then the wide-angle camera can pre-collect multiple frames of marked images, which record the position of the subject at different times.

[0313] Afterwards, an image processing method, such as an optical flow method, can be used to calculate the movement speed of the object from the images collected at different times. According to the movement speed of the object, the stroboscopic frequency of the flash at the next moment can be determined, and the stroboscopic frequency of the flash at the next moment can be used to control the flash to flash, so that the image collected by the main camera remains clear.

[0314] It is understandable that in the process of stroboscopic photography, the flash is used to provide exposure for the subject. In order to achieve that the stroboscopic frequency of the flash can adapt to the movement of the subject, when the subject moves fast, the stroboscopic frequency of the flash needs to be increased, and when the subject moves slowly, the stroboscopic frequency of the flash needs to be reduced. Therefore, the stroboscopic frequency of the flash is directly proportional to the movement speed of the subject.

[0315] It should be noted that the relationship between the movement speed of the subject and the stroboscopic frequency of the flash can be determined according to specific needs, but it is necessary to satisfy that the movement speed of the subject and the stroboscopic frequency of the flash are in direct proportion, and the specific form of the relationship is not limited in the embodiments of the present application.

[0316] Combine the following Fig. 20 The process of determining the strobe frequency of a flash lamp is described as an example.

[0317] like Fig. 20 As shown, assuming that there are three frames of marked images captured at adjacent moments, they may be image 1, image 2, and image 3, wherein in image 1 the object is at position 2001, in image 2 the object is at position 2002, and in image 3 the object is at position 2003. It can be understood that the object moves from position 2001 to position 2002, and from position 2002 to position 2003 at adjacent moments.

[0318] The optical flow method can be used to calculate the speed of the object v1 based on Image 1 and Image 2. The strobe frequency f1 can be obtained based on v1. The strobe frequency of the flash at the next moment can be controlled according to the frequency f1 to achieve appropriate exposure of the image captured by the main camera at the next moment.

[0319] Similarly, according to the above logic, the speed of the photographed object calculated according to Image 2 and Image 3 is v2, and the strobe frequency f2 can be obtained according to v2. The strobe frequency of the flash at the next moment is controlled according to the frequency f2 to achieve appropriate exposure of the image captured by the main camera at the next moment.

[0320] It should be noted that the image processing method used in the above embodiment is the optical flow method, and in practical applications, other image processing algorithms may also be used, which is not limited in the embodiments of the present application.

[0321] S1605: Control the continuous flashing of the flash according to the stroboscopic frequency of the flash.

[0322] According to the above calculation method, the stroboscopic frequency of the flash can be calculated, and the continuous flashing of the flash can be controlled according to the frequency of the flash, so that the flash frequency can be dynamically adjusted. Therefore, according to the different movement speeds of the photographed object, different flash frequencies can be adjusted to achieve reasonable exposure of the photographed object and improve the quality of the output image.

[0323] S1606: Collect original images through the main camera.

[0324] When the subject enters the main camera, the image captured by the main camera can be understood as the original image. By capturing the original image through the main camera, the motion details of the subject at different times can be captured more clearly. Therefore, using the main camera to capture the original image can not only ensure the high definition of the captured image, but also ensure the quality of the final output image.

[0325] Combine the following Fig.21 The process of collecting raw images by the main camera is introduced as an example.

[0326] like Fig.21 As shown in (a) in the figure, it is assumed that the terminal device turns on the stroboscopic photography mode. In this mode, it is assumed that there are the framing conditions of the framing frames shown in Examples 1, 2 and 3. Each example includes: a framing frame 2101 of the wide-angle camera, a framing frame 2102 of the main camera, and the photographed object and background content in the framing frame. In the current step, the framing condition of the framing frame corresponding to the main camera is described.

[0327] exist Fig.21In Example 1 shown in (a) of FIG. 1 , when the object moves to position 2103, it enters the first field of view of the main camera. At this time, the main camera will capture the original image of the object when it moves, such as Fig.21 Image 1 shown in (b).

[0328] exist Fig.21 In Example 2 shown in (a) of FIG. 1 , when the photographed object moves to position 2105, it is within the first field of view of the main camera. At this time, the main camera will capture the original image of the photographed object when it moves, such as Fig.21 Image 2 shown in (b).

[0329] In Example 2, position 2104 can be understood as the position of the photographed object at the last moment of position 2105, which is indicated by a dotted line and corresponds to position 2103 in Example 1. It can be understood that the photographed object moves from position 2104 to position 2105.

[0330] exist Fig.21 In Example 3 shown in (a) of FIG. 1 , when the photographed object moves to position 2108, it is within the first field of view of the main camera. At this time, the main camera will capture the original image of the photographed object when it moves, such as Fig.21 Image 3 shown in (b).

[0331] Among them, in Example 3, position 2107 can be understood as the position of the photographed object at the previous moment of position 2108, represented by a dotted line, and corresponds to position 2105 in Example 2, and position 2106 can be understood as the position of the photographed object at the previous two moments of position 2108, represented by a dotted line, and corresponds to position 2104 in Example 2, and corresponds to position 2103 in Example 1. It can be understood that the photographed object moves from position 2106 to position 2107, and then to position 2108.

[0332] It should be noted that the three examples illustrated above are the framing frames and captured images corresponding to the continuous motion of the subject. In actual operation, there is no limit on the number of frames for capturing the original image, but what needs to be satisfied is that the original image is captured based on the continuous motion of the subject.

[0333] S1607: Extract the area containing the photographed object from the original image to generate an intermediate image.

[0334] The intermediate image refers to an image generated after extracting the area containing the photographed object from the original image. The extraction of the area containing the photographed object can be completed by an image processing algorithm, and operations such as background subtraction and edge detection are performed on the photographed object to generate an image that only includes the area containing the photographed object. Therefore, by extracting the area containing the photographed object from the original image, unnecessary losses generated when the terminal device performs image fusion operations can be reduced. At the same time, by extracting the area containing the photographed object, the interference of the background content in the original image on the image processing can be reduced.

[0335] Combine the following Fig. 22 The process of extracting a region including a photographed object is described as an example.

[0336] like Fig. 22 As shown in (a), assuming that there are original images 1, 2 and 3 captured by the main camera, the three frames of images may include a photographed object 2201 and background content 2202 that is unrelated to the photographed object.

[0337] like Fig. 22 As shown in (b), after the area including the photographed object is extracted from image 1, image 4 is generated; after the area including the photographed object is extracted from image 2, image 5 is generated; after the area including the photographed object is extracted from image 3, image 6 is generated.

[0338] Among them, image 4, image 5 and image 6 only include the photographed object 2201. At this time, image 4, image 5 and image 6 can all be understood as intermediate images.

[0339] It should be noted that, when extracting the area containing the photographed object, the image processing algorithm used can be determined according to the specific situation, and the embodiments of the present application do not limit this.

[0340] S1608: Perform image fusion processing on the intermediate image to generate a real-time rendering image.

[0341] After obtaining multiple frames of intermediate images, in order to clearly present the motion state of the photographed object at different times, it is necessary to perform image fusion processing on the multiple frames, and the fused image is a real-time rendering image. Among them, using the black frame and the intermediate image for fusion processing or using the gray frame and the intermediate image for fusion processing can ensure that the focus of the final image is on the photographed object without being disturbed by the background content.

[0342] It can be understood that a black frame refers to a completely dark frame in an image sequence, and its pixel value is basically zero. During image fusion, a black frame can be used for background subtraction. A gray frame has a certain grayscale value, and its pixel value distribution is usually relatively uniform. During the image fusion process, a gray frame can be used for brightness calibration and balance. Therefore, by fusing a black frame with an intermediate image, background interference can be avoided. At the same time, by fusing a gray frame with an intermediate image, brightness calibration and balance can be used.

[0343] Combine the following Fig.23 The process of image fusion is introduced exemplarily.

[0344] like Fig.23 As shown, assuming that there are intermediate images 1, 2, and 3, when image fusion processing is performed on the three frames, image 4 can be inserted between image 1 and image 2, and image 5 can be inserted between image 2 and image 3. Image 1, image 2, image 3, image 4, and image 5 are fused to obtain image 6. It can be understood that image 6 is a real-time rendered image.

[0345] In one embodiment, image 4 and image 5 are both black frames, and image 6 is a fusion result image of the intermediate image and the black frame; in another embodiment, image 4 and image 5 are both gray frames, and image 6 is a fusion result image of the intermediate image and the gray frame.

[0346] It should be noted that the number of black frames or gray frames inserted between image frames can be determined according to actual needs and is not limited in this application.

[0347] S1609: Displaying a real-time rendered image in the shooting interface.

[0348] After the intermediate image is fused, the real-time rendered image can be displayed in the shooting interface of the terminal device. Therefore, setting the real-time rendered image in the shooting interface can help users browse the shooting effect in real time, thereby improving user friendliness.

[0349] Combine the following Fig.24 An exemplary introduction to the effect of displaying a real-time rendered image is given.

[0350] like Fig.24 As shown, assuming that the terminal device turns on the stroboscopic photography mode, the shooting interface of the terminal device will display a real-time rendering image 2401. According to the real-time rendering image, the motion trajectory of the shooting object can be observed in real time.

[0351] S1610, determine whether the field of view of the main camera contains the photographed object, if so, execute S1603, if not, execute S1611.

[0352] When the subject is not within the field of view of the main camera, the main camera can no longer shoot the subject, so the flash needs to stop flashing. Therefore, by judging whether the subject is within the field of view of the main camera, the loss of the terminal device can be saved, and the waste of resources caused by turning on the flash when the main camera can no longer shoot the subject can be avoided.

[0353] Combine the following Fig.25 The process of determining whether the field of view of the main camera contains the photographed object is exemplarily introduced.

[0354] like Fig.25 As shown, assuming that the terminal device has turned on the stroboscopic photography mode, the shooting interface includes the viewfinder 2501 of the main camera, the viewfinder 2502 of the wide-angle camera, the shooting object and the picture in the viewfinder.

[0355] like Fig.25 As shown in (a), when the photographed object is at position 2503, which is within the field of view of the main camera 2501, a real-time rendering image 2504 is presented, and the wide-angle camera continues to collect marked images and subsequent operations.

[0356] like Fig.25 As shown in (b), when the object is at position 2505, which is not within the field of view of the main camera 2501, a real-time rendering image 2506 is presented, and the flash stops flashing. Since the main camera has finished shooting images, the real-time rendering image 2506 is the last real-time rendering image.

[0357] S1611: The flashlight stops flashing and outputs a stroboscopic image.

[0358] When the flash stops flashing, the subject will no longer be exposed and no image will be collected. At this time, the real-time rendered image when the flash is turned off can be understood as a stroboscopic image. Therefore, after turning off the flash, the stroboscopic image is output instead of collecting images, which can reduce unnecessary operations and improve the sensitivity of the solution.

[0359] like Fig.25 The real-time rendered image 2506 shown in (b) can be understood as the final output stroboscopic image.

[0360] In summary, this embodiment obtains the real-time motion trajectory of the photographed object by starting the stroboscopic photography mode. By collecting the marker image through the wide-angle camera, on the one hand, a wider scene can be captured, and on the other hand, the collected marker image can be used to calculate the stroboscopic frequency of the flash at the next moment, ensuring that the original image collected by the main camera can be clear; by collecting the original image through the main camera, a more accurate focus can be achieved on the photographed object. It can be understood that when photographing the photographed object, the flash can be used for auxiliary distance measurement to obtain a first distance. According to the first distance, the target power of the main flash of the flash is calculated. According to the target power of the flash and the stroboscopic frequency of the flash calculated by the marker image, the original image is collected when the main brightness reaches the peak. By using the combination of the main camera and the wide-angle camera, the motion state of the photographed object at different times can be captured, and the motion state at different times can be presented in the same image to enhance the visual effect of the image. Moreover, in the shooting interface, a real-time rendering image is provided, which enhances the user-friendliness of the present application solution.

[0361] Based on the above embodiments, Fig.26 The focusing processing method provided in this application is further explained. Fig.26 A schematic diagram of the flow of the focus processing method provided in an embodiment of the present application.

[0362] S2601. In response to a first operation of triggering shooting, control a flash of a terminal device to perform a first lighting operation.

[0363] In this embodiment, the first operation is used to trigger shooting, for example, it may be the operation of the user clicking the shutter as described in the above embodiment.

[0364] Exemplarily, the specific implementation of the first operation in response to triggering shooting may refer to the introduction of S801 in the above embodiment, and will not be repeated here.

[0365] In this embodiment, the first lighting operation is used to obtain the distance between the photographed object and the terminal device, for example, it may be the pre-flash of starting the flash light as described in the above embodiment.

[0366] Exemplarily, the specific implementation of controlling the flashlight of the terminal device to perform the first lighting operation may refer to the introduction of S803 in the above embodiment, and will not be repeated here.

[0367] S2602: Acquire a first distance based on a first lighting operation of the flash, where the first distance is the distance between the photographed object and the terminal device.

[0368] In one implementation, the first distance is obtained based on the first lighting operation of the flash, for example, in the following manner:

[0369] First, based on the first lighting operation of the flash, the first moment of lighting the flash is acquired; and,

[0370] Acquire a second moment when the light of the flash lamp returns to the terminal device, wherein the light of the flash lamp returns to the terminal device after being reflected by the photographed object;

[0371] The second moment when the light of the flash returns to the terminal device is obtained, for example, after the first moment, based on the first sensor for detecting exposure in the terminal device, whether the exposure parameter changes. In the case where the exposure parameter changes, the second moment is obtained, and the second moment is the moment when the change in the exposure parameter is detected.

[0372] Exemplarily, the specific implementation of obtaining the second moment when the light of the flash lamp returns to the terminal device can refer to the introduction of S804 in the above embodiment, and will not be repeated here.

[0373] Secondly, the first duration is determined according to the first moment and the second moment.

[0374] In the embodiment of the present application, the first duration is the time difference between the first moment and the second moment, for example, it can be the time difference between the light emitted by the pre-flash of the flashlight introduced in the above embodiment from the emission to the return of the terminal device. Exemplarily, the specific implementation of the current step can refer to the introduction of S805 in the above embodiment, and will not be repeated here.

[0375] Finally, the first distance is obtained according to the first duration and the speed corresponding to the light.

[0376] Exemplarily, the specific implementation of the current step may refer to the introduction of S806 in the above embodiment, and will not be repeated here.

[0377] After executing the first lighting operation, it is also necessary to obtain first brightness information, which is brightness information detected by a preset sensor in the terminal device.

[0378] Exemplarily, the specific implementation of the current step may refer to the introduction of S1202 in the above embodiment, and will not be repeated here.

[0379] S2603: Adjust shooting parameters according to the first distance, where the shooting parameters include a lens position.

[0380] In one implementation, the shooting parameters are adjusted according to the first distance, for example, in the following manner:

[0381] First, according to the first distance, a target value of a shooting parameter is determined.

[0382] In the process of determining the target value of the shooting parameter, for example, the target value of the lens position can be determined based on the first distance and the focal length of the lens in the terminal device.

[0383] Exemplarily, the specific implementation of the current step may refer to the introduction of S807 in the above embodiment, and will not be repeated here.

[0384] According to the first brightness information, a convergence value of the exposure parameter and a convergence value of the white balance parameter are determined.

[0385] Exemplarily, the specific implementation of the current step may refer to the introduction of S1203 in the above embodiment, and will not be repeated here.

[0386] The target power corresponding to the second lighting operation is determined according to the first distance, and the first distance is in direct proportion to the target power.

[0387] In this embodiment, the second lighting operation is used for exposure when capturing an image, for example, it may be the main flash of the flashlight as described in the above embodiment.

[0388] Exemplarily, the specific implementation of the current step may refer to the introduction of S1204 in the above embodiment, and will not be repeated here.

[0389] According to the target power, second brightness information of the flash lamp when performing a second lighting operation is predicted.

[0390] Exemplarily, the specific implementation of the current step may refer to the introduction of S1205 in the above embodiment, and will not be repeated here.

[0391] The convergence value of the exposure parameter and the convergence value of the white balance parameter are adjusted based on the second brightness information to obtain the target value of the exposure parameter and the target value of the white balance parameter.

[0392] Exemplarily, the specific implementation of the current step may refer to the introduction of S1206 in the above embodiment, and will not be repeated here.

[0393] Secondly, the flash is controlled to perform a second lighting operation, and during the second lighting operation, when it is determined that the brightness of the flash has reached a preset peak value, the shooting parameter is adjusted based on the target value of the shooting parameter. The flash is controlled to perform the second lighting operation according to the target power.

[0394] For example, the specific implementation of controlling the flash to perform the second lighting operation can refer to the introduction of S1207 in the above embodiment, which is not repeated here. The specific implementation of adjusting the shooting parameters based on the target values ​​of the shooting parameters can refer to the introduction of S1208 in the above embodiment, which is not repeated here.

[0395] S2604: Capture an image based on the adjusted shooting parameters to obtain a first image.

[0396] In one implementation, image acquisition is performed based on the adjusted shooting parameters. For example, the specific implementation of the current step can refer to the introduction of S1209 in the above embodiment, which will not be repeated here.

[0397] After the captured image is obtained, the main flash of the flash is turned off and the first image is output. Exemplarily, the specific implementation of the current step can refer to the introduction of S1210 in the above embodiment, which will not be repeated here.

[0398] In this embodiment, a focus processing method is provided, which can solve the focus problem in a dark environment at a low cost. It can be understood that by using a flash to emit light, recording the first moment when the flash is activated and the second moment when the AE change is detected, the first duration is calculated, and the first distance between the terminal device and the photographed object is calculated according to the first duration and the speed of light, and the lens focus is adjusted according to the first distance to obtain a first image. The embodiment of the present application uses the existing component of the flash to solve the focus problem in a dark environment and reduce the cost of solving the problem.

[0399] At the same time, by using the first distance, dynamically adjusting the target power of the flash when starting the main flash can ensure that the image exposure is appropriate, neither overexposed nor underexposed, and starting the flash according to the target power can avoid unnecessary damage to the terminal device. When the main brightness reaches its peak, adjusting the parameters to the target value can ensure that when the exposure conditions meet the requirements, the image parameters are also optimal.

[0400] Based on the above embodiments, Fig. 27 The stroboscopic image acquisition method provided in the present application is further described. Fig. 27 A schematic diagram of the flow of the stroboscopic photography method provided in an embodiment of the present application.

[0401] S2701. When the shooting mode triggered by the first operation is a stroboscopic mode, a start instruction is sent to a first camera and a second camera of the terminal device respectively, and the field of view of the first camera is greater than the field of view of the second camera.

[0402] Exemplarily, the specific implementation of the first operation triggering the stroboscopic mode may refer to the introduction of S1601 in the above embodiment, and will not be repeated here.

[0403] In the embodiment of the present application, the first camera may be, for example, the wide-angle camera introduced in the above embodiment, and the second camera may be, for example, the main camera introduced in the above embodiment.

[0404] The specific implementation of sending the start instruction to the first camera and the second camera of the terminal device respectively can refer to the introduction of S1602 in the above embodiment, and will not be repeated here.

[0405] S2702: When the photographed object enters the field of view of the first camera, sequentially collect multiple marked images based on the first camera, and dynamically determine the flash frequency of the flash in the process of collecting the marked images.

[0406] In one implementation, the flash frequency of the flashlight is dynamically determined by collecting the marker image, for example, in the following manner:

[0407] First, in the process of capturing the marked image, the real-time moving speed of the photographed object is determined based on the first marked image and the second marked image, the first marked image being the original image closest to the capture time, and the second marked image being the marked image adjacent to the first marked image.

[0408] In the embodiment of the present application, the marked image may be, for example, an image captured by the wide-angle camera introduced in the above embodiment, and the original image may be, for example, an image captured by the main camera introduced in the above embodiment.

[0409] Exemplarily, the specific implementation of collecting the marked image may refer to the introduction of S1603 in the above embodiment, and will not be described in detail here.

[0410] Secondly, the flash frequency of the flash is dynamically determined according to the real-time moving speed of the photographed object, and the flash frequency is directly proportional to the real-time moving speed.

[0411] Exemplarily, the specific implementation of determining the flash frequency of the flash light may refer to the introduction of S1604 in the above embodiment, and will not be described in detail here.

[0412] S2703: When the photographed object enters the field of view of the second camera, control the flash to flash continuously based on the dynamically determined flash frequency, and in the process of the flash continuing to flash, sequentially collect multiple original images based on the second camera.

[0413] For example, the specific implementation of controlling the flash light to flash continuously based on the dynamically determined flash frequency can refer to the introduction of S1605 in the above embodiment, which will not be described in detail here. The specific implementation of sequentially collecting multiple original images based on the second camera can refer to the introduction of S1606 in the above embodiment, which will not be described in detail here.

[0414] S2704: Generate a stroboscopic image based on the multiple original images.

[0415] In one implementation, the stroboscopic image may be generated, for example, in the following manner:

[0416] First, for any one of the plurality of original images, an object extraction is performed on a photographed object contained in the original image to generate an intermediate image containing the photographed object.

[0417] Exemplarily, the specific implementation of object extraction for the photographed object contained in the original image may refer to the introduction of S1607 in the above embodiment, and will not be repeated here.

[0418] Secondly, the intermediate images corresponding to the multiple original images are fused to obtain a real-time rendered image.

[0419] Exemplarily, the specific implementation of performing fusion processing on multiple intermediate images to obtain a real-time rendered image may refer to the introduction of S1608 in the above embodiment, and will not be repeated here.

[0420] Then, a real-time rendering image is displayed in the first area of ​​the shooting interface. The real-time rendering image is an intermediate result image generated in the process of sequentially fusing multiple intermediate images.

[0421] Exemplarily, the specific implementation of displaying the real-time rendered image in the first area of ​​the shooting interface can refer to the introduction of S1609 in the above embodiment, and will not be repeated here.

[0422] Finally, when the duration range of the second camera does not include the photographed object, the flash is controlled to stop flashing to obtain a stroboscopic image.

[0423] Exemplarily, the specific implementation of determining whether the time range of the second camera includes the photographed object can refer to the introduction of S1610 in the above embodiment, and will not be repeated here.

[0424] When it is determined that the duration range of the second camera does not include the photographed object, the flash stops flashing and outputs a stroboscopic image. The specific implementation of the current step can refer to the introduction of S1611 in the above embodiment, and will not be repeated here.

[0425] In this embodiment, when the user selects the stroboscopic photography mode, a motion state diagram of the photographed object at different times can be provided, which improves the visual effect of the image. Moreover, a real-time rendered image is displayed on the shooting interface, which improves the user-friendliness of the present application method. It can be understood that a wide-angle camera is used to obtain a wider field of view, and a main camera is used to obtain higher-definition details. Combining a wide-angle camera with a main camera can ensure that the output stroboscopic image has both details and a relatively comprehensive display of the motion trajectory of the photographed object.

[0426] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0427] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0428] The focus processing method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above focus processing method.

[0429] The focus processing method provided in the embodiment of the present application can be applied to an electronic device with a shooting function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the above related description, which will not be repeated here.

[0430] In one implementation, an embodiment of the present application provides an electronic device, Fig.28 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0431] like Fig.28 As shown, the electronic device 280 includes: a processor 2801 and a memory 2802; the memory 2802 stores computer-executable instructions; the processor 2801 executes the computer-executable instructions stored in the memory 2802, so that the electronic device 280 executes the above method.

[0432] When the memory 2802 is independently provided, the electronic device further includes a bus 2803 for connecting the memory 2802 and the processor 2801 .

[0433] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0434] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0435] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0436] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0437] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0438] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A focusing processing method, characterized in that: Applied to terminal equipment, including: In response to a first operation of triggering shooting, controlling a flash of the terminal device to perform a first lighting operation; Acquire a first distance based on a first lighting operation of the flash, where the first distance is the distance between the photographed object and the terminal device; adjusting shooting parameters according to the first distance, the shooting parameters including lens position; Image acquisition is performed based on the adjusted shooting parameters to obtain a first image.

2. The method according to claim 1, characterized in that The acquiring a first distance based on a first lighting operation of the flash lamp comprises: Based on a first lighting operation of the flashlight, acquiring a first moment of lighting of the flashlight; and Acquire a second moment when the light of the flash lamp returns to the terminal device, wherein the light of the flash lamp returns to the terminal device after being reflected by the photographed object; Determine a first duration according to the first moment and the second moment; The first distance is acquired according to the first duration and a speed corresponding to the light.

3. The method according to claim 2, characterized in that The obtaining of the second moment when the light of the flash lamp returns to the terminal device includes: After the first moment, based on a first sensor for detecting exposure in the terminal device, detecting whether an exposure parameter changes; In the case where the exposure parameter changes, the second moment is acquired, where the second moment is the moment when the change of the exposure parameter is detected.

4. The method according to any one of claims 1 to 3, characterized in that: The adjusting the shooting parameters according to the first distance includes: determining a target value of the shooting parameter according to the first distance; controlling the flash to perform a second lighting operation, and in the process of performing the second lighting operation, adjusting the shooting parameter based on the target value of the shooting parameter; Image acquisition is performed based on the adjusted shooting parameters to obtain a first image.

5. The method according to claim 4, characterized in that After performing the first lighting operation, the method further includes: Acquire first brightness information, where the first brightness information is brightness information detected by a preset sensor in the terminal device.

6. The method according to claim 5, characterized in that The shooting parameters also include exposure parameters and white balance parameters; The step of determining a target value of a shooting parameter according to the first distance includes: Determining a target value of the lens position according to the first distance and a focal length of a lens in the terminal device; Determining a convergence value of the exposure parameter and a convergence value of the white balance parameter according to the first brightness information; determining a target power corresponding to the second lighting operation according to the first distance, wherein the first distance is in direct proportion to the target power; According to the target power, the second brightness information of the flash when performing the second lighting operation is predicted, and the convergence value of the exposure parameter and the convergence value of the white balance parameter are adjusted based on the second brightness information to obtain the target value of the exposure parameter and the target value of the white balance parameter.

7. The method according to claim 6, characterized in that The controlling the flash lamp to perform a second lighting operation includes: According to the target power, the flash lamp is controlled to perform the second lighting operation.

8. The method according to claim 7, characterized in that The step of adjusting the shooting parameter based on the target value of the shooting parameter during the execution of the second lighting operation includes: During the execution of the second lighting operation, when it is determined that the brightness of the flash has reached a preset peak value, each of the shooting parameters is adjusted to a corresponding target value.

9. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the shooting mode triggered by the first operation is a stroboscopic mode, sending a start instruction to a first camera and a second camera of the terminal device respectively, and the field of view of the first camera is greater than the field of view of the second camera; When the photographed object enters the field of view of the first camera, sequentially collecting a plurality of marked images based on the first camera, and dynamically determining the flash frequency of the flash light during the process of collecting the marked images; When the photographed object enters the field of view of the second camera, the flash is controlled to flash continuously based on the dynamically determined flash frequency, and during the process of the flash continuously flashing, a plurality of original images are sequentially collected based on the second camera; A stroboscopic image is generated according to the plurality of original images.

10. The method according to claim 9, characterized in that The step of dynamically determining the flash frequency of the flash lamp during the process of acquiring the marked image includes: In the process of acquiring the marked image, determining the real-time moving speed of the photographed object according to a first marked image and a second marked image, wherein the first marked image is an original image at a most recent acquisition time, and the second marked image is a marked image adjacent to the first marked image; The flash frequency of the flashlight is dynamically determined according to the real-time moving speed of the photographed object, and the flash frequency is directly proportional to the real-time moving speed.

11. The method according to claim 9, characterized in that The method further comprises: When the duration range of the second camera does not include the photographed object, the flash is controlled to stop flashing.

12. The method according to claim 9, characterized in that The step of generating a stroboscopic image according to the plurality of original images comprises: For any one of the plurality of original images, extracting a photographed object contained in the original image to generate an intermediate image containing the photographed object; A fusion process is performed on the intermediate images corresponding to the plurality of original images to obtain a stroboscopic image.

13. The method according to claim 12, characterized in that The method further comprises: A real-time rendering image is displayed in the first area of ​​the shooting interface. The real-time rendering image is an intermediate result image generated by sequentially fusing the intermediate images corresponding to the multiple original images.

14. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the method as claimed in any one of claims 1 to 13.

15. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes any one of the methods described in claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device is caused to perform the method as claimed in any one of claims 1 to 13.

17. A computer program product, characterized in that The computer program product comprises a computer program code, and when the computer program code is run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 13.

Images