Automatic focusing method and electronic equipment

By setting the obscured pixel points on the camera photosensitive device, obtaining and preprocessing the phase difference data, and calculating the lens movement distance, the problem of poor autofocus effect in bright screens or luminescent scenes is solved, and fast and accurate autofocus is achieved. Reduces the push of the focus motor and improves imaging quality.

CN117135449BActive Publication Date: 2024-11-15HONOR DEVICE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310079207.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-01-13
Publication Date
2024-11-15
Estimated Expiration
2043-01-13

AI Technical Summary

Technical Problem

The automatic focus effect of the camera when taking pictures may be poor in bright screens or luminescent objects scenes, resulting in frequent adjustment of the focus motor, causing the focus to pull and affect the imaging quality.

Method used

By setting the obscured pixel points on the photosensitive device, obtaining phase difference data, counting the phase difference value of multi-frame images, calculating the moving distance of the lens, and pre-processing the PD value using the weighted average value to reduce the drastic changes in the PD value and avoiding focus The motor pushes back and forth for fast and accurate autofocus.

Benefits of technology

It effectively reduces the back and forth push of the focus motor, improves the accuracy and stability of autofocus, avoids the pull-out phenomenon, and improves the imaging effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117135449B_ABST
    Figure CN117135449B_ABST
Patent Text Reader

Abstract

The present application discloses an automatic focusing method and electronic device, and relates to the field of terminal technology. The electronic device includes a camera, and the camera includes a lens and a photosensitive device. The left and right sides of the photosensitive device are respectively provided with shielded pixels, and these shielded pixels are used to obtain the phase difference when the camera collects images. After the camera starts the photo preview, the phase difference data of multiple frames of images are fused as the preprocessed phase difference data of one frame of image; then the moving distance of the lens is calculated according to the preprocessed phase difference data to achieve automatic focusing. The change of phase difference is reduced, and the influence of the drastic change of phase difference on the moving position of the lens is reduced, and the focus pulling phenomenon caused by the focus motor driving the lens to push back and forth is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an automatic focusing method and electronic device. Background Art

[0002] The camera will perform auto focus (AF) when taking pictures. The camera can use the focus motor to drive the lens to adjust the focus distance so that the image of the subject is clear. In some scenes, such as when the subject is a bright screen or a light, the auto focus effect may be poor. In these scenes, some cameras will zoom continuously and cannot complete the focus. Some cameras will zoom multiple times, resulting in a pull focus phenomenon, and finally adjust to a blurred focus distance.

[0003] How to automatically focus the camera to avoid the focus pulling phenomenon and achieve clear imaging effects is a problem that needs to be solved. Summary of the invention

[0004] The embodiments of the present application provide an autofocus method and an electronic device, which can reduce the back and forth pushing of the focus motor during the autofocus process, avoid the focus pulling phenomenon, and achieve fast and accurate focusing.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, an autofocus method is provided, which is applied to an electronic device, wherein the electronic device includes a camera, the camera includes a lens and a photosensitive device, and shielded pixels are respectively arranged on the left and right sides of the photosensitive device. After the camera is started to preview a photo, phase difference data when the camera captures each frame of an image is obtained; the phase difference data is the value of the phase difference of the image of the subject through the shielded pixels on the left and right sides of the photosensitive device; then, according to the phase difference data of the i-th frame image captured by the camera and N (N is greater than or equal to 1) frames of images before the i-th frame image is captured by the camera, the movement distance of the lens is obtained; and the lens is moved according to the calculated movement distance to achieve autofocus.

[0007] In this method, the phase difference (PD) values ​​of multiple frames are counted, and the moving distance of the lens is calculated based on the phase difference (PD) values ​​of multiple frames. In this way, the influence of the drastic change of the PD value on the moving position of the lens can be reduced, thereby avoiding the focus motor driving the lens to push back and forth and reducing the phenomenon of pulling focus.

[0008] In combination with the first aspect, in one embodiment, the movement distance of the lens is obtained based on the phase difference data between the i-th frame image captured by the camera and the N-frame image image captured by the camera before the i-th frame image, including: calculating the weighted average of the phase difference data between the i-th frame image captured by the camera and the N-frame image image captured before the i-th frame image, and obtaining the movement distance of the lens based on the weighted average.

[0009] In this method, the weighted average of the PD values ​​of the N+1 frames of images is taken as the PD value of a frame of image after preprocessing. The PD values ​​of multiple frames of images are averaged, so that the change of the PD value after processing is reduced, avoiding the drastic change of the PD value.

[0010] In combination with the first aspect, in one embodiment, the camera's photo preview interface includes a focus frame, the phase difference data is the phase difference value corresponding to the focus frame, and the weighted average of the phase difference data of the i-th frame image captured by the camera and the N frames of images before the i-th frame image is calculated, including: calculating the weighted average of the phase difference value of the focus frame when the camera captures the i-th frame image and the N frames of images before the i-th frame image.

[0011] In this method, the camera driver outputs the PD value of the entire focus frame, and directly averages the PD values ​​of the focus frames of multiple frames.

[0012] In combination with the first aspect, in one embodiment, the camera's photo preview interface includes a focus frame, the focus frame includes multiple sub-windows, the phase difference data includes phase difference values ​​corresponding to each sub-window of the focus frame, and the weighted average of the phase difference data of the i-th frame image and N frames of images before the i-th frame image are calculated by the camera, including: for the first sub-window of the focus frame (the first sub-window is any sub-window of the focus frame), calculating the weighted average of the phase difference values ​​of the first sub-window when the camera captures the i-th frame image and N frames of images before the i-th frame image; and then calculating the weighted average of the weighted averages of the phase difference values ​​of each sub-window of the i-th frame image.

[0013] In this method, the weighted average of the phase differences of each sub-window in multiple frames of images is first calculated, and then the PD values ​​of each sub-window after preprocessing (i.e., calculating the weighted average of the phase differences) in a frame of image are averaged to obtain the preprocessed PD value of the frame of image.

[0014] In combination with the first aspect, in one implementation, the i-th frame image is an image captured after a first preset time period after the camera is started.

[0015] Within a short period of time (within the first preset time period) after the camera is just started, the PD value obtained is still unstable. This is normal and does not require preprocessing. The PD data of the image frames collected within the first preset time period after the camera is started can be discarded, and the PD data of the image frames collected after the first preset time period after the camera is started can be directly preprocessed.

[0016] In combination with the first aspect, in one embodiment, each phase difference data corresponds to a confidence level, and the higher the confidence level, the more reliable the PD data. Phase difference data with a confidence level greater than a preset threshold can be used as valid phase difference data, and phase difference data with a confidence level less than or equal to the preset threshold can be used as invalid phase difference data. According to the phase difference data of the i-th frame image and the N frames before the i-th frame image collected by the camera, the movement distance of the lens is obtained, including: according to the effective phase difference data of the i-th frame image and the N frames before the i-th frame image collected by the camera, the movement distance of the lens is obtained. In this way, unreliable PD data can be discarded, so that the acquired focus position is more accurate.

[0017] In combination with the first aspect, in one embodiment, before obtaining the moving distance of the lens according to the phase difference data after preprocessing (calculating the weighted average) of the i-th frame image, the method further includes: obtaining the difference between the phase difference data after preprocessing the i-th frame image and the phase difference data after preprocessing the i-1-th frame image. The difference can indicate the fluctuation of the PD value after preprocessing, and the smaller the difference, the smaller the fluctuation of the PD value after preprocessing; the larger the difference, the larger the fluctuation of the PD value after preprocessing.

[0018] In combination with the first aspect, in one embodiment, obtaining the movement distance of the lens based on the phase difference data after preprocessing the i-th frame image includes: if the difference is less than or equal to a preset fluctuation threshold, obtaining the movement distance of the lens based on the phase difference data after preprocessing the i-th frame image.

[0019] In this method, if it is determined that the difference between the preprocessed PD values ​​of two adjacent frames of images is less than or equal to a preset fluctuation threshold, it indicates that the fluctuation of the preprocessed PD value is small. Using the preprocessed PD value to calculate the moving distance of the lens can avoid causing the focus motor to be pushed back and forth.

[0020] In combination with the first aspect, in one embodiment, the electronic device may further include a depth sensor, which is used to measure the distance between the subject and the lens. If it is determined that the difference is greater than a preset fluctuation threshold, the depth sensor is used to obtain the position of the subject; and the movement distance of the lens is obtained based on the position of the subject.

[0021] In this method, if it is determined that the difference between the PD values ​​of two adjacent frames of images after preprocessing is greater than the preset fluctuation threshold, it indicates that the fluctuation of the preprocessed PD value is still quite violent. Using the preprocessed PD value to calculate the moving distance of the lens will still cause the focus motor to push back and forth. Therefore, instead of using the PD value to obtain the moving distance of the lens, a depth sensor is used to obtain the object distance of the subject, and the moving distance of the lens is calculated based on the object distance obtained by the depth sensor to achieve automatic focus. This can avoid the focus motor from pushing back and forth due to violent fluctuations in the PD value.

[0022] In combination with the first aspect, in one implementation, the depth sensor includes a camera array, a distance sensor, a structured light sensor, or a time-of-flight TOF sensor.

[0023] In a second aspect, an electronic device is provided, which has the function of implementing the method described in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0024] In a third aspect, an electronic device is provided, comprising: a processor, a memory and a camera; the camera comprises a lens and a photosensitive device, and shielded pixels are respectively arranged on the left and right sides of the photosensitive device; the memory is used to store computer execution instructions, and when the electronic device is running, the processor executes the computer execution instructions stored in the memory, so that the electronic device performs a method as described in any one of the above-mentioned first aspects.

[0025] In a fourth aspect, an electronic device is provided, comprising: a processor; the processor is used to couple with a memory, and after reading instructions in the memory, execute a method as described in any one of the above-mentioned first aspects according to the instructions.

[0026] In a fifth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on a computer, the computer can execute any of the methods described in the first aspect.

[0027] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect.

[0028] In a seventh aspect, a device (for example, the device may be a chip system) is provided, the device including a processor for supporting an electronic device to implement the functions involved in the first aspect above. In one possible design, the device also includes a memory for storing program instructions and data necessary for the electronic device. When the device is a chip system, it may be composed of a chip, or may include a chip and other discrete devices.

[0029] Among them, the technical effects brought about by any design method in the second to seventh aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a scenario in which the camera automatically focuses when taking photos;

[0031] Figure 2 A schematic diagram of another scenario of automatic focusing when taking photos with a camera;

[0032] Figure 3 A schematic diagram of a scene example when taking photos with a camera;

[0033] Figure 4 A schematic diagram of a scenario example applicable to the auto-focus method provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of a flow chart of an auto-focusing method provided in an embodiment of the present application;

[0037] Figure 8 Another schematic diagram of the process of the automatic focusing method provided in the embodiment of the present application;

[0038] Fig. 9 A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include such expressions as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are a kind of "or" relationship.

[0040] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0041] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0042] In some scenarios, the camera's autofocus effect is poor. For example, Figure 1 Figure 1 shows a scenario where the camera automatically focuses when taking pictures. Figure 1 As shown, the user uses the camera of the mobile phone 100 to take a photo of the large screen, and the subject is the large screen with a bright screen. For example, Figure 2 FIG. 4 shows another scenario in which the camera automatically focuses when taking pictures. Figure 2As shown, the user uses the camera of the mobile phone 100 to take a close-up photo of a lighted lamp, and the object is the lighted lamp. In these scenes, the auto focus effect may be poor.

[0043] In some cameras, phase detection auto focus (PDAF) is used for auto focus, that is, auto focus is achieved through phase detection. In the PDAF method, some shield pixels (SP) are reserved on the left and right sides of the camera's photosensitive device for phase detection. The distance and change of the image of the object through the shielded pixels on the left and right sides are detected to determine the focus offset, thereby achieving accurate focus.

[0044] Specifically, the control unit of the camera (such as the camera driver) obtains the phase difference of the SP images of the left and right sides of the object on the photosensitive device, and maps the object to a certain position on the lens moving axis according to the phase difference of the SP images of the left and right sides of the object; that is, the detected phase difference (phase difference, PD) of the SP images of the left and right sides of the object on the photosensitive device is converted into a defocus value (defocus value, DV). In one implementation, DV = PD*DCC; wherein DCC is a defocus conversion coefficient, which is a fixed parameter of the camera (camera intrinsic parameter). The control unit of the camera can obtain the DV value according to the detected PD value and the value of the camera intrinsic parameter DCC. The control unit of the camera transmits the DV value to the focus motor of the camera. The focus motor of the camera drives the lens of the camera to move forward or backward according to the DV value, so that the lens of the camera adjusts the focal length according to the DV value to achieve automatic focus.

[0045] In the prior art, a focus frame is generally set during autofocusing, and the focus frame can be divided into a center frame or a face frame. Figure 3 As shown, the photo preview interface of the mobile phone 100 displays a focus frame 101. It should be noted that in some implementations, the focus frame can be displayed on the photo preview interface. In other implementations, the focus frame may not be displayed on the interface.

[0046] The focus frame can be divided into multiple sub-windows. For example, a central frame is divided into 9 sub-windows of 3*3. The camera can calculate the PD value of each sub-window separately, and finally select the PD value of one of the multiple sub-windows or the entire focus frame as the final focus process.

[0047] In some scenarios, such as the above Figure 1 or Figure 2In the photo-taking scene shown, the PD value acquired by the camera changes dramatically. Figure 4 The diagram is a schematic diagram of the PD value changes of the 9 sub-windows. Figure 4 As shown, there are 9 subwindows from subwindow 0 to subwindow 8, and the PD value of each subwindow changes dramatically with the frame number. For example, the PD value of subwindow 7 in the first frame is 0.1, the PD value in the second frame is 5, the PD value in the third frame is 0, the PD value in the fourth frame is 5, the PD value in the fifth frame is 9, ..., the PD value in the tenth frame is -3, ..., the PD value in the thirteenth frame is 10, ....

[0048] In the prior art, the PD value of a sub-window or the entire focus frame is used for focusing. The camera calculates the corresponding DV value based on the PD value of each frame, and then the focus motor of the camera adjusts the focal length of the lens based on the DV value of each frame. Regardless of which sub-window is selected for focus, or the PD value of the entire focus frame is used for focus, since the PD value changes dramatically, the corresponding DV value also changes dramatically. The focus motor of the camera adjusts the focal length of the lens significantly in each frame, and the lens is constantly focusing, resulting in an obvious focus pulling phenomenon, which seriously affects the user experience.

[0049] The embodiment of the present application provides an autofocus method, which pre-processes the acquired PD value, reduces the change of the PD value between adjacent frames, avoids the focus motor from pushing back and forth, and optimizes the focusing process.

[0050] The method provided in the embodiment of the present application can be applied to electronic devices including autofocus cameras. The above electronic devices may include mobile phones, tablet computers, laptops, personal computers (PCs), ultra-mobile personal computers (UMPCs), handheld computers, netbooks, smart home devices (e.g., smart TVs, smart screens, large screens, smart speakers, smart air conditioners, etc.), personal digital assistants (PDAs), wearable devices (e.g., smart watches, smart bracelets, etc.), vehicle-mounted devices, virtual reality devices, etc., and the embodiments of the present application do not impose any restrictions on this.

[0051] like Figure 5, which is a schematic diagram of the structure of an electronic device 100. The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a display screen 191, an indicator 192, a camera 193, etc. The sensor module 180 may include a temperature sensor, an ambient light sensor, a distance sensor, a structured light sensor, a time of flight (TOF) sensor, etc.

[0052] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0053] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0054] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0055] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0056] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0057] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0058] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of an electronic device. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0059] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 191, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0060] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0061] The electronic device 100 implements the display function through a GPU, a display screen 191, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 191 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0062] The display screen 191 is used to display images, videos, etc. The display screen 191 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, Micro-OLED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc.

[0063] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 191 and application processor.

[0064] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0065] Camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1. In the embodiment of the present application, the camera 193 can be used to capture video images.

[0066] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency point, a digital signal processor is used to perform Fourier transform on the frequency point energy.

[0067] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. In this way, the electronic device can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0068] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0069] The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor, etc. For example, music playing, recording, etc. In the embodiment of the present application, the audio module 170 can be used to collect audio in recording video.

[0070] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 are arranged in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" and "microphone", is used to convert sound signals into electrical signals. The headphone interface 170D is used to connect wired headphones. The headphone interface 170D can be a USB interface 130, or it can be a 3.5mm open mobile electronic equipment platform (open mobile terminal platform, OMTP) standard interface, and the cellular telecommunications industry association of the USA (cellular telecommunications industry association of the USA, CTIA) standard interface.

[0071] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, audio, video and other files are stored in the external memory card.

[0072] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 can execute instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data (such as video files) created during the use of the electronic device, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0073] The button 190 includes a power button, a volume button, etc. The button 190 may be a mechanical button or a touch button. The indicator 192 may be an indicator light, which may be used to indicate the charging status, the change of the power, or may be used to indicate messages, missed calls, notifications, etc.

[0074] A temperature sensor is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

[0075] The ambient light sensor is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 191 according to the perceived brightness of the ambient light. The ambient light sensor can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor can also cooperate with the proximity light sensor to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0076] The distance sensor is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, such as shooting scenes, the electronic device 100 can use the distance sensor to measure the distance to achieve fast focusing.

[0077] The structured light sensor can calculate the position and depth of the object according to the change of the light signal caused by the object, and then restore the entire three-dimensional space. In some embodiments, the electronic device 100 can use the structured light sensor to measure the distance between the object and the electronic device 100. In the embodiment of the present application, the structured light sensor can be used to measure the distance between the object and the lens to achieve auto focus.

[0078] The TOF sensor is used to measure the time or phase difference between the light pulse and the object to be measured to obtain the distance and depth data of the object to be measured, thereby obtaining a three-dimensional model of the object to be measured. In the embodiment of the present application, the TOF sensor can be used to measure the distance between the object to be photographed and the lens to achieve autofocus.

[0079] In some embodiments, the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplarily described.

[0080] Figure 6 A software architecture diagram of an electronic device provided in an embodiment of the present application.

[0081] It is understandable that the layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, The system can include the application (App) layer, the application framework (FWK) layer, the hardware abstraction layer (HAL) and the kernel layer. Figure 6 As shown, The system may also include the Android runtime and system libraries.

[0082] The application layer can include a series of application packages. Figure 6 As shown, the application package may include applications such as camera, gallery, calendar, call, music, short message, etc. The camera application is used to take photos and videos.

[0083] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. The application framework layer provides programming services to the application layer through the API interface. Figure 6As shown, the application framework layer includes the media framework, media codec services, etc. The media framework is used to manage multimedia resources such as photos, pictures, videos, and audio. The media codec services are used to manage the codecs of multimedia resources such as photos, pictures, videos, and audio.

[0084] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0085] The core library consists of two parts: one part is the functional functions that Java voice needs to call, and the other part is the Android core library.

[0086] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0087] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0088] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0089] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0090] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing, etc.

[0091] A 2D graphics engine is a drawing engine for 2D drawings.

[0092] The hardware abstraction layer is used to abstract the hardware, encapsulate the kernel layer driver, and provide an interface to the upper layer. Figure 6 As shown, the hardware abstraction layer includes a camera abstraction layer, a display abstraction layer, an audio abstraction layer, etc.

[0093] The kernel layer provides low-level drivers for various hardware of electronic devices. For example, Figure 6 As shown, the kernel layer may include camera driver, display driver, audio driver, etc.

[0094] In an embodiment of the present application, the camera driver obtains the PD value of each frame of the image based on the phase difference between the images formed by the left and right sides SP of the camera photosensitive device; optionally, the PD value of each sub-window of the focus frame can be obtained separately. The camera abstraction layer obtains the PD value of each frame (or each sub-window in each frame) during the camera shooting process from the camera driver, pre-processes the obtained PD value, and calculates the focal length (defocus rate) of the lens based on the PD value. The camera abstraction layer can also calculate the movement distance L of the lens based on the focal length (defocus rate) of the lens and the current position of the lens. Furthermore, the camera abstraction layer transmits the calculated movement distance L to the focus motor of the camera. In this way, the focus motor drives the lens to move position to achieve automatic focus.

[0095] It is understandable that in other embodiments, the camera driver may also pre-process the acquired PD value and calculate the focal length (defocus rate) of the lens according to the PD value. The camera driver calculates the moving distance L of the lens according to the focal length (defocus rate) of the lens and the current position of the lens. Further, the camera driver transmits the calculated moving distance L to the focus motor of the camera. In this way, the focus motor drives the lens to move position to achieve automatic focus.

[0096] The following describes in detail the auto-focus method provided by the embodiments of the present application in conjunction with the accompanying drawings.

[0097] For example, Figure 7 A schematic diagram of a flow chart of the automatic focusing method provided in the embodiment of the present application. Figure 7 As shown, the method includes:

[0098] S701, the camera starts to take pictures and preview; the camera collects images in real time.

[0099] S702: The camera driver obtains the PD data when the camera captures each frame of image.

[0100] The camera may include components such as a lens and a photosensitive device. Among them, some shield pixels (SP) are reserved on the left and right sides of the photosensitive device. The phase difference (PD) between the images formed by the SP on the left and right sides when the lens captures a frame of image can be detected, and the PD data when the camera captures each frame of image can be obtained in real time.

[0101] In one implementation, a focus frame is set during autofocus, and the focus frame is a preset area within the lens framing area. The object that the focus frame is aimed at is used as the target object for autofocus. The purpose of autofocus is to make the target object image clear when taking pictures.

[0102] The focus frame can be divided into multiple sub-windows. Exemplarily, a focus frame is divided into 9 sub-windows of 3*3. When the camera captures a frame of image, the camera driver can obtain the PD value corresponding to each sub-window respectively, or obtain the PD value corresponding to the entire focus frame. Among them, the specific method of obtaining the PD value corresponding to the focus frame and each sub-window of the focus frame can adopt the method in conventional technology, and the embodiment of the present application does not limit this.

[0103] The PD data when the camera captures a frame of image may be the PD value corresponding to the entire focus frame when the frame of image is captured; or may include the PD values ​​corresponding to each sub-window of the focus frame when the frame of image is captured.

[0104] Exemplarily, a focus frame has 9 sub-windows, which are respectively recorded as sub-window 0, sub-window 1, sub-window 2, sub-window 3, sub-window 4, sub-window 5, sub-window 6, sub-window 7 and sub-window 8. The PD value corresponding to sub-window 0 is PD0, the PD value corresponding to sub-window 1 is PD1, the PD value corresponding to sub-window 2 is PD2, the PD value corresponding to sub-window 3 is PD3, the PD value corresponding to sub-window 4 is PD4, the PD value corresponding to sub-window 5 is PD5, the PD value corresponding to sub-window 6 is PD6, the PD value corresponding to sub-window 7 is PD7, and the PD value corresponding to sub-window 8 is PD8. The PD data of a frame of image may include PD0, PD1, PD2, PD3, PD4, PD5, PD6, PD7 and PD8.

[0105] S703, the camera driver determines whether the image frame is captured within the first preset time after the camera is started. If it is an image frame captured within the first preset time after the camera is started, the PD data of the frame image is not preprocessed; if it is not an image frame captured within the first preset time after the camera is started, the PD data of the frame image is preprocessed and S704 is executed.

[0106] Within a short period of time (within the first preset time period) after the camera is just started, the acquired PD value is not stable. In one implementation, the PD data of the image frames collected within the first preset time period after the camera is started can be discarded, and the PD data of the image frames collected after the first preset time period after the camera is started can be directly preprocessed.

[0107] S704, the camera driver determines the confidence of the PD data of each frame of image. If the confidence of the PD data is greater than a preset threshold, the PD data is determined to be valid PD data; if the confidence of the PD data is less than or equal to the preset threshold, the PD data is determined to be invalid PD data.

[0108] In one implementation, when the camera driver outputs the PD data, it also outputs the confidence level corresponding to the PD data. The confidence level is used to indicate the credibility of the PD data; the higher the confidence level, the more credible the PD data.

[0109] In one implementation, if it is determined that the confidence of the PD data is greater than a preset threshold, the PD data is determined to be valid PD data, and PD data preprocessing can be performed based on the PD data. If it is determined that the confidence of the PD data is less than or equal to the preset threshold, the frame PD data is determined to be invalid PD data, and the PD data is not used for PD data preprocessing.

[0110] S705 . The camera driver obtains the pre-processed PD data of the i-th frame image according to the valid PD data of the N frames of images before the i-th frame image and the valid PD data of the i-th frame image.

[0111] The i-th frame image is any frame among the image frames captured after the first preset time after the camera is started. According to the valid PD data of the N frames before the i-th frame image and the valid PD data of the i-th frame image, the PD data can be preprocessed to obtain the PD data of the i-th frame image after preprocessing. For example, the first 10 frames of images captured after the camera is started are unstable, and the PD data preprocessing is not performed on the first 10 frames of images, and the PD data preprocessing starts from the first 11 frames of images captured after the camera is started; that is, the i-th frame image is any frame after the 10th frame image captured after the camera is started.

[0112] Wherein, N is a preset value, and N is greater than or equal to 1; illustratively, the value of N is 2.

[0113] Exemplarily, PD data preprocessing may be performed based on the valid PD data of the 9th frame image, the valid PD data of the 10th frame image, and the valid PD data of the 11th frame image to obtain the preprocessed PD data of the 11th frame image; PD data preprocessing may be performed based on the valid PD data of the 10th frame image, the valid PD data of the 11th frame image, and the valid PD data of the 12th frame image to obtain the preprocessed PD data of the 12th frame image; PD data preprocessing may be performed based on the valid PD data of the 11th frame image, the valid PD data of the 12th frame image, and the valid PD data of the 13th frame image to obtain the preprocessed PD data of the 13th frame image; and so on, to obtain the preprocessed PD data of each frame image.

[0114] In one implementation, a weighted average of effective PD values ​​of N frames of images before the i-th frame of image and an effective PD value of the i-th frame of image may be calculated as the PD value of the i-th frame of image after preprocessing.

[0115] In one example, the PD data is the PD value corresponding to the entire focus frame. Take N=2 as an example. The PD value of the i-th frame image after preprocessing is the weighted average of the PD value of the entire focus frame of the i-2-th frame image, the PD value of the entire focus frame of the i-1-th frame image, and the PD value of the entire focus frame of the i-th frame image. For example, when the weighting coefficient of each item (each frame image) is 1, the PD value of the i-th frame image after preprocessing is (the PD value of the entire focus frame of the i-2-th frame image + the PD value of the entire focus frame of the i-1-th frame image + the PD value of the entire focus frame of the i-th frame image) / 3.

[0116] In one example, the PD data includes PD values ​​corresponding to each sub-window of the focus frame. Still taking N=2 as an example.

[0117] The preprocessed PD value of subwindow 0 of the i-th frame image is the weighted average of the PD value of subwindow 0 of the i-2-th frame image, the PD value of subwindow 0 of the i-1-th frame image, and the PD value of subwindow 0 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of subwindow 0 of the i-th frame image is (PD value of subwindow 0 of the i-2-th frame image + PD value of subwindow 0 of the i-1-th frame image + PD value of subwindow 0 of the i-th frame image) / 3.

[0118] The preprocessed PD value of subwindow 1 of the i-th frame image is a weighted average of the PD value of subwindow 1 of the i-2-th frame image, the PD value of subwindow 1 of the i-1-th frame image, and the PD value of subwindow 1 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of subwindow 1 of the i-th frame image is (PD value of subwindow 1 of the i-2-th frame image + PD value of subwindow 1 of the i-1-th frame image + PD value of subwindow 1 of the i-th frame image) / 3.

[0119] The preprocessed PD value of subwindow 2 of the i-th frame image is a weighted average of the PD value of subwindow 2 of the i-2-th frame image, the PD value of subwindow 2 of the i-1-th frame image, and the PD value of subwindow 2 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of subwindow 2 of the i-th frame image is (PD value of subwindow 2 of the i-2-th frame image + PD value of subwindow 2 of the i-1-th frame image + PD value of subwindow 2 of the i-th frame image) / 3.

[0120] The preprocessed PD value of the subwindow 3 of the i-th frame image is the weighted average of the PD value of the subwindow 3 of the i-2-th frame image, the PD value of the subwindow 3 of the i-1-th frame image, and the PD value of the subwindow 3 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of the subwindow 3 of the i-th frame image is (the PD value of the subwindow 3 of the i-2-th frame image + the PD value of the subwindow 3 of the i-1-th frame image + the PD value of the subwindow 3 of the i-th frame image) / 3.

[0121] The preprocessed PD value of the subwindow 4 of the i-th frame image is the weighted average of the PD value of the subwindow 4 of the i-2-th frame image, the PD value of the subwindow 4 of the i-1-th frame image, and the PD value of the subwindow 4 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of the subwindow 4 of the i-th frame image is (the PD value of the subwindow 4 of the i-2-th frame image + the PD value of the subwindow 4 of the i-1-th frame image + the PD value of the subwindow 4 of the i-th frame image) / 3.

[0122] The preprocessed PD value of the subwindow 5 of the i-th frame image is the weighted average of the PD value of the subwindow 5 of the i-2-th frame image, the PD value of the subwindow 5 of the i-1-th frame image, and the PD value of the subwindow 5 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of the subwindow 5 of the i-th frame image is (the PD value of the subwindow 5 of the i-2-th frame image + the PD value of the subwindow 5 of the i-1-th frame image + the PD value of the subwindow 5 of the i-th frame image) / 3.

[0123] The preprocessed PD value of the subwindow 6 of the i-th frame image is a weighted average of the PD values ​​of the subwindow 6 of the i-2-th frame image, the PD value of the subwindow 6 of the i-1-th frame image, and the PD value of the subwindow 6 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of the subwindow 6 of the i-th frame image is (the PD value of the subwindow 6 of the i-2-th frame image + the PD value of the subwindow 6 of the i-1-th frame image + the PD value of the subwindow 6 of the i-th frame image) / 3.

[0124] The preprocessed PD value of the subwindow 7 of the i-th frame image is the weighted average of the PD values ​​of the subwindow 7 of the i-2th frame image, the PD value of the subwindow 7 of the i-1th frame image, and the PD value of the subwindow 7 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of the subwindow 7 of the i-th frame image is (the PD value of the subwindow 7 of the i-2th frame image + the PD value of the subwindow 7 of the i-1th frame image + the PD value of the subwindow 7 of the i-th frame image) / 3.

[0125] The preprocessed PD value of the subwindow 8 of the i-th frame image is the weighted average of the PD value of the subwindow 8 of the i-2-th frame image, the PD value of the subwindow 8 of the i-1-th frame image, and the PD value of the subwindow 8 of the i-th frame image. For example, when the weight coefficient of each item (each frame image) is 1, the preprocessed PD value of the subwindow 8 of the i-th frame image is (the PD value of the subwindow 8 of the i-2-th frame image + the PD value of the subwindow 8 of the i-1-th frame image + the PD value of the subwindow 8 of the i-th frame image) / 3.

[0126] In one implementation, the preprocessed PD value of the i-th frame image is the weighted average of the preprocessed PD values ​​of the subwindows of the i-th frame image. For example, when the weight coefficient of each subwindow is 1, the preprocessed PD value of the i-th frame image is (the preprocessed PD value of subwindow 0 of the i-th frame image + the preprocessed PD value of subwindow 1 of the i-th frame image + the preprocessed PD value of subwindow 2 of the i-th frame image + the preprocessed PD value of subwindow 3 of the i-th frame image + the preprocessed PD value of subwindow 4 of the i-th frame image + the preprocessed PD value of subwindow 5 of the i-th frame image + the preprocessed PD value of subwindow 6 of the i-th frame image + the preprocessed PD value of subwindow 7 of the i-th frame image + the preprocessed PD value of subwindow 8 of the i-th frame image) / 9.

[0127] S706: The camera driver obtains the moving distance of the lens according to the PD value after preprocessing the i-th frame image.

[0128] In one implementation, the camera driver obtains the value of the defocus rate (DV) according to the PD value of the i-th frame image after preprocessing and the value of the camera internal reference DCC.

[0129] It can be understood that the lens can move forward or backward along the axis, and each preset position of the lens corresponds to a defocus rate (DV) value. The camera driver can obtain the corresponding preset position of the lens according to the defocus rate (DV) value, that is, the target position of the lens movement. The camera driver calculates the movement distance of the lens according to the target position of the lens movement and the current position of the lens.

[0130] S707: The camera driver transmits the moving distance of the lens to the focus motor.

[0131] S708: The focus motor drives the lens to move forward or backward according to the calculated moving distance, so that the lens moves to the target position to achieve automatic focusing.

[0132] If the preprocessed PD value remains stable, that is, the preprocessed PD values ​​of images of different frames no longer change, then the obtained defocus rate (DV) value no longer changes, that is, the target position of the lens movement no longer changes, the movement distance of the lens is 0, and the focus is successful.

[0133] It should be noted that the embodiment of the present application takes the camera driver performing preprocessing of PD data as an example to introduce the autofocus method provided by the embodiment of the present application. In specific implementation, the PD data preprocessing method provided by the embodiment of the present application can be implemented by other modules. The embodiment of the present application does not limit this.

[0134] The autofocus method provided in the embodiment of the present application averages the PD values ​​of multiple frames of images and calculates the moving distance of the lens according to the averaged PD value. In this way, the drastic change of the PD value can be reduced, thereby avoiding the focus motor driving the lens to push back and forth and reducing the phenomenon of pulling focus.

[0135] In the autofocus method provided in the embodiment of the present application, after preprocessing the PD data, the fluctuation amplitude of the preprocessed PD data can also be judged; if it is determined that the fluctuation amplitude of the preprocessed PD data is greater than the preset fluctuation threshold, the PD data can be not used for autofocusing, but other methods can be used for autofocusing. In this way, it can be avoided that the drastic fluctuation of the PD data causes the focus motor to drive the lens back and forth, affecting the autofocus effect.

[0136] For example, Figure 8 As shown, the automatic focusing method provided in the embodiment of the present application may also include:

[0137] S801: The camera driver obtains the difference between the PD value of the i-th frame image after preprocessing and the PD value of the i-1-th frame image after preprocessing.

[0138] After obtaining the PD value of each frame of image after processing, the camera driver can obtain the difference between the PD values ​​of two adjacent frames of image after preprocessing.

[0139] S802: The camera driver determines whether the difference is greater than a preset fluctuation threshold.

[0140] The greater the difference between the preprocessed PD values ​​of two adjacent frames of images, the greater the fluctuation of the preprocessed PD value.

[0141] If it is determined that the difference between the preprocessed PD values ​​of two adjacent frames is less than or equal to the preset fluctuation threshold, it indicates that the fluctuation of the preprocessed PD value is small, and the use of the preprocessed PD value to calculate the movement distance of the lens will not cause the focus motor to push back and forth. In one implementation, if it is determined that the difference between the preprocessed PD value of the i-th frame image and the preprocessed PD value of the i-1-th frame image is less than or equal to the preset fluctuation threshold, S706 can be continued to obtain the movement distance of the lens according to the preprocessed PD value of the i-th frame image.

[0142] If it is determined that the difference between the PD value of the preprocessed i-th frame image and the PD value of the preprocessed i-1-th frame image is greater than the preset fluctuation threshold, it indicates that the fluctuation of the preprocessed PD value is still quite drastic, and the use of the preprocessed PD value to calculate the moving distance of the lens will still cause the focus motor to push back and forth. In one implementation, the moving distance of the lens is not obtained by using the PD value. Instead, S803 is executed to perform autofocus in other ways.

[0143] S803: The camera driver uses a depth sensor to obtain the position of the object being photographed, and calculates the moving distance of the lens according to the position of the object being photographed.

[0144] In one implementation, a depth sensor is provided on the electronic device, and the depth sensor can be used to measure the distance between the object and the lens and obtain the position of the object. Exemplarily, the depth sensor can include a camera array, a distance sensor, a structured light sensor, a TOF sensor, etc.

[0145] The camera driver can adopt the conventional method to calculate the position of the lens according to the position of the object being photographed. Further, the camera driver calculates the moving distance of the lens according to the focus position of the lens and the current position of the lens.

[0146] Afterwards, S707 and S708 may be executed to achieve automatic focusing.

[0147] The autofocus method provided in the embodiment of the present application, after acquiring the PD data, averages the PD values ​​of multiple frames of images, that is, pre-processes the PD data. The fluctuation amplitude of the pre-processed PD data is also judged; if it is determined that the fluctuation amplitude of the pre-processed PD data is less than or equal to the preset fluctuation threshold, the movement distance of the lens is calculated according to the averaged PD value. In this way, the drastic changes in the PD value can be reduced, thereby avoiding the focus motor driving the lens back and forth and reducing the focus pulling phenomenon. If it is determined that the fluctuation amplitude of the pre-processed PD data is greater than the preset fluctuation threshold, the PD data is not used for autofocus, but a depth sensor is used to obtain the position of the subject, and autofocus is performed according to the obtained position of the subject. In this way, in the scene where the PD data fluctuates violently, it can also be avoided that the focus motor drives the lens back and forth.

[0148] It is understandable that the electronic device provided in the embodiment of the present application includes a hardware structure and / or software module corresponding to each function in order to realize the above functions. Those skilled in the art should easily realize that, in conjunction with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiment of the present application.

[0149] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0150] In one example, see Fig. 9 , which shows a possible structural diagram of the electronic device involved in the above embodiment. The electronic device 900 includes: a processing unit 910, a storage unit 920, an image acquisition unit 930 and a display unit 940.

[0151] The processing unit 910 is used to control and manage the actions of the electronic device 900. For example, it acquires PD data, pre-processes the PD data, and determines the moving distance of the lens according to the PD data.

[0152] The storage unit 920 is used to store program codes and data of the electronic device 900. For example, it stores acquired PD data, calculates the moving distance of the lens, and the like.

[0153] The image acquisition unit 930 is used to acquire images.

[0154] The display unit 940 is used to display the interface of the electronic device 900. For example, it displays a photo preview interface.

[0155] Of course, the unit modules in the above-mentioned electronic device 900 include but are not limited to the above-mentioned processing unit 910, storage unit 920, image acquisition unit 930 and display unit 940.

[0156] Optionally, the electronic device 900 may further include an audio unit, a communication unit, etc. The audio unit is used to collect audio, play audio, etc. The communication unit is used to support the electronic device 900 to communicate with other devices.

[0157] Among them, the processing unit 910 can be a processor or a controller, for example, a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The storage unit 920 can be a memory. The image acquisition unit 930 can be a camera, etc. The display unit 940 can be a display screen, etc. The audio unit can include a microphone, a speaker, etc. The communication unit can include a mobile communication unit and / or a wireless communication unit.

[0158] For example, the processing unit 910 is a processor (such as Figure 5 The processor 110 shown in FIG. 1 ), the storage unit 920 may be a memory (eg, Figure 5 The internal memory 121 shown in FIG. 1 ), the image acquisition unit 930 may be a camera (eg Figure 5 The camera 193 shown in FIG. 1 ), the display unit 940 may be a display screen (such as Figure 5 The audio unit may be an audio module (such as Figure 5 The communication unit may include a mobile communication unit (such as Figure 5 The mobile communication module 150 shown) and the wireless communication unit (such as Figure 5 The wireless communication module 160 shown in FIG. 1 ). The electronic device 900 provided in the embodiment of the present application may be Figure 5 The electronic device 100 shown in FIG. The processor, memory, display screen, camera, audio module, mobile communication unit, wireless communication unit, etc. may be connected together, for example, via a bus.

[0159] The embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected by lines. For example, the interface circuit can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as a processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

[0160] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the mobile phone in the above-mentioned method embodiment.

[0161] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the mobile phone in the above method embodiment.

[0162] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0163] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0165] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0167] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An automatic focusing method, applied to an electronic device, characterized in that: The electronic device includes a camera, the camera includes a lens and a photosensitive device, and shielded pixels are respectively arranged on the left and right sides of the photosensitive device. The electronic device also includes a depth sensor, and the depth sensor is used to measure the distance between the object to be photographed and the lens. The method includes: Starting the camera to preview the photo; Obtaining phase difference data when the camera captures each frame of image; the phase difference data is the value of the phase difference of the image of the object being photographed through the obscured pixels on the left and right sides; If the difference between the first value and the second value is less than or equal to a preset fluctuation threshold, the moving distance of the lens is obtained according to the phase difference data of the i-th frame image collected by the camera and the N-frame images before the i-th frame image; wherein the first value is a weighted average of the phase difference data of the i-th frame image collected by the camera and the N-frame images before the i-th frame image, and the second value is a weighted average of the phase difference data of the i-1-th frame image collected by the camera and the N-frame images before the i-1-th frame image, and N is greater than or equal to 1; If the difference between the first value and the second value is greater than a preset fluctuation threshold, the depth sensor is used to obtain the position of the object, and the movement distance of the lens is obtained according to the position of the object; The lens is moved according to the moving distance to achieve automatic focusing.

2. The method according to claim 1, characterized in that The step of acquiring the moving distance of the lens according to the phase difference data of the i-th frame image and N frames of image before the i-th frame image collected by the camera comprises: A weighted average value of phase difference data between an i-th frame image and N frames of images before the i-th frame image captured by the camera is calculated, and a moving distance of the lens is obtained according to the weighted average value.

3. The method according to claim 2, characterized in that The camera's photo preview interface includes a focus frame, and the phase difference data is a phase difference value corresponding to the focus frame. The calculating a weighted average of phase difference data of the i-th frame image captured by the camera and N frames of images before the i-th frame image includes: A weighted average of phase differences of a focus frame when the camera captures an i-th image frame and N frames of image before the i-th image frame is calculated.

4. The method according to claim 2, characterized in that: The camera's photo preview interface includes a focus frame, the focus frame includes a plurality of sub-windows, and the phase difference data includes phase difference values ​​corresponding to each sub-window of the focus frame. The calculating a weighted average of phase difference data of the i-th frame image captured by the camera and N frames of images before the i-th frame image includes: Calculating a weighted average of phase difference values ​​of a first sub-window when the camera captures an i-th frame image and N frames of images before the i-th frame image; the first sub-window is any sub-window of the focus frame; Calculate the weighted average of the weighted averages of the phase difference values ​​of each sub-window of the i-th frame image.

5. The method according to any one of claims 1 to 4, characterized in that: The i-th frame image is an image captured after a first preset time period after the camera is started.

6. The method according to any one of claims 1 to 4, characterized in that: Each phase difference data corresponds to a confidence level; The step of acquiring the moving distance of the lens according to the phase difference data of the i-th frame image and N frames of image before the i-th frame image collected by the camera comprises: The moving distance of the lens is obtained according to the effective phase difference data of the i-th frame image and N frames of images before the i-th frame image collected by the camera; wherein the effective phase difference data is the phase difference data with a confidence level greater than a preset threshold.

7. The method according to any one of claims 1 to 4, characterized in that: The depth sensor includes a camera array, a distance sensor, a structured light sensor or a time-of-flight TOF sensor.

8. An electronic device, characterized in that: The electronic device comprises: a processor, a memory and a camera, the camera comprises a lens and a photosensitive device, and shielded pixels are respectively arranged on the left and right sides of the photosensitive device; the processor is coupled to the memory; the memory is used to store computer program code; the computer program code comprises computer instructions, and when the processor executes the above-mentioned computer instructions, the electronic device executes the method as claimed in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.