Focusing method, electronic equipment and storage medium

Abstract

The invention provides a focusing method, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: firstly, carrying out calibration and calibration on the difference of the NxN pixel sensitivity of a focus plane position corresponding to an NxN OCL sensor pre-configured on a photographing device so as to reduce the difference of the NxN pixel sensitivity of the focus plane position, and then obtaining K calibration images at preset K focusing positions by using the calibrated NxN OCL sensor, and calibrating the difference value of the NxN pixel sensitivities under the K focusing positions by using the N * N pixel sensitivities and a focusing response function, and fitting a focusing parameter in the calibration process. Then, using the calibrated NxN OCL sensor to obtain P target images at P focusing positions, and calculating a difference value of NxN pixel sensitivity corresponding to the P target images; and accurately fitting a quasi-focus position by using the obtained difference value and the focusing parameter, and performing accurate focusing according to the lens position of the photographing equipment determined according to the quasi-focus position.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a focusing method, an electronic device and a storage medium. Background Art

[0002] With the continuous development of technology, people can use mobile phones, tablet computers, cameras and other photographic devices to record every detail of their lives anytime and anywhere. During the photo-taking process, whether the focus can be achieved and whether the focus is accurate often affects the clarity of the photo.

[0003] Currently, commonly used focusing methods include on-chip phase focusing and contrast focusing. Although these two focusing methods can show better focus and defocus recognition capabilities for areas with rich textures and accurately determine the focus position, when it comes to shooting pure color scenes, such as shooting the sky or white walls, the recognition effect of these two focusing methods is weak, and it is easy to have problems such as repeated focus push and pull, hysteresis or focus errors. In order to improve this situation, the current improvement method is usually to add an additional laser focus module to the camera device to improve the focus effect when shooting pure color scenes by sensing the absolute distance. However, this improvement scheme has the problem of ranging distance limitation and increased cost. Summary of the invention

[0004] In order to solve the above problems, the present application provides a focusing method, an electronic device and a storage medium, the purpose of which is to utilize the multi-pixel phase focusing sensor configured on the camera to achieve accurate estimation of the quasi-focus of pure color scene shooting. Accurate focusing can be achieved without adding any additional focusing devices, thereby not only solving the problem of poor focusing effect in pure color scenes, but also reducing the focusing cost.

[0005] In a first aspect, the present application provides a focusing method, the method comprising: firstly calibrating the difference of NxN pixel sensitivities at the focal plane position corresponding to the NxN pixel phase focusing sensor pre-configured on the photographing device, so as to calibrate the difference value of the NxN pixel sensitivities at the focal plane position, so as to minimize the difference of the NxN pixel sensitivities at the focal plane position, and then using the calibrated NxN pixel phase focusing sensor to obtain K calibration images at preset K focusing positions, so as to use the K calibration images and the focus response function (the specific source is not limited, such as being pre-constructed by the NxN pixel phase focusing sensor module factory end) to calibrate the different difference values ​​of the NxN pixel sensitivities at the preset K focusing positions, and in the calibration process, fitting the focus parameters of the focus response function. Then, using the calibrated NxN pixel phase focusing sensor, P target images are obtained at P focusing positions, and the difference values ​​of the NxN pixel sensitivities corresponding to the P target images are calculated. Then, the focus parameters of the focus response function and the difference in the NxN pixel sensitivities corresponding to the P target images can be used to accurately fit the quasi-focal position, and the lens position of the camera device determined according to the quasi-focal position can be precisely focused to obtain an image with a better focus effect.

[0006] It can be seen that in the above-mentioned focusing method, there is no need to add any new focusing devices to the photographing device (such as a mobile phone). Only by utilizing the pixel sensitivity difference corresponding to the NxN pixel phase focusing sensor (referring to an image sensor that obtains image data) pre-configured in the photographing device (such as a mobile phone), accurate estimation of the quasi-focus and recognition of the distance for shooting pure color scenes can be achieved, which not only solves the problem of poor focusing effect in pure color scenes, but also reduces the focusing cost.

[0007] In a possible implementation, the value of N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; there are four pixels corresponding to the microlens; calibrating the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor to calibrate the difference in sensitivity of NxN pixels at the focal plane position, and reducing the difference in sensitivity of NxN pixels at the focal plane position, may include: calibrating the difference in sensitivity of four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor to calibrate the difference in sensitivity of four pixels at the focal plane position, and reducing the difference in sensitivity of four pixels at the focal plane position, so as to improve the focusing effect.

[0008] In a possible implementation, calibrating the difference in sensitivity of four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor to calibrate the difference in sensitivity of the four pixels at the focal plane position and reduce the difference in sensitivity of the four pixels at the focal plane position may include: using a camera to photograph a light plate with uniform illumination, and setting half of the sum of the focus position of the lens at infinity and the focus position of the lens at the closest focus distance as the calibration focus position; using the 2x2 pixel phase focus sensor to obtain a calibration image at the calibration focus position; wherein the calibration image includes four color channels; each sub-channel under each color channel shares the color of the corresponding color channel Color filter; using a preset pixel scaling algorithm to perform pixel reduction processing on the calibration image to obtain a reduced calibration image; merging the sub-channels under each color channel contained in the reduced calibration image, and calculating the difference between each merged channel and its pixel mean for the color channel; using the difference between each merged channel and its pixel mean for the color channel, calculating the compensation value between each merged channel and its pixel mean for the color channel; and using the compensation value to compensate the corresponding channel pixels, so as to more accurately calibrate the difference in sensitivity of the four pixels at the focal plane position, and minimize the difference in sensitivity of the four pixels at the focal plane position.

[0009] In a possible implementation, the preset pixel scaling algorithm is a bilinear interpolation algorithm.

[0010] In a possible implementation, merging the sub-channels under each color channel contained in the reduced calibration image and calculating the difference between each merged channel and the color channel pixel mean value may include: calculating the mean value of each color channel pixel contained in the reduced calibration image; calculating the ratio of each merged channel pixel to the mean value of the corresponding color channel pixel, and using the difference between the ratio and the number 1 as the difference value of the corresponding channel, so as to improve the calculation efficiency and accuracy of the difference value.

[0011] In one possible implementation, the difference values ​​between each merged channel and its pixel mean for the color channel are used to calculate the compensation values ​​of each merged channel and its pixel mean for the color channel, which can include: respectively calculating the sum of the difference values ​​between each merged channel and its pixel mean for the color channel and the number 1, and calculating the ratio of the number 1 to the sum as the compensation value of the corresponding channel and its pixel mean for the color channel, so as to improve the calculation accuracy of the compensation value.

[0012] In a possible implementation, the value of K is 5; the K focus positions are preset as 5 step positions corresponding to dividing the lens travel into 5 equal parts; using a calibrated NxN pixel phase focus sensor, K calibration images are obtained at the preset K focus positions, which may include: using a calibrated NxN pixel phase focus sensor, obtaining a calibration image at each of the 5 step positions corresponding to dividing the lens travel into 5 equal parts, to obtain 5 calibration images, so as to improve calibration efficiency.

[0013] In a possible implementation, the K calibration images and the focus response function are used to calibrate different difference values ​​of NxN pixel sensitivity at K preset focus positions, and during the calibration process, the focus parameters of the focus response function are fitted, including: calculating the average pixel value of each color channel included in the K calibration images; and using the average pixel value to calculate the interpolation of each color channel; using the interpolation and the focus response function, calibrating different difference values ​​of NxN pixel sensitivity at K preset focus positions, and during the calibration process, fitting the focus parameters of the focus response function. Thereby, the fitting accuracy of the focus parameters can be improved.

[0014] In a possible implementation, the value of N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; four pixels correspond to the microlens; using K calibration images and a focus response function, calibrating different difference values ​​of NxN pixel sensitivity at K preset focus positions, and fitting the focus parameters of the focus response function during the calibration process, which may include: calculating the average pixel value of the four color channels contained in the five calibration images; and using the average pixel value to calculate the interpolation of the four color channels; using the interpolation and the focus response function, calibrating different difference values ​​of the sensitivity of the four pixels at the five preset focus positions, and during the calibration process, more quickly fitting the focus parameters of the focus response function to improve the fitting efficiency.

[0015] In a possible implementation, the value of P is 2; using a calibrated NxN pixel phase focus sensor, P target images are obtained at P focus positions, and the difference in NxN pixel sensitivities corresponding to the P target images is calculated, which may include: using a calibrated NxN pixel phase focus sensor, two target images are obtained at two focus positions, and the difference in NxN pixel sensitivities corresponding to the two target images is calculated to improve calculation efficiency.

[0016] In a possible implementation, using the focus parameter of the focus response function and the difference value of the NxN pixel sensitivity corresponding to the P target images, fitting the quasi-focus position, and focusing according to the lens position of the camera device determined by the quasi-focus position, may include: using the focus parameter of the focus response function and the difference value of the NxN pixel sensitivity corresponding to the two target images, fitting the quasi-focus position by the least square method, and focusing according to the lens position of the camera device determined by the quasi-focus position. Thereby, the focusing effect can be improved and the focusing cost can be reduced.

[0017] In a second aspect, the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor being used to call and execute the computer program to implement the focusing method described in any one of the first aspects above.

[0018] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of an electronic device, is used to implement the focusing method described in any one of the first aspects above.

[0019] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computer executes the focusing method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a scenario provided for an embodiment of the present application;

[0021] Figure 2 A schematic diagram of an electronic device provided in an embodiment of the present application;

[0022] Figure 3 A software structure block diagram of an electronic device provided in an embodiment of the present application;

[0023] Figure 4 A flowchart of a focusing method provided in an embodiment of the present application;

[0024] Figure 5 An example diagram of the brightness distribution of pixels at the front focus, quasi focus, and back focus positions provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of four color channels R, Gr, Gb, and B included in the calibration image provided in an embodiment of the present application;

[0026] Figure 7 This is an example diagram of the fitting process of the focus response function provided in an embodiment of the present application;

[0027] Figure 8This is an example diagram of a focus scene for a camera device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be used as limitations on the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context.

[0029] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0030] The multiple involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0031] In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant concepts or technologies is first given:

[0032] Contrast Detection Auto Focus (CDAF) is a technology that achieves focusing by analyzing the contrast information of an image. The principle is to determine the clarity of the image based on the contrast of the image, thereby adjusting the focal length of the camera lens to achieve a clear effect. In contrast focusing, a photographic device (such as a camera) takes several images at different focal lengths, and then determines which focal length has a clearer image by analyzing the contrast of the objects in these images. Images with higher contrast usually indicate clearer images. This focusing method uses the contrast of the edges and details of objects in the image to determine the image clarity.

[0033] Phase Detection Autofocus (PDAF) is a technology used for automatic focus (AF) that uses pixels on the image sensor to compare different phases to determine the focus position. This technology is commonly used in cameras and other photographic devices to focus quickly and accurately. PDAF uses the light phase difference between specific pixels on the image sensor to determine the focus position. On the image sensor, light is divided into two or more different paths to reach the pixel after passing through the lens. The optical path difference of these paths will cause the light phase difference between different pixels. When the image is not in focus, by analyzing the light phase difference between two pixels, the phase-focused image sensor can detect the focus deviation and quantify this deviation. By continuously adjusting the lens position and detecting the change in phase difference, the best focus position can be quickly located.

[0034] In order to enable persons skilled in the art to more clearly understand the solution of the present application, the application scenarios of the technical solution of the present application are described below.

[0035] See also Figure 1 , which shows a scenario schematic diagram provided in an embodiment of the present application.

[0036] In this example scenario, it is shown that when (a user) uses a mobile phone, camera or other camera device to shoot a pure color scene, no matter whether contrast focus or phase focus is used, the position of the lens needs to be repeatedly pushed and pulled. However, no matter how the lens position is moved, the corresponding phase difference (PD value) or contrast value is the same as Figure 1 The straight line shown in the figure cannot find the exact focus and lens position. This is because the contrast focusing method usually relies on the contrast difference in the image to focus. Figure 1 In the shooting of the pure color scene shown, there is a lack of obvious features or contrast in the image, which makes it difficult for the contrast focus method to determine the focus position. In a pure color scene, all areas in the image look similar, lack obvious edges or details, and therefore the contrast is very low. The contrast focus method needs to find the contrast difference between areas in the image to determine the focus position. However, in a pure color scene, this contrast difference is small or almost non-existent, and it is difficult for the contrast focus method to accurately identify the focus. Therefore, in this case, problems such as repeated focus push and pull, hysteresis, or focus errors will occur. The phase focus method also has difficulty obtaining sufficient phase difference information from a pure color scene because the entire scene or area lacks obvious features or details. This makes it impossible for the phase focus method to accurately determine the focus position.

[0037] In order to improve the above situation, the current improvement method is usually to add a laser focus module to mobile phones, cameras and other photographic devices to improve the focus effect when shooting pure color scenes by sensing the absolute distance. However, this improvement solution has the problem of limited ranging distance and increased cost.

[0038] In order to overcome the above technical problems, the present application provides a focusing method. By utilizing the pixel sensitivity difference corresponding to the NxN pixel phase focusing sensor (referring to an image sensor that obtains image data, which can be represented by an NxN On-ChipLens sensor, or simply referred to as an NxN OCL sensor) pre-installed on the camera device, accurate estimation of the quasi-focus and recognition of the distance of pure color scene shooting can be achieved, and accurate focusing can be achieved without adding any new focusing devices, thereby not only solving the problem of poor focusing effect in pure color scenes, but also reducing the focusing cost.

[0039] Among them, the present application does not limit the value of N, and it can be set according to actual conditions and empirical values. It only needs to be a positive integer greater than 1. For example, N can be set to 2, then the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor (i.e., QPD (Quadrant Photodiode) sensor or 2x2 OCL sensor). In this way, the sensitivity difference of the four pixels corresponding to the 2x2 OCL sensor can be used to achieve accurate estimation of the quasi-focus and distance recognition of pure color scene shooting, so as to further achieve precise focusing.

[0040] It should be noted that the focusing method provided in the embodiments of the present application can be applicable to electronic devices with camera functions (i.e., camera devices) such as mobile phones, tablet computers, cameras, personal digital assistants (PDAs), desktops, laptops, notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, and wearable devices.

[0041] In order to enable those skilled in the art to more clearly understand the focusing method provided by the present application, the hardware architecture and software system architecture of the electronic device that implements the focusing method in real time are first introduced in detail below.

[0042] See also Figure 2 , which shows a schematic diagram of an electronic device (taking a mobile phone or other photographing device as an example) provided in an embodiment of the present application.

[0043] like Figure 2As shown, the electronic device 200 may include a processor 210, a mobile communication module 220, a wireless communication module 230, a sensor module 240, a display screen 250, an internal memory 260, a camera 270, an audio module 280, a speaker 280A, a receiver 280B, a microphone 280C, an earphone interface 280D, an antenna group 1 and an antenna group 2.

[0044] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0045] The processor 210 may include one or more processing units, for example: the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions. For example, according to the sensitivity difference of each pixel corresponding to the NxN pixel phase focus sensor 240 pre-installed on the electronic device (photographing device) 200, accurate estimation of the quasi-focus and recognition of the distance of the pure color scene shooting can be achieved, so as to further achieve accurate focus on the pure color scene and improve the focusing effect.

[0046] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0047] In some embodiments, the processor 210 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0048] The sensor module 240 can be used to obtain data signals of various aspects related to the electronic device 200 as a basis for implementing corresponding functions. In some embodiments, the sensor module 240 may include but is not limited to a pressure sensor, a gyroscope sensor, an image sensor, and the like. Among them, the image sensor (the present application adopts an NxN OCL sensor) can be used to obtain image data (such as a Raw image) around the electronic device, and transmit the obtained image data to the electronic device 200. So that the electronic device 200 can calculate the difference in the sensitivity of the NxN pixels under each color channel corresponding to the NxN pixel phase focus sensor based on these image data, so that the quasi-focus position can be accurately fitted according to the value of the difference, thereby achieving precise focusing. For details, please refer to the relevant introduction of the subsequent embodiments.

[0049] The internal memory 260 can be used to store computer executable program codes, which include instructions. The internal memory 260 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound acquisition function, an image shooting function, etc.), etc. The data storage area may store data created during the use of the electronic device 200 (such as audio data, image data, etc.), etc. In addition, the internal memory 260 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 210 executes various functional applications and data processing of the electronic device 200 by running instructions stored in the internal memory 260 and / or instructions stored in a memory provided in the processor.

[0050] In some embodiments, the internal memory 260 stores instructions for executing the focusing method. The processor 210 can execute the instructions stored in the internal memory 260 to implement the following functions: first, calibrate the difference in sensitivity of the NxN pixels at the focal plane position corresponding to the NxN OCL sensor pre-configured in the electronic device (photographing device) 200 to calibrate the difference in sensitivity of the NxN pixels at the focal plane position, so that the difference in sensitivity of the NxN pixels at the focal plane position is minimized, and then use the calibrated NxN pixel phase focusing sensor to obtain K calibration images at the preset K focal positions, so as to use the K calibration images and the focus response function (the specific source is not limited, such as can be pre-constructed by the NxN OCL sensor module factory end) to calibrate the different difference values ​​of the sensitivity of the NxN pixels at the preset K focal positions, and fit the focus parameters of the focus response function during the calibration process. Then, use the calibrated NxN pixel phase focusing sensor to obtain P target images at P focal positions, and calculate the difference values ​​of the sensitivity of the NxN pixels corresponding to the P target images. Then, the focus parameters of the focus response function and the difference in the NxN pixel sensitivities corresponding to the P target images can be used to accurately fit the quasi-focal position, and the lens position of the camera device determined according to the quasi-focal position can be precisely focused to obtain an image with a better focus effect.

[0051] The display screen 250 is used to display images, videos, etc., such as displaying pure color scenes such as the sky or white walls captured. The display screen 250 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 200 may include one or more display screens 250.

[0052] The camera 270 is used to capture still images or videos. The object generates an optical image through the lens and is projected onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include one or more cameras 193.

[0053] The electronic device 200 implements the display function through a GPU, a display screen 250, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 250 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0054] The electronic device 200 can implement audio functions such as music playback, recording, and voice input and output through the audio module 280, the speaker 280A, the receiver 280B, the microphone 280C, the headphone jack 280D, and the application processor.

[0055] The audio module 280 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 280 can also be used to encode and decode audio signals. In some embodiments, the audio module 280 can be arranged in the processor 210, or some functional modules of the audio module 280 can be arranged in the processor 210.

[0056] The speaker 280A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 200 can listen to music or listen to a hands-free call through the speaker 280A.

[0057] The receiver 280B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 200 receives a call or voice message, the voice can be received by placing the receiver 280B close to the human ear.

[0058] Microphone 280C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by approaching the microphone 280C with his mouth to input the sound signal into the microphone 280C.

[0059] The earphone interface 280D is used to connect a wired earphone, and the standard attributes of the interface are not limited.

[0060] It should be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely a schematic illustration and does not constitute a structural limitation on the electronic device 200 .

[0061] The wireless communication function of the electronic device 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 220, the wireless communication module 230, the modem processor and the baseband processor.

[0062] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0063] The mobile communication module 220 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 200. The mobile communication module 220 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 220 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 220 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 220 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 220 can be set in the same device as at least some of the modules of the processor 210.

[0064] The wireless communication module 230 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 200. The wireless communication module 230 can be one or more devices integrating at least one communication processing module. The wireless communication module 230 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 210. The wireless communication module 230 can also receive the signal to be sent from the processor 210, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0065] In addition, on top of the above components, the electronic device 200 runs an operating system, such as an iOS operating system, an Android operating system, a Windows operating system, etc. Application programs can be installed and run on the operating system.

[0066] See also Figure 3 , which shows a schematic diagram of the software structure of an electronic device (taking a mobile phone or other photographing device as an example) provided in an embodiment of the present application.

[0067] The software system of the electronic device (taking a mobile phone as an example) 200 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 200.

[0068] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: application layer (APK), application framework layer (Framework), hardware abstraction layer (HAL), driver layer, and hardware layer.

[0069] The application layer can include a series of application packages (APP). Figure 3 As shown, the application package may include camera, navigation, WLAN, Bluetooth, gallery and other applications. When the user holds the electronic device 200 to take a photo (such as taking a photo Figure 1 When shooting a pure color scene as shown in the figure), the camera application can communicate with the camera-related devices in the camera access interface of the framework layer to request camera functions and obtain image data, etc.

[0070] The application framework layer (framework layer) provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a window manager, a notification manager, a resource manager, a camera access interface (including but not limited to camera management, camera devices, etc.), etc. The application framework layer may be used to implement the interaction between the camera service and the camera API. That is, it provides a unified interface so that different camera hardware can interact with different camera applications. The framework layer is also used to handle many common aspects of camera functions, such as auto focus, motion focus, etc.

[0071] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0072] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0073] In this embodiment, the hardware abstraction layer (HAL) provides a set of standard interfaces, so that the framework layer can communicate with camera hardware from various manufacturers without knowing the underlying hardware details. The hardware abstraction layer (HAL) stores the hardware abstraction layer and the camera algorithm library. It should be noted that the focus algorithm provided by this application can be stored in the camera algorithm library of the HAL layer, such as Figure 3 shown.

[0074] Among them, the focusing algorithm provided by the present application is used to accurately fit the quasi-focus position by calculating the difference value of the NxN pixel sensitivity under each color channel corresponding to the NxN pixel phase focusing sensor 240, so as to further achieve precise focusing. Specifically, firstly, the difference of the NxN pixel sensitivity at the focus plane position corresponding to the NxN OCL sensor pre-configured in the electronic device (taking a mobile phone as an example) 200 is calibrated to calibrate the difference value of the NxN pixel sensitivity at the focus plane position, so that the difference of the NxN pixel sensitivity at the focus plane position is minimized, and then the calibrated NxN pixel phase focusing sensor is used to obtain K calibration images at the preset K focus positions, so as to calibrate the different difference values ​​of the NxN pixel sensitivity at the preset K focus positions using the K calibration images and the focus response function, and fit the focus parameters of the focus response function during the calibration process. Then, the calibrated NxN pixel phase focusing sensor is used to obtain P target images at P focus positions, and the difference value of the NxN pixel sensitivity corresponding to the P target images is calculated. Then, the focus parameters of the focus response function and the difference in the NxN pixel sensitivities corresponding to the P target images can be used to accurately fit the quasi-focal position, and the lens position of the camera device determined according to the quasi-focal position can be precisely focused to obtain an image with a better focus effect.

[0075] The hardware layer may include the hardware components of the aforementioned electronic device (taking a mobile phone as an example) 200. Exemplarily, Figure 3 The NxN pixel phase focus sensor, image signal processor, digital signal processor, and graphics processor were demonstrated.

[0076] Among them, the NxN pixel phase focus sensor is used for image exposure and focus processing, etc.

[0077] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 200 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0078] In this way, interoperability of different camera applications and hardware devices on the Android platform can be achieved through the application layer, framework layer, HAL layer, driver layer and hardware layer, which can realize accurate estimation of the quasi-focus when shooting pure-color scenes, thereby achieving precise focus without adding any additional focusing devices, which not only improves the focusing effect for pure-color scenes, but also reduces the focusing cost.

[0079] The technical solutions involved in the following embodiments can all be implemented in electronic devices having the above-mentioned hardware architecture and software architecture.

[0080] Next, the specific implementation process of the focusing method provided by this application will be described in detail:

[0081] like Figure 4 As shown, the specific implementation process of the focusing method may include the following steps S401-S405:

[0082] S401: Calibrate the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor to calibrate the difference in sensitivity of NxN pixels at the focal plane position and reduce the difference in sensitivity of NxN pixels at the focal plane position; wherein N is a positive integer greater than 1.

[0083] It should be noted that in order to achieve accurate focusing of pure color scenes by photographing devices (such as mobile phones, tablets, etc.) without adding any new focusing devices, this application sets the image sensor pre-installed in the photographing device as an NxN pixel phase focusing sensor (NxN OCL sensor), which is used to accurately fit the quasi-focus position by calculating the difference in the sensitivity of NxN pixels under each color channel corresponding to the NxN pixel phase focusing sensor, so as to further achieve accurate focusing. In this way, accurate focusing can be achieved without adding any new focusing devices, which can not only solve the problem of poor focusing effect in pure color scenes, but also reduce the focusing cost.

[0084] Among them, the present application does not limit the specific composition and structure of the NxN OCL sensor. Its structural characteristics may include: including a microlens, and one microlens corresponds to NxN pixels, where N is a positive integer greater than 1. For example, when the value of N is 2, the NxN OCL sensor is a 2x2 OCL sensor, and one microlens corresponds to four pixels. Next, the structure of the NxN OCL sensor is described in detail by taking the 2x2 OCL sensor as an example, which may specifically include:

[0085] (1) Microlens: In a 2*2 OCL sensor, each microlens is located at the center of four adjacent pixels, forming a 2x2 pixel area. The function of the microlens is to focus light from different directions onto the corresponding pixel. This layout helps to enhance the light receiving efficiency and improve the light utilization of the sensor.

[0086] (2) Color filter array (CFA): In a 2*2 OCL sensor, the color filter is usually arranged in a four-pixel Bayer pattern. In this layout, each 2x2 pixel area contains a red filter, a blue filter, and two green filters. This arrangement enables each pixel to capture information of the three colors of red (R), green (G), and blue (B), thereby achieving full-color image acquisition.

[0087] (3) Pixel arrangement: Four pixels are distributed around each microlens in a 2x2 array, with each pixel corresponding to a filter. This layout design enables each microlens to focus and distribute light to its corresponding pixel, thereby capturing light signals more efficiently.

[0088] In this way, based on the above-mentioned characteristics of the NxN OCL sensor (taking a 2x2 OCL sensor as an example), the present application can use the NxN pixels under the microlens to sense the matching of the lens principal ray angle and the sensor principal ray angle, calculate the sensitivity difference of the NxN pixels, and use it to achieve accurate estimation of the quasi-focus and distance recognition of pure color scene shooting, thereby achieving precise focusing.

[0089] Specifically, still taking the 2x2 OCL sensor as an example, when the chief ray angle (CRA) is aligned to the microlenses of the four pixels, that is, there is a strict CRA match, the light will be more evenly distributed to the four pixels, making their sensitivity relatively consistent. If the CRA cannot be strictly matched to the microlenses of the four pixels, the angle of the incident light may cause slight differences on different pixels. Among them, the chief ray angle (CRA) refers to the angle between the line from the optical center of the optical system to a certain pixel and the normal of the pixel plane.

[0090] When the lens's main ray CRA and the 2x 2OCL sensor's main ray angle are strictly matched (e.g., no more than plus or minus 2° to 3°), the sensitivity difference of the four pixels is the lowest. When the CRA matching relationship between the lens and the sensor changes from matching to mismatching, the sensitivity difference of the four pixels will gradually increase.

[0091] Therefore, in camera modules such as mobile phones, changes in the focus position will cause the chief ray angle (CRA) of the lens to change, which means that the CRA adaptation relationship between the lens and the sensor is changing at different focus positions. Therefore, the difference in the sensitivity of the four pixels at different focus positions is also changing.

[0092] Based on this, the present application proposes to utilize the quantitative relationship between the 2*2 OCL sensitivity difference and the position of the camera lens to achieve quasi-focus judgment when shooting pure color scenes. Figure 5 As shown in the figure, at the front focus, quasi focus, and back focus positions, the angle of light perceived by a single pixel changes, and after being focused and refracted by the microlens, the brightness distribution of the four pixels also changes. Figure 5 It can be seen that the sensitivity difference of pixels at the front focus and back focus positions is large, and the sensitivity difference of pixels at the quasi-focus position is small. When the matching relationship between the lens and the sensor is fixed, the relationship between the sensitivity difference change of NxN pixels under one microlens and the focus position is also fixed. Such characteristics constitute the basis for the depth perception of the captured image in this application. Therefore, the quasi-focus position can be determined by fitting and other methods by pre-calibrating the relationship between the sensitivity difference of NxN pixels and the focus position of the lens, and collecting a small number of data points of the focus position and the sensitivity difference of NxN pixels through camera control.

[0093] In the specific implementation process, it is first necessary to calibrate the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor (NxNOCL sensor) configured in the photographing device, so as to calibrate the difference value of the sensitivity of NxN pixels at the focal plane position (which can be represented by Channel Diff), and reduce the difference in sensitivity of NxN pixels at the focal plane position, thereby obtaining a calibrated NxN OCL sensor for executing the subsequent step S402.

[0094] In some implementations, the value of N can be 2, then the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor (2x2 OCL sensor), and the 2x2 OCL sensor includes a microlens, and there are four pixels under the microlens. In this way, in order to improve the focusing effect, it is first necessary to perform quad-Bayer coding sensitivity correction (QSC) on the 2x2 OCL sensor configured in the camera device to correct the uniformity of the sensitivity of the 2*2 pixels, that is, to calibrate the difference value (Channel Diff) of the sensitivity of the four pixels at the focal plane position, so that the difference value of the sensitivity of the four pixels at the focal plane position is minimized. The specific implementation process may include the following steps S4011-S4015:

[0095] S4011: Using a camera to photograph a light panel with uniform illumination, and setting half of the sum of the focus position of the lens at infinity and the focus position of the lens at the minimum focus distance as the calibration focus position.

[0096] In this implementation, when performing QSC calibration, the camera module of the photographing device (mobile phone, camera, etc.) can first be used to shoot a uniform light plate (the specific structure is not limited, such as the structural characteristics of the uniform light plate can be large-area luminescence, monochromatic temperature, D65 light source and 3000lux brightness, etc.). For example, when the focal length of the lens is 8.6mm, the object distance can be defined as 1 times the focal length to improve the calibration accuracy. The setting rule of the focus position of the camera module of the photographing device (mobile phone, camera, etc.) is: define the focus position of the lens at infinity as z1, define the focus position of the lens at the closest focus distance as z2, and then set the focus position to (z1+z2) / 2.

[0097] S4012: Using a 2x2 pixel phase focus sensor, at a calibrated focus position, a calibration image is acquired; wherein the calibration image includes four color channels; and each sub-channel under each color channel shares a color filter of the corresponding color channel.

[0098] In this implementation, after the focus position is determined in step S4011, a calibration image can be obtained by using a 2x2 OCL sensor at the calibration focus position (e.g., using a 2x2 OCL sensor to output a Raw image with full pixel resolution at the calibration focus position). The calibration image (e.g., Raw image) includes four color channels, such as Figure 6 The four color channels R, Gr, Gb, and B are shown. Each sub-channel under each color channel shares the color filter of the corresponding color channel. For example, for Figure 6 Under each color channel shown, there are subchannels with subscripts 0, 1, 2, and 3. Under the R channel, its four subchannels R0, R1, R2, and R3 share the color filter of the R channel. Under the Gr channel, its four subchannels Gr0, Gr1, Gr2, and Gr3 share the color filter of the Gr channel. Under the Gb channel, its four subchannels Gb0, Gb1, Gb2, and Gb3 share the color filter of the Gb channel. Under the B channel, its four subchannels B0, B1, B2, and B3 share the color filter of the B channel.

[0099] S4013: Using a preset pixel scaling algorithm, perform pixel reduction processing on the calibration image to obtain a reduced calibration image.

[0100] In this implementation, after obtaining the calibration image (such as the Raw image) through step S4012, in order to speed up the calculation efficiency, a pre-set pixel scaling algorithm can be further used to perform pixel reduction processing on the calibration image (such as the Raw image) to obtain the pixel-reduced calibration image for executing the subsequent step S4014.

[0101] Among them, the present application does not limit the specific content of the preset pixel scaling algorithm, which can be selected according to actual conditions and empirical values. For example, the preset pixel scaling algorithm can be set to a bilinear interpolation algorithm, so that the bilinear interpolation algorithm can be used to perform pixel reduction processing on the calibration image (such as the Raw image) to reduce the amount of calibration data and speed up calculation efficiency. In addition, the present application does not limit the specific pixel values ​​of the calibration image (such as the Raw image) and the reduced calibration image. Assuming that the calibration image (such as the Raw image) is L pixels × H pixels (the values ​​of L and H are not limited, for example, L and H can be 512 and 224 respectively), after using the bilinear interpolation algorithm to perform pixel reduction processing on it, 64 pixels × 28 pixels can be obtained.

[0102] S4014: Merge the sub-channels under each color channel included in the reduced calibration image, and calculate the difference between each merged channel and the pixel mean of the color channel.

[0103] In this implementation, after obtaining the reduced calibration image (the size may be 64 pixels × 28 pixels) through step S4013, further, the sub-channels under each color channel contained in the reduced calibration image may be merged first, taking the four sub-channels R0, R1, R2, and R3 under the R color channel of the 2x2 OCL sensor as an example, the size of each sub-channel may be 16 pixels × 12 pixels, and the size of the 16-channel integrated QSC is 16*12*16. Among them, 16 pixels = 64 pixels / (2*2), 12 pixels = 28 pixels / (2*2).

[0104] Then, the mean value of each color channel pixel contained in the reduced calibration image is calculated. For example, taking the four sub-channels R0, R1, R2, and R3 corresponding to the R color channel of the 2x2 OCL sensor as an example, this color channel covers 2x 2 pixels, and the pixel values ​​corresponding to the sub-channel numbers R0, R1, R2, and R3 are represented by R0, R1, R2, and R3 respectively, then the mean value of the color channel pixel is calculated as:

[0105] For a specific merged channel pixel, its ratio to the average value of the corresponding color channel pixel can be further calculated. For example, still taking R0 corresponding to the R color channel of the 2x2 OCL sensor as an example, its ratio to the average value of the R color channel pixel is: Then use the difference between the ratio and the number 1 as the difference value of the corresponding channel and use D0 to represent it, that is, Thus, when the mean value of the R color channel pixel When 10 is 10 and the pixel value R0 is 9, it can be calculated that the value of D0 is -10%, that is, D0 = 9 / 10--1 = -10%.

[0106] S4015: Calculate compensation values ​​of each merged channel and its corresponding color channel pixel average using the difference values ​​between each merged channel and its corresponding color channel pixel average; and use the compensation values ​​to compensate the corresponding channel pixels to calibrate the difference values ​​of the sensitivity of the four pixels at the focal plane position, and reduce the difference in the sensitivity of the four pixels at the focal plane position.

[0107] In this implementation, after obtaining the difference values ​​between each merged channel and its pixel mean for the color channel (such as the value of D0) through step S4014, the sum of the difference values ​​between each merged channel and its pixel mean for the color channel (such as the value of D0) and the number 1 can be further calculated respectively, and the ratio of the number 1 to the sum is calculated as the compensation value of the corresponding channel and its pixel mean for the color channel.

[0108] For example, taking R0 of the 2x2 OCL sensor corresponding to the R color channel as an example, its ratio to the mean value of the R color channel pixels is Then use the difference between the ratio and the number 1 as the difference value of the corresponding channel and use D0 to represent it, that is, After further calculating the sum of D0 and the number 1 (i.e. 1+D0), the ratio of the value 1 to the sum is calculated as the compensation value of R0 and its mean value of the R color channel pixel, and is expressed as QR0, i.e. QR0=1 / (1+D0). In this way, when the value of D0 is -10%, it can be calculated that the value of QR0 is 1.111, i.e. QR0=1 / (1+D0)=1 / 0.9≈1.111.

[0109] Furthermore, the compensation value (such as QR0) can be used to compensate the corresponding channel pixel (such as R0). For example, for pixel R0, it can be multiplied by the compensation value QR0 so that the de-compensated pixel value is close to the mean value of the R color channel pixel. By analogy, the difference in sensitivity of the four pixels at the focal plane position can be calibrated, and the difference in sensitivity of the four pixels at the focal plane position can be reduced (that is, they are all close to the mean value of the corresponding color channel pixels).

[0110] In this way, after the calibration by the QSC, the pixel difference of each sub-channel in the Raw image finally output by the 2x2 OCL sensor can be reduced to the minimum, and a calibrated 2x2 OCL sensor is obtained to perform the subsequent step S402.

[0111] It should be noted that when the value of N is other positive integers greater than 2, the calibration process for the NxN OCL sensor can be implemented by referring to the method for calibrating the 2x2 OCL sensor described in the above steps S4011-S4045, and the specific calibration process is not repeated here.

[0112] S402: Using the calibrated NxN pixel phase focus sensor, obtain K calibration images at preset K focus positions; where K is a positive integer greater than 0.

[0113] In this embodiment, after the NxN pixel phase focus sensor configured in the camera device is calibrated in step S401, the calibrated NxN pixel phase focus sensor can be further used to obtain an image (defined as a calibration image, with no specific size limit) at a preset K focus position to obtain K calibration images for performing the subsequent step S403. K is a positive integer greater than 0.

[0114] Among them, it should be noted that the present application does not limit the value of K and the specific values ​​of the preset K focus positions, which can be set according to actual conditions and experience values. In some embodiments, the value of K can be set to 5, and the preset K focus positions can be set to the 5 step positions corresponding to the 5 equal divisions of the lens stroke. For example, assuming that the lens stroke (in mm) is divided into 800 steps, and then divided into 5 equal divisions, the AF positions can be set to the 1st step, the 200th step, the 400th step, the 600th step, and the 800th step, respectively. And obtain an image at each AF position to obtain 5 calibration images, which are used to calibrate the difference in the sensitivity of the NxN pixels corresponding to the NxN OCL sensor at each AF position (Channel Diff).

[0115] It should also be noted that, for each calibration image, taking the color encoding scheme of the NxN Raw image as an example, there are four color channels of R, Gr, Gb, and B, and there are NxN color sub-channels under the color channel corresponding to one microlens.

[0116] S403: Using K calibration images and a focus response function, calibrate different difference values ​​of NxN pixel sensitivities at K preset focus positions, and during the calibration process, fit the focus parameters of the focus response function.

[0117] After obtaining K calibration images at the preset K focus positions using the calibrated NxN OCL sensor in step S402, the average pixel value of each color channel contained in the K calibration images can be further calculated, and the interpolation of each color channel can be calculated using the obtained average pixel value. Then, the different difference values ​​of NxN pixel sensitivity at the preset K focus positions are calibrated using the interpolation and focus response function, and the focus parameters of the focus response function are fitted during the calibration process to perform the subsequent step S404.

[0118] Among them, the present application does not limit the specific value and source of the focus response function, which can be set according to actual conditions and empirical values, such as being pre-built by the NxN OCL sensor module factory, etc., and can be represented by F(x, p0), specifically: Wherein, p0 represents the position of the lens in the accurate focus state, x represents any focus position during the shooting process, and a and b represent the focus parameters of the focus response function.

[0119] For example: still assuming that the value of N is 2, the NxN OCL sensor is a 2x2 OCL sensor, and the 2x2 OCL sensor includes a microlens, and there are four pixels under the microlens. Assume that the value of K is 5, and an image is obtained at each of the five step positions corresponding to the lens stroke divided into five equal parts, and five calibration images are obtained. Each calibration image contains four color channels, R, Gr, Gb, and B, and 16 sub-channels, R0, R1, R2, R3, Gr0, Gr1, Gr2, Gr3, Gb0, Gb1, Gb2, Gb3, B0, B1, B2, and B3.

[0120] On this basis, we can further use algorithms such as bilinear interpolation to calculate the average pixel value of the four color channels contained in the five calibration images, and use the obtained average pixel value to calculate the interpolation of the four color channels. Then, we can use interpolation and focus response function to calibrate the different differences in sensitivity of the four pixels at the five focus positions, and in the calibration process, fit the values ​​of the focus parameters a and b of the focus response function.

[0121] Among them, the bilinear interpolation algorithm is a method of estimating based on the values ​​of four neighboring points. Assume that there are four points, namely (x1, y1), (x2, y1), (x1, y2), (x2, y2), which correspond to values ​​f(x1, y1), f(x2, y1), f(x1, y2), f(x2, y2), respectively. At this time, to estimate a point (x, y), so that x1≤x≤x2, y1≤y≤y2 holds, the corresponding calculation formula of the bilinear interpolation algorithm is as follows:

[0122] f(x,y)=f(x1,y1)×(x2-x)(y2-y)+f(x2,y1)×(x-x1)(y2-y)+f(x1,y2)×(x2-x)(y-y1)+f(x2,y2)×(x-x1)(y-y1)

[0123] Where f(x, y) represents the estimated value for the point (x, y).

[0124] In addition, for the 16 sub-channels R0, R1, R2, R3, Gr0, Gr1, Gr2, Gr3, Gb0, Gb1, Gb2, Gb3, B0, B1, B2, and B3 contained in any calibration image, after using R0, R1, R2, R3, Gr0, Gr1, Gr2, Gr3, Gb0, Gb1, Gb2, Gb3, B0, B1, B2, and B3 to represent the pixel values ​​of the corresponding channels respectively, the average values ​​of R0, R1, R2, and R3 can be calculated as <r>, the average value of Gr0, Gr1, Gr2, and Gr3 is <gr>The average values ​​of Gb0, Gb1, Gb2, and Gb3 are <gb>The average value of B0, B1, B2, and B3 is .

[0125] Then, the calculation formula of the above bilinear interpolation algorithm can be used to calculate the interpolation of the four color channels respectively, which is specifically expressed as follows:

[0126] R Diff max =max{R i - <r>} / <r>

[0127] Gr Diff max =max{Gr i - <gr>} / <gr>

[0128] Gb Diff max =max{Gb i - <gb>} / <gb>

[0129] B Diff max =max{B i - } /

[0130] Among them, the value of i is 0, 1, 2, 3.

[0131] Thus, the global four-pixel sensitivity difference value in the calibration image obtained at the corresponding focus position can be calculated. The specific calculation formula is as follows:

[0132] Channel Diff=max{R Diff max ,Gr Diff max ,Gb Diff max ,B Diff max }

[0133] Similarly, the global four-pixel sensitivity difference value (Channel Diff) in the four calibration images obtained at the other four focus positions can be obtained.

[0134] Then define the five points of the calibration image obtained when the lens focus position is the 1st step, 200th step, 400th step, 600th step and 800th step as P1, P2, P3, P4, P5, and the corresponding lens step positions are defined as x1, x2, x3, x4, x5, and the global four-pixel sensitivity difference values ​​(Channel Diff) in the five calibration images obtained by the above calculation steps are divided into F1, F2, F3, F4, F5. Then set p0 corresponding to the quasi-focus position to the position of the 400th step (the specific value is not limited, and it is usually set to the 400th step to ensure clear imaging), and place the corresponding light box position at the object distance corresponding to the 400-step image distance. In this way, by converting 400 steps into lens travel, the lens advancement can be determined. For example, if the full lens travel is 2mm, the lens advancement is 1mm after conversion.

[0135] On this basis, the values ​​of x1, x2, x3, x4, x5 and F1, F2, F3, F4, F5 and p0 are substituted into the focus response function By performing least square fitting, the specific values ​​of focus parameters a and b can be obtained.

[0136] For example, when the values ​​of x1, x2, x3, x4, and x5 are 1, 200, 400, 600, and 800, respectively, the corresponding values ​​of F1, F2, F3, F4, and F5 are 7%, 4%, 2.2%, 4.2%, and 7.2%, and when p0 = 400, the following can be fitted: Figure 7 The curve shown in FIG. 1 shows a and determines that a=4.9992 and b=0.0003.

[0137] S404: Using the calibrated NxN pixel phase focus sensor, obtain P target images at P focus positions, and calculate the difference values ​​of NxN pixel sensitivities corresponding to the P target images; wherein P is a positive integer greater than 0.

[0138] In this embodiment, after the NxN pixel phase focus sensor configured in the camera device is calibrated in step S401, the calibrated NxN pixel phase focus sensor can be further used to obtain an image (defined as a target image, with no specific size limit) at each of the P focus positions to obtain P calibration images, and calculate the difference values ​​of the NxN pixel sensitivities corresponding to the P target images (such as the four-pixel sensitivity difference value Channel Diff) to perform the subsequent step S405. Wherein, K is a positive integer greater than 0.

[0139] It should be noted that the present application does not limit the value of P and the specific values ​​of the P focus positions, which can be set according to actual conditions and empirical values. In some embodiments, the value of P can be set to 2, and the two focus positions can be set to any two lens positions. An image is obtained at each of the two positions to obtain two target images, and then the difference in the NxN pixel sensitivity corresponding to the two target images (Channel Diff) is calculated through a similar implementation method to the above step S403.

[0140] S405: fitting a quasi-focus position using the focus parameters of the focus response function and the difference in NxN pixel sensitivities corresponding to the P target images, and focusing is performed according to the lens position of the camera device determined by the quasi-focus position.

[0141] In this embodiment, after fitting the specific values ​​of focus parameters a and b of the focus response function in step S403 and calculating the difference value (Channel Diff) of the NxN pixel sensitivity corresponding to P (e.g., 2) target images (e.g., 2) in step S404, the specific values ​​of focus parameters a and b of the focus response function and the difference value (Channel Diff) of the NxN pixel sensitivity corresponding to the P (e.g., 2) target images can be further used to fit the quasi-focal position by the least squares method, and the lens position of the camera device determined by the quasi-focal position can be used to achieve precise focusing, so that precise focusing can be achieved by using only the configured image sensor (especially the NxN pixel phase focus sensor) without adding any additional focusing devices, thereby not only solving the problem of poor focusing effect in pure color scenes, but also reducing the focusing cost.

[0142] Example: To facilitate understanding of the focusing method provided by this application, this application will be explained through an example scene of shooting a pure color scene, such as Figure 8 As shown, in this example scenario, Figure 8 (a) shows that when a user uses a mobile phone, camera or other camera device to shoot a pure color scene (such as the sky or a white wall), no matter whether the contrast focus or phase focus is used, the position of the lens needs to be repeatedly pushed and pulled. However, no matter how the lens position is moved, the corresponding phase difference (PD value) or contrast value is the same as above. Figure 1 The straight line shown in FIG. 1 cannot find the quasi-focus and accurate lens position, and the focusing effect is poor. After the focusing method provided by the present application is used for focus optimization processing, as shown in FIG. Figure 8 As shown in (b), the quasi-focus position can be fitted by moving the lens only twice. Figure 8 The position indicated by x0 shown in (c) is as follows: Figure 8 As shown in (c), assuming that the lens is moved to the position of x1 = 200 steps for the first time, the Channel Diff value F1 = 6.21%, and the lens is moved to the position of x2 = 300 steps for the second time, the Channel Diff value F2 = 4.68%, and then the quasi-focus x0 = 550 steps can be fitted by the least squares method, that is, Figure 8 The position indicated by x0 shown in (c) above realizes depth perception of pure color scenes and enables precise focusing at the quasi-focus x0, which not only solves the problem of poor focusing effect in pure color scenes but also reduces the focusing cost.

[0143] In addition, the present application also provides an electronic device (i.e., a camera device). For the hardware structure and software framework of the electronic device, please refer to Figure 2 and Figure 3 The electronic device includes a memory and a processor, the memory stores a computer program, and the processor is used to call and execute the computer program to implement the focusing method provided in the above description.

[0144] The present application also provides a computer-readable storage medium in an embodiment, on which a computer program is stored. When the computer program is executed by a processor of a terminal device, it is used to implement the focusing method provided in the above description.

[0145] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application. < / gb> < / gb> < / gr> < / gr> < / r> < / r> < / gb> < / gr> < / r>

Claims

1. A focusing method, characterized in that: Applied to a photographing device, the photographing device is equipped with an NxN pixel phase focusing sensor, and the method comprises: Calibrate the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor to calibrate the difference in sensitivity of NxN pixels at the focal plane position and reduce the difference in sensitivity of NxN pixels at the focal plane position; N is a positive integer greater than 1; Using the calibrated NxN pixel phase focus sensor, obtain K calibration images at preset K focus positions, where K is a positive integer greater than 0; Using the K calibration images and the focus response function, calibrate different difference values ​​of NxN pixel sensitivity at the preset K focus positions, and in the calibration process, fit the focus parameters of the focus response function; Using the calibrated NxN pixel phase focus sensor, P target images are acquired at P focus positions, and the difference values ​​of the NxN pixel sensitivities corresponding to the P target images are calculated; where P is a positive integer greater than 0; The quasi-focus position is fitted by utilizing the focus parameters of the focus response function and the difference in the NxN pixel sensitivities corresponding to the P target images, and focusing is performed according to the lens position of the camera device determined by the quasi-focus position.

2. The method according to claim 1, characterized in that The value of N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; the microlens corresponds to four pixels; the difference in sensitivity of the NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor is calibrated to calibrate the difference in sensitivity of the NxN pixels at the focal plane position, and reduce the difference in sensitivity of the NxN pixels at the focal plane position, including: The difference in sensitivity of four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor is calibrated to calibrate the difference in sensitivity of the four pixels at the focal plane position and reduce the difference in sensitivity of the four pixels at the focal plane position.

3. The method according to claim 2, characterized in that The calibrating the difference in sensitivity of four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor to calibrate the difference in sensitivity of the four pixels at the focal plane position and reduce the difference in sensitivity of the four pixels at the focal plane position includes: Using the photographing device to photograph a light plate with uniform illumination, and setting half of the sum of the focus position of the lens at infinity and the focus position of the lens at the closest focus distance as the calibration focus position; Using the 2x2 pixel phase focus sensor, a calibration image is acquired at the calibration focus position; the calibration image includes four color channels; wherein each sub-channel under each color channel shares a color filter of the corresponding color channel; Using a preset pixel scaling algorithm, the calibration image is subjected to pixel reduction processing to obtain a reduced calibration image; Merging the sub-channels under each color channel contained in the reduced calibration image, and calculating the difference between each merged channel and the pixel mean of the color channel; The compensation values ​​of each merged channel and its corresponding color channel pixel average are calculated by using the difference values ​​between the merged channels and their corresponding color channel pixel averages; and the corresponding channel pixels are compensated by using the compensation values ​​to calibrate the difference values ​​of the sensitivity of the four pixels at the focal plane position and reduce the difference in the sensitivity of the four pixels at the focal plane position.

4. The method according to claim 3, characterized in that The preset pixel scaling algorithm is a bilinear interpolation algorithm.

5. The method according to claim 3, characterized in that: The step of merging the sub-channels under each color channel contained in the reduced calibration image and calculating the difference between each merged channel and the pixel mean of the color channel includes: Calculating the mean value of each color channel pixel included in the reduced calibration image; The ratio of each merged channel pixel to the average value of the corresponding color channel pixel is calculated, and the difference between the ratio and the number 1 is used as the difference value of the corresponding channel.

6. The method according to claim 5, characterized in that The method of calculating the compensation values ​​of each merged channel and the pixel mean of each color channel by using the difference values ​​of each merged channel and the pixel mean of each color channel comprises: The sum of the difference between each merged channel and the pixel mean of the color channel and the number 1 is calculated respectively, and the ratio of the number 1 to the sum is calculated as the compensation value of the corresponding channel and the pixel mean of the color channel.

7. The method according to claim 1, characterized in that The value of K is 5; the preset K focus positions are 5 step positions corresponding to dividing the lens travel into 5 equal parts; The method uses the calibrated NxN pixel phase focus sensor to obtain K calibration images at preset K focus positions, including: Using the calibrated NxN pixel phase focus sensor, a calibration image is acquired at each of the five step positions corresponding to the lens travel divided into five equal parts, thereby obtaining five calibration images.

8. The method according to claim 1, characterized in that The method of using the K calibration images and the focus response function to calibrate different difference values ​​of the NxN pixel sensitivities at the preset K focus positions, and fitting the focus parameters of the focus response function during the calibration process, includes: Calculating the average pixel value of each color channel contained in the K calibration images; and calculating the interpolation value of each color channel using the average pixel value; The interpolation and focus response function are used to calibrate different difference values ​​of NxN pixel sensitivity at the preset K focus positions, and in the calibration process, the focus parameters of the focus response function are fitted.

9. The method according to claim 7, characterized in that: The value of N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; four pixels correspond to each other under the microlens; the different difference values ​​of the sensitivity of the NxN pixels at the preset K focus positions are calibrated by using the K calibration images and the focus response function, and during the calibration process, the focus parameters of the focus response function are fitted, including; Calculating the average pixel value of the four color channels included in the five calibration images; and calculating the interpolation of the four color channels using the average pixel value; The interpolation and focus response function are used to calibrate different difference values ​​of the four-pixel sensitivity at the five preset focus positions, and during the calibration process, the focus parameters of the focus response function are fitted.

10. The method according to claim 1, characterized in that The value of P is 2; the method of using the calibrated NxN pixel phase focus sensor to obtain P target images at P focus positions, and calculating the difference values ​​of the NxN pixel sensitivities corresponding to the P target images, includes: Using the calibrated NxN pixel phase focus sensor, two target images are acquired at two focus positions, and the difference in the NxN pixel sensitivities corresponding to the two target images is calculated.

11. The method according to claim 10, characterized in that The method of using the focus parameter of the focus response function and the difference between the NxN pixel sensitivities corresponding to the P target images to fit a quasi-focus position, and focusing according to the lens position of the camera device determined by the quasi-focus position, includes: The quasi-focus position is fitted by the least squares method using the focus parameters of the focus response function and the difference in NxN pixel sensitivities corresponding to the two target images, and focusing is performed based on the lens position of the camera device determined by the quasi-focus position.

12. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor is used to call and execute the computer program to implement the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by an electronic device, the method according to any one of claims 1 to 11 is implemented.

Images