Focusing method, electronic device and storage medium

Abstract

The present application provides a focusing method, electronic device and storage medium, which relate to the field of image processing technology. The method includes: first calibrating the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN OCL sensor pre-configured on the photographic device to reduce the difference in sensitivity of NxN pixels at the focal plane position, and then using the calibrated NxN OCL sensor to obtain K calibration images at preset K focus positions, so as to use them and the focus response function to calibrate the difference in sensitivity of NxN pixels at these K focus positions, and in the calibration process, fit the focus parameters. Then, using the calibrated NxN OCL sensor, obtain P target images at P focus positions, and calculate the difference in sensitivity of NxN pixels corresponding to the P target images. Then, using the obtained difference value and the focus parameter, accurately fit the quasi-focus position, and accurately focus the lens position of the photographic device determined according to the quasi-focus position.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a focusing method, electronic device and storage medium. Background Art

[0002] With the continuous development of technology, people can use mobile phones, tablet computers, cameras and other photography devices to record every detail of their lives anytime and anywhere. During the photography process, whether the focus can be achieved and whether the focus is accurate often affects the clarity of the photo.

[0003] Currently, commonly used focusing methods include on-chip phase focusing and contrast focusing. Although these two focusing methods can show better focus and defocus recognition capabilities for areas with rich textures and accurately determine the focus position, when it comes to shooting pure color scenes, such as shooting the sky or white walls, the recognition effect of these two focusing methods is weak, and it is easy to have problems such as repeated focus pushing and pulling, hysteresis or focus errors. To improve this situation, the current improvement method is usually to add an additional laser focus module to the camera device to improve the focus effect when shooting pure color scenes by sensing the absolute distance. However, this improvement solution has the problems of ranging distance limitation and increased cost. Summary of the Invention

[0004] In order to solve the above problems, the present application provides a focusing method, an electronic device and a storage medium, the purpose of which is to use the multi-pixel phase focusing sensor configured on the photographing device to achieve accurate estimation of the quasi-focus of pure color scene shooting. Accurate focusing can be achieved without adding any additional focusing devices, thereby not only solving the problem of poor focusing effect in pure color scenes, but also reducing the focusing cost.

[0005] In a first aspect, the present application provides a focusing method, comprising: first, calibrating the difference in sensitivity of NxN pixels at a focal plane position corresponding to an NxN pixel phase focus sensor pre-configured on a camera device to calibrate the difference in sensitivity of the NxN pixels at the focal plane position, thereby minimizing the difference in sensitivity of the NxN pixels at the focal plane position; then, using the calibrated NxN pixel phase focus sensor, acquiring K calibration images at K preset focal positions; and using the K calibration images and a focus response function (the specific source of which is not limited, such as one pre-constructed by the NxN pixel phase focus sensor module manufacturer) to calibrate the different difference values ​​of the NxN pixel sensitivity at the preset K focal positions. During the calibration process, focusing parameters of the focus response function are fitted. Next, using the calibrated NxN pixel phase focus sensor, acquiring P target images at P focal positions, and calculating the difference in sensitivity of the NxN pixels corresponding to the P target images. The focus parameters of the focus response function and the difference in the NxN pixel sensitivities corresponding to the P target images can then be used to accurately fit the quasi-focus position. The lens position of the camera device determined based on the quasi-focus position can then be precisely focused to obtain an image with a better focus effect.

[0006] It can be seen that in the above-mentioned focusing method, there is no need to add any new focusing devices to the photographing device (such as a mobile phone). Only by utilizing the pixel sensitivity differences corresponding to the NxN pixel phase focusing sensor (referring to an image sensor that obtains image data) pre-configured in the photographing device (such as a mobile phone), accurate estimation of the quasi-focus and identification of the distance for shooting pure-color scenes can be achieved, which not only solves the problem of poor focusing effect in pure-color scenes, but also reduces the focusing cost.

[0007] In one possible implementation, the value of N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; there are four pixels corresponding to the microlens; calibrating the difference in sensitivity of the NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor to calibrate the difference in sensitivity of the NxN pixels at the focal plane position and reducing the difference in sensitivity of the NxN pixels at the focal plane position may include: calibrating the difference in sensitivity of the four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor to calibrate the difference in sensitivity of the four pixels at the focal plane position and reducing the difference in sensitivity of the four pixels at the focal plane position to improve the focusing effect.

[0008] In one possible implementation, calibrating the difference in sensitivity of four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor to calibrate the difference in sensitivity of the four pixels at the focal plane position and reduce the difference in sensitivity of the four pixels at the focal plane position can include: using a camera to photograph a light plate with uniform illumination, and setting half of the sum of the focus position of the lens at infinity and the focus position of the lens at the minimum focus distance as the calibration focus position; using the 2x2 pixel phase focus sensor to obtain a calibration image at the calibration focus position; wherein the calibration image includes four color channels; each sub-channel under each color channel shares the color of the corresponding color channel Color filter; using a preset pixel scaling algorithm, the calibration image is pixel-reduced to obtain a reduced calibration image; the sub-channels under each color channel contained in the reduced calibration image are merged, and the difference values ​​between each merged channel and the average pixel value of the color channel are calculated; using the difference values ​​between each merged channel and the average pixel value of the color channel, the compensation values ​​between each merged channel and the average pixel value of the color channel are calculated; and the corresponding channel pixels are compensated using the compensation values ​​to more accurately calibrate the difference values ​​of the sensitivity of the four pixels at the focal plane position, and minimize the difference in the sensitivity of the four pixels at the focal plane position.

[0009] In a possible implementation, the preset pixel scaling algorithm is a bilinear interpolation algorithm.

[0010] In one possible implementation, merging the subchannels of each color channel included in the reduced calibration image and calculating the difference between each merged channel and the mean of its corresponding color channel pixels may include: calculating the mean of each color channel pixel included in the reduced calibration image; calculating the ratio of each merged channel pixel to the mean of the corresponding color channel pixel; and using the difference between the ratio and the number 1 as the difference value for the corresponding channel. This improves the efficiency and accuracy of the difference calculation.

[0011] In one possible implementation, the difference values ​​between each merged channel and its corresponding color channel pixel mean are used to calculate the compensation values ​​of each merged channel and its corresponding color channel pixel mean, which can include: respectively calculating the sum of the difference values ​​between each merged channel and its corresponding color channel pixel mean and the number 1, and calculating the ratio of the number 1 to the sum as the compensation value of the corresponding channel and its corresponding color channel pixel mean, so as to improve the calculation accuracy of the compensation value.

[0012] In one possible implementation, the value of K is 5; the K focus positions are preset as 5 step positions corresponding to dividing the lens travel into 5 equal parts; and using a calibrated NxN pixel phase focus sensor to obtain K calibration images at the preset K focus positions may include: using the calibrated NxN pixel phase focus sensor to obtain a calibration image at each of the 5 step positions corresponding to dividing the lens travel into 5 equal parts, thereby obtaining 5 calibration images to improve calibration efficiency.

[0013] In one possible implementation, the K calibration images and the focus response function are used to calibrate the different differences in NxN pixel sensitivity at K preset focus positions, and during the calibration process, focus parameters of the focus response function are fitted. This includes: calculating the average pixel value of each color channel included in the K calibration images; and using the average pixel value to calculate an interpolation of each color channel; using this interpolation and the focus response function, calibrating the different differences in NxN pixel sensitivity at K preset focus positions, and fitting the focus parameters of the focus response function during the calibration process. This improves the accuracy of the focus parameter fitting.

[0014] In one possible implementation, N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; there are four pixels corresponding to the microlens; K calibration images and a focus response function are used to calibrate the different difference values ​​of the NxN pixel sensitivities at the preset K focus positions, and during the calibration process, the focus parameters of the focus response function are fitted, which may include: calculating the average pixel value of the four color channels contained in the five calibration images; and using the average pixel value to calculate the interpolation of the four color channels; using the interpolation and the focus response function, the different difference values ​​of the four-pixel sensitivities at the preset five focus positions are calibrated, and during the calibration process, the focus parameters of the focus response function are fitted more quickly to improve the fitting efficiency.

[0015] In one possible implementation, the value of P is 2; using a calibrated NxN pixel phase focus sensor, P target images are obtained at P focus positions, and the difference in NxN pixel sensitivities corresponding to the P target images is calculated. This may include: using a calibrated NxN pixel phase focus sensor, two target images are obtained at two focus positions, and the difference in NxN pixel sensitivities corresponding to the two target images is calculated to improve computational efficiency.

[0016] In one possible implementation, using focus parameters of a focus response function and the difference in NxN pixel sensitivities corresponding to P target images to fit a near-focus position, and performing focusing based on the camera lens position determined by the near-focus position, may include: using focus parameters of a focus response function and the difference in NxN pixel sensitivities corresponding to two target images to fit a near-focus position using a least squares method, and performing focusing based on the camera lens position determined by the near-focus position. This can improve focusing performance and reduce focusing costs.

[0017] In a second aspect, the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor is used to call and execute the computer program to implement the focusing method described in any one of the first aspects above.

[0018] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of an electronic device, is used to implement the focusing method described in any one of the first aspects above.

[0019] In a fourth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the focusing method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a scenario provided for an embodiment of the present application;

[0021] Figure 2 A schematic diagram of an electronic device provided in an embodiment of the present application;

[0022] Figure 3 A software structure diagram of the electronic device provided in the embodiment of the present application;

[0023] Figure 4 A flowchart of the focusing method provided in an embodiment of the present application;

[0024] Figure 5 An example diagram of the brightness distribution of pixels at the front focus, near focus, and back focus positions provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the four color channels R, Gr, Gb, and B included in the calibration image provided in an embodiment of the present application;

[0026] Figure 7 This is an example diagram of the fitting process of the focus response function provided in an embodiment of the present application;

[0027] Figure 8This is an example diagram of a focus scene for a camera device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and claims of this application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless the context clearly indicates otherwise.

[0029] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] The "multiple" involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first" and "second" are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance or order.

[0031] To make the description of the following embodiments clear and concise, a brief introduction to the relevant concepts or technologies is first given:

[0032] Contrast Detection Auto Focus (CDAF) is a technology that achieves focusing by analyzing image contrast information. It determines image sharpness based on image contrast, adjusting the camera lens's focus to achieve clarity. In contrast-detection auto focus, a camera captures several images at different focal lengths and then analyzes the contrast of objects in these images to determine which focal length produces the sharpest image. Images with higher contrast generally indicate sharper images. This focusing method uses the contrast between edges and details in an image to determine image sharpness.

[0033] Phase Detection Autofocus (PDAF) is a technology used for automatic focus (AF) that uses pixels on an image sensor to determine the focus position by comparing them at different phases. This technology is commonly used in cameras and other photographic devices and can focus quickly and accurately. PDAF uses the difference in light phase between specific pixels on an image sensor to determine the focus position. On an image sensor, light passes through a lens and is divided into two or more different paths to reach the pixels. The optical path difference between these paths causes a difference in light phase between different pixels. When the image is out of focus, the phase-focus image sensor can detect focus deviation and quantify this deviation by analyzing the light phase difference between two pixels. By continuously adjusting the lens position and detecting changes in the phase difference, the optimal focus position can be quickly located.

[0034] In order to enable people skilled in the art to understand the solution of the present application more clearly, the application scenarios of the technical solution of the present application are described below.

[0035] See also Figure 1 , which shows a scenario schematic diagram provided by an embodiment of the present application.

[0036] In this example scenario, a user uses a mobile phone, camera, or other camera device to shoot a pure color scene. Regardless of whether contrast or phase focus is used, the lens position needs to be repeatedly pushed and pulled. However, no matter how the lens position is moved, the corresponding phase difference (PD value) or contrast value remains the same. Figure 1 The straight line shown in the figure cannot find the exact focus and lens position. This is because the contrast focus method usually relies on the contrast difference in the image to focus. Figure 1 In the image shown, of a solid-color scene, the lack of distinct features or contrast makes it difficult for the contrast-based focusing method to determine the focus position. In solid-color scenes, all areas of the image appear similar, lacking clear edges or details, resulting in very low contrast. The contrast-based focusing method relies on finding contrast differences between image areas to determine the focus position. However, in solid-color scenes, these contrast differences are minimal or almost non-existent, making it difficult for the contrast-based focusing method to accurately identify the focus position. Consequently, issues such as repeated focus shifts, lags, or focus errors can occur. Phase-based focusing also struggles to obtain sufficient phase difference information in solid-color scenes because the entire scene or area lacks distinct features or details. Consequently, the phase-based focusing method also struggles to accurately determine the focus position.

[0037] To address this issue, current approaches typically involve adding a laser focus module to mobile phones, cameras, and other photographic devices. This module uses absolute distance sensing to improve focus when capturing pure-color scenes. However, this approach suffers from limitations in distance measurement and increased costs.

[0038] In order to overcome the above technical problems, the present application provides a focusing method. By utilizing the pixel sensitivity differences corresponding to the NxN pixel phase focus sensor (referring to an image sensor that obtains image data, which can be represented by an NxN On-ChipLens sensor, or simply referred to as an NxN OCL sensor) pre-installed on the camera device, accurate estimation of the quasi-focus and recognition of the distance for shooting pure color scenes can be achieved. Without adding any new focusing devices, accurate focusing can be achieved, thereby not only solving the problem of poor focusing effect in pure color scenes, but also reducing focusing costs.

[0039] Among them, this application does not limit the value of N, and it can be set according to actual conditions and empirical values. It only needs to be a positive integer greater than 1. For example, N can be set to 2, then the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor (i.e., QPD (Quadrant Photodiode) sensor or 2x2 OCL sensor). In this way, the sensitivity difference of the four pixels corresponding to the 2x2 OCL sensor can be used to achieve accurate estimation of the quasi-focus and recognition of the distance for pure color scene shooting, so as to further achieve precise focusing.

[0040] It should be noted that the focusing method provided in the embodiments of the present application can be applied to electronic devices with a camera function (i.e., camera devices), such as mobile phones, tablet computers, cameras, personal digital assistants (PDAs), desktop computers, laptop computers, notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, and wearable devices.

[0041] In order to enable people skilled in the art to more clearly understand the focusing method provided by the present application, the hardware architecture and software system architecture of the electronic device that implements the focusing method in real time are first introduced in detail below.

[0042] See also Figure 2 , which shows a schematic diagram of an electronic device (taking a mobile phone or other photographing device as an example) provided in an embodiment of the present application.

[0043] like Figure 2As shown, the electronic device 200 may include a processor 210, a mobile communication module 220, a wireless communication module 230, a sensor module 240, a display screen 250, an internal memory 260, a camera 270, an audio module 280, a speaker 280A, a receiver 280B, a microphone 280C, an earphone interface 280D, an antenna group 1 and an antenna group 2.

[0044] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0045] The processor 210 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may generate an operation control signal based on an instruction opcode and a timing signal to control instruction fetching and execution. For example, the controller may implement accurate near-focus estimation and distance recognition for pure color scene photography based on the sensitivity differences of each pixel corresponding to the NxN pixel phase focus sensor 240 pre-installed on the electronic device (photographing device) 200, thereby further achieving accurate focus on pure color scenes and improving the focusing effect.

[0046] Processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 210 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 210. If processor 210 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 210 latency, and thus improves system efficiency.

[0047] In some embodiments, the processor 210 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0048] The sensor module 240 can be used to obtain data signals of various aspects related to the electronic device 200 as a basis for implementing corresponding functions. In some embodiments, the sensor module 240 may include but is not limited to a pressure sensor, a gyroscope sensor, an image sensor, and the like. Among them, the image sensor (the present application adopts an NxN OCL sensor) can be used to obtain image data (such as a Raw image) around the electronic device and transmit the obtained image data to the electronic device 200. So that the electronic device 200 can calculate the difference in NxN pixel sensitivity under each color channel corresponding to the NxN pixel phase focus sensor based on these image data, so that the quasi-focus position can be accurately fitted according to the value of the difference, thereby achieving precise focusing. For details, please refer to the relevant introduction of the subsequent embodiments.

[0049] The internal memory 260 can be used to store computer executable program code, which includes instructions. The internal memory 260 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound acquisition function, an image shooting function, etc.), etc. The data storage area may store data created during the use of the electronic device 200 (such as audio data, image data, etc.), etc. In addition, the internal memory 260 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 210 executes various functional applications and data processing of the electronic device 200 by running instructions stored in the internal memory 260 and / or instructions stored in a memory provided in the processor.

[0050] In some embodiments, the internal memory 260 stores instructions for executing a focusing method. The processor 210 can execute the instructions stored in the internal memory 260 to implement the following functions: first, calibrate the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN OCL sensor pre-configured in the electronic device (camera) 200 to calibrate the difference in sensitivity of the NxN pixels at the focal plane position, thereby minimizing the difference in sensitivity of the NxN pixels at the focal plane position. Then, using the calibrated NxN pixel phase focus sensor, K calibration images are acquired at preset K focal positions. Using the K calibration images and a focus response function (the specific source is not limited, such as one pre-constructed by the NxN OCL sensor module manufacturer), the different difference values ​​of the NxN pixel sensitivity at the preset K focal positions are calibrated. During the calibration process, focus parameters of the focus response function are fitted. Next, using the calibrated NxN pixel phase focus sensor, P target images are acquired at P focal positions, and the difference values ​​of the NxN pixel sensitivity corresponding to these P target images are calculated. The focus parameters of the focus response function and the difference in the NxN pixel sensitivities corresponding to the P target images can then be used to accurately fit the quasi-focus position. The lens position of the camera device determined based on the quasi-focus position can then be precisely focused to obtain an image with a better focus effect.

[0051] The display screen 250 is used to display images, videos, and the like, such as a captured scene of pure color, such as the sky or a white wall. The display screen 250 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, the electronic device 200 may include one or more display screens 250.

[0052] The camera 270 is used to capture still images or videos, and the object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include one or more cameras 193.

[0053] Electronic device 200 implements display functionality through a GPU, display screen 250, and an application processor. The GPU is a microprocessor for image processing that connects display screen 250 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or modify display information.

[0054] The electronic device 200 can implement audio functions such as music playback, recording, and voice input and output through the audio module 280, speaker 280A, receiver 280B, microphone 280C, headphone jack 280D, and application processor.

[0055] The audio module 280 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 280 can also be used to encode and decode audio signals. In some embodiments, the audio module 280 can be provided in the processor 210, or some functional modules of the audio module 280 can be provided in the processor 210.

[0056] The speaker 280A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 200 can listen to music or listen to hands-free calls through the speaker 280A.

[0057] The receiver 280B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 200 receives a call or a voice message, the user can place the receiver 280B close to the ear to hear the voice.

[0058] Microphone 280C, also known as a "microphone" or "speaker," is used to convert sound signals into electrical signals. When making a call or sending a voice message, a user can speak by placing their mouth close to microphone 280C, inputting the sound signal into microphone 280C.

[0059] The headphone jack 280D is used to connect a wired headphone, and the standard attributes of the interface are not limited.

[0060] It should be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 200.

[0061] The wireless communication function of the electronic device 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 220, the wireless communication module 230, the modem processor and the baseband processor.

[0062] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0063] The mobile communication module 220 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 200. The mobile communication module 220 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 220 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 220 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 220 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 220 can be set in the same device as at least some of the modules of the processor 210.

[0064] The wireless communication module 230 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 200. The wireless communication module 230 can be one or more devices integrating at least one communication processing module. The wireless communication module 230 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210. The wireless communication module 230 can also receive the signal to be sent from the processor 210, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0065] In addition, on top of the above components, the electronic device 200 runs an operating system, such as an iOS operating system, an Android operating system, a Windows operating system, etc. Applications can be installed and run on the operating system.

[0066] See also Figure 3 , which shows a schematic diagram of the software structure of an electronic device (taking a mobile phone or other photographing device as an example) provided in an embodiment of the present application.

[0067] The software system of the electronic device (taking a mobile phone as an example) 200 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 200.

[0068] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: application program layer (APK), application framework layer (Framework), hardware abstraction layer (HAL), driver layer, and hardware layer, from top to bottom.

[0069] The application layer can include a series of application packages (APP). Figure 3 As shown, the application package may include camera, navigation, WLAN, Bluetooth, gallery and other applications. When the user holds the electronic device 200 to shoot (such as shooting Figure 1 (as shown when shooting a pure color scene), the camera application can communicate with the camera-related devices in the camera access interface of the framework layer to request camera functions and obtain image data, etc.

[0070] The application framework layer (framework layer) provides application programming interface (API) and programming framework for the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a window manager, a notification manager, a resource manager, and camera access interfaces (including but not limited to camera management and camera devices). The application framework layer enables interaction between camera services and camera APIs. In other words, it provides a unified interface that enables different camera hardware to interact with different camera applications. The framework layer also handles many common aspects of camera functionality, such as autofocus and motion focus.

[0071] The window manager is used to manage window programs. It can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0072] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0073] In this embodiment, the hardware abstraction layer (HAL) provides a set of standard interfaces, which enables the framework layer to communicate with camera hardware from various manufacturers without having to understand the underlying hardware details. The hardware abstraction layer (HAL) stores the hardware abstraction layer and the camera algorithm library. It should be noted that the focus algorithm provided by this application can be stored in the camera algorithm library of the HAL layer, such as Figure 3 shown.

[0074] The focusing algorithm provided herein is used to accurately fit a near-focus position by calculating the difference in NxN pixel sensitivities across each color channel corresponding to the NxN pixel phase focus sensor 240, thereby further achieving precise focusing. Specifically, the difference in NxN pixel sensitivities at the focal plane corresponding to the pre-configured NxN OCL sensor in the electronic device 200 (for example, a mobile phone) is first calibrated to minimize the difference in NxN pixel sensitivities at the focal plane. K calibration images are then acquired at K preset focal positions using the calibrated NxN pixel phase focus sensor. The K calibration images and a focus response function are then used to calibrate the difference in NxN pixel sensitivities at the K preset focal positions. During the calibration process, focus parameters of the focus response function are fitted. Next, P target images are acquired at P focal positions using the calibrated NxN pixel phase focus sensor, and the difference in NxN pixel sensitivities corresponding to these P target images is calculated. The focus parameters of the focus response function and the difference in the NxN pixel sensitivities corresponding to the P target images can then be used to accurately fit the quasi-focus position. The lens position of the camera device determined based on the quasi-focus position can then be precisely focused to obtain an image with a better focus effect.

[0075] The hardware layer may include the hardware components of the aforementioned electronic device (taking a mobile phone as an example) 200. For example, Figure 3 The NxN pixel phase focus sensor, image signal processor, digital signal processor, and graphics processor were demonstrated.

[0076] Among them, the NxN pixel phase focus sensor is used for image exposure and focus processing, etc.

[0077] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 200 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0078] In this way, the interoperability of different camera applications and hardware devices on the Android platform can be achieved through the application layer, framework layer, HAL layer, driver layer and hardware layer, which can achieve accurate estimation of the focus when shooting pure-color scenes, thereby achieving precise focus without adding any additional focusing devices. This not only improves the focusing effect for pure-color scenes, but also reduces the focusing cost.

[0079] The technical solutions involved in the following embodiments can all be implemented in electronic devices having the above-mentioned hardware architecture and software architecture.

[0080] Next, the specific implementation process of the focusing method provided by this application will be introduced in detail:

[0081] like Figure 4 As shown, the specific implementation process of the focusing method may include the following steps S401-S405:

[0082] S401: Calibrate the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor to calibrate the difference in sensitivity of NxN pixels at the focal plane position and reduce the difference in sensitivity of NxN pixels at the focal plane position; where N is a positive integer greater than 1.

[0083] It should be noted that in order to achieve precise focusing of pure color scenes by camera devices (such as mobile phones and tablets) without adding any new focusing devices, this application sets the image sensor pre-installed in the camera device as an NxN pixel phase focus sensor (NxN OCL sensor), which is used to accurately fit the quasi-focus position by calculating the difference in the sensitivity of the NxN pixels under each color channel corresponding to the NxN pixel phase focus sensor, so as to further achieve precise focusing. In this way, precise focusing can be achieved without adding any new focusing devices, which not only solves the problem of poor focusing effect in pure color scenes, but also reduces focusing costs.

[0084] The present application does not limit the specific composition and structure of the NxN OCL sensor. Its structural characteristics may include: including a microlens, and one microlens corresponds to NxN pixels, where N is a positive integer greater than 1. For example, when the value of N is 2, the NxN OCL sensor is a 2x2 OCL sensor, and one microlens corresponds to four pixels. Next, the structure of the NxN OCL sensor is described in detail using a 2x2 OCL sensor as an example. Specifically, it may include:

[0085] (1) Microlens: In a 2x2 OCL sensor, each microlens is located at the center of four adjacent pixels, forming a 2x2 pixel area. The function of the microlens is to focus light from different directions onto the corresponding pixel. This layout helps to enhance the light reception efficiency and improve the light utilization rate of the sensor.

[0086] (2) Color filter array (CFA): In a 2x2 OCL sensor, the color filters are typically arranged in a four-pixel Bayer pattern. In this layout, each 2x2 pixel area contains a red filter, a blue filter, and two green filters. This arrangement enables each pixel to capture information for the three colors red (R), green (G), and blue (B), thus enabling full-color image acquisition.

[0087] (3) Pixel arrangement: Four pixels are arranged in a 2x2 array around each microlens, with each pixel corresponding to a filter. This layout design enables each microlens to focus and distribute light to its corresponding pixel, thereby more efficiently capturing light signals.

[0088] In this way, based on the above-mentioned characteristics of the NxN OCL sensor (taking a 2x2 OCL sensor as an example), this application can use the NxN pixels under the microlens to sense the matching of the lens principal ray angle and the sensor principal ray angle, calculate the sensitivity difference of the NxN pixels, and use it to achieve accurate estimation of the quasi-focus and distance recognition for pure color scene shooting, thereby achieving precise focusing.

[0089] Specifically, still taking the 2x2 OCL sensor as an example, when the chief ray angle (CRA) is aligned with the microlenses of the four pixels, that is, there is a strict CRA match, the light will be more evenly distributed to the four pixels, making their sensitivity relatively consistent. If the CRA cannot be strictly matched to the microlenses of the four pixels, the angle of the incident light may cause slight differences on different pixels. Among them, the chief ray angle (CRA) refers to the angle between the line connecting the optical center of the optical system to a certain pixel and the normal of the pixel plane.

[0090] When the lens's chief ray angle (CRA) and the 2x2 OCL sensor's chief ray angle are strictly aligned (for example, within ±2° to 3°), the sensitivity difference between the four pixels is minimal. As the CRA alignment between the lens and sensor shifts from alignment to mismatch, the sensitivity difference between the four pixels gradually increases.

[0091] Therefore, in camera modules such as mobile phones, changes in the focus position will cause the chief ray angle (CRA) of the lens to change. This means that the CRA adaptation relationship between the lens and sensor changes at different focus positions. Therefore, the difference in the sensitivity of the four pixels at different focus positions also changes.

[0092] Based on this, this application proposes that the quantitative relationship between the 2*2 OCL sensitivity difference and the camera lens position can be used to achieve accurate focus judgment when shooting pure color scenes. Figure 5 As shown in the figure, at the front focus, quasi focus, and back focus positions, the angle of light perceived by a single pixel changes, and after being focused and refracted by the microlens, the brightness distribution of the four pixels also changes. Figure 5 As can be seen, the sensitivity difference between pixels at the front and back focus positions is large, while the sensitivity difference between pixels at the near-focus position is small. When the lens and sensor are fixed, the relationship between the sensitivity difference of NxN pixels under a microlens and the focus position is also fixed. This characteristic forms the basis for depth perception of captured images in this application. Therefore, by pre-calibrating the relationship between the NxN pixel sensitivity difference and the lens focus position, and collecting a small number of data points of the focus position and NxN pixel sensitivity difference through camera control, the near-focus position can be determined through fitting and other methods.

[0093] During the specific implementation process, it is first necessary to calibrate the difference in sensitivity of the NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor (NxNOCL sensor) configured in the camera device to calibrate the difference in sensitivity of the NxN pixels at the focal plane position (which can be represented by Channel Diff), and reduce the difference in sensitivity of the NxN pixels at the focal plane position, thereby obtaining a calibrated NxN OCL sensor for executing the subsequent step S402.

[0094] In some implementations, the value of N can be 2, then the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor (2x2 OCL sensor), and the 2x2 OCL sensor includes a microlens, and there are four pixels corresponding to the microlens. In this way, in order to improve the focusing effect, it is first necessary to perform quad-Bayer coding sensitivity correction (QSC) on the 2x2 OCL sensor configured in the camera device to correct the uniformity of the sensitivity of the 2*2 pixels, that is, to calibrate the difference in sensitivity of the four pixels at the focal plane position (Channel Diff) so that the difference in sensitivity of the four pixels at the focal plane position is minimized. The specific implementation process may include the following steps S4011-S4015:

[0095] S4011: Using a camera to photograph a light panel with uniform illumination, and setting half of the sum of the focus position of the lens at infinity and the focus position of the lens at the minimum focus distance as the calibration focus position.

[0096] In this implementation, when performing QSC calibration, the camera module of a photographic device (such as a mobile phone or camera) can first be used to capture a uniform light plate (the specific structure is not limited; for example, the structural characteristics of the uniform light plate can include large-area illumination, monochromatic temperature, D65 light source, and 3000 lux brightness). For example, when the lens focal length is 8.6mm, the object distance can be defined as 1 times the focal length to improve calibration accuracy. The focus position of the camera module of the photographic device (such as a mobile phone or camera) is set as follows: the focus position of the lens at infinity is defined as z1, the focus position of the lens at the minimum focus distance is defined as z2, and then the focus position is set to (z1+z2) / 2.

[0097] S4012: Acquire a calibration image at a calibration focus position using a 2x2 pixel phase focus sensor; wherein the calibration image includes four color channels; and each sub-channel under each color channel shares a color filter of the corresponding color channel.

[0098] In this implementation, after the focus position is determined in step S4011, a calibration image can be obtained by using a 2x2 OCL sensor at the calibration focus position (e.g., using a 2x2 OCL sensor to output a full-pixel resolution Raw image at the calibration focus position). The calibration image (e.g., Raw image) includes four color channels, such as Figure 6 The four color channels R, Gr, Gb, and B are shown. Each sub-channel under each color channel shares the color filter of the corresponding color channel. For example, Figure 6 Each color channel shown has subchannels numbered 0, 1, 2, and 3. For the R channel, its four subchannels, R0, R1, R2, and R3, share the R channel's color filter. For the Gr channel, its four subchannels, Gr0, Gr1, Gr2, and Gr3, share the Gr channel's color filter. For the Gb channel, its four subchannels, Gb0, Gb1, Gb2, and Gb3, share the Gb channel's color filter. For the B channel, its four subchannels, B0, B1, B2, and B3, share the B channel's color filter.

[0099] S4013: Using a preset pixel scaling algorithm, perform pixel reduction processing on the calibration image to obtain a reduced calibration image.

[0100] In this implementation, after obtaining the calibration image (such as the Raw image) through step S4012, in order to speed up the calculation efficiency, a pre-set pixel scaling algorithm can be further used to perform pixel reduction processing on the calibration image (such as the Raw image) to obtain a pixel-reduced calibration image for executing the subsequent step S4014.

[0101] Among them, this application does not limit the specific content of the preset pixel scaling algorithm, which can be selected according to actual conditions and experience values. For example, the preset pixel scaling algorithm can be set to a bilinear interpolation algorithm, so that the bilinear interpolation algorithm can be used to perform pixel reduction processing on the calibration image (such as the Raw image) to reduce the amount of calibration data and speed up calculation efficiency. In addition, this application does not limit the specific pixel values ​​of the calibration image (such as the Raw image) and the reduced calibration image. Assuming that the calibration image (such as the Raw image) is L pixels × H pixels (the values ​​of L and H are not limited, for example, L and H can be 512 and 224 respectively), after using the bilinear interpolation algorithm to perform pixel reduction processing on it, 64 pixels × 28 pixels can be obtained.

[0102] S4014: Merge the sub-channels under each color channel included in the reduced calibration image, and calculate the difference between each merged channel and the pixel mean of the color channel.

[0103] In this implementation, after obtaining the reduced calibration image (which may have a size of 64 pixels × 28 pixels) in step S4013, the sub-channels of each color channel contained in the reduced calibration image may be merged. Taking the four sub-channels R0, R1, R2, and R3 of the R color channel of a 2x2 OCL sensor as an example, each sub-channel may have a size of 16 pixels × 12 pixels, and the size of the 16-channel integrated QSC is 16*12*16. Here, 16 pixels = 64 pixels / (2*2), and 12 pixels = 28 pixels / (2*2).

[0104] Then, calculate the mean value of each color channel pixel in the reduced calibration image. For example, taking the four sub-channels R0, R1, R2, and R3 corresponding to the R color channel of a 2x2 OCL sensor as an example, this color channel covers 2x 2 pixels, and the pixel values ​​corresponding to the sub-channel numbers R0, R1, R2, and R3 are represented by R0, R1, R2, and R3 respectively. The calculated mean value of the color channel pixels is:

[0105] For a specific merged channel pixel, its ratio to the average value of the corresponding color channel pixels can be further calculated. For example, taking R0 in the R color channel of the 2x2 OCL sensor as an example, its ratio to the average value of the R color channel pixels is: Then use the difference between the ratio and the number 1 as the difference value of the corresponding channel and express it with D0, that is, Thus, when the mean value of the R color channel pixel When the value of pixel R0 is 9, the value of D0 can be calculated to be -10%, that is, D0 = 9 / 10--1 = -10%.

[0106] S4015: Calculate the compensation values ​​of each merged channel and its corresponding color channel pixel average using the difference values ​​between each merged channel and its corresponding color channel pixel average; and use the compensation values ​​to compensate the corresponding channel pixels to calibrate the difference values ​​of the sensitivity of the four pixels at the focal plane position and reduce the difference in the sensitivity of the four pixels at the focal plane position.

[0107] In this implementation, after obtaining the difference values ​​between each merged channel and its corresponding color channel pixel mean (such as the value of D0) through step S4014, the sum of the difference values ​​between each merged channel and its corresponding color channel pixel mean (such as the value of D0) and the number 1 can be further calculated respectively, and the ratio of the number 1 to the sum value can be calculated as the compensation value of the corresponding channel and its corresponding color channel pixel mean.

[0108] For example, taking R0 of the 2x2 OCL sensor corresponding to the R color channel as an example, its ratio to the mean value of the R color channel pixels is Then use the difference between the ratio and the number 1 as the difference value of the corresponding channel and express it with D0, that is, After further calculating the sum of D0 and the number 1 (i.e., 1+D0), the ratio of the value 1 to the sum is calculated as the compensation value of R0 and its mean value for the R color channel pixel, and is expressed as QR0, i.e., QR0=1 / (1+D0). In this way, when the value of D0 is -10%, it can be calculated that the value of QR0 is 1.111, i.e., QR0=1 / (1+D0)=1 / 0.9≈1.111.

[0109] Furthermore, the compensation value (such as QR0) can be used to compensate the corresponding channel pixel (such as R0). For example, for pixel R0, it can be multiplied by the compensation value QR0 so that the pixel value after compensation is close to the mean value of the R color channel pixel. Similarly, the difference in the sensitivity of the four pixels at the focal plane position can be calibrated, and the difference in the sensitivity of the four pixels at the focal plane position can be reduced (that is, they are all close to the mean value of the corresponding color channel pixels).

[0110] In this way, after calibration by the QSC, the pixel differences of each sub-channel in the Raw image finally output by the 2x2 OCL sensor can be minimized, thereby obtaining a calibrated 2x2 OCL sensor for executing the subsequent step S402.

[0111] It should be noted that when the value of N is another positive integer greater than 2, the calibration process for the NxN OCL sensor can be implemented by referring to the calibration method for the 2x2 OCL sensor described in steps S4011-S4045 above, and the specific calibration process is not repeated here.

[0112] S402: Using the calibrated NxN pixel phase focus sensor, obtain K calibration images at K preset focus positions; where K is a positive integer greater than 0.

[0113] In this embodiment, after calibrating the NxN pixel phase focus sensor configured in the camera device in step S401, the calibrated NxN pixel phase focus sensor can be used to capture an image (defined as a calibration image, with no specific size restrictions) at K preset focus positions, thereby obtaining K calibration images for executing the subsequent step S403. Here, K is a positive integer greater than 0.

[0114] Among them, it should be noted that this application does not limit the value of K and the specific values ​​of the preset K focus positions, which can be set according to actual conditions and experience values. In some embodiments, the value of K can be set to 5, and the preset K focus positions can be set to 5 step positions corresponding to the 5 equal divisions of the lens stroke. For example, assuming that the lens stroke (can be in mm) is divided into 800 steps, and then divided into 5 equal divisions, the AF positions can be set to the 1st step, the 200th step, the 400th step, the 600th step and the 800th step respectively. And acquire an image at each AF position to obtain 5 calibration images, which are used to calibrate the difference in NxN pixel sensitivity (Channel Diff) corresponding to the NxN OCL sensor at each AF position.

[0115] It should also be noted that for each calibration image, taking the color encoding scheme of the NxN Raw image as an example, there are four color channels: R, Gr, Gb, and B, and there are NxN color sub-channels under the color channel corresponding to one microlens.

[0116] S403: Using K calibration images and a focus response function, calibrate different difference values ​​of NxN pixel sensitivities at K preset focus positions, and during the calibration process, fit the focus parameters of the focus response function.

[0117] After acquiring K calibration images at K preset focus positions using the calibrated NxN OCL sensor in step S402, the average pixel value for each color channel in these K calibration images is calculated. Interpolation values ​​for each color channel are then calculated using these average pixel values. This interpolation value and the focus response function are then used to calibrate the differences in NxN pixel sensitivity at the K preset focus positions. During the calibration process, focus parameters of the focus response function are fitted and used to execute step S404.

[0118] The present application does not limit the specific value and source of the focus response function, which can be set according to actual conditions and empirical values. For example, it can be pre-built by the NxN OCL sensor module manufacturer, and can be represented by F(x, p0), specifically: Where p0 represents the position of the lens in the focus state, x represents any focus position during the shooting process, and a and b represent the focus parameters of the focus response function.

[0119] For example, assuming N is 2, an NxN OCL sensor is a 2x2 OCL sensor containing a microlens, each corresponding to four pixels. Furthermore, assuming K is 5, an image is acquired at each of the five steps corresponding to the lens travel divided into five equal parts. Five calibration images are obtained, each containing four color channels (R, Gr, Gb, and B), and 16 subchannels (R0, R1, R2, R3, Gr0, Gr1, Gr2, Gr3, Gb0, Gb1, Gb2, Gb3, B0, B1, B2, and B3).

[0120] On this basis, we can further use algorithms such as bilinear interpolation to calculate the average pixel value of the four color channels contained in the five calibration images, and use the obtained average pixel value to calculate the interpolation of the four color channels. Then, we can use interpolation and the focus response function to calibrate the different differences in the sensitivity of the four pixels at the five focus positions. During the calibration process, we can fit the focus parameters a and b of the focus response function.

[0121] Among them, the bilinear interpolation algorithm is a method of estimating based on the values ​​of four neighboring points. Assume that there are four points, namely (x1, y1), (x2, y1), (x1, y2), (x2, y2), which correspond to the values ​​​​f(x1, y1), f(x2, y1), f(x1, y2), f(x2, y2), respectively. At this time, to estimate a point (x, y), so that x1≤x≤x2, y1≤y≤y2 holds, the corresponding bilinear interpolation algorithm calculation formula is as follows:

[0122] f(x,y)=f(x1,y1)×(x2-x)(y2-y)+f(x2,y1)×(x-x1)(y2-y)+f(x1,y2)×(x2-x)(y-y1)+f(x2,y2)×(x-x1)(y-y1)

[0123] Here, f(x, y) represents the estimated value for the point (x, y).

[0124] In addition, for the 16 sub-channels R0, R1, R2, R3, Gr0, Gr1, Gr2, Gr3, Gb0, Gb1, Gb2, Gb3, B0, B1, B2, and B3 contained in any calibration image, after using R0, R1, R2, R3, Gr0, Gr1, Gr2, Gr3, Gb0, Gb1, Gb2, Gb3, B0, B1, B2, and B3 to represent the pixel values ​​of the corresponding channels respectively, the average value of R0, R1, R2, and R3 can be calculated as <r>The average value of Gr0, Gr1, Gr2 and Gr3 is <gr>The average value of Gb0, Gb1, Gb2, and Gb3 is <gb>The average value of B0, B1, B2, and B3 is .

[0125] Then, the calculation formula of the above bilinear interpolation algorithm can be used to calculate the interpolation of the four color channels respectively, which is specifically expressed as follows:

[0126] R Diff max =max{R i - <r>} / <r>

[0127] Gr Diff max =max{Gr i - <gr>} / <gr>

[0128] Gb Diff max =max{Gb i - <gb>} / <gb>

[0129] B Diff max =max{B i - } /

[0130] Among them, the value of i is 0, 1, 2, 3.

[0131] Thus, the global four-pixel sensitivity difference value in the calibration image obtained at the corresponding focus position can be calculated. The specific calculation formula is as follows:

[0132] Channel Diff=max{R Diff max ,Gr Diff max ,Gb Diff max ,B Diff max }

[0133] Similarly, the global four-pixel sensitivity difference value (Channel Diff) in the four calibration images obtained at the other four focus positions can be obtained.

[0134] The five points in the calibration images obtained when the lens focus positions are at the 1st, 200th, 400th, 600th, and 800th steps, respectively, are defined as P1, P2, P3, P4, and P5. The corresponding lens step positions are defined as x1, x2, x3, x4, and x5, respectively. The global four-pixel sensitivity difference (Channel Diff) values ​​in the five calibration images obtained through the above calculation steps are divided into F1, F2, F3, F4, and F5. The p0 corresponding to the near-focus position is then set to the position of the 400th step (the specific value is not limited; to ensure clear imaging, it is usually set to the 400th step). The corresponding light box position is placed at the object distance corresponding to the 400th step image distance. By converting the 400th step length to the lens travel, the lens advance can be determined. For example, if the full lens travel is 2mm, the converted lens advance is 1mm.

[0135] On this basis, the values ​​of x1, x2, x3, x4, x5 and F1, F2, F3, F4, F5 and p0 are substituted into the focus response function By performing least square fitting, the specific values ​​of focus parameters a and b can be obtained.

[0136] For example, when the values ​​of x1, x2, x3, x4, and x5 are 1, 200, 400, 600, and 800 respectively, the corresponding values ​​of F1, F2, F3, F4, and F5 are 7%, 4%, 2.2%, 4.2%, and 7.2%, and when p0 = 400, the following can be fitted: Figure 7 The curve shown in FIG. 1 is used to determine a=4.9992 and b=0.0003.

[0137] S404: Using the calibrated NxN pixel phase focus sensor, obtain P target images at P focus positions, and calculate the difference in NxN pixel sensitivities corresponding to the P target images; where P is a positive integer greater than 0.

[0138] In this embodiment, after calibrating the NxN pixel phase focus sensor configured in the camera device in step S401, the calibrated NxN pixel phase focus sensor can be further used to acquire an image (defined as a target image, with no specific size limitation) at each of P focus positions to obtain P calibration images. The sensitivity differences of the NxN pixels corresponding to these P target images (e.g., a four-pixel sensitivity difference value, Channel Diff) are then calculated for use in executing the subsequent step S405. Here, K is a positive integer greater than 0.

[0139] It should be noted that the present application does not limit the value of P or the specific values ​​of the P focus positions, and they can be set based on actual conditions and empirical values. In some embodiments, the value of P can be set to 2, and the two focus positions can be set to any two lens positions. An image is acquired at each of these two positions to obtain two target images. Then, through a similar implementation method as described in step S403 above, the difference in the NxN pixel sensitivities corresponding to the two target images (Channel Diff) is calculated.

[0140] S405: Using the focus parameters of the focus response function and the difference in NxN pixel sensitivities corresponding to the P target images, a near-focus position is fitted, and focusing is performed based on the lens position of the camera device determined by the near-focus position.

[0141] In this embodiment, after fitting the specific values ​​of focus parameters a and b of the focus response function in step S403 and calculating the difference value (Channel Diff) of the NxN pixel sensitivities corresponding to P (e.g., 2) target images (e.g., 2) in step S404, the specific values ​​of focus parameters a and b of the focus response function and the difference value (Channel Diff) of the NxN pixel sensitivities corresponding to the P (e.g., 2) target images can be further used to fit a quasi-focus position using the least squares method. The lens position of the camera device determined by the quasi-focus position is then used to achieve precise focusing. Thus, precise focusing can be achieved using only the configured image sensor (especially the NxN pixel phase focus sensor) without adding any additional focusing devices. This not only solves the problem of poor focusing effect in pure color scenes, but also reduces focusing costs.

[0142] Example: To facilitate understanding of the focusing method provided by this application, this application will illustrate an example scenario of shooting a pure color scene, such as Figure 8 As shown, in this example scenario, Figure 8 (a) shows that when a user uses a mobile phone, camera or other camera device to shoot a pure color scene (such as the sky or a white wall), whether using contrast focus or phase focus, the user needs to repeatedly push and pull the lens position. However, no matter how the lens position is moved, the corresponding phase difference (PD value) or contrast value is the same. Figure 1 The straight line shown in FIG. 1 is unable to find the quasi-focus and accurate lens position, and the focusing effect is poor. After the focusing method provided by this application is used for focus optimization, as shown in FIG. Figure 8 As shown in (b), the quasi-focus position can be fitted by moving the lens only twice. Figure 8 The position indicated by x0 shown in (c) is as follows: Figure 8 As shown in (c), assuming that the lens is moved to the position of x1 = 200 steps for the first time, the Channel Diff value F1 = 6.21% is obtained, and the lens is moved to the position of x2 = 300 steps for the second time, the Channel Diff value F2 = 4.68% is obtained. Then, the least squares method can be used to fit the quasi-focus x0 = 550 steps, that is, Figure 8 The position indicated by x0 shown in (c) above realizes depth perception of pure color scenes and enables precise focusing at the quasi-focus x0, which not only solves the problem of poor focusing effect in pure color scenes but also reduces the focusing cost.

[0143] In addition, the present application also provides an electronic device (ie, a camera device). For the hardware structure and software framework of the electronic device, please refer to Figure 2 and Figure 3 The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor is configured to call and execute the computer program to implement the focusing method provided in the above description.

[0144] The present application also provides a computer-readable storage medium in an embodiment, on which a computer program is stored. When the computer program is executed by a processor of a terminal device, it is used to implement the focusing method provided in the above description.

[0145] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application. < / gb> < / gb> < / gr> < / gr> < / r> < / r> < / gb> < / gr> < / r>

Claims

1. A focusing method, characterized in that: Applied to a photographing device equipped with an NxN pixel phase focus sensor, the method includes: Calibrating the difference in sensitivity of NxN pixels at a focal plane position corresponding to the NxN pixel phase focus sensor to calibrate the difference in sensitivity of the NxN pixels at the focal plane position and reduce the difference in sensitivity of the NxN pixels at the focal plane position; where N is a positive integer greater than 1; Using the calibrated NxN pixel phase focus sensor, obtain K calibration images at K preset focus positions, where K is a positive integer greater than 0; Using the K calibration images and the focus response function, calibrate different difference values ​​of NxN pixel sensitivities at the preset K focus positions, and during the calibration process, fit the focus parameters of the focus response function; Using a calibrated NxN pixel phase focus sensor, acquire P target images at P focus positions, and calculate the difference in sensitivity of the NxN pixels corresponding to the P target images, where P is a positive integer greater than 0; The quasi-focus position is fitted using the focus parameters of the focus response function and the difference in NxN pixel sensitivities corresponding to the P target images, and focusing is performed based on the lens position of the camera device determined by the quasi-focus position.

2. The method according to claim 1, characterized in that The value of N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; and four pixels correspond to each other under the microlens. The calibrating the difference in sensitivity of NxN pixels at the focal plane position corresponding to the NxN pixel phase focus sensor to calibrate the difference in sensitivity of the NxN pixels at the focal plane position and reduce the difference in sensitivity of the NxN pixels at the focal plane position includes: The difference in sensitivity of the four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor is calibrated to calibrate the difference in sensitivity of the four pixels at the focal plane position and reduce the difference in sensitivity of the four pixels at the focal plane position.

3. The method according to claim 2, characterized in that The calibrating the difference in sensitivity of four pixels at the focal plane position corresponding to the 2x2 pixel phase focus sensor to calibrate the difference in sensitivity of the four pixels at the focal plane position and reduce the difference in sensitivity of the four pixels at the focal plane position includes: Using the camera to photograph a light panel with uniform illumination, and setting half of the sum of the focus position of the lens at infinity and the focus position of the lens at the minimum focus distance as the calibration focus position; Acquire a calibration image at the calibration focus position using the 2x2 pixel phase focus sensor; the calibration image comprises four color channels; wherein each sub-channel under each color channel shares the color filter of the corresponding color channel; Using a preset pixel scaling algorithm, the calibration image is subjected to pixel reduction processing to obtain a reduced calibration image; Merging the sub-channels under each color channel contained in the reduced calibration image, and calculating the difference between each merged channel and the pixel mean of the color channel; The compensation values ​​of each merged channel and its corresponding color channel pixel average are calculated by using the difference values ​​between the merged channels and their corresponding color channel pixel averages; and the corresponding channel pixels are compensated by using the compensation values ​​to calibrate the difference values ​​of the sensitivity of the four pixels at the focal plane position and reduce the difference in the sensitivity of the four pixels at the focal plane position.

4. The method according to claim 3, characterized in that The preset pixel scaling algorithm is a bilinear interpolation algorithm.

5. The method according to claim 3, characterized in that Merging the sub-channels under each color channel contained in the reduced calibration image and calculating the difference between each merged channel and the pixel mean of the color channel, including: Calculating the mean value of each color channel pixel contained in the reduced calibration image; Calculate the ratio of each merged channel pixel to the mean value of the corresponding color channel pixel, and use the difference between the ratio and the number 1 as the difference value of the corresponding channel.

6. The method according to claim 5, characterized in that The calculating of compensation values ​​of each merged channel and its corresponding color channel pixel mean by using the difference values ​​of each merged channel and its corresponding color channel pixel mean includes: Calculate the sum of the difference between each merged channel and the pixel mean of the corresponding color channel and the number 1, and calculate the ratio of the number 1 to the sum as the compensation value of the corresponding channel and the pixel mean of the corresponding color channel.

7. The method according to claim 1, characterized in that The value of K is 5; the preset K focus positions are 5 step positions corresponding to dividing the lens travel into 5 equal parts; The method of obtaining K calibration images at K preset focus positions using the calibrated NxN pixel phase focus sensor includes: Using the calibrated NxN pixel phase focus sensor, a calibration image is acquired at each of the five step positions corresponding to the lens travel being divided into five equal parts, resulting in five calibration images.

8. The method according to claim 1, characterized in that The method of calibrating different difference values ​​of NxN pixel sensitivities at the preset K focus positions by using the K calibration images and the focus response function, and fitting the focus parameters of the focus response function during the calibration process, includes: Calculating the average pixel value of each color channel contained in the K calibration images; and calculating the interpolation value of each color channel using the average pixel value; The interpolation and focus response function are used to calibrate different difference values ​​of NxN pixel sensitivities at the preset K focus positions, and during the calibration process, focus parameters of the focus response function are fitted.

9. The method according to claim 7, characterized in that The value of N is 2; the NxN pixel phase focus sensor is a 2x2 pixel phase focus sensor; the 2x2 pixel phase focus sensor includes a microlens; four pixels correspond to each other under the microlens; using the K calibration images and the focus response function to calibrate different difference values ​​of the NxN pixel sensitivity at the preset K focus positions, and during the calibration process, fitting the focus parameters of the focus response function, including: Calculating the average pixel value of the four color channels contained in the five calibration images; and calculating the interpolation of the four color channels using the average pixel value; The interpolation and focus response function are used to calibrate different difference values ​​of the four-pixel sensitivity at the five preset focus positions, and during the calibration process, the focus parameters of the focus response function are fitted.

10. The method according to claim 1, characterized in that The value of P is 2. The method of acquiring P target images at P focus positions using the calibrated NxN pixel phase focus sensor and calculating the difference in NxN pixel sensitivities corresponding to the P target images includes: Using the calibrated NxN pixel phase focus sensor, two target images are acquired at two focus positions, and the difference in NxN pixel sensitivities corresponding to the two target images is calculated.

11. The method according to claim 10, characterized in that The method of fitting a quasi-focus position by using a focus parameter of the focus response function and a difference in NxN pixel sensitivities corresponding to the P target images, and focusing according to a lens position of the camera device determined by the quasi-focus position, includes: The focus parameter of the focus response function and the difference in NxN pixel sensitivities corresponding to the two target images are used to fit the quasi-focus position through the least squares method, and the lens position of the camera device determined according to the quasi-focus position is used for focusing.

12. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor is configured to call and execute the computer program to implement the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which implements the method according to any one of claims 1 to 11 when executed by an electronic device.

Images