Phase focusing method, electronic device, storage medium, and program product

Abstract

Embodiments of the present application provide a phase focusing method, an electronic device, a storage medium and a program product, which are applied to the technical field of electronics. The method comprises the following steps: acquiring a first phase image collected by phase detection pixels and a first coordinate of a position of an optical anti-shake module at a target time, under the condition that the optical anti-shake module is started; determining a phase gain compensation matrix according to the first coordinate and a plurality of phase gain calibration matrices, the relative positions of the optical center of the lens and the optical center of the image sensor being different during calibration; compensating the first phase image by using the phase gain compensation matrix; and performing phase focusing based on a second phase image obtained after compensation. Therefore, the embodiments of the present application can make the brightness values of the pixels in the second phase image obtained after compensation more uniform, so as to make the phase difference information determined based on the second phase image more accurate, thereby improving the focusing stability and the focusing accuracy in the optical anti-shake mode.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to a phase focusing method, electronic device, storage medium, and program product. Background Technology

[0002] With the continuous development of electronic technology, mobile phones, tablets, and other electronic devices have become common tools in people's daily lives and work. Currently, some electronic devices are equipped with cameras, providing users with photo or video recording functions.

[0003] In some electronic devices, the camera is equipped with features that support phase detection autofocus (PDAF) and optical image stabilization (OIS).

[0004] However, when the camera activates its optical image stabilization module, there are issues with focusing stability and accuracy. Summary of the Invention

[0005] This application provides a phase-detection autofocus method, electronic device, storage medium, and program product, which can improve focusing stability and accuracy when the optical image stabilization module is activated for optical image stabilization.

[0006] In a first aspect, embodiments of this application propose a phase-detection autofocus method applied to an electronic device. The electronic device includes a camera, which comprises an optical image stabilization module, a lens, and an image sensor. The image sensor includes multiple phase detection pixels. The phase-detection autofocus method includes: when the optical image stabilization module is activated, the electronic device acquires a first phase image captured by the phase detection pixels; the electronic device acquires a first coordinate of the position of the optical image stabilization module at a target time, where the target time is the time at which the focus point in the original image acquired by the image sensor is located during acquisition, and the original image corresponds to the first phase image; the electronic device determines a phase gain compensation matrix based on the first coordinate and multiple phase gain calibration matrices, wherein the relative positions of the optical center of the lens and the optical center of the image sensor are different during calibration; the electronic device compensates the first phase image using the phase gain compensation matrix to obtain a second phase image; and the electronic device performs phase-detection autofocus based on the second phase image.

[0007] In this way, when the camera activates its optical image stabilization (OIS) module, a phase gain compensation matrix is ​​calculated based on the first coordinate of the OIS module's position at the target time and multiple phase gain calibration matrices. This matrix is ​​then used to compensate for the brightness of the first phase image captured by the phase detection pixels, resulting in more uniform pixel brightness values ​​in the second phase image obtained after brightness compensation. Therefore, when using the second phase image to determine phase difference information, the determined phase difference information is more accurate, allowing for a more precise calculation of the defocus distance corresponding to the phase difference information. This improves focusing stability and accuracy in OIS mode.

[0008] In one possible implementation, before the electronic device acquires the first coordinates of the optical image stabilization module's position at the target time, the method further includes: the electronic device acquiring the second coordinates of the focus point in the original image; the electronic device determining the target duration based on the second coordinates; and the electronic device determining the target time as the sum of the initial time when it begins acquiring the original image and the target duration. In this way, by using the second coordinates of the focus point in the original image to estimate the target time at the time of acquisition, the estimated target time can be made more accurate.

[0009] In one possible implementation, the electronic device determines the target duration based on the second coordinate, including: the electronic device calculates the target duration using the following formula:

[0010]

[0011] Among them, T lag Y represents the target duration, EIT represents the exposure duration of the original image, and Y represents the target duration. n Here, H represents the coordinate component along the height direction of the original image in the second coordinate system, where H is the height of the original image, and RowNum is the total number of rows in the photosensitive unit array of the image sensor. time This refers to the reading time required to read image data generated by a row of photosensitive cells in the photosensitive cell array. This allows for a more accurate prediction of the target time of the focus point in the original image during acquisition.

[0012] In one possible implementation, the movable directions of the optical image stabilization module include a first movable direction and a second movable direction, which are perpendicular to each other and both perpendicular to the optical axis of the camera. The first movable direction includes a first direction and a second direction that are opposite to each other, and the second movable direction includes a third direction and a fourth direction that are opposite to each other. Multiple phase gain calibration matrices include a first type of phase gain calibration matrix and / or a second type of phase gain calibration matrix. The first type of phase gain calibration matrix includes a first phase gain calibration matrix, a second phase gain calibration matrix, and a third phase gain calibration matrix. The second type of phase gain calibration matrix includes a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix. In this calibration process, the optical center of the lens and the optical center of the image sensor are aligned along the optical axis of the camera when the first and fourth phase gain calibration matrices are used. In the second phase gain calibration matrix, the optical center of the lens is offset by a first distance relative to the optical center of the image sensor along a first direction, where the first distance is the maximum distance the lens can move along that direction. In the third phase gain calibration matrix, the optical center of the lens is offset by a second distance relative to the optical center of the image sensor along a second direction, where the second distance is the maximum distance the lens can move along that direction. In the fifth phase gain calibration matrix, the optical center of the lens is offset by a third distance relative to the optical center of the image sensor along a third direction, where the third distance is the maximum distance the lens can move along that direction. In the sixth phase gain calibration matrix, the optical center of the lens is offset by a fourth distance relative to the optical center of the image sensor along a fourth direction, where the fourth distance is the maximum distance the lens can move along that direction. This allows for the calculation of the phase gain compensation matrix using multiple phase gain calibration matrices, enabling precise brightness compensation of the first phase image regardless of the optical image stabilization module's position.

[0013] In one possible implementation, the first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix; the second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix; the third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix; and the first coordinate includes a first coordinate component along a first movement direction. The electronic device determines a phase gain compensation matrix based on the first coordinate and multiple phase gain calibration matrices, including: the electronic device determining a first left phase gain compensation sub-matrix based on the first coordinate component, the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix, and the third left phase gain calibration sub-matrix; and the electronic device determining a first right phase gain compensation sub-matrix based on the first coordinate component, the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix, and the third right phase gain calibration sub-matrix. Thus, in scenarios involving left and right phase detection focusing, the phase gain compensation matrix can be accurately calculated to compensate for the brightness of the first phase image.

[0014] In one possible implementation, the electronic device determines the first left-phase gain compensation sub-matrix based on the first coordinate component, the first left-phase gain calibration sub-matrix, the second left-phase gain calibration sub-matrix, and the third left-phase gain calibration sub-matrix, including: the electronic device calculates the first left-phase gain compensation sub-matrix using the following formula:

[0015]

[0016] Among them, GainMap L Q is the first left-side phase gain compensation submatrix. 0L Q is the first left-side phase gain calibration submatrix. 1L Q is the second left-side phase gain calibration submatrix. 2L X is the third left phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This is the first coordinate component. Accordingly, the electronic device determines the first right-phase gain compensation sub-matrix based on the first coordinate component, the first right-phase gain calibration sub-matrix, the second right-phase gain calibration sub-matrix, and the third right-phase gain calibration sub-matrix, including: the electronic device calculates the first right-phase gain compensation sub-matrix using the following formula:

[0017]

[0018] Among them, GainMap R Q is the first right phase gain compensation submatrix. 0R Q is the first right phase gain calibration submatrix. 1RQ is the second right phase gain calibration submatrix. 2R X is the third right phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This is the first coordinate component. Thus, in scenarios involving left and right phase detection focusing, the phase gain compensation matrix can be accurately calculated to compensate for the brightness of the first phase image.

[0019] In one possible implementation, the fourth phase gain calibration matrix includes a first upper phase gain calibration sub-matrix and a first lower phase gain calibration sub-matrix; the fifth phase gain calibration matrix includes a second upper phase gain calibration sub-matrix and a second lower phase gain calibration sub-matrix; the sixth phase gain calibration matrix includes a third upper phase gain calibration sub-matrix and a third lower phase gain calibration sub-matrix; and the first coordinate includes a second coordinate component along the second movement direction. The electronic device determines the phase gain compensation matrix based on the first coordinate and multiple phase gain calibration matrices, including: the electronic device determining the first upper phase gain compensation sub-matrix based on the second coordinate component, the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix; and the electronic device determining the first lower phase gain compensation sub-matrix based on the second coordinate component, the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix. Thus, in scenarios involving upper and lower phase detection focusing, the phase gain compensation matrix can be accurately calculated to compensate for the brightness of the first phase image.

[0020] In one possible implementation, the electronic device determines the first upper phase gain compensation sub-matrix based on the second coordinate component, the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix, including: the electronic device calculates the first upper phase gain compensation sub-matrix using the following formula:

[0021]

[0022] Among them, GainMap T Q is the first upper phase gain compensation submatrix. 0T Q is the first upper phase gain calibration submatrix. 3T Q is the second upper phase gain calibration submatrix. 4T Y is the third upper phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS YThis is the second coordinate component. Accordingly, the electronic device determines the first lower phase gain compensation sub-matrix based on the second coordinate component, the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix. This includes: the electronic device calculates the first lower phase gain compensation sub-matrix using the following formula:

[0023]

[0024] Among them, GainMap B Q is the first lower phase gain compensation submatrix. 0B Q is the first lower phase gain calibration submatrix. 3B The second lower phase gain calibration submatrix, Q 4B Y is the third lower phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS Y This is the second coordinate component. Thus, in scenarios involving upper and lower phase detection focusing, the phase gain compensation matrix can be accurately calculated to compensate for the brightness of the first phase image.

[0025] In one possible implementation, the phase gain compensation matrix includes a first phase gain compensation sub-matrix and a second phase gain compensation sub-matrix. The electronic device uses the phase gain compensation matrix to compensate for a first phase image to obtain a second phase image, including: the electronic device splitting the first phase image into a first phase sub-image and a second phase sub-image; the electronic device adjusting the size of the first phase gain compensation sub-matrix to obtain a third phase gain compensation sub-matrix, the size of which is equal to the size of the first phase sub-image; the electronic device using the third phase gain compensation sub-matrix to compensate for the first phase sub-image to obtain a third phase sub-image; the electronic device adjusting the size of the second phase gain compensation sub-matrix to obtain a fourth phase gain compensation sub-matrix, the size of which is equal to the size of the second phase sub-image; and the electronic device using the fourth phase gain compensation sub-matrix to compensate for the second phase sub-image to obtain a fourth phase sub-image. Wherein, the first phase sub-image is the first left phase sub-image, the second phase sub-image is the first right phase sub-image, the third phase sub-image is the second left phase sub-image, and the fourth phase sub-image is the second right phase sub-image; the first phase gain compensation sub-matrix is ​​the first left phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​the first right phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​the second left phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​the second right phase gain compensation sub-matrix. Alternatively, the first phase sub-image is the first upper phase sub-image, the second phase sub-image is the first lower phase sub-image, the third phase sub-image is the second upper phase sub-image, and the fourth phase sub-image is the second lower phase sub-image; the first phase gain compensation sub-matrix is ​​the first upper phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​the first lower phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​the second upper phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​the second lower phase gain compensation sub-matrix. Thus, in scenarios where left and right phase detection focusing is used, brightness compensation can be performed on the first left phase sub-image and the first right phase sub-image respectively; in scenarios where top and bottom phase detection focusing is used, brightness compensation can be performed on the first top phase sub-image and the first bottom phase sub-image respectively.

[0026] In one possible implementation, the electronic device compensates the first phase sub-image using a third phase gain compensation sub-matrix to obtain a third phase sub-image. This involves multiplying the brightness value of each pixel in the first phase sub-image by a compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix. Correspondingly, the electronic device compensates the second phase sub-image using a fourth phase gain compensation sub-matrix to obtain a fourth phase sub-image. This involves multiplying the brightness value of each pixel in the second phase sub-image by a compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix. In this way, brightness compensation of the first phase image can be achieved by multiplying the compensation coefficient by the brightness value, resulting in a relatively simple compensation method.

[0027] In one possible implementation, the electronic device compensates the first phase sub-image using a third phase gain compensation sub-matrix to obtain a third phase sub-image. This involves multiplying the brightness value of each pixel within the focus area of ​​the first phase sub-image by a compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix. Correspondingly, the electronic device compensates the second phase sub-image using a fourth phase gain compensation sub-matrix to obtain a fourth phase sub-image. This involves multiplying the brightness value of each pixel within the focus area of ​​the second phase sub-image by a compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix. By performing brightness compensation only on the brightness values ​​of each pixel within the focus area of ​​both the first and second phase sub-images, the computational load when performing brightness compensation on the first phase image can be reduced.

[0028] In one possible implementation, before the electronic device determines the phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, the method further includes: the electronic device acquiring multiple pre-calibrated phase gain calibration matrices, each phase gain calibration matrix including a first phase gain calibration sub-matrix and a second phase gain calibration sub-matrix. Each compensation coefficient in the first phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each first image block in the first phase test sub-image; each compensation coefficient in the second phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each second image block in the second phase test sub-image; the maximum brightness value is the maximum value between the brightness values ​​of each first image block and each second image block; the first and second phase test sub-images are obtained by splitting the phase test image; the phase test image is a phase image acquired when the lens's focus position is a preset focus position and the relative position between the optical center of the lens and the optical center of the image sensor is a preset position. In this way, multiple phase gain calibration matrices can be pre-calibrated to quickly calculate the phase gain compensation matrix, thereby increasing the speed of brightness compensation for the first phase image and thus improving the speed of phase focusing.

[0029] In one possible implementation, each compensation coefficient in the first phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each first image block in the first phase test sub-image; and each compensation coefficient in the second phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each second image block in the second phase test sub-image. This simplifies the calculation of the phase gain calibration matrix.

[0030] Secondly, embodiments of this application propose an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call the computer program to execute the phase focusing method described above.

[0031] Thirdly, embodiments of this application propose a computer-readable storage medium storing a computer program or instructions, which, when executed, implements the aforementioned phase focusing method.

[0032] Fourthly, embodiments of this application propose a computer program product, including a computer program that, when run, causes the computer to execute the aforementioned phase focusing method.

[0033] The possible implementations of aspects two through four have similar effects to those of aspect one and the possible designs of aspect one, and will not be elaborated upon here. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the camera structure provided in an embodiment of this application;

[0035] Figure 2 This is a schematic diagram of the structure of the first image sensor in the camera provided in the embodiments of this application;

[0036] Figure 3 This is a schematic diagram of the structure of the second image sensor in the camera provided in the embodiments of this application;

[0037] Figure 4 This is a schematic diagram of the structure of a third image sensor in a camera provided in an embodiment of this application;

[0038] Figure 5 This is a schematic diagram illustrating how, when the optical image stabilization module in the camera is activated and the optical displacement between the optical center of the lens and the optical center of the image sensor is zero, the brightness compensation of the first phase image is performed using the first phase gain calibration matrix.

[0039] Figure 6 This is a schematic diagram illustrating how, when the optical image stabilization module in the camera is activated and the optical displacement between the optical center of the lens and the optical center of the image sensor is not equal to zero, the first phase gain calibration matrix is ​​used to perform brightness compensation on the first phase image.

[0040] Figure 7 A schematic diagram of the hardware system structure of the electronic device provided in the embodiments of this application;

[0041] Figure 8 A schematic diagram of the software system structure of the electronic device provided in the embodiments of this application;

[0042] Figure 9 A schematic diagram of an application scenario provided in this application embodiment;

[0043] Figure 10 A flowchart of a phase focusing method provided in an embodiment of this application;

[0044] Figure 11 A schematic diagram showing the displacement of the optical image stabilization module during the acquisition of the original image provided in this embodiment of the application;

[0045] Figure 12 A schematic diagram showing the offset of the optical center of the lens relative to the optical center of the image sensor during the calibration process of the multiple phase gain calibration matrices provided in the embodiments of this application;

[0046] Figure 13 A flowchart illustrating the calibration process of the first phase gain calibration matrix provided in this application embodiment;

[0047] Figure 14 This is a schematic diagram illustrating how, in an embodiment of this application, when the optical image stabilization module in the camera is activated and the optical displacement between the optical center of the lens and the optical center of the image sensor is not equal to 0, a phase gain compensation matrix is ​​used to compensate for the brightness of the first phase image.

[0048] Figure 15 A flowchart illustrating the brightness compensation of a first phase image provided in an embodiment of this application;

[0049] Figure 16 This is a schematic diagram of the structure of a phase focusing device provided in an embodiment of this application;

[0050] Figure 17 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0051] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" do not necessarily imply that they are different.

[0052] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0053] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0054] Currently, some electronic devices are equipped with cameras, which users can use to take photos or videos while using these devices.

[0055] When a user holds an electronic device to take a picture, the device may shake to some extent, causing the image captured by the camera to be blurry. To enhance the stability of the images captured by the camera, optical image stabilization technology has been introduced into the camera, that is, an optical image stabilization module is set in the camera.

[0056] Furthermore, to improve the clarity of images captured by the camera, electronic devices need to perform automatic focus (AF). AF methods can include phase detection autofocus (PD), which involves using phase-detection pixels in the camera to capture a phase image to obtain phase difference (PD) information. Based on this phase difference, the defocus distance is calculated, and the focusing motor moves the lens according to this distance to adjust the distance between the lens and the image sensor, thus achieving phase focus. The defocus distance refers to the distance between the lens's current position and its position when in focus; it can also be understood as the distance from the focal point to the image plane.

[0057] like Figure 1 As shown, the camera may include a lens 10, an image sensor 20, a lens drive assembly, a bracket 50, and a circuit board 60.

[0058] The lens 10 and the image sensor 20 are arranged sequentially along the optical axis of the camera. The lens 10 may include one or more optical lenses, which can be convex or concave lenses; when the lens 10 includes multiple optical lenses, these lenses can be stacked sequentially along the optical axis of the camera. The image sensor 20, also known as a camera sensor, is fixed to and electrically connected to the circuit board 60.

[0059] The lens drive assembly is used to move the lens 10 to achieve autofocus and / or optical image stabilization. The lens drive assembly may include an optical image stabilization module and a focus drive module.

[0060] The optical image stabilization module may include an optical image stabilization motor 30, which drives the lens 10 to move to achieve optical image stabilization. Optical image stabilization technology corrects "optical axis misalignment" through a floating lens. Its principle is to detect minute vibrations using a gyroscope or accelerometer in the electronic device, and then send the detected vibration data to a microprocessor, such as a driver chip electrically connected to the optical image stabilization motor 30. The microprocessor calculates the amount of displacement to be compensated based on the vibration data, and then drives the optical image stabilization motor 30 to move, thereby moving the lens 10 along a first and / or second movement direction to compensate for the vibration direction and displacement of the lens 10, effectively improving the image blur caused by electronic device vibration. The first and second movement directions are perpendicular to each other, and both are perpendicular to the optical axis of the camera.

[0061] Therefore, the optical image stabilization motor 30 can drive the lens 10 to move along a direction perpendicular to the optical axis of the camera (i.e., along the first movement direction and / or the second movement direction) to achieve optical image stabilization. It should be understood that... Figure 1 The location of the optical image stabilization motor 30 is only schematically shown in this embodiment. The specific location of the optical image stabilization motor 30 is not limited in this application embodiment.

[0062] The optical image stabilization motor 30 can be a voice coil motor (VCM), a shape memory alloy (SMA) motor, a stepping motor, a piezoelectric motor, etc. It should be understood that the specific structure of the optical image stabilization motor 30 can be designed and selected according to the chosen driving method, and this application embodiment does not limit this.

[0063] The focus drive module may include a focus motor 40, which can be mounted on the bracket 50. The focus motor 40 is used to move the lens 10 to achieve autofocus. Specifically, the focus motor 40 can move the lens 10 along the optical axis of the camera to achieve autofocus. It should be understood that... Figure 1 The location of the focusing motor 40 is only schematically shown in the illustration, and the specific location of the focusing motor 40 is not limited in the embodiments of this application.

[0064] The focusing motor 40 can be a voice coil motor, a shape memory alloy motor, a stepper motor, a piezoelectric motor, etc. It should be understood that the specific structure of the focusing motor 40 can be designed and selected according to the chosen driving method, and this application embodiment does not limit this.

[0065] In some embodiments, such as Figure 1 As shown, the optical image stabilization motor 30 and the focusing motor 40 can be two independent components, which respectively drive the lens 10 for optical image stabilization and autofocus. In other embodiments, the optical image stabilization motor 30 and the focusing motor 40 can also be the same component, which can drive the lens 10 for both optical image stabilization and autofocus.

[0066] In this embodiment, the image sensor 20 includes a microlens array, a filter unit array, and a photosensitive unit array. The microlens array and the filter unit array are both located between the lens 10 and the photosensitive unit array, and the filter unit array is located between the microlens array and the photosensitive unit array.

[0067] like Figures 2 to 4 As shown, the microlens array includes multiple microlenses (MLs) 21 arranged in an array. Each microlens 21 has a light-focusing function and can be an on-chip microlens (OCL). The filter unit array includes multiple filter units 22, such as a red (R) filter unit corresponding to a red pixel, a green (G) filter unit corresponding to a green pixel, and a blue (B) filter unit corresponding to a blue pixel. The photosensitive unit array includes multiple photosensitive units 23.

[0068] The light reflected from the subject passes through the lens 10 and then enters the microlens 21. The microlens 21 focuses the incident light, so that the focused light passes through the light filtering unit 22 and is projected onto the photosensitive unit 23. The photosensitive unit 23 converts the light signal into an electrical signal for imaging.

[0069] In this way, the microlens array, the filter unit array, and the photosensitive unit array can together constitute the pixel array of the image sensor.

[0070] In some image sensors, such as Figure 2 As shown in (a) and (b), the pixel array of the image sensor may include multiple pixels 24 distributed in an array, such as red pixels, green pixels and blue pixels, etc. Each pixel 24 may include a microlens 21, a filter unit 22 covered by the microlens 21 and a photosensitive unit 23 covered by the filter unit 22, and each photosensitive unit 23 may include a photosensitive element.

[0071] Some pixels in the image sensor can be designated as phase detection pixels for phase detection. These detection pixels can be reserved shield pixels (SPs), which are pixels whose areas of the photosensitive unit 23 are partially obscured. For example, some green pixels in the image sensor can be designated as phase detection pixels.

[0072] Specifically, a light-shielding material layer 25 can be provided between the photosensitive unit 23 and the filter unit 22. The light-shielding material layer 25 can block part of the photosensitive unit 23. The light-shielding material layer 25 can block the light entering the photosensitive unit 23 below it, so that the light can only enter the photosensitive unit 23 that is not blocked by the light-shielding material layer 25.

[0073] like Figure 2 As shown in (a) and (b), the phase detection pixel may include a left phase detection pixel and a right phase detection pixel. For example, the left phase detection pixel may be a masked pixel on the right side that is partially blocked by the light-shielding material layer 25, and the right phase detection pixel may be a masked pixel on the left side that is partially blocked by the light-shielding material layer 25.

[0074] In this way, of the imaging beam incident on the right phase detection pixel, only the right-hand beam can be imaged on the photosensitive portion of the right phase detection pixel (i.e., the portion not blocked by the light-blocking material layer 25), forming a right phase sub-image; similarly, of the imaging beam incident on the left phase detection pixel, only the left-hand beam can be imaged on the photosensitive portion of the left phase detection pixel (i.e., the portion not blocked by the light-blocking material layer 25), forming a left phase sub-image. Thus, by comparing the left and right phase sub-images, phase difference information can be obtained.

[0075] It should be noted that, Figure 2 The image sensor shown in (a) is... Figure 2 The image sensor shown in (b) is a cross-sectional view obtained along section A-A'.

[0076] In other image sensors, such as Figure 3 As shown in (a) and (b), the pixel array of the image sensor may include multiple pixels 24 arranged in an array, such as red pixels, green pixels, and blue pixels. Each pixel 24 may include a microlens 21, a filter unit 22 covered by the microlens 21, and a photosensitive unit 23 covered by the filter unit 22. Each photosensitive unit 23 may include two arranged side by side (e.g., Figure 3 The photosensitive elements (as shown, arranged horizontally side by side), such as photosensitive unit 23, may include a first photosensitive element 231 and a second photosensitive element 232. Figure 3 The image sensors shown in (a) and (b) can also be called dual-pixel autofocus image sensors (Dual PD image sensors).

[0077] Each pixel 24 in the image sensor includes a left phase detection pixel and a right phase detection pixel, meaning that all pixels in the image sensor can be used for phase detection. Any pixel 24 in the pixel array of the image sensor can be divided into a left phase detection pixel and a right phase detection pixel. The left phase detection pixel includes a first photosensitive element 231, a corresponding filter unit 22, and a microlens 21. The right phase detection pixel includes a second photosensitive element 232, a corresponding filter unit 22, and a microlens 21. The first and second photosensitive elements 231 share the same color filter unit 22 and the same microlens 21.

[0078] In this way, when light is incident on pixel 24, it passes sequentially through microlens 21 and filter unit 22 and is focused onto first photosensitive element 231 and second photosensitive element 232, respectively. First photosensitive element 231 performs photoelectric conversion on the incident light to obtain a left phase sub-image, and second photosensitive element 232 performs photoelectric conversion on the incident light to obtain a right phase sub-image. Thus, by comparing the left and right phase sub-images, phase difference information can be obtained.

[0079] It should be noted that, Figure 3 The image sensor shown in (a) is... Figure 3 The image sensor shown in (b) is a cross-sectional view obtained along section C-C'.

[0080] In some image sensors, such as Figure 4 As shown in (a) and (b), the pixel array of the image sensor may include multiple pixel units 240 arranged in an array, such as red pixel units, green pixel units, and blue pixel units. Each pixel unit 240 includes four adjacent pixels of the same color. For example, one pixel unit 240 includes a first red pixel (R0 pixel), a second red pixel (R1 pixel), a third red pixel (R2 pixel), and a fourth red pixel (R3 pixel).

[0081] Each pixel unit 240 in the image sensor includes four pixels of the same color, which share the same color filter unit 22 and the same microlens 21. That is, each pixel unit 240 may include a microlens 21, a filter unit 22 covered by the microlens 21, and four photosensitive units 23 covered by the filter unit 22. Each photosensitive unit 23 includes a photosensitive element. Figure 4 The image sensor shown in (a) and (b) can also be called a full-pixel phase-detection image sensor. The full-pixel phase-detection image sensor relies on the separation effect of the microlens 21 on the focusing phase to achieve phase detection and phase focusing.

[0082] Each pixel unit 240 in the image sensor may include a left phase detection pixel and a right phase detection pixel, meaning that all pixels in the image sensor can be used for phase detection. For any pixel unit 240 in the pixel array of the image sensor, the pixels can be divided into left phase detection pixels and right phase detection pixels.

[0083] For example, for one of the pixel units 240, the first red pixel (R0 pixel) can be used as the left phase detection pixel, and the second red pixel (R1 pixel) can be used as the right phase detection pixel; and the third red pixel (R2 pixel) can be used as the left phase detection pixel, and the fourth red pixel (R3 pixel) can be used as the right phase detection pixel.

[0084] When light is incident on pixel unit 240, it passes sequentially through microlens 21 and filter unit 22, and is focused onto photosensitive units 23 corresponding to the first red pixel (R0 pixel), the second red pixel (R1 pixel), the third red pixel (R2 pixel), and the fourth red pixel (R3 pixel), respectively. The photosensitive units 23 corresponding to the first red pixel (R0 pixel) and the third red pixel (R2 pixel) perform photoelectric conversion on the incident light to obtain a left phase sub-image; the photosensitive units 23 corresponding to the second red pixel (R1 pixel) and the fourth red pixel (R3 pixel) perform photoelectric conversion on the incident light to obtain a right phase sub-image. Thus, by comparing the left and right phase sub-images, phase difference information can be obtained.

[0085] It should be noted that, Figure 4 The image sensor shown in (a) is... Figure 4 The image sensor shown in (b) is a cross-sectional view obtained along section D-D'.

[0086] It is understood that the pixel array of the full-pixel phase-detection autofocus image sensor in this application embodiment, in addition to employing Figure 4 In addition to the four-Bayer array shown, a nine-Bayer array or a sixteen-Bayer array can also be used. The specific form of the pixel array of the image sensor is not limited in the embodiments of this application.

[0087] When the pixel array of the full-pixel phase-detection autofocus image sensor is a nine-Bayer array, each pixel unit 240 in the pixel array includes nine adjacent pixels of the same color, and these nine pixels of the same color share the same color filter unit 22 and the same microlens 21. When the pixel array of the full-pixel phase-detection autofocus image sensor is a sixteen-Bayer array, each pixel unit 240 in the pixel array includes sixteen adjacent pixels of the same color, and these sixteen pixels of the same color share the same color filter unit 22 and the same microlens 21.

[0088] Understandable, Figures 2 to 4 The image sensors shown are all designed for scenarios requiring left-right phase detection autofocus. In practical applications, phase detection autofocus methods can include, but are not limited to, left-right phase detection autofocus, top-bottom phase detection autofocus, and four-phase detection autofocus.

[0089] Left and right phase detection autofocus refers to setting left and right phase detection pixels in the image sensor, using the left phase detection pixel to acquire a left phase sub-image, and using the right phase detection pixel to acquire a right phase sub-image, calculating phase difference information based on the left and right phase sub-images, and then calculating the defocus distance based on the phase difference information. The focusing motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, thereby achieving phase focusing.

[0090] Upper and lower phase detection autofocus refers to setting upper and lower phase detection pixels in the image sensor. Specifically, for... Figure 2 The image sensor shown can use pixels partially blocked by the light-blocking material layer 25 on the lower side as upper phase detection pixels, and pixels partially blocked by the light-blocking material layer 25 on the upper side as lower phase detection pixels. For... Figure 3 The image sensor shown can arrange the first photosensitive element 231 and the second photosensitive element 232 side by side along the vertical direction, so that each pixel 24 includes an upper phase detection pixel and a lower phase detection pixel. For Figure 4 The image sensor shown includes an upper phase detection pixel and a lower phase detection pixel in each pixel unit 240. For example, for one pixel unit 240, a first red pixel (R0 pixel) can be used as an upper phase detection pixel, and a third red pixel (R2 pixel) can be used as a lower phase detection pixel; and a second red pixel (R1 pixel) can be used as an upper phase detection pixel, and a fourth red pixel (R3 pixel) can be used as a lower phase detection pixel.

[0091] Furthermore, an upper phase sub-image is acquired using the upper phase detection pixel, and a lower phase sub-image is acquired using the lower phase detection pixel. Phase difference information is calculated based on the upper and lower phase sub-images, and then the defocus distance is calculated based on the phase difference information. The focusing motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, thereby achieving phase focusing.

[0092] Four-phase detection autofocus refers to setting up left, right, top, and bottom phase detection pixels in the image sensor. Phase difference information is calculated using the phase sub-images acquired by these four phase detection pixels. Then, the defocus distance is calculated based on the phase difference information. The focusing motor moves the lens based on the defocus distance to adjust the distance between the lens and the image sensor, thus achieving phase focusing.

[0093] The aforementioned camera includes an optical image stabilization module, a lens 10, and an image sensor 20, which comprises multiple phase detection pixels. These phase detection pixels may include a left phase detection pixel and a right phase detection pixel; or, they may include an upper phase detection pixel and a lower phase detection pixel; or, they may include a left phase detection pixel, a right phase detection pixel, an upper phase detection pixel, and a lower phase detection pixel.

[0094] However, when the camera activates its optical image stabilization (OIS) module, the OIS module moves, causing the lens 10 to move as well. This results in a misalignment between the optical center of the lens 10 and the optical center of the image sensor 20 (i.e., the optical displacement between them is not zero). Consequently, after the light passing through the lens 10 is focused by the microlens 21, the phase detection pixels in the image sensor 20 receive uneven light, leading to uneven brightness values ​​in the first phase image captured by the phase detection pixels. For example, the left and right phase detection pixels may receive uneven light, or the upper and lower phase detection pixels may receive uneven light.

[0095] The brightness differences of pixels in the first phase image captured by the phase detection pixels will affect the calculation of phase difference information. If the brightness values ​​of pixels in the first phase image captured by the phase detection pixels are uneven, the phase difference information determined using the first phase image will be inaccurate, resulting in an inaccurate defocus distance determined based on the phase difference information. Consequently, the focusing stability and accuracy in optical image stabilization mode will be low.

[0096] In related technologies, such as Figure 5As shown in (a), when the camera activates the optical image stabilization module for optical image stabilization and the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0, the first phase image can be acquired through the phase detection pixels in the image sensor 20.

[0097] When the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is equal to 0, the acquired first phase image is as follows: Figure 5 As shown in (b), it includes a first left-phase sub-image acquired by the left-phase detection pixels and a first right-phase sub-image acquired by the right-phase detection pixels. Figure 5 In the first phase image shown in (b), the horizontal axis represents multiple image blocks in the first left phase sub-image and the first right phase sub-image. For example, the first left phase sub-image is divided into 17 image blocks along the horizontal axis, and the first right phase sub-image is also divided into 17 image blocks along the horizontal axis. The vertical axis represents the brightness value of each image block in the first left phase sub-image and the first right phase sub-image. This brightness value is the average of the brightness values ​​of all pixels in each image block.

[0098] Related technologies can be adopted such as Figure 5 The first phase gain calibration matrix shown in (c) is for... Figure 5 The first phase image shown in (b) is subjected to brightness compensation, and the brightness-compensated first phase image is as follows: Figure 5 As shown in (d) in the figure. During calibration, the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is equal to 0.

[0099] exist Figure 5 In the first phase gain calibration matrix shown in (c), the first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix. The horizontal axis represents multiple image blocks, and the vertical axis represents the compensation coefficient corresponding to each image block in the first left phase gain calibration sub-matrix and the first right phase gain calibration sub-matrix.

[0100] exist Figure 5 In the compensated first phase image shown in (d), the compensated first phase image includes a compensated first left phase sub-image and a compensated first right phase sub-image. The horizontal axis represents multiple image blocks in the compensated first left and first right phase sub-images, and the vertical axis represents the brightness value of each image block in the compensated first left and first right phase sub-images.

[0101] Specifically, it can be adopted Figure 5 The first left-side phase gain calibration submatrix shown in (c) is for... Figure 5 The first left-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 5 The compensated first left-phase sub-image is shown in (d) in the figure; using Figure 5 The first right phase gain calibration submatrix shown in (c) is for... Figure 5 The first right-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 5 The compensated first right-phase sub-image is shown in (d) in the figure.

[0102] It can be seen that when the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is zero, the brightness values ​​of the pixels in the first phase image acquired by the phase detection pixels are uneven. However, after using the first phase gain calibration matrix to compensate for the brightness of the first phase image, the brightness values ​​of the pixels in the compensated first phase image become more uniform. Thus, when using the compensated first phase image to determine phase difference information, the determined phase difference information is more accurate. The determined phase difference information has a linear relationship with the defocus distance. Based on this linear relationship, the autofocus system can calculate the defocus distance corresponding to the phase difference information to perform the judgment of the focus position and the focusing action.

[0103] It should be noted that due to the manufacturing process of the image sensor and the fact that the microlenses 21 set at various positions are not completely consistent, when the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0, the phase detection pixels in different positions of the image sensor 20 receive uneven light, which in turn causes uneven brightness values ​​of the pixels in the first phase image acquired by the phase detection pixels.

[0104] However, the relevant technology uses a first phase gain calibration matrix to compensate for the brightness of the first phase image. This method can only compensate for the uneven brightness values ​​of pixels in the first phase image caused by the uneven light received by phase detection pixels in different directions when the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0.

[0105] like Figure 6 As shown in (a), when the camera activates the optical image stabilization module for optical image stabilization, and the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 can be d, where d is not equal to 0, the first phase image can be acquired through the phase detection pixels in the image sensor 20.

[0106] When the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is not equal to zero, the acquired first phase image is as follows: Figure 6As shown in (b), it includes a first left-phase sub-image acquired by the left-phase detection pixels and a first right-phase sub-image acquired by the right-phase detection pixels. Figure 6 In the first phase image shown in (b), the horizontal axis represents multiple image blocks in the first left phase sub-image and the first right phase sub-image, and the vertical axis represents the brightness value of each image block in the first left phase sub-image and the first right phase sub-image. The brightness value is the average of the brightness values ​​of all pixels in each image block.

[0107] If we still adopt the following... Figure 6 The first phase gain calibration matrix shown in (c) is for... Figure 6 The first phase image shown in (b) is subjected to brightness compensation, and the brightness-compensated first phase image is as follows: Figure 6 As shown in (d) in the figure.

[0108] exist Figure 6 In the first phase gain calibration matrix shown in (c), the first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix. The horizontal axis represents multiple image blocks, and the vertical axis represents the compensation coefficient corresponding to each image block in the first left phase gain calibration sub-matrix and the first right phase gain calibration sub-matrix.

[0109] exist Figure 6 In the compensated first phase image shown in (d), the compensated first phase image includes a compensated first left phase sub-image and a compensated first right phase sub-image. The horizontal axis represents multiple image blocks in the compensated first left and first right phase sub-images, and the vertical axis represents the brightness value of each image block in the compensated first left and first right phase sub-images.

[0110] Specifically, it can be adopted Figure 6 The first left-side phase gain calibration submatrix shown in (c) is for... Figure 6 The first left-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 6 The compensated first left-phase sub-image is shown in (d) in the figure; using Figure 6 The first right phase gain calibration submatrix shown in (c) is for... Figure 6 The first right-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 6 The compensated first right-phase sub-image is shown in (d) in the figure.

[0111] It can be seen that when the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is not zero, the brightness values ​​of the pixels in the first phase image acquired by the phase detection pixels are uneven. If the first phase gain calibration matrix is ​​still used to compensate for the brightness of the first phase image, the first phase gain calibration matrix becomes ineffective, and the brightness values ​​of the pixels in the compensated first phase image are still uneven.

[0112] Thus, when the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is not equal to 0, if the first phase image after compensation by the first phase gain calibration matrix is ​​used to determine the phase difference information, the determined phase difference information will have a deviation, resulting in a non-linear relationship between the determined phase difference information and the defocus distance. If a linear relationship is still used to calculate the defocus distance corresponding to the phase difference information, the determined defocus distance will be inaccurate, which will lead to misjudgment of the focusing system under image stabilization conditions, that is, the focusing stability and focusing accuracy in optical image stabilization mode will be low.

[0113] Based on this, this application provides a phase focusing method. When the optical image stabilization module is activated, a first phase image is acquired by the phase detection pixels, and the first coordinate of the position of the optical image stabilization module at the target time is obtained. The target time is the time when the focus point in the original image acquired by the image sensor is located at the time of acquisition. A phase gain compensation matrix is ​​determined based on the first coordinate and multiple phase gain calibration matrices. The relative positions of the optical center of the lens and the optical center of the image sensor are different when the multiple phase gain calibration matrices are calibrated. The phase gain compensation matrix is ​​used to compensate the first phase image to obtain a second phase image, and phase focusing is performed based on the second phase image.

[0114] Therefore, in this embodiment, when the camera activates the optical image stabilization module for optical image stabilization, a phase gain compensation matrix is ​​calculated based on the first coordinate of the optical image stabilization module's position at the target time and multiple phase gain calibration matrices. This matrix is ​​then used to compensate the brightness of the first phase image captured by the phase detection pixels, resulting in more uniform pixel brightness values ​​in the second phase image obtained after brightness compensation. This makes the determined phase difference information more accurate when using the second phase image to determine phase difference information. The determined phase difference information has a linear relationship with the defocus distance, and based on this linear relationship, the defocus distance corresponding to the phase difference information can be accurately calculated, thereby improving focusing stability and accuracy in optical image stabilization mode.

[0115] The phase focusing method provided in this application can be applied to electronic devices equipped with cameras. Electronic devices include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technology or device form used in the electronic devices.

[0116] To better understand the embodiments of this application, the structure of the electronic device of the embodiments of this application is described below.

[0117] Figure 7 A schematic diagram of the hardware system structure of electronic device 700 is shown. Electronic device 700 may include a processor 710, an external memory interface 720, an internal memory 721, a universal serial bus (USB) interface 730, a charging management module 740, a power management module 741, a battery 742, antenna 1, antenna 2, a mobile communication module 750, a wireless communication module 760, an audio module 770, a speaker 770A, a receiver 770B, a microphone 770C, a headphone jack 770D, a sensor module 780, buttons 790, a motor 791, an indicator 792, a camera 793, a display screen 794, and a subscriber identification module (SIM) card interface 795, etc. The sensor module 780 may include a gyroscope sensor 780A and an accelerometer sensor 780B.

[0118] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 700. In other embodiments of this application, the electronic device 700 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0119] The processor 710 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0120] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0121] The processor 710 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 710 is a cache memory. This memory can store instructions or data that the processor 710 has just used or that are used repeatedly. If the processor 710 needs to use the instruction or data again, it can retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 710, and thus improves the efficiency of the system.

[0122] The charging management module 740 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 740 receives charging input from the wired charger via a USB interface 730. In some wireless charging embodiments, the charging management module 740 receives wireless charging input via the wireless charging coil of the electronic device 700. While charging the battery 742, the charging management module 740 can also supply power to the electronic device via the power management module 741.

[0123] The power management module 741 connects the battery 742, the charging management module 740, and the processor 710. The power management module 741 receives input from the battery 742 and / or the charging management module 740, providing power to the processor 710, internal memory 721, display screen 794, camera 793, and wireless communication module 760, etc. The power management module 741 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 741 may also be located within the processor 710. In other embodiments, the power management module 741 and the charging management module 740 may be located in the same device.

[0124] The wireless communication function of electronic device 700 can be implemented through antenna 1, antenna 2, mobile communication module 750, wireless communication module 760, modem processor and baseband processor, etc.

[0125] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 750 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on electronic devices 700. The mobile communication module 750 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc.

[0126] The wireless communication module 760 can provide solutions for wireless communication applications on the electronic device 700, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 760 can be one or more devices integrating at least one communication processing module. The wireless communication module 760 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 710. The wireless communication module 760 can also receive signals to be transmitted from processor 710, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0127] In some embodiments, antenna 1 of electronic device 700 is coupled to mobile communication module 750, and antenna 2 is coupled to wireless communication module 760, enabling electronic device 700 to communicate with networks and other devices via wireless communication technology.

[0128] Electronic device 700 implements display functions through a GPU, a display screen 794, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 794 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 710 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0129] The display screen 794 is used to display images, display videos, and receive swipe operations, etc. The display screen 794 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 700 may include one or more display screens 794.

[0130] Electronic device 700 can achieve shooting function through ISP, camera 793, video codec, GPU, display 794 and application processor.

[0131] The ISP (Image Signal Processor) is used to process data fed back from the camera 793. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 793.

[0132] Camera 793 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through a lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, electronic device 700 may include one or more cameras 793.

[0133] The external memory interface 720 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 700. The external memory card communicates with the processor 710 through the external memory interface 720 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0134] Internal memory 721 can be used to store executable program code, including instructions. Internal memory 721 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.). The data storage area may store data created during the use of electronic device 700 (such as audio data, phonebook, etc.). Furthermore, internal memory 721 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 710 executes various functional applications and data processing of electronic device 700 by running instructions stored in internal memory 721 and / or instructions stored in memory located in the processor.

[0135] Electronic device 700 can implement audio functions such as music playback and recording through audio module 770, speaker 770A, receiver 770B, microphone 770C, headphone jack 770D, and application processor.

[0136] The gyroscope sensor 780A can be used to determine the motion attitude of the electronic device 700. In some embodiments, the gyroscope sensor 780A can determine the angular velocity of the electronic device 700 around three axes (i.e., the x-axis, y-axis, and z-axis). The gyroscope sensor 780A can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 780A detects the angle of the electronic device 700's shake, calculates the distance the lens needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 700 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 780A can also be used in scenarios such as navigation and motion-sensing games.

[0137] The accelerometer 780B can detect the magnitude of acceleration of electronic device 700 in various directions (typically three axes). When electronic device 700 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic device and applied to applications such as screen orientation switching and pedometers.

[0138] Buttons 790 include a power button, volume buttons, etc. Buttons 790 can be mechanical buttons or touch buttons. Electronic device 700 can receive button input and generate key signal inputs related to user settings and function control. Motor 791 can generate vibration alerts. Motor 791 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 792 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 795 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 795 to achieve contact and separation with electronic device 70.

[0139] The software system of the electronic device 700 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc. This application embodiment uses the layered architecture Android system as an example to illustrate the software structure of the electronic device 700.

[0140] Figure 8 This is a schematic diagram of the software system architecture of an electronic device 700 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer, and the kernel layer.

[0141] The application layer can include a series of application packages. For example... Figure 8 As shown, the application package can include applications such as camera, settings, and calendar.

[0142] In this context, a camera app is an application with shooting and video recording functions. Electronic devices can respond to a user's action of opening the camera app to take photos or record videos. It's understandable that the photo and video recording functions of a camera app can also be invoked by other applications.

[0143] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes a set of predefined functions.

[0144] Among them, such as Figure 8 As shown, the application framework layer can also include a camera service, which can be called by camera applications to enable functions such as taking photos or recording videos.

[0145] In addition, such as Figure 8 As shown, the application framework layer may also include a window manager, content provider, resource manager, and view system, etc.

[0146] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0147] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0148] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0149] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0150] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0151] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0152] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0153] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0154] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0155] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG2, H.262, MP3, AAC, AMR, JPG, and PNG.

[0156] 3D graphics processing libraries are used to implement 3D graphics drawing, image rendering, compositing, and layer processing. 2D graphics engines are drawing engines for 2D graphics.

[0157] The Hardware Abstraction Layer (HAL) is a layer of abstraction situated between the kernel layer and the Android runtime. The HAL can be a wrapper around the kernel layer's hardware drivers, providing a calling interface for the application framework layer. In this embodiment, the HAL may include a camera hardware abstraction layer (camera HAL).

[0158] The kernel layer is the layer between hardware and software. The kernel layer includes at least camera drivers, sensor drivers, and display drivers. In some embodiments, the camera driver controls the operation of the camera, the sensor driver controls the operation of the sensor, and the display driver controls the display screen to show images.

[0159] The hardware can be a camera, a sensor, or a display screen, etc. In this embodiment, the camera can be a front-facing camera or a rear-facing camera.

[0160] In some embodiments, during the execution of the phase focusing method of the present application embodiments by the electronic device, the camera driver can drive the camera to acquire a first phase image captured by the phase detection pixels when the optical image stabilization module is activated, and send the first phase image to the camera hardware abstraction module; the camera hardware abstraction module can also acquire the first coordinates of the position of the optical image stabilization module at the target time; the camera hardware abstraction module determines the phase gain compensation matrix according to the first coordinates and multiple phase gain calibration matrices, and uses the phase gain compensation matrix to compensate the first phase image to obtain a second phase image, and then performs phase focusing based on the second phase image.

[0161] Specifically, the camera hardware abstraction module can determine the phase difference information based on the second phase image, and then calculate the defocus distance based on the phase difference information. The camera hardware abstraction module can send the defocus distance to the focusing motor in the camera through the camera driver. The focusing motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, thereby achieving phase focusing.

[0162] It should be noted that although the embodiments of this application are described using the Android system, the principle of the phase focusing method is also applicable to electronic devices with operating systems such as iOS or Windows.

[0163] The technical solution of this application and how it solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be implemented independently or in combination with each other. The same or similar concepts or processes may not be described again in some embodiments.

[0164] For ease of understanding, this application uses a mobile phone as an electronic device. First, we will use some user interfaces shown in this application to illustrate the application scenarios of the phase focusing method.

[0165] When a user turns on the screen of an electronic device and the device is in an unlocked state, the electronic device can display something like this: Figure 9 The first interface 901 is shown in (a) of the diagram. The first interface 901 can be the desktop of the electronic device, on which icons of multiple installed applications are displayed, such as file management application icon, email application icon, weather application icon, calculator application icon, clock application icon, voice recorder application icon, music application icon, settings application icon, address book application icon, phone application icon, messaging application icon, and camera application icon 9011, etc.

[0166] Users can perform touch operations on the camera application icon 9011, such as tapping or long-pressing, so that the electronic device receives the user's touch operation on the camera application icon 9011 and launches the camera application in response to the touch operation.

[0167] After the camera app is launched, the electronic device can display something like this: Figure 9 The second interface 902 is shown in (b) above. The second interface 902 can be a preview interface provided by the camera application to implement the shooting function, which includes a preview area 9021, shooting controls 9022, and function controls corresponding to various shooting modes.

[0168] The preview area 9021 can be used to display the raw image captured by the image sensor in the camera. The shooting control 9022 is used to trigger the shooting operation of the electronic device. The functional controls corresponding to various shooting modes may include night mode controls, portrait mode controls, photo mode controls, video mode controls, professional mode controls, and more controls for enabling more functions in the camera application.

[0169] One possible approach is for a user to perform a touch operation, such as a click, on a specific location of the original image displayed within the preview area 9021. This causes the electronic device to receive the user's touch input, and in response, selects the touched location as the focus point, and then selects a rectangular or circular area centered on the focus point as the focal area. For example... Figure 9 As shown in (c), the electronic device can display a focus frame 9023 within the preview area 9021, and the area within the focus frame 9023 is the focus area.

[0170] Another possible approach is for the electronic device to invoke an image recognition model to identify the target object displayed in the preview area 9021, determine the focus point based on the location of the target object, and select a rectangular or circular area centered on the focus point as the focus area. The target object can be a human body, a face, etc.

[0171] Therefore, in this embodiment, the camera application can be launched by touching the camera application icon 9011. After the camera application is launched, the focus area is determined by the user touching a certain position of the original image displayed in the preview area 9021, or by using an image recognition model to recognize the original image displayed in the preview area 9021. After determining the focus area, the electronic device can execute the process corresponding to the phase focusing method provided in this embodiment to perform phase focusing.

[0172] In addition, users can access the camera app of their electronic device by calling the corresponding interface through a third-party application installed on the device. Alternatively, when the electronic device is locked, users can instruct the device to open the camera app by swiping right on the screen. Or, if the electronic device is locked and the lock screen includes a camera app icon, users can tap the icon to open the camera app.

[0173] It should be understood that the above is an example of how to open the camera application; the camera application can also be opened by voice command or other commands, and this application does not limit this in any way.

[0174] The phase focusing method provided in this application embodiment can be applied not only to the preview scene of the camera application mentioned above, but also to the shooting scene of the camera application.

[0175] For example, Figure 10 A flowchart of a phase-detection autofocus method provided in this application embodiment is available. This method can be applied to an electronic device, which may include a camera, an optical image stabilization module, a lens 10, and an image sensor 20. The image sensor 20 includes multiple phase-detection pixels. The image sensor 20 can be... Figures 2 to 4 Any of the image sensors shown. (Refer to...) Figure 10 As shown, this phase-detection focusing method may specifically include the following steps:

[0176] S1001, When the optical image stabilization module is activated, the electronic device acquires the first phase image captured by the phase detection pixels.

[0177] In some embodiments, when the camera activates the optical image stabilization module for optical image stabilization, the camera can simultaneously acquire the original image and the first phase image, with the original image corresponding to the first phase image.

[0178] The raw image refers to the image captured by all pixels of the image sensor in the camera. The raw image can be in RAW format and can be displayed in the preview area of ​​the electronic device. The first phase image refers to the image captured by multiple phase detection pixels of the image sensor in the camera.

[0179] Understandably, regarding Figure 2 The image sensor shown has some pixels set as phase detection pixels, meaning the image sensor includes both phase detection pixels and non-phase detection pixels. In this case, the size of the first phase image will be smaller than the size of the original image.

[0180] S1002, the electronic device acquires the second coordinates of the focus point in the original image.

[0181] In some embodiments, the electronic device can acquire the second coordinates of the focus point in the original image acquired by the image sensor. The second coordinates of the focus point can be (Xn, Yn), where Xn is the coordinate component along the width direction of the original image, and Yn is the coordinate component along the width direction of the original image. n The coordinate components in the second coordinate system are along the height direction of the original image.

[0182] The focus point can refer to the center position of the focus frame. The focus point can be determined by the touch position when a user touches a location in the original image, or it can be determined by identifying the target object in the original image and determining its location.

[0183] S1003, the electronic device determines the target duration based on the second coordinate.

[0184] In some embodiments, the electronic device can determine the target duration based on a second coordinate of the focus point in the original image. The target duration refers to the time elapsed from the initial moment when the original image is acquired to the moment when the focus point in the original image is located at the time of acquisition.

[0185] Specifically, the electronic device can determine the target duration based on the coordinate components along the height direction of the original image in the second coordinate system.

[0186] In one implementation, the electronic device calculates the target duration using the following formula:

[0187]

[0188] Among them, T lag Y represents the target duration, EIT represents the exposure duration of the original image, and Y represents the target duration. n Here, H represents the coordinate component along the height direction of the original image in the second coordinate system, where H is the height of the original image, and RowNum is the total number of rows in the photosensitive unit array of the image sensor. time The reading time for reading image data generated by reading a row of photosensitive cells in the photosensitive cell array.

[0189] When an image sensor acquires a raw image, it starts controlling the exposure of the photosensitive unit array from the exposure start time corresponding to the raw image, and then sequentially reads the image data generated by each row of photosensitive units in the array from the exposure end time corresponding to the raw image to obtain the raw image. Therefore, the acquisition time of this frame of raw image includes the exposure time and the reading time of the raw image.

[0190] like Figure 11 As shown in (a), the starting point of the exposure of the original image is defined as the start of frame (SOF) point, and the ending point of the original image is defined as the end of frame (EOF) point. The first timestamp under the SOF point is defined as T1, and the second timestamp under the EOF point is defined as T2.

[0191] During the acquisition of the original image, the acquisition duration is the time interval between the second timestamp T2 and the first timestamp T1. The first timestamp T1 represents the timestamp of the start of exposure of the original image, which is also the timestamp of the moment when the image sensor begins to acquire the original image; the second timestamp T2 represents the timestamp of the end of reading the original image, which is also the timestamp of the moment when the image sensor ends to acquire the original image.

[0192] Therefore, in the above formula, This can represent the reading time required to read the focus point location in the original image. The electronic device can use half the exposure time of the original image, plus the sum of the reading time required to read the focus point location in the original image, as the target duration. The exposure time of the original image can also refer to the exposure time of the first phase image.

[0193] S1004, the electronic device determines the target time by the sum of the initial time when it starts acquiring the original image and the target duration; the target time is the time when the focus point in the original image acquired by the image sensor is at the time of acquisition.

[0194] It should be understood that the target time is actually estimated by the electronic device, representing the possible time at which the focus point in the original image was located during acquisition. The target time is essentially an estimated time.

[0195] S1005, The electronic device acquires the first coordinates of the position of the optical image stabilization module at the target time.

[0196] During the process of the camera activating the optical image stabilization module to acquire the original image and the first phase image, the optical image stabilization module can move to move the lens along the first movement direction and / or the second movement direction to compensate for the effects of shaking and achieve the purpose of optical image stabilization.

[0197] In this way, the coordinates of the position of the optical image stabilization module after each movement can be statistically determined during the acquisition process of the original image and the first phase image, that is, during the time interval between the second timestamp T2 and the first timestamp T1.

[0198] like Figure 11As shown in (b), during the acquisition of the original image and the first phase image, the optical image stabilization module can move along the first moving direction and the second moving direction. OIS-X represents a schematic diagram of the displacement of the optical image stabilization module when it moves along the first moving direction, and OIS-Y represents a schematic diagram of the displacement of the optical image stabilization module when it moves along the second moving direction.

[0199] After estimating the target time, the electronic device can extract the first coordinate of the optical image stabilization module's position at the target time from the coordinates of the position it held after each movement. The first coordinate of the optical image stabilization module's position at the target time can be (OIS... X OIS Y OIS X For the first coordinate component along the first direction of movement, OIS Y This represents the second coordinate component along the second direction of movement.

[0200] For example, the time interval between the moment the image sensor finishes acquiring the original image and the moment it begins acquiring the original image is 20ms (milliseconds). That is, the time interval between the second timestamp and the first timestamp is 20ms. The optical image stabilization module moves once every 1ms. Therefore, the optical image stabilization module can move 20 times during the original image acquisition process. This means that the coordinates of the optical image stabilization module at 20 different locations are acquired during the original image acquisition process. Starting from the initial moment when the original image acquisition begins, assuming the focus point in the original image is acquired at 10ms, the coordinates of the optical image stabilization module's position at 10ms are obtained as the first coordinate.

[0201] It is understandable that the coordinates of the position of the optical image stabilization module after each movement actually represent the relative position between the optical image stabilization module and the optical center of the image sensor after the movement. The relative position between the optical image stabilization module and the optical center of the image sensor after the movement can also be used to characterize the relative position between the optical center of the lens 10 and the optical center of the image sensor 20 after the optical image stabilization module moves to move the lens 10.

[0202] For example, when the optical image stabilization module is not activated, the coordinates of the optical image stabilization module can be (0, 0), which can indicate that the optical center of the lens 10 and the optical center of the image sensor 20 coincide along the optical axis of the camera.

[0203] S1006, the electronic device determines the phase gain compensation matrix based on the first coordinate and multiple phase gain calibration matrices; the relative positions of the optical center of the lens and the optical center of the image sensor are different during the calibration of the multiple phase gain calibration matrices.

[0204] Before determining the phase gain compensation matrix based on the first coordinate and multiple phase gain calibration matrices, the electronic device can also acquire multiple pre-calibrated phase gain calibration matrices.

[0205] The multiple phase gain calibration matrices include a first type of phase gain calibration matrix and / or a second type of phase gain calibration matrix. The first type of phase gain calibration matrix includes a first phase gain calibration matrix, a second phase gain calibration matrix, and a third phase gain calibration matrix; and during calibration, the relative positions of the optical center of the lens 10 and the optical center of the image sensor 20 are different. The second type of phase gain calibration matrix includes a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix; and during calibration, the relative positions of the optical center of the lens 10 and the optical center of the image sensor 20 are different.

[0206] The optical image stabilization module has two movable directions: a first movable direction and a second movable direction. This means the optical image stabilization module can drive the lens 10 to move along the first and second movable directions. The first and second movable directions are perpendicular to each other and both are perpendicular to the optical axis of the camera. The first movable direction includes a first direction and a second direction that are opposite to each other, and the second movable direction includes a third direction and a fourth direction that are opposite to each other. For example, such as... Figure 12 As shown, the first direction can be the X1 direction, the second direction can be the X2 direction, the third direction can be the Y1 direction, and the fourth direction can be the Y2 direction.

[0207] like Figure 12 As shown in (a) to (e), these represent the relative positions between the optical center of lens 10 and the optical center of image sensor 20 during the calibration of each phase gain calibration matrix. The intersection of the two dashed lines indicates the optical center of image sensor 20.

[0208] like Figure 12 As shown in (a), during the calibration of the first and fourth phase gain calibration matrices, the optical center of the lens 10 coincides with the optical center of the image sensor 20 along the optical axis of the camera. In this case, the relative position of the optical center of the lens 10 with respect to the optical center of the image sensor 20 is G0, and the position coordinates of G0 are (0, 0).

[0209] The first phase gain calibration matrix includes the first left phase gain calibration submatrix Q. 0L and the first right phase gain calibration submatrix Q 0RThe fourth phase gain calibration matrix includes the first upper phase gain calibration submatrix Q. 0T and the first lower phase gain calibration submatrix Q 0B .Right now Figure 12 The phase gain calibration matrix Q0 shown can actually include the first left phase gain calibration submatrix Q. 0L The first right phase gain calibration submatrix Q 0R The first upper phase gain calibration submatrix Q 0T and the first lower phase gain calibration submatrix Q 0B .

[0210] like Figure 12 As shown in (b), during the calibration of the second phase gain calibration matrix, the optical center of the lens 10 is offset by a first distance relative to the optical center of the image sensor 20 along a first direction; the first distance is the maximum distance that the lens 10 can move along the first direction. In this case, the relative position of the optical center of the lens 10 with respect to the optical center of the image sensor 20 is G1, and the position coordinates of position G1 are (X... lim1 ,0),X lim1 This is the first distance.

[0211] in, Figure 12 The second phase gain calibration matrix Q1 shown can actually include the second left phase gain calibration submatrix Q. 1L Second right phase gain calibration submatrix Q 1R .

[0212] like Figure 12 As shown in (c), during calibration, the optical center of lens 10 is offset by a second distance relative to the optical center of image sensor 20 along the second direction; the second distance is the maximum distance that lens 10 can move along the second direction. In this case, the relative position of the optical center of lens 10 with respect to the optical center of image sensor 20 is G2, and the position coordinates of position G2 are (-X). lim2 ,0),X lim2 This is the second distance.

[0213] in, Figure 12 The third phase gain calibration matrix Q2 shown can actually include the third left phase gain calibration submatrix Q. 2L and the third right phase gain calibration submatrix Q 2R .

[0214] like Figure 12As shown in (d), during calibration, the optical center of lens 10 is offset by a third distance relative to the optical center of image sensor 20 along a third direction; the third distance is the maximum distance that lens 10 can move along the third direction. In this case, the relative position of the optical center of lens 10 with respect to the optical center of image sensor 20 is G3, and the position coordinates of position G3 are (0, Y). lim1 ), Y lim1 This is the third distance.

[0215] in, Figure 12 The fifth phase gain calibration matrix Q3 shown can actually include the second upper phase gain calibration submatrix Q. 3T Second lower phase gain calibration submatrix Q 3B .

[0216] like Figure 12 As shown in (e), during calibration, the optical center of lens 10 is offset by a fourth distance relative to the optical center of image sensor 20 along the fourth direction; the fourth distance is the maximum distance that lens 10 can move along the fourth direction. In this case, the relative position of the optical center of lens 10 with respect to the optical center of image sensor 20 is G4, and the position coordinates of position G4 are (0, -Y). lim2 ), Y lim2 This is the fourth distance.

[0217] in, Figure 12 The sixth phase gain calibration matrix Q4 shown can actually include the third upper phase gain calibration submatrix Q. 4T and the third lower phase gain calibration submatrix Q 4B .

[0218] In some embodiments, the first distance X lim1 With the second distance X lim2 They can be equal or unequal; the third distance Y lim1 Distance Y to the fourth lim2 The distances can be equal or unequal, and this application does not limit this. Furthermore, the first distance, the second distance, the third distance, and the fourth distance are all positive numbers.

[0219] In some embodiments, each phase gain calibration matrix includes a first phase gain calibration submatrix and a second phase gain calibration submatrix. When the phase gain calibration matrix is ​​a first phase gain calibration matrix, the first phase gain calibration submatrix may be a first left phase gain calibration submatrix, and the second phase gain calibration submatrix may be a first right phase gain calibration submatrix; when the phase gain calibration matrix is ​​a second phase gain calibration matrix, the first phase gain calibration submatrix may be a second left phase gain calibration submatrix, and the second phase gain calibration submatrix may be a second right phase gain calibration submatrix; when the phase gain calibration matrix is ​​a third phase gain calibration matrix, the first phase gain calibration submatrix may be a third left phase gain calibration submatrix, and the second phase gain calibration submatrix may be a third right phase gain calibration submatrix. When the phase gain calibration matrix is ​​the fourth phase gain calibration matrix, the first phase gain calibration submatrix can be the first upper phase gain calibration submatrix, and the second phase gain calibration submatrix can be the first lower phase gain calibration submatrix; when the phase gain calibration matrix is ​​the fifth phase gain calibration matrix, the first phase gain calibration submatrix can be the second upper phase gain calibration submatrix, and the second phase gain calibration submatrix can be the second lower phase gain calibration submatrix; when the phase gain calibration matrix is ​​the sixth phase gain calibration matrix, the first phase gain calibration submatrix can be the third upper phase gain calibration submatrix, and the second phase gain calibration submatrix can be the third lower phase gain calibration submatrix.

[0220] Each compensation coefficient in the first phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each first image block in the first phase test sub-image; each compensation coefficient in the second phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each second image block in the second phase test sub-image; the maximum brightness value is the maximum of the brightness values ​​of each first image block and each second image block. The first and second phase test sub-images are obtained by splitting the phase test image; the phase test image is a phase image acquired when the lens focus position is a preset focus position and the relative position between the optical center of the lens and the optical center of the image sensor is a preset position.

[0221] In one implementation, each compensation coefficient in the first phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each first image block in the first phase test sub-image; and each compensation coefficient in the second phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each second image block in the second phase test sub-image.

[0222] Taking the calibration process of the first phase gain calibration matrix as an example, Figure 13 A flowchart illustrating the calibration process of the first phase gain calibration matrix provided in this embodiment of the application. (Refer to...) Figure 13 As shown, the calibration process may specifically include the following steps:

[0223] S1301 uses phase detection pixels to acquire phase test images when the lens is at a preset focus position and the optical center of the lens coincides with the optical center of the image sensor along the optical axis of the camera.

[0224] The focus position of the lens in the camera is set to a preset focus position, which is half the optical travel of the lens; and the optical center of the lens is set to coincide with the optical center of the image sensor along the optical axis of the camera, that is, the optical displacement between the optical center of the lens and the optical center of the image sensor is 0.

[0225] With the lens focused at a preset focus position and the optical center of the lens and the optical center of the image sensor aligned along the optical axis of the camera, the camera is used to photograph a uniform light plate, and a phase test image is acquired through the phase detection pixels in the camera.

[0226] When photographing a uniformly lit surface, the object distance can be 10 mm. A uniformly lit surface refers to a planar light source with uniform illumination and color temperature within its effective area.

[0227] S1302, split the phase test image into a left phase test sub-image and a right phase test sub-image.

[0228] The phase test image can include a left phase test sub-image and a right phase test sub-image. The left phase test sub-image is obtained by separating the image acquired by the left phase detection pixels from the phase test image; correspondingly, the right phase test sub-image is obtained by separating the image acquired by the right phase detection pixels from the phase test image.

[0229] S1303, obtain the brightness value of each first image block in the left phase test sub-image and the brightness value of each second image block in the right phase test sub-image.

[0230] The left phase test sub-image is divided into multiple first image blocks, each of which includes multiple pixels. The average brightness value of all pixels in each first image block is taken as the brightness value of that first image block.

[0231] Accordingly, the right phase test sub-image is divided into multiple second image blocks, each of which includes multiple pixels. The average brightness value of all pixels in each second image block is taken as the brightness value of that second image block.

[0232] For example, the left phase test sub-image is divided into 17×13 first image blocks, and the brightness value of the first image block in the i-th row and j-th column is lpd. ij It is actually the average brightness value of all pixels in the first image block in the i-th row and j-th column. The right phase test sub-image is divided into 17×13 second image blocks, and the brightness value of the second image block in the i-th row and j-th column is rpd. ij It is actually the average brightness value of all pixels in the second image block in the i-th row and j-th column. Here, i and j are both positive integers.

[0233] S1304, calculate the ratio of the maximum brightness value to the brightness value of each first image block in the left phase test sub-image to obtain each compensation coefficient in the first left phase gain calibration sub-matrix.

[0234] In some embodiments, the maximum brightness value is the maximum value among the brightness values ​​of each first image block in the left phase test sub-image and the brightness values ​​of each second image block in the right phase test sub-image.

[0235] Specifically, each compensation coefficient in the first left phase gain calibration submatrix can be calculated using the following formula:

[0236] lgain ij =max{lpd ij ,rpd ij} / lpd ij

[0237] Among them, lgain ij lpd is the compensation coefficient located in the i-th row and j-th column of the first left phase gain calibration submatrix. ij rpd is the brightness value of the first image block in the i-th row and j-th column of the left phase test sub-image. ij This represents the brightness value of the second image block in the i-th row and j-th column of the right phase test sub-image.

[0238] For example, dividing the left phase test sub-image into 17×13 first image blocks and the right phase test sub-image into 17×13 second image blocks can make the first left phase gain calibration sub-matrix include 17×13 compensation coefficients.

[0239] S1305, calculate the ratio of the maximum brightness value to the brightness value of each second image block in the right phase test sub-image to obtain each compensation coefficient in the first right phase gain calibration sub-matrix.

[0240] Specifically, each compensation coefficient in the first right phase gain calibration submatrix can be calculated using the following formula:

[0241] rgain ij =max{lpd ij ,rpd ij} / rpd ij

[0242] Among them, rgain ij lpd is the compensation coefficient located in the i-th row and j-th column of the first right phase gain calibration submatrix. ij rpd is the brightness value of the first image block in the i-th row and j-th column of the left phase test sub-image. ij This represents the brightness value of the second image block in the i-th row and j-th column of the right phase test sub-image.

[0243] For example, dividing the left phase test sub-image into 17×13 first image blocks and the right phase test sub-image into 17×13 second image blocks can make the first right phase gain calibration sub-matrix include 17×13 compensation coefficients.

[0244] This embodiment divides the left-phase test sub-image into multiple first image blocks, with the brightness value of each first image block being the average of the brightness values ​​of all pixels within that first image block. This allows each first image block to correspond to a compensation coefficient in a first left-phase gain calibration sub-matrix. Similarly, by dividing the right-phase test sub-image into multiple second image blocks, with the brightness value of each second image block being the average of the brightness values ​​of all pixels within that second image block, each second image block can also correspond to a compensation coefficient in a first right-phase gain calibration sub-matrix. This reduces the number of parameters in the first phase gain calibration matrix and other phase gain calibration matrices, thereby reducing the computational load when calculating the phase gain compensation matrix.

[0245] In summary, following steps S1301 to S1305, the first phase gain calibration matrix can be obtained. In practical applications, if the optical displacement between the optical center of the lens and the optical center of the image sensor is zero, the first phase gain calibration matrix can be used to compensate for the brightness of the first phase image, making the brightness values ​​of the pixels in the compensated first phase image more uniform.

[0246] In the actual calibration process, the optical image stabilization module can be controlled sequentially to move the lens to a position such as... Figure 12 The G0 position shown in (a) is as follows: Figure 12 The G1 position shown in (b) and as shown in the figure Figure 12 The position G2 is shown in (c) in the diagram.

[0247] With the lens focused at a preset focus position and the optical center of the lens and the optical center of the image sensor aligned along the optical axis of the camera, the camera captures an image of a uniform light plate. A phase test image is then acquired through the phase detection pixels in the camera. Following steps S1302 to S1305, a first phase gain calibration matrix is ​​obtained. This first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix.

[0248] Accordingly, with the lens focusing position set at a preset focusing position and the optical center of the lens offset by a first distance relative to the optical center of the image sensor along a first direction, a camera is used to photograph a uniform light plate to acquire a phase test image through the phase detection pixels in the camera. Following steps S1302 to S1305 described above, a second phase gain calibration matrix is ​​obtained. The second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix.

[0249] With the lens's focus position set at a preset focus position and the lens's optical center offset by a second distance relative to the image sensor's optical center along a second direction, a camera is used to photograph a uniform light plate. A phase test image is then acquired through the phase detection pixels in the camera, and the process follows steps S1302 to S1305 to obtain a third phase gain calibration matrix. This third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix.

[0250] It is worth noting that when the first phase gain calibration matrix, the second phase gain calibration matrix, and the third phase gain calibration matrix are obtained, the first phase test sub-image refers to the left phase test sub-image, and the second phase test sub-image refers to the right phase test sub-image.

[0251] In the actual calibration process, the optical image stabilization module can also be controlled sequentially to move the lens to a position such as... Figure 12 The G0 position shown in (a) is as follows: Figure 12 The G3 position shown in (d) and as shown in the figure Figure 12 The position G4 is shown in (e) in the diagram.

[0252] With the lens focused at a preset focus position and the optical center of the lens and the optical center of the image sensor aligned along the optical axis of the camera, the camera captures a uniform light plate to obtain a phase test image via the phase detection pixels in the camera. The phase test image is then divided into an upper phase test sub-image and a lower phase test sub-image. The brightness value of each first image block in the upper phase test sub-image and the brightness value of each second image block in the lower phase test sub-image are obtained. The ratio of the maximum brightness value to the brightness value of each first image block in the upper phase test sub-image is calculated to obtain each compensation coefficient in the first upper phase gain calibration sub-matrix. Similarly, the ratio of the maximum brightness value to the brightness value of each second image block in the lower phase test sub-image is calculated to obtain each compensation coefficient in the first lower phase gain calibration sub-matrix. The fourth phase gain calibration matrix includes the first upper phase gain calibration sub-matrix and the first lower phase gain calibration sub-matrix. The maximum brightness value is the maximum value among the brightness values ​​of each first image block in the upper phase test sub-image and the brightness values ​​of each second image block in the lower phase test sub-image.

[0253] Accordingly, with the lens focusing position set at a preset focus position and the optical center of the lens offset by a third distance relative to the optical center of the image sensor along a third direction, a camera is used to photograph a uniform light plate to acquire a phase test image through the phase detection pixels in the camera. The phase test image is then divided into an upper phase test sub-image and a lower phase test sub-image. The brightness value of each first image block in the upper phase test sub-image and the brightness value of each second image block in the lower phase test sub-image are obtained. The ratio of the maximum brightness value to the brightness value of each first image block in the upper phase test sub-image is calculated to obtain each compensation coefficient in the second upper phase gain calibration sub-matrix. The ratio of the maximum brightness value to the brightness value of each second image block in the lower phase test sub-image is also calculated to obtain each compensation coefficient in the second lower phase gain calibration sub-matrix. The fifth phase gain calibration matrix includes the second upper phase gain calibration sub-matrix and the second lower phase gain calibration sub-matrix.

[0254] With the lens focused at a preset focus position and the optical center of the lens offset by a fourth distance relative to the optical center of the image sensor along a fourth direction, a camera is used to photograph a uniform light plate to acquire a phase test image through the phase detection pixels in the camera. The phase test image is then divided into an upper phase test sub-image and a lower phase test sub-image. The brightness value of each first image block in the upper phase test sub-image and the brightness value of each second image block in the lower phase test sub-image are obtained. The ratio of the maximum brightness value to the brightness value of each first image block in the upper phase test sub-image is calculated to obtain each compensation coefficient in the third upper phase gain calibration sub-matrix. The ratio of the maximum brightness value to the brightness value of each second image block in the lower phase test sub-image is also calculated to obtain each compensation coefficient in the third lower phase gain calibration sub-matrix. The sixth phase gain calibration matrix includes the third upper phase gain calibration sub-matrix and the third lower phase gain calibration sub-matrix.

[0255] It is worth noting that when the fourth, fifth, and sixth phase gain calibration matrices are obtained through calibration, the first phase test sub-image refers to the upper phase test sub-image, and the second phase test sub-image refers to the lower phase test sub-image.

[0256] It should be noted that the camera used in the calibration process of the phase gain calibration matrix in this application embodiment may or may not be the same camera used to acquire the first phase image during the execution of the phase focusing method. If the camera used in the calibration process of the phase gain calibration matrix is ​​not the same camera used to acquire the first phase image during the execution of the phase focusing method, the type of camera used in the calibration process of the phase gain calibration matrix and the type of camera used to acquire the first phase image during the execution of the phase focusing method can be the same. For example, the structural composition of the camera used in the calibration process of the phase gain calibration matrix can be the same as that of the camera used to acquire the first phase image during the execution of the phase focusing method.

[0257] After obtaining multiple phase gain calibration matrices in advance according to the above method, the electronic device can determine the phase gain compensation matrix based on the first coordinate and the multiple phase gain calibration matrices.

[0258] In one scenario, for left and right phase detection autofocus, the electronic device can calculate the phase gain compensation matrix based on the first coordinates of the optical image stabilization module's position at the target time, the first phase gain calibration matrix, the second phase gain calibration matrix, and the third phase gain calibration matrix.

[0259] The first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix; the second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix; and the third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix. The first coordinate includes a first coordinate component OIS along the first movement direction. X .

[0260] Specifically, the electronic device can determine the phase gain compensation matrix in the following ways: the electronic device determines the first left phase gain compensation sub-matrix based on the first coordinate component, the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix, and the third left phase gain calibration sub-matrix; the electronic device determines the first right phase gain compensation sub-matrix based on the first coordinate component, the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix, and the third right phase gain calibration sub-matrix.

[0261] In scenarios involving left and right phase detection focusing, the phase gain compensation matrix can include a first left phase gain compensation sub-matrix and a first right phase gain compensation sub-matrix.

[0262] In one implementation, the electronic device calculates the first left-phase gain compensation sub-matrix using the following formula:

[0263]

[0264] Among them, GainMap L Q is the first left-side phase gain compensation submatrix. 0L Q is the first left-side phase gain calibration submatrix. 1L Q is the second left-side phase gain calibration submatrix. 2L X is the third left phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This is the first coordinate component.

[0265] Therefore, in the above formula, the compensation coefficient at the corresponding position in the first left-phase gain compensation sub-matrix is ​​calculated by using each compensation coefficient in the first left-phase gain calibration sub-matrix, the second left-phase gain calibration sub-matrix, and the third left-phase gain calibration sub-matrix.

[0266] For example, the compensation coefficients in the i-th row and j-th column of the first left-phase gain compensation sub-matrix are calculated using the compensation coefficients in the i-th row and j-th column of the first left-phase gain calibration sub-matrix, the second left-phase gain calibration sub-matrix, and the third left-phase gain calibration sub-matrix, respectively.

[0267] When the first left-phase gain calibration sub-matrix, the second left-phase gain calibration sub-matrix, and the third left-phase gain calibration sub-matrix all contain 17×13 compensation coefficients, the calculated first left-phase gain compensation sub-matrix also contains 17×13 compensation coefficients.

[0268] Accordingly, the electronic device calculates the first right phase gain compensation sub-matrix using the following formula:

[0269]

[0270] Among them, GainMap R Q is the first right phase gain compensation submatrix. 0R Q is the first right phase gain calibration submatrix. 1R Q is the second right phase gain calibration submatrix. 2R X is the third right phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This is the first coordinate component.

[0271] Therefore, in the above formula, the compensation coefficient at the corresponding position in the first right phase gain compensation sub-matrix is ​​calculated by using each compensation coefficient in the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix, and the third right phase gain calibration sub-matrix.

[0272] For example, the compensation coefficients in the i-th row and j-th column of the first right-phase gain compensation sub-matrix can be calculated using the compensation coefficients in the i-th row and j-th column of the first right-phase gain calibration sub-matrix, the second right-phase gain calibration sub-matrix, and the third right-phase gain calibration sub-matrix, respectively.

[0273] When the first right-phase gain calibration sub-matrix, the second right-phase gain calibration sub-matrix, and the third right-phase gain calibration sub-matrix all contain 17×13 compensation coefficients, the calculated first right-phase gain compensation sub-matrix also contains 17×13 compensation coefficients.

[0274] It should be understood that in scenarios involving left and right phase detection focusing, it is only necessary to pre-calibrate the first phase gain calibration matrix, the second phase gain calibration matrix, and the third phase gain calibration matrix to calculate the phase gain compensation matrix for brightness compensation of the first phase image.

[0275] In another scenario, for scenarios involving top and bottom phase detection autofocus, the electronic device can calculate the phase gain compensation matrix based on the first coordinate, fourth phase gain calibration matrix, fifth phase gain calibration matrix, and sixth phase gain calibration matrix of the optical image stabilization module at the target time.

[0276] The fourth phase gain calibration matrix includes a first upper phase gain calibration sub-matrix and a first lower phase gain calibration sub-matrix; the fifth phase gain calibration matrix includes a second upper phase gain calibration sub-matrix and a second lower phase gain calibration sub-matrix; and the sixth phase gain calibration matrix includes a third upper phase gain calibration sub-matrix and a third lower phase gain calibration sub-matrix. The first coordinate includes the second coordinate component OIS along the second movement direction. Y .

[0277] Specifically, the electronic device can determine the phase gain compensation matrix in the following ways: the electronic device determines the first upper phase gain compensation sub-matrix based on the second coordinate component, the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix; the electronic device determines the first lower phase gain compensation sub-matrix based on the second coordinate component, the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix.

[0278] In scenarios involving upper and lower phase detection focusing, the phase gain compensation matrix can include a first upper phase gain compensation sub-matrix and a first lower phase gain compensation sub-matrix.

[0279] In one implementation, the electronic device calculates the first upper phase gain compensation sub-matrix using the following formula:

[0280]

[0281] Among them, GainMap T Q is the first upper phase gain compensation submatrix. 0T Q is the first upper phase gain calibration submatrix. 3T Q is the second upper phase gain calibration submatrix. 4T Y is the third upper phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS Y This is the second coordinate component.

[0282] Therefore, in the above formula, the compensation coefficient at the corresponding position in the first upper phase gain compensation submatrix is ​​calculated by using each compensation coefficient in the first upper phase gain calibration submatrix, the second upper phase gain calibration submatrix, and the third upper phase gain calibration submatrix.

[0283] For example, the compensation coefficients in the i-th row and j-th column of the first upper phase gain compensation sub-matrix are used respectively in the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix to calculate the compensation coefficients in the i-th row and j-th column of the first upper phase gain compensation sub-matrix.

[0284] When the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix all contain 17×13 compensation coefficients, the calculated first upper phase gain compensation sub-matrix also contains 17×13 compensation coefficients.

[0285] Accordingly, the electronic device calculates the first lower phase gain compensation sub-matrix using the following formula:

[0286]

[0287] Among them, GainMap B Q is the first lower phase gain compensation submatrix. 0B Q is the first lower phase gain calibration submatrix. 3B The second lower phase gain calibration submatrix, Q 4B Y is the third lower phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS Y This is the second coordinate component.

[0288] Therefore, in the above formula, the compensation coefficient at the corresponding position in the first lower phase gain compensation submatrix is ​​calculated by using each compensation coefficient in the first lower phase gain calibration submatrix, the second lower phase gain calibration submatrix, and the third lower phase gain calibration submatrix.

[0289] For example, the compensation coefficients in the i-th row and j-th column of the first lower phase gain compensation sub-matrix can be calculated using the compensation coefficients in the i-th row and j-th column of the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix, respectively.

[0290] When the first, second, and third lower phase gain calibration sub-matrixes include 17×13 compensation coefficients, the calculated first lower phase gain compensation sub-matrix also includes 17×13 compensation coefficients.

[0291] It should be understood that in scenarios involving upper and lower phase detection focusing, it is only necessary to pre-calibrate the fourth phase gain calibration matrix, the fifth phase gain calibration matrix, and the sixth phase gain calibration matrix to calculate the phase gain compensation matrix for brightness compensation of the first phase image.

[0292] In a four-phase detection autofocus scenario, it is necessary to pre-calibrate the first phase gain calibration matrix, the second phase gain calibration matrix, the third phase gain calibration matrix, the fourth phase gain calibration matrix, the fifth phase gain calibration matrix, and the sixth phase gain calibration matrix. These matrices are then used to calculate the phase gain compensation matrices corresponding to the left and right phase detection autofocus and the phase gain compensation matrices corresponding to the top and bottom phase detection autofocus, respectively. The brightness compensation of the first phase image is then performed sequentially using the phase gain compensation matrices corresponding to the left and right phase detection autofocus and the phase gain compensation matrices corresponding to the top and bottom phase detection autofocus.

[0293] S1007, the electronic device uses a phase gain compensation matrix to compensate the first phase image to obtain the second phase image.

[0294] After calculating the phase gain compensation matrix, the electronic device can use the phase gain compensation matrix to perform brightness compensation on the first phase image to obtain the second phase image, making the brightness values ​​of the pixels in the second phase image obtained after brightness compensation more uniform.

[0295] In some embodiments, the phase gain compensation matrix includes a first phase gain compensation sub-matrix and a second phase gain compensation sub-matrix. The electronic device can compensate for the first phase image to obtain a second phase image in the following manner: the electronic device splits the first phase image into a first phase sub-image and a second phase sub-image; the electronic device adjusts the size of the first phase gain compensation sub-matrix to obtain a third phase gain compensation sub-matrix, the size of which is equal to the size of the first phase sub-image; the electronic device uses the third phase gain compensation sub-matrix to compensate for the first phase sub-image to obtain a third phase sub-image; the electronic device adjusts the size of the second phase gain compensation sub-matrix to obtain a fourth phase gain compensation sub-matrix, the size of which is equal to the size of the second phase sub-image; the electronic device uses the fourth phase gain compensation sub-matrix to compensate for the second phase sub-image to obtain a fourth phase sub-image.

[0296] In one implementation, the electronic device can compensate for the first phase sub-image to obtain the third phase sub-image as follows: the electronic device multiplies the brightness value of each pixel in the first phase sub-image by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image. Correspondingly, the electronic device can compensate for the second phase sub-image to obtain the fourth phase sub-image as follows: the electronic device multiplies the brightness value of each pixel in the second phase sub-image by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image.

[0297] For example, the electronic device uses the brightness value of the pixel located in the m-th row and n-th column of the first phase sub-image, multiplied by the compensation coefficient in the m-th row and n-th column of the third phase gain compensation sub-matrix, to obtain the brightness value of the pixel located in the m-th row and n-th column of the third phase sub-image. The electronic device uses the brightness value of the pixel located in the m-th row and n-th column of the second phase sub-image, multiplied by the compensation coefficient in the m-th row and n-th column of the fourth phase gain compensation sub-matrix, to obtain the brightness value of the pixel located in the m-th row and n-th column of the fourth phase sub-image. Here, m and n are both positive integers.

[0298] In another implementation, the electronic device can compensate the first phase sub-image to obtain the third phase sub-image as follows: the electronic device multiplies the brightness value of each pixel within the focus area of ​​the first phase sub-image by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image. Correspondingly, the electronic device can compensate the second phase sub-image to obtain the fourth phase sub-image as follows: the electronic device multiplies the brightness value of each pixel within the focus area of ​​the second phase sub-image by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image.

[0299] It is understandable that when the electronic device obtains the second coordinates of the focus point in the original image, it can also obtain information such as the width and height of the focus frame, thereby enabling the electronic device to determine the coordinate position of the focus area in the original image.

[0300] In this way, the electronic device can perform brightness compensation on the brightness value of each pixel in the focus area of ​​the first phase sub-image to obtain the third phase sub-image; and perform brightness compensation on the brightness value of each pixel in the focus area of ​​the second phase sub-image to obtain the fourth phase sub-image.

[0301] If the electronic device only performs brightness compensation on the brightness value of each pixel in the focus area of ​​the first phase sub-image and the second phase sub-image, the amount of calculation required to perform brightness compensation on the first phase image can be reduced.

[0302] In a scenario involving left and right phase detection focusing, the first phase sub-image is the first left phase sub-image, the second phase sub-image is the first right phase sub-image, the third phase sub-image is the second left phase sub-image, and the fourth phase sub-image is the second right phase sub-image; the first phase gain compensation sub-matrix is ​​the first left phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​the first right phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​the second left phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​the second right phase gain compensation sub-matrix. The second phase image includes both the second left phase sub-image and the second right phase sub-image.

[0303] Specifically, the electronic device splits the first phase image into a first left-phase sub-image and a first right-phase sub-image. The first left-phase sub-image is obtained by separating the image acquired by the left-phase detection pixels from the first phase image; correspondingly, the first right-phase sub-image is obtained by separating the image acquired by the right-phase detection pixels from the first phase image. The first left-phase sub-image includes brightness values ​​corresponding to multiple left-phase detection pixels, and the first right-phase sub-image includes brightness values ​​corresponding to multiple right-phase detection pixels.

[0304] Since the size of the first left-phase sub-image may not be consistent with the size of the first left-phase gain compensation sub-matrix, the electronic device can use bilinear interpolation to adjust the size of the first left-phase gain compensation sub-matrix to obtain a second left-phase gain compensation sub-matrix, the size of which is equal to the size of the first left-phase sub-image.

[0305] For example, if the first left-phase gain compensation sub-matrix includes 17×13 compensation coefficients and the size of the first left-phase sub-image is 2000 pixels × 2000 pixels, then the electronic device can adjust the size of the first left-phase gain compensation sub-matrix from 17×13 to 2000×2000 to obtain the second left-phase gain compensation sub-matrix, that is, the second left-phase gain compensation sub-matrix includes 2000×2000 compensation coefficients.

[0306] After adjusting the size of the first left-phase gain compensation sub-matrix to obtain the second left-phase gain compensation sub-matrix, the electronic device uses the second left-phase gain compensation sub-matrix to compensate the first left-phase sub-image to obtain the second left-phase sub-image.

[0307] In one implementation, the electronic device uses the brightness value of each pixel in the first left-phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second left-phase gain compensation sub-matrix to obtain the second left-phase sub-image. In another implementation, the electronic device uses the brightness value of each pixel within the focus area of ​​the first left-phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second left-phase gain compensation sub-matrix to obtain the second left-phase sub-image.

[0308] Correspondingly, since the size of the first right-phase sub-image may not be consistent with the size of the first right-phase gain compensation sub-matrix, the electronic device can use bilinear interpolation to adjust the size of the first right-phase gain compensation sub-matrix to obtain the second right-phase gain compensation sub-matrix, the size of which is equal to the size of the first right-phase sub-image.

[0309] For example, if the first right-phase gain compensation sub-matrix includes 17×13 compensation coefficients and the size of the first right-phase sub-image is 2000 pixels × 2000 pixels, then the electronic device can adjust the size of the first right-phase gain compensation sub-matrix from 17×13 to 2000×2000 to obtain the second right-phase gain compensation sub-matrix, that is, the second right-phase gain compensation sub-matrix includes 2000×2000 compensation coefficients.

[0310] After adjusting the size of the first right-phase gain compensation sub-matrix to obtain the second right-phase gain compensation sub-matrix, the electronic device uses the second right-phase gain compensation sub-matrix to compensate the first right-phase sub-image to obtain the second right-phase sub-image.

[0311] In one implementation, the electronic device uses the brightness value of each pixel in the first right-phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second right-phase gain compensation sub-matrix to obtain the second right-phase sub-image. In another implementation, the electronic device uses the brightness value of each pixel within the focus area of ​​the first right-phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second right-phase gain compensation sub-matrix to obtain the second right-phase sub-image.

[0312] In a scenario involving upper and lower phase detection autofocus, the first phase sub-image is the first upper phase sub-image, the second phase sub-image is the first lower phase sub-image, the third phase sub-image is the second upper phase sub-image, and the fourth phase sub-image is the second lower phase sub-image. The first phase gain compensation sub-matrix is ​​the first upper phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​the first lower phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​the second upper phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​the second lower phase gain compensation sub-matrix. The second phase image includes both the second upper phase sub-image and the second lower phase sub-image.

[0313] Specifically, the electronic device splits the first phase image into a first upper phase sub-image and a first lower phase sub-image. The first upper phase sub-image is obtained by separating the image acquired by the upper phase detection pixels from the first phase image; correspondingly, the first lower phase sub-image is obtained by separating the image acquired by the lower phase detection pixels from the first phase image. The first upper phase sub-image includes brightness values ​​corresponding to multiple upper phase detection pixels, and the first lower phase sub-image includes brightness values ​​corresponding to multiple lower phase detection pixels.

[0314] Since the size of the first upper phase sub-image may not be consistent with the size of the first upper phase gain compensation sub-matrix, the electronic device can use bilinear interpolation to adjust the size of the first upper phase gain compensation sub-matrix to obtain a second upper phase gain compensation sub-matrix, the size of which is equal to the size of the first upper phase sub-image.

[0315] For example, if the first upper phase gain compensation sub-matrix includes 17×13 compensation coefficients and the size of the first upper phase sub-image is 2000 pixels × 2000 pixels, then the electronic device can adjust the size of the first upper phase gain compensation sub-matrix from 17×13 to 2000×2000 to obtain the second upper phase gain compensation sub-matrix, that is, the second upper phase gain compensation sub-matrix includes 2000×2000 compensation coefficients.

[0316] After adjusting the size of the first upper phase gain compensation sub-matrix to obtain the second upper phase gain compensation sub-matrix, the electronic device uses the second upper phase gain compensation sub-matrix to compensate the first upper phase sub-image to obtain the second upper phase sub-image.

[0317] In one implementation, the electronic device uses the brightness value of each pixel in the first upper-phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second upper-phase gain compensation sub-matrix to obtain the second upper-phase sub-image. In another implementation, the electronic device uses the brightness value of each pixel within the focus area of ​​the first upper-phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second upper-phase gain compensation sub-matrix to obtain the second upper-phase sub-image.

[0318] Correspondingly, since the size of the first lower phase sub-image may not be consistent with the size of the first lower phase gain compensation sub-matrix, the electronic device can use bilinear interpolation to adjust the size of the first lower phase gain compensation sub-matrix to obtain the second lower phase gain compensation sub-matrix, the size of which is equal to the size of the first lower phase sub-image.

[0319] For example, the first lower phase gain compensation sub-matrix also includes 17×13 compensation coefficients, and the size of the first lower phase sub-image is 2000 pixels × 2000 pixels. Then the electronic device can adjust the size of the first lower phase gain compensation sub-matrix from 17×13 to 2000×2000 to obtain the second lower phase gain compensation sub-matrix, that is, the second lower phase gain compensation sub-matrix includes 2000×2000 compensation coefficients.

[0320] After adjusting the size of the first lower phase gain compensation sub-matrix to obtain the second lower phase gain compensation sub-matrix, the electronic device uses the second lower phase gain compensation sub-matrix to compensate the first lower phase sub-image to obtain the second lower phase sub-image.

[0321] In one implementation, the electronic device uses the brightness value of each pixel in the first lower phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second lower phase gain compensation sub-matrix to obtain the second lower phase sub-image. In another implementation, the electronic device uses the brightness value of each pixel within the focus area of ​​the first lower phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the second lower phase gain compensation sub-matrix to obtain the second lower phase sub-image.

[0322] In a four-phase detection focusing scenario, the electronic device can first use the brightness compensation method corresponding to the left and right phase detection focusing scenario to obtain the second left phase sub-image and the second right phase sub-image. Then, the second left phase sub-image and the second right phase sub-image are combined into a phase image. Finally, the brightness compensation method corresponding to the upper and lower phase detection focusing scenario is used to obtain the second upper phase sub-image and the second lower phase sub-image.

[0323] Of course, in the scenario of four-phase detection focusing, the electronic device can first use the brightness compensation method corresponding to the scenario of upper and lower phase detection focusing to obtain the second upper phase sub-image and the second lower phase sub-image, and then combine the second upper phase sub-image and the second lower phase sub-image into a phase image, and then continue to use the brightness compensation method corresponding to the scenario of left and right phase detection focusing to obtain the second left phase sub-image and the second right phase sub-image.

[0324] S1008, the electronic device performs phase focusing based on the second phase image.

[0325] After compensating the first phase image using a phase gain compensation matrix to obtain a second phase image, the electronic device can determine the phase difference information based on the brightness-compensated second phase image, and then determine the defocus distance based on the phase difference information. The focusing motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, thereby achieving phase focusing.

[0326] In a left-right phase detection focusing scenario, the second phase image includes a second left phase sub-image and a second right phase sub-image. The phase difference information of the first movement direction can be determined using the second left phase sub-image and the second right phase sub-image. The first movement direction can be a horizontal direction (i.e., the left-right direction). Furthermore, the defocus distance is determined based on the phase difference information of the first movement direction.

[0327] In a scenario involving vertical phase detection autofocus, the second phase image includes a second upper phase sub-image and a second lower phase sub-image. The phase difference information in a second movement direction can be determined using the second upper phase sub-image and the second lower phase sub-image. The second movement direction can be a vertical direction (i.e., the up-down direction). Furthermore, the defocus distance is determined based on the phase difference information in the second movement direction.

[0328] In a four-phase detection autofocus scenario, the phase difference information in the first movement direction can be determined using the second left and second right phase sub-images, and the phase difference information in the second movement direction can be determined using the second upper and second lower phase sub-images. Furthermore, the defocus distance is determined jointly based on the phase difference information in the first and second movement directions.

[0329] It is understood that in the phase focusing process of this application embodiment, when obtaining the second coordinate of the focus point in the original image in step S1002, the second coordinate of the focus point can refer to the second coordinate of the focus point in the (N-1)th frame of the original image. When obtaining the first coordinate of the position of the optical image stabilization module at the target time in step S1005, it refers to the first coordinate of the position of the optical image stabilization module at the target time during the acquisition of the Nth frame of the original image. Since the position of the focus point in the (N-1)th frame of the original image is basically unchanged from that in the Nth frame of the original image, the second coordinate of the focus point in the (N-1)th frame of the original image can be used to determine the first coordinate of the position of the optical image stabilization module at the target time during the acquisition of the Nth frame of the original image. Furthermore, when compensating the first phase image using the phase gain compensation matrix in step S1007, it can also be the first phase image corresponding to the Nth frame of the original image that is compensated. In addition, when performing phase focusing based on the second phase image in step S1008, phase focusing can be performed on the (N+1)th frame of the original image. Here, N is an integer greater than 1.

[0330] In summary, this embodiment of the application, when the camera activates the optical image stabilization module for optical image stabilization, calculates a phase gain compensation matrix based on the first coordinates of the optical image stabilization module's position at the target time and multiple phase gain calibration matrices. This matrix is ​​then used to compensate the brightness of the first phase image captured by the phase detection pixels, resulting in more uniform pixel brightness values ​​in the second phase image obtained after brightness compensation. Consequently, when using the second phase image to determine phase difference information, the determined phase difference information is more accurate, allowing for precise calculation of the defocus distance corresponding to the phase difference information. This improves focusing stability and accuracy in optical image stabilization mode.

[0331] Taking a scenario with left and right phase detection autofocus as an example, Figure 14 This is a schematic diagram illustrating how, in an embodiment of this application, when the optical image stabilization module in the camera is activated and the optical displacement between the optical center of the lens and the optical center of the image sensor is not equal to 0, a phase gain compensation matrix is ​​used to compensate for the brightness of the first phase image.

[0332] like Figure 14As shown in (a), when the camera activates the optical image stabilization module for optical image stabilization, and the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 can be d, where d is not equal to 0, the first phase image can be acquired through the phase detection pixels in the image sensor 20.

[0333] When the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is not equal to zero, the acquired first phase image is as follows: Figure 14 As shown in (b), it includes a first left-phase sub-image acquired by the left-phase detection pixels and a first right-phase sub-image acquired by the right-phase detection pixels. Figure 14 In the first phase image shown in (b), the horizontal axis represents multiple image blocks in the first left phase sub-image and the first right phase sub-image, and the vertical axis represents the brightness value of each image block in the first left phase sub-image and the first right phase sub-image. The brightness value is the average of the brightness values ​​of all pixels in each image block.

[0334] The embodiments of this application can be adopted as follows: Figure 14 The phase gain compensation matrix shown in (c) is for... Figure 14 The first phase image shown in (b) is subjected to brightness compensation, and the resulting second phase image is as follows: Figure 14 As shown in (d) in the figure.

[0335] exist Figure 14 In the phase gain compensation matrix shown in (c), the phase gain compensation matrix includes a first left phase gain compensation sub-matrix and a first right phase gain compensation sub-matrix. The horizontal axis represents multiple image blocks, and the vertical axis represents the compensation coefficient corresponding to each image block in the first left phase gain compensation sub-matrix and the first right phase gain compensation sub-matrix.

[0336] exist Figure 14 In the second phase image shown in (d), the second phase image includes a second left phase sub-image and a second right phase sub-image. The horizontal axis represents multiple image blocks in the second left and second right phase sub-images, and the vertical axis represents the brightness value of each image block in the second left and second right phase sub-images.

[0337] Specifically, it can be adopted Figure 14 The first left-side phase gain compensation submatrix shown in (c) is for... Figure 14 The first left-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 14 The second left-phase sub-image shown in (d) is used; Figure 14 The first right phase gain compensation submatrix shown in (c) is for... Figure 14The first right-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 14 The second right-phase sub-image is shown in (d) in the image.

[0338] It can be seen that when the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is not zero, the brightness values ​​of the pixels in the first phase image acquired by the phase detection pixels are uneven. However, after using a phase gain compensation matrix to compensate the brightness of the first phase image, the brightness values ​​of the pixels in the resulting second phase image become more uniform. This allows for more accurate determination of phase difference information when using the second phase image, enabling precise calculation of the defocus distance corresponding to the phase difference, thereby improving focusing stability and accuracy in optical image stabilization mode.

[0339] Figure 15 This is a flowchart illustrating brightness compensation for a first phase image, provided as an embodiment of this application. (Refer to...) Figure 15 As shown, the electronic device may include a processor, memory, and a camera. The camera may include an image sensor, a driver chip in the optical image stabilization module, and a Hall sensor, etc. The Hall sensor is electrically connected to the driver chip in the optical image stabilization module, and the driver chip, image sensor, and memory in the optical image stabilization module are each electrically connected to the processor. For example, the image sensor and the processor can be electrically connected via a mobile industry processor interface (MIPI).

[0340] The processor can be a central processing unit or a coprocessor, and the memory can be electrically erasable programmable read-only memory (EEPROM).

[0341] A Hall sensor is used to collect Hall data, which indicates the coordinates of the optical image stabilization module's position after each movement. This allows the Hall sensor to acquire the initial coordinates of the optical image stabilization module's position at the target time. The Hall sensor then sends these initial coordinates to the driver chip within the optical image stabilization module, which in turn sends them to the processor.

[0342] The phase detection pixels in the image sensor can acquire a first phase image, and the image sensor can send the acquired first phase image to the processor.

[0343] The memory can store multiple phase gain calibration matrices, such as the first phase gain calibration matrix, the second phase gain calibration matrix, the third phase gain calibration matrix, the fourth phase gain calibration matrix, the fifth phase gain calibration matrix, and the sixth phase gain calibration matrix.

[0344] The processor can execute the brightness compensation algorithm in the phase-detection autofocus method provided in this application embodiment. Specifically, the processor can acquire a first phase image from the image sensor, acquire the first coordinates of the position of the optical image stabilization module at the target time from the driver chip in the optical image stabilization module, and acquire multiple phase gain calibration matrices, including a first phase gain calibration matrix, a second phase gain calibration matrix, a third phase gain calibration matrix, a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix, from the memory. This allows the processor to execute the brightness compensation algorithm to determine the phase gain compensation matrix based on the first coordinates of the position of the optical image stabilization module at the target time and the multiple phase gain calibration matrices, and then use the phase gain compensation matrix to compensate the first phase image to obtain a second phase image.

[0345] The above combination Figures 9 to 15 The phase focusing method provided in the embodiments of this application has been described. The apparatus for performing the above method provided in the embodiments of this application is described below. Figure 16 As shown, Figure 16 This is a schematic diagram of a phase focusing device provided in an embodiment of this application. The phase focusing device may be an electronic device as described in this application embodiment, or a chip or chip system within an electronic device.

[0346] like Figure 16 As shown, the phase focusing device 1600 may include a processing unit 1601. The processing unit 1601 is used to support the phase focusing device 1600 in performing the above-described processing steps.

[0347] Specifically, processing unit 1601 is used to acquire a first phase image captured by phase detection pixels when the optical image stabilization module is activated; processing unit 1601 is used to acquire the first coordinates of the position of the optical image stabilization module at a target time, where the target time is the time when the focus point in the original image captured by the image sensor is captured, and the original image corresponds to the first phase image; processing unit 1601 is used to determine a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, where the relative positions of the optical center of the lens and the optical center of the image sensor are different during calibration; processing unit 1601 is used to compensate the first phase image using the phase gain compensation matrix to obtain a second phase image; processing unit 1601 is used to perform phase focusing based on the second phase image.

[0348] In one possible implementation, the phase focusing device 1600 further includes a storage unit 1602. The storage unit 1602 and the processing unit 1601 are connected via a line. The storage unit 1602 may include one or more memories, which can be devices in one or more devices or circuits used to store programs or data. The storage unit 1602 can exist independently and be connected to the processing unit 1601 via a communication bus. Alternatively, the storage unit 1602 can be integrated with the processing unit 1601.

[0349] Storage unit 1602 can store computer-executable instructions for methods in an electronic device, so that processing unit 1601 executes the methods in the above embodiments. Storage unit 1602 can be a register, cache, or random access memory (RAM), etc., and storage unit 1602 can be integrated with processing unit 1601. Storage unit 1602 can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, and storage unit 1602 can be independent of processing unit 1601.

[0350] Figure 17 This is a schematic diagram of a chip structure provided in an embodiment of this application. Figure 17 As shown, chip 1700 includes one or more (including two) processors 1701, communication lines 1702 and communication interfaces 1703. Optionally, chip 1700 also includes memory 1704.

[0351] In some implementations, memory 1704 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof.

[0352] The methods described in the embodiments of this application can be applied to, or implemented by, processor 1701. Processor 1701 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the hardware of processor 1701 or by instructions in software form. Processor 1701 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. Processor 1701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0353] The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory. This storage medium is located in memory 1704, and processor 1701 reads information from memory 1704 and, in conjunction with its hardware, completes the steps of the above method.

[0354] The processor 1701, memory 1704 and communication interface 1703 can communicate with each other via communication line 1702.

[0355] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.

[0356] This application also provides a computer program product comprising one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. For example, available media may include magnetic media (e.g., floppy disk, hard disk, or magnetic tape), optical media (e.g., digital versatile disc (DVD)), or semiconductor media (e.g., solid-state disk (SSD)).

[0357] This application provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to perform the phase focusing method described above.

[0358] This application provides a chip. The chip includes a processor, which is used to call a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those in the related embodiments described above, and will not be repeated here.

[0359] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program or instructions. When the computer program or instructions are executed by a processor, they implement the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0360] As one possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM, or other optical disc storage; computer-readable media may include disk storage or other disk storage devices. Furthermore, any connecting cable may also be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. As used herein, disks and optical discs include optical discs (CD), laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0361] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0362] The above specific embodiments further illustrate the purpose, technical solution and beneficial effects of this application. It should be understood that the above are only specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of this application should be included within the scope of protection of this application.

Claims

1. A phase-detection focusing method, characterized in that, The method is applied to an electronic device, the electronic device including a camera, the camera including an optical image stabilization module, a lens, and an image sensor, the image sensor including multiple phase detection pixels; the method includes: When the optical image stabilization module is activated, the electronic device acquires the first phase image captured by the phase detection pixel; The electronic device acquires the first coordinates of the position of the optical image stabilization module at the target time; the target time is the time when the focus point in the original image acquired by the image sensor is located at the time of acquisition, and the original image corresponds to the first phase image; The electronic device determines a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices. The relative positions of the optical center of the lens and the optical center of the image sensor differ during calibration. The phase gain calibration matrix is ​​a compensation coefficient matrix predetermined based on the ratio of the brightness value of each image block to the maximum brightness value in the phase test image under calibration conditions. The phase test image is a phase image acquired when the lens's focus position is a preset focus position and the relative position of the optical center of the lens and the optical center of the image sensor is a preset position. The maximum brightness value is the maximum value among the brightness values ​​of each image block. The electronic device uses the phase gain compensation matrix to compensate the first phase image to obtain a second phase image; The electronic device performs phase focusing based on the second phase image.

2. The method according to claim 1, characterized in that, Before the electronic device acquires the first coordinates of the position of the optical image stabilization module at the target time, it further includes: The electronic device acquires the second coordinates of the focus point in the original image; The electronic device determines the target duration based on the second coordinates; The electronic device determines the target time by summing the initial time when it begins acquiring the original image with the target duration.

3. The method according to claim 2, characterized in that, The electronic device determines the target duration based on the second coordinates, including: The electronic device calculates the target duration using the following formula: Among them, T lag The target duration is EIT, the exposure duration of the original image is Y. n Let H be the coordinate component along the height direction of the original image in the second coordinate system, where H is the height of the original image, and RowNum is the total number of rows in the photosensitive unit array of the image sensor. time The reading time for reading image data generated by a row of photosensitive units in the photosensitive unit array.

4. The method according to claim 1, characterized in that, The movable direction of the optical image stabilization module includes a first movable direction and a second movable direction, the first movable direction and the second movable direction are perpendicular to each other, and both the first movable direction and the second movable direction are perpendicular to the optical axis of the camera; the first movable direction includes a first direction and a second direction that are opposite to each other, and the second movable direction includes a third direction and a fourth direction that are opposite to each other; The plurality of phase gain calibration matrices include a first type of phase gain calibration matrix and / or a second type of phase gain calibration matrix; the first type of phase gain calibration matrix includes a first phase gain calibration matrix, a second phase gain calibration matrix, and a third phase gain calibration matrix; the second type of phase gain calibration matrix includes a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix; Wherein, during the calibration of the first phase gain calibration matrix and the fourth phase gain calibration matrix, the optical center of the lens coincides with the optical center of the image sensor along the optical axis of the camera; When calibrating the second phase gain calibration matrix, the optical center of the lens is offset by a first distance relative to the optical center of the image sensor along the first direction; the first distance is the maximum distance that the lens can move along the first direction. During calibration, the optical center of the lens is offset by a second distance relative to the optical center of the image sensor along the second direction; the second distance is the maximum distance the lens can move along the second direction. During calibration, the optical center of the lens is offset by a third distance relative to the optical center of the image sensor along the third direction; the third distance is the maximum distance the lens can move along the third direction. During calibration, the optical center of the lens is offset by a fourth distance relative to the optical center of the image sensor along the fourth direction; the fourth distance is the maximum distance the lens can move along the fourth direction.

5. The method according to claim 4, characterized in that, The first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix; the second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix; the third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix; the first coordinate includes a first coordinate component along the first movement direction; The electronic device determines a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, including: The electronic device determines the first left phase gain compensation submatrix based on the first coordinate component, the first left phase gain calibration submatrix, the second left phase gain calibration submatrix, and the third left phase gain calibration submatrix; The electronic device determines the first right phase gain compensation submatrix based on the first coordinate component, the first right phase gain calibration submatrix, the second right phase gain calibration submatrix, and the third right phase gain calibration submatrix.

6. The method according to claim 5, characterized in that, The electronic device determines a first left-phase gain compensation sub-matrix based on the first coordinate component, the first left-phase gain calibration sub-matrix, the second left-phase gain calibration sub-matrix, and the third left-phase gain calibration sub-matrix, including: The electronic device calculates the first left-phase gain compensation sub-matrix using the following formula: Among them, GainMap L Let Q be the first left-phase gain compensation submatrix. 0L Q is the first left phase gain calibration submatrix. 1L Q is the second left phase gain calibration submatrix. 2L Let X be the third left-phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This refers to the first coordinate component; Accordingly, the electronic device determines a first right-phase gain compensation sub-matrix based on the first coordinate component, the first right-phase gain calibration sub-matrix, the second right-phase gain calibration sub-matrix, and the third right-phase gain calibration sub-matrix, including: The electronic device calculates the first right-phase gain compensation sub-matrix using the following formula: Among them, GainMap R Let Q be the first right phase gain compensation submatrix. 0R Q is the first right phase gain calibration submatrix. 1R Q is the second right phase gain calibration submatrix. 2R Let X be the third right phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This is the first coordinate component.

7. The method according to claim 4, characterized in that, The fourth phase gain calibration matrix includes a first upper phase gain calibration sub-matrix and a first lower phase gain calibration sub-matrix; the fifth phase gain calibration matrix includes a second upper phase gain calibration sub-matrix and a second lower phase gain calibration sub-matrix; the sixth phase gain calibration matrix includes a third upper phase gain calibration sub-matrix and a third lower phase gain calibration sub-matrix; the first coordinate includes a second coordinate component along the second movement direction; The electronic device determines a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, including: The electronic device determines the first upper phase gain compensation submatrix based on the second coordinate component, the first upper phase gain calibration submatrix, the second upper phase gain calibration submatrix, and the third upper phase gain calibration submatrix; The electronic device determines the first lower phase gain compensation submatrix based on the second coordinate component, the first lower phase gain calibration submatrix, the second lower phase gain calibration submatrix, and the third lower phase gain calibration submatrix.

8. The method according to claim 7, characterized in that, The electronic device determines a first upper phase gain compensation submatrix based on the second coordinate component, the first upper phase gain calibration submatrix, the second upper phase gain calibration submatrix, and the third upper phase gain calibration submatrix, including: The electronic device calculates the first upper phase gain compensation sub-matrix using the following formula: Among them, GainMap T Let Q be the first upper phase gain compensation submatrix. 0T Q is the first upper phase gain calibration submatrix. 3T Q is the second upper phase gain calibration submatrix. 4T Y is the third upper phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS Y This is the second coordinate component; Accordingly, the electronic device determines a first lower phase gain compensation sub-matrix based on the second coordinate component, the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix, including: The electronic device calculates the first lower phase gain compensation sub-matrix using the following formula: Among them, GainMap B Let Q be the first lower phase gain compensation submatrix. 0B Let Q be the first lower phase gain calibration submatrix. 3B Q is the second lower phase gain calibration submatrix. 4B Y is the third lower phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS Y This is the second coordinate component.

9. The method according to any one of claims 1 to 8, characterized in that, The phase gain compensation matrix includes a first phase gain compensation submatrix and a second phase gain compensation submatrix; the electronic device uses the phase gain compensation matrix to compensate the first phase image to obtain a second phase image, including: The electronic device splits the first phase image into a first phase sub-image and a second phase sub-image; The electronic device adjusts the size of the first phase gain compensation sub-matrix to obtain a third phase gain compensation sub-matrix; the size of the third phase gain compensation sub-matrix is ​​equal to the size of the first phase sub-image. The electronic device uses the third phase gain compensation sub-matrix to compensate the first phase sub-image to obtain the third phase sub-image; The electronic device adjusts the size of the second phase gain compensation sub-matrix to obtain a fourth phase gain compensation sub-matrix; the size of the fourth phase gain compensation sub-matrix is ​​equal to the size of the second phase sub-image. The electronic device uses the fourth phase gain compensation sub-matrix to compensate the second phase sub-image to obtain the fourth phase sub-image; Wherein, the first phase sub-image is the first left phase sub-image, the second phase sub-image is the first right phase sub-image, the third phase sub-image is the second left phase sub-image, and the fourth phase sub-image is the second right phase sub-image; the first phase gain compensation sub-matrix is ​​the first left phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​the first right phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​the second left phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​the second right phase gain compensation sub-matrix; Alternatively, the first phase sub-image is a first upper phase sub-image, the second phase sub-image is a first lower phase sub-image, the third phase sub-image is a second upper phase sub-image, and the fourth phase sub-image is a second lower phase sub-image; the first phase gain compensation sub-matrix is ​​a first upper phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​a first lower phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​a second upper phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​a second lower phase gain compensation sub-matrix.

10. The method according to claim 9, characterized in that, The electronic device uses the third phase gain compensation sub-matrix to compensate the first phase sub-image to obtain a third phase sub-image, including: The electronic device uses the brightness value of each pixel in the first phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image; Accordingly, the electronic device uses the fourth phase gain compensation sub-matrix to compensate the second phase sub-image to obtain the fourth phase sub-image, including: The electronic device uses the brightness value of each pixel in the second phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image.

11. The method according to claim 9, characterized in that, The electronic device uses the third phase gain compensation sub-matrix to compensate the first phase sub-image to obtain a third phase sub-image, including: The electronic device uses the brightness value of each pixel in the focus area of ​​the first phase sub-image, multiplied by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix, to obtain the third phase sub-image; Accordingly, the electronic device uses the fourth phase gain compensation sub-matrix to compensate the second phase sub-image to obtain the fourth phase sub-image, including: The electronic device uses the brightness value of each pixel in the focus area of ​​the second phase sub-image, multiplied by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix, to obtain the fourth phase sub-image.

12. The method according to any one of claims 1 to 8, characterized in that, Before the electronic device determines the phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, the method further includes: The electronic device acquires the pre-calibrated plurality of phase gain calibration matrices; each of the phase gain calibration matrices includes a first phase gain calibration submatrix and a second phase gain calibration submatrix; Wherein, each compensation coefficient in the first phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each first image block in the first phase test sub-image; each compensation coefficient in the second phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each second image block in the second phase test sub-image; the maximum brightness value is the maximum value among the brightness values ​​of each first image block and each second image block; The first phase test sub-image and the second phase test sub-image are obtained by splitting the phase test image; the phase test image is a phase image acquired when the focus position of the lens is a preset focus position and the relative position between the optical center of the lens and the optical center of the image sensor is a preset position.

13. The method according to claim 12, characterized in that, Each compensation coefficient in the first phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each first image block in the first phase test sub-image; Each compensation coefficient in the second phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each second image block in the second phase test sub-image.

14. An electronic device, characterized in that, It includes a memory and a processor, the memory being used to store a computer program, and the processor being used to invoke the computer program to perform the phase focusing method as described in any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions that, when executed, implement the phase focusing method as described in any one of claims 1 to 13.

16. A computer program product, characterized in that, Includes a computer program that, when run, causes a computer to perform the phase focusing method as described in any one of claims 1 to 13.

Images