Phase focusing method, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a phase focusing method, electronic equipment, a storage medium and a program product, and is applied to the technical field of electronics. According to the method, under the condition that an optical anti-shake module is started, a first phase image collected by a phase detection pixel and a first coordinate of the position where the optical anti-shake module is located at a target moment are obtained, and a phase gain compensation matrix is determined according to the first coordinate and a plurality of phase gain calibration matrixes; the relative positions of the optical center of the lens and the optical center of the image sensor are different when the plurality of phase gain calibration matrixes are calibrated, the phase gain compensation matrix is adopted to compensate the first phase image, and phase focusing is carried out based on a second phase image obtained after compensation. Therefore, according to the embodiment of the invention, the brightness value of the pixel in the second phase image obtained after compensation can be more uniform, so that the phase difference information determined based on the second phase image is more accurate, and the focusing stability and the focusing accuracy in an optical anti-shake mode are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronics, and particularly relates to a phase focusing method, an electronic device, a storage medium and a program product. BACKGROUND

[0002] With the continuous development of electronic technology, electronic devices such as mobile phones and tablet computers have become common tools in people's daily life and work. At present, some electronic devices are provided with cameras, and the cameras provide users with photographing or video recording functions.

[0003] In some electronic devices, the cameras provided therein can support phase detection autofocus (PDAF) and optical image stabilization (OIS).

[0004] However, in the case where the camera starts the optical image stabilization module to perform optical image stabilization, there is a problem of low focusing stability and focusing accuracy. SUMMARY

[0005] Embodiments of the present application provide a phase focusing method, an electronic device, a storage medium and a program product, which can improve the focusing stability and focusing accuracy in the case where the optical image stabilization module is started to perform optical image stabilization.

[0006] In a first aspect, embodiments of the present application provide a phase focusing method applied to an electronic device, the electronic device comprising a camera, the camera comprising an optical image stabilization module, a lens and an image sensor, the image sensor comprising a plurality of phase detection pixels. The phase focusing method comprises: in the case where the optical image stabilization module is started, the electronic device acquires a first phase image collected by the phase detection pixels; the electronic device acquires a first coordinate of a position of the optical image stabilization module at a target time, the target time being a time at which a focus point in an original image collected by the image sensor is located at the time of collection, the original image corresponding to the first phase image; the electronic device determines a phase gain compensation matrix according to the first coordinate and a plurality of phase gain calibration matrices, the plurality of phase gain calibration matrices being different in relative positions of an optical center of the lens and an optical center of the image sensor at the time of calibration; the electronic device compensates the first phase image by using the phase gain compensation matrix to obtain a second phase image; and the electronic device performs phase focusing based on the second phase image.

[0007] In this way, in a case where the camera starts the optical anti-shake module to perform optical anti-shake, the phase gain compensation matrix is calculated according to the first coordinate of the position of the optical anti-shake module at the target moment and the plurality of phase gain calibration matrices, so as to perform brightness compensation on the first phase image collected by the phase detection pixel, which can make the brightness values of the pixels in the second phase image obtained after brightness compensation more uniform. Therefore, when the second phase image is used to determine the phase difference information, the determined phase difference information can be more accurate, so as to accurately calculate the defocus distance corresponding to the phase difference information, thereby improving the focusing stability and the focusing accuracy in the optical anti-shake mode.

[0008] In a possible implementation, before the electronic device obtains the first coordinate of the position of the optical anti-shake module at the target moment, the method further includes: obtaining, by the electronic device, a second coordinate of a focus point in the original image; determining, by the electronic device, a target time length according to the second coordinate; and determining, by the electronic device, a target moment as a sum of an initial moment when the original image starts to be collected and the target time length. In this way, the target moment at which the focus point in the original image is collected is estimated according to the second coordinate of the focus point in the original image, so that the estimated target moment can be more accurate.

[0009] In a possible implementation, the electronic device determines the target time length according to the second coordinate, and the method includes: calculating, by the electronic device, the target time length according to the following formula:

[0010]

[0011] wherein, T lag is the target time length, EIT is an exposure time length of the original image, Y n is a coordinate component in a height direction of the original image in the second coordinate, H is a height of the original image, RowNum is a total number of rows of the photosensitive unit array in the image sensor, H time is a reading time length when image data generated by a row of photosensitive units in the photosensitive unit array is read. In this way, the target moment at which the focus point in the original image is collected can be more accurately estimated.

[0012] In a possible implementation, the movable directions of the optical anti-shake module include a first movable direction and a second movable direction, the first movable direction and the second movable direction are perpendicular to each other, and the first movable direction and the second movable direction are both perpendicular to the optical axis direction of the camera; the first movable direction includes a first direction and a second direction that are opposite to each other, and the second movable direction includes a third direction and a fourth direction that are opposite to each other. The plurality of phase gain calibration matrices include a first type of phase gain calibration matrix and / or a second type of phase gain calibration matrix; the first type of phase gain calibration matrix includes a first phase gain calibration matrix, a second phase gain calibration matrix, and a third phase gain calibration matrix; and the second type of phase gain calibration matrix includes a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix. In the calibration of the first phase gain calibration matrix and the fourth phase gain calibration matrix, the optical center of the lens and the optical center of the image sensor coincide along the optical axis direction of the camera; in the calibration of the second phase gain calibration matrix, the optical center of the lens is offset from the optical center of the image sensor by a first distance along the first direction, and the first distance is the maximum distance that the lens can move along the first direction; in the calibration of the third phase gain calibration matrix, the optical center of the lens is offset from the optical center of the image sensor by a second distance along the second direction, and the second distance is the maximum distance that the lens can move along the second direction; in the calibration of the fifth phase gain calibration matrix, the optical center of the lens is offset from the optical center of the image sensor by a third distance along the third direction, and the third distance is the maximum distance that the lens can move along the third direction; and in the calibration of the sixth phase gain calibration matrix, the optical center of the lens is offset from the optical center of the image sensor by a fourth distance along the fourth direction, and the fourth distance is the maximum distance that the lens can move along the fourth direction. In this way, the plurality of phase gain calibration matrices can be used to calculate a phase gain compensation matrix, which can accurately compensate the first phase image when the optical anti-shake module moves to any position.

[0013] In a possible implementation, the first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix; the second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix; and the third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix. The first coordinate includes a first coordinate component in the first movement direction. The electronic device determines the phase gain compensation matrix according to the first coordinate and the plurality of phase gain calibration matrices, including: determining, by the electronic device, a first left phase gain compensation sub-matrix according to the first coordinate component, the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix, and the third left phase gain calibration sub-matrix; and determining, by the electronic device, a first right phase gain compensation sub-matrix according to the first coordinate component, the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix, and the third right phase gain calibration sub-matrix. In this way, in the scene of left-right phase detection focusing, the phase gain compensation matrix can be accurately calculated to perform brightness compensation on the first phase image.

[0014] In a possible implementation, the electronic device determines the first left phase gain compensation sub-matrix according to the first coordinate component, the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix, and the third left phase gain calibration sub-matrix, including: calculating, by the electronic device, the first left phase gain compensation sub-matrix according to the following formula:

[0015]

[0016] wherein GainMap L is the first left phase gain compensation sub-matrix, Q 0L is the first left phase gain calibration sub-matrix, Q 1L is the second left phase gain calibration sub-matrix, Q 2L is the third left phase gain calibration sub-matrix, X lim1 is the first distance, X lim2 is the second distance, OIS X is the first coordinate component. Correspondingly, the electronic device determines the first right phase gain compensation sub-matrix according to the first coordinate component, the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix, and the third right phase gain calibration sub-matrix, including: calculating, by the electronic device, the first right phase gain compensation sub-matrix according to the following formula:

[0017]

[0018] wherein GainMap R is the first right phase gain compensation sub-matrix, Q 0R is the first right phase gain calibration sub-matrix, Q 1RQ is a third right phase gain calibration sub-matrix 2R X is a third right phase gain calibration sub-matrix lim1 X is a first distance lim2 OIS is a second distance X X is a first coordinate component. In this way, in a left-right phase detection focusing scene, the phase gain compensation matrix can be accurately calculated to perform brightness compensation on the first phase image.

[0019] In a possible implementation, the fourth phase gain calibration matrix includes a first upper phase gain calibration sub-matrix and a first lower phase gain calibration sub-matrix; the fifth phase gain calibration matrix includes a second upper phase gain calibration sub-matrix and a second lower phase gain calibration sub-matrix; the sixth phase gain calibration matrix includes a third upper phase gain calibration sub-matrix and a third lower phase gain calibration sub-matrix; the first coordinate includes a second coordinate component in the second moving direction. The electronic device determines the phase gain compensation matrix according to the first coordinate and the plurality of phase gain calibration matrices, including: the electronic device determines a first upper phase gain compensation sub-matrix according to the second coordinate component, the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix; and the electronic device determines a first lower phase gain compensation sub-matrix according to the second coordinate component, the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix. In this way, in an up-down phase detection focusing scene, the phase gain compensation matrix can be accurately calculated to perform brightness compensation on the first phase image.

[0020] In a possible implementation, the electronic device determines the first upper phase gain compensation sub-matrix according to the second coordinate component, the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix, including: the electronic device calculates the first upper phase gain compensation sub-matrix through the following formula:

[0021]

[0022] GainMap T Q is a first upper phase gain compensation sub-matrix 0T Q is a first upper phase gain calibration sub-matrix 3T Q is a second upper phase gain calibration sub-matrix 4T Y is a third upper phase gain calibration sub-matrix lim1 Y is a third distance lim2 OIS is a fourth distance Yis a second coordinate component. Correspondingly, the electronic device determines the first down phase gain compensation sub-matrix according to the second coordinate component, the first down phase gain calibration sub-matrix, the second down phase gain calibration sub-matrix, and the third down phase gain calibration sub-matrix, including: the electronic device calculates the first down phase gain compensation sub-matrix through the following formula:

[0023]

[0024] wherein, GainMap B is the first down phase gain compensation sub-matrix, Q 0B is the first down phase gain calibration sub-matrix, Q 3B is the second down phase gain calibration sub-matrix, Q 4B is the third down phase gain calibration sub-matrix, Y lim1 is the third distance, Y lim2 is the fourth distance, OIS Y is the second coordinate component. In this way, in the scene of up-down phase detection focusing, the phase gain compensation matrix can be accurately calculated to perform brightness compensation on the first phase image.

[0025] In a possible implementation, the phase gain compensation matrix comprises a first phase gain compensation sub-matrix and a second phase gain compensation sub-matrix. The electronic device compensates the first phase image by using the phase gain compensation matrix to obtain a second phase image, comprising: the electronic device splits the first phase image into a first phase sub-image and a second phase sub-image; the electronic device adjusts the size of the first phase gain compensation sub-matrix to obtain a third phase gain compensation sub-matrix, the size of the third phase gain compensation sub-matrix being equal to the size of the first phase sub-image; the electronic device compensates the first phase sub-image by using the third phase gain compensation sub-matrix to obtain a third phase sub-image; the electronic device adjusts the size of the second phase gain compensation sub-matrix to obtain a fourth phase gain compensation sub-matrix, the size of the fourth phase gain compensation sub-matrix being equal to the size of the second phase sub-image; and the electronic device compensates the second phase sub-image by using the fourth phase gain compensation sub-matrix to obtain a fourth phase sub-image. The first phase sub-image is a first left phase sub-image, the second phase sub-image is a first right phase sub-image, the third phase sub-image is a second left phase sub-image, and the fourth phase sub-image is a second right phase sub-image; the first phase gain compensation sub-matrix is a first left phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is a first right phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is a second left phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is a second right phase gain compensation sub-matrix. Alternatively, the first phase sub-image is a first upper phase sub-image, the second phase sub-image is a first lower phase sub-image, the third phase sub-image is a second upper phase sub-image, and the fourth phase sub-image is a second lower phase sub-image; the first phase gain compensation sub-matrix is a first upper phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is a first lower phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is a second upper phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is a second lower phase gain compensation sub-matrix. In this way, in a left-right phase detection focusing scenario, the first left phase sub-image and the first right phase sub-image can be compensated for brightness respectively; and in an up-down phase detection focusing scenario, the first upper phase sub-image and the first lower phase sub-image can be compensated for brightness respectively.

[0026] In a possible implementation, the electronic device compensates the first phase sub-image by using a third phase gain compensation sub-matrix to obtain a third phase sub-image, including: the electronic device multiplies the brightness value of each pixel in the first phase sub-image by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image. Correspondingly, the electronic device compensates the second phase sub-image by using a fourth phase gain compensation sub-matrix to obtain a fourth phase sub-image, including: the electronic device multiplies the brightness value of each pixel in the second phase sub-image by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image. In this way, the product of the compensation coefficient and the brightness value can be used to realize the brightness compensation of the first phase image, and the compensation manner is relatively simple.

[0027] In a possible implementation, the electronic device compensates the first phase sub-image by using a third phase gain compensation sub-matrix to obtain a third phase sub-image, including: the electronic device multiplies the brightness value of each pixel in the first phase sub-image by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image. Correspondingly, the electronic device compensates the second phase sub-image by using a fourth phase gain compensation sub-matrix to obtain a fourth phase sub-image, including: the electronic device multiplies the brightness value of each pixel in the second phase sub-image by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image. In this way, the product of the compensation coefficient and the brightness value can be used to realize the brightness compensation of the first phase image, and the compensation manner is relatively simple.

[0028] In a possible implementation, before the electronic device determines the phase gain compensation matrix according to the first coordinate and the plurality of phase gain calibration matrices, the method further includes: obtaining, by the electronic device, the plurality of phase gain calibration matrices pre-calibrated, each of the plurality of phase gain calibration matrices including a first phase gain calibration sub-matrix and a second phase gain calibration sub-matrix. Each compensation coefficient in the first phase gain calibration sub-matrix is calculated according to a maximum luminance value and a luminance value of each first image patch in the first phase test sub-image. Each compensation coefficient in the second phase gain calibration sub-matrix is calculated according to the maximum luminance value and a luminance value of each second image patch in the second phase test sub-image. The maximum luminance value is a maximum value of the luminance value of each first image patch and the luminance value of each second image patch. The first phase test sub-image and the second phase test sub-image are obtained by splitting the phase test image. The phase test image is a phase image collected when the focusing position of the lens is a preset focusing position and the relative position of the optical center of the lens and the optical center of the image sensor is a preset position. In this way, the plurality of phase gain calibration matrices can be pre-calibrated to quickly calculate the phase gain compensation matrix, thereby improving the speed of luminance compensation on the first phase image and improving the speed of phase focusing.

[0029] In a possible implementation, each compensation coefficient in the first phase gain calibration sub-matrix is a ratio of the maximum luminance value to the luminance value of each first image patch in the first phase test sub-image. Each compensation coefficient in the second phase gain calibration sub-matrix is a ratio of the maximum luminance value to the luminance value of each second image patch in the second phase test sub-image. In this way, the calculation of the phase gain calibration matrix is relatively simple.

[0030] In a second aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, the memory is configured to store a computer program, and the processor is configured to invoke the computer program to execute the phase focusing method described above.

[0031] In a third aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program or instructions, when the computer program or instructions are executed, the phase focusing method described above is implemented.

[0032] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, when the computer program is executed, the computer program causes a computer to execute the phase focusing method described above.

[0033] The possible implementation of the second aspect to the fourth aspect has similar effects to the first aspect and the possible design of the first aspect, and will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A structure diagram of a camera provided for an embodiment of the present application is shown in FIG. 1.

[0035] Figure 2 A structure diagram of a first image sensor in a camera provided for an embodiment of the present application is shown in FIG. 2.

[0036] Figure 3 A structure diagram of a second image sensor in a camera provided for an embodiment of the present application is shown in FIG. 3.

[0037] Figure 4 A structure diagram of a third image sensor in a camera provided for an embodiment of the present application is shown in FIG. 4.

[0038] Figure 5 A diagram for a related art in which, when an optical image stabilization module in a camera is started, and an optical displacement between an optical center of a lens and an optical center of an image sensor is equal to 0, a first phase gain calibration matrix is used to perform brightness compensation on a first phase image is shown in FIG. 5.

[0039] Figure 6 A diagram for a related art in which, when an optical image stabilization module in a camera is started, and an optical displacement between an optical center of a lens and an optical center of an image sensor is not equal to 0, a first phase gain calibration matrix is used to perform brightness compensation on a first phase image is shown in FIG. 6.

[0040] Figure 7 A hardware system structure diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 7.

[0041] Figure 8 A software system structure diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 8.

[0042] Figure 9 An interface diagram of an application scenario provided for an embodiment of the present application is shown in FIG. 9.

[0043] Figure 10 A flowchart of a phase focusing method provided for an embodiment of the present application is shown in FIG. 10.

[0044] Figure 11 A displacement diagram of an optical image stabilization module in a process of collecting an original image provided for an embodiment of the present application is shown in FIG. 11.

[0045] Figure 12 A diagram for a case in which an optical center of a lens is offset relative to an optical center of an image sensor in a calibration process of a plurality of phase gain calibration matrices provided for an embodiment of the present application is shown in FIG. 12.

[0046] Figure 13 A flowchart of a calibration process of a first phase gain calibration matrix provided for an embodiment of the present application is shown in FIG. 13.

[0047] Figure 14 A schematic diagram of the embodiment of the present application for brightness compensation of the first phase image by using a phase gain compensation matrix for the case that the optical displacement between the optical center of the lens and the optical center of the image sensor is not equal to 0 in starting the optical image stabilization module of the camera;

[0048] Figure 15 A flow chart of the embodiment of the present application for brightness compensation of the first phase image;

[0049] Figure 16 A structural schematic diagram of a phase focusing device provided by the embodiment of the present application;

[0050] Figure 17 A structural schematic diagram of a chip provided by the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and effects are distinguished by using "first", "second", etc. For example, the first chip and the second chip are only used to distinguish different chips, and do not limit the sequence. Those skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution sequence, and "first", "second", etc. also do not necessarily mean different.

[0052] It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0053] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The association relationship of the associated objects is described by "and / or", which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c, can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0054] At present, some electronic devices are provided with cameras, and the cameras in the electronic devices can be used for photographing or video recording during use of the electronic devices by users.

[0055] When a user holds the electronic device to take a picture, the electronic device may vibrate to a certain extent, resulting in imaging blur of the image collected by the camera. In order to enhance the stability of the image collected by the camera, an optical image stabilization technology is introduced into the camera, that is, an optical image stabilization module is arranged in the camera.

[0056] In addition, in order to improve the clarity of the image collected by the camera, the electronic device needs to perform automatic focus (AF). The automatic focus mode can include phase focus, which refers to obtaining phase difference (PD) information by using a phase image collected by a phase detection pixel in the camera, and then calculating a defocus distance based on the phase difference information. A focus motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, so as to realize phase focus. The defocus distance refers to the distance between the current position of the lens and the position of the lens when the lens is in focus, which can also be understood as the distance between the focal point and the imaging surface.

[0057] As shown in Figure 1 , the camera can include a lens 10, an image sensor 20, a lens driving assembly, a support 50, and a circuit board 60.

[0058] The lens 10 and the image sensor 20 are sequentially arranged along the optical axis direction of the camera. The lens 10 can include one or more optical lenses, which can be convex lenses or concave lenses. When the lens 10 includes multiple optical lenses, the multiple optical lenses can be sequentially stacked along the optical axis direction of the camera. The image sensor 20 can also be referred to as a camera sensor, and the image sensor 20 is fixed on the circuit board 60 and electrically connected to the circuit board 60.

[0059] The lens driving assembly is used to drive the lens 10 to move to realize automatic focus and / or optical image stabilization. The lens driving assembly can include an optical image stabilization module and a focus driving module.

[0060] The optical image stabilization module can include an optical image stabilization motor 30 for driving the lens 10 to move to achieve optical image stabilization. The optical image stabilization technology is to correct the "optical axis deviation" through the floating lens of the lens. The principle is that the micro vibration is detected by the gyroscope sensor or the acceleration sensor in the electronic device, and then the detected vibration data is sent to the microprocessor, such as the driving chip electrically connected with the optical image stabilization motor 30. The microprocessor calculates the displacement amount to be compensated according to the vibration data, and then drives the optical image stabilization motor 30 to move through the displacement amount, thereby driving the lens 10 to move along the first moving direction and / or the second moving direction, to compensate for the shaking direction and shaking displacement of the lens 10, thereby effectively improving the imaging blur problem caused by the shaking of the electronic device. The first moving direction and the second moving direction are perpendicular to each other, and both the first moving direction and the second moving direction are perpendicular to the optical axis direction of the camera.

[0061] Therefore, the optical image stabilization motor 30 can drive the lens 10 to move along the direction perpendicular to the optical axis direction of the camera (i.e., along the first moving direction and / or the second moving direction) to perform optical image stabilization. It should be understood that, Figure 1 The position of the optical image stabilization motor 30 is only schematically shown in the figure, and the specific position of the optical image stabilization motor 30 is not limited in the embodiments of the present application.

[0062] The optical image stabilization motor 30 can be a voice coil motor (VCM), a shape memory alloy (SMA) motor, a stepping motor, a piezoelectric motor, etc. It should be understood that the specific structure of the optical image stabilization motor 30 can be designed and selected according to the selected driving mode, and the embodiments of the present application do not limit this.

[0063] The focusing driving module can include a focusing motor 40, which can be arranged on the bracket 50 and used to drive the lens 10 to move to achieve automatic focusing. Specifically, the focusing motor 40 can drive the lens 10 to move along the optical axis direction of the camera to perform automatic focusing. It should be understood that, Figure 1 The position of the focusing motor 40 is only schematically shown in the figure, and the specific position of the focusing motor 40 is not limited in the embodiments of the present application.

[0064] The focusing motor 40 can be a voice coil motor, a shape memory alloy motor, a stepping motor, a piezoelectric motor, etc. It should be understood that the specific structure of the focusing motor 40 can be designed and selected according to the selected driving mode, and the embodiments of the present application do not limit this.

[0065] In some embodiments, asFigure 1 As shown, the optical image stabilization motor 30 and the focusing motor 40 can be two independent components, which respectively drive the lens 10 for optical image stabilization and autofocus. In other embodiments, the optical image stabilization motor 30 and the focusing motor 40 can also be the same component, which can drive the lens 10 for both optical image stabilization and autofocus.

[0066] In this embodiment, the image sensor 20 includes a microlens array, a filter unit array, and a photosensitive unit array. The microlens array and the filter unit array are both located between the lens 10 and the photosensitive unit array, and the filter unit array is located between the microlens array and the photosensitive unit array.

[0067] like Figures 2 to 4 As shown, the microlens array includes multiple microlenses (MLs) 21 arranged in an array. Each microlens 21 has a light-focusing function and can be an on-chip microlens (OCL). The filter unit array includes multiple filter units 22, such as a red (R) filter unit corresponding to a red pixel, a green (G) filter unit corresponding to a green pixel, and a blue (B) filter unit corresponding to a blue pixel. The photosensitive unit array includes multiple photosensitive units 23.

[0068] The light reflected from the subject passes through the lens 10 and then enters the microlens 21. The microlens 21 focuses the incident light, so that the focused light passes through the light filtering unit 22 and is projected onto the photosensitive unit 23. The photosensitive unit 23 converts the light signal into an electrical signal for imaging.

[0069] In this way, the microlens array, the filter unit array, and the photosensitive unit array can together constitute the pixel array of the image sensor.

[0070] In some image sensors, such as Figure 2 As shown in (a) and (b), the pixel array of the image sensor may include multiple pixels 24 distributed in an array, such as red pixels, green pixels and blue pixels, etc. Each pixel 24 may include a microlens 21, a filter unit 22 covered by the microlens 21 and a photosensitive unit 23 covered by the filter unit 22, and each photosensitive unit 23 may include a photosensitive element.

[0071] Some pixels in the image sensor can be designated as phase detection pixels for phase detection. These detection pixels can be reserved shield pixels (SPs), which are pixels whose areas of the photosensitive unit 23 are partially obscured. For example, some green pixels in the image sensor can be designated as phase detection pixels.

[0072] Specifically, a light-shielding material layer 25 can be provided between the photosensitive unit 23 and the filter unit 22. The light-shielding material layer 25 can block part of the photosensitive unit 23. The light-shielding material layer 25 can block the light entering the photosensitive unit 23 below it, so that the light can only enter the photosensitive unit 23 that is not blocked by the light-shielding material layer 25.

[0073] like Figure 2 As shown in (a) and (b), the phase detection pixel may include a left phase detection pixel and a right phase detection pixel. For example, the left phase detection pixel may be a masked pixel on the right side that is partially blocked by the light-shielding material layer 25, and the right phase detection pixel may be a masked pixel on the left side that is partially blocked by the light-shielding material layer 25.

[0074] In this way, of the imaging beam incident on the right phase detection pixel, only the right-hand beam can be imaged on the photosensitive portion of the right phase detection pixel (i.e., the portion not blocked by the light-blocking material layer 25), forming a right phase sub-image; similarly, of the imaging beam incident on the left phase detection pixel, only the left-hand beam can be imaged on the photosensitive portion of the left phase detection pixel (i.e., the portion not blocked by the light-blocking material layer 25), forming a left phase sub-image. Thus, by comparing the left and right phase sub-images, phase difference information can be obtained.

[0075] It should be noted that, Figure 2 The image sensor shown in (a) is... Figure 2 The image sensor shown in (b) is a cross-sectional view obtained along section A-A'.

[0076] In other image sensors, such as Figure 3 As shown in (a) and (b), the pixel array of the image sensor may include multiple pixels 24 arranged in an array, such as red pixels, green pixels, and blue pixels. Each pixel 24 may include a microlens 21, a filter unit 22 covered by the microlens 21, and a photosensitive unit 23 covered by the filter unit 22. Each photosensitive unit 23 may include two arranged side by side (e.g., Figure 3 The photosensitive elements (as shown, arranged horizontally side by side), such as photosensitive unit 23, may include a first photosensitive element 231 and a second photosensitive element 232. Figure 3 The image sensors shown in (a) and (b) can also be called dual-pixel autofocus image sensors (Dual PD image sensors).

[0077] Each pixel 24 in the image sensor includes a left phase detection pixel and a right phase detection pixel, that is, all the pixels in the image sensor can be used for phase detection. For any one pixel 24 in the pixel array of the image sensor, it can be divided into a left phase detection pixel and a right phase detection pixel. The left phase detection pixel includes the first photosensitive element 231, the light filtering unit 22 corresponding to the first photosensitive element 231, and the microlens 21, and the right phase detection pixel includes the second photosensitive element 232, the light filtering unit 22 corresponding to the second photosensitive element 232, and the microlens 21, and the first photosensitive element 231 and the second photosensitive element 232 share the same color of the light filtering unit 22 and also share the same microlens 21.

[0078] In this way, when the light is incident on the pixel 24, the light is sequentially focused on the first photosensitive element 231 and the second photosensitive element 232 through the microlens 21 and the light filtering unit 22, respectively, the first photosensitive element 231 photoelectrically converts the incident light to obtain a left phase sub-image, and the second photosensitive element 232 photoelectrically converts the incident light to obtain a right phase sub-image. In this way, by comparing the left phase sub-image and the right phase sub-image, phase difference information can be obtained.

[0079] It should be noted that, Figure 3 The image sensor shown in (a) of FIG. 1A is an image sensor shown in (a) of FIG. 1B. Figure 3 The image sensor shown in (b) of FIG. 1A is a cross-sectional view of the image sensor shown in (b) of FIG. 1B along the section C-C’.

[0080] In some other image sensors, as shown in (a) and (b) of FIG. 1C, the pixel array of the image sensor can include a plurality of pixel units 240 arranged in an array, such as red pixel units, green pixel units, and blue pixel units. Figure 4

[0081] The four same-color pixels included in each pixel unit 240 in the image sensor share the same color of the light filtering unit 22 and share the same microlens 21, that is, each pixel unit 240 can include a microlens 21, a light filtering unit 22 covered by the microlens 21, and four photosensitive units 23 covered by the light filtering unit 22, and each photosensitive unit 23 includes a photosensitive element. Figure 4 The image sensors shown in (a) and (b) of FIG. 1C can also be referred to as full-pixel phase focus image sensors, and the full-pixel phase focus image sensor relies on the separation effect of the microlens 21 on the focusing phase to achieve phase detection and phase focusing. ​

[0082] Each pixel unit 240 in the image sensor can include a left phase detection pixel and a right phase detection pixel, that is, all pixels in the image sensor can be used for phase detection. The pixels included in any one pixel unit 240 in the pixel array of the image sensor can be divided into a left phase detection pixel and a right phase detection pixel.

[0083] For example, for one of the pixel units 240, a first red pixel (R0 pixel) can be used as a left phase detection pixel, and a second red pixel (R1 pixel) can be used as a right phase detection pixel; and a third red pixel (R2 pixel) can be used as a left phase detection pixel, and a fourth red pixel (R3 pixel) can be used as a right phase detection pixel.

[0084] In this way, when light is incident on the pixel unit 240, the light passes through the microlens 21 and the filter unit 22 in sequence, and is focused to the photosensitive unit 23 corresponding to the first red pixel (R0 pixel), the photosensitive unit 23 corresponding to the second red pixel (R1 pixel), the photosensitive unit 23 corresponding to the third red pixel (R2 pixel), and the photosensitive unit 23 corresponding to the fourth red pixel (R3 pixel), respectively. The photosensitive unit 23 corresponding to the first red pixel (R0 pixel) and the photosensitive unit 23 corresponding to the third red pixel (R2 pixel) perform photoelectric conversion on the incident light to obtain a left phase sub-image; and the photosensitive unit 23 corresponding to the second red pixel (R1 pixel) and the photosensitive unit 23 corresponding to the fourth red pixel (R3 pixel) perform photoelectric conversion on the incident light to obtain a right phase sub-image. In this way, by comparing the left phase sub-image and the right phase sub-image, phase difference information can be obtained.

[0085] It should be noted that, Figure 4 The image sensor shown in (a) in FIG. 10 is Figure 4 The image sensor shown in (b) in FIG. 10 is a sectional view along the section D-D'.

[0086] It can be understood that, in addition to the four-bayer array shown in FIG. 10, the pixel array of the full-pixel phase focus image sensor in the embodiments of the present application can also use a nine-bayer array or a sixteen-bayer array, and the like. The specific form of the pixel array of the image sensor is not limited in the embodiments of the present application. Figure 4 It can be understood that, in addition to the four-bayer array shown in FIG. 10, the pixel array of the full-pixel phase focus image sensor in the embodiments of the present application can also use a nine-bayer array or a sixteen-bayer array, and the like. The specific form of the pixel array of the image sensor is not limited in the embodiments of the present application.

[0087] When the pixel array of the full-pixel phase focus image sensor is a nine-bayer array, each pixel unit 240 in the pixel array of the full-pixel phase focus image sensor includes nine adjacent same-color pixels, and the nine same-color pixels share a same-color filter unit 22 and share a same microlens 21. When the pixel array of the full-pixel phase focus image sensor is a sixteen-bayer array, each pixel unit 240 in the pixel array of the full-pixel phase focus image sensor includes sixteen adjacent same-color pixels, and the sixteen same-color pixels share a same-color filter unit 22 and share a same microlens 21.

[0088] It can be understood that, Figures 2 to 4 The image sensors shown are all for left-right phase detection focusing. In actual applications, the phase detection focusing mode can include, but is not limited to, left-right phase detection focusing, up-down phase detection focusing, and four-phase detection focusing, etc.

[0089] Left-right phase detection focusing refers to setting left phase detection pixels and right phase detection pixels in the image sensor, using the left phase detection pixels to collect a left phase sub-image, and using the right phase detection pixels to collect a right phase sub-image, calculating phase difference information according to the left phase sub-image and the right phase sub-image, then calculating a defocus distance based on the phase difference information, and moving the lens based on the defocus distance by a focusing motor to adjust the distance between the lens and the image sensor, thereby realizing phase focusing.

[0090] Up-down phase detection focusing refers to setting up phase detection pixels and down phase detection pixels in the image sensor. Specifically, for the image sensor shown in FIG. 1, Figure 2 For the image sensor shown in FIG. 2, the shielded pixels partially shielded by the light-shielding material layer 25 on the lower side can be used as the up phase detection pixels, and the shielded pixels partially shielded by the light-shielding material layer 25 on the upper side can be used as the down phase detection pixels. For the image sensor shown in FIG. 3, Figure 3 For the image sensor shown in FIG. 4, the first light sensing element 231 and the second light sensing element 232 are arranged side by side in the vertical direction, so that each pixel 24 includes an up phase detection pixel and a down phase detection pixel. For the image sensor shown in FIG. 5, Figure 4 For the image sensor shown in FIG. 6, each pixel unit 240 includes an up phase detection pixel and a down phase detection pixel. For example, for one of the pixel units 240, the first red pixel (R0 pixel) can be used as the up phase detection pixel, and the third red pixel (R2 pixel) can be used as the down phase detection pixel; and the second red pixel (R1 pixel) can be used as the up phase detection pixel, and the fourth red pixel (R3 pixel) can be used as the down phase detection pixel.

[0091] And, the upper phase detection pixels are used to collect an upper phase sub-image, and the lower phase detection pixels are used to collect a lower phase sub-image, phase difference information is calculated according to the upper phase sub-image and the lower phase sub-image, and then a defocus distance is calculated based on the phase difference information, and a focusing motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, so as to realize phase focusing.

[0092] Four-phase detection focusing refers to that left phase detection pixels, right phase detection pixels, upper phase detection pixels and lower phase detection pixels are arranged in the image sensor, phase sub-images collected by the phase detection pixels in the four directions are used to calculate phase difference information, and then a defocus distance is calculated based on the phase difference information, and a focusing motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, so as to realize phase focusing.

[0093] For the camera described above, the camera has an optical anti-shake module, a lens 10 and an image sensor 20, and the image sensor 20 includes a plurality of phase detection pixels. The phase detection pixels can include left phase detection pixels and right phase detection pixels, or the phase detection pixels can include upper phase detection pixels and lower phase detection pixels, or the phase detection pixels can include left phase detection pixels, right phase detection pixels, upper phase detection pixels and lower phase detection pixels.

[0094] However, in the case that the camera starts the optical anti-shake module to perform optical anti-shake, the optical anti-shake module will move to drive the lens 10 to move, which will cause the optical center of the lens 10 to deviate from the optical center of the image sensor 20 (i.e., the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is not equal to 0), so that the light passing through the lens 10 is focused again after passing through the microlens 21, and the light received by the phase detection pixels in different directions of the image sensor 20 is uneven, thereby causing the brightness values of the pixels in the first phase image collected by the phase detection pixels to be uneven. For example, the light received by the left phase detection pixels and the right phase detection pixels is uneven, or the light received by the upper phase detection pixels and the lower phase detection pixels is uneven.

[0095] The brightness difference of the pixels in the first phase image collected by the phase detection pixels will affect the calculation of the phase difference information. If the brightness values of the pixels in the first phase image collected by the phase detection pixels are uneven, the determined phase difference information will be inaccurate when the first phase image is used to determine the phase difference information, so that the defocus distance determined according to the phase difference information is also inaccurate, thereby causing the focusing stability and the focusing accuracy in the optical anti-shake mode to be not high.

[0096] In the related art, as Figure 5In (a) of FIG. 1, when the camera starts the optical image stabilization module to perform optical image stabilization, and the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0, the first phase image can be collected by the phase detection pixels in the image sensor 20.

[0097] In (a) of FIG. 1, when the camera starts the optical image stabilization module to perform optical image stabilization, and the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0, the first phase image can be collected by the phase detection pixels in the image sensor 20. Figure 5 In (b) of FIG. 1, the first phase image includes a first left phase sub-image collected by the left phase detection pixels and a first right phase sub-image collected by the right phase detection pixels. Figure 5 In the first phase image shown in (b) of FIG. 1, the horizontal coordinate represents a plurality of image blocks in the first left phase sub-image and the first right phase sub-image, for example, the first left phase sub-image is split into 17 image blocks along the horizontal direction, and the first right phase sub-image is also split into 17 image blocks along the horizontal direction; the vertical coordinate represents the brightness value of each image block in the first left phase sub-image and the first right phase sub-image, which is the average of the brightness values of all pixels in each image block.

[0098] The related art can perform brightness compensation on the first phase image shown in (b) of FIG. 1 by using the first phase gain calibration matrix shown in (c) of FIG. 1, and the brightness compensated first phase image is shown in (d) of FIG. 1. Figure 5 The related art can perform brightness compensation on the first phase image shown in (b) of FIG. 1 by using the first phase gain calibration matrix shown in (c) of FIG. 1, and the brightness compensated first phase image is shown in (d) of FIG. 1. Figure 5 The related art can perform brightness compensation on the first phase image shown in (b) of FIG. 1 by using the first phase gain calibration matrix shown in (c) of FIG. 1, and the brightness compensated first phase image is shown in (d) of FIG. 1. Figure 5 The first phase gain calibration matrix is calibrated when the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0.

[0099] In the first phase gain calibration matrix shown in (c) of FIG. 1, the first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix. The horizontal coordinate represents a plurality of image blocks, and the vertical coordinate represents the compensation coefficient corresponding to each image block in the first left phase gain calibration sub-matrix and the first right phase gain calibration sub-matrix. Figure 5 In the compensated first phase image shown in (d) of FIG. 1, the compensated first phase image includes a compensated first left phase sub-image and a compensated first right phase sub-image. The horizontal coordinate represents a plurality of image blocks in the compensated first left phase sub-image and the compensated first right phase sub-image, and the vertical coordinate represents the brightness value of each image block in the compensated first left phase sub-image and the compensated first right phase sub-image.

[0100] Figure 5 Specifically, the first left phase gain calibration sub-matrix shown in (c) of FIG. 1 can be used to compensate the first left phase sub-image shown in (b) of FIG. 1, and the compensated first left phase sub-image is shown in (d) of FIG. 1.

[0101] Specifically, the first left phase gain calibration sub-matrix shown in (c) of FIG. 1 can be used to compensate the first left phase sub-image shown in (b) of FIG. 1, and the compensated first left phase sub-image is shown in (d) of FIG. 1. Figure 5 ​​Figure 5 The first left-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 5 The compensated first left-phase sub-image is shown in (d) in the figure; using Figure 5 The first right phase gain calibration submatrix shown in (c) is for... Figure 5 The first right-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 5 The compensated first right-phase sub-image is shown in (d) in the figure.

[0102] It can be seen that when the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is zero, the brightness values ​​of the pixels in the first phase image acquired by the phase detection pixels are uneven. However, after using the first phase gain calibration matrix to compensate for the brightness of the first phase image, the brightness values ​​of the pixels in the compensated first phase image become more uniform. Thus, when using the compensated first phase image to determine phase difference information, the determined phase difference information is more accurate. The determined phase difference information has a linear relationship with the defocus distance. Based on this linear relationship, the autofocus system can calculate the defocus distance corresponding to the phase difference information to perform the judgment of the focus position and the focusing action.

[0103] It should be noted that due to the manufacturing process of the image sensor and the fact that the microlenses 21 set at various positions are not completely consistent, when the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0, the phase detection pixels in different positions of the image sensor 20 receive uneven light, which in turn causes uneven brightness values ​​of the pixels in the first phase image acquired by the phase detection pixels.

[0104] However, the relevant technology uses a first phase gain calibration matrix to compensate for the brightness of the first phase image. This method can only compensate for the uneven brightness values ​​of pixels in the first phase image caused by the uneven light received by phase detection pixels in different directions when the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is equal to 0.

[0105] like Figure 6 As shown in (a), when the camera activates the optical image stabilization module for optical image stabilization, and the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 can be d, where d is not equal to 0, the first phase image can be acquired through the phase detection pixels in the image sensor 20.

[0106] When the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is not equal to zero, the acquired first phase image is as follows: Figure 6As shown in (b), it includes a first left-phase sub-image acquired by the left-phase detection pixels and a first right-phase sub-image acquired by the right-phase detection pixels. Figure 6 In the first phase image shown in (b), the horizontal axis represents multiple image blocks in the first left phase sub-image and the first right phase sub-image, and the vertical axis represents the brightness value of each image block in the first left phase sub-image and the first right phase sub-image. The brightness value is the average of the brightness values ​​of all pixels in each image block.

[0107] If we still adopt the following... Figure 6 The first phase gain calibration matrix shown in (c) is for... Figure 6 The first phase image shown in (b) is subjected to brightness compensation, and the brightness-compensated first phase image is as follows: Figure 6 As shown in (d) in the figure.

[0108] exist Figure 6 In the first phase gain calibration matrix shown in (c), the first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix. The horizontal axis represents multiple image blocks, and the vertical axis represents the compensation coefficient corresponding to each image block in the first left phase gain calibration sub-matrix and the first right phase gain calibration sub-matrix.

[0109] exist Figure 6 In the compensated first phase image shown in (d), the compensated first phase image includes a compensated first left phase sub-image and a compensated first right phase sub-image. The horizontal axis represents multiple image blocks in the compensated first left and first right phase sub-images, and the vertical axis represents the brightness value of each image block in the compensated first left and first right phase sub-images.

[0110] Specifically, it can be adopted Figure 6 The first left-side phase gain calibration submatrix shown in (c) is for... Figure 6 The first left-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 6 The compensated first left-phase sub-image is shown in (d) in the figure; using Figure 6 The first right phase gain calibration submatrix shown in (c) is for... Figure 6 The first right-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 6 The compensated first right-phase sub-image is shown in (d) in the figure.

[0111] It can be seen that in the case that the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is not equal to 0, the brightness values of the pixels in the first phase image collected by the phase detection pixels are uneven. If the first phase gain calibration matrix is still used to compensate the brightness of the first phase image, the first phase gain calibration matrix is invalid at this time, and the brightness values of the pixels in the compensated first phase image are still uneven.

[0112] In this way, in the case that the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is not equal to 0, if the first phase image compensated by the first phase gain calibration matrix is used to determine the phase difference information, the determined phase difference information is deviated, resulting in that the determined phase difference information and the defocus distance are actually in a nonlinear relationship. If the linear relationship is still used to calculate the defocus distance corresponding to the phase difference information, the determined defocus distance is inaccurate, thereby causing the misjudgment of the focusing system in the anti-shake working condition, that is, the focusing stability and the focusing accuracy in the optical anti-shake mode are not high.

[0113] Based on this, the embodiment of the present application provides a phase focusing method. In the case that the optical anti-shake module is started, the first phase image collected by the phase detection pixels is obtained, and the first coordinate of the position of the optical anti-shake module at a target time is obtained. The target time is the time at which the focusing point in the original image collected by the image sensor is located at the time of collection. The phase gain compensation matrix is determined according to the first coordinate and a plurality of phase gain calibration matrices. The relative positions of the optical center of the lens and the optical center of the image sensor are different when the plurality of phase gain calibration matrices are calibrated. The first phase image is compensated by using the phase gain compensation matrix to obtain the second phase image, and the phase focusing is performed based on the second phase image.

[0114] Therefore, the embodiment of the present application can calculate the phase gain compensation matrix according to the first coordinate of the position of the optical anti-shake module at the target time and the plurality of phase gain calibration matrices in the case that the camera starts the optical anti-shake module to perform optical anti-shake, so as to compensate the brightness of the first phase image collected by the phase detection pixels. The brightness values of the pixels in the second phase image obtained after the brightness compensation are more uniform. In this way, when the second phase image is used to determine the phase difference information, the determined phase difference information is more accurate, the determined phase difference information and the defocus distance are in a linear relationship, the defocus distance corresponding to the phase difference information can be accurately calculated based on the linear relationship, thereby improving the focusing stability and the focusing accuracy in the optical anti-shake mode.

[0115] The phase focusing method provided by the embodiments of the present application can be applied to an electronic device with a camera. The electronic device includes a terminal device, which can also be referred to as a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The electronic device can be a mobile phone, a smart television, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form of the electronic device.

[0116] In order to better understand the embodiments of the present application, the structure of the electronic device of the embodiments of the present application is introduced as follows.

[0117] Figure 7 A schematic diagram of the hardware system structure of the electronic device 700 is shown. The electronic device 700 can include a processor 710, an external memory interface 720, an internal memory 721, a universal serial bus (USB) interface 730, a charge management module 740, a power management module 741, a battery 742, an antenna 1, an antenna 2, a mobile communication module 750, a wireless communication module 760, an audio module 770, a loudspeaker 770A, a receiver 770B, a microphone 770C, a headset interface 770D, a sensor module 780, a key 790, a motor 791, an indicator 792, a camera 793, a display screen 794, and a subscriber identification module (SIM) card interface 795, etc. The sensor module 780 can include a gyroscope sensor 780A and an acceleration sensor 780B.

[0118] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 700. In some other embodiments of the present application, the electronic device 700 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software or a combination of software and hardware.

[0119] The processor 710 can include one or more processing units, for example: the processor 710 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0120] The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0121] The memory in the processor 710 can also be provided for storing instructions and data. In some embodiments, the memory in the processor 710 is a cache memory. The memory can save instructions or data that the processor 710 has just used or repeatedly uses. If the processor 710 needs to use the instructions or data again, it can be called from the memory. Avoiding repeated access, reducing the waiting time of the processor 710, thus improving the efficiency of the system.

[0122] The charging management module 740 is used to receive charging input from a charger. Among them, the charger can be a wireless charger, or a wired charger. In some wired charging embodiments, the charging management module 740 can receive the charging input of the wired charger through the USB interface 730. In some wireless charging embodiments, the charging management module 740 can receive the wireless charging input through the wireless charging coil of the electronic device 700. The charging management module 740 charges the battery 742, and at the same time, it can also supply power to the electronic device through the power management module 741.

[0123] The power management module 741 is configured to connect the battery 742 and the charging management module 740 to the processor 710. The power management module 741 receives input from the battery 742 and / or the charging management module 740 to supply power to the processor 710, the internal memory 721, the display screen 794, the camera 793, the wireless communication module 760, and the like. The power management module 741 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health (leakage, impedance), and the like. In some embodiments, the power management module 741 can also be disposed in the processor 710. In some embodiments, the power management module 741 and the charging management module 740 can also be disposed in the same device.

[0124] The wireless communication function of the electronic device 700 can be implemented by the antenna 1, the antenna 2, the mobile communication module 750, the wireless communication module 760, the modem processor, and the baseband processor, and the like.

[0125] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. The mobile communication module 750 can provide a solution for wireless communication including 2G / 3G / 4G / 5G and the like applied to the electronic device 700. The mobile communication module 750 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like.

[0126] The wireless communication module 760 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like applied to the electronic device 700. The wireless communication module 760 can be one or more devices integrated with at least one communication processing module. The wireless communication module 760 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering on the electromagnetic wave signals, and transmits the processed signals to the processor 710. The wireless communication module 760 can also receive signals to be transmitted from the processor 710, perform frequency modulation and amplification on the signals, and radiate the signals as electromagnetic waves via the antenna 2.

[0127] In some embodiments, antenna 1 and mobile communication module 750 of electronic device 700 are coupled, and antenna 2 and wireless communication module 760 are coupled, so that electronic device 700 can communicate with a network and other devices through wireless communication technology.

[0128] Electronic device 700 implements a display function through a GPU, display screen 794, and an application processor, etc. The GPU is a microprocessor for image processing, connected to display screen 794 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 710 can include one or more GPUs that execute program instructions to generate or change display information.

[0129] Display screen 794 is used to display images, display videos, and receive sliding operations, etc. Display screen 794 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, electronic device 700 can include 1 or more display screens 794.

[0130] Electronic device 700 can implement a shooting function through an ISP, camera 793, a video codec, a GPU, display screen 794, and an application processor, etc.

[0131] The ISP is used to process data fed back by camera 793. For example, when taking a photo, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize algorithms for image noise, brightness, and skin color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be disposed in camera 793.

[0132] The camera 793 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, or the like format image signal. In some embodiments, the electronic device 700 can include one or more cameras 793.

[0133] The external memory interface 720 can be configured to connect with an external memory card, such as a micro SD card, to extend the memory capacity of the electronic device 700. The external memory card communicates with the processor 710 via the external memory interface 720 to perform data storage functions. For example, music, video, and the like files can be saved in the external memory card.

[0134] The internal memory 721 can be configured to store computer executable program codes including instructions. The internal memory 721 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program (such as a sound play application, an image play application, and the like) required by at least one function, and the like. The data storage area can store data (such as audio data, a phonebook, and the like) created during the use of the electronic device 700, and the like. In addition, the internal memory 721 can include a high speed random access memory, and can further include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like. The processor 710 performs various function applications and data processing of the electronic device 700 by running instructions stored in the internal memory 721 and / or instructions stored in a memory disposed in the processor.

[0135] The electronic device 700 can implement audio functions through an audio module 770, a speaker 770A, a receiver 770B, a microphone 770C, an earphone interface 770D, and an application processor, and the like. For example, music play, voice recording, and the like.

[0136] The gyroscope sensor 780A can be used to determine the motion posture of the electronic device 700. In some embodiments, the angular velocity of the electronic device 700 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyroscope sensor 780A. The gyroscope sensor 780A can be used for anti-shake photography. For example, when the shutter is pressed, the gyroscope sensor 780A detects the angle of shaking of the electronic device 700, calculates the distance that the lens needs to compensate according to the angle, and lets the lens offset the shaking of the electronic device 700 by reverse movement to achieve anti-shake. The gyroscope sensor 780A can also be used in scenarios such as navigation and motion sensing games.

[0137] The acceleration sensor 780B can detect the magnitude of acceleration of the electronic device 700 in various directions (generally three axes). When the electronic device 700 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device, applied to landscape / portrait switching, pedometer, and other application programs.

[0138] The keys 790 include a power key, a volume key, and the like. The keys 790 can be mechanical keys. They can also be touch keys. The electronic device 700 can receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 700. The motor 791 can generate a vibration prompt. The motor 791 can be used for incoming call vibration prompts and also for touch vibration feedback. The indicator 792 can be an indicator light that can be used to indicate the charging state, the power change, and also to indicate messages, missed calls, notifications, and the like. The SIM card interface 795 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 795 to achieve contact and separation with the electronic device 700.

[0139] The software system of the electronic device 700 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture, etc. The embodiments of the present application take the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 700.

[0140] Figure 8 FIG. 7 is a schematic diagram of the software system structure of the electronic device 700 according to an embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, the hardware abstraction layer, and the kernel layer.

[0141] The application layer can include a series of application packages. As shown in FIG. 7, the application packages can include camera, settings, and calendar applications. Figure 8 ​

[0142] The camera application is an application having a photographing and video recording function, and the electronic device can respond to a user's operation of opening the camera application to perform photographing or video recording. It can be understood that the photographing and video recording function of the camera application can also be invoked by other applications.

[0143] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0144] As shown in FIG. 1, the application framework layer can further include a camera service, which can be invoked by the camera application to implement photographing or video recording and the like. Figure 8 In addition, as shown in FIG. 1, the application framework layer can further include a window manager, a content provider, a resource manager, and a view system, and the like.

[0145] Figure 8 The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen, and the like.

[0146] The content provider is used to store and obtain data, and make the data accessible to the application program. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phone books, and the like.

[0147] The resource manager provides various resources for the application program, such as localized strings, icons, pictures, layout files, video files, and the like.

[0148] The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, and the like. The view system can be used to build an application program. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0149] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0150] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0151] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0152] ​The application layer and the application framework layer run in a virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the functions of management of object life cycle, stack management, thread management, management of security and exception, and garbage collection.

[0153] The system library can include a plurality of functional modules. For example, a surface manager, media libraries, a three-dimensional graphics processing library (e.g., OpenGL ES), a two-dimensional graphics engine (e.g., SGL), and the like.

[0154] The surface manager is used to manage the display subsystem and provides fusion of 2D and 3D layers for a plurality of applications.

[0155] The media libraries support playback and recording of a plurality of commonly used audio, video formats, and static image files. The media libraries can support a plurality of audio and video encoding formats, such as MPEG2, H.262, MP3, AAC, AMR, JPG, PNG, and the like.

[0156] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing. The two-dimensional graphics engine is a drawing engine for 2D drawing.

[0157] The hardware abstraction layer is an abstracted layer between the kernel layer and the Android runtime. The hardware abstraction layer can be a package of hardware drivers of the kernel layer and provides a calling interface for the application framework layer. In the embodiments of the present application, the hardware abstraction layer can include a camera hardware abstraction module (camera HAL).

[0158] The kernel layer is a layer between hardware and software. The kernel layer at least includes a camera driver, a sensor driver, and a display driver. In some embodiments, the camera driver is used to control the operation of a camera, the sensor driver is used to control the operation of a sensor, and the display driver is used to control the display of an image on a display screen.

[0159] The hardware can be a camera, a sensor, and a display screen. In the embodiments of the present application, the camera can be a front camera or a rear camera.

[0160] In some embodiments, during the execution of the phase focusing method of the embodiments of the present application, the camera driver can drive the camera to acquire a first phase image collected by the phase detection pixels in the case of starting the optical image stabilization module, and send the first phase image to the camera hardware abstraction module; the camera hardware abstraction module can also acquire a first coordinate of the position of the optical image stabilization module at a target time; the camera hardware abstraction module determines a phase gain compensation matrix according to the first coordinate and a plurality of phase gain calibration matrices, and compensates the first phase image by using the phase gain compensation matrix to obtain a second phase image, and then performs phase focusing based on the second phase image.

[0161] Specifically, the camera hardware abstraction module can determine phase difference information based on the second phase image, and then calculate the defocus distance based on the phase difference information; the camera hardware abstraction module can send the defocus distance to the focus motor in the camera through the camera driver, and the focus motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, thereby realizing phase focusing.

[0162] It should be noted that although the embodiments of the present application are described based on the Android system, the principle of the phase focusing method is also applicable to electronic devices running iOS or windows operating systems.

[0163] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be implemented independently or in combination, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0164] For ease of understanding, the embodiments of the present application take a mobile phone as an electronic device, and first illustrate the application scenarios of the phase focusing method in combination with some user interfaces shown in the embodiments of the present application.

[0165] When the user turns on the screen of the electronic device and controls the electronic device to be in an unlocked state, the electronic device can display a first interface 901 as shown in (a) of FIG. 9A. Figure 9 The first interface 901 can be the desktop of the electronic device, and the icons of a plurality of installed application programs are displayed on the desktop of the electronic device, such as a file management application icon, an email application icon, a weather application icon, a calculator application icon, a clock application icon, a sound recorder application icon, a music application icon, a settings application icon, a contacts application icon, a phone application icon, a message application icon, and a camera application icon 9011.

[0166] The user can perform a touch operation on the camera application icon 9011, which can be a click operation, a long press operation, etc., so that the electronic device receives the touch operation of the user on the camera application icon 9011, and the electronic device starts the camera application in response to the touch operation.

[0167] After the camera application is started, the electronic device can display a second interface 902 as shown in (b) of FIG. 9B. The second interface 902 can be a preview interface provided by the camera application for implementing a shooting function, which includes a preview area 9021, a shooting control 9022, and function controls corresponding to multiple shooting modes, etc. Figure 9

[0168] The preview area 9021 can be used to display an original image captured by an image sensor in a camera lens. The shooting control 9022 is used to trigger a shooting operation of the electronic device. The function controls corresponding to multiple shooting modes can include a night scene mode control, a portrait mode control, a still image shooting mode control, a video recording mode control, a professional mode control, and a more control for enabling more functions in the camera application, etc.

[0169] In an implementable manner, the user can perform a touch operation, such as a click operation, on a certain position of the original image displayed in the preview area 9021, so that the electronic device receives the touch operation of the user on the certain position of the original image, and the electronic device selects the position touched by the user as a focus point and selects a rectangular area or a circular area centered on the focus point as a focus area. Figure 9 As shown in (c) of FIG. 9B, the electronic device can display a focus frame 9023 in the preview area 9021, and the area in the focus frame 9023 is the focus area.

[0170] In another implementable manner, the electronic device can call an image recognition model to recognize the original image displayed in the preview area 9021 to determine a target object in the original image, determine a focus point according to the position of the target object, and select a rectangular area or a circular area centered on the focus point as a focus area. The target object can be a human body, a human face, etc.

[0171] Therefore, the embodiments of the present application can start the camera application by performing a touch operation on the camera application icon 9011, and after the camera application is started, the focus area can be determined by the way that the user performs a touch operation on a certain position of the original image displayed in the preview area 9021, or by the way that an image recognition model is used to recognize the original image displayed in the preview area 9021. After the focus area is determined, the electronic device can perform the process corresponding to the phase focusing method provided by the embodiments of the present application to perform phase focusing.

[0172] ​In addition, the user can also call the corresponding interface through a third-party application installed on the electronic device to access the camera application of the electronic device to start the camera application. Alternatively, when the electronic device is in a locked state, the user can indicate the electronic device to start the camera application by swiping right on the display screen of the electronic device. Alternatively, when the electronic device is in a locked state, the lock screen interface includes an icon of the camera application, and the user indicates the electronic device to start the camera application by clicking the icon of the camera application.

[0173] It should be understood that the above is an example of the operation of starting the camera application; the electronic device can also be instructed to start the camera application by voice instruction or other operation, which is not limited in the present application.

[0174] The phase focusing method provided by the embodiments of the present application can be applied not only to the preview scene of the camera application but also to the shooting scene of the camera application.

[0175] For example, Figure 10 A flowchart of a phase focusing method provided by the embodiments of the present application can be applied to an electronic device, which can include a camera head including an optical image stabilization module, a lens 10, and an image sensor 20 including a plurality of phase detection pixels. The image sensor 20 can be any one of the image sensors shown in Figures 2 to 4 Referring to Figure 10 The phase focusing method can specifically include the following steps:

[0176] S1001, the electronic device acquires a first phase image collected by the phase detection pixels under the condition that the optical image stabilization module is started.

[0177] In some embodiments, under the condition that the camera head starts the optical image stabilization module for optical image stabilization, the camera head can synchronously collect a raw image and a first phase image, the raw image corresponding to the first phase image.

[0178] The raw image refers to an image collected by all pixels included in the image sensor in the camera head, and the raw image can be a RAW format image. The raw image can be displayed in the preview area of the electronic device. The first phase image refers to an image collected by the plurality of phase detection pixels included in the image sensor in the camera head.

[0179] It can be understood that, for the image sensor shown in Figure 2 In this case, the size of the first phase image is smaller than the size of the raw image.

[0180] S1002, the electronic device obtains a second coordinate of the focus point in the original image.

[0181] In some embodiments, the electronic device can obtain a second coordinate of the focus point in the original image captured by the image sensor. The second coordinate of the focus point can be (Xn, Yn), Xn is a coordinate component in the second coordinate along the width direction of the original image, Yn is a coordinate component in the second coordinate along the height direction of the original image. n Yn is a coordinate component in the second coordinate along the height direction of the original image.

[0182] The focus point can refer to the center position of the focus frame. The focus point can be determined by the touch position when the user performs a touch operation on a certain position of the original image, or the focus point can be determined by the position of the target object after the target object in the original image is identified.

[0183] S1003, the electronic device determines a target time length according to the second coordinate.

[0184] In some embodiments, the electronic device can determine a target time length according to the second coordinate of the focus point in the original image. The target time length refers to the time length elapsed from the initial time when the original image is started to be captured to the time when the focus point in the original image is captured.

[0185] Specifically, the electronic device can determine the target time length according to the coordinate component in the second coordinate along the height direction of the original image.

[0186] In an implementation manner, the electronic device calculates the target time length by the following formula:

[0187]

[0188] wherein, T lag is the target time length, EIT is the exposure time length of the original image, Yn is the coordinate component in the second coordinate along the height direction of the original image, H is the height of the original image, RowNum is the total number of rows of the photosensitive unit array in the image sensor, H n is the reading time length when reading the image data generated by one row of photosensitive units in the photosensitive unit array. time

[0189] The image sensor starts to control the photosensitive unit array to expose from the exposure start time corresponding to the original image when capturing the original image, and starts to read the image data generated by each row of photosensitive units in the photosensitive unit array in turn from the exposure end time corresponding to the original image to obtain the original image. Therefore, the capture time length of the original image includes the exposure time length of the original image and the reading time length of the original image.

[0190] ​As shown in (a) of Figure 11 , a start point of the exposure of the original image is defined as a start of frame (SOF) point, and an end point of the original image is defined as an end of frame (EOF) point. A first timestamp at the SOF point is defined as T1, and a second timestamp at the EOF point is defined as T2.

[0191] In the process of collecting the original image, the collection duration of the original image is the time interval between the second timestamp T2 and the first timestamp T1. The first timestamp T1 represents the timestamp of the exposure start time of the original image, and is also the timestamp of the time when the image sensor starts collecting the original image. The second timestamp T2 represents the timestamp of the reading end time of the original image, and is also the timestamp of the time when the image sensor ends collecting the original image.

[0192] Therefore, in the above formula, the reading duration required for reading the focus point in the original image can be represented. The electronic device can use the sum of half of the exposure duration of the original image and the reading duration required for reading the focus point in the original image as the target duration. The exposure duration of the original image can also refer to the exposure duration of the first phase image.

[0193] In S1004, the electronic device determines the sum of the initial time when the original image starts to be collected and the target duration as a target time. The target time is the time when the focus point in the original image collected by the image sensor is collected.

[0194] It should be understood that the target time is actually the time when the focus point in the original image collected by the image sensor is collected, which is estimated by the electronic device. The target time is actually an estimated time.

[0195] In S1005, the electronic device obtains a first coordinate of the position of the optical image stabilization module at the target time.

[0196] In the process of collecting the original image and the first phase image by starting the optical image stabilization module of the camera, the optical image stabilization module can move to drive the lens to move along the first moving direction and / or the second moving direction to compensate for the influence of the shaking, so as to achieve the purpose of optical image stabilization.

[0197] In this way, the coordinates of the positions of the optical image stabilization module after each movement in the process of collecting the original image and the first phase image, i.e., in the time interval between the second timestamp T2 and the first timestamp T1, can be counted.

[0198] As shown in (a) of Figure 11In (b) shown in the figure, in the process of capturing the original image and the first phase image, the optical image stabilization module can move along the first moving direction and the second moving direction, OIS-X represents the displacement schematic diagram of the optical image stabilization module when moving along the first moving direction, and OIS-Y represents the displacement schematic diagram of the optical image stabilization module when moving along the second moving direction.

[0199] After the electronic device estimates the target moment, the electronic device can extract the first coordinate of the position of the optical image stabilization module at the target moment from the coordinates of the positions of the optical image stabilization module after each movement. The first coordinate of the position of the optical image stabilization module at the target moment can be (OIS X , OIS Y ), OIS X is the first coordinate component along the first moving direction, and OIS Y is the second coordinate component along the second moving direction.

[0200] For example, the time interval between the moment when the image sensor finishes capturing the original image and the moment when the image sensor starts capturing the original image is 20 ms (milliseconds), that is, the time interval between the second time stamp and the first time stamp is 20 ms, and the optical image stabilization module moves once every 1 ms. Therefore, the number of movements of the optical image stabilization module in the process of capturing the original image can be 20, that is, 20 coordinates of the positions of the optical image stabilization module are obtained in the process of capturing the original image. Starting from the initial moment when the original image starts to be captured, it is assumed that the position of the focus point in the original image is captured at the 10th ms, and the coordinate of the position of the optical image stabilization module at the 10th ms is obtained as the first coordinate.

[0201] It can be understood that the coordinate of the position of the optical image stabilization module after each movement actually represents the relative position between the optical image stabilization module and the optical center of the image sensor after the movement of the optical image stabilization module. The relative position between the optical image stabilization module and the optical center of the image sensor after the movement of the optical image stabilization module can also be used to represent the relative position between the optical center of the lens 10 and the optical center of the image sensor 20 after the movement of the optical image stabilization module to drive the lens 10 to move.

[0202] For example, in the case where the optical image stabilization module is not started by the camera, the coordinate of the position of the optical image stabilization module can be (0, 0), which can represent that the optical center of the lens 10 and the optical center of the image sensor 20 coincide along the optical axis direction of the camera.

[0203] S1006, the electronic device determines a phase gain compensation matrix according to the first coordinate and a plurality of phase gain calibration matrices. The relative positions of the optical centers of the lens and the image sensor are different when the plurality of phase gain calibration matrices are calibrated.

[0204] Before determining the phase gain compensation matrix based on the first coordinate and multiple phase gain calibration matrices, the electronic device can also acquire multiple pre-calibrated phase gain calibration matrices.

[0205] The multiple phase gain calibration matrices include a first type of phase gain calibration matrix and / or a second type of phase gain calibration matrix. The first type of phase gain calibration matrix includes a first phase gain calibration matrix, a second phase gain calibration matrix, and a third phase gain calibration matrix; and during calibration, the relative positions of the optical center of the lens 10 and the optical center of the image sensor 20 are different. The second type of phase gain calibration matrix includes a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix; and during calibration, the relative positions of the optical center of the lens 10 and the optical center of the image sensor 20 are different.

[0206] The optical image stabilization module has two movable directions: a first movable direction and a second movable direction. This means the optical image stabilization module can drive the lens 10 to move along the first and second movable directions. The first and second movable directions are perpendicular to each other and both are perpendicular to the optical axis of the camera. The first movable direction includes a first direction and a second direction that are opposite to each other, and the second movable direction includes a third direction and a fourth direction that are opposite to each other. For example, such as... Figure 12 As shown, the first direction can be the X1 direction, the second direction can be the X2 direction, the third direction can be the Y1 direction, and the fourth direction can be the Y2 direction.

[0207] like Figure 12 As shown in (a) to (e), these represent the relative positions between the optical center of lens 10 and the optical center of image sensor 20 during the calibration of each phase gain calibration matrix. The intersection of the two dashed lines indicates the optical center of image sensor 20.

[0208] like Figure 12 As shown in (a), during the calibration of the first and fourth phase gain calibration matrices, the optical center of the lens 10 coincides with the optical center of the image sensor 20 along the optical axis of the camera. In this case, the relative position of the optical center of the lens 10 with respect to the optical center of the image sensor 20 is G0, and the position coordinates of G0 are (0, 0).

[0209] The first phase gain calibration matrix includes the first left phase gain calibration submatrix Q. 0L and the first right phase gain calibration submatrix Q 0RThe fourth phase gain calibration matrix includes the first upper phase gain calibration submatrix Q. 0T and the first lower phase gain calibration submatrix Q 0B .Right now Figure 12 The phase gain calibration matrix Q0 shown can actually include the first left phase gain calibration submatrix Q. 0L The first right phase gain calibration submatrix Q 0R The first upper phase gain calibration submatrix Q 0T and the first lower phase gain calibration submatrix Q 0B .

[0210] like Figure 12 As shown in (b), during the calibration of the second phase gain calibration matrix, the optical center of the lens 10 is offset by a first distance relative to the optical center of the image sensor 20 along a first direction; the first distance is the maximum distance that the lens 10 can move along the first direction. In this case, the relative position of the optical center of the lens 10 with respect to the optical center of the image sensor 20 is G1, and the position coordinates of position G1 are (X... lim1 ,0),X lim1 This is the first distance.

[0211] in, Figure 12 The second phase gain calibration matrix Q1 shown can actually include the second left phase gain calibration submatrix Q. 1L Second right phase gain calibration submatrix Q 1R .

[0212] like Figure 12 As shown in (c), during calibration, the optical center of lens 10 is offset by a second distance relative to the optical center of image sensor 20 along the second direction; the second distance is the maximum distance that lens 10 can move along the second direction. In this case, the relative position of the optical center of lens 10 with respect to the optical center of image sensor 20 is G2, and the position coordinates of position G2 are (-X). lim2 ,0),X lim2 This is the second distance.

[0213] in, Figure 12 The third phase gain calibration matrix Q2 shown can actually include the third left phase gain calibration submatrix Q. 2L and the third right phase gain calibration submatrix Q 2R .

[0214] like Figure 12As shown in (d), during calibration, the optical center of lens 10 is offset by a third distance relative to the optical center of image sensor 20 along a third direction; the third distance is the maximum distance that lens 10 can move along the third direction. In this case, the relative position of the optical center of lens 10 with respect to the optical center of image sensor 20 is G3, and the position coordinates of position G3 are (0, Y). lim1 ), Y lim1 This is the third distance.

[0215] in, Figure 12 The fifth phase gain calibration matrix Q3 shown can actually include the second upper phase gain calibration submatrix Q. 3T Second lower phase gain calibration submatrix Q 3B .

[0216] like Figure 12 As shown in (e), during calibration, the optical center of lens 10 is offset by a fourth distance relative to the optical center of image sensor 20 along the fourth direction; the fourth distance is the maximum distance that lens 10 can move along the fourth direction. In this case, the relative position of the optical center of lens 10 with respect to the optical center of image sensor 20 is G4, and the position coordinates of position G4 are (0, -Y). lim2 ), Y lim2 This is the fourth distance.

[0217] in, Figure 12 The sixth phase gain calibration matrix Q4 shown can actually include the third upper phase gain calibration submatrix Q. 4T and the third lower phase gain calibration submatrix Q 4B .

[0218] In some embodiments, the first distance X lim1 With the second distance X lim2 They can be equal or unequal; the third distance Y lim1 Distance Y to the fourth lim2 The distances can be equal or unequal, and this application does not limit this. Furthermore, the first distance, the second distance, the third distance, and the fourth distance are all positive numbers.

[0219] In some embodiments, each phase gain calibration matrix comprises a first phase gain calibration sub-matrix and a second phase gain calibration sub-matrix. In the case that the phase gain calibration matrix is the first phase gain calibration matrix, the first phase gain calibration sub-matrix can be the first left phase gain calibration sub-matrix, and the second phase gain calibration sub-matrix can be the first right phase gain calibration sub-matrix; in the case that the phase gain calibration matrix is the second phase gain calibration matrix, the first phase gain calibration sub-matrix can be the second left phase gain calibration sub-matrix, and the second phase gain calibration sub-matrix can be the second right phase gain calibration sub-matrix; in the case that the phase gain calibration matrix is the third phase gain calibration matrix, the first phase gain calibration sub-matrix can be the third left phase gain calibration sub-matrix, and the second phase gain calibration sub-matrix can be the third right phase gain calibration sub-matrix. In the case that the phase gain calibration matrix is the fourth phase gain calibration matrix, the first phase gain calibration sub-matrix can be the first upper phase gain calibration sub-matrix, and the second phase gain calibration sub-matrix can be the first lower phase gain calibration sub-matrix; in the case that the phase gain calibration matrix is the fifth phase gain calibration matrix, the first phase gain calibration sub-matrix can be the second upper phase gain calibration sub-matrix, and the second phase gain calibration sub-matrix can be the second lower phase gain calibration sub-matrix; in the case that the phase gain calibration matrix is the sixth phase gain calibration matrix, the first phase gain calibration sub-matrix can be the third upper phase gain calibration sub-matrix, and the second phase gain calibration sub-matrix can be the third lower phase gain calibration sub-matrix.

[0220] wherein each compensation coefficient in the first phase gain calibration sub-matrix is calculated according to the maximum luminance value and the luminance value of each first image patch in the first phase test sub-image; each compensation coefficient in the second phase gain calibration sub-matrix is calculated according to the maximum luminance value and the luminance value of each second image patch in the second phase test sub-image; the maximum luminance value is the maximum value among the luminance value of each first image patch and the luminance value of each second image patch. The first phase test sub-image and the second phase test sub-image are obtained by splitting the phase test image; the phase test image is collected under the condition that the focusing position of the lens is the preset focusing position, and the relative position of the optical center of the lens and the optical center of the image sensor is the preset position.

[0221] In an implementation, each compensation coefficient in the first phase gain calibration sub-matrix is the ratio of the maximum luminance value and the luminance value of each first image patch in the first phase test sub-image; each compensation coefficient in the second phase gain calibration sub-matrix is the ratio of the maximum luminance value and the luminance value of each second image patch in the second phase test sub-image.

[0222] Taking the calibration process of the first phase gain calibration matrix as an example,Figure 13 The flowchart of the calibration process of the first phase gain calibration matrix provided by the embodiments of the present application is shown in FIG. 13. Referring to FIG. 13, the calibration process can specifically include the following steps: Figure 13

[0223] S1301, in the case that the focusing position of the lens is a preset focusing position, and the optical center of the lens and the optical center of the image sensor coincide along the optical axis direction of the camera, a phase detection pixel is used to capture a phase test image.

[0224] The focusing position of the lens in the camera is set as the preset focusing position, and the preset focusing position is at 1 / 2 of the optical travel of the lens; and the optical center of the lens and the optical center of the image sensor are set to coincide along the optical axis direction of the camera, i.e., the optical displacement between the optical center of the lens and the optical center of the image sensor is 0.

[0225] In the case that the focusing position of the lens is set at the preset focusing position, and the optical center of the lens and the optical center of the image sensor are set to coincide along the optical axis direction of the camera, a uniform light plate is photographed by the camera to capture a phase test image by a phase detection pixel in the camera.

[0226] When the uniform light plate is photographed, the object distance can be 10 mm (millimeters). The uniform light plate refers to a planar light source with uniform illuminance and consistent color temperature in the effective area.

[0227] S1302, the phase test image is split into a left phase test sub-image and a right phase test sub-image.

[0228] The phase test image can include the left phase test sub-image and the right phase test sub-image. The left phase test sub-image is obtained by separating the image captured by the left phase detection pixel from the phase test image; correspondingly, the right phase test sub-image is obtained by separating the image captured by the right phase detection pixel from the phase test image.

[0229] S1303, the brightness value of each first image block in the left phase test sub-image and the brightness value of each second image block in the right phase test sub-image are obtained.

[0230] The left phase test sub-image is divided into a plurality of first image blocks, each first image block including a plurality of pixels, and the average value of the brightness values of all pixels in each first image block is taken as the brightness value of this first image block.

[0231] Correspondingly, the right phase test sub-image is divided into a plurality of second image blocks, each second image block including a plurality of pixels, and the average value of the brightness values of all pixels in each second image block is taken as the brightness value of this second image block. ​

[0232] For example, the left phase test sub-image is divided into 17x13 first image patches, and the luminance value of the first image patch in the ith row and jth column is lpd ij which is actually the average of the luminance values of all pixels in the first image patch in the ith row and jth column. The right phase test sub-image is divided into 17x13 second image patches, and the luminance value of the second image patch in the ith row and jth column is rpd ij which is actually the average of the luminance values of all pixels in the second image patch in the ith row and jth column. Wherein, i and j are positive integers.

[0233] S1304, the ratio of the maximum luminance value to the luminance value of each first image patch in the left phase test sub-image is calculated to obtain each compensation coefficient in the first left phase gain calibration sub-matrix.

[0234] In some embodiments, the maximum luminance value is the maximum value in the luminance value of each first image patch in the left phase test sub-image and the luminance value of each second image patch in the right phase test sub-image.

[0235] Specifically, each compensation coefficient in the first left phase gain calibration sub-matrix can be calculated by the following formula:

[0236] lgain ij = max{lpd ij ,rpd ij} / lpd ij

[0237] Wherein, lgain ij is the compensation coefficient in the first left phase gain calibration sub-matrix located in the ith row and jth column, lpd ij is the luminance value of the first image patch in the ith row and jth column in the left phase test sub-image, rpd ij is the luminance value of the second image patch in the ith row and jth column in the right phase test sub-image.

[0238] For example, the left phase test sub-image is divided into 17x13 first image patches, and the right phase test sub-image is divided into 17x13 second image patches, which can make the first left phase gain calibration sub-matrix include 17x13 compensation coefficients.

[0239] S1305, the ratio of the maximum luminance value to the luminance value of each second image patch in the right phase test sub-image is calculated to obtain each compensation coefficient in the first right phase gain calibration sub-matrix.

[0240] Specifically, each compensation coefficient in the first right phase gain calibration sub-matrix can be calculated by the following formula:

[0241] rgain ij = max{lpd ij , rpd ij} / rpd ij

[0242] wherein rgain ij is a compensation coefficient in the first right phase gain calibration sub-matrix at the i-th row and the j-th column, lpd ij is a luminance value of the first image patch at the i-th row and the j-th column in the left phase test sub-image, and rpd ij is a luminance value of the second image patch at the i-th row and the j-th column in the right phase test sub-image.

[0243] For example, the left phase test sub-image is divided into 17x13 first image patches, and the right phase test sub-image is divided into 17x13 second image patches, so that the first right phase gain calibration sub-matrix includes 17x13 compensation coefficients.

[0244] In the embodiments of the present application, the left phase test sub-image is divided into a plurality of first image patches, and the luminance value of each first image patch is the average of the luminance values of all pixels in the first image patch, so that each first image patch can correspond to a compensation coefficient in the first left phase gain calibration sub-matrix. In addition, the right phase test sub-image is divided into a plurality of second image patches, and the luminance value of each second image patch is the average of the luminance values of all pixels in the second image patch, so that each second image patch can correspond to a compensation coefficient in the first right phase gain calibration sub-matrix. In this way, the parameter quantity of the first phase gain calibration matrix and other phase gain calibration matrices is relatively small, thereby reducing the calculation amount in subsequent calculation of the phase gain compensation matrix.

[0245] In summary, according to the steps of S1301 to S1305 described above, the first phase gain calibration matrix can be obtained. In actual application, if the optical displacement between the optical center of the lens and the optical center of the image sensor is equal to 0, the first phase gain calibration matrix can be used to perform luminance compensation on the first phase image, so that the luminance values of the pixels in the compensated first phase image are more uniform.

[0246] In the actual calibration process, the optical image stabilization module can be controlled to move the lens to the G0 position shown in (a) of FIG. 13A, the G1 position shown in (b) of FIG. 13B, and the G2 position shown in (c) of FIG. 13C in sequence. Figure 12 Figure 12 Figure 12

[0247] ​​​In a case that the focusing position of the lens is set at the preset focusing position, and the optical center of the lens and the optical center of the image sensor are set to coincide along the optical axis direction of the camera, the camera is used to capture the uniform light plate to acquire a phase test image through the phase detection pixels in the camera, and the steps of S1302 to S1305 are performed to obtain a first phase gain calibration matrix. The first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix.

[0248] Correspondingly, in a case that the focusing position of the lens is set at the preset focusing position, and the optical center of the lens is offset from the optical center of the image sensor by a first distance along a first direction, the camera is used to capture the uniform light plate to acquire a phase test image through the phase detection pixels in the camera, and the steps of S1302 to S1305 are performed to obtain a second phase gain calibration matrix. The second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix.

[0249] In a case that the focusing position of the lens is set at the preset focusing position, and the optical center of the lens is offset from the optical center of the image sensor by a second distance along a second direction, the camera is used to capture the uniform light plate to acquire a phase test image through the phase detection pixels in the camera, and the steps of S1302 to S1305 are performed to obtain a third phase gain calibration matrix. The third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix.

[0250] It is worth noting that in the calibration of the first phase gain calibration matrix, the second phase gain calibration matrix and the third phase gain calibration matrix, the first phase test sub-image refers to the left phase test sub-image, and the second phase test sub-image refers to the right phase test sub-image.

[0251] In the actual calibration process, the optical image stabilization module can also be controlled to move the lens to the G0 position shown in (a) of FIG. 13A, the G3 position shown in (d) of FIG. 13B, and the G4 position shown in (e) of FIG. 13B in sequence. Figure 12 Figure 12 Figure 12

[0252] ​​​In a case that the focusing position of the lens is set at the preset focusing position, and the optical center of the lens and the optical center of the image sensor are set to coincide along the optical axis direction of the camera, the camera is used to capture a uniform light plate to acquire a phase test image through the phase detection pixels in the camera. The phase test image is split into an upper phase test sub-image and a lower phase test sub-image; the brightness value of each first image block in the upper phase test sub-image and the brightness value of each second image block in the lower phase test sub-image are obtained; the ratio of the maximum brightness value to the brightness value of each first image block in the upper phase test sub-image is calculated to obtain each compensation coefficient in a first upper phase gain calibration sub-matrix; and the ratio of the maximum brightness value to the brightness value of each second image block in the lower phase test sub-image is calculated to obtain each compensation coefficient in a first lower phase gain calibration sub-matrix. The fourth phase gain calibration matrix includes the first upper phase gain calibration sub-matrix and the first lower phase gain calibration sub-matrix; and the maximum brightness value is the maximum value of the brightness value of each first image block in the upper phase test sub-image and the brightness value of each second image block in the lower phase test sub-image.

[0253] Correspondingly, in a case that the focusing position of the lens is set at the preset focusing position, and the optical center of the lens is offset from the optical center of the image sensor by a third distance along a third direction, the camera is used to capture a uniform light plate to acquire a phase test image through the phase detection pixels in the camera. The phase test image is split into an upper phase test sub-image and a lower phase test sub-image; the brightness value of each first image block in the upper phase test sub-image and the brightness value of each second image block in the lower phase test sub-image are obtained; the ratio of the maximum brightness value to the brightness value of each first image block in the upper phase test sub-image is calculated to obtain each compensation coefficient in a second upper phase gain calibration sub-matrix; and the ratio of the maximum brightness value to the brightness value of each second image block in the lower phase test sub-image is calculated to obtain each compensation coefficient in a second lower phase gain calibration sub-matrix. The fifth phase gain calibration matrix includes the second upper phase gain calibration sub-matrix and the second lower phase gain calibration sub-matrix.

[0254] In a case where the focusing position of the lens is set at the preset focusing position, and the optical center of the lens is offset from the optical center of the image sensor by the fourth distance in the fourth direction, the camera is used to capture the uniform light plate to acquire a phase test image through the phase detection pixels in the camera. The phase test image is split into an upper phase test sub-image and a lower phase test sub-image; the brightness value of each first image block in the upper phase test sub-image and the brightness value of each second image block in the lower phase test sub-image are acquired; the ratio of the maximum brightness value to the brightness value of each first image block in the upper phase test sub-image is calculated to obtain each compensation coefficient in a third upper phase gain calibration sub-matrix; and the maximum brightness value and the brightness value of each second image block in the lower phase test sub-image are calculated to obtain each compensation coefficient in a third lower phase gain calibration sub-matrix. The sixth phase gain calibration matrix includes the third upper phase gain calibration sub-matrix and the third lower phase gain calibration sub-matrix.

[0255] It is worth noting that when the fourth phase gain calibration matrix, the fifth phase gain calibration matrix and the sixth phase gain calibration matrix are calibrated, the first phase test sub-image refers to the upper phase test sub-image, and the second phase test sub-image refers to the lower phase test sub-image.

[0256] It should be noted that the camera used in the calibration process of the phase gain calibration matrix of the embodiments of the present application can be the same camera as the camera used to acquire the first phase image in the execution process of the phase focusing method, or can not be the same camera. In a case where the camera used in the calibration process of the phase gain calibration matrix is not the same camera as the camera used to acquire the first phase image in the execution process of the phase focusing method, the type of the camera used in the calibration process of the phase gain calibration matrix can be the same as the type of the camera used to acquire the first phase image in the execution process of the phase focusing method, for example, the structure of the camera used in the calibration process of the phase gain calibration matrix can be the same as the structure of the camera used to acquire the first phase image in the execution process of the phase focusing method.

[0257] After the plurality of phase gain calibration matrices are calibrated in the above manner, the electronic device can determine a phase gain compensation matrix according to the first coordinate and the plurality of phase gain calibration matrices.

[0258] In one case, for a left-right phase detection focusing scene, the electronic device can calculate a phase gain compensation matrix according to the first coordinate of the position of the optical image stabilization module at the target time, the first phase gain calibration matrix, the second phase gain calibration matrix and the third phase gain calibration matrix.

[0259] The first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix; the second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix; and the third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix. The first coordinate includes a first coordinate component OIS X .

[0260] Specifically, the electronic device can determine the phase gain compensation matrix in the following manner: the electronic device determines a first left phase gain compensation sub-matrix according to the first coordinate component, the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix, and the third left phase gain calibration sub-matrix; and the electronic device determines a first right phase gain compensation sub-matrix according to the first coordinate component, the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix, and the third right phase gain calibration sub-matrix.

[0261] In the scene of left-right phase detection focusing, the phase gain compensation matrix can include a first left phase gain compensation sub-matrix and a first right phase gain compensation sub-matrix.

[0262] In an implementation manner, the electronic device calculates the first left phase gain compensation sub-matrix according to the following formula:

[0263]

[0264] wherein GainMap L is the first left phase gain compensation sub-matrix, Q 0L is the first left phase gain calibration sub-matrix, Q 1L is the second left phase gain calibration sub-matrix, Q 2L is the third left phase gain calibration sub-matrix, X lim1 is the first distance, X lim2 is the second distance, OIS X is the first coordinate component.

[0265] Therefore, in the above formula, each compensation coefficient in the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix, and the third left phase gain calibration sub-matrix is used to calculate the compensation coefficient at the corresponding position in the first left phase gain compensation sub-matrix.

[0266] For example, the compensation coefficient in the i-th row and the j-th column in the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix, and the third left phase gain calibration sub-matrix is used to calculate the compensation coefficient in the i-th row and the j-th column in the first left phase gain compensation sub-matrix.

[0267] In the case that the first left phase gain calibration sub-matrix, the second left phase gain calibration sub-matrix and the third left phase gain calibration sub-matrix include 17x13 compensation coefficients, the calculated first left phase gain compensation sub-matrix also includes 17x13 compensation coefficients.

[0268] Correspondingly, the electronic device calculates the first right phase gain compensation sub-matrix by the following formula:

[0269]

[0270] wherein GainMap R is the first right phase gain compensation sub-matrix, Q 0R is the first right phase gain calibration sub-matrix, Q 1R is the second right phase gain calibration sub-matrix, Q 2R is the third right phase gain calibration sub-matrix, X lim1 is the first distance, X lim2 is the second distance, OIS X is the first coordinate component.

[0271] Therefore, in the above formula, each compensation coefficient in the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix and the third right phase gain calibration sub-matrix is used to calculate the compensation coefficient at the corresponding position in the first right phase gain compensation sub-matrix.

[0272] For example, the compensation coefficient in the i-th row and the j-th column in the first right phase gain compensation sub-matrix is calculated by using the compensation coefficient in the i-th row and the j-th column in the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix and the third right phase gain calibration sub-matrix respectively.

[0273] In the case that the first right phase gain calibration sub-matrix, the second right phase gain calibration sub-matrix and the third right phase gain calibration sub-matrix include 17x13 compensation coefficients, the calculated first right phase gain compensation sub-matrix also includes 17x13 compensation coefficients.

[0274] It should be understood that in the scene of left-right phase detection focusing, only the first phase gain calibration matrix, the second phase gain calibration matrix and the third phase gain calibration matrix obtained by pre-calibration can be used to calculate the phase gain compensation matrix to perform brightness compensation on the first phase image.

[0275] In another case, for the up-down phase detection focusing scene, the electronic device can calculate the phase gain compensation matrix according to the first coordinate of the position of the optical image stabilization module at the target moment, the fourth phase gain calibration matrix, the fifth phase gain calibration matrix and the sixth phase gain calibration matrix.

[0276] The fourth phase gain calibration matrix includes a first upper phase gain calibration sub-matrix and a first lower phase gain calibration sub-matrix; the fifth phase gain calibration matrix includes a second upper phase gain calibration sub-matrix and a second lower phase gain calibration sub-matrix; and the sixth phase gain calibration matrix includes a third upper phase gain calibration sub-matrix and a third lower phase gain calibration sub-matrix. The first coordinate includes a second coordinate component OIS Y .

[0277] Specifically, the electronic device can determine the phase gain compensation matrix in the following manner: the electronic device determines a first upper phase gain compensation sub-matrix according to the second coordinate component, the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix; and the electronic device determines a first lower phase gain compensation sub-matrix according to the second coordinate component, the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix.

[0278] In the scene of up-down phase detection focusing, the phase gain compensation matrix can include a first upper phase gain compensation sub-matrix and a first lower phase gain compensation sub-matrix.

[0279] In an implementation manner, the electronic device calculates the first upper phase gain compensation sub-matrix according to the following formula:

[0280]

[0281] wherein GainMap T is the first upper phase gain compensation sub-matrix, Q 0T is the first upper phase gain calibration sub-matrix, Q 3T is the second upper phase gain calibration sub-matrix, Q 4T is the third upper phase gain calibration sub-matrix, Y lim1 is the third distance, Y lim2 is the fourth distance, OIS Y is the second coordinate component.

[0282] Therefore, in the above formula, each compensation coefficient in the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix is used to calculate the compensation coefficient at the corresponding position in the first upper phase gain compensation sub-matrix.

[0283] For example, the compensation coefficient in the i-th row and the j-th column in the first upper phase gain calibration sub-matrix, the second upper phase gain calibration sub-matrix, and the third upper phase gain calibration sub-matrix is used to calculate the compensation coefficient in the i-th row and the j-th column in the first upper phase gain compensation sub-matrix.

[0284] In the case that the first, second and third upper phase gain calibration sub-matrices comprise 17x13 compensation coefficients, the calculated first upper phase gain compensation sub-matrix also comprises 17x13 compensation coefficients.

[0285] Correspondingly, the electronic device calculates the first lower phase gain compensation sub-matrix by the following formula:

[0286]

[0287] wherein GainMap B is the first lower phase gain compensation sub-matrix, Q 0B is the first lower phase gain calibration sub-matrix, Q 3B is the second lower phase gain calibration sub-matrix, Q 4B is the third lower phase gain calibration sub-matrix, Y lim1 is the third distance, Y lim2 is the fourth distance, OIS Y is the second coordinate component.

[0288] Therefore, in the above formula, each compensation coefficient in the first, second and third lower phase gain calibration sub-matrices is used to calculate the compensation coefficient at the corresponding position in the first lower phase gain compensation sub-matrix.

[0289] For example, the compensation coefficient in the i-th row and j-th column in the first lower phase gain compensation sub-matrix is calculated by using the compensation coefficient in the i-th row and j-th column in the first, second and third lower phase gain calibration sub-matrices.

[0290] In the case that the first, second and third lower phase gain calibration sub-matrices comprise 17x13 compensation coefficients, the calculated first lower phase gain compensation sub-matrix also comprises 17x13 compensation coefficients.

[0291] It should be understood that in the scene of up-down phase detection focusing, only the fourth, fifth and sixth phase gain calibration matrices obtained by pre-calibration can be required to calculate the phase gain compensation matrix to perform brightness compensation on the first phase image.

[0292] In the scene of four-phase detection focusing, the first phase gain calibration matrix, the second phase gain calibration matrix, the third phase gain calibration matrix, the fourth phase gain calibration matrix, the fifth phase gain calibration matrix, and the sixth phase gain calibration matrix are needed to be calibrated in advance, to calculate the phase gain compensation matrix corresponding to left and right phase detection focusing and the phase gain compensation matrix corresponding to up and down phase detection focusing, and to use the phase gain compensation matrix corresponding to left and right phase detection focusing and the phase gain compensation matrix corresponding to up and down phase detection focusing to sequentially perform brightness compensation on the first phase image.

[0293] In S1007, the electronic device compensates the first phase image by using the phase gain compensation matrix to obtain a second phase image.

[0294] After the electronic device calculates the phase gain compensation matrix, the electronic device can use the phase gain compensation matrix to perform brightness compensation on the first phase image to obtain a second phase image, so that the brightness values of the pixels in the second phase image obtained after brightness compensation are more uniform.

[0295] In some embodiments, the phase gain compensation matrix includes a first phase gain compensation sub-matrix and a second phase gain compensation sub-matrix. The electronic device can compensate the first phase image in the following manner to obtain a second phase image: the electronic device splits the first phase image into a first phase sub-image and a second phase sub-image; the electronic device adjusts the size of the first phase gain compensation sub-matrix to obtain a third phase gain compensation sub-matrix, the size of the third phase gain compensation sub-matrix being equal to the size of the first phase sub-image; the electronic device compensates the first phase sub-image by using the third phase gain compensation sub-matrix to obtain a third phase sub-image; the electronic device adjusts the size of the second phase gain compensation sub-matrix to obtain a fourth phase gain compensation sub-matrix, the size of the fourth phase gain compensation sub-matrix being equal to the size of the second phase sub-image; and the electronic device compensates the second phase sub-image by using the fourth phase gain compensation sub-matrix to obtain a fourth phase sub-image.

[0296] In an implementation, the electronic device can compensate the first phase sub-image in the following manner to obtain a third phase sub-image: the electronic device multiplies the brightness value of each pixel in the first phase sub-image by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image. Correspondingly, the electronic device can compensate the second phase sub-image in the following manner to obtain a fourth phase sub-image: the electronic device multiplies the brightness value of each pixel in the second phase sub-image by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image.

[0297] For example, the electronic device multiplies the luminance value of the pixel at the mth row and the nth column in the first phase sub-image by the compensation coefficient at the mth row and the nth column in the third phase gain compensation sub-matrix to obtain the luminance value of the pixel at the mth row and the nth column in the third phase sub-image. The electronic device multiplies the luminance value of the pixel at the mth row and the nth column in the second phase sub-image by the compensation coefficient at the mth row and the nth column in the fourth phase gain compensation sub-matrix to obtain the luminance value of the pixel at the mth row and the nth column in the fourth phase sub-image. Here, m and n are positive integers.

[0298] In another implementation, the electronic device can compensate the first phase sub-image to obtain the third phase sub-image in the following manner: the electronic device multiplies the luminance value of each pixel in the focus region in the first phase sub-image by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image. Correspondingly, the electronic device can compensate the second phase sub-image to obtain the fourth phase sub-image in the following manner: the electronic device multiplies the luminance value of each pixel in the focus region in the second phase sub-image by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image.

[0299] It can be understood that when the electronic device obtains the second coordinate of the focus point in the original image, it can also correspondingly obtain information such as the width of the focus frame and the height of the focus frame, so that the electronic device can determine the coordinate position of the focus region in the original image.

[0300] In this way, the electronic device can perform luminance compensation on the luminance value of each pixel in the focus region in the first phase sub-image to obtain the third phase sub-image, and perform luminance compensation on the luminance value of each pixel in the focus region in the second phase sub-image to obtain the fourth phase sub-image.

[0301] If the electronic device only performs luminance compensation on the luminance value of each pixel in the focus region in the first phase sub-image and the second phase sub-image, the calculation amount when performing luminance compensation on the first phase image can be reduced.

[0302] In the scene of left-right phase detection focusing, the first phase sub-image is a first left phase sub-image, the second phase sub-image is a first right phase sub-image, the third phase sub-image is a second left phase sub-image, and the fourth phase sub-image is a second right phase sub-image; the first phase gain compensation sub-matrix is a first left phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is a first right phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is a second left phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is a second right phase gain compensation sub-matrix. Here, the second phase image includes the second left phase sub-image and the second right phase sub-image.

[0303] Specifically, the electronic device splits the first phase image into a first left phase sub-image and a first right phase sub-image. The first left phase sub-image is obtained by separating the image captured by the left phase detection pixels from the first phase image; correspondingly, the first right phase sub-image is obtained by separating the image captured by the right phase detection pixels from the first phase image. The first left phase sub-image includes luminance values corresponding to a plurality of left phase detection pixels, and the first right phase sub-image includes luminance values corresponding to a plurality of right phase detection pixels.

[0304] Since the size of the first left phase sub-image may not be consistent with the size of the first left phase gain compensation sub-matrix, the electronic device can adjust the size of the first left phase gain compensation sub-matrix in a bilinear interpolation manner to obtain a second left phase gain compensation sub-matrix, and the size of the second left phase gain compensation sub-matrix is equal to the size of the first left phase sub-image.

[0305] For example, the first left phase gain compensation sub-matrix includes 17x13 compensation coefficients, and the size of the first left phase sub-image is 2000x2000 pixels. The electronic device can adjust the size of the first left phase gain compensation sub-matrix from 17x13 to 2000x2000 to obtain the second left phase gain compensation sub-matrix, i.e., the second left phase gain compensation sub-matrix includes 2000x2000 compensation coefficients.

[0306] After adjusting the size of the first left phase gain compensation sub-matrix to obtain the second left phase gain compensation sub-matrix, the electronic device compensates the first left phase sub-image using the second left phase gain compensation sub-matrix to obtain a second left phase sub-image.

[0307] In one implementation, the electronic device multiplies the luminance value of each pixel in the first left phase sub-image by the compensation coefficient at the corresponding position in the second left phase gain compensation sub-matrix to obtain the second left phase sub-image. In another implementation, the electronic device multiplies the luminance value of each pixel in the focus region in the first left phase sub-image by the compensation coefficient at the corresponding position in the second left phase gain compensation sub-matrix to obtain the second left phase sub-image.

[0308] Correspondingly, since the size of the first right phase sub-image may not be consistent with the size of the first right phase gain compensation sub-matrix, the electronic device can adjust the size of the first right phase gain compensation sub-matrix in a bilinear interpolation manner to obtain a second right phase gain compensation sub-matrix, and the size of the second right phase gain compensation sub-matrix is equal to the size of the first right phase sub-image.

[0309] For example, the first right phase gain compensation sub-matrix includes 17*13 compensation coefficients, and the size of the first right phase sub-image is 2000*2000 pixels. The electronic device can adjust the size of the first right phase gain compensation sub-matrix from 17*13 to 2000*2000, to obtain a second right phase gain compensation sub-matrix, i.e., the second right phase gain compensation sub-matrix includes 2000*2000 compensation coefficients.

[0310] After adjusting the size of the first right phase gain compensation sub-matrix to obtain the second right phase gain compensation sub-matrix, the electronic device compensates the first right phase sub-image by using the second right phase gain compensation sub-matrix to obtain a second right phase sub-image.

[0311] In one implementation, the electronic device multiplies the brightness value of each pixel in the first right phase sub-image by the compensation coefficient at the corresponding position in the second right phase gain compensation sub-matrix to obtain the second right phase sub-image. In another implementation, the electronic device multiplies the brightness value of each pixel in the focus region in the first right phase sub-image by the compensation coefficient at the corresponding position in the second right phase gain compensation sub-matrix to obtain the second right phase sub-image.

[0312] In the scene of up-down phase detection focusing, the first phase sub-image is a first up phase sub-image, the second phase sub-image is a first down phase sub-image, the third phase sub-image is a second up phase sub-image, and the fourth phase sub-image is a second down phase sub-image. The first phase gain compensation sub-matrix is a first up phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is a first down phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is a second up phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is a second down phase gain compensation sub-matrix. The second phase image includes the second up phase sub-image and the second down phase sub-image.

[0313] Specifically, the electronic device splits the first phase image into a first up phase sub-image and a first down phase sub-image. The first up phase sub-image is obtained by separating the image captured by the up phase detection pixels from the first phase image. Correspondingly, the first down phase sub-image is obtained by separating the image captured by the down phase detection pixels from the first phase image. The first up phase sub-image includes brightness values corresponding to a plurality of up phase detection pixels, and the first down phase sub-image includes brightness values corresponding to a plurality of down phase detection pixels.

[0314] Since the size of the first up phase sub-image may not be consistent with the size of the first up phase gain compensation sub-matrix, the electronic device can adjust the size of the first up phase gain compensation sub-matrix by using bilinear interpolation to obtain a second up phase gain compensation sub-matrix, and the size of the second up phase gain compensation sub-matrix is equal to the size of the first up phase sub-image.

[0315] For example, the first upper phase gain compensation sub-matrix includes 17*13 compensation coefficients, and the size of the first upper phase sub-image is 2000*2000 pixels. The electronic device can adjust the size of the first upper phase gain compensation sub-matrix from 17*13 to 2000*2000, to obtain a second upper phase gain compensation sub-matrix, that is, the second upper phase gain compensation sub-matrix includes 2000*2000 compensation coefficients.

[0316] After adjusting the size of the first upper phase gain compensation sub-matrix to obtain the second upper phase gain compensation sub-matrix, the electronic device compensates the first upper phase sub-image by using the second upper phase gain compensation sub-matrix to obtain a second upper phase sub-image.

[0317] In an implementation, the electronic device multiplies the brightness value of each pixel in the first upper phase sub-image by the compensation coefficient at the corresponding position in the second upper phase gain compensation sub-matrix to obtain the second upper phase sub-image. In another implementation, the electronic device multiplies the brightness value of each pixel in the focusing area in the first upper phase sub-image by the compensation coefficient at the corresponding position in the second upper phase gain compensation sub-matrix to obtain the second upper phase sub-image.

[0318] Correspondingly, since the size of the first lower phase sub-image and the size of the first lower phase gain compensation sub-matrix can be inconsistent, the electronic device can adjust the size of the first lower phase gain compensation sub-matrix by using the bilinear interpolation to obtain a second lower phase gain compensation sub-matrix, and the size of the second lower phase gain compensation sub-matrix is equal to the size of the first lower phase sub-image.

[0319] For example, the first lower phase gain compensation sub-matrix also includes 17*13 compensation coefficients, and the size of the first lower phase sub-image is 2000*2000 pixels. The electronic device can adjust the size of the first lower phase gain compensation sub-matrix from 17*13 to 2000*2000 to obtain a second lower phase gain compensation sub-matrix, that is, the second lower phase gain compensation sub-matrix includes 2000*2000 compensation coefficients.

[0320] After adjusting the size of the first lower phase gain compensation sub-matrix to obtain the second lower phase gain compensation sub-matrix, the electronic device compensates the first lower phase sub-image by using the second lower phase gain compensation sub-matrix to obtain a second lower phase sub-image.

[0321] In one implementation, the electronic device multiplies the luminance value of each pixel in the first lower phase sub-image by the compensation coefficient at the corresponding position in the second lower phase gain compensation sub-matrix to obtain the second lower phase sub-image. In another implementation, the electronic device multiplies the luminance value of each pixel in the focus region in the first lower phase sub-image by the compensation coefficient at the corresponding position in the second lower phase gain compensation sub-matrix to obtain the second lower phase sub-image.

[0322] In the scene of four-phase detection focusing, the electronic device can first obtain the second left phase sub-image and the second right phase sub-image by using the luminance compensation manner corresponding to the scene of left-right phase detection focusing, and then continue to obtain the second upper phase sub-image and the second lower phase sub-image by using the luminance compensation manner corresponding to the scene of up-down phase detection focusing after synthesizing the second left phase sub-image and the second right phase sub-image into one phase image.

[0323] Of course, in the scene of four-phase detection focusing, the electronic device can first obtain the second upper phase sub-image and the second lower phase sub-image by using the luminance compensation manner corresponding to the scene of up-down phase detection focusing, and then continue to obtain the second left phase sub-image and the second right phase sub-image by using the luminance compensation manner corresponding to the scene of left-right phase detection focusing after synthesizing the second upper phase sub-image and the second lower phase sub-image into one phase image.

[0324] S1008, the electronic device performs phase focusing based on the second phase image.

[0325] After the first phase image is compensated by using the phase gain compensation matrix to obtain the second phase image, the electronic device can determine phase difference information based on the second phase image obtained after luminance compensation, and then determine the defocus distance based on the phase difference information. The focus motor drives the lens to move based on the defocus distance to adjust the distance between the lens and the image sensor, thereby realizing phase focusing.

[0326] In the scene of left-right phase detection focusing, the second phase image includes the second left phase sub-image and the second right phase sub-image, and the phase difference information of the first movement direction can be determined through the second left phase sub-image and the second right phase sub-image. The first movement direction can be the horizontal direction (i.e., the left-right direction). Moreover, the defocus distance is determined based on the phase difference information of the first movement direction.

[0327] In the scene of up-down phase detection focusing, the second phase image includes the second upper phase sub-image and the second lower phase sub-image, and the phase difference information of the second movement direction can be determined through the second upper phase sub-image and the second lower phase sub-image. The second movement direction can be the vertical direction (i.e., the up-down direction). Moreover, the defocus distance is determined based on the phase difference information of the second movement direction.

[0328] In the scene of four-phase detection focusing, the phase difference information of the first moving direction can be determined by the second left phase sub-image and the second right phase sub-image, and the phase difference information of the second moving direction can be determined by the second upper phase sub-image and the second lower phase sub-image. And the defocus distance is determined based on the phase difference information of the first moving direction and the phase difference information of the second moving direction.

[0329] It can be understood that in the phase focusing process of the embodiment of the application, the second coordinate of the focus point in the step S1002 of acquiring the second coordinate of the focus point in the original image can refer to the second coordinate of the focus point in the N-1th frame of original image. The first coordinate of the position of the optical image stabilization module at the target time in the step S1005 of acquiring the first coordinate of the position of the optical image stabilization module at the target time is the first coordinate of the position of the optical image stabilization module at the target time in the Nth frame of original image acquisition process; since the positions of the focus point in the N-1th frame of original image and the focus point in the Nth frame of original image remain basically unchanged, the first coordinate of the position of the optical image stabilization module at the target time in the Nth frame of original image acquisition process can be determined by the second coordinate of the focus point in the N-1th frame of original image. And the step S1007 of compensating the first phase image by using the phase gain compensation matrix can also compensate the first phase image corresponding to the Nth frame of original image. In addition, the step S1008 of performing phase focusing based on the second phase image can perform phase focusing on the N+1th frame of original image. Wherein, N is an integer greater than 1.

[0330] In summary, the embodiment of the application can calculate the phase gain compensation matrix according to the first coordinate of the position of the optical image stabilization module at the target time and a plurality of phase gain calibration matrices when the optical image stabilization module is started to perform optical image stabilization, so as to perform brightness compensation on the first phase image collected by the phase detection pixel, which can make the brightness values of the pixels in the second phase image obtained after brightness compensation more uniform. In this way, when the phase difference information is determined by using the second phase image, the determined phase difference information can be more accurate, so as to accurately calculate the defocus distance corresponding to the phase difference information, thereby improving the focusing stability and the focusing accuracy in the optical image stabilization mode.

[0331] Taking the scene of left-right phase detection focusing as an example, Figure 14 For the case that the optical displacement between the optical center of the lens and the optical center of the image sensor is not equal to 0 when the optical image stabilization module in the camera is started, the schematic diagram of the embodiment of the application for performing brightness compensation on the first phase image by using the phase gain compensation matrix.

[0332] As Figure 14As shown in (a), when the camera activates the optical image stabilization module for optical image stabilization, and the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 can be d, where d is not equal to 0, the first phase image can be acquired through the phase detection pixels in the image sensor 20.

[0333] When the optical displacement between the optical center of lens 10 and the optical center of image sensor 20 is not equal to zero, the acquired first phase image is as follows: Figure 14 As shown in (b), it includes a first left-phase sub-image acquired by the left-phase detection pixels and a first right-phase sub-image acquired by the right-phase detection pixels. Figure 14 In the first phase image shown in (b), the horizontal axis represents multiple image blocks in the first left phase sub-image and the first right phase sub-image, and the vertical axis represents the brightness value of each image block in the first left phase sub-image and the first right phase sub-image. The brightness value is the average of the brightness values ​​of all pixels in each image block.

[0334] The embodiments of this application can be adopted as follows: Figure 14 The phase gain compensation matrix shown in (c) is for... Figure 14 The first phase image shown in (b) is subjected to brightness compensation, and the resulting second phase image is as follows: Figure 14 As shown in (d) in the figure.

[0335] exist Figure 14 In the phase gain compensation matrix shown in (c), the phase gain compensation matrix includes a first left phase gain compensation sub-matrix and a first right phase gain compensation sub-matrix. The horizontal axis represents multiple image blocks, and the vertical axis represents the compensation coefficient corresponding to each image block in the first left phase gain compensation sub-matrix and the first right phase gain compensation sub-matrix.

[0336] exist Figure 14 In the second phase image shown in (d), the second phase image includes a second left phase sub-image and a second right phase sub-image. The horizontal axis represents multiple image blocks in the second left and second right phase sub-images, and the vertical axis represents the brightness value of each image block in the second left and second right phase sub-images.

[0337] Specifically, it can be adopted Figure 14 The first left-side phase gain compensation submatrix shown in (c) is for... Figure 14 The first left-phase sub-image shown in (b) is subjected to brightness compensation to obtain... Figure 14 The second left-phase sub-image shown in (d) is used; Figure 14 The first right phase gain compensation submatrix shown in (c) is for... Figure 14the first right phase sub-image shown in (b) in FIG. 6 is subjected to brightness compensation to obtain a second right phase sub-image shown in (d) in FIG. 6. Figure 15 the first right phase sub-image shown in (b) in FIG. 6 is subjected to brightness compensation to obtain a second right phase sub-image shown in (d) in FIG. 6.

[0338] It can be seen that, in the case that the optical displacement between the optical center of the lens 10 and the optical center of the image sensor 20 is not equal to 0, the brightness values of the pixels in the first phase image collected by the phase detection pixels are not uniform. After the first phase image is subjected to brightness compensation by using the phase gain compensation matrix, the brightness values of the pixels in the second phase image obtained after compensation are more uniform. In this way, when the phase difference information is determined by using the second phase image, the determined phase difference information is more accurate, so that the defocus distance corresponding to the phase difference information can be accurately calculated, thereby improving the focusing stability and the focusing accuracy in the optical anti-shake mode.

[0339] Figure 15 A flowchart for brightness compensation of the first phase image is provided for the embodiments of the present application. As shown in Figures 9 to 15 The electronic device can include a processor, a memory and a camera, and the camera can include an image sensor, a driving chip in an optical anti-shake module and a Hall sensor, etc. The Hall sensor is electrically connected with the driving chip in the optical anti-shake module, and the driving chip in the optical anti-shake module, the image sensor and the memory are respectively electrically connected with the processor. For example, the image sensor and the processor can be electrically connected through a mobile industry processor interface (MIPI).

[0340] The processor can be a central processor or a coprocessor, and the memory can be an electrically erasable programmable read only memory (EEPROM).

[0341] The Hall sensor is used to collect Hall data, which can be used to indicate the coordinates of the position of the optical anti-shake module after each movement. In this way, the Hall sensor can collect a first coordinate of the position of the optical anti-shake module at a target time. The Hall sensor can send the first coordinate of the position of the optical anti-shake module at the target time to the driving chip in the optical anti-shake module, and the driving chip in the optical anti-shake module can send the first coordinate of the position of the optical anti-shake module at the target time to the processor.

[0342] The phase detection pixels in the image sensor can collect a first phase image, and the image sensor can send the collected first phase image to the processor.

[0343] The memory can store multiple phase gain calibration matrices, such as the first phase gain calibration matrix, the second phase gain calibration matrix, the third phase gain calibration matrix, the fourth phase gain calibration matrix, the fifth phase gain calibration matrix, and the sixth phase gain calibration matrix.

[0344] The processor can execute the brightness compensation algorithm in the phase-detection autofocus method provided in this application embodiment. Specifically, the processor can acquire a first phase image from the image sensor, acquire the first coordinates of the position of the optical image stabilization module at the target time from the driver chip in the optical image stabilization module, and acquire multiple phase gain calibration matrices, including a first phase gain calibration matrix, a second phase gain calibration matrix, a third phase gain calibration matrix, a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix, from the memory. This allows the processor to execute the brightness compensation algorithm to determine the phase gain compensation matrix based on the first coordinates of the position of the optical image stabilization module at the target time and the multiple phase gain calibration matrices, and then use the phase gain compensation matrix to compensate the first phase image to obtain a second phase image.

[0345] The above combination Figure 16 The phase focusing method provided in the embodiments of this application has been described. The apparatus for performing the above method provided in the embodiments of this application is described below. Figure 16 As shown, Figure 16 This is a schematic diagram of a phase focusing device provided in an embodiment of this application. The phase focusing device may be an electronic device as described in this application embodiment, or a chip or chip system within an electronic device.

[0346] like Figure 17 As shown, the phase focusing device 1600 may include a processing unit 1601. The processing unit 1601 is used to support the phase focusing device 1600 in performing the above-described processing steps.

[0347] Specifically, processing unit 1601 is used to acquire a first phase image captured by phase detection pixels when the optical image stabilization module is activated; processing unit 1601 is used to acquire the first coordinates of the position of the optical image stabilization module at a target time, where the target time is the time when the focus point in the original image captured by the image sensor is captured, and the original image corresponds to the first phase image; processing unit 1601 is used to determine a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, where the relative positions of the optical center of the lens and the optical center of the image sensor are different during calibration; processing unit 1601 is used to compensate the first phase image using the phase gain compensation matrix to obtain a second phase image; processing unit 1601 is used to perform phase focusing based on the second phase image.

[0348] In a possible implementation, the phase focusing device 1600 further includes a storage unit 1602. The storage unit 1602 and the processing unit 1601 are connected through a line. The storage unit 1602 can include one or more memories, and the memory can be a device for storing programs or data in one or more devices or circuits. The storage unit 1602 can exist independently and be connected to the processing unit 1601 through a communication bus. The storage unit 1602 can also be integrated with the processing unit 1601.

[0349] The storage unit 1602 can store computer execution instructions of the method in the electronic device, so as to enable the processing unit 1601 to perform the method in the above-described embodiments. The storage unit 1602 can be a register, a cache or a random access memory (RAM), and the storage unit 1602 can be integrated with the processing unit 1601. The storage unit 1602 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, and the storage unit 1602 can be independent of the processing unit 1601.

[0350] Figure 17 A structure schematic diagram of a chip provided by the embodiments of the present application is shown in FIG. 17. As shown in FIG. 17, the chip 1700 includes one or more than two (including two) processors 1701, a communication line 1702 and a communication interface 1703. Optionally, the chip 1700 further includes a memory 1704. Figure 1

[0351] In some embodiments, the memory 1704 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0352] The method described in the above embodiments of the present application can be applied to the processor 1701 or implemented by the processor 1701. The processor 1701 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 1701. The processor 1701 described above can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, a discrete gate or a transistor logic device, or a discrete hardware component, and the processor 1701 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. ​

[0353] The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware code processing executed by a processor, or a combination of hardware and software modules in the code processing. The software module can be located in a storage medium such as a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The storage medium is located in the storage 1704, and the processor 1701 reads information in the storage 1704 and combines hardware to complete the steps of the above method.

[0354] The processor 1701, the storage 1704, and the communication interface 1703 can communicate through the communication line 1702.

[0355] In the above embodiments, the instructions stored in the storage for the processor to execute can be implemented in the form of a computer program product. The computer program product can be written in the storage in advance, or downloaded and installed in the storage in the form of software.

[0356] The embodiments of the present application also provide a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that the computer can store or the data storage device such as a server, data center, etc. integrated with one or more available media. For example, the available media can include magnetic media (such as floppy disk, hard disk or magnetic tape), optical media (such as digital versatile disc (DVD)), or semiconductor media (such as solid state disk (SSD)) and the like.

[0357] The embodiments of the present application provide an electronic device, which includes a processor and a storage. The storage is configured to store a computer program, and the processor is configured to execute the computer program to perform the phase focusing method described above.

[0358] The embodiment of the present application provides a chip. The chip comprises a processor, and the processor is used to call a computer program in a memory to execute the technical solutions in the above embodiment. The implementation principle and technical effects are similar to those of the above related embodiments, and will not be repeated here.

[0359] The embodiment of the present application further provides a computer readable storage medium. The computer readable storage medium stores computer programs or instructions. The computer programs or instructions are executed by the processor to realize the above method. The method described in the above embodiment can be realized by software, hardware, firmware or any combination thereof in whole or in part. If realized in software, the functions can be stored as one or more instructions or codes on a computer readable medium or transmitted on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can also include any medium that can transfer computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0360] As a possible design, the computer readable medium can include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disk storage; the computer readable medium can include a magnetic disk storage or other magnetic disk storage device. Moreover, any connection line can also be appropriately referred to as a computer readable medium. For example, if software is transmitted from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a DSL or a wireless technology (such as infrared, radio and microwave), the coaxial cable, the optical fiber cable, the twisted pair, the DSL or the wireless technology (such as infrared, radio and microwave) is included in the definition of the medium. As used herein, the disk and the optical disk include a compact disc (CD), a laser disc, an optical disc, a DVD, a floppy disk and a Blu-ray disc, wherein the disk is usually reproduced in a magnetic manner, and the optical disk is optically reproduced by laser. The combination of the above should also be included in the scope of the computer readable medium.

[0361] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the methods, devices (systems) and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks ​ one flow or multiple flows and / or blocks.

[0362] The above detailed description has been given to the purpose of further explaining the object, technical solutions and beneficial effects of the present application. It should be understood that the above is only a specific implementation of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A phase-detection focusing method, characterized in that, The method is applied to an electronic device, the electronic device including a camera, the camera including an optical image stabilization module, a lens, and an image sensor, the image sensor including multiple phase detection pixels; the method includes: When the optical image stabilization module is activated, the electronic device acquires the first phase image captured by the phase detection pixel; The electronic device acquires the first coordinates of the position of the optical image stabilization module at the target time; the target time is the time when the focus point in the original image acquired by the image sensor is located at the time of acquisition, and the original image corresponds to the first phase image; The electronic device determines a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices; the relative positions of the optical center of the lens and the optical center of the image sensor are different during calibration of the multiple phase gain calibration matrices. The electronic device uses the phase gain compensation matrix to compensate the first phase image to obtain a second phase image; The electronic device performs phase focusing based on the second phase image.

2. The method according to claim 1, characterized in that, Before the electronic device acquires the first coordinates of the position of the optical image stabilization module at the target time, it further includes: The electronic device acquires the second coordinates of the focus point in the original image; The electronic device determines the target duration based on the second coordinates; The electronic device determines the target time by summing the initial time when it begins acquiring the original image with the target duration.

3. The method according to claim 2, characterized in that, The electronic device determines the target duration based on the second coordinates, including: The electronic device calculates the target duration using the following formula: Among them, T lag The target duration is EIT, the exposure duration of the original image is Y. n Let H be the coordinate component along the height direction of the original image in the second coordinate system, where H is the height of the original image, and RowNum is the total number of rows in the photosensitive unit array of the image sensor. time The reading time for reading image data generated by a row of photosensitive units in the photosensitive unit array.

4. The method according to claim 1, characterized in that, The movable direction of the optical image stabilization module includes a first movable direction and a second movable direction, the first movable direction and the second movable direction are perpendicular to each other, and both the first movable direction and the second movable direction are perpendicular to the optical axis of the camera; the first movable direction includes a first direction and a second direction that are opposite to each other, and the second movable direction includes a third direction and a fourth direction that are opposite to each other; The plurality of phase gain calibration matrices include a first type of phase gain calibration matrix and / or a second type of phase gain calibration matrix; the first type of phase gain calibration matrix includes a first phase gain calibration matrix, a second phase gain calibration matrix, and a third phase gain calibration matrix; the second type of phase gain calibration matrix includes a fourth phase gain calibration matrix, a fifth phase gain calibration matrix, and a sixth phase gain calibration matrix; Wherein, during the calibration of the first phase gain calibration matrix and the fourth phase gain calibration matrix, the optical center of the lens coincides with the optical center of the image sensor along the optical axis of the camera; When calibrating the second phase gain calibration matrix, the optical center of the lens is offset by a first distance relative to the optical center of the image sensor along the first direction; the first distance is the maximum distance that the lens can move along the first direction. During calibration, the optical center of the lens is offset by a second distance relative to the optical center of the image sensor along the second direction; the second distance is the maximum distance the lens can move along the second direction. During calibration, the optical center of the lens is offset by a third distance relative to the optical center of the image sensor along the third direction; the third distance is the maximum distance the lens can move along the third direction. During calibration, the optical center of the lens is offset by a fourth distance relative to the optical center of the image sensor along the fourth direction; the fourth distance is the maximum distance the lens can move along the fourth direction.

5. The method according to claim 4, characterized in that, The first phase gain calibration matrix includes a first left phase gain calibration sub-matrix and a first right phase gain calibration sub-matrix; the second phase gain calibration matrix includes a second left phase gain calibration sub-matrix and a second right phase gain calibration sub-matrix; the third phase gain calibration matrix includes a third left phase gain calibration sub-matrix and a third right phase gain calibration sub-matrix; the first coordinate includes a first coordinate component along the first movement direction; The electronic device determines a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, including: The electronic device determines the first left phase gain compensation submatrix based on the first coordinate component, the first left phase gain calibration submatrix, the second left phase gain calibration submatrix, and the third left phase gain calibration submatrix; The electronic device determines the first right phase gain compensation submatrix based on the first coordinate component, the first right phase gain calibration submatrix, the second right phase gain calibration submatrix, and the third right phase gain calibration submatrix.

6. The method according to claim 5, characterized in that, The electronic device determines a first left-phase gain compensation sub-matrix based on the first coordinate component, the first left-phase gain calibration sub-matrix, the second left-phase gain calibration sub-matrix, and the third left-phase gain calibration sub-matrix, including: The electronic device calculates the first left-phase gain compensation sub-matrix using the following formula: Among them, GainMap L Let Q be the first left-phase gain compensation submatrix. 0L Q is the first left phase gain calibration submatrix. 1L Q is the second left phase gain calibration submatrix. 2L Let X be the third left-phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This refers to the first coordinate component; Accordingly, the electronic device determines a first right-phase gain compensation sub-matrix based on the first coordinate component, the first right-phase gain calibration sub-matrix, the second right-phase gain calibration sub-matrix, and the third right-phase gain calibration sub-matrix, including: The electronic device calculates the first right-phase gain compensation sub-matrix using the following formula: Among them, GainMap R Let Q be the first right phase gain compensation submatrix. 0R Q is the first right phase gain calibration submatrix. 1R Q is the second right phase gain calibration submatrix. 2R Let X be the third right phase gain calibration submatrix. lim1 Let X be the first distance. lim2 For the second distance, OIS X This is the first coordinate component.

7. The method according to claim 4, characterized in that, The fourth phase gain calibration matrix includes a first upper phase gain calibration sub-matrix and a first lower phase gain calibration sub-matrix; the fifth phase gain calibration matrix includes a second upper phase gain calibration sub-matrix and a second lower phase gain calibration sub-matrix; the sixth phase gain calibration matrix includes a third upper phase gain calibration sub-matrix and a third lower phase gain calibration sub-matrix; the first coordinate includes a second coordinate component along the second movement direction; The electronic device determines a phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, including: The electronic device determines the first upper phase gain compensation submatrix based on the second coordinate component, the first upper phase gain calibration submatrix, the second upper phase gain calibration submatrix, and the third upper phase gain calibration submatrix; The electronic device determines the first lower phase gain compensation submatrix based on the second coordinate component, the first lower phase gain calibration submatrix, the second lower phase gain calibration submatrix, and the third lower phase gain calibration submatrix.

8. The method according to claim 7, characterized in that, The electronic device determines a first upper phase gain compensation submatrix based on the second coordinate component, the first upper phase gain calibration submatrix, the second upper phase gain calibration submatrix, and the third upper phase gain calibration submatrix, including: The electronic device calculates the first upper phase gain compensation sub-matrix using the following formula: Among them, GainMap T Let Q be the first upper phase gain compensation submatrix. 0T Q is the first upper phase gain calibration submatrix. 3T Q is the second upper phase gain calibration submatrix. 4T Y is the third upper phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS Y This is the second coordinate component; Accordingly, the electronic device determines a first lower phase gain compensation sub-matrix based on the second coordinate component, the first lower phase gain calibration sub-matrix, the second lower phase gain calibration sub-matrix, and the third lower phase gain calibration sub-matrix, including: The electronic device calculates the first lower phase gain compensation sub-matrix using the following formula: Among them, GainMap B Let Q be the first lower phase gain compensation submatrix. 0B Let Q be the first lower phase gain calibration submatrix. 3B Q is the second lower phase gain calibration submatrix. 4B Y is the third lower phase gain calibration submatrix. lim1 For the third distance, Y lim2 For the fourth distance, OIS Y This is the second coordinate component.

9. The method according to any one of claims 1 to 8, characterized in that, The phase gain compensation matrix includes a first phase gain compensation submatrix and a second phase gain compensation submatrix; the electronic device uses the phase gain compensation matrix to compensate the first phase image to obtain a second phase image, including: The electronic device splits the first phase image into a first phase sub-image and a second phase sub-image; The electronic device adjusts the size of the first phase gain compensation sub-matrix to obtain a third phase gain compensation sub-matrix; the size of the third phase gain compensation sub-matrix is ​​equal to the size of the first phase sub-image. The electronic device uses the third phase gain compensation sub-matrix to compensate the first phase sub-image to obtain the third phase sub-image; The electronic device adjusts the size of the second phase gain compensation sub-matrix to obtain a fourth phase gain compensation sub-matrix; the size of the fourth phase gain compensation sub-matrix is ​​equal to the size of the second phase sub-image. The electronic device uses the fourth phase gain compensation sub-matrix to compensate the second phase sub-image to obtain the fourth phase sub-image; Wherein, the first phase sub-image is the first left phase sub-image, the second phase sub-image is the first right phase sub-image, the third phase sub-image is the second left phase sub-image, and the fourth phase sub-image is the second right phase sub-image; the first phase gain compensation sub-matrix is ​​the first left phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​the first right phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​the second left phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​the second right phase gain compensation sub-matrix; Alternatively, the first phase sub-image is a first upper phase sub-image, the second phase sub-image is a first lower phase sub-image, the third phase sub-image is a second upper phase sub-image, and the fourth phase sub-image is a second lower phase sub-image; the first phase gain compensation sub-matrix is ​​a first upper phase gain compensation sub-matrix, the second phase gain compensation sub-matrix is ​​a first lower phase gain compensation sub-matrix, the third phase gain compensation sub-matrix is ​​a second upper phase gain compensation sub-matrix, and the fourth phase gain compensation sub-matrix is ​​a second lower phase gain compensation sub-matrix.

10. The method according to claim 9, characterized in that, The electronic device uses the third phase gain compensation sub-matrix to compensate the first phase sub-image to obtain a third phase sub-image, including: The electronic device uses the brightness value of each pixel in the first phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix to obtain the third phase sub-image; Accordingly, the electronic device uses the fourth phase gain compensation sub-matrix to compensate the second phase sub-image to obtain the fourth phase sub-image, including: The electronic device uses the brightness value of each pixel in the second phase sub-image and multiplies it by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix to obtain the fourth phase sub-image.

11. The method according to claim 9, characterized in that, The electronic device uses the third phase gain compensation sub-matrix to compensate the first phase sub-image to obtain a third phase sub-image, including: The electronic device uses the brightness value of each pixel in the focus area of ​​the first phase sub-image, multiplied by the compensation coefficient at the corresponding position in the third phase gain compensation sub-matrix, to obtain the third phase sub-image; Accordingly, the electronic device uses the fourth phase gain compensation sub-matrix to compensate the second phase sub-image to obtain the fourth phase sub-image, including: The electronic device uses the brightness value of each pixel in the focus area of ​​the second phase sub-image, multiplied by the compensation coefficient at the corresponding position in the fourth phase gain compensation sub-matrix, to obtain the fourth phase sub-image.

12. The method according to any one of claims 1 to 8, characterized in that, Before the electronic device determines the phase gain compensation matrix based on the first coordinates and multiple phase gain calibration matrices, the method further includes: The electronic device acquires the pre-calibrated plurality of phase gain calibration matrices; each of the phase gain calibration matrices includes a first phase gain calibration submatrix and a second phase gain calibration submatrix; Wherein, each compensation coefficient in the first phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each first image block in the first phase test sub-image; each compensation coefficient in the second phase gain calibration sub-matrix is ​​calculated based on the maximum brightness value and the brightness value of each second image block in the second phase test sub-image; the maximum brightness value is the maximum value among the brightness values ​​of each first image block and each second image block; The first phase test sub-image and the second phase test sub-image are obtained by splitting the phase test image; the phase test image is a phase image acquired when the focus position of the lens is a preset focus position and the relative position between the optical center of the lens and the optical center of the image sensor is a preset position.

13. The method according to claim 12, characterized in that, Each compensation coefficient in the first phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each first image block in the first phase test sub-image; Each compensation coefficient in the second phase gain calibration sub-matrix is ​​the ratio of the maximum brightness value to the brightness value of each second image block in the second phase test sub-image.

14. An electronic device, characterized in that, It includes a memory and a processor, the memory being used to store a computer program, and the processor being used to invoke the computer program to perform the phase focusing method as described in any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions that, when executed, implement the phase focusing method as described in any one of claims 1 to 13.

16. A computer program product, characterized in that, Includes a computer program, which, when run, causes a computer to perform the phase focusing method as described in any one of claims 1 to 13.

Images