Focus control method, device, image sensor, electronic device and computer-readable storage medium

Abstract

The present application relates to a focus control method and device, an image sensor, an electronic device, and a computer-readable storage medium, which are applied to an image sensor. The method includes: determining a phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different. According to the phase information output mode, a phase array corresponding to a pixel array is output; wherein, the phase array includes phase information corresponding to a target pixel in the pixel array. The phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference. Therefore, the accuracy of the phase information output under different light intensities can be improved, thereby improving the accuracy of the focus control.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a focus control method, device, image sensor, electronic device and computer-readable storage medium. Background Art

[0002] With the development of electronic devices, more and more users use electronic devices to capture images. In order to ensure the clarity of the captured images, it is usually necessary to focus the camera module of the electronic device, that is, to adjust the distance between the lens and the image sensor so that the image of the subject is on the focal plane. Traditional focusing methods include phase detection auto focus (English: phase detection auto focus; abbreviated: PDAF).

[0003] Traditional phase detection autofocus mainly calculates the phase difference based on the RGB pixel array, and then controls the motor based on the phase difference. The motor then drives the lens to move to the appropriate position for focusing so that the subject is imaged on the focal plane.

[0004] However, since the sensitivity of the RGB pixel array is different under different light intensities, the accuracy of the phase difference calculated by the RGB pixel array is low under certain light intensities, which in turn greatly reduces the accuracy of focusing. Summary of the invention

[0005] Embodiments of the present application provide a focus control method, device, electronic device, image sensor, and computer-readable storage medium, which can improve the accuracy of focus.

[0006] In one aspect, an image sensor is provided, the image sensor comprising a microlens array, a pixel array and a filter array, the filter array comprising a minimum repeating unit, the minimum repeating unit comprising a plurality of filter groups, the filter groups comprising a color filter and a panchromatic filter; the color filter has a narrower spectral response than the panchromatic filter, the color filter and the panchromatic filter both comprising 9 sub-filters arranged in an array;

[0007] The pixel array includes a plurality of pixel groups, the pixel groups are full-color pixel groups or color pixel groups, each of the full-color pixel groups corresponds to the full-color filter, and each of the color pixel groups corresponds to the color filter; the full-color pixel group and the color pixel group each include 9 pixels, the pixels of the pixel array are arranged corresponding to the sub-filters of the filter array, and each pixel corresponds to a photosensitive element;

[0008] The microlens array includes a plurality of microlens groups, each microlens group corresponds to the full-color pixel group or the color pixel group, the microlens group includes a plurality of microlenses, and at least one microlens among the plurality of microlenses corresponds to at least two pixels.

[0009] On the other hand, a focus control method is provided, which is applied to the image sensor as described above, and the method comprises:

[0010] According to the light intensity of the current shooting scene, determine a phase information output mode adapted to the light intensity of the current shooting scene; wherein in different phase information output modes, the size of the output phase array is different;

[0011] According to the phase information output mode, output a phase array corresponding to the pixel array; wherein the phase array includes phase information corresponding to a target pixel in the pixel array;

[0012] The phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference.

[0013] On the other hand, a focus control device is provided, which is applied to the image sensor as described above, and the device comprises:

[0014] A phase information output mode determination module, used to determine a phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different;

[0015] A phase array output module, configured to output a phase array corresponding to the pixel array according to the phase information output mode; wherein the phase array includes phase information corresponding to a target pixel in the pixel array;

[0016] A focus control module is used to calculate the phase difference of the pixel array based on the phase array, and perform focus control according to the phase difference.

[0017] On the other hand, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the focus control method as described above.

[0018] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0019] On the other hand, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the steps of the focus control method as described above when executed by a processor.

[0020] The above-mentioned focus control method, device, image sensor, electronic device and computer-readable storage medium determine the phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different. According to the phase information output mode, a phase array corresponding to the pixel array is output; wherein the phase array includes the phase information corresponding to the target pixel in the pixel array. The phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference.

[0021] Under different light intensities of the current shooting scene, the accuracy of the original phase information that can be collected is different. Therefore, according to the light intensity of the current shooting scene, different phase information output modes can be used for the same pixel array, and phase arrays of different sizes can be output based on the original phase information. Since the signal-to-noise ratio of phase arrays of different sizes is different, the accuracy of the phase information output under different light intensities can be improved, thereby improving the accuracy of focus control. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a schematic diagram of the structure of an electronic device in one embodiment;

[0024] Figure 2 This is a schematic diagram of the principle of phase detection autofocus;

[0025] Figure 3 A schematic diagram of arranging phase detection pixels in pairs among pixels included in an image sensor;

[0026] Figure 4 is an exploded schematic diagram of an image sensor in one embodiment;

[0027] Figure 5 is a connection diagram of a pixel array and a readout circuit in one embodiment;

[0028] Figure 6A schematic diagram of an arrangement of a minimum repeating unit of a pixel array in one embodiment;

[0029] Figure 7 A schematic diagram of an arrangement of a minimum repeating unit of a pixel array in another embodiment;

[0030] Figure 8 is a schematic diagram of an arrangement of the minimum repeating unit of a microlens array in one embodiment;

[0031] Fig. 9 is a schematic diagram of an arrangement of the minimum repeating unit of a microlens array in another embodiment;

[0032] Fig.10 is a flow chart of a focus control method in one embodiment;

[0033] Fig.11 A flowchart of a method for outputting a phase array corresponding to a pixel array according to a phase information output mode in one embodiment;

[0034] Fig.12 for Fig.11 A flowchart of a method for determining a phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene;

[0035] Fig.13 for Fig.12 A flow chart of a method for determining a phase information output mode adapted to the light intensity of a current shooting scene according to a target light intensity range;

[0036] Fig.14 A schematic diagram of generating a full-size phase array in one embodiment;

[0037] Fig.15 A flowchart of a method for generating a first-size phase array in one embodiment;

[0038] Fig.16 A schematic diagram of generating a second-size phase array in one embodiment;

[0039] Fig.17 is a structural block diagram of a focus control device in one embodiment;

[0040] Fig.18 for Fig.17 The structural block diagram of the phase array output module;

[0041] Fig.19 FIG. 1 is a schematic diagram of the internal structure of an electronic device in an embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] It is understood that the terms "first", "second", "third", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first size may be referred to as a second size, and similarly, a second size may be referred to as a first size. Both the first size and the second size are sizes, but they are not the same size. The first preset threshold may be referred to as the second preset threshold, and similarly, the second preset threshold may be referred to as the first preset threshold. Both the first preset threshold and the second preset threshold are preset thresholds, but they are not the same preset threshold.

[0044] Figure 1 FIG. 1 is a schematic diagram of an application environment of a focus control method in an embodiment. Figure 1 As shown, the application environment includes an electronic device 100. The electronic device 100 includes an image sensor, and the image sensor includes a pixel array. The electronic device determines a phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different; according to the phase information output mode, a phase array corresponding to the pixel array is output; wherein the phase array includes phase information corresponding to the pixel array; the phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference. The electronic device can be any terminal device with image processing function, such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a wearable device (smart bracelet, smart watch, smart glasses, smart gloves, smart socks, smart belt, etc.), a VR (virtual reality) device, a smart home, an unmanned vehicle, etc.

[0045] Among them, the electronic device 100 includes a camera 20, a processor 30 and a housing 40. The camera 20 and the processor 30 are both arranged in the housing 40. The housing 40 can also be used to install functional modules such as a power supply device and a communication device of the terminal 100, so that the housing 40 provides dustproof, drop-proof, waterproof and other protections for the functional modules. The camera 20 can be a front camera, a rear camera, a side camera, an under-screen camera, etc., which is not limited here. The camera 20 includes a lens and an image sensor 21. When the camera 20 takes an image, light passes through the lens and reaches the image sensor 21. The image sensor 21 is used to convert the light signal irradiated on the image sensor 21 into an electrical signal.

[0046] Figure 2 Schematic diagram of the principle of phase detection auto focus (PDAF). Figure 2 As shown, M1 is the position of the image sensor when the imaging device is in a focused state, wherein the focused state refers to a state of successful focusing. When the image sensor is at position M1, the imaging light g reflected from the object W to the lens Lens in different directions converges on the image sensor, that is, the imaging light g reflected from the object W to the lens Lens in different directions forms an image at the same position on the image sensor, and at this time, the image sensor forms a clear image.

[0047] M2 and M3 are possible positions of the image sensor when the imaging device is not in focus, such as Figure 2 As shown in FIG. 1 , when the image sensor is located at position M2 or position M3, the imaging light g reflected from the object W to the lens Lens in different directions will form images at different positions. Figure 2 When the image sensor is located at position M2, the imaging light g reflected from the object W to the lens Lens in different directions forms images at positions A and B respectively. When the image sensor is located at position M3, the imaging light g reflected from the object W to the lens Lens in different directions forms images at positions C and D respectively. At this time, the image sensor image is not clear.

[0048] In PDAF technology, the positional differences of the images formed by the imaging light rays entering the lens from different directions in the image sensor can be obtained, for example, Figure 2 As shown, the difference between position A and position B can be obtained, or the difference between position C and position D can be obtained; after obtaining the difference in position of the image formed in the image sensor by the imaging light entering the lens from different directions, the defocus distance can be obtained according to the difference and the geometric relationship between the lens and the image sensor in the camera. The so-called defocus distance refers to the distance between the current position of the image sensor and the position where the image sensor should be in the focused state; the imaging device can focus according to the obtained defocus distance.

[0049] It can be seen that when the focus is achieved, the calculated PD value is 0. Conversely, the larger the calculated value, the farther the position is from the focus point, and the smaller the value, the closer the focus point is. When using PDAF to focus, the defocus distance can be obtained by calculating the PD value and then calibrating the corresponding relationship between the PD value and the defocus distance. Then, the lens is controlled to move to the focus point according to the defocus distance to achieve focus.

[0050] In the related art, some phase detection pixels may be arranged in pairs among the pixels included in the image sensor, such as Figure 3 As shown, the image sensor may be provided with phase detection pixel pairs (hereinafter referred to as pixel pairs) A, pixel pairs B and pixel pairs C. In each pixel pair, one phase detection pixel performs left shielding (English: Left Shield) and the other phase detection pixel performs right shielding (English: Right Shield).

[0051] For a phase detection pixel point that is blocked on the left side, only the right side of the imaging light beam directed to the phase detection pixel point can form an image on the photosensitive part (that is, the unblocked part) of the phase detection pixel point. For a phase detection pixel point that is blocked on the right side, only the left side of the imaging light beam directed to the phase detection pixel point can form an image on the photosensitive part (that is, the unblocked part) of the phase detection pixel point. In this way, the imaging light beam can be divided into two parts, the left and the right, and the phase difference can be obtained by comparing the images formed by the left and the right imaging light beams.

[0052] In one embodiment, it is further described that the image sensor includes a pixel array and a filter array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, the filter groups include color filters and panchromatic filters, in the filter groups, the color filters are arranged in a first diagonal direction, the panchromatic filters are arranged in a second diagonal direction, and the first diagonal direction is different from the second diagonal direction; the color filter has a narrower spectral response than the panchromatic filter, and the color filter and the panchromatic filter both include 9 sub-filters arranged in an array;

[0053] Among them, the pixel array includes multiple full-color pixel groups and multiple color pixel groups, each full-color pixel group corresponds to a full-color filter, and each color pixel group corresponds to a color filter; the full-color pixel group and the color pixel group each include 9 pixels, the pixels of the pixel array are arranged corresponding to the sub-filters of the filter array, and each pixel corresponds to a photosensitive element.

[0054] The microlens array includes a plurality of microlens groups, each microlens group corresponds to a full-color pixel group or a color pixel group, the microlens group includes a plurality of microlenses, and at least one microlens among the plurality of microlenses corresponds to at least two pixels.

[0055] In the embodiment of the present application, since at least one microlens among the plurality of microlenses corresponds to at least two pixels, the phase difference of the pixel array can be calculated based on the phase information of the at least two pixels.

[0056] like Figure 4 As shown, the image sensor 21 includes a microlens array 22 , a filter array 23 , and a pixel array 24 .

[0057] The microlens array 22 includes a plurality of minimum repeating units 221, the minimum repeating unit 221 includes a plurality of microlens groups 222, and the microlens group 222 includes a plurality of microlenses 2221. The sub-filters in the filter array 23 and the pixels in the pixel array 24 are arranged in a one-to-one correspondence, and at least one microlens in the plurality of microlenses 2221 corresponds to at least two pixels. The microlens 2221 is used to gather the incident light, and the gathered light passes through the corresponding sub-filter and is then projected onto the pixel and received by the corresponding pixel, which then converts the received light into an electrical signal.

[0058] The filter array 23 includes a plurality of minimal repeating units 231. The minimal repeating unit 231 may include a plurality of filter groups 232. Each filter group 232 includes a panchromatic filter 233 and a color filter 234, and the color filter 234 has a narrower spectral response than the panchromatic filter 233. Each panchromatic filter 233 includes nine sub-filters 2331, and each color filter 234 includes nine sub-filters 2341. Different filter groups also include different color filters 234.

[0059] The colors corresponding to the wavelength band of the light transmitted by the color filter 234 of the filter group 232 in the minimum repeating unit 231 include color a, color b and / or color c. The colors corresponding to the wavelength band of the light transmitted by the color filter 234 of the filter group 232 include color a, color b and color c, or color a, color b or color c, or color a and color b, or color b and color c, or color a and color c. Among them, color a is red, color b is green, and color c is blue, or for example, color a is magenta, color b is cyan, and color c is yellow, etc., which is not limited here.

[0060] In one embodiment, the width of the wavelength band of the light transmitted by the color filter 234 is smaller than the width of the wavelength band of the light transmitted by the full-color filter 233. For example, the wavelength band of the light transmitted by the color filter 234 may correspond to the wavelength band of red light, the wavelength band of green light, or the wavelength band of blue light, and the wavelength band of the light transmitted by the full-color filter 233 is the wavelength band of all visible light. In other words, the color filter 234 only allows light of a specific color to be transmitted, while the full-color filter 233 can pass light of all colors. Of course, the wavelength band of the light transmitted by the color filter 234 may also correspond to the wavelength band of other colored light, such as magenta light, purple light, cyan light, yellow light, etc., which is not limited here.

[0061] In one embodiment, the ratio of the number of color filters 234 to the number of full-color filters 233 in the filter group 232 can be 1:3, 1:1 or 3:1. For example, if the ratio of the number of color filters 234 to the number of full-color filters 233 is 1:3, then the number of color filters 234 is 1, and the number of full-color filters 233 is 3. At this time, the number of full-color filters 233 is large. Compared with the traditional case of only color filters, more phase information can be obtained through the full-color filters 233 in dim light, so the focusing quality is better; or, if the ratio of the number of color filters 234 to the number of full-color filters 233 is 1:1, then the number of color filters 234 is 1, and the number of full-color filters 233 is 3. The number of color filters 234 is 2, and the number of full-color filters 233 is 2. At this time, better color performance can be obtained, and more phase information can be obtained through the full-color filter 233 in dim light, so the focus quality is also better; or, the ratio of the number of color filters 234 to the number of full-color filters 233 is 3:1, then the number of color filters 234 is 3, and the number of full-color filters 233 is 1. At this time, better color performance can be obtained, and similarly, the focus quality in dim light can also be improved.

[0062] The pixel array 24 includes a plurality of pixels, and the pixels of the pixel array 24 are arranged corresponding to the sub-filters of the filter array 23. The pixel array 24 is configured to receive light passing through the filter array 23 to generate an electrical signal.

[0063] The pixel array 24 is configured to receive the light passing through the filter array 23 to generate an electrical signal, which means that the pixel array 24 is used to perform photoelectric conversion on the light of a given set of scenes of the photographed object passing through the filter array 23 to generate an electrical signal. The light of a given set of scenes of the photographed object is used to generate image data. For example, if the photographed object is a building, the given set of scenes of the photographed object refers to the scene where the building is located, and the scene may also contain other objects.

[0064] In one embodiment, the pixel array 24 may be an RGBW pixel array, including a plurality of minimum repeating units 241, wherein the minimum repeating unit 241 includes a plurality of pixel groups 242, wherein the plurality of pixel groups 242 include a panchromatic pixel group 243 and a color pixel group 244. Each panchromatic pixel group 243 includes 9 panchromatic pixels 2431, and each color pixel group 244 includes 9 color pixels 2441. Each panchromatic pixel 2431 corresponds to a sub-filter 2331 in the panchromatic filter 233, and the panchromatic pixel 2431 receives light passing through the corresponding sub-filter 2331 to generate an electrical signal. Each color pixel 2441 corresponds to a sub-filter 2341 of the color filter 234, and the color pixel 2441 receives light passing through the corresponding sub-filter 2341 to generate an electrical signal. Each pixel corresponds to a photosensitive element. That is, each panchromatic pixel 2431 corresponds to a photosensitive element; each color pixel 2441 corresponds to a photosensitive element.

[0065] The image sensor 21 in this embodiment includes a filter array 23 and a pixel array 24. The filter array 23 includes a minimum repeating unit 231. The minimum repeating unit 231 includes multiple filter groups 232. The filter groups include a panchromatic filter 233 and a color filter 234. The color filter 234 has a narrower spectral response than the panchromatic filter 233, so that more light can be obtained during shooting, thereby eliminating the need to adjust shooting parameters. The focus quality in dark light is improved without affecting the stability of shooting. When focusing in dark light, both stability and quality can be taken into account, and the stability and quality of focusing in dark light are both high. Furthermore, each full-color filter 233 includes 9 sub-filters 2331, each color filter 234 includes 9 sub-filters 2341, the pixel array 24 includes multiple full-color pixels 2431 and multiple color pixels 2441, each full-color pixel 2431 corresponds to a sub-filter 2331 of the full-color filter 233, and each color pixel 2441 corresponds to a sub-filter 2341 of the color filter 234, the full-color pixels 2431 and the color pixels 2441 are used to receive light passing through the corresponding sub-filters to generate electrical signals, and when focusing in dim light, the phase information of the pixels corresponding to the 9 sub-filters can be combined and output to obtain phase information with a higher signal-to-noise ratio, and in scenes with sufficient light, the phase information of the pixels corresponding to each sub-filter can be output separately to obtain phase information with higher resolution and signal-to-noise ratio, so that it can adapt to different application scenarios and improve the focusing quality in each scenario.

[0066] In one embodiment, Figure 4As shown, the minimum repeating unit 231 in the filter array 23 includes four filter groups 232, and the four filter groups 232 are arranged in a matrix. Each filter group 232 includes a full-color filter 233 and a color filter 234, and each full-color filter 233 and each color filter 234 has 9 sub-filters, so the filter group 232 includes 36 sub-filters in total.

[0067] Similarly, the pixel array 24 includes a plurality of minimal repeating units 241 corresponding to the plurality of minimal repeating units 231. Each minimal repeating unit 241 includes four pixel groups 242, and the four pixel groups 242 are arranged in a matrix. Each pixel group 242 corresponds to a filter group 232.

[0068] like Figure 5 As shown, the readout circuit 25 is electrically connected to the pixel array 24, and is used to control the exposure of the pixel array 24 and the reading and output of the pixel values ​​of the pixel points. The readout circuit 25 includes a vertical drive unit 251, a control unit 252, a column processing unit 253 and a horizontal drive unit 254. The vertical drive unit 251 includes a shift register and an address decoder. The vertical drive unit 251 includes a readout scan and a reset scan function. The control unit 252 configures the timing signal according to the operation mode, and uses a variety of timing signals to control the vertical drive unit 251, the column processing unit 253 and the horizontal drive unit 254 to work together. The column processing unit 253 may have an analog-to-digital (A / D) conversion function for converting an analog pixel signal into a digital format. The horizontal drive unit 254 includes a shift register and an address decoder. The horizontal drive unit 254 sequentially scans the pixel array 24 column by column.

[0069] In one embodiment, Figure 6 As shown, each filter group 232 includes a color filter 234 and a full-color filter 233. Each full-color filter 233 in the filter group 232 is arranged in a first diagonal direction D1, and each color filter 234 in the filter group 232 is arranged in a second diagonal direction. The first diagonal D1 direction and the second diagonal D2 direction are different, which can take into account both color performance and low-light focusing quality.

[0070] The direction of the first diagonal line D1 is different from the direction of the second diagonal line D2. Specifically, the direction of the first diagonal line D1 is not parallel to the direction of the second diagonal line D2, or the direction of the first diagonal line D1 is perpendicular to the direction of the second diagonal line D2.

[0071] In other embodiments, one color filter 234 and one panchromatic filter 233 may be located on the first diagonal line D1 , and another color filter 234 and another panchromatic filter 233 may be located on the second diagonal line D2 .

[0072] In one embodiment, each pixel corresponds to a photosensitive element. The photosensitive element is an element that can convert a light signal into an electrical signal. For example, the photosensitive element may be a photodiode. Figure 6 As shown, each full-color pixel 2331 includes 1 photodiode PD (PhotoDiode), and each color pixel 2341 includes 1 photodiode PD (PhotoDiode).

[0073] In one embodiment, Figure 6 As shown, the minimum repeating unit 231 in the filter array 23 includes four filter groups 232, and the four filter groups 232 are arranged in a matrix. Each filter group 232 includes two full-color filters 233 and two color filters 234. The full-color filter 233 includes nine sub-filters 2331, and the color filter 234 includes nine sub-filters 2341. The minimum repeating unit 231 is 12 rows and 12 columns of 144 sub-filters, and the arrangement is as follows:

[0074]

[0075] Wherein, w represents the full-color sub-filter 2331, and a, b and c all represent the color sub-filter 2341. The full-color sub-filter 2331 refers to a sub-filter that can filter out all light outside the visible light band, and the color sub-filter 2341 includes a red sub-filter, a green sub-filter, a blue sub-filter, a magenta sub-filter, a cyan sub-filter and a yellow sub-filter. The red sub-filter is a sub-filter that filters out all light except red light, the green sub-filter is a sub-filter that filters out all light except green light, the blue sub-filter is a sub-filter that filters out all light except blue light, the magenta sub-filter is a sub-filter that filters out all light except magenta light, the cyan sub-filter is a sub-filter that filters out all light except cyan light, and the yellow sub-filter is a sub-filter that filters out all light except yellow light.

[0076] a can be a red sub-filter, a green sub-filter, a blue sub-filter, a magenta sub-filter, a cyan sub-filter or a yellow sub-filter, b can be a red sub-filter, a green sub-filter, a blue sub-filter, a magenta sub-filter, a cyan sub-filter or a yellow sub-filter, and c can be a red sub-filter, a green sub-filter, a blue sub-filter, a magenta sub-filter, a cyan sub-filter or a yellow sub-filter. For example, b is a red sub-filter, a is a green sub-filter, and c is a blue sub-filter; or, c is a red sub-filter, a is a green sub-filter, and b is a blue sub-filter; for another example, c is a red sub-filter, a is a green sub-filter, and b is a blue sub-filter; or, a is a red sub-filter, b is a blue sub-filter, and c is a green sub-filter, etc., which are not limited here; for another example, b is a magenta sub-filter, a is a cyan sub-filter, and b is a yellow sub-filter, etc. In other embodiments, the color filter may further include sub-filters of other colors, such as an orange sub-filter, a purple sub-filter, etc., which are not limited here.

[0077] In one embodiment, Figure 7 As shown, the minimum repeating unit 231 in the filter array 23 includes four filter groups 232, and the four filter groups 232 are arranged in a matrix. Each filter group 232 includes a color filter 234 and a full-color filter 233. Each color filter 234 in the filter group 232 is arranged in the direction of the first diagonal line D1, and each full-color filter 233 in the filter group 232 is arranged in the direction of the second diagonal line D2. And the pixel array ( Figure 7 Not shown, for reference Figure 6 ) are arranged correspondingly to the sub-filters of the filter array, and each pixel corresponds to a photosensitive element.

[0078] In one embodiment, each filter group 232 includes two full-color filters 233 and two color filters 234, the full-color filter 233 includes nine sub-filters 2331, and the color filter 234 includes nine sub-filters 2341. The minimum repeating unit 231 is 12 rows and 12 columns of 144 sub-filters. Figure 5 As shown, the arrangement is:

[0079]

[0080] Among them, w represents the full-color sub-filter, and a, b and c all represent color sub-filters.

[0081] 12 rows, 12 columns and 144 sub-filters combine the advantages of quad and RGBW. The advantage of quad is that it can locally merge the same pixels 2x2 and 3x3 (binning) to obtain images of different resolutions with high signal-to-noise ratio. The full-size quad output has high pixels, and a full-size and full-resolution image is obtained with higher clarity. The advantage of RGBW is that W pixels are used to increase the overall light intake of the image, thereby improving the image quality signal-to-noise ratio.

[0082] In one embodiment, Figure 8 As shown in (a), the microlens array 22 includes a plurality of minimum repeating units 221, the minimum repeating unit 221 includes a plurality of microlens groups 222, and the microlens group 222 includes a plurality of microlenses 2221. Each microlens group 222 corresponds to a full-color pixel group 243 or a color pixel group 244, and the microlens group 222 includes a plurality of microlenses 2211. For example, the microlens group 222 includes five microlenses 2211, and the five microlenses 2211 include four first microlenses 2211a and one second microlens 2211b; wherein the first microlenses 2211a correspond to two pixels respectively, and the second microlens 2211b corresponds to one pixel. And the four first microlenses include two first microlenses in a first direction and two first microlenses in a second direction; the first microlenses in the first direction are arranged along the first direction in the microlens group; and the first microlenses in the second direction are arranged along the second direction in the microlens group. The second microlens 2211 b is located at the center of the microlens group 222 , and the second microlens 2211 b corresponds to the central pixel of the pixel group corresponding to the microlens group.

[0083] like Figure 8 As shown in (a), if the second microlens 2211b is located at the center of the microlens group, then the two first microlenses ( Figure 8 (a) gray filling) and two first micro lenses in the second direction ( Figure 8 (a) oblique line filling) is arranged around the second microlens 2211b;

[0084] The two first microlenses in the first direction are centrally symmetrically arranged relative to the second microlens 2211b along a first diagonal direction; the two first microlenses in the second direction are centrally symmetrically arranged relative to the second microlens 2211b along a second diagonal direction.

[0085] like Figure 8As shown in (b), in another embodiment, the second microlens 2211b is located at one of the four corners of the microlens group, and the second microlens 2211b corresponds to a pixel at one of the four corners in the pixel group corresponding to the microlens group. For example, the second microlens 2211b is located at the lower right corner of the microlens group 222. Of course, the second microlens 2211b can also be located at the lower left corner of the microlens group 222; the second microlens 2211b can also be located at the upper left corner of the microlens group 222; the second microlens 2211b can also be located at the upper right corner of the microlens group 222, and this application does not limit this.

[0086] In one embodiment, one microlens in the plurality of microlenses corresponds to at least four pixels, that is, one microlens in the plurality of microlenses 2211 corresponds to at least four pixels.

[0087] In one embodiment, Fig. 9 As shown, for example, the microlens group 222 includes four microlenses 2211, and the four microlenses include one third microlens 2211c, two fourth microlenses 2211d, and one fifth microlens 2211e; wherein the third microlens 2211c corresponds to four pixels, the fourth microlens 2211d corresponds to two pixels, and the fifth microlens 2211e corresponds to one pixel. The fourth microlens 2211d here can be the same microlens as the first microlens 2211a; the fifth microlens 2211e here can be the same microlens as the second microlens 2211b, and the present application does not limit this.

[0088] In one embodiment, Fig.10 As shown, a focus control method is provided, which is applied to an image sensor as in the above embodiment, the image sensor includes a pixel array and a filter array, and the method includes:

[0089] Step 1020, determining a phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different.

[0090] In different shooting scenes or at different times, the light intensity of the current shooting scene is not the same. Since the sensitivity of the RGB pixel array is different under different light intensities, the phase difference calculated by the RGB pixel array is less accurate under some light intensities, which in turn greatly reduces the accuracy of focusing. Among them, light intensity is also called illumination intensity. Light intensity is a physical term that refers to the luminous flux of visible light received per unit area, referred to as illuminance, and the unit is lux (Lux or lx). Light intensity is used to indicate the intensity of light and the degree of illumination of the surface area of ​​an object. The following table shows the light intensity values ​​under different weather and locations:

[0091] Table 1-1

[0092] Weather and location Light intensity value Sunny day with direct sunlight on the ground 100000lx Sunny day indoor center 200lx Cloudy outdoor 50-500lx Cloudy indoor 5-50lx Moonlight (Full Moon) 2500lx Clear Moon Night 0.2lx Night 0.0011lx

[0093] It can be seen from Table 1-1 above that the light intensity of the current shooting scene varies greatly in different shooting scenes or at different times.

[0094] In order to solve the problem that the phase difference calculated by the RGB pixel array is less accurate under certain light intensities, which leads to a significant reduction in the accuracy of focusing, under different light intensities of the current shooting scene, a phase information output mode adapted to the light intensity of the current shooting scene is determined, and then different phase information output modes are used to output the phase information of the pixel array. The phase information output mode refers to a mode in which the original phase information of the pixel array is processed to generate the final output phase information of the pixel array.

[0095] Among them, in different phase information output modes, the size of the phase array output is different. That is, under different light intensities of the current shooting scene, the size of the phase array output by the same pixel array is different. In other words, under different light intensities of the current shooting scene, the phase information corresponding to the same pixel array is directly output as the phase array corresponding to the pixel array or merged to a certain extent to generate the phase array corresponding to the pixel array. For example, if the light intensity of the current shooting scene is large, the phase information corresponding to the same pixel array can be directly output as the phase array corresponding to the pixel array. At this time, the size of the phase array output is equal to the size of the pixel array. If the light intensity of the current shooting scene is small, the phase information corresponding to the same pixel array can be merged to a certain extent to generate the phase array corresponding to the pixel array. At this time, the size of the phase array output is smaller than the size of the pixel array.

[0096] Since the signal-to-noise ratios of phase arrays of different sizes are different, the accuracy of the phase information output under different light intensities can be improved, thereby improving the accuracy of focusing.

[0097] Step 1040: output a phase array corresponding to the pixel array according to the phase information output mode; wherein the phase array includes phase information corresponding to the target pixel in the pixel array.

[0098] After determining the phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene, the phase information corresponding to the pixel array can be output according to the phase information output mode. Specifically, when outputting the phase information corresponding to the pixel array, it can be output in the form of a phase array. The phase array includes the phase information corresponding to the pixel array.

[0099] Specifically, under different light intensities in the current shooting scene, the phase information corresponding to the same pixel array is directly output as the phase array corresponding to the pixel array or merged to a certain extent to generate the phase array corresponding to the pixel array according to the phase information output mode adapted to the light intensity. This application does not limit this.

[0100] Step 1060: Calculate the phase difference of the pixel array based on the phase array, and perform focus control according to the phase difference.

[0101] After outputting the phase array corresponding to the pixel array according to the phase information output mode, the phase difference of the pixel array can be calculated based on the phase information in the phase array. Assuming that the phase array of the pixel array in the second direction can be obtained, the phase difference is calculated based on the two adjacent phase information in the second direction, and finally the phase difference of the entire pixel array in the second direction is obtained. Assuming that the phase array of the pixel array in the first direction can be obtained, the phase difference is calculated based on the two adjacent phase information in the first direction, and finally the phase difference of the entire pixel array in the first direction is obtained, and the second direction is different from the first direction. Among them, the second direction can be the vertical direction of the pixel array, the first direction can be the horizontal direction of the pixel array, and the second direction is perpendicular to the first direction. Of course, the phase difference of the entire pixel array in the second direction and the first direction can be obtained at the same time, and the phase difference of the pixel array in other directions can also be calculated, such as the diagonal direction (including the first diagonal direction, and the second diagonal direction perpendicular to the first diagonal direction), etc., and this application does not limit this.

[0102] When the focus control is performed based on the calculated phase difference, since the collected phase difference parallel to the texture feature in a certain direction on the preview image corresponding to the current shooting scene is almost 0, it is obviously impossible to focus based on the collected phase difference parallel to the direction. Therefore, if the preview image corresponding to the current shooting scene includes the texture feature in the first direction, the phase difference of the pixel array in the second direction is calculated based on the phase array of the pixel array in the second direction. The focus control is performed according to the phase difference of the pixel array in the second direction.

[0103] For example, assuming that the second direction is the vertical direction of the pixel array, the first direction is the horizontal direction of the pixel array, and the second direction is perpendicular to the first direction. Then, the preview image includes texture features in the first direction, which means that the preview image includes horizontal stripes, which can be solid color, horizontal stripes. At this time, the preview image corresponding to the current shooting scene includes texture features in the horizontal direction, and focus control is performed based on the phase difference in the vertical direction.

[0104] If the preview image corresponding to the current shooting scene includes texture features in the second direction, focus control is performed based on the phase difference in the first direction. If the preview image corresponding to the current shooting scene includes texture features in the first diagonal direction, focus control is performed based on the phase difference in the second diagonal direction, and vice versa. In this way, the phase difference can be accurately collected for texture features in different directions, and then the focus can be accurately achieved.

[0105] In an embodiment of the present application, a phase information output mode adapted to the light intensity of the current shooting scene is determined according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different. According to the phase information output mode, a phase array corresponding to the pixel array is output; wherein, the phase array includes phase information corresponding to the pixel array. The phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference.

[0106] Under different light intensities of the current shooting scene, the accuracy of the original phase information that can be collected is different. Therefore, according to the light intensity of the current shooting scene, different phase information output modes can be used for the same pixel array, and phase arrays of different sizes can be output based on the original phase information. Since the signal-to-noise ratio of phase arrays of different sizes is different, the accuracy of the phase information output under different light intensities can be improved, thereby improving the accuracy of focus control.

[0107] In the previous embodiment, if Fig.11 As shown, step 1040, outputting a phase array corresponding to the pixel array according to the phase information output mode, includes:

[0108] Step 1042, according to the phase information output mode, determine a target pixel group from the color pixel group and the full-color pixel group in the pixel array, determine a target microlens corresponding to the target pixel group, and use at least two pixels corresponding to the target microlens as target pixels.

[0109] First, according to the light intensity of the current shooting scene, a phase information output mode adapted to the light intensity of the current shooting scene is determined. That is, the phase information output mode determined under different light intensities is different, and under different phase information output modes, the size of the output phase array is also different.

[0110] Secondly, according to the phase information output mode, the target pixel group is determined from the color pixel group and the full-color pixel group in the pixel array. Because the phase information output mode adapted to the light intensity of the current shooting scene is determined based on the light intensity of the current shooting scene, and the color pixels and the full-color pixels have different sensitivities under different light intensities, therefore, under different phase information output modes, the target pixel group can be determined from the color pixel group and the full-color pixel group in the pixel array.

[0111] Specifically, in different phase information output modes, all or part of the color pixel groups and the full-color pixel groups in the pixel array can be screened out as the target pixel groups. All or part of the full-color pixel groups can also be screened out as the target pixel groups from the color pixel groups and the full-color pixel groups in the pixel array. All or part of the color pixel groups and all or part of the full-color pixel groups can also be screened out as the target pixel groups from the color pixel groups and the full-color pixel groups in the pixel array, and this application does not limit this.

[0112] Finally, a target microlens corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens are used as target pixels.

[0113] The target pixel group includes a plurality of microlenses, some of which correspond to at least two pixels, and some of which correspond to one pixel. Therefore, it is necessary to determine the target microlens from the microlenses corresponding to at least two pixels, that is, all the microlenses corresponding to at least two pixels can be determined as the target microlenses, or some of the microlenses corresponding to at least two pixels can be screened out and determined as the target microlenses, which is not limited in the present application.

[0114] After the target microlens is determined, at least two pixels corresponding to the target microlens are used as target pixels, that is, at least two pixels corresponding to the target microlens are used as target pixels, or all pixels corresponding to the target microlens are used as target pixels, which is not limited in the present application.

[0115] Step 1044 , for each target pixel group, obtain the phase information of the target pixel.

[0116] Step 1046: Generate a phase array corresponding to the pixel array according to the phase information of the target pixel.

[0117] After the target pixel is determined from the target microlens, the phase information of the target pixel is obtained for each target pixel group. A phase array corresponding to the pixel array can be generated directly based on the phase information of the target pixel. The phase information of the target pixel can also be merged to a certain extent to generate a phase array corresponding to the pixel array, which is not limited in this application.

[0118] In the embodiment of the present application, different phase information output modes correspond to different light intensities, and the target pixel group can be determined from the color pixel group and the full-color pixel group in the pixel array under different phase information output modes. Thus, the target pixel group can be determined based on at least one pixel group in the color pixel group and the full-color pixel group. Then the target pixel is determined from the target pixel group, and a phase array corresponding to the pixel array is generated according to the phase information of the target pixel. The phase array corresponding to the pixel array is generated from the two dimensions of determining the target pixel group and determining the target pixel. Ultimately, the accuracy of the generated phase array is improved.

[0119] Continuing from the previous embodiment, step 1042, determining a target microlens corresponding to the target pixel group, and taking at least two pixels corresponding to the target microlens as target pixels, includes:

[0120] Determine a target microlens in a first direction corresponding to the target pixel group; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction;

[0121] At least two pixels corresponding to the target microlens in the first direction are used as target pixels.

[0122] After determining the target pixel group from the color pixel group and the full-color pixel group in the pixel array according to the phase information output mode, determine the target microlens corresponding to the target pixel group. Specifically, assuming that the full-color pixel group in the pixel array is determined as the target pixel group, the target microlens is determined from the microlens corresponding to the full-color pixel group. Because the target microlens corresponds to at least two pixels, combined with Figure 8 As shown, the microlenses corresponding to the full-color pixel group include four first microlenses 2211a and one second microlens 2211b, wherein the first microlenses 2211a correspond to two pixels respectively, and the second microlens 2211b corresponds to one pixel.

[0123] Therefore, a target microlens in the first direction is further determined from the four first microlenses 2211a. The target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction. Assuming that the first direction is a horizontal direction, then Figure 8 It can be seen that the four first microlenses 2211a include a first microlens 2211a in the first direction (filled in gray in the figure) and a first microlens 2211a in the second direction (filled in oblique lines in the figure). Then, the target microlens in the first direction corresponding to the target pixel group is determined to be the first microlens 2211a in the first direction (filled in gray in the figure). Of course, the first direction and the second direction here can be interchangeable, and this application does not limit this.

[0124] At least two pixels corresponding to the first microlens 2211a (filled in gray in the figure) in the first direction are used as target pixels. Then, for each target pixel group, the phase information of the target pixel in the second direction can be obtained. And according to the phase information of the target pixel, a phase array corresponding to the pixel array is generated.

[0125] Combination Fig. 9 As shown, the full-color pixel group includes four microlenses 2211, including one third microlens 2211c, two fourth microlenses 2211d and one fifth microlens 2211e; wherein the third microlens 2211c corresponds to four pixels, the fourth microlens 2211d corresponds to two pixels, and the fifth microlens 2211e corresponds to one pixel.

[0126] Therefore, the target microlens in the first direction is further determined from one third microlens 2211c and two fourth microlenses 2211d. Since the third microlens 2211c includes 4 pixels arranged in a 2×2 array, the third microlens 2211c can be considered as a microlens in the first direction or a microlens in the second direction. Then, the target microlens in the first direction corresponding to the target pixel group is determined to be the first microlens 2211a (filled in gray in the figure) and the third microlens 2211c in the first direction.

[0127] At least two pixels corresponding to the first microlens 2211a (filled in gray in the figure) in the first direction and at least four pixels corresponding to the third microlens 2211c are taken as target pixels. Then, for each target pixel group, the phase information of the target pixel in the second direction can be obtained. And according to the phase information of the target pixel, a phase array corresponding to the pixel array is generated.

[0128] In an embodiment of the present application, after determining the target pixel group from the color pixel group and the full-color pixel group in the pixel array according to the phase information output mode, the target microlens corresponding to the target pixel group is determined. Specifically, the microlens in the first direction can be determined as the target microlens from the microlenses corresponding to the target pixel group. Then, at least two pixels corresponding to the first microlens in the first direction can be used as target pixels. Then, the phase information of the target pixel in the second direction is obtained. And based on the phase information of the target pixel, a phase array corresponding to the pixel array is generated. Finally, for each target pixel group, the phase array corresponding to the pixel array in the second direction can be obtained.

[0129] Optionally, step 1042, determining a target microlens corresponding to the target pixel group, and taking at least two pixels corresponding to the target microlens as target pixels, further includes:

[0130] Determine a target microlens in a second direction corresponding to the target pixel group; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; the second direction is perpendicular to the first direction;

[0131] At least two pixels corresponding to the target microlenses in the second direction are used as target pixels.

[0132] After determining the target pixel group from the color pixel group and the full-color pixel group in the pixel array according to the phase information output mode, determine the target microlens corresponding to the target pixel group. Specifically, assuming that the full-color pixel group in the pixel array is determined as the target pixel group, the target microlens is determined from the microlens corresponding to the full-color pixel group. Because the target microlens corresponds to at least two pixels, combined with Figure 8 As shown, the microlenses corresponding to the full-color pixel group include four first microlenses 2211a and one second microlens 2211b, wherein the first microlenses 2211a correspond to two pixels respectively, and the second microlens 2211b corresponds to one pixel.

[0133] Therefore, a target microlens in the first direction is further determined from the four first microlenses 2211a. The target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction. The second direction is perpendicular to the first direction. Assuming that the first direction is a horizontal direction, the second direction is a vertical direction. Figure 8 It can be seen that the four first microlenses 2211a include the first microlens 2211a in the first direction (filled in gray in the figure) and the first microlens 2211a in the second direction (filled with oblique lines in the figure). Then, the target microlens in the second direction corresponding to the target pixel group is determined to be the first microlens 2211a in the second direction (filled with oblique lines in the figure). Of course, the first direction and the second direction here can be interchangeable, and this application does not limit this.

[0134] At least two pixels corresponding to the first microlens 2211a (filled with diagonal lines in the figure) in the second direction are used as target pixels. Then, for each target pixel group, the phase information of the target pixel in the second direction can be obtained. And according to the phase information of the target pixel, a phase array corresponding to the pixel array is generated.

[0135] Combination Fig. 9 As shown, the full-color pixel group includes four microlenses 2211, including one third microlens 2211c, two fourth microlenses 2211d and one fifth microlens 2211e; wherein the third microlens 2211c corresponds to four pixels, the fourth microlens 2211d corresponds to two pixels, and the fifth microlens 2211e corresponds to one pixel.

[0136] Therefore, the target microlens in the second direction is further determined from one third microlens 2211c and two fourth microlenses 2211d. Since the third microlens 2211c includes 4 pixels arranged in a 2×2 array, the third microlens 2211c can be considered as a microlens in the first direction or a microlens in the second direction. Then, the target microlens in the second direction corresponding to the target pixel group is determined to be the first microlens 2211a (filled with oblique lines in the figure) and the third microlens 2211c in the second direction.

[0137] At least two pixels corresponding to the first microlens 2211a (filled with oblique lines in the figure) in the second direction and at least four pixels corresponding to the third microlens 2211c are taken as target pixels. Then, for each target pixel group, the phase information of the target pixel in the first direction can be obtained. And according to the phase information of the target pixel, a phase array corresponding to the pixel array is generated.

[0138] In an embodiment of the present application, after determining the target pixel group from the color pixel group and the full-color pixel group in the pixel array according to the phase information output mode, the target microlens corresponding to the target pixel group is determined. Specifically, the microlens in the second direction can be determined as the target microlens from the microlens corresponding to the target pixel group. Then, at least two pixels corresponding to the first microlens in the second direction can be used as target pixels. Then, the phase information of the target pixel in the first direction is obtained. And based on the phase information of the target pixel, a phase array corresponding to the pixel array is generated. Finally, for each target pixel group, the phase array corresponding to the pixel array in the first direction can be obtained.

[0139] Optionally, step 1042, determining a target microlens corresponding to the target pixel group, and taking at least two pixels corresponding to the target microlens as target pixels, includes:

[0140] Determine a target microlens in a first direction and a target microlens in a second direction corresponding to the target pixel group; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; the second direction is perpendicular to the first direction;

[0141] At least two pixels corresponding to the target microlens in the first direction and at least two pixels corresponding to the target microlens in the second direction are used as target pixels.

[0142] After determining the target pixel group from the color pixel group and the full-color pixel group in the pixel array according to the phase information output mode, determine the target microlens corresponding to the target pixel group. Specifically, assuming that the full-color pixel group in the pixel array is determined as the target pixel group, the target microlens is determined from the microlens corresponding to the full-color pixel group. Because the target microlens corresponds to at least two pixels, combined with Figure 8 As shown, the microlenses corresponding to the full-color pixel group include four first microlenses 2211a and one second microlens 2211b, wherein the first microlenses 2211a correspond to two pixels respectively, and the second microlens 2211b corresponds to one pixel.

[0143] Therefore, a target microlens in the first direction and a target microlens in the second direction are further determined from the four first microlenses 2211a. The target microlens in the first direction corresponds to at least two target pixels arranged and adjacent to each other along the first direction, and the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent to each other along the first direction. Assuming that the first direction is the horizontal direction, then Figure 8 It can be seen that the four first microlenses 2211a include a first microlens 2211a in the first direction (gray filled in the figure) and a first microlens 2211a in the second direction (diagonal filled in the figure). Then, the target microlens in the first direction corresponding to the target pixel group is determined to be the first microlens 2211a in the first direction (gray filled in the figure), and the target microlens in the second direction corresponding to the target pixel group is determined to be the first microlens 2211a in the second direction (diagonal filled in the figure). Of course, the first direction and the second direction here can be interchangeable, and this application does not limit this.

[0144] At least two pixels corresponding to the first microlens 2211a in the first direction (filled in gray in the figure) and the first microlens 2211a in the second direction (filled in oblique lines in the figure) are taken as target pixels. Then, for each target pixel group, the phase information of the target pixel in the first direction and the phase information of the second direction can be obtained. And according to the phase information of the target pixel, a phase array corresponding to the pixel array is generated.

[0145] Combination Fig. 9 As shown, the full-color pixel group includes four microlenses 2211, including one third microlens 2211c, two fourth microlenses 2211d and one fifth microlens 2211e; wherein the third microlens 2211c corresponds to four pixels, the fourth microlens 2211d corresponds to two pixels, and the fifth microlens 2211e corresponds to one pixel.

[0146] Therefore, the target microlens in the first direction and the target microlens in the second direction are further determined from one third microlens 2211c and two fourth microlenses 2211d. Since the third microlens 2211c includes 4 pixels arranged in a 2×2 array, the third microlens 2211c can be considered as a microlens in the first direction or a microlens in the second direction. Then, the target microlens in the first direction and the target microlens in the second direction corresponding to the target pixel group are determined to be the first microlens 2211a in the first direction (filled with gray in the figure), the first microlens 2211a in the second direction (filled with oblique lines in the figure), and the third microlens 2211c.

[0147] The first microlens 2211a in the first direction (filled in gray in the figure), at least two pixels corresponding to the first microlens 2211a in the second direction (filled in oblique lines in the figure), and at least four pixels corresponding to the third microlens 2211c are taken as target pixels. Then, for each target pixel group, the phase information of the target pixel in the first direction and the second direction can be obtained. And according to the phase information of the target pixel, a phase array corresponding to the pixel array is generated.

[0148] In an embodiment of the present application, after determining the target pixel group from the color pixel group and the full-color pixel group in the pixel array according to the phase information output mode, the target microlens corresponding to the target pixel group is determined. Specifically, the microlens in the first direction can be determined as the target microlens from the microlens corresponding to the target pixel group. Then, at least two pixels corresponding to the first microlens in the first direction can be used as target pixels. Then, the phase information of the target pixel in the second direction is obtained. At least two pixels corresponding to the first microlens in the second direction can also be used as target pixels. Then, the phase information of the target pixel in the first direction is obtained. And based on the phase information of the target pixel, a phase array corresponding to the pixel array is generated. Finally, for each target pixel group, the phase array corresponding to the pixel array in the first direction and the second direction can be obtained. Thereby, the phase difference in the two directions can be calculated, and then focusing can be achieved in the two directions.

[0149] In the previous embodiment, it is described that according to the light intensity of the current shooting scene, different phase information output modes are used for the same pixel array, a phase array corresponding to the pixel array is output, and the phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference. In this embodiment, Fig.12 As shown, the specific implementation steps of step 1020, which are to determine the phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene, include:

[0150] Step 1022, determining the target light intensity range to which the light intensity of the current shooting scene belongs; wherein different light intensity ranges correspond to different phase information output modes;

[0151] Step 1024: Determine a phase information output mode that matches the light intensity of the current shooting scene according to the target light intensity range.

[0152] Specifically, the light intensity can be divided into different light intensity ranges in order of magnitude based on different preset thresholds of light intensity. The preset threshold of light intensity can be determined according to exposure parameters and the size of pixels in the pixel array. The exposure parameters include shutter speed, lens aperture size, and light sensitivity (ISO).

[0153] Then, different phase information output modes are set for different light intensity ranges. Specifically, according to the order of light intensity in the light intensity range, the sizes of the phase arrays output by the phase information output modes set for different light intensity ranges decrease in sequence.

[0154] Determine which light intensity range the light intensity of the current shooting scene falls into, and use the light intensity range as the target light intensity range to which the light intensity of the current shooting scene belongs. Use the phase information output mode corresponding to the target light intensity range as the phase information output mode adapted to the light intensity of the current shooting scene.

[0155] In an embodiment of the present application, when determining a phase information output mode that matches the light intensity of the current shooting scene based on the light intensity of the current shooting scene, since different light intensity ranges correspond to different phase information output modes, first determine the target light intensity range to which the light intensity of the current shooting scene belongs. Then, based on the target light intensity range, determine a phase information output mode that matches the light intensity of the current shooting scene. Different phase information output modes are set in advance for different light intensity ranges, and the size of the phase array output in each phase information output mode is different. Therefore, based on the light intensity of the current shooting scene, the phase information of the pixel array can be calculated more finely to achieve more accurate focusing.

[0156] Following the previous embodiment, it is further described that the phase information output mode includes a full-size output mode and a first-size output mode, and the size of the phase array in the full-size output mode is larger than the size of the phase array in the first-size output mode. Fig.13 As shown, step 1024, determining a phase information output mode adapted to the light intensity of the current shooting scene according to the target light intensity range, includes:

[0157] Step 1024a, if the light intensity of the current shooting scene is greater than the first preset threshold, determining that the phase information output mode adapted to the light intensity of the current shooting scene is the full-size output mode;

[0158] Step 1024b: If the light intensity of the current shooting scene is greater than the second preset threshold and less than or equal to the first preset threshold, determine that the phase information output mode adapted to the light intensity of the current shooting scene is the first size output mode.

[0159] Specifically, if one of the light intensity ranges is greater than the first preset threshold, the phase information output mode corresponding to the light intensity range is the full-size output mode. If it is determined that the light intensity of the current shooting scene is greater than the first preset threshold, the light intensity of the current shooting scene falls within the light intensity range. That is, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the full-size output mode. Among them, the full-size output mode is used to output the phase array, that is, all the original phase information of the pixel array is output to generate the phase array of the pixel array.

[0160] If one of the light intensity ranges is greater than the second preset threshold and less than or equal to the first preset threshold, the phase information output mode corresponding to the light intensity range is the first size output mode. If it is determined that the light intensity of the current shooting scene is greater than the second preset threshold and less than or equal to the first preset threshold, the light intensity of the current shooting scene falls within the light intensity range. That is, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the first size output mode. Among them, the first size output mode is used to output the phase array, that is, the original phase information of the pixel array is merged and output to generate the phase array of the pixel array.

[0161] In the embodiment of the present application, since the size of the phase array in the full-size output mode is larger than the size of the phase array in the first-size output mode, if the light intensity of the current shooting scene is greater than the first preset threshold, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the full-size output mode. If the light intensity of the current shooting scene is greater than the second preset threshold, and less than or equal to the first preset threshold, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the first-size output mode. That is, if the light intensity of the current shooting scene is large, the full-size output mode is used to output a phase array with the same size as the pixel array, and if the light intensity of the current shooting scene is second, the first-size output mode is used to output a phase array smaller than the pixel array size. That is, when the light intensity of the current shooting scene is second, the signal-to-noise ratio of the phase information is improved by reducing the phase array.

[0162] In the previous embodiment, it is described that the pixel array may be an RGBW pixel array, including a plurality of minimum repeating units 241, the minimum repeating unit 241 includes a plurality of pixel groups 242, and the plurality of pixel groups 242 include a full-color pixel group 243 and a color pixel group 244. Each full-color pixel group 243 includes 9 full-color pixels 2431, and each color pixel group 244 includes 9 color pixels 2441. Each full-color pixel 2431 includes 2 sub-pixels arranged in an array, and each color pixel 2441 includes 2 sub-pixels arranged in an array.

[0163] In this embodiment, when the pixel array is an RGBW pixel array, it is described in detail that if the phase information output mode is a full-size output mode, the color pixel group and the full-color pixel group in the pixel array are selected as the target pixel group, including:

[0164] A color pixel group in a pixel array is used as a target pixel group; or a color pixel group and a full-color pixel group in the pixel array are used as a target pixel group; wherein one target pixel group corresponds to one pixel group;

[0165] Accordingly,

[0166] According to the phase information of the target pixel, a phase array corresponding to the pixel array is generated, including:

[0167] According to the phase information of the target pixel, a full-size phase array corresponding to the pixel array is generated.

[0168] This embodiment is a specific implementation step of outputting a phase array corresponding to a pixel array in full-size output mode when the light intensity of the current shooting scene is greater than a first preset threshold. The first preset threshold may be 2000 lux, which is not limited in this application. That is, at this time, the light intensity is greater than 2000 lux. Fig.11 As shown, first, a color pixel group is determined from the pixel array as a target pixel group for calculating phase information. Because when the light intensity of the current shooting scene is greater than the first preset threshold, that is, in a scene with sufficient light, the panchromatic pixel has high sensitivity and is easily saturated in a scene with sufficient light, and after saturation, the correct phase information cannot be obtained, so the phase information of the color pixel group can be used to achieve phase focusing (PDAF). Of course, the color pixel group and the panchromatic pixel group in the pixel array can also be used as the target pixel group.

[0169] Specifically, if the phase information of the color pixel group is used to implement phase focusing (PDAF), the phase information of some color pixel groups in the pixel array can be used to implement phase focusing, for example, the phase information of at least one pixel group among the red pixel group, the green pixel group and the blue pixel group is used to implement phase focusing. It is also possible to use some pixels in some color pixel groups to implement phase focusing, for example, the phase information of some pixels in the red pixel group is used to implement phase focusing, and this application does not limit this. Since only the phase information of the color pixel group is used for phase focusing at this time, the amount of data of the output phase information is reduced, thereby improving the efficiency of phase focusing.

[0170] Secondly, a target microlens is determined from the microlenses corresponding to the target pixel group, and at least two pixels corresponding to the target microlens are used as target pixels.

[0171] Specifically, a target microlens in the first direction corresponding to the target pixel group can be determined; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction. The at least two pixels corresponding to the target microlens in the first direction are taken as target pixels. A target microlens in the second direction corresponding to the target pixel group can also be determined; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; the second direction is perpendicular to the first direction. The at least two pixels corresponding to the target microlens in the second direction are taken as target pixels. A target microlens in the first direction and a target microlens in the second direction corresponding to the target pixel group can also be determined; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction. The at least two pixels corresponding to the target microlens in the first direction and the at least two pixels corresponding to the target microlens in the second direction are taken as target pixels.

[0172] Again, for each target pixel group, the phase information of each target pixel in the target pixel group is obtained. Each pixel corresponds to a photosensitive element. In the embodiment of the present application, it is assumed that all color pixel groups in the pixel array are used as target pixel groups, and for each color pixel group, the phase information of each target pixel in the color pixel group is obtained. Finally, the phase information of all target pixels is output as a full-size phase array corresponding to the pixel array.

[0173] Combination Fig.14 As shown, a 12×12 pixel array may include 2 red pixel groups, 4 green pixel groups, 2 blue pixel groups, and 8 full-color pixel groups. Figure 7 and Figure 8As shown, it can be assumed that a represents green, b represents red, c represents blue, and w represents full color. And assuming that all color pixel groups in the pixel array are taken as target pixel groups, the phase information of each pixel group is calculated in sequence for the 2 red pixel groups, 4 green pixel groups and 2 blue pixel groups included in the pixel array. For example, the phase information of the red pixel group 244 is calculated for the red pixel group, and the red pixel group includes 9 red pixels arranged in a 3×3 array, which are numbered red pixel 1, red pixel 2, red pixel 3, red pixel 4, red pixel 5, red pixel 6, red pixel 7, red pixel 8, and red pixel 9 in sequence. Among them, each pixel corresponds to a photosensitive element. That is, the phase information of red pixel 1 is L1, and the phase information of red pixel 2 is R1; the phase information of red pixel 8 is L2, and the phase information of red pixel 9 is L2; ​​the phase information of red pixel 3 is U1, and the phase information of red pixel 6 is D1; ​​the phase information of red pixel 4 is U1, and the phase information of red pixel 7 is D1. Among them, the pixel 5 located at the center of the red pixel group corresponds to a microlens, so no phase information is obtained.

[0174] Therefore, if the target microlens in the first direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens in the first direction are used as target pixels, assuming that the first direction is the horizontal direction, L1, R1, L2, and R2 are output in sequence to obtain the phase information of the red pixel group.

[0175] In this way, the above processing is performed in turn on the other 1 red pixel group, 4 green pixel groups and 2 blue pixel groups included in the pixel array to obtain the phase information L3, R3, L4, R4, L5, R5, L6, R6, L7, R7, L8, R8, L9, R9, L10, R10, L11, R11, L12, R12, L13, R13, L14, R14, L15, R15, L16, R16 of the pixel array.

[0176] Finally, the phase information of all target pixels is output as a full-size phase array corresponding to the pixel array. That is, L1, R1, L2, R2, L3, R3, L4, R4, L5, R5, L6, R6, L7, R7, L8, R8, L9, R9, L10, R10, L11, R11, L12, R12, L13, R13, L14, R14, L15, R15, L16, R16 are arranged in sequence to generate a full-size phase array. And the size of the full-size phase array is equivalent to the size of 4×8 pixels arranged in the array. Here, the size of a pixel refers to the area size of a pixel, which is related to the length and width of the pixel.

[0177] Among them, pixel is the smallest photosensitive unit on the photosensitive device (CCD or CMOS) of a digital camera. Among them, CCD is the abbreviation of charge coupled device, and CMOS (Complementary Metal-Oxide-Semiconductor), which can be interpreted as complementary metal oxide semiconductor. Generally, pixels do not have a fixed size, and the size of pixels is related to the size and resolution of the display screen. For example, if the size of the display screen is 4.5 inches and the resolution of the display screen is 1280×720, the length of the display screen is 99.6mm and the width is 56mm. Then the length of a pixel is 99.6mm / 1280=0.0778mm, and the width is also 56mm / 720=0.0778mm. In this example, the size of the 4×8 pixels arranged in the array is: 4×0.0778mm in length and 8×0.0778mm in width. Of course, this application does not limit this. Then, the full-size phase array has a length of 4×0.0778 mm and a width of 8×0.0778 mm. Of course, in other embodiments, the pixel may not be a rectangle with equal length and width, and the pixel may have other heterogeneous structures, which is not limited in this application.

[0178] Similarly, if the target microlens in the second direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens in the second direction are used as target pixels, assuming that the second direction is the vertical direction, U1, D1, U2, and D2 are output in sequence to obtain the phase information of the red pixel group.

[0179] In this way, the above processing is performed in turn on the other 1 red pixel group, 4 green pixel groups and 2 blue pixel groups included in the pixel array to obtain the phase information U3, D3, U4, D4, U5, D5, U6, D6, U7, D7, U8, D8, U9, D9, U10, D10, U11, D11, U12, D12, U13, D13, U14, D14, U15, D15, U16, D16 of the pixel array.

[0180] Finally, the phase information of all target pixels is output as a full-size phase array corresponding to the pixel array. That is, U1, D1, U2, D2, U3, D3, U4, D4, U5, D5, U6, D6, U7, D7, U8, D8, U9, D9, U10, D10, U11, D11, U12, D12, U13, D13, U14, D14, U15, D15, U16, D16 are arranged in sequence to generate a full-size phase array. And the size of the full-size phase array is equivalent to the size of 4×8 pixels arranged in the array.

[0181] Similarly, if the target microlens in the first direction and the target microlens in the second direction corresponding to the target pixel group are determined, and at least two pixels corresponding to the target microlens in the first direction and the target microlens in the second direction are used as target pixels. Assuming that the first direction is the horizontal direction and the second direction is the vertical direction, L1, R1, L2, R2, U1, D1, U2, and D2 are output in sequence to obtain the phase information of the red pixel group. Then, the above processing is performed on the other 1 red pixel group, 4 green pixel groups, and 2 blue pixel groups included in the pixel array in sequence to obtain the phase information of the pixel array. Finally, the phase information of all target pixels is output as a full-size phase array corresponding to the pixel array. And the size of the full-size phase array is equivalent to the size of 8×8 pixels arranged in the array.

[0182] At this point, the phase array can be input into ISP (Image Signal Processing), and the phase difference of the pixel array can be calculated by ISP based on the phase array. Then, the defocus distance is calculated based on the phase difference, and the DAC code value corresponding to the defocus distance is calculated. Finally, the code value is converted into a driving current through the driver IC of the motor (VCM), and the motor drives the lens to move to a clear position. Thus, focus control is achieved based on the phase difference.

[0183] In an embodiment of the present application, when the light intensity of the current shooting scene is greater than the first preset threshold, a phase array corresponding to the pixel array is output in full-size output mode. The color pixel group in the pixel array is used as the target pixel group, and for each target pixel group, the phase information of each pixel in the target pixel group is obtained. And the phase information of the target pixel can be obtained from at least one of the microlenses in the first direction and the microlens in the second direction, so a variety of phase information can be obtained. Finally, according to the phase information of the target pixel, a full-size phase array corresponding to the pixel array is generated. Since only the phase information of the color pixel group is used for phase focusing at this time, the amount of data of the output phase information is reduced, thereby improving the efficiency of phase focusing.

[0184] In one embodiment, if the phase information output mode is the first size output mode, the color pixel group and the full-color pixel group in the pixel array are selected as the target pixel group, including:

[0185] At least one of a color pixel group and a full-color pixel group in a pixel array is used as a target pixel group; the target pixel group corresponds to a pixel group;

[0186] Accordingly,

[0187] According to the phase information of the target pixel, a phase array corresponding to the pixel array is generated, including:

[0188] Merging the phase information of the target pixels in the target pixel group to generate multiple groups of intermediate merged phase information;

[0189] A phase array of a first size corresponding to the pixel array is generated according to the multiple sets of intermediate combined phase information.

[0190] This embodiment is a specific implementation step of outputting a phase array corresponding to a pixel array according to a first size output mode when the light intensity of the current shooting scene is greater than a second preset threshold and less than or equal to a first preset threshold. The second preset threshold may be 500 lux, which is not limited in this application. That is, at this time, the light intensity is greater than 500 lux and less than or equal to 2000 lux. Fig.15 As shown, at least one of the color pixel group and the full-color pixel group is first determined from the pixel array as the target pixel group for calculating the phase information. Because when the light intensity of the current shooting scene is greater than the second preset threshold and less than or equal to the first preset threshold, that is, in a scene with slightly weak light, the full-color pixels are not easy to saturate in the scene with slightly weak light, so at this time, the phase information of the full-color pixel group can be used to achieve phase focusing (PDAF). And in a scene with slightly weak light, the color pixels can also obtain accurate phase information in the scene with slightly weak light, so at this time, the phase information of the color pixel group can also be used to achieve phase focusing (PDAF). Therefore, when outputting the phase array corresponding to the pixel array according to the first size output mode, the color pixel group can be selected as the target pixel group, the full-color pixel group can be selected as the target pixel group, and the color pixel group and the full-color pixel group can also be selected as the target pixel group, and this application does not limit this.

[0191] Specifically, for the case where the target pixel group is a color pixel group, phase information of some color pixel groups in the pixel array can be used to achieve phase focusing, or some color pixels in some color pixel groups can be used to achieve phase focusing, which is not limited in this application. Similarly, for the case where the target pixel group is a panchromatic pixel group, phase information of some panchromatic pixel groups in the pixel array can be used to achieve phase focusing, or some panchromatic pixels in some panchromatic pixel groups can be used to achieve phase focusing, which is not limited in this application. Similarly, for the case where the target pixel group is a color pixel group and a panchromatic pixel group, phase information of some panchromatic pixel groups and some color pixel groups in the pixel array can be used to achieve phase focusing, or some panchromatic pixels in some panchromatic pixel groups and some color pixels in some color pixel groups can be used to achieve phase focusing, which is not limited in this application.

[0192] Since only the phase information of some pixel groups can be used for phase focusing, or only the phase information of some pixels in some pixel groups can be used for phase focusing, the amount of data of the output phase information is reduced, thereby improving the efficiency of phase focusing.

[0193] Secondly, a target microlens is determined from the microlenses corresponding to the target pixel group, and at least two pixels corresponding to the target microlens are used as target pixels.

[0194] Specifically, a target microlens in the first direction corresponding to the target pixel group can be determined; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction. The at least two pixels corresponding to the target microlens in the first direction are taken as target pixels. A target microlens in the second direction corresponding to the target pixel group can also be determined; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; the second direction is perpendicular to the first direction. The at least two pixels corresponding to the target microlens in the second direction are taken as target pixels. A target microlens in the first direction and a target microlens in the second direction corresponding to the target pixel group can also be determined; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction. The at least two pixels corresponding to the target microlens in the first direction and the at least two pixels corresponding to the target microlens in the second direction are taken as target pixels.

[0195] Again, for each target pixel group, the phase information of each target pixel in the target pixel group is obtained. Each pixel corresponds to a photosensitive element. In the embodiment of the present application, on the one hand, it is assumed that all full-color pixel groups in the pixel array are used as target pixel groups, and for each full-color pixel group, the phase information of each target pixel in the full-color pixel group is obtained. The phase information of the target pixels is merged to generate multiple groups of intermediate merged phase information; finally, a first-size phase array corresponding to the pixel array is generated according to the multiple groups of intermediate merged phase information.

[0196] Combination Fig.15As shown, a pixel array may include 2 red pixel groups, 4 green pixel groups, 2 blue pixel groups, and 8 full-color pixel groups. Assuming that all full-color pixel groups in the pixel array are taken as target pixel groups, the phase information of each pixel group is calculated in sequence for the 8 full-color pixel groups included in the pixel array. For example, the phase information of the full-color pixel group is calculated for the full-color pixel group, and the full-color pixel group includes 9 full-color pixels arranged in a 3×3 array, which are numbered as full-color pixel 1, full-color pixel 2, full-color pixel 3, full-color pixel 4, full-color pixel 5, full-color pixel 6, full-color pixel 7, full-color pixel 8, and full-color pixel 9 in sequence. Among them, each pixel corresponds to a photosensitive element. That is, the phase information of full-color pixel 1 is L1, the phase information of full-color pixel 2 is R1; the phase information of full-color pixel 8 is L2, the phase information of full-color pixel 9 is L2; ​​the phase information of full-color pixel 3 is U1, the phase information of full-color pixel 6 is D1; ​​the phase information of full-color pixel 4 is U1, the phase information of full-color pixel 7 is D1. Among them, pixel 5 at the center of the full-color pixel group corresponds to a microlens, so no phase information is obtained.

[0197] Therefore, if the target microlens of the first direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens of the first direction are used as target pixels. Assuming that the first direction is the horizontal direction, L1, R1, L2, and R2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information. For example, L1 and L2 are merged to generate the first intermediate merged phase information L; R1 and R2 are merged to generate the first intermediate merged phase information R.

[0198] In this way, the above processing is performed on the other full-color pixel groups included in the pixel array in turn to obtain the phase information L3, R3, L4, R4, L5, R5, L6, R6, L7, R7, L8, R8, L9, R9, L10, R10, L11, R11, L12, R12, L13, R13, L14, R14, L15, R15, L16, R16 of the pixel array. Then L3 and L4 are combined to generate the first intermediate combined phase information L; R3 and R4 are combined to generate the first intermediate combined phase information R. L5 and L6 are combined to generate the first intermediate combined phase information L; R5 and R6 are combined to generate the first intermediate combined phase information R. L7 and L8 are combined to generate the first intermediate combined phase information L; R7 and R8 are combined to generate the first intermediate combined phase information R. L9 and L10 are combined to generate the first intermediate combined phase information L; R9 and R10 are combined to generate the first intermediate combined phase information R. L11 and L12 are combined to generate the first intermediate combined phase information L; R11 and R12 are combined to generate the first intermediate combined phase information R. L13 and L14 are combined to generate the first intermediate combined phase information L; R13 and R14 are combined to generate the first intermediate combined phase information R. L15 and L16 are combined to generate the first intermediate combined phase information L; R15 and R16 are combined to generate the first intermediate combined phase information R.

[0199] Finally, a first-size phase array corresponding to the pixel array is generated according to multiple sets of intermediate combined phase information. In this way, a first-size phase array is generated by sequentially arranging 8 sets of first intermediate combined phase information L and first intermediate combined phase information R. And the size of the first-size phase array is equivalent to the size of 4×4 pixels arranged in the array.

[0200] Of course, the 8 groups of first intermediate merged phase information L and the first intermediate merged phase information R can also be merged or transformed to generate a first size phase array corresponding to the pixel array. The transformation processing here can be correction and other processing on the 8 groups of first intermediate merged phase information L and the first intermediate merged phase information R, which is not limited in this application. When the 8 groups of first intermediate merged phase information L and the first intermediate merged phase information R are merged, the phase array equivalent to 4×4 pixels can be merged into a phase array of 4×2 pixels. Of course, the present application does not limit the specific size of the merged phase array. Here, the size of a pixel refers to the area size of a pixel, which is related to the length and width of the pixel.

[0201] Among them, a pixel is the smallest photosensitive unit on a digital camera photosensitive device (CCD or CMOS). In general, a pixel does not have a fixed size, and the size of a pixel is related to the size and resolution of the display screen. For example, if the size of the display screen is 4.5 inches and the resolution of the display screen is 1280×720, the length of the display screen is 99.6 mm and the width is 56 mm. The length of a pixel is 99.6 mm / 1280=0.0778 mm, and the width is also 56 mm / 720=0.0778 mm. In this example, the size of the 4×4 pixels arranged in the array is: 4×0.0778 mm in length and 4×0.0778 mm in width. Of course, this application does not limit this. Then, the size of the full-size phase array is 4×0.0778 mm in length and 4×0.0778 mm in width. Of course, in other embodiments, the pixel may not be a rectangle with equal length and width, and the pixel may also have other heterogeneous structures, which this application does not limit.

[0202] Similarly, if the target microlens in the second direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens in the second direction are used as target pixels. Assuming that the second direction is the vertical direction, U1, D1, U2, and D2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information. For example, U1 and U2 are merged to generate the first intermediate merged phase information U; D1 and D2 are merged to generate the first intermediate merged phase information D.

[0203] In this way, the above processing is performed on the other full-color pixel groups included in the pixel array in turn to obtain the phase information U3, D3, U4, D4, U5, D5, U6, D6, U7, D7, U8, D8, U9, D9, U10, D10, U11, D11, U12, D12, U13, D13, U14, D14, U15, D15, U16, D16 of the pixel array. Then, U3 and U4 are combined to generate the first intermediate combined phase information U; D3 and D4 are combined to generate the first intermediate combined phase information D. U5 and U6 are combined to generate the first intermediate combined phase information U; D5 and D6 are combined to generate the first intermediate combined phase information D. U7 and U8 are combined to generate the first intermediate combined phase information U; D7 and D8 are combined to generate the first intermediate combined phase information D. Merge U9 and U10 to generate the first intermediate merged phase information U; merge D9 and D10 to generate the first intermediate merged phase information D. Merge U11 and U12 to generate the first intermediate merged phase information U; merge D11 and D12 to generate the first intermediate merged phase information D. Merge U13 and U14 to generate the first intermediate merged phase information U; merge D13 and D14 to generate the first intermediate merged phase information D. Merge U15 and U16 to generate the first intermediate merged phase information U; merge D15 and D16 to generate the first intermediate merged phase information D.

[0204] Finally, a first-size phase array corresponding to the pixel array is generated according to multiple sets of intermediate combined phase information. In this way, a first-size phase array is generated by sequentially arranging 8 sets of first intermediate combined phase information U and first intermediate combined phase information D. And the size of the first-size phase array is equivalent to the size of 4×4 pixels arranged in the array.

[0205] Similarly, if the target microlens in the first direction and the target microlens in the second direction corresponding to the target pixel group are determined, and at least two pixels corresponding to the target microlens in the first direction and the target microlens in the second direction are used as target pixels. Assuming that the first direction is the horizontal direction and the second direction is the vertical direction, L1, R1, L2, R2, U1, D1, U2, and D2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information. For example, L1 and L2 are merged to generate the first intermediate merged phase information L; R1 and R2 are merged to generate the first intermediate merged phase information R. U1 and U2 are merged to generate the first intermediate merged phase information U; D1 and D2 are merged to generate the first intermediate merged phase information D.

[0206] Then, the above processing is performed on the other full-color pixel groups included in the pixel array in turn to obtain the phase information of the pixel array. Finally, the phase information of the target pixel is merged to generate multiple sets of intermediate merged phase information, and the multiple sets of intermediate merged phase information are used as the first size phase array corresponding to the pixel array. And the size of the first full-size phase array is equivalent to the size of 8×8 pixels arranged in the array.

[0207] On the other hand, assuming that all color pixel groups in the pixel array are taken as target pixel groups, for each color pixel group, the phase information of each target pixel in the color pixel group is obtained. The phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information; finally, a first-size phase array corresponding to the pixel array is generated according to the multiple sets of intermediate merged phase information. The specific calculation process is the same as the process of calculating the first-size phase array when all panchromatic pixel groups in the pixel array are taken as target pixel groups, and will not be repeated here.

[0208] On the other hand, assuming that all panchromatic pixel groups and color pixel groups in the pixel array are taken as target pixel groups, for each panchromatic pixel group and each color pixel group, the phase information of each target pixel in the panchromatic pixel group and each color pixel group is obtained. The phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information; finally, a first-size phase array corresponding to the pixel array is generated according to the multiple sets of intermediate merged phase information. The specific calculation process is the same as the process of calculating the first-size phase array when all panchromatic pixel groups in the pixel array are taken as target pixel groups, and will not be repeated here.

[0209] At this time, the phase array can be input into the ISP, and the phase difference of the pixel array can be calculated by the ISP based on the phase array. Then, the defocus distance is calculated based on the phase difference, and the DAC code value corresponding to the defocus distance is calculated. Finally, the code value is converted into a driving current through the driver IC of the motor (VCM), and the motor drives the lens to move to a clear position. Thus, focus control is achieved based on the phase difference.

[0210] In an embodiment of the present application, when the light intensity of the current shooting scene is greater than the second preset threshold and less than or equal to the first preset threshold, because the light intensity at this time is slightly weak, the phase information collected by the color pixel group or the full-color pixel group is not very accurate, and some color pixel groups or some full-color pixel groups may not collect phase information. Therefore, at least one of the color pixel group or the full-color pixel group in the pixel array is used as the target pixel group, and for each target pixel group, the phase information of each pixel in the target pixel group is obtained. And the phase information of the target pixel can be obtained from at least one of the microlenses in the first direction and the microlenses in the second direction, so a variety of phase information can be obtained.

[0211] The phase information of the target pixels is combined to generate multiple sets of intermediate combined phase information. Based on the multiple sets of intermediate combined phase information, a first size phase array corresponding to the pixel array is generated. The phase information of the target pixels is combined to a certain extent using the first size output mode to improve the accuracy of the output phase information and the signal-to-noise ratio of the phase information. Finally, phase focusing is performed based on the first size phase array corresponding to the pixel array, so as to improve the focusing accuracy.

[0212] Continuing from the previous embodiment, if the target pixels are at least two pixels corresponding to the target microlenses in the first direction, the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information, including:

[0213] Acquire at least two target microlenses in the first direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least two target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses;

[0214] The phase information of the target pixels at the same position are combined to generate multiple sets of first combined phase information.

[0215] Combination Fig.15 As shown, if the target microlens of the first direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens of the first direction are used as target pixels, assuming that the first direction is the horizontal direction, L1, R1, L2, and R2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information.

[0216] Specific, combined Figure 8 As shown in (a), at least two target microlenses in the first direction (gray filled in the figure) in the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least two target microlenses in the first direction. Among them, the target pixels at the same position are in the same orientation in the first direction in the corresponding target microlenses. For example, the target pixels at the same position are determined from the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a in the first direction (gray filled in the figure). That is, the pixel on the left side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel, and the pixel on the right side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel. That is, L1 and L2 are merged to generate the first intermediate merged phase information L; R1 and R2 are merged to generate the first intermediate merged phase information R.

[0217] Based on the multiple sets of first intermediate combined phase information L and the first intermediate combined phase information R, multiple sets of first combined phase information are generated.

[0218] In an embodiment of the present application, at least two target microlenses in a first direction within a target pixel group are obtained, and target pixels at the same position are determined from the pixels corresponding to the at least two target microlenses in the first direction. The phase information of the target pixels at the same position is merged to generate multiple sets of first merged phase information. The phase information of the pixels is merged to a certain extent using a first size output mode to improve the accuracy of the output phase information and the signal-to-noise ratio of the phase information. Finally, phase focusing is performed based on a first size phase array corresponding to the pixel array, so that the focusing accuracy can be improved.

[0219] Continuing from the previous embodiment, assuming that the first direction is a vertical direction, the vertical direction is named as the second direction here, if the target pixels are at least two pixels corresponding to the target microlenses in the second direction, the phase information of the target pixels in the target pixel group is merged to generate multiple groups of intermediate merged phase information, including:

[0220] Acquire at least two target microlenses in the second direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least two target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses;

[0221] The phase information of the target pixels at the same position are combined to generate multiple sets of second combined phase information.

[0222] Combination Fig.15 As shown, if the target microlens in the second direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens in the second direction are used as target pixels, assuming that the second direction is a vertical direction, U1, D1, U2, and D2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information.

[0223] Specific, combined Figure 8As shown in (a), at least two target microlenses in the second direction (filled with oblique lines in the figure) in the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least two target microlenses in the second direction. Among them, the target pixels at the same position are in the same orientation in the second direction in the corresponding target microlenses. For example, the target pixels at the same position are determined from the pixels 3, 6, 4, and 7 corresponding to the target microlens 2211a (filled with oblique lines in the figure) in the second direction. That is, the pixels above the target microlens 2211a (filled with oblique lines in the figure) in the second direction are determined as target pixels, and the pixels below the target microlens 2211a (filled with oblique lines in the figure) in the second direction are determined as target pixels. That is, U1 and U2 are merged to generate the second intermediate merged phase information U; D1 and D2 are merged to generate the second intermediate merged phase information D.

[0224] Based on the multiple sets of second intermediate combined phase information U and the second intermediate combined phase information D, multiple sets of second combined phase information are generated.

[0225] In an embodiment of the present application, at least two target microlenses in the second direction within the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least two target microlenses in the second direction. The phase information of the target pixels at the same position is merged to generate multiple sets of second merged phase information. The phase information of the pixels is merged to a certain extent using the first size output mode to improve the accuracy of the output phase information and the signal-to-noise ratio of the phase information. Finally, phase focusing is performed based on the first size phase array corresponding to the pixel array, so that the focusing accuracy can be improved.

[0226] Continuing from the previous embodiment, if the target pixels are at least two pixels corresponding to the target microlenses in the first direction and at least two pixels corresponding to the target microlenses in the second direction, the phase information of the target pixels in the target pixel group is merged to generate multiple sets of intermediate merged phase information, including:

[0227] Acquire at least two target micro-lenses in the first direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least two target micro-lenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target micro-lenses;

[0228] Merging the phase information of target pixels at the same position to generate multiple sets of first merged phase information;

[0229] Acquire at least two target microlenses in the second direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least two target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses;

[0230] Merging the phase information of target pixels at the same position to generate multiple sets of second merged phase information;

[0231] Based on the multiple sets of first combined phase information and the multiple sets of second combined phase information, multiple sets of intermediate combined phase information are generated.

[0232] Specific, combined Fig.15 As shown, if the target microlens of the first direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens of the first direction are used as target pixels, assuming that the first direction is the horizontal direction, L1, R1, L2, and R2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information.

[0233] Specific, combined Figure 8 As shown in (a), at least two target microlenses of the first direction (gray filled in the figure) in the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least two target microlenses of the first direction. Among them, the target pixels at the same position are in the same orientation in the first direction in the corresponding target microlenses. For example, the target pixels at the same position are determined from the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a of the first direction (gray filled in the figure). That is, the pixel on the left side of the target microlens 2211a of the first direction (gray filled in the figure) is determined as the target pixel, and the pixel on the right side of the target microlens 2211a of the first direction (gray filled in the figure) is determined as the target pixel. That is, L1 and L2 are merged to generate the first intermediate merged phase information L; R1 and R2 are merged to generate the first intermediate merged phase information R.

[0234] Based on the multiple sets of first intermediate combined phase information L and the first intermediate combined phase information R, multiple sets of first combined phase information are generated.

[0235] If the target microlens in the second direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens in the second direction are used as target pixels, assuming that the second direction is a vertical direction, U1, D1, U2, and D2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of intermediate merged phase information.

[0236] Specifically, at least two target microlenses in the second direction (filled with oblique lines in the figure) in the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least two target microlenses in the second direction. The target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses. For example, the target pixels at the same position are determined from the pixels 3, 6, 4, and 7 corresponding to the target microlens 2211a (filled with oblique lines in the figure) in the second direction. That is, the pixels above the target microlens 2211a (filled with oblique lines in the figure) in the second direction are determined as target pixels, and the pixels below the target microlens 2211a (filled with oblique lines in the figure) in the second direction are determined as target pixels. That is, U1 and U2 are merged to generate the second intermediate merged phase information U; D1 and D2 are merged to generate the second intermediate merged phase information D.

[0237] Based on the multiple sets of second intermediate combined phase information U and the second intermediate combined phase information D, multiple sets of second combined phase information are generated.

[0238] Finally, based on the multiple sets of first combined phase information and the multiple sets of second combined phase information, multiple sets of intermediate combined phase information are generated.

[0239] In an embodiment of the present application, at least two target microlenses in a first direction and at least two target microlenses in a second direction are obtained within a target pixel group, and target pixels at the same position are determined from the pixels corresponding to the target microlenses. The phase information of the target pixels at the same position is merged to generate multiple groups of first merged phase information and second merged phase information. Finally, based on the multiple groups of first merged phase information and the multiple groups of second merged phase information, multiple groups of intermediate merged phase information are generated. The first size output mode is used to merge the phase information of the pixels to a certain extent, thereby improving the accuracy of the output phase information and improving the signal-to-noise ratio of the phase information. Finally, phase focusing is performed based on the first size phase array corresponding to the pixel array, so as to improve the focusing accuracy.

[0240] Continuing from the previous embodiment, if the target pixel group includes a color pixel group and a full-color pixel group, generating a first-size phase array of the pixel array in the first direction according to the target phase information includes:

[0241] Determine a first phase weight corresponding to the color pixel group and a second phase weight corresponding to the panchromatic pixel group according to the light intensity of the current shooting scene; wherein the first phase weight corresponding to the color pixel group is different under different light intensities, and the second phase weight corresponding to the panchromatic pixel group is different under different light intensities;

[0242] A first-size phase array of the pixel array in a first direction is generated based on the target phase information and the first phase weight of the color pixel group and the target phase information and the second phase weight of the panchromatic pixel group.

[0243] Specifically, in a scenario where the light intensity of the current shooting scene is greater than the second preset threshold and less than or equal to the first preset threshold, and the determined target pixel group includes a color pixel group and a panchromatic pixel group, then, when generating a first-size phase array based on the phase information of the target pixel group, the weights between different pixel groups can be considered. Among them, the first phase weight corresponding to the color pixel group and the second phase weight corresponding to the panchromatic pixel group can be determined according to the light intensity of the current shooting scene. Specifically, the closer the light intensity of the current shooting scene is to the second preset threshold, the smaller the first phase weight corresponding to the color pixel group is at this time, and the larger the second phase weight corresponding to the panchromatic pixel group is. Because the light intensity is relatively small in a scenario where the light intensity is greater than the second preset threshold and less than or equal to the first preset threshold, the larger the second phase weight corresponding to the panchromatic pixel group is, the more accurate the phase information obtained is. As the light intensity increases, the closer the light intensity of the current shooting scene is to the first preset threshold, the larger the first phase weight corresponding to the color pixel group is at this time, and the smaller the second phase weight corresponding to the panchromatic pixel group is. Because the light intensity is relatively large in the scenario where it is greater than the second preset threshold and less than or equal to the first preset threshold, the larger the first phase weight corresponding to the color pixel group, the more comprehensive and accurate the phase information obtained. Among them, the first phase weight corresponding to the color pixel group is different under different light intensities, and the second phase weight corresponding to the full-color pixel group is different under different light intensities. For example, when the light intensity of the current shooting scene is 2000lux, the first phase weight corresponding to the color pixel group is determined to be 40%, wherein the phase weight of the green pixel group is 20%, the phase weight of the red pixel group is 10%, and the phase weight of the blue pixel group is 10%. And the second phase weight corresponding to the full-color pixel group is determined to be 60%, which is not limited in this application.

[0244] Then, based on the target phase information and the first phase weight of the color pixel group, the target phase information and the second phase weight of the full-color pixel group, a first-size phase array of the pixel array in the first direction can be generated. For example, for the pixel array, based on the target phase information and the phase weight of 10% of the first red pixel group, the target phase information and the phase weight of 10% of the second red pixel group, the target phase information and the phase weight of 10% of the first blue pixel group, the target phase information and the phase weight of 10% of the second blue pixel group, and the target phase information and the phase weight of 20% of each green pixel group, and the target phase information and the phase weight of 60% of each full-color pixel group, the phase information of the pixel array in the first direction is calculated by summing up, and the first-size phase array is obtained.

[0245] In an embodiment of the present application, when performing phase focusing, if it is determined that the target pixel group includes a color pixel group and a full-color pixel group, a first-size phase array of the pixel array can be generated based on the target phase information of the color pixel group and its first phase weight, and the target phase information of the full-color pixel group and its second phase weight. In this way, based on the target phase information of the color pixel group and the full-color pixel group, a first-size phase array of the pixel array is jointly generated, which can improve the comprehensiveness of the phase information. At the same time, the phase weights of the target phase information of the color pixel group and the full-color pixel group are different under different light intensities. In this way, the accuracy of the phase information can be improved by adjusting the weight size under different light intensities.

[0246] In the foregoing embodiment, it is described that the pixel array may be an RGBW pixel array, including a plurality of minimum repeating units 241, the minimum repeating unit 241 includes a plurality of pixel groups 242, and the plurality of pixel groups 242 include a full-color pixel group 243 and a color pixel group 244. Each full-color pixel group 243 includes 9 full-color pixels 2431, and each color pixel group 244 includes 9 color pixels 2441.

[0247] In this embodiment, when the pixel array is an RGBW pixel array, the phase information output mode further includes a second size output mode; wherein the size of the phase array in the first size output mode is greater than the size of the phase array in the second size output mode;

[0248] Combination Fig.13 As shown, step 1024, determining a phase information output mode adapted to the light intensity of the current shooting scene according to the target light intensity range, includes:

[0249] Step 1024c: if the light intensity of the current shooting scene is less than the second preset threshold, determine that the phase information output mode adapted to the light intensity of the current shooting scene is the second size output mode.

[0250] Specifically, if one of the light intensity ranges is less than the second preset threshold, the phase information output mode corresponding to the light intensity range is the second size output mode. If it is determined that the light intensity of the current shooting scene is less than the second preset threshold, the light intensity of the current shooting scene falls into the light intensity range. That is, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the second size output mode. Among them, the second preset threshold can be 500 lux, which is not limited in this application. That is, it is at dusk or in an environment where the light intensity is less than 500 lux.

[0251] Among them, the second size output mode is used to output the phase array, that is, the original phase information of the pixel array is combined and output to generate the phase array of the pixel array. In other words, the size of the pixel array is larger than the size of the phase array of the pixel array. For example, if the size of the pixel array is 12×12, the size of the phase array of each target pixel group in the pixel array is 4×2, and the size of the phase array is not limited in this application.

[0252] In the embodiment of the present application, since the size of the phase array in the first size output mode is greater than or equal to the size of the phase array in the second size output mode, if the light intensity of the current shooting scene is less than the second preset threshold, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the second size output mode, and the second size output mode is adopted to output the phase array with the same size as the pixel array. That is, when the light intensity of the current shooting scene is weaker, the signal-to-noise ratio of the phase information is improved by reducing the phase array.

[0253] In one embodiment, if the phase information output mode is the second size output mode, the color pixel group and the full-color pixel group in the pixel array are selected as the target pixel group, including:

[0254] Two color pixel groups adjacent to each other along a first diagonal direction in the pixel array are used as target pixel groups and two panchromatic pixel groups adjacent to each other along a second diagonal direction in the pixel array are used as target pixel groups;

[0255] According to the phase information of the target pixel, a phase array corresponding to the pixel array is generated, including:

[0256] Merging the phase information of the target pixels of the two full-color pixel groups in the target pixel group, and merging the phase information of the target pixels of the two color pixel groups in the target pixel group to generate multiple groups of target merged phase information;

[0257] The phase information of the plurality of groups of targets is combined to generate a phase array of a second size corresponding to the pixel array.

[0258] This embodiment is a specific implementation step of outputting a phase array corresponding to a pixel array according to a second size output mode when the light intensity of the current shooting scene is less than a second preset threshold. Wherein, at this time, it is at dusk or in an environment where the light intensity is less than 500 lux. Wherein, firstly, two color pixel groups adjacent along the first diagonal direction in the pixel array are used as target pixel groups and two panchromatic pixel groups adjacent along the second diagonal direction are used as target pixel groups. Because when the light intensity of the current shooting scene is less than the second preset threshold, that is, in a dark light scene, the panchromatic pixels can capture more light information in an extremely dark light scene, so at this time, two color pixel groups adjacent along the first diagonal direction in the pixel array can be used as target pixel groups and two panchromatic pixel groups adjacent along the second diagonal direction can be used as target pixel groups. The phase information of the target pixels of the two panchromatic pixel groups in the target pixel group is merged, and the phase information of the target pixels of the two color pixel groups in the target pixel group is merged to generate multiple groups of target merged phase information. According to the multiple groups of target merged phase information, a second size phase array corresponding to the pixel array is generated. The process of merging the phase information of the target pixels of the two full-color pixel groups in the target pixel group can refer to the process of merging the phase information of the target pixels of the two full-color pixel groups in the next embodiment. The process of merging the phase information of the target pixels of the two color pixel groups in the target pixel group can also refer to the process of merging the phase information of the target pixels of the two full-color pixel groups in the next embodiment. This will not be repeated in this implementation.

[0259] In one embodiment, if the phase information output mode is the second size output mode, then taking the color pixel group and the panchromatic pixel group in the pixel array as the target pixel group includes:

[0260] Taking the two adjacent full-color pixel groups along the second diagonal direction as target pixel groups;

[0261] Generating a phase array corresponding to the pixel array according to the phase information of the target pixel includes:

[0262] Merging the phase information of the target pixels of the two full-color pixel groups in the target pixel group to generate multiple groups of target merged phase information;

[0263] A second-size phase array corresponding to the pixel array is generated according to the multiple sets of target combined phase information.

[0264] This embodiment is a specific implementation step of outputting a phase array corresponding to a pixel array according to a second size output mode when the light intensity of the current shooting scene is less than a second preset threshold. In this case, it is dusk or in an environment where the light intensity is less than 500 lux. Fig.16 As shown, first, two color pixel groups adjacent to each other along the first diagonal direction in the pixel array are used as target pixel groups and two panchromatic pixel groups adjacent to each other along the second diagonal direction are used as target pixel groups. Because the light intensity of the current shooting scene is less than the second preset threshold, that is, in a dark scene, the panchromatic pixel can capture more light information in an extremely dark scene, so at this time, the phase information of the panchromatic pixel group can be used to achieve phase focusing (PDAF). Therefore, when outputting the phase array corresponding to the pixel array according to the second size output mode, the panchromatic pixel group can be selected as the target pixel group, or the color pixel group and the panchromatic pixel group can be selected as the target pixel group.

[0265] Specifically, for the case where the target pixel group is a full-color pixel group, phase focusing can be achieved using the phase information of some full-color pixel groups in the pixel array, or some full-color pixels in some full-color pixel groups can be used to achieve phase focusing, and this application does not limit this. For the case where the target pixel group is a color pixel group and a full-color pixel group, phase focusing can be achieved using the phase information of some color pixel groups and some full-color pixel groups in the pixel array, or some color pixels in some color pixel groups and some full-color pixels in some full-color pixel groups can be used to achieve phase focusing, and this application does not limit this.

[0266] Since only the phase information of some pixel groups can be used for phase focusing, or only the phase information of some pixels in some pixel groups can be used for phase focusing, the amount of data of the output phase information is reduced, thereby improving the efficiency of phase focusing.

[0267] Secondly, a target microlens is determined from the microlenses corresponding to the target pixel group, and at least two pixels corresponding to the target microlens are used as target pixels.

[0268] Specifically, a target microlens in the first direction corresponding to the target pixel group can be determined; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction. The at least two pixels corresponding to the target microlens in the first direction are taken as target pixels. A target microlens in the second direction corresponding to the target pixel group can also be determined; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; the second direction is perpendicular to the first direction. The at least two pixels corresponding to the target microlens in the second direction are taken as target pixels. A target microlens in the first direction and a target microlens in the second direction corresponding to the target pixel group can also be determined; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction. The at least two pixels corresponding to the target microlens in the first direction and the at least two pixels corresponding to the target microlens in the second direction are taken as target pixels.

[0269] Again, for each target pixel group, the phase information of each target pixel in the target pixel group is obtained. Each pixel corresponds to a photosensitive element. On the one hand, assuming that the two adjacent full-color pixel groups along the second diagonal direction in the pixel array are taken as target pixel groups, for the full-color pixel groups in each target pixel group, the phase information of each target pixel in the full-color pixel group is obtained. The phase information of the target pixels of the two full-color pixel groups in the target pixel group is merged, and the phase information of the target pixels of the two color pixel groups in the target pixel group is merged to generate multiple groups of target merged phase information; finally, according to the multiple groups of target merged phase information, a second size phase array corresponding to the pixel array is generated.

[0270] Combination Fig.16As shown, a pixel array may include 2 red pixel groups, 4 green pixel groups, 2 blue pixel groups, and 8 full-color pixel groups. Assuming that the two full-color pixel groups adjacent to each other along the second diagonal direction in the pixel array are both taken as target pixel groups, the phase information of each target pixel group is calculated in sequence for the 8 full-color pixel groups included in the pixel array, i.e., 4 pairs of full-color pixel groups. For example, for the full-color pixel group in each target pixel group, the phase information of the full-color pixel group is calculated. Specifically, for the first full-color pixel group in the target pixel group, the first full-color pixel group includes 9 full-color pixels arranged in a 3×3 array, which are numbered in sequence as full-color pixel 1, full-color pixel 2, full-color pixel 3, full-color pixel 4, full-color pixel 5, full-color pixel 6, full-color pixel 7, full-color pixel 8, and full-color pixel 9. Among them, each pixel corresponds to a photosensitive element. That is, the phase information of full-color pixel 1 is L1, the phase information of full-color pixel 2 is R1; the phase information of full-color pixel 8 is L2, the phase information of full-color pixel 9 is L2; ​​the phase information of full-color pixel 3 is U1, the phase information of full-color pixel 6 is D1; ​​the phase information of full-color pixel 4 is U1, the phase information of full-color pixel 7 is D1. Among them, pixel 5 at the center of the full-color pixel group corresponds to a microlens, so no phase information is obtained.

[0271] Similarly, for the second full-color pixel group in the target pixel group, the second full-color pixel group includes 9 full-color pixels arranged in a 3×3 array, which are numbered as full-color pixel 1, full-color pixel 2, full-color pixel 3, full-color pixel 4, full-color pixel 5, full-color pixel 6, full-color pixel 7, full-color pixel 8, and full-color pixel 9. Among them, each pixel corresponds to a photosensitive element. That is, the phase information of full-color pixel 1 is L1, and the phase information of full-color pixel 2 is R1; the phase information of full-color pixel 8 is L2, and the phase information of full-color pixel 9 is L2; ​​the phase information of full-color pixel 3 is U1, and the phase information of full-color pixel 6 is D1; ​​the phase information of full-color pixel 4 is U1, and the phase information of full-color pixel 7 is D1. Among them, pixel 5 at the center of the full-color pixel group corresponds to a microlens, so no phase information is obtained.

[0272] Therefore, if the target microlens of the first direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens of the first direction are used as target pixels. Assuming that the first direction is the horizontal direction, L1, R1, L2, and R2 corresponding to the first full-color pixel group are output in sequence, and L1, R1, L2, and R2 corresponding to the second full-color pixel group are output in sequence. Then the phase information of the target pixels is merged to generate multiple sets of target merged phase information. For example, L1 and L2 corresponding to the first full-color pixel group are merged with L1 and L2 corresponding to the second full-color pixel group to generate the first target merged phase information L; R1 and R2 corresponding to the first full-color pixel group are merged with R1 and R2 corresponding to the second full-color pixel group to generate the first target merged phase information R.

[0273] In this way, the above processing is performed on the other full-color pixel groups included in the pixel array in turn, and then L3 and L4 corresponding to the first full-color pixel group are merged with L3 and L4 corresponding to the second full-color pixel group to generate the first target merged phase information L; R3 and R4 corresponding to the first full-color pixel group are merged with R3 and R4 corresponding to the second full-color pixel group to generate the first target merged phase information R. L5 and L6 corresponding to the first full-color pixel group are merged with L5 and L6 corresponding to the second full-color pixel group to generate the first target merged phase information L; R6 and R6 corresponding to the first full-color pixel group are merged with R5 and R6 corresponding to the second full-color pixel group to generate the first target merged phase information R. L3 and L4 corresponding to the first full-color pixel group are merged with L3 and L4 corresponding to the second full-color pixel group to generate the first target merged phase information L; R7 and R8 corresponding to the first full-color pixel group are merged with R7 and R8 corresponding to the second full-color pixel group to generate the first target merged phase information R.

[0274] Finally, according to the multiple sets of target merged phase information, a second size phase array corresponding to the pixel array is generated. In this way, the second size phase array is generated by sequentially arranging the four sets of first target merged phase information L and the first target merged phase information R. And the size of the second size phase array is equivalent to the size of the 4×2 pixels arranged in the array.

[0275] Of course, it is also possible to merge or transform the four groups of first target merged phase information L and the first target merged phase information R to generate a second size phase array corresponding to the pixel array. The transformation processing here may be correction and other processing on the four groups of first target merged phase information L and the first target merged phase information R, and the present application does not limit this. When merging the four groups of first target merged phase information L and the first target merged phase information R, a phase array equivalent to a size of 4×2 pixels can be merged into a phase array of a size of 2×2 pixels. Of course, the present application does not limit the specific size of the merged phase array. Here, the size of a pixel refers to the area size of a pixel, and the area size is related to the length and width of the pixel.

[0276] Among them, a pixel is the smallest photosensitive unit on a digital camera photosensitive device (CCD or CMOS). In general, a pixel does not have a fixed size, and the size of a pixel is related to the size and resolution of the display screen. For example, if the size of the display screen is 4.5 inches and the resolution of the display screen is 1280×720, the length of the display screen is 99.6 mm and the width is 56 mm. The length of a pixel is 99.6 mm / 1280=0.0778 mm, and the width is also 56 mm / 720=0.0778 mm. In this example, the size of the 4×2 pixels arranged in the array is: 4×0.0778 mm in length and 2×0.0778 mm in width. Of course, this application does not limit this. Then, the size of the full-size phase array is 4×0.0778 mm in length and 2×0.0778 mm in width. Of course, in other embodiments, the pixel may not be a rectangle with equal length and width, and the pixel may also have other heterogeneous structures, which this application does not limit.

[0277] Similarly, if the target microlens of the second direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens of the second direction are used as target pixels. Assuming that the second direction is a vertical direction, U1, D1, U2, and D2 corresponding to the first full-color pixel group are output in sequence, and U1, D1, U2, and D2 corresponding to the second full-color pixel group are output in sequence. Then the phase information of the target pixels is merged to generate multiple sets of target merged phase information. For example, U1 and U2 corresponding to the first full-color pixel group are merged with U1 and U2 corresponding to the second full-color pixel group to generate the first target merged phase information U; D1 and D2 corresponding to the first full-color pixel group are merged with D1 and D2 corresponding to the second full-color pixel group to generate the first target merged phase information D.

[0278] In this way, the above processing is performed on the other full-color pixel groups included in the pixel array in turn, and then U3 and U4 corresponding to the first full-color pixel group are merged with U3 and U4 corresponding to the second full-color pixel group to generate the first target merged phase information U; D3 and D4 corresponding to the first full-color pixel group are merged with D3 and D4 corresponding to the second full-color pixel group to generate the first target merged phase information D. U5 and U6 corresponding to the first full-color pixel group are merged with U5 and U6 corresponding to the second full-color pixel group to generate the first target merged phase information U; D6 and D6 corresponding to the first full-color pixel group are merged with D5 and D6 corresponding to the second full-color pixel group to generate the first target merged phase information D. U3 and U4 corresponding to the first full-color pixel group are merged with U3 and U4 corresponding to the second full-color pixel group to generate the first target merged phase information U; D7 and D8 corresponding to the first full-color pixel group are merged with D7 and D8 corresponding to the second full-color pixel group to generate the first target merged phase information D.

[0279] Finally, according to the multiple sets of target merged phase information, a second size phase array corresponding to the pixel array is generated. In this way, the second size phase array is generated by sequentially arranging the four sets of first target merged phase information U and the first target merged phase information D. And the size of the second size phase array is equivalent to the size of the 4×2 pixels arranged in the array.

[0280] Similarly, if the target microlens in the first direction and the target microlens in the second direction corresponding to the target pixel group are determined, and at least two pixels corresponding to the target microlens in the first direction and the target microlens in the second direction are used as target pixels. Assuming that the first direction is the horizontal direction and the second direction is the vertical direction, L1, R1, L2, R2, U1, D1, U2, and D2 are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of target merged phase information. For example, L1 and L2 corresponding to the first full-color pixel group are merged with L1 and L2 corresponding to the second full-color pixel group to generate the first target merged phase information L; R1 and R2 corresponding to the first full-color pixel group are merged with R1 and R2 corresponding to the second full-color pixel group to generate the first target merged phase information R. Merge U1 and U2 corresponding to the first full-color pixel group with U1 and U2 corresponding to the second full-color pixel group to generate first target merged phase information U; merge D1 and D2 corresponding to the first full-color pixel group with D1 and D2 corresponding to the second full-color pixel group to generate first target merged phase information D.

[0281] Then, the above processing is performed on the other full-color pixel groups included in the pixel array in turn to obtain the phase information of the pixel array. Finally, the phase information of the target pixels is merged to generate multiple sets of target merged phase information, and the multiple sets of target merged phase information are used as the second size phase array corresponding to the pixel array. And the size of the second full-size phase array is equivalent to the size of 4×4 pixels arranged in the array.

[0282] On the other hand, assuming that all panchromatic pixel groups and color pixel groups in the pixel array are taken as target pixel groups, for each panchromatic pixel group and each color pixel group, the phase information of each target pixel in the panchromatic pixel group and each color pixel group is obtained. The phase information of the target pixels is merged to generate multiple sets of target merged phase information; finally, a second-size phase array corresponding to the pixel array is generated according to the multiple sets of target merged phase information. The specific calculation process is the same as the process of calculating the second-size phase array when all panchromatic pixel groups in the pixel array are taken as target pixel groups, and will not be repeated here.

[0283] At this time, the phase array can be input into the ISP, and the phase difference of the pixel array can be calculated by the ISP based on the phase array. Then, the defocus distance is calculated based on the phase difference, and the DAC code value corresponding to the defocus distance is calculated. Finally, the code value is converted into a driving current through the driver IC of the motor (VCM), and the motor drives the lens to move to a clear position. Thus, focus control is achieved based on the phase difference.

[0284] In an embodiment of the present application, when the light intensity of the current shooting scene is less than the second preset threshold, because the light intensity at this time is weaker, the phase information collected by the color pixel group is not very accurate, and some color pixel groups may not collect phase information. Therefore, two color pixel groups adjacent to each other along the first diagonal direction and two full-color pixel groups adjacent to each other along the second diagonal direction in the pixel array are used as target pixel groups, or two full-color pixel groups adjacent to each other along the second diagonal direction are used as target pixel groups, and for each target pixel group, the phase information of each pixel in the target pixel group is obtained. And the phase information of the target pixel can be obtained from at least one of the microlenses in the first direction and the microlenses in the second direction, so a variety of phase information can be obtained.

[0285] The phase information of the target pixels is combined to generate multiple sets of target combined phase information. Based on the multiple sets of target combined phase information, a second size phase array corresponding to the pixel array is generated. The phase information of the target pixels is greatly combined using the second size output mode to improve the accuracy of the output phase information and the signal-to-noise ratio of the phase information. Finally, phase focusing is performed based on the second size phase array corresponding to the pixel array to improve the accuracy of focusing.

[0286] Continuing from the previous embodiment, if at least two pixels corresponding to the target microlenses in the first direction are taken as target pixels, the phase information of the target pixels is merged to generate multiple sets of target merged phase information, including:

[0287] Acquire at least four target microlenses in the first direction of two full-color pixel groups in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least four target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses;

[0288] The phase information of the target pixels at the same position are combined to generate multiple sets of first combined phase information.

[0289] Combination Fig.16 As shown, if the target microlens of the first direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens of the first direction in the first full-color pixel group and the second full-color pixel group are used as target pixels. Among them, the target pixels at the same position are in the same orientation in the first direction in the corresponding target microlens. Assuming that the first direction is the horizontal direction, L1, R1, L2, and R2 corresponding to the first full-color pixel group are output in sequence, and L1, R1, L2, and R2 corresponding to the second full-color pixel group are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of target merged phase information.

[0290] Specifically, at least four target microlenses in the first direction (gray filled in the figure) are obtained from the first full-color pixel group and the second full-color pixel group in the target pixel group, and the target pixels in the same position are determined from the pixels corresponding to the at least four target microlenses in the first direction. For example, the target pixels in the same position are determined from the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a in the first direction (gray filled in the figure) in the first full-color pixel group, and the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a in the first direction (gray filled in the figure) in the second full-color pixel group. That is, the pixel on the left side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel, and the pixel on the right side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel. That is, L1 and L2 in the first full-color pixel group and L1 and L2 in the second full-color pixel group are merged to generate the first target merged phase information L; R1 and R2 in the first full-color pixel group and R1 and R2 in the second full-color pixel group are merged to generate the first target merged phase information R.

[0291] Based on the multiple sets of first target combined phase information L and the first target combined phase information R, multiple sets of first combined phase information are generated.

[0292] In an embodiment of the present application, at least four target microlenses in the first direction within the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least four target microlenses in the first direction. The phase information of the target pixels at the same position is merged to generate multiple sets of first merged phase information. The phase information of the pixels is significantly merged using the second size output mode to improve the accuracy of the output phase information and the signal-to-noise ratio of the phase information. Finally, phase focusing is performed based on the second size phase array corresponding to the pixel array, so that the focusing accuracy can be improved.

[0293] Continuing from the previous embodiment, assuming that the first direction is a vertical direction, the vertical direction is named as the second direction here. If at least two pixels corresponding to the target microlens in the second direction are used as target pixels, the phase information of the target pixels in the target pixel group is merged to generate multiple groups of target merged phase information, including:

[0294] Acquire at least four target micro-lenses in the second direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least four target micro-lenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target micro-lenses;

[0295] The phase information of the target pixels at the same position are combined to generate multiple sets of second combined phase information.

[0296] Combination Fig.16 As shown, if the target microlens in the second direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens in the second direction in the first full-color pixel group and the second full-color pixel group are used as target pixels. The target pixels at the same position are in the same orientation in the second direction in the corresponding target microlens. Assuming that the second direction is the vertical direction, U1, D1, U2, and D2 corresponding to the first full-color pixel group are output in sequence, and U1, D1, U2, and D2 corresponding to the second full-color pixel group are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of target merged phase information.

[0297] Specifically, at least four target microlenses in the second direction (filled with oblique lines in the figure) are obtained from the first full-color pixel group and the second full-color pixel group in the target pixel group, and the target pixels in the same position are determined from the pixels corresponding to the at least four target microlenses in the second direction. For example, the target pixels in the same position are determined from the pixels 3, 5, 4, and 7 corresponding to the target microlens 2211a in the second direction (filled with oblique lines in the figure) in the first full-color pixel group, and the pixels 3, 6, 3, and 7 corresponding to the target microlens 2211a in the second direction (filled with oblique lines in the figure) in the second full-color pixel group. That is, the pixels above the target microlens 2211a in the second direction (filled with oblique lines in the figure) are determined as target pixels, and the pixels below the target microlens 2211a in the second direction (filled with oblique lines in the figure) are determined as target pixels. That is, U1 and U2 in the first full-color pixel group and U1 and U2 in the second full-color pixel group are merged to generate the second target merged phase information U; D1 and D2 in the first full-color pixel group and D1 and D2 in the second full-color pixel group are merged to generate the second target merged phase information D.

[0298] Based on the multiple sets of second target combined phase information U and the second target combined phase information D, multiple sets of second combined phase information are generated.

[0299] In an embodiment of the present application, at least four target microlenses in the second direction within the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least four target microlenses in the second direction. The phase information of the target pixels at the same position is merged to generate multiple sets of second merged phase information. The phase information of the pixels is significantly merged using the second size output mode to improve the accuracy of the output phase information and the signal-to-noise ratio of the phase information. Finally, phase focusing is performed based on the second size phase array corresponding to the pixel array, so that the focusing accuracy can be improved.

[0300] Continuing from the previous embodiment, if at least two pixels corresponding to the target microlens in the first direction and at least two pixels corresponding to the target microlens in the second direction are used as target pixels, the phase information of the target pixels of the two full-color pixel groups in the target pixel group is combined to generate multiple groups of target combined phase information, including:

[0301] Acquire at least four target microlenses in a first direction of two full-color pixel groups in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least four target microlenses in the first direction,

[0302] Merging the phase information of target pixels at the same position to generate multiple sets of first merged phase information;

[0303] Acquire at least four target micro-lenses in the second direction of two full-color pixel groups in the target pixel group, and determine the target pixels at the same position from the pixels corresponding to the at least four target micro-lenses in the second direction;

[0304] The phase information of the target pixels at the same position are combined to generate multiple sets of second combined phase information.

[0305] Specific, combined Fig.16 As shown, if the target microlens in the first direction corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens in the first direction in the first full-color pixel group and the second full-color pixel group are used as target pixels. Assuming that the first direction is the horizontal direction, L1, R1, L2, and R2 corresponding to the first full-color pixel group are output in sequence, and L1, R1, L2, and R2 corresponding to the second full-color pixel group are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of target merged phase information.

[0306] Specifically, at least four target microlenses in the first direction (gray filled in the figure) are obtained from the first full-color pixel group and the second full-color pixel group in the target pixel group, and the target pixels in the same position are determined from the pixels corresponding to the at least four target microlenses in the first direction. For example, the target pixels in the same position are determined from the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a in the first direction (gray filled in the figure) in the first full-color pixel group, and the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a in the first direction (gray filled in the figure) in the second full-color pixel group. That is, the pixel on the left side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel, and the pixel on the right side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel. That is, L1 and L2 in the first full-color pixel group and L1 and L2 in the second full-color pixel group are merged to generate the first target merged phase information L; R1 and R2 in the first full-color pixel group and R1 and R2 in the second full-color pixel group are merged to generate the first target merged phase information R.

[0307] Based on the multiple sets of first target combined phase information L and the first target combined phase information R, multiple sets of first combined phase information are generated.

[0308] Combination Fig.16As shown, if the target microlens in the first direction and the target microlens in the second direction corresponding to the target pixel group are determined, and at least two pixels corresponding to the target microlens in the first direction in the first full-color pixel group and the second full-color pixel group are used as target pixels, and at least two pixels corresponding to the target microlens in the second direction in the first full-color pixel group and the second full-color pixel group are also used as target pixels. Assuming that the first direction is the horizontal direction and the second direction is the vertical direction, L1, R1, L2, and R2 corresponding to the first full-color pixel group are output in sequence, L1, R1, L2, and R2 corresponding to the second full-color pixel group are output in sequence, U1, D1, U2, and D2 corresponding to the first full-color pixel group are output in sequence, and U1, D1, U2, and D2 corresponding to the second full-color pixel group are output in sequence, and then the phase information of the target pixels is merged to generate multiple sets of target merged phase information.

[0309] Specifically, at least four target microlenses in the first direction (gray filled in the figure) are obtained from the first full-color pixel group and the second full-color pixel group in the target pixel group, and the target pixels in the same position are determined from the pixels corresponding to the at least four target microlenses in the first direction. For example, the target pixels in the same position are determined from the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a in the first direction (gray filled in the figure) in the first full-color pixel group, and the pixels 1, 2, 8, and 9 corresponding to the target microlens 2211a in the first direction (gray filled in the figure) in the second full-color pixel group. That is, the pixel on the left side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel, and the pixel on the right side of the target microlens 2211a in the first direction (gray filled in the figure) is determined as the target pixel. That is, L1 and L2 in the first full-color pixel group and L1 and L2 in the second full-color pixel group are merged to generate the first target merged phase information L; R1 and R2 in the first full-color pixel group and R1 and R2 in the second full-color pixel group are merged to generate the first target merged phase information R.

[0310] Further, at least four target microlenses in the second direction (filled with oblique lines in the figure) are obtained from the first full-color pixel group and the second full-color pixel group in the target pixel group, and the target pixels in the same position are determined from the pixels corresponding to the at least four target microlenses in the second direction. For example, the target pixels in the same position are determined from the pixels 3, 5, 4, and 7 corresponding to the target microlens 2211a in the second direction (filled with oblique lines in the figure) in the first full-color pixel group, and the pixels 3, 6, 3, and 7 corresponding to the target microlens 2211a in the second direction (filled with oblique lines in the figure) in the second full-color pixel group. That is, the pixels above the target microlens 2211a in the second direction (filled with oblique lines in the figure) are determined as target pixels, and the pixels below the target microlens 2211a in the second direction (filled with oblique lines in the figure) are determined as target pixels. That is, U1 and U2 in the first full-color pixel group and U1 and U2 in the second full-color pixel group are merged to generate the second target merged phase information U; D1 and D2 in the first full-color pixel group and D1 and D2 in the second full-color pixel group are merged to generate the second target merged phase information D.

[0311] Based on the multiple sets of first target combined phase information L, the first target combined phase information R, and the multiple sets of second target combined phase information U, the second target combined phase information D, multiple sets of second combined phase information are generated.

[0312] In an embodiment of the present application, at least four target microlenses in the first direction and at least four target microlenses in the second direction of two full-color pixel groups in the target pixel group are obtained, and the target pixels at the same position are determined from the pixels corresponding to the at least four target microlenses in the first direction and the at least four target microlenses in the second direction. The phase information of the target pixels at the same position is merged to generate multiple sets of second merged phase information. Therefore, the phase information can be obtained based on the target microlenses in the first direction and the second direction at the same time, and then the phase information of the pixels can be greatly merged using the second size output mode to improve the accuracy of the output phase information and the signal-to-noise ratio of the phase information. Finally, phase focusing can be performed based on the second size phase array corresponding to the pixel array to improve the accuracy of focusing.

[0313] In one embodiment, if the target pixel group is two color pixel groups adjacent along a first diagonal direction and two panchromatic pixel groups adjacent along a second diagonal direction in the pixel array, a second-size phase array corresponding to the pixel array is generated according to multiple sets of target merged phase information, including:

[0314] Determine a third phase weight corresponding to the color pixel group and a fourth phase weight corresponding to the panchromatic pixel group according to the light intensity of the current shooting scene; wherein the third phase weight corresponding to the color pixel group is different under different light intensities, and the fourth phase weight corresponding to the panchromatic pixel group is different under different light intensities;

[0315] A phase array of a second size of the pixel array in the second direction is generated based on the target phase information and the third phase weight of the color pixel group and the target phase information and the fourth phase weight of the panchromatic pixel group.

[0316] Specifically, in a scenario where the light intensity of the current shooting scene is less than the second preset threshold, and the determined target pixel group includes two color pixel groups adjacent to each other along the first diagonal direction and two panchromatic pixel groups adjacent to each other along the second diagonal direction in the pixel array, then, when generating the second-size phase array based on the phase information of the target pixel group, the weights between different pixel groups can be considered. Among them, the third phase weight corresponding to the color pixel group and the fourth phase weight corresponding to the panchromatic pixel group can be determined according to the light intensity of the current shooting scene. Specifically, the further away the light intensity of the current shooting scene is from the second preset threshold, the smaller the third phase weight corresponding to the color pixel group is at this time, and the larger the fourth phase weight corresponding to the panchromatic pixel group is. Because the light intensity is relatively small in the scenario where the light intensity is less than the second preset threshold, the larger the fourth phase weight corresponding to the panchromatic pixel group is, the more accurate the phase information obtained is. As the light intensity increases, the closer the light intensity of the current shooting scene is to the second preset threshold, the larger the third phase weight corresponding to the color pixel group is at this time, and the smaller the fourth phase weight corresponding to the panchromatic pixel group is. Because the light intensity is relatively large in the scene less than the second preset threshold at this time, the larger the third phase weight corresponding to the color pixel group, the more comprehensive and accurate the phase information obtained. Among them, the third phase weight corresponding to the color pixel group is different under different light intensities, and the fourth phase weight corresponding to the full-color pixel group is different under different light intensities. For example, when the light intensity of the current shooting scene is 450lux, the third phase weight corresponding to the color pixel group is determined to be 40%, wherein the phase weight of the green pixel group is 20%, the phase weight of the red pixel group is 10%, and the phase weight of the blue pixel group is 10%. And the fourth phase weight corresponding to the full-color pixel group is determined to be 60%, which is not limited in this application.

[0317] Then, based on the multiple target merged phase information and the third phase weight of the color pixel group, the multiple target merged phase information and the fourth phase weight of the full-color pixel group, a second-size phase array of the pixel array can be generated. For example, for the pixel array, based on the multiple target merged phase information and the phase weight of 10% of the first red pixel group, the multiple target merged phase information and the phase weight of 10% of the second red pixel group, the multiple target merged phase information and the phase weight of 10% of the first blue pixel group, the multiple target merged phase information and the phase weight of 10% of the second blue pixel group, the multiple target merged phase information and the phase weight of 20% of each green pixel group, and the multiple target merged phase information and the phase weight of 60% of each full-color pixel group, the phase information of the pixel array in the first direction is calculated by summing up, and the second-size phase array is obtained.

[0318] In an embodiment of the present application, when performing phase focusing, if the target pixel group is determined to be two color pixel groups adjacent to each other along the first diagonal direction and two full-color pixel groups adjacent to each other along the second diagonal direction in the pixel array, a second-size phase array of the pixel array can be generated based on multiple groups of target merged phase information of the color pixel groups and their third phase weights, and multiple groups of target merged phase information of the full-color pixel groups and their fourth phase weights. In this way, based on the target phase information of the color pixel groups and the full-color pixel groups, a second-size phase array of the pixel array is jointly generated, which can improve the comprehensiveness of the phase information. At the same time, the phase weights of the target phase information of the color pixel groups and the full-color pixel groups are different under different light intensities. In this way, the accuracy of the phase information can be improved by adjusting the weight size under different light intensities.

[0319] In one embodiment, before outputting the phase array corresponding to the pixel array according to the phase information output mode, the method further includes:

[0320] Determining a target pixel array from a plurality of pixel arrays in the image sensor according to a preset extraction ratio and a preset extraction position of the pixel array for focus control;

[0321] According to the phase information output mode, a phase array corresponding to the pixel array is output, including:

[0322] According to the phase information output mode, a phase array corresponding to the target pixel array is output.

[0323] Specifically, the image sensor has a large area and contains tens of thousands of pixel arrays as the smallest units. If all phase information is extracted from the image sensor for phase focusing, the actual amount of calculation will be too large due to the large amount of phase information data, thus wasting system resources and reducing the image processing speed.

[0324] In order to save system resources and improve the image processing speed, the pixel array for focus control can be extracted from multiple pixel arrays in the image sensor according to a preset extraction ratio and a preset extraction position. For example, extraction can be performed according to a preset extraction ratio of 3%, that is, one pixel array is extracted from 32 pixel arrays as a pixel array for focus control. And the extracted pixel arrays are arranged as the vertices of a hexagon, that is, the extracted pixel arrays constitute a hexagon. In this way, phase information can be evenly acquired. Of course, the present application does not limit the preset extraction ratio and the preset extraction position.

[0325] Then, according to the light intensity of the current shooting scene, a phase information output mode adapted to the light intensity of the current shooting scene can be determined. And for the pixel array used for focus control, according to the phase information output mode, a phase array corresponding to the pixel array is output; wherein the phase array includes the phase information corresponding to the target pixel in the pixel array. Finally, the phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference.

[0326] In an embodiment of the present application, a target pixel array is determined from multiple pixel arrays in an image sensor according to a preset extraction ratio and a preset extraction position of a pixel array for focus control. In this way, it is not necessary to use all phase information in the image sensor for focusing, but only the phase information corresponding to the target pixel array is used for focusing, which greatly reduces the amount of data and improves the speed of image processing. At the same time, according to the preset extraction position, the target pixel array is determined from multiple pixel arrays in the image sensor, and the phase information can be obtained more evenly. Ultimately, the accuracy of phase focusing is improved.

[0327] In one embodiment, a focus control method is provided, further comprising:

[0328] A first preset threshold and a second preset threshold of light intensity are determined according to the exposure parameter and the size of the pixel.

[0329] Specifically, when determining the threshold of light intensity, it can be determined according to exposure parameters and pixel size, wherein the exposure parameters include shutter speed, lens aperture size and light sensitivity (ISO).

[0330] In an embodiment of the present application, a first preset threshold value and a second preset threshold value of light intensity are determined according to exposure parameters and pixel size, and the light intensity range is divided into three ranges. Thus, in each light intensity range, a phase information output mode corresponding to the light intensity range is adopted, thereby achieving more refined calculation of phase information.

[0331] In one embodiment, Fig.17 As shown, a focus control device 1700 is provided, which is applied to an image sensor, and the device includes:

[0332] The phase information output mode determination module 1720 is used to determine the phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different;

[0333] The phase array output module 1740 is used to output a phase array corresponding to the pixel array according to the phase information output mode; wherein the phase array includes phase information corresponding to the target pixel in the pixel array;

[0334] The focus control module 1760 is used to calculate the phase difference of the pixel array based on the phase array, and perform focus control according to the phase difference.

[0335] In one embodiment, Fig.18 As shown, the phase array output module 1740 includes:

[0336] A target pixel determination unit 1742 is used to determine a target pixel group from the color pixel group and the panchromatic pixel group in the pixel array according to the phase information output mode, determine a target microlens corresponding to the target pixel group, and use at least two pixels corresponding to the target microlens as target pixels;

[0337] A phase information generating unit 1744 is used to obtain phase information of target pixels for each target pixel group;

[0338] The phase array generating unit 1746 is used to generate a phase array corresponding to the pixel array according to the phase information of the target pixel.

[0339] In one embodiment, the target pixel determination unit 1742 is further used to determine a target microlens in a first direction corresponding to the target pixel group; the target microlens in the first direction corresponds to at least two target pixels arranged along the first direction and adjacent to each other; and at least two pixels corresponding to the target microlens in the first direction are used as target pixels.

[0340] In one embodiment, the target pixel determination unit 1742 is further used to determine a target microlens in a second direction corresponding to the target pixel group; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; and at least two pixels corresponding to the target microlens in the second direction are used as target pixels.

[0341] In one embodiment, the target pixel determination unit 1742 is further used to determine a target microlens in a first direction and a target microlens in a second direction corresponding to the target pixel group; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; the second direction is perpendicular to the first direction; and at least two pixels corresponding to the target microlens in the first direction and at least two pixels corresponding to the target microlens in the second direction are used as target pixels.

[0342] In one embodiment, the phase information output mode determination module 1720 is also used to determine the target light intensity range to which the light intensity of the current shooting scene belongs; wherein different light intensity ranges correspond to different phase information output modes; and according to the target light intensity range, determine the phase information output mode that is adapted to the light intensity of the current shooting scene.

[0343] In one embodiment, the phase information output mode includes a full-size output mode and a first-size output mode, and the size of the phase array in the full-size output mode is greater than or equal to the size of the phase array in the first-size output mode;

[0344] The phase information output mode determination module 1720 includes:

[0345] a full-size output mode determination unit, configured to determine that a phase information output mode adapted to the light intensity of the current shooting scene is a full-size output mode if the light intensity of the current shooting scene is greater than a first preset threshold;

[0346] The first size output mode determination unit is used to determine that the phase information output mode adapted to the light intensity of the current shooting scene is the first size output mode if the light intensity of the current shooting scene is greater than the second preset threshold and less than or equal to the first preset threshold; the first preset threshold is greater than the second preset threshold.

[0347] In one embodiment, if the phase information output mode is the full-size output mode, the target pixel determination unit 1742 is further configured to use the color pixel group in the pixel array as the target pixel group; or use the color pixel group and the full-color pixel group in the pixel array as the target pixel group; wherein one target pixel group corresponds to one pixel group;

[0348] Accordingly,

[0349] The phase array generating unit 1746 is further configured to generate a full-size phase array corresponding to the pixel array according to the phase information of the target pixel.

[0350] In one embodiment, if the phase information output mode is the first size output mode, the target pixel determination unit 1742 is further configured to use at least one of a color pixel group and a full-color pixel group in the pixel array as a target pixel group; the target pixel group corresponds to one pixel group;

[0351] Accordingly,

[0352] The phase array generating unit 1746 is further configured to merge the phase information of the target pixels in the target pixel group to generate multiple sets of intermediate merged phase information; and generate a first size phase array corresponding to the pixel array according to the multiple sets of intermediate merged phase information.

[0353] In one embodiment, if the target pixels are at least two pixels corresponding to the target microlenses in the first direction, the phase array generation unit 1746 is further used to obtain at least two target microlenses in the first direction within the target pixel group, and determine the target pixels at the same position from the pixels corresponding to the at least two target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses; and the phase information of the target pixels at the same position is merged to generate multiple groups of first merged phase information.

[0354] In one embodiment, if the target pixels are at least two pixels corresponding to the target microlenses in the second direction, the phase array generation unit 1746 is further used to obtain at least two target microlenses in the second direction within the target pixel group, and determine the target pixels at the same position from the pixels corresponding to the at least two target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses; and the phase information of the target pixels at the same position is merged to generate multiple groups of second merged phase information.

[0355] In one embodiment, if the target pixels are at least two pixels corresponding to the target microlenses in the first direction and at least two pixels corresponding to the target microlenses in the second direction, the phase array generation unit 1746 is further used to obtain at least two target microlenses in the first direction within the target pixel group, and determine the target pixels at the same position from the pixels corresponding to the at least two target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses; merge the phase information of the target pixels at the same position to generate multiple sets of first merged phase information; obtain at least two target microlenses in the second direction within the target pixel group, and determine the target pixels at the same position from the pixels corresponding to the at least two target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses; merge the phase information of the target pixels at the same position to generate multiple sets of second merged phase information; and generate multiple sets of intermediate merged phase information based on the multiple sets of first merged phase information and the multiple sets of second merged phase information.

[0356] In one embodiment, if the target pixel group includes a color pixel group and a full-color pixel group, the phase array generation unit 1746 is further used to determine a first phase weight corresponding to the color pixel group and a second phase weight corresponding to the full-color pixel group according to the light intensity of the current shooting scene; wherein the first phase weight corresponding to the color pixel group is different under different light intensities, and the second phase weight corresponding to the full-color pixel group is different under different light intensities; based on multiple groups of intermediate merged phase information and first phase weights corresponding to the color pixel group, and multiple groups of intermediate merged phase information and second phase weights corresponding to the full-color pixel group, a first size phase array of the pixel array is generated.

[0357] In one embodiment, the phase information output mode further includes a second size output mode; wherein the size of the phase array in the first size output mode is greater than or equal to the size of the phase array in the second size output mode;

[0358] The phase information output mode determination module 1720 includes:

[0359] The second size output mode determining unit is used to determine the phase information output mode adapted to the light intensity of the current shooting scene as the second size output mode if the light intensity of the current shooting scene is less than a second preset threshold.

[0360] In one embodiment, if the phase information output mode is the second size output mode, the target pixel determination unit 1742 is further configured to use two color pixel groups adjacent to each other along a first diagonal direction in the pixel array as target pixel groups and two panchromatic pixel groups adjacent to each other along a second diagonal direction as target pixel groups;

[0361] The phase array generation unit 1746 is also used to merge the phase information of the target pixels of the two full-color pixel groups in the target pixel group, and merge the phase information of the target pixels of the two color pixel groups in the target pixel group to generate multiple groups of target merged phase information; and generate a second size phase array corresponding to the pixel array according to the multiple groups of target merged phase information.

[0362] In one embodiment, if the phase information output mode is the second size output mode, the target pixel determination unit 1742 is further configured to use two adjacent full-color pixel groups along the second diagonal direction as the target pixel groups;

[0363] The phase array generation unit 1746 is further used to merge the phase information of the target pixels of the two full-color pixel groups in the target pixel group to generate multiple groups of target merged phase information; and generate a second size phase array corresponding to the pixel array according to the multiple groups of target merged phase information.

[0364] In one embodiment, if at least two pixels corresponding to the target microlenses in the first direction are taken as target pixels, the phase array generation unit 1746 is further used to obtain at least four target microlenses in the first direction of two full-color pixel groups in the target pixel group, and determine the target pixels at the same position from the pixels corresponding to the at least four target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses; and the phase information of the target pixels at the same position is merged to generate multiple groups of first merged phase information.

[0365] In one embodiment, if at least two pixels corresponding to the target microlenses in the second direction are taken as target pixels, the phase array generation unit 1746 is further used to obtain at least four target microlenses in the second direction in the two full-color pixel groups of the target pixel group, and determine the target pixels at the same position from the pixels corresponding to the at least four target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses; and the phase information of the target pixels at the same position is merged to generate multiple groups of second merged phase information.

[0366] In one embodiment, if at least two pixels corresponding to the target microlens in the first direction and at least two pixels corresponding to the target microlens in the second direction are used as target pixels, the phase array generation unit 1746 is further configured to:

[0367] Acquire at least four target microlenses in the first direction of two full-color pixel groups in the target pixel group, and determine target pixels at the same position from the pixels corresponding to the at least four target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses; merge the phase information of the target pixels at the same position to generate multiple groups of first merged phase information;

[0368] At least four target microlenses in the second direction of two full-color pixel groups in the target pixel group are obtained, and target pixels at the same position are determined from the pixels corresponding to the at least four target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses; and phase information of the target pixels at the same position is merged to generate multiple groups of second merged phase information.

[0369] In one embodiment, if the target pixel groups are two color pixel groups adjacent to each other along a first diagonal direction and two full-color pixel groups adjacent to each other along a second diagonal direction in the pixel array, the phase array generation unit 1746 is further used to determine the third phase weight corresponding to the color pixel group and the fourth phase weight corresponding to the full-color pixel group according to the light intensity of the current shooting scene; wherein the third phase weight corresponding to the color pixel group is different under different light intensities, and the fourth phase weight corresponding to the full-color pixel group is different under different light intensities; based on the multiple groups of target merged phase information and the third phase weight corresponding to the color pixel groups, and the multiple groups of target merged phase information and the fourth phase weight corresponding to the full-color pixel groups, a second size phase array of the pixel array is generated.

[0370] In one embodiment, a focus control device is provided, further comprising:

[0371] A target pixel array determination module, used to determine a target pixel array from a plurality of pixel arrays in the image sensor according to a preset extraction ratio and a preset extraction position of a pixel array for focus control;

[0372] The phase array output module 1740 is further configured to output a phase array corresponding to the target pixel array according to a phase information output mode.

[0373] In one embodiment, a focus control device is provided, further comprising:

[0374] The threshold determination module is used to determine a first preset threshold, a second preset threshold and a third preset threshold of light intensity according to exposure parameters and pixel size.

[0375] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0376] The division of the various modules in the above focus control device is only for illustration. In other embodiments, the focus control device may be divided into different modules as needed to complete all or part of the functions of the above focus control device.

[0377] For the specific definition of the focus control device, please refer to the definition of the focus control method above, which will not be repeated here. Each module in the above-mentioned focus control device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0378] Fig.19 Schematic diagram of the internal structure of an electronic device in an embodiment. The electronic device can be any terminal device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a car computer, a wearable device, etc. The electronic device includes a processor and a memory connected via a system bus. Among them, the processor may include one or more processing units. The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processing), etc. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement a focus control method provided in each of the following embodiments. The internal memory provides a cache operating environment for the operating system computer program in the non-volatile storage medium.

[0379] The implementation of each module in the focus control device provided in the embodiment of the present application can be in the form of a computer program. The computer program can be run on an electronic device. The program module constituted by the computer program can be stored in the memory of the electronic device. When the computer program is executed by the processor, the steps of the method described in the embodiment of the present application are implemented.

[0380] The embodiment of the present application further provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, the processors execute the steps of the focus control method.

[0381] An embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute a focus control method.

[0382] Any reference to memory, storage, database or other medium used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include ROM (Read-Only Memory), PROM (Programmable Read-only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-only Memory) or flash memory. Volatile memory may include RAM (Random Access Memory), which is used as an external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), Double Data Rate DDRSDRAM (Double Data Rate Synchronous Dynamic Random Access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory), SLDRAM (Sync Link Dynamic Random Access Memory), RDRAM (Rambus Dynamic Random Access Memory), and DRDRAM (Direct Rambus Dynamic Random Access Memory).

[0383] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image sensor, characterized in that: The image sensor comprises a microlens array, a pixel array and a filter array, the filter array comprises a minimum repeating unit, the minimum repeating unit comprises a plurality of filter groups, the filter groups comprise color filters and panchromatic filters; the color filters in different filter groups are different; the color filters have a narrower spectral response than the panchromatic filters, the color filters and the panchromatic filters both comprise 9 sub-filters arranged in an array; the ratio of the number of the color filters to the number of the panchromatic filters is any one of 1:3, 1:1 and 3:1; The pixel array includes a plurality of pixel groups, the pixel groups are full-color pixel groups or color pixel groups, each of the full-color pixel groups corresponds to the full-color filter, and each of the color pixel groups corresponds to the color filter; the full-color pixel group and the color pixel group each include 9 pixels, the pixels of the pixel array are arranged corresponding to the sub-filters of the filter array, and each pixel corresponds to a photosensitive element; The microlens array includes a plurality of microlens groups, each microlens group corresponds to the pixel group, the microlens group includes a plurality of microlenses, and at least one microlens in the plurality of microlenses corresponds to at least two pixels; The microlens group includes 5 microlenses, and the 5 microlenses include 4 first microlenses and 1 second microlens; wherein the first microlenses correspond to 2 pixels respectively, and the second microlens corresponds to 1 pixel; The four first microlenses include two first microlenses in a first direction and two first microlenses in a second direction; the first microlenses in the first direction are arranged along the first direction in the microlens group; the first microlenses in the second direction are arranged along the second direction in the microlens group; the second microlens is located at the center of the microlens group, and the second microlens corresponds to the central pixel of the pixel group corresponding to the microlens group, or the second microlens is located at one of the four corners of the microlens group, and the second microlens corresponds to a pixel at one of the four corners in the pixel group corresponding to the microlens group; If the second microlens is located at the center of the microlens group, two first microlenses in the first direction and two first microlenses in the second direction are arranged around the second microlens; wherein the two first microlenses in the first direction are arranged centrally symmetrically with respect to the second microlens along a first diagonal direction; and the two first microlenses in the second direction are arranged centrally symmetrically with respect to the second microlens along a second diagonal direction.

2. The image sensor according to claim 1, characterized in that There are four filter groups, and the four filter groups are arranged in a matrix.

3. The image sensor according to claim 2, characterized in that In each of the filter groups, the panchromatic filters are arranged in a first diagonal direction, and the color filters are arranged in a second diagonal direction, and the first diagonal direction is different from the second diagonal direction.

4. The image sensor according to claim 3, characterized in that: The filter set includes 2 full-color filters and 2 color filters, and the minimum repeating unit is 12 rows and 12 columns of 144 sub-filters, arranged as follows: Among them, w represents the full-color sub-filter, and a, b and c all represent color sub-filters.

5. The image sensor according to claim 3, characterized in that: The filter set includes 2 full-color filters and 2 color filters, and the minimum repeating unit is 12 rows and 12 columns of 144 sub-filters, arranged as follows: Among them, w represents the full-color sub-filter, and a, b and c all represent color sub-filters.

6. The image sensor according to claim 1, characterized in that: One microlens among the plurality of microlenses corresponds to at least four pixels.

7. The image sensor according to claim 6, characterized in that: The microlens group includes four microlenses, and the four microlenses include one third microlens, two fourth microlenses and one fifth microlens; wherein the third microlenses correspond to four pixels, the fourth microlenses correspond to two pixels, and the fifth microlens corresponds to one pixel.

8. A focus control method, characterized in that: Applied to the image sensor according to any one of claims 1 to 7, the method comprises: According to the light intensity of the current shooting scene, determine a phase information output mode adapted to the light intensity of the current shooting scene; wherein in different phase information output modes, the size of the output phase array is different; According to the phase information output mode, output a phase array corresponding to the pixel array; wherein the phase array includes phase information corresponding to the target pixel in the pixel array; The phase difference of the pixel array is calculated based on the phase array, and focus control is performed according to the phase difference.

9. The method according to claim 8, characterized in that Outputting a phase array corresponding to the pixel array according to the phase information output mode includes: According to the phase information output mode, a target pixel group is determined from the color pixel group and the full-color pixel group in the pixel array, a target microlens corresponding to the target pixel group is determined, and at least two pixels corresponding to the target microlens are used as target pixels; For each of the target pixel groups, acquiring phase information of the target pixel; A phase array corresponding to the pixel array is generated according to the phase information of the target pixel.

10. The method according to claim 9, characterized in that Determining a target microlens corresponding to the target pixel group, and taking at least two pixels corresponding to the target microlens as target pixels, comprises: Determine a target microlens in a first direction corresponding to the target pixel group; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction; At least two pixels corresponding to the target microlenses in the first direction are used as target pixels.

11. The method according to claim 9, characterized in that Determining a target microlens corresponding to the target pixel group, and taking at least two pixels corresponding to the target microlens as target pixels, comprises: Determine a target microlens in a first direction and a target microlens in a second direction corresponding to the target pixel group; the target microlens in the first direction corresponds to at least two target pixels arranged and adjacent along the first direction; the target microlens in the second direction corresponds to at least two target pixels arranged and adjacent along the second direction; the second direction is perpendicular to the first direction; At least two pixels corresponding to the target microlens in the first direction and at least two pixels corresponding to the target microlens in the second direction are used as target pixels.

12. The method according to claim 10 or 11, characterized in that: The step of determining a phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene comprises: Determine the target light intensity range to which the light intensity of the current shooting scene belongs; wherein different light intensity ranges correspond to different phase information output modes; According to the target light intensity range, a phase information output mode adapted to the light intensity of the current shooting scene is determined.

13. The method according to claim 12, characterized in that The phase information output mode includes a full-size output mode and a first-size output mode, and the size of the phase array in the full-size output mode is greater than or equal to the size of the phase array in the first-size output mode; The step of determining a phase information output mode adapted to the light intensity of the current shooting scene according to the target light intensity range includes: If the light intensity of the current shooting scene is greater than a first preset threshold, determining that the phase information output mode adapted to the light intensity of the current shooting scene is the full-size output mode; If the light intensity of the current shooting scene is greater than the second preset threshold and less than or equal to the first preset threshold, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the first size output mode; the first preset threshold is greater than the second preset threshold.

14. The method according to claim 13, characterized in that If the phase information output mode is a full-size output mode, then taking the color pixel group and the full-color pixel group in the pixel array as the target pixel group includes: Using a color pixel group in the pixel array as a target pixel group, or using a color pixel group and a full-color pixel group in the pixel array as a target pixel group; wherein one target pixel group corresponds to one pixel group; Accordingly, Generating a phase array corresponding to the pixel array according to the phase information of the target pixel includes: A full-size phase array corresponding to the pixel array is generated according to the phase information of the target pixel.

15. The method according to claim 13, characterized in that If the phase information output mode is the first size output mode, then taking the color pixel group and the full-color pixel group in the pixel array as the target pixel group includes: At least one of the color pixel group and the full-color pixel group in the pixel array is used as a target pixel group; one target pixel group corresponds to one pixel group; Accordingly, Generating a phase array corresponding to the pixel array according to the phase information of the target pixel includes: Merging the phase information of the target pixels in the target pixel group to generate multiple groups of intermediate merged phase information; A phase array of a first size corresponding to the pixel array is generated according to the multiple sets of intermediate combined phase information.

16. The method according to claim 15, characterized in that If the target pixels are at least two pixels corresponding to the target microlenses in the first direction, then merging the phase information of the target pixels to generate multiple sets of intermediate merged phase information includes: Acquire at least two target micro-lenses in the first direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least two target micro-lenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target micro-lenses; The phase information of the target pixels at the same position are combined to generate multiple sets of first combined phase information.

17. The method according to claim 15, characterized in that If the target pixels are at least two pixels corresponding to the target microlenses in the first direction and at least two pixels corresponding to the target microlenses in the second direction, merging the phase information of the target pixels in the target pixel group to generate multiple sets of intermediate merged phase information includes: Acquire at least two target micro-lenses in the first direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least two target micro-lenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target micro-lenses; Merging the phase information of the target pixels at the same position to generate multiple sets of first merged phase information; Acquire at least two target microlenses in the second direction in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least two target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses; Merging the phase information of the target pixels at the same position to generate multiple sets of second merged phase information; Based on the multiple sets of first combined phase information and the multiple sets of second combined phase information, multiple sets of intermediate combined phase information are generated.

18. The method according to claim 15, characterized in that If the target pixel group includes a color pixel group and a full-color pixel group, then generating a first-size phase array corresponding to the pixel array according to the multiple sets of intermediate combined phase information includes: Determine a first phase weight corresponding to the color pixel group and a second phase weight corresponding to the panchromatic pixel group according to the light intensity of the current shooting scene; A first-size phase array of the pixel array is generated based on multiple sets of intermediate combined phase information and the first phase weights corresponding to the color pixel groups, and multiple sets of intermediate combined phase information and the second phase weights corresponding to the panchromatic pixel groups.

19. The method according to claim 13, characterized in that The phase information output mode also includes a second size output mode; wherein the size of the phase array in the first size output mode is greater than or equal to the size of the phase array in the second size output mode; The step of determining a phase information output mode adapted to the light intensity of the current shooting scene according to the target light intensity range includes: If the light intensity of the current shooting scene is less than a second preset threshold, the phase information output mode adapted to the light intensity of the current shooting scene is determined to be the second size output mode.

20. The method according to claim 19, characterized in that If the phase information output mode is the second size output mode, then taking the color pixel group and the full-color pixel group in the pixel array as the target pixel group includes: Using two color pixel groups adjacent to each other along a first diagonal direction in the pixel array as target pixel groups and two panchromatic pixel groups adjacent to each other along a second diagonal direction as target pixel groups; Generating a phase array corresponding to the pixel array according to the phase information of the target pixel includes: Merging the phase information of the target pixels of the two full-color pixel groups in the target pixel group, and merging the phase information of the target pixels of the two color pixel groups in the target pixel group to generate multiple groups of target merged phase information; A second-size phase array corresponding to the pixel array is generated according to the multiple sets of target combined phase information.

21. The method according to claim 19, characterized in that If the phase information output mode is the second size output mode, then taking the color pixel group and the full-color pixel group in the pixel array as the target pixel group includes: Taking the two adjacent full-color pixel groups along the second diagonal direction as target pixel groups; Generating a phase array corresponding to the pixel array according to the phase information of the target pixel includes: Merging the phase information of the target pixels of the two full-color pixel groups in the target pixel group to generate multiple groups of target merged phase information; A second-size phase array corresponding to the pixel array is generated according to the multiple sets of target combined phase information.

22. The method according to claim 21, characterized in that If at least two pixels corresponding to the target microlens in the first direction are taken as target pixels, then merging the phase information of the target pixels of the two full-color pixel groups in the target pixel group to generate multiple groups of target merged phase information includes: Acquire at least four target microlenses in the first direction of two full-color pixel groups in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least four target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses; The phase information of the target pixels at the same position are combined to generate multiple sets of first combined phase information.

23. The method according to claim 21, characterized in that If at least two pixels corresponding to the target microlens in the first direction and at least two pixels corresponding to the target microlens in the second direction are used as target pixels, then merging the phase information of the target pixels of the two full-color pixel groups in the target pixel group to generate multiple groups of target merged phase information includes: Acquire at least four target microlenses in the first direction of two full-color pixel groups in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least four target microlenses in the first direction; the target pixels at the same position are at the same orientation in the first direction in the corresponding target microlenses; Merging the phase information of the target pixels at the same position to generate multiple sets of first merged phase information; Acquire at least four target microlenses in the second direction of two full-color pixel groups in the target pixel group, and determine target pixels at the same position from pixels corresponding to the at least four target microlenses in the second direction; the target pixels at the same position are at the same orientation in the second direction in the corresponding target microlenses; The phase information of the target pixels at the same position are combined to generate multiple sets of second combined phase information.

24. The method according to claim 20, characterized in that The step of combining phase information of multiple groups of targets to generate a second-size phase array corresponding to the pixel array includes: Determining a third phase weight corresponding to the color pixel group and a fourth phase weight corresponding to the panchromatic pixel group according to the light intensity of the current shooting scene; A second size phase array of the pixel array is generated based on the multiple sets of target merged phase information corresponding to the color pixel group and the third phase weight, the multiple sets of target merged phase information corresponding to the panchromatic pixel group and the fourth phase weight.

25. The method according to claim 8, characterized in that Before outputting the phase array corresponding to the pixel array according to the phase information output mode, the method further includes: Determining a target pixel from a plurality of pixel arrays in the image sensor according to a preset extraction ratio and a preset extraction position of a pixel array for focus control; Outputting a phase array corresponding to the pixel array according to the phase information output mode includes: According to the phase information output mode, a phase array corresponding to the pixel array is output.

26. The method according to claim 13, characterized in that The method further comprises: A first preset threshold and a second preset threshold of the light intensity are determined according to exposure parameters and the size of the pixel.

27. A focus control device, characterized in that: Applied to the image sensor according to any one of claims 1 to 7, the device comprises: A phase information output mode determination module, used to determine a phase information output mode adapted to the light intensity of the current shooting scene according to the light intensity of the current shooting scene; wherein, in different phase information output modes, the size of the output phase array is different; A phase array output module, configured to output a phase array corresponding to a pixel array according to the phase information output mode; wherein the phase array includes phase information corresponding to a target pixel in the pixel array; A focus control module is used to calculate the phase difference of the pixel array based on the phase array, and perform focus control according to the phase difference.

28. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the focus control method according to any one of claims 8 to 26.

29. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the focus control method according to any one of claims 8 to 26 are implemented.

Images