Image sensor, exposure method, exposure device, and electronic device
By using a morphological switching structure in the image sensor to drive the movement of the microlens array, each pixel group can output image information and phase information at different microlens array positions, solving the problem of improving phase focus capability but reducing image clarity in the prior art , achieving high accuracy and high resolution image acquisition.
Patent Information
- Application Number
- CN202411094073.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-09
AI Technical Summary
While existing image sensors improve phase focus capabilities, image clarity is difficult to improve, especially due to the high proportion of PDAF pixels, the perceived color pixel coverage decreases and the focus pixel arrangement limitations.
Using an image sensor including a pixel array and a microlens array, the microlens array is driven to move the microlens array through a morphological switching structure. Each pixel group can be switched from the first microlens to at least two second microlens, or from at least two second microlens. The lens is switched to the first microlens, thereby outputting image information and phase information at different microlens array positions.
It realizes full pixel support for PDAF, improves the clarity and focus capability of the image sensor, and combines high accuracy and high resolution image acquisition.
Smart Images

Figure CN119094877A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of imaging technology, and specifically relates to an image sensor, an exposure method, an exposure device and an electronic device. Background Art
[0002] With the development of shooting technology, users have higher and higher requirements on the focusing ability of image sensors.
[0003] In the related art, when shooting, phase detection autofocus (PDAF) technology can usually be used for autofocus. Specifically, phase focus can be performed by an image sensor that supports PDAF technology, wherein at least some pixels in the image sensor that supports PDAF technology can output phase information. Generally, the higher the percentage of pixels that output phase information from the image sensor, the higher the accuracy of autofocus.
[0004] However, the higher the proportion of pixels in the image sensor that support PDAF, the lower the clarity of the image captured by the image sensor may be due to factors such as reduced pixel coverage for perceived color and limitations on the arrangement of focus pixels. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide an image sensor, an exposure method, an exposure device and an electronic device, which can improve the image clarity while improving the phase focusing capability.
[0006] In a first aspect, an embodiment of the present application provides an image sensor, which includes: a corresponding pixel array and a microlens array, and a morphology switching structure connected to the microlens array; the pixel array includes a plurality of pixel groups, each pixel group includes N pixel units, and N is an integer greater than 1; the microlens array includes a plurality of microlens groups, each microlens group includes a first microlens or at least two second microlenses; wherein each pixel group is correspondingly provided with a first microlens or at least two second microlenses; the morphology switching structure is used to drive the microlens array to move, so as to control each pixel group to switch from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens, wherein when the pixel group corresponds to the first microlens, the pixel group is used to obtain phase information, and when the pixel group corresponds to the at least two second microlenses, the pixel group is used to obtain image information.
[0007] In a second aspect, an embodiment of the present application provides an exposure method, which is applied to an image sensor as in the first aspect, and the exposure method includes: controlling a pixel array of the image sensor to perform row-by-row exposure to obtain a first image frame; controlling the pixel array to stop exposure at a first moment, and switching the microlens group corresponding to each pixel group through a morphology switching structure of the image sensor; wherein the first moment is a moment when the pixel array corresponding to the first image frame has not completed exposure, and the pixel array corresponding to the second image frame starts to be exposed; controlling the pixel array of the image sensor to continue to perform row-by-row exposure; controlling the pixel array to stop exposure at a second moment, and switching the microlens group corresponding to each pixel group through a morphology switching structure; wherein the second moment is a moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure; controlling the pixel array of the image sensor to continue to perform row-by-row exposure until the pixel array corresponding to the second image frame is completely exposed; and obtaining target image information or target phase information based on first data corresponding to the first image frame and second data corresponding to the second image frame.
[0008] In a third aspect, an embodiment of the present application provides an exposure device, which may include an image sensor as in the first aspect, and the exposure device may also include a control module; the control module is used to: control the pixel array of the image sensor to perform row-by-row exposure to obtain a first image frame; control the pixel array to stop exposure at a first moment, and switch the microlens group corresponding to each pixel group through the morphology switching structure of the image sensor; wherein the first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure, and the pixel array corresponding to the second image frame starts to be exposed; control the pixel array of the image sensor to continue to perform row-by-row exposure; control the pixel array to stop exposure at a second moment, and switch the microlens group corresponding to each pixel group through the morphology switching structure; wherein the second moment is the moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure; control the pixel array of the image sensor to continue to perform row-by-row exposure until the pixel array corresponding to the second image frame is exposed; obtain target image information or target phase information based on first data corresponding to the first image frame and second data corresponding to the second image frame.
[0009] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be executed on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the second aspect are implemented.
[0010] In a fifth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method in the second aspect are implemented.
[0011] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method of the second aspect.
[0012] In a seventh aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method of the second aspect.
[0013] In the image sensor provided in the embodiment of the present application, since the microlens array is driven to move forward and backward through the morphology switching structure, each pixel group can be switched from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens, so that each pixel group can output image information and phase information respectively before and after the movement of the microlens array. Therefore, the image sensor provided in the embodiment of the present application can realize full-pixel support for PDAF, and can improve the clarity of the captured image on the basis of improving the phase focusing capability.
[0014] The exposure method provided by the embodiment of the present application, in the process of controlling the pixel array to perform row-by-row exposure, can control the pixel array to stop exposure at the first time when the first image frame is not completed and the second image frame starts to be exposed, and control the morphology switching structure to switch the microlens group corresponding to each pixel group in the pixel array; then control the pixel array to continue to be exposed, and can control the pixel array to stop exposure again at the second moment when the first image frame is completed and the second image frame is not completed, and control the morphology switching structure to switch the microlens group corresponding to each pixel group again to control the pixel array to continue to be exposed; so that each pixel group in the pixel array can output one type of information, such as image information, in the exposure period before the first moment and after the second moment, and output another type of information, such as phase information, in the exposure period after the first moment and before the second moment. Therefore, it is ensured that the first data and the second data include the image information or phase information output by each pixel unit in the pixel array. In this way, the target phase information for autofocus can be generated according to the phase information output by each pixel group, or the image information with higher clarity can be obtained according to the image information output by each pixel group, so that the image sensor can have both the ability to collect clear images and the ability to accurately focus. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 is one of the structural schematic diagrams of the image sensor provided in the embodiment of the present application;
[0016] Figure 2A It is one of the structural schematic diagrams of an image sensor in the related art;
[0017] Figure 2BThis is the second structural schematic diagram of an image sensor in the related art;
[0018] Figure 3A This is the second structural schematic diagram of the image sensor provided in the embodiment of the present application;
[0019] Figure 3B This is the third structural schematic diagram of the image sensor provided in the embodiment of the present application;
[0020] Figure 4A This is the fourth structural schematic diagram of the image sensor provided in the embodiment of the present application;
[0021] Figure 4B This is the fifth structural diagram of the image sensor provided in the embodiment of the present application;
[0022] Figure 5 This is the sixth structural diagram of the image sensor provided in the embodiment of the present application;
[0023] Figure 6 This is the seventh structural diagram of the image sensor provided in the embodiment of the present application;
[0024] Figure 7 is a structural schematic diagram of a camera module provided in an embodiment of the present application;
[0025] Figure 8 It is a schematic diagram of the process of the exposure method provided in the embodiment of the present application;
[0026] Fig. 9 is a schematic diagram of an exposure method provided in an embodiment of the present application;
[0027] Fig.10 is a schematic structural diagram of an exposure device provided in an embodiment of the present application;
[0028] Fig.11 It is one of the structural schematic diagrams of the electronic device provided in the embodiment of the present application;
[0029] Fig.12 This is the second structural diagram of the electronic device provided in the embodiment of the present application;
[0030] in, Figures 1 to 7 The reference numerals in the drawings are:
[0031] 100, image sensor; 10, pixel array; 11, pixel group; 12, pixel unit; 20, microlens array; 21, lens group; 22, first microlens; 23, second microlens; 30, morphology switching structure; 40, exposure reading control module; 50, exposure trigger control module; 60, conversion circuit; 200, camera module; 210, lens group; 220, focus motor; 230, filter; 240, base. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0033] The terms involved in the embodiments of the present application are explained below.
[0034] Image sensor: a device that converts optical images into electronic signals, widely used in digital cameras and other electronic optical devices. Today, image sensors are mainly divided into two types: Charge-coupled Device (CCD) sensors and Complementary Metal-Oxide-Semiconductor (CMOS) active pixel sensors. Among them, CMOS active pixel sensors are hereinafter referred to as CMOS sensors.
[0035] The pixels of CMOS sensors can only sense brightness. If you want to sense color, you need to cover the pixels with a color filter array (CFA). The function of CFA is to filter the light of other bands and allow the light of the required band to pass through and be photoelectrically converted by the pixels, so that the pixels have the ability to sense color. It should be noted that the way of setting CFA is a waste of light energy, and the image sensor's ability to perceive color is relatively limited under this method.
[0036] PDAF: is a technology that achieves fast and accurate focusing by detecting the phase difference between the object and the imaging sensor. Taking a mobile phone as an example, the phase detection autofocus on the mobile phone can directly integrate the autofocus sensor and the pixel sensor. The autofocus sensor can form a phase detection unit for phase detection on the pixel sensor, such as a 2*2 phase detection unit, so that a PDAF sensor can be obtained. Then, the left and right opposite pairs of pixels can be taken out from each phase detection unit of the PDAF sensor, and the phase information such as the amount of light entering the objects in the scene can be detected respectively. By comparing the phase information of the paired pixels, the accurate focus point can be quickly determined. Then the lens group can be pushed to the corresponding position at one time through the focus motor to quickly complete the autofocus.
[0037] At present, there are two types of PDAF sensors. One is a density-type PDAF image sensor, that is, only some of the pixels of the image sensor support PDAF. For example, a conventional image sensor has 1200W pixels. If it supports 6% PDAF, it means that only 72W pixels can output PDAF information. Since there is only a part of the phase difference information, the final calculated focus point may be inaccurate, resulting in poor focusing ability of the image sensor. The other is an image sensor that supports full PDAF. The current mainstream full PDAF sensor is a Quad Phase Detection (QPD) type image sensor. In a QPD type image sensor, each pixel can output PDAF information, that is, an image sensor with 1200W pixels can output 1200W PDAF phase difference information. However, the QPD type image sensor has a loss in image clarity.
[0038] Staggered High Dynamic Range (HDR): also called row-interleaved HDR, its principle is: when each row of pixel units of CMOS pixels is exposed and read, the next exposure starts immediately; that is, the reading of the image signal of the next original image of a row of pixel units does not have to wait for the complete reading of the image signal of the previous original image, but can be read directly after each row of pixel units is exposed, and the reading of image data of different original images can be carried out simultaneously.
[0039] For example, after the nth row of pixels has finished exposure and reading (such as long exposure), it immediately performs a second exposure (such as short exposure); at this time, the n+1th row of pixels also performs the same operation. It looks like the interlacing of yarns in textiles, so it is called: row interlaced exposure technology.
[0040] It can be understood that since the interval between reading two frames of original images is greatly shortened, artifacts and smearing phenomena are greatly improved.
[0041] It can be understood that staggered HDR can be achieved through staggered exposure mode. The reading speeds of staggered exposure mode and normal exposure mode are as follows:
[0042] In the normal exposure mode, the next frame of image data is read out only after the previous frame of image data is read out, i.e., frame-to-frame readout; while the Staggered exposure mode realizes line-to-line readout; i.e., the readout of the next frame of image data in the current line does not have to wait for the previous frame of image data to be fully read out, but can be directly read out after each line is exposed, and the readout of different frames can be parallel. This line-based reading method further shortens the time interval between frames and further optimizes ghosts. At the same time, multiple frames (such as long frames, medium frames, and short frames) of image data are not divided into three frames for output, but are overlapped and intertwined and output as one frame of data. Then, this frame of image data is parsed to separate multiple frames of image data.
[0043] 2Stagger: The image sensor completes one Stagger exposure and generates two frames of images. Furthermore, the two frames of images are fused to obtain a fused image.
[0044] For example, an image signal processor (ISP) may be used to perform fusion processing on two frames of images to obtain a final fused image.
[0045] 3Stagger: The sensor completes a Stagger exposure and can generate three frames of images. Furthermore, the ISP can fuse these three frames into one frame.
[0046] Other terms
[0047] The terms "first", "second", etc. in the specification of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0048] The terms "at least one (item)", "at least one of" and the like in the specification of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".
[0049] The image sensor, exposure method, exposure device, electronic device and medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0050] With the development of shooting technology, users have higher and higher requirements on the focusing ability of image sensors.
[0051] In the related art, when shooting, phase detection autofocus (PDAF) technology can usually be used for autofocus. Specifically, phase focus can be performed by an image sensor that supports PDAF technology, wherein at least some pixels in the image sensor that supports PDAF technology can output phase information. Generally, the higher the percentage of pixels that output phase information from the image sensor, the higher the accuracy of autofocus.
[0052] However, the higher the proportion of pixels in the image sensor that support PDAF, the lower the clarity of the image captured by the image sensor may be due to factors such as reduced pixel coverage for perceived color and limitations on the arrangement of focus pixels.
[0053] In order to solve the above technical problems, an embodiment of the present application provides an image sensor, including: a corresponding pixel array and a microlens array, and a morphology switching structure connected to the microlens array; the pixel array includes a plurality of pixel groups, each pixel group includes N pixel units, N is an integer greater than 1; the microlens array includes a plurality of microlens groups, each microlens group includes a first microlens or at least two second microlenses; wherein each pixel group is correspondingly provided with a first microlens or at least two second microlenses; the morphology switching structure is used to drive the microlens array to move, so as to control each pixel group to switch from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens, wherein when the pixel group corresponds to the first microlens, the pixel group is used to obtain phase information, and when the pixel group corresponds to the at least two second microlenses, the pixel group is used to obtain image information. In this way, since the microlens array is driven to move forward and backward through the morphology switching structure, each pixel group can be switched from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens, so that each pixel group can output image information and phase information respectively before and after the movement of the microlens array. Therefore, the image sensor provided in the embodiment of the present application improves the clarity of the captured image on the basis of realizing full-pixel support for PDAF.
[0054] The embodiment of the present application provides an image sensor. Figure 1 FIG. 1 shows a schematic diagram of the structure of an image sensor provided in an embodiment of the present application. Figure 1 As shown, the image sensor 100 provided in the embodiment of the present application may include a correspondingly arranged pixel array 10 and a microlens array 20, and a morphology switching structure 30 connected to the microlens array 20; the pixel array 10 includes a plurality of pixel groups 11, each pixel group 11 includes N pixel units 12, N is an integer greater than 1; the microlens array 20 includes a plurality of microlens groups 21, each microlens group 21 includes a first microlens 22 or at least two second microlenses 23; wherein each pixel group 11 is correspondingly arranged with a first microlens 22 or at least two second microlenses 23. The morphology switching structure 30 is used to drive the microlens array 20 to move, so as to control each pixel group 11 to switch from the corresponding first microlens 22 to the corresponding at least two second microlenses 23, or from the corresponding at least two second microlenses 23 to the corresponding first microlens 22; wherein, when the pixel group 11 corresponds to the first microlens 22, the pixel group 11 is used to obtain phase information, and when the pixel group 11 corresponds to the at least two second microlenses 23, the pixel group 11 is used to obtain image information.
[0055] In some embodiments of the present application, the area of the light incident surface of the first microlens is larger than the area of the light incident surface of the second microlens. This ensures that the first microlens can correspond to all pixel units in a pixel group. For example, the area of the light incident surface of the first microlens is N times the area of the light incident surface of the second microlens.
[0056] In some embodiments of the present application, the number of rows of lens arrays in the microlens array is greater than the number of rows of pixel groups in the pixel array, so as to ensure that before and after the microlens array moves, each pixel group is provided with a corresponding first microlens or at least one second microlens. Of course, in some embodiments, the number of rows of lens arrays in the microlens array may also be equal to the number of rows of pixel groups in the pixel array. For example, if the pixel array includes L rows of pixel groups, then the lens array may include L+T rows of lens groups, where L and T are both positive integers.
[0057] In some embodiments of the present application, for each pixel group in the pixel array, the number of pixel units in a pixel group is 2, 4, 6, 9 or 16, and the color of the pixel units in each pixel group is the same. That is, N can be: 2, 4, 6, 9 or 16, etc.
[0058] Among them, when N=4, the above-mentioned pixel array can be called a 4-in-1 pixel array; when N=9, the above-mentioned pixel array can be called a 9-in-1 pixel array; when N=16, the above-mentioned pixel array can be called a 16-in-1 pixel array; among them, "4-in-1", "9-in-1" and "16-in-1" refer to pixel merging technology, that is, merging multiple small pixels into one large pixel to improve the resolution and clarity of the image.
[0059] "Four-in-one" is to arrange four pixel units of the same color together to form a large pixel unit to achieve higher sensitivity; "nine-in-one" is to merge nine pixel units into one large pixel unit to improve image clarity and detail; "sixteen-in-one" is to merge sixteen pixel units into one large pixel unit. This technology can further improve image clarity and detail, but it will also reduce the image resolution accordingly.
[0060] In some embodiments of the present application, the same color of the pixel units in each pixel group can be understood as: the same color channel of the pixel units in each pixel group.
[0061] In some embodiments of the present application, adjacent pixel groups in a pixel array have different colors or color channels.
[0062] In some embodiments of the present application, the color channels may be a red (Red, R) channel, a green (Green, G) channel, and a blue (Blue, B) channel, wherein the G channel includes a Gr channel and a Gb channel.
[0063] In some embodiments of the present application, a color filter array (CFA) array may be provided on the pixel array, wherein the CFA may include a plurality of filter groups, each filter group corresponding to a color channel.
[0064] It can be understood that in the embodiment of the present application, each pixel group corresponds to a filter group, and the color channel of each pixel group is determined by the filter group corresponding to each pixel group.
[0065] In some embodiments of the present application, when a pixel group is provided with a first microlens, the N pixel units in the pixel group all correspond to the first microlens. For example, taking N=4 as an example, Figure 2A Schematic diagram of the structure of the pixel group corresponding to the first microlens. Figure 2A As shown, each of the four pixel groups in the pixel array corresponds to a first microlens, that is, each of the four pixel units in each pixel group corresponds to a first microlens. At this time, each of the four pixel groups can output phase information, through which accurate autofocus can be achieved.
[0066] It can be understood that if each pixel group in an image sensor is correspondingly provided with a first microlens, the image sensor can be called a QPD sensor with optimal focusing capability.
[0067] In some embodiments of the present application, when a pixel group is provided with a corresponding first microlens, the phase information output by the pixel group may be any one of the following: the up-down phase difference of the pixel group, the left-right phase difference of the pixel group. For example, taking N=4, that is, the pixel group includes 4 pixel units, as an example, Figure 2A As shown in FIG. 1 , a first microlens is disposed corresponding to each pixel group. Figure 2A Taking the first pixel group in as an example, if the pixel group outputs a left-right phase difference, the R1+R3 pixel unit can be used as the left phase, and the R2+R4 pixel unit can be used as the right phase; if the pixel group outputs an up-down phase difference, the R1+R2 pixel unit can be used as the up phase, and the R3+R4 pixel unit can be used as the down phase.
[0068] It can be understood that both the left-right phase difference and the up-down phase difference of the pixel group can be used for PDAF. The difference is that the left-right phase difference is more advantageous for vertical stripe scenes, while the up-down phase difference is more advantageous for horizontal stripe scenes.
[0069] In some embodiments of the present application, when at least two second microlenses are correspondingly arranged in a pixel group, the corresponding relationship between the N pixel units in the pixel group and the at least two second microlenses may be any one of the following:
[0070] 1) Each of the at least two second microlenses corresponds to a pixel unit in the pixel group, and different second microlenses correspond to different pixel units in the pixel group. It should be noted that when each second microlens corresponds to a pixel unit in a pixel group, the clarity of the image information collected by the pixel group is the highest.
[0071] For example, taking N=4 as an example, Figure 2B is a schematic diagram of the structure corresponding to the pixel group and the lens unit. Figure 2B As shown, at least two second microlenses are arranged corresponding to the four pixel groups in the pixel array. At this time, the pixel units in each pixel group correspond to one second microlens, so that each pixel group can output image information, so that the image data output by the image sensor has the best resolution, so that the image sensor can capture high-definition images.
[0072] It can be understood that if each pixel group in an image sensor is correspondingly provided with N first micro lenses, the image sensor can be called a four-in-one sensor with optimal clarity.
[0073] 2) Each of the at least two second microlenses corresponds to at least two pixel units in the pixel group, and different second microlenses correspond to different pixel units in the pixel group. The number of pixel units corresponding to each second microlens is less than N.
[0074] In some embodiments of the present application, for any microlens group including at least two second microlenses, the arrangement of at least two second microlenses in a microlens group is adapted to the arrangement of pixel units in a pixel group.
[0075] For example, taking the number of second microlenses in a microlens group as N, the N second microlenses in the microlens group may be distributed in an M*M array, where N is equal to the square of M and M is a positive integer.
[0076] In some embodiments of the present application, the plurality of microlens groups include at least two first-type microlens groups and at least two second-type microlens groups. Each first-type microlens group includes a first microlens, and each second-type microlens group includes at least two second microlenses. In this way, before and after the microlens array moves, the pixel array includes a pixel group corresponding to the first microlens and a pixel group corresponding to at least two second microlenses.
[0077] In some embodiments of the present application, the microlens group including the first microlens and the pixel group including at least one second microlens in the microlens array may be arranged regularly according to a certain rule, or may be arranged irregularly.
[0078] For ease of description, in the following embodiments, unless otherwise specified, a microlens group including a first microlens and a pixel group including at least one second microlens in a microlens array are regularly arranged as an example for illustration.
[0079] The corresponding relationship between the microlens array and the pixel array before and after the microlens array moves will be described below with reference to the accompanying drawings.
[0080] For example, suppose Figure 3A and Figure 3B Schematic diagram of the correspondence between the microlens array and the pixel array before and after the microlens array moves. Figure 3A As shown, before the microlens array moves, the R pixel group 11-1 corresponds to a first microlens 22, and the Gr pixel group 11-2 corresponds to four second microlenses, that is, each pixel unit in the Gr pixel group 11-2 corresponds to a second microlens 23. If the morphology switching structure drives the microlens array to move two pixel groups along the column scanning direction of the pixel array, that is, move two pixel groups downward, then: Figure 3B As shown, four second microlenses 23 are correspondingly arranged in the R pixel group 11 - 1 , that is, each pixel unit in the R pixel group 11 - 1 corresponds to one second microlens 23 ; and the Gr pixel group 11 - 2 corresponds to one first microlens 22 .
[0081] In some embodiments of the present application, Figures 1 to 4B As shown, each pixel group in the same row of pixel groups corresponds to a first microlens; or, each pixel group in the same row of pixel groups corresponds to at least two second microlenses.
[0082] It should be noted that “each pixel group in the same row corresponds to a first microlens” can be understood as: each pixel group in the same row corresponds to a first microlens, and different pixel groups correspond to different first microlenses.
[0083] For example, the pixel groups in the i-th row include K pixel groups, and the K pixel groups may correspond one-to-one to the K first microlenses.
[0084] It should be noted that “each pixel group in the same row corresponds to at least two second microlenses” can be understood as: each pixel group in the same row corresponds to at least one second microlens, and different pixel groups correspond to different at least two second microlenses.
[0085] In this way, since each pixel group in the same row corresponds to the first microlens; or, each pixel group in the same row corresponds to at least two second microlenses. That is, the pixel groups in the same row correspond to the same type of microlenses, the morphology switching structure can be controlled to drive the microlens array to move along the column direction, so as to control each of the pixel groups to switch from corresponding to the first microlens to corresponding to at least two of the second microlenses, or from corresponding to at least two of the second microlenses to corresponding to the first microlens. In this way, each pixel unit can output different information, such as image information or phase information, before and after the microlens array moves.
[0086] The arrangement of the lens groups in the lens array is described below in conjunction with the corresponding relationship between the pixel groups and the lens groups.
[0087] In some embodiments of the present application, when the arrangement is method a, such as Figure 3A As shown, when the i-th row of pixel groups corresponds to the first microlens 22, the i+1-th row of pixel groups corresponds to at least two second microlenses 23, the i+2-th row of pixel groups corresponds to at least two second microlenses 23, and the i+3-th row of pixel groups corresponds to the first microlens 22; wherein i can be a positive integer.
[0088] Or, when the arrangement is method b, such as Figure 4A As shown, when the i-th row of pixel groups corresponds to the first microlens 22, the i+1-th row of pixel groups corresponds to at least two second microlenses, the i+2-th row of pixel groups corresponds to the first microlens, and the i+3-th row of pixel groups corresponds to at least two second microlenses; wherein i can be a positive integer.
[0089] Or, when the arrangement is method c, such as Figure 4B As shown, when the i-th row of pixel groups corresponds to the first microlens 22, the i+1-th row of pixel groups corresponds to the first microlens 22, the i+2-th row of pixel groups corresponds to at least two second microlenses 23, and the i+3-th row of pixel groups corresponds to at least two second microlenses 23; wherein i can be a positive integer.
[0090] Or, when the arrangement is mode d, such as Figure 3B As shown, when the i-th row of pixel groups corresponds to at least two second microlenses 23, the i+1-th row of pixel groups corresponds to the first microlenses 22, the i+2-th row of pixel groups corresponds to the first microlenses 22, and the i+3-th row of pixel groups corresponds to at least two second microlenses 23; wherein i can be a positive integer.
[0091] Alternatively, when the arrangement is method e, when the i-th row of pixel groups corresponds to at least two second microlenses, the i+1-th row of pixel groups corresponds to the first microlens, the i+2-th row of pixel groups corresponds to two second microlenses, and the i+3-th row of pixel groups corresponds to the first microlens; wherein i can be a positive integer.
[0092] Alternatively, when the arrangement is manner f, when the i-th row of pixel groups corresponds to at least two second microlenses, the i+1-th row of pixel groups corresponds to at least two second microlenses, the i+2-th row of pixel groups corresponds to first microlenses, and the i+3-th row of pixel groups corresponds to first microlenses; wherein i can be a positive integer.
[0093] In some embodiments of the present application, when the arrangement between the pixel groups and the microlens groups is a, c, d or f as described above, the morphology switching structure can drive the microlens array to move two pixel groups along the column direction of the image sensor to control each pixel group to switch from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens.
[0094] In some embodiments of the present application, when the arrangement between the pixel group and the microlens group is as described above b or e, the morphology switching structure can drive the microlens array to move one pixel group along the column direction of the image sensor to control each pixel group to switch from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens.
[0095] In this way, for four adjacent rows of pixel groups, since these four rows of pixel groups can correspond to the first microlens, at least two second microlenses, at least two second microlenses, and the first microlens in sequence, or can correspond to the first microlens, at least two second microlenses, the first microlens, and at least two second microlenses in sequence, or can correspond to the first microlens, the first microlens, at least two second microlenses, and at least two second microlenses in sequence, the flexibility of correspondingly setting the lens groups and the pixel groups can be improved.
[0096] In some embodiments of the present application, the morphology switching structure can drive the microlens array to move as a whole, wherein each time the morphology switching structure drives the microlens array to move once, the lens group corresponding to each pixel group in the pixel array switches once.
[0097] For example, if the i-th pixel group of the pixel array corresponds to the first microlens before the microlens array moves, and the i+1-th pixel group corresponds to at least two i-th microlenses before the microlens array moves, then: after the microlens array completes one movement, the i-th pixel group corresponds to at least two i-th microlenses, and the i+1-th pixel group corresponds to one first microlens, where i is a positive integer.
[0098] In some embodiments of the present application, the morphology switching structure may include any of the following: a lead screw assembly, a slide rail assembly, or any other structure that can drive the microlens array to move.
[0099] For example, taking the form switching structure including a lead screw assembly as an example, the lead screw assembly may include a lead screw, a slider sleeved on the lead screw, and a motor connected to the lead screw, wherein the microlens array is connected to the slider. Thus, when the motor drives the lead screw to rotate, the slider can drive the microlens array to move as a whole along the lead screw.
[0100] In some embodiments of the present application, the morphology switching structure may also drive one or more microlens groups in the microlens array to move individually each time, and the specific movement may be determined according to actual usage requirements.
[0101] In some embodiments of the present application, the morphology switching structure may be configured to be movable along at least one of the row direction and the column direction of the pixel array. The specific configuration structure may refer to the related art, and the present application does not limit it.
[0102] In the image sensor provided by the embodiment of the present application, since the microlens array is driven to move forward and backward by the morphology switching structure, each pixel group can be switched from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens, so that each pixel group can output image information and phase information respectively before and after the movement of the microlens array. Therefore, the image sensor provided by the embodiment of the present application improves the clarity of the captured image on the basis of realizing full-pixel support for PDAF.
[0103] In some embodiments of the present application, Figure 1 ,like Figure 5 As shown, the image sensor 100 may further include: an exposure reading control module 40 and an exposure triggering control module 50; the exposure reading control module 40 is connected to the pixel array 10, and the exposure triggering control module 50 is respectively connected to the exposure reading control module 40, the pixel array 10 and the morphology switching structure 30; the exposure reading control module 40 may be used to control the pixel array 10 to perform row-by-row exposure to obtain a first image frame; the exposure triggering control module 50 may be used to:
[0104] At a first moment, the pixel array 10 is controlled to stop exposure, and the form switching structure 30 is controlled to drive the microlens array 20 to move, and after the microlens array 20 has completed moving, the pixel array 10 is controlled to continue to expose row by row; wherein the first moment is the moment when the pixel array 10 corresponding to the first image frame has not completed exposure, and the pixel array 10 corresponding to the second image frame starts to expose;
[0105] At the second moment, the pixel array 10 is controlled to stop exposure, and the form switching structure 30 is controlled to drive the microlens array 20 to move, and after the microlens array 20 has completed moving, the pixel array 10 is controlled to continue to expose row by row; wherein the second moment is the moment when the pixel array 10 corresponding to the first image frame has completed exposure, and the pixel array 10 corresponding to the second image frame has not completed exposure.
[0106] In some embodiments of the present application, Fig. 9 As shown in FIG. 1 , it is a schematic diagram of the exposure time of the first image frame and the second image frame; wherein the first moment can be Fig. 9 The A moment in the second moment can be Fig. 9 The B moment in the film.
[0107] It should be noted that each time the exposure trigger control module controls the morphology switching structure to drive the microlens array to move once, the microlens corresponding to each pixel group in the pixel array changes once. Specifically, the microlens corresponding to the first microlens is switched to the corresponding at least two second microlenses, or the microlens corresponding to the at least two second microlenses is switched to the corresponding first microlens.
[0108] For the description of the exposure trigger control module controlling the morphology switching structure to drive the microlens array to move, refer to the relevant description in the above embodiments, which will not be described again here to avoid repetition.
[0109] In some embodiments of the present application, the exposure trigger control module and the form switching structure may be electrically connected.
[0110] In some embodiments of the present application, "connected to the pixel array" can be understood as: connected to each pixel unit in the pixel array, so as to realize exposure control of each pixel unit. Among them, the exposure reading control module is connected to each pixel unit in the pixel array, and the exposure triggering control module is connected to each pixel unit in the pixel array.
[0111] In some embodiments of the present application, "the exposure reading control module can be used to control the pixel array to expose row by row to obtain a first image frame" can be understood as: the exposure reading control module controls the first image frame to be obtained after each row of the pixel array is exposed and read.
[0112] It should be noted that "controlling the pixel array to continue exposure" can be understood as that when the pixel array stops exposure, the next pixel unit of the pixel unit that has recently completed exposure starts to continue exposure. For example, assuming that after the exposure of the i-th pixel unit of the pixel array is completed, the pixel array stops exposure, then when the exposure trigger control module controls the pixel array to continue exposure, the pixel array can start exposure from the i+1-th pixel unit, where i is a positive integer.
[0113] In some embodiments of the present application, the pixel unit completing exposure may include at least one of the following: the pixel unit ends photosensitizing, and photosensitivity data of the pixel unit has been read out.
[0114] In some embodiments of the present application, the exposure trigger control module may determine whether the microlens array has completed the movement in the following manner.
[0115] Method 1: Determine whether the microlens array has completed the movement by using the displacement information detected by the displacement detection component in the morphology switching structure, such as a displacement sensor.
[0116] For example, the displacement detection component can be connected to the microlens array and the exposure trigger control module, so that the displacement information of the microlens array can be detected, and the detected displacement information can be fed back to the exposure trigger control module, so that the exposure trigger control module can determine whether the microlens array has completed the movement based on the displacement information.
[0117] Method 2: Determine whether the microlens array has completed the movement by controlling the time difference between the moment when the morphology switching structure drives the microlens array to move and the current moment. For example, if the exposure trigger control module controls the morphology switching structure to drive the microlens array at time t0, then the exposure trigger control module can determine that the microlens array has completed the movement after a preset time after time t0. The preset time is determined by the movement speed of the microlens array driven by the morphology switching structure.
[0118] Method 3: Determined by the stroke of the form switching structure. For example, if the form switching structure is a motor + lead screw structure, the exposure trigger control module can obtain the number of revolutions of the motor to determine whether the movement is completed.
[0119] In this way, in the process of controlling the pixel array to perform row-by-row exposure by the exposure reading control module, the exposure trigger control module can control the pixel array to stop exposure at the first time when the first image frame has not completed exposure and the second image frame has started to be exposed, and control the morphology switching structure to drive the microlens array to move, and control the pixel array to continue to expose after the microlens array has moved; and can control the pixel array to stop exposure again at the second moment when the first image frame has completed exposure and the second image frame has not completed exposure, and control the morphology switching structure to drive the microlens array to move, and control the pixel array to continue to expose after the microlens array has moved; so that each pixel group in the pixel array can output one type of information, such as image information, in the exposure period before the first moment and after the second moment, and output another type of information, such as phase information, in the exposure period after the first moment and before the second moment. Therefore, it is ensured that the data corresponding to the first image frame and the data corresponding to the second image frame include the image information or phase information output by each pixel unit in the pixel array. In this way, a full-phase image for autofocus can be generated based on the phase information output by each pixel group, or an image with higher clarity can be generated based on the image information output by each pixel group, so that the image sensor provided in the embodiment of the present application can have both the ability to capture clear images and the ability to accurately focus.
[0120] The following is a schematic diagram of the circuit between the exposure reading control module, the exposure triggering control module, the morphology switching structure and the pixel array.
[0121] In some embodiments of the present application, Figure 5 ,like Figure 6As shown, the pixel unit 12 may include: a photodiode PD1, a reset transistor RST1, a floating switch TG1, a selection transistor RSeL1, a source follower SF1 and a capacitor FD1, wherein a first end of the photodiode PD1 is grounded, a second end of the photodiode PD1 is connected to a first end of the floating switch TG1, a second end of the floating switch TG1 is connected to a first control line TG, a third end of the floating switch TG1 is connected to a first end of the capacitor FD1, a second end of the capacitor FD1 is grounded, a first end of the reset transistor RST1 is connected to a second control line RST, a second end of the reset transistor RST1 is connected to a power line VDD, and a third end of the reset transistor RST1 is connected to the capacitor F The first end of the signal amplifier SF1 is connected to the power line VDD, the second end of the signal amplifier SF1 is connected to the first end of the capacitor FD1, the third end of the signal amplifier SF1 is connected to the first end of the selection transistor RSeL1, the second end of the selection transistor RSeL1 is connected to the conversion circuit 60, and the third end of the selection transistor RSeL1 is connected to the third control line ROW-Sel, so that when the pixel unit 12 receives light, the pixel unit 12 can output analog image data to the analog-to-digital conversion circuit through the photodiode PD1, the reset transistor RST1, the floating switch TG1, the selection transistor RSeL, the signal amplifier SF1 and the capacitor FD1.
[0122] Among them, PD1 is controlled by TG1, that is, TG1 is responsible for switching PD1 to control the light-sensing moment of PD1. RST1 is responsible for clearing the residual photogenerated electrons in PD1 and FD1. FD1 is equivalent to a capacitor and is responsible for carrying the charge transmitted from PD1. SF1 is responsible for transferring the charge in FD1 to the selection transistor RseL1. RseL1 is responsible for controlling the output of the pixel unit. When RseL1 is turned on, the charge in FD1 will be transferred to the output circuit through SF1.
[0123] In some embodiments provided in this application, Figure 6 As shown, the conversion circuit 60 may include: an analog to digital converter (ADC), an ISP and a mobile industry processor interface MIPI connected in sequence. The ADC is connected to the second end of the selection transistor RSeL1, and is used to convert the analog image signal obtained by the PD1 to a digital image signal, and output the digital image signal to the ISP. The ISP is used to convert the digital image signal output by the ADC into image information; and the MIPI is used to output the image information.
[0124] In some embodiments of the present application, Figure 6The pixel unit 12 shown can be called a pinned photodiode pixel (PDD) pixel unit, referred to as a PDD pixel unit. The PPD pixel includes a photosensitive area of PD1, i.e., a photosensitive diode PD, and four transistors, which are: a reset transistor RST, a floating switch TX1, a row selector SET, and a signal amplifier SF, so it is also called a 4T pixel unit. The PPD pixel unit allows the introduction of a correlated double sampling circuit, eliminating the kTC noise introduced by the reset, the 1 / f noise introduced by the MOS tube, and the offset noise. The working mode of the PDD pixel unit is as follows:
[0125] 1. Reset. First activate RST and TG to clear the residual electrons in PD and FD.
[0126] 2. PD photosensitivity. When RST and TG are disconnected, the pixel begins to be sensitive to light. The electron-hole pairs generated by light irradiation will separate due to the existence of the PD electric field, with electrons moving to the n region and holes moving to the p region.
[0127] 3. Charge transfer. When the PD is exposed to light at a specified time, the TG is activated to completely transfer the charge from the PD to the FD for readout. The mechanism here is similar to the charge transfer in the CCD.
[0128] 4. Signal level readout. Next, the voltage signal of FD is output (Vout) to ADC for digital-to-analog conversion through SF (source follower) and SET (row selector), and then the digital signal can be output.
[0129] In some embodiments of the present application, at least one pixel unit corresponds to a conversion circuit.
[0130] In some embodiments of the present application, Figure 6 As shown, the exposure trigger control module 50 can be connected to the pixel unit by connecting to the first control line TG, the second control line RST, the third control line ROW-Sel, the reset line RST and the power line VDD, thereby realizing connection with the pixel array.
[0131] In some embodiments of the present application, Figure 6 As shown, the exposure reading control module 40 can be connected to the first control line TG, the second control line RST, the third control line ROW-Sel, the reset line RST and the power line VDD to achieve connection with the pixel unit, thereby achieving connection with the pixel array.
[0132] In some embodiments of the present application, the exposure reading control module may be specifically used to control the pixel array to perform row-by-row exposure in a staggered exposure mode.
[0133] It can be understood that the reading speeds of the interlaced exposure mode and the normal exposure mode are as follows:
[0134] In the normal exposure mode, the next frame of image data is read out only after the previous frame of image data is read out, that is, frame-to-frame reading; while the Staggered exposure mode realizes line-to-line reading; that is, the reading of the next frame of image data in the current line does not have to wait for the previous frame of image data to be fully read out, but can be read out directly after each line is exposed, and the reading of different frames can be parallel. This line-based reading method further shortens the time interval between frames and further optimizes ghosts. At the same time, multiple frames (such as long frames, medium frames and short frames) of image data are not divided into three frames for output, but overlapped and intertwined together and output as one frame of image data. Then the frame of image data is parsed to separate multiple frames of image data.
[0135] It should be noted that the above embodiment is illustrated by using an interlaced exposure mode. In actual implementation, the exposure reading control module can also control the pixel array to be exposed according to a traditional exposure mode. The exposure reading control module is the overall exposure control structure of the image sensor.
[0136] In this way, since the exposure reading control module can control the pixel array to be exposed in an interlaced exposure mode, the exposure and reading time intervals between the multiple frames of image data exposed by the pixel array can be shortened, thereby optimizing the ghost images between the images corresponding to the multiple frames of image data, thereby improving the image quality.
[0137] In some embodiments of the present application, the exposure reading control module can also be used to send first information to the exposure trigger control module, and the first information can be used to indicate the first moment and the second moment; the exposure trigger control module can also be used to determine the first moment and the second moment based on the first information.
[0138] In some embodiments of the present application, the exposure reading control module may send first information to the exposure trigger control module after controlling the pixel array to start exposure, so that the exposure trigger control module can determine the first moment and the second moment.
[0139] In this way, since the exposure reading control module can send the first information indicating the first moment and the second moment to the exposure trigger control module, the exposure trigger control module can accurately control the pixel array to terminate exposure and switch the microlens group corresponding to the pixel group.
[0140] In some embodiments of the present application, the first information includes any one of the following: the moment information of the first moment and the moment information of the second moment; the exposure parameter of the first image frame and the exposure parameter of the second image frame. In this way, the flexibility of the exposure trigger control module in determining the first moment can be improved.
[0141] The present application embodiment provides a camera module, such as Figure 7 As shown, the camera module includes the image sensor 100 in the above embodiment.
[0142] In some embodiments of the present application, Figure 7 As shown, the camera module 200 may also include: a lens group 210, a focus motor 220, a filter 230, a conversion circuit and a memory. The lens group 210 is embedded in the focus motor 220, the lens group 210 is arranged opposite to the image sensor 100, the filter 230 is located between the lens group 210 and the image sensor 100, and the conversion circuit is connected to the output end of the image sensor (such as the second end of the selection transistor RSeL) to convert the analog image signal obtained by the image sensor into a digital image signal and output it. The memory is connected to the output end of the conversion circuit to store the digital image signal converted by the conversion circuit.
[0143] In some embodiments of the present application, Figure 7 As shown, the camera module 200 may further include a base 240 and a shell connected to the base, and the image sensor 100 is disposed on the base 240 .
[0144] In some embodiments of the present application, the focus motor may be connected to a first spring sheet and a second spring sheet disposed on the housing, wherein the housing is connected to the base. Figure 7 The specific structure is not shown in the figure, and the specific structure is shown in the related art. When focusing, the focus motor is energized to generate magnetic force to compress the first spring sheet or the second spring sheet to push the lens group to the focus position. It can be understood that the final position of the focus motor can be controlled by the magnitude of the current or voltage applied to the focus motor.
[0145] In some embodiments of the present application, the filter may be a CFA filter.
[0146] It can be understood that the light that converges into the lens group is projected onto the filter, and the function of the filter is to filter out unnecessary light that passes through the lens group, preventing the image sensor from generating false colors / ripples, so as to improve the effective resolution and color reproduction of the image sensor. The light that passes through the filter can be sensed by the image sensor.
[0147] In some embodiments of the present application, the lens group may be composed of multiple glass lenses or plastic lenses. When using the camera module to take pictures, the light will pass through a series of refractions of the lens and finally be imaged on the image sensor. The more times of refraction, the more prominent the correction and focusing effect of the light will be, and the better the imaging effect will be when taking pictures.
[0148] It can be understood that the shooting principle of the camera module can be as follows: when the camera module is aimed at a scene to take a picture, the light first passes through the lens and shines on the image sensor. The sensor converts the light signal into an analog image signal. The analog image signal is converted into a digital signal through the analog-to-digital converter in the conversion circuit. Finally, it is processed by the ISP in the conversion circuit into visual image data and output to the memory.
[0149] In the camera module provided by the embodiment of the present application, since the microlens array is driven to move forward and backward through the morphology switching structure, each pixel group can be switched from the corresponding first microlens to the corresponding at least two second microlenses, or from the corresponding at least two second microlenses to the corresponding first microlens, so that each pixel group can output image information and phase information respectively before and after the movement of the microlens array. Therefore, the camera module provided by the embodiment of the present application improves the clarity of the captured image on the basis of realizing full-pixel support for PDAF.
[0150] The exposure method provided in the embodiment of the present application can be executed by an exposure control device, an electronic device, or a functional module or entity in an electronic device. The exposure method provided in the embodiment of the present application is described by taking an electronic device executing the exposure method as an example.
[0151] The present application embodiment provides an exposure method, which is applied to the image sensor in the above embodiment, such as Figure 8 As shown, the exposure method provided in the embodiment of the present application may include steps 801 to 806.
[0152] Step 801: The electronic device controls the pixel array of the image sensor to perform row-by-row exposure to obtain a first image frame.
[0153] It should be noted that “the electronic device controls the pixel array of the image sensor to perform row-by-row exposure to obtain the first image frame” can be understood as: the electronic device controls the pixel array to perform row-by-row exposure starting from the first row of pixel units. After the last row of pixel units in the pixel array completes the row-by-row exposure, the first image frame can be obtained.
[0154] In some embodiments of the present application, the electronic device may control the pixel array to perform row-by-row exposure according to a first exposure parameter. The first exposure parameter may include an exposure parameter of a first image frame and an exposure parameter of a second image frame.
[0155] In some embodiments of the present application, the exposure duration corresponding to the exposure parameters of the first image frame is greater than the exposure duration corresponding to the exposure parameters of the second image frame. That is, the first image frame is a long image frame, and the second image frame is a short image frame. In other embodiments, the exposure duration of the first image frame may be the same as the exposure duration of the second image frame.
[0156] In some embodiments of the present application, the electronic device may determine a first exposure parameter according to a shooting scene, that is, the first exposure parameter is an automatic exposure (Auto Exposure, AE) parameter.
[0157] In some embodiments of the present application, the electronic device may also determine preset auto focus (Auto Focus) parameters and auto white balance (Auto White Balance) parameters according to the above-mentioned shooting scene, so that the preview image of the image sensor conforms to the perception of the human eye.
[0158] In some embodiments of the present application, the electronic device can control the pixel array to perform row-by-row exposure in an interlaced exposure mode. In this way, since the electronic device can control the pixel array to perform row-by-row exposure in an interlaced exposure mode, the exposure and reading time interval between the corresponding data of at least two image frames exposed by the pixel array can be shortened, thereby optimizing the ghost image between the at least two image frames, thereby improving the image capture quality.
[0159] Step 802: The electronic device controls the pixel array to stop exposure at a first moment, and switches the microlens group corresponding to each pixel group through the morphology switching structure of the image sensor. The first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure and the pixel array corresponding to the second image frame starts to be exposed.
[0160] It should be noted that the electronic device can first control the pixel array to stop exposure, and then switch the microlens group corresponding to each pixel group through the morphology switching structure of the image sensor.
[0161] In some embodiments of the present application, the electronic device can switch the microlens group corresponding to each pixel group by controlling the morphology switching structure to drive the microlens group to move.
[0162] It should be noted that “switching the microlens group corresponding to each pixel group” may include: the pixel group is switched from the corresponding first microlens to the corresponding at least two second microlenses, so that the pixel group can output phase information before switching the corresponding microlens group, and can output image information after switching the corresponding microlens group; or, the pixel group is switched from the corresponding at least two second microlenses to the corresponding first microlens, so that each pixel group can output image information before switching the corresponding microlens group, and can output phase information after switching the corresponding microlens group.
[0163] Step 803: The electronic device controls the pixel array of the image sensor to continue to perform row-by-row exposure.
[0164] It can be understood that the electronic device can control the pixel array to continue exposure row by row from the position where the exposure was stopped.
[0165] Step 804: the electronic device controls the pixel array to stop exposure at the second moment, and switches the microlens group corresponding to each pixel group through the morphology switching structure.
[0166] The second moment is a moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure.
[0167] Step 805: The electronic device controls the pixel array of the image sensor to continue to perform row-by-row exposure until the exposure of the pixel array corresponding to the second image frame is completed.
[0168] For other descriptions of step 804 and step 805, refer to the relevant descriptions of steps 802 and 803 in the above embodiment.
[0169] Step 806: The electronic device obtains target image information or target phase information based on the first data corresponding to the first image frame and the second data corresponding to the second image frame.
[0170] In some embodiments of the present application, the first data includes image information output by a part of pixel groups of the pixel array and phase information output by another part of pixel groups of the pixel array; the second data includes phase information output by a part of pixel groups of the pixel array and image information output by another part of pixel groups of the pixel array.
[0171] In some embodiments of the present application, the target phase information can be used for autofocus, and the target image information can generate a higher-definition image, such as a high-definition image.
[0172] In some embodiments of the present application, the first data and the second data include image information or phase information corresponding to each pixel unit in the pixel array; the above step 806 may include the following steps 806A and 806B.
[0173] Step 806A: The electronic device synthesizes the image information in the first data and the image information in the second data to obtain target image information.
[0174] Step 806B: The electronic device synthesizes the phase information in the first data with the phase information in the second data to obtain target phase information.
[0175] In some embodiments of the present application, the exposure duration of the first image frame and the exposure duration of the second image frame may be the same or different.
[0176] Among them, when the exposure time of the first image frame and the exposure time of the second image frame are the same, the phase information in the first data and the phase information in the second data can be directly fused to obtain the target phase information, and the image information in the first data and the image information in the second data can be directly fused to obtain the target image information.
[0177] If the exposure duration of the first image frame is different from that of the second image frame, the electronic device may first adjust the exposure gain corresponding to the first data and the second data so that the total exposure amounts corresponding to the two are the same, and then synthesize them according to demand.
[0178] In some embodiments of the present application, in a focusing scene, the electronic device may execute step 806B, and in a shooting scene (ie, when an image needs to be captured), the electronic device may execute step 806A.
[0179] In this way, since the electronic device can synthesize the image information in the first data and the image information in the second data to obtain the target image information; or can synthesize the image information in the first data and the image information in the second image to obtain the target image information; therefore, the electronic device can synthesize the target phase information in the focusing stage to achieve fast and accurate autofocus based on the target phase information, and obtain a high-definition image by synthesizing the target image information in the shooting stage. Therefore, the image sensor provided in the embodiment of the present application improves the clarity of the captured image on the basis of achieving full pixel support for PDAF.
[0180] In the exposure method provided in the embodiment of the present application, in the process of controlling the pixel array to perform row-by-row exposure, the pixel array can be controlled to stop exposure at the first time when the first image frame is not completed and the second image frame starts to be exposed, and the morphology switching structure is controlled to switch the microlens group corresponding to each pixel group in the pixel array; then the pixel array is controlled to continue to be exposed, and the pixel array can be controlled to stop exposure again at the second moment when the first image frame is completed and the second image frame is not completed, and the morphology switching structure is controlled to switch the microlens group corresponding to each pixel group again to control the pixel array to continue to be exposed; so that each pixel group in the pixel array can output one type of information, such as image information, in the exposure period before the first moment and after the second moment, and output another type of information, such as phase information, in the exposure period after the first moment and before the second moment. Therefore, it is ensured that the first data and the second data include the image information or phase information output by each pixel unit in the pixel array. In this way, the target phase information for autofocus can be generated according to the phase information output by each pixel group, or the image information with higher clarity can be obtained according to the image information output by each pixel group, so that the image sensor can have both the ability to collect clear images and the ability to accurately focus.
[0181] In some embodiments provided in the present application, the exposure method provided in the embodiments of the present application may further include:
[0182] Step 807: The electronic device determines the first moment according to the exposure parameters of the first image frame, and determines the second moment according to the exposure parameters of the second image frame.
[0183] In this way, since the exposure parameters of the image include the exposure time of the image, the first moment is determined according to the exposure parameters of the first image frame, and the second moment is determined according to the exposure parameters of the second image frame, which can improve the accuracy of the first moment and the second moment.
[0184] The exposure method provided in the embodiment of the present application is described below with reference to examples.
[0185] For example, assuming that the pixel group in the image sensor includes 4 pixel units, the exposure mode is 2stagger, that is, the stagger outputs two frames of long and short images, and the electronic device is a mobile phone, then the detailed process of realizing the image sensor outputting an image compatible with PDAF and image clarity is as follows:
[0186] Step 1: The mobile phone presets AE parameters, AF parameters and AWB parameters according to the user's photo-taking scene, so that the camera's preview image conforms to human eye perception.
[0187] Step 2: When the user presses the camera button, the mobile phone outputs the preset AE parameters to the exposure reading control module, so that the exposure reading control module controls the pixel array to start exposure according to the stagger exposure mode.
[0188] Step 3, after reaching Fig. 9 At the moment shown by point A in the figure, that is, before the start of exposure of the short image in the stagger exposure mode, the mobile phone first controls the pixel array to sense light line by line and reads out the image data of the long image. At this time, the corresponding relationship between the microlens array and the pixel array is as follows: Figure 3A As shown, it can be seen that at this time, the first R pixel group in the upper left corner of the pixel array corresponds to a first microlens, that is, the photosensitivity form of the R pixel group is a phase detection form, so that the R pixel group can output image data of a QPD structure.
[0189] Step 4, after reaching Fig. 9 When the pixel array reaches point A in the image, the pixel array needs to operate the first row of pixel units to start photosensing and reading out again to output the data of the short image. At this time, the mobile phone can control the exposure or reading action of the pixel array to be paused through the exposure mode trigger control module, and control the morphology switching module through the exposure mode trigger control module to push the microlens array up or down to adjust the microlens or first lens unit corresponding to each pixel group. Among them, after the microlens array completes the movement, the corresponding relationship between the microlens array and the pixel array is as follows: Figure 3B As shown, after the microlens completes the movement, the mobile phone can trigger the control module through the exposure mode to control the pixel array to continue the exposure and data readout actions.
[0190] Step 5, after reaching Fig. 9 At the moment shown by point B in the figure, the long image in the stagger exposure mode has been completely exposed and read out. At this time, the mobile phone can control the exposure or readout action of the pixel array to be paused through the exposure mode trigger control module, and control the morphology switching module through the exposure mode trigger control module to push the microlens array up or down to adjust the microlens or first lens unit corresponding to each pixel group. Among them, after the microlens array completes the movement, the corresponding relationship between the microlens array and the pixel array is as follows Figure 3A shown.
[0191] Step 6: After the microlens has finished moving, the mobile phone can trigger the control module through the exposure mode to control the pixel array to continue the exposure and data readout action until Fig. 9 At the moment shown by point C in the figure, both frames of images in the stagger exposure mode are output.
[0192] It can be understood that, during the exposure of the pixel array in the stagger exposure mode, the photosensitive form of each pixel group in the pixel array is switched twice. Fig. 9 As shown, the long image is divided into two parts, X1 and Y1, and the short image is divided into two parts, X2 and Y2.
[0193] Step 7. During the focusing stage, the mobile phone can synthesize the phase information in the short image and the long image, also known as QPD data, to obtain a full-phase image, that is, an image with the best focusing capability.
[0194] During the shooting stage, the mobile phone can synthesize the image information in the short image and the long image, also known as tetra data, to obtain the image with the best clarity.
[0195] In this way, in the process of controlling the pixel array to be exposed in the stagger exposure mode, the microlens group corresponding to each pixel group in the pixel array is switched multiple times by controlling the movement of the microlens array, so that each pixel group outputs both image information and phase information in the process of the pixel array being exposed in the stagger exposure mode. Therefore, the data output by each pixel group can not only perform accurate PDAF, but also generate high-definition images. In this way, the image collected by the image sensor can be compatible with both clarity and focusing capabilities.
[0196] It should be noted that the above embodiment is described by taking the electronic device controlling the pixel array to expose two image frames as an example. In actual implementation, the electronic device can control the pixel array to expose three or more image frames in an interlaced exposure manner.
[0197] Taking the exposure of three image frames as an example, the electronic device can obtain a high-definition image or perform accurate autofocus according to the data corresponding to the three image frames by using the processing method of step 806 .
[0198] For example, assuming that the electronic device obtains data of three image frames, namely data 1, data 2 and data 3, the electronic device can first obtain first image information by combining the image information in data 1 and the image information in data 2, and then obtain second image information by combining the image information in data 2 and the image information in data 3, and then fuse the first image information and the second image information to obtain target image information. The method for obtaining target phase information is similar to the method for obtaining target image information.
[0199] The exposure method provided in the embodiment of the present application can be performed by an exposure device. In the embodiment of the present application, the exposure device provided in the embodiment of the present application is described by taking the exposure method performed by the exposure device as an example.
[0200] The present application embodiment provides an exposure device, which can be an image sensor in the above embodiment, such as Fig.10 As shown, the exposure device 1000 further includes a control module 1001. The control module 1001 can be used for:
[0201] Controlling the pixel array of the image sensor to perform row-by-row exposure to obtain a first image frame;
[0202] At a first moment, the pixel array is controlled to stop exposure, and the microlens group corresponding to each pixel group is switched through the morphology switching structure of the image sensor; wherein the first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure, and the pixel array corresponding to the second image frame starts to be exposed; the pixel array of the image sensor is controlled to continue to be exposed row by row; at a second moment, the pixel array is controlled to stop exposure, and the microlens group corresponding to each pixel group is switched through the morphology switching structure; wherein the second moment is the moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure; the pixel array of the image sensor is controlled to continue to be exposed row by row until the exposure of the pixel array corresponding to the second image frame is completed; based on the first data corresponding to the first image frame and the second data corresponding to the second image frame, the target image information or the target phase information is obtained.
[0203] In some embodiments of the present application, the first data and the second data include image information or phase information corresponding to each pixel group in the pixel array;
[0204] The control module 1001 is further configured to synthesize the image information in the first data and the image information in the second data to obtain target image information; or,
[0205] The phase information in the first data is synthesized with the phase information in the second data to obtain target phase information.
[0206] In the exposure device provided by the embodiment of the present application, in the process of controlling the pixel array to perform row-by-row exposure, the pixel array can be controlled to stop exposure at the first time when the first image frame is not completed and the second image frame starts to be exposed, and the morphology switching structure can be controlled to switch the microlens group corresponding to each pixel group in the pixel array; then the pixel array is controlled to continue to be exposed, and the pixel array can be controlled to stop exposure again at the second moment when the first image frame is completed and the second image frame is not completed, and the morphology switching structure can be controlled to switch the microlens group corresponding to each pixel group again to control the pixel array to continue to be exposed; so that each pixel group in the pixel array can output one type of information, such as image information, in the exposure period before the first moment and after the second moment, and output another type of information, such as phase information, in the exposure period after the first moment and before the second moment. Therefore, it is ensured that the first data and the second data include the image information or phase information output by each pixel unit in the pixel array. In this way, the target phase information for autofocus can be generated according to the phase information output by each pixel group, or the image information with higher clarity can be obtained according to the image information output by each pixel group, so that the image sensor can have both the ability to collect clear images and the ability to accurately focus.
[0207] The exposure device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or a device other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0208] The exposure device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0209] The exposure device provided in the embodiment of the present application can achieve Fig. 9 The various processes implemented by the method embodiment shown achieve the same technical effect, and to avoid repetition, they will not be described again here.
[0210] Alternatively, if Fig.11 As shown, an embodiment of the present application further provides an electronic device 1100, including a processor 1101 and a memory 1102, wherein the memory 1102 stores programs or instructions that can be executed on the processor 1101, and when the program or instructions are executed by the processor 1101, the various steps of the above-mentioned exposure method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described again here.
[0211] It should be noted that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.
[0212] Fig.12 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0213] The electronic device 1500 includes but is not limited to: a radio frequency unit 1501, a network module 1502, an audio output unit 1503, an input unit 1504, a sensor 1505, a display unit 1506, a user input unit 1507, an interface unit 1508, a memory 1509, and a processor 1510 and other components.
[0214] The sensor 1501 includes an image sensor, a pixel array and a microlens array corresponding to the image sensor, and a morphology switching structure connected to the microlens array;
[0215] The pixel array includes a plurality of pixel groups, each pixel group includes N pixel units, where N is an integer greater than 1;
[0216] The microlens array includes a plurality of microlens groups, each microlens group includes a first microlens or at least two second microlenses; wherein each pixel group is correspondingly provided with a first microlens or at least two second microlenses;
[0217] The morphology switching structure is used to drive the microlens array to move, so as to control each pixel group to switch from corresponding to the first microlens to corresponding to at least two second microlenses, or from corresponding to at least two second microlenses to corresponding to the first microlens, wherein when the pixel group corresponds to the first microlens, the pixel group is used to obtain phase information, and when the pixel group corresponds to at least two second microlenses, the pixel group is used to obtain image information.
[0218] In some embodiments of the present application, each pixel group in the same row of pixel groups corresponds to a first microlens;
[0219] Alternatively, each pixel group in the same row corresponds to at least two second micro lenses.
[0220] In some embodiments of the present application, when the pixel group in the i-th row corresponds to the first microlens, the pixel group in the i+1-th row corresponds to at least two second microlenses, the pixel group in the i+2-th row corresponds to at least two second microlenses, and the pixel group in the i+3-th row corresponds to the first microlens;
[0221] Alternatively, when the pixel group in the i-th row corresponds to the first microlens, the pixel group in the i+1-th row corresponds to at least two second microlenses, the pixel group in the i+2-th row corresponds to the first microlens, and the pixel group in the i+3-th row corresponds to at least two second microlenses;
[0222] Alternatively, when the pixel group in the i-th row corresponds to the first microlens, the pixel group in the i+1-th row corresponds to the first microlens, the pixel group in the i+2-th row corresponds to at least two second microlenses, and the pixel group in the i+3-th row corresponds to at least two second microlenses;
[0223] In the case where the pixel group in the i-th row corresponds to at least two of the second microlenses, the pixel group in the i+1-th row corresponds to the first microlens, the pixel group in the i+2-th row corresponds to the first microlens, and the pixel group in the i+3-th row corresponds to at least two of the second microlenses;
[0224] Wherein, i is a positive integer.
[0225] In some embodiments of the present application, the number of pixel units in a pixel group is 2, 4, 6, 9 or 16, and the colors of the pixel units in each pixel group are the same.
[0226] In some embodiments of the present application, the image sensor further includes: an exposure reading control module and an exposure triggering control module; the exposure reading control module is connected to the pixel array, and the exposure triggering control module is respectively connected to the exposure reading control module, the pixel array and the morphology switching structure;
[0227] An exposure reading control module, used for controlling the pixel array to perform row-by-row exposure to obtain a first image frame;
[0228] Exposure trigger control module, used for:
[0229] At a first moment, the pixel array is controlled to stop exposure, and the morphology switching structure is controlled to drive the microlens array to move, and after the microlens array has moved, the pixel array is controlled to continue to be exposed row by row; wherein the first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure and the pixel array corresponding to the second image frame starts to be exposed;
[0230] At the second moment, the pixel array is controlled to stop exposure, and the morphology switching structure is controlled to drive the microlens array to move, and after the microlens array has moved, the pixel array is controlled to continue to expose row by row; wherein the second moment is the moment when the pixel array corresponding to the first image frame has completed exposure and the pixel array corresponding to the second image frame has not completed exposure.
[0231] In some embodiments of the present application, the exposure reading control module is specifically used to control the pixel array to perform row-by-row exposure in an interlaced exposure mode.
[0232] In some embodiments of the present application, the exposure reading control module is further used to send first information to the exposure triggering control module, where the first information is used to indicate the first moment and the second moment;
[0233] The exposure trigger control module is further used to determine the first moment and the second moment according to the first information.
[0234] In some embodiments of the present application, the first information includes any of the following:
[0235] time information of a first moment and time information of a second moment;
[0236] The exposure parameters of the first image frame and the exposure parameters of the second image frame.
[0237] Those skilled in the art will appreciate that the electronic device 1500 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1510 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.12 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0238] The processor 1510 is configured to:
[0239] Controlling the pixel array of the image sensor to perform row-by-row exposure to obtain a first image frame;
[0240] At a first moment, the pixel array is controlled to stop exposure, and the microlens group corresponding to each pixel group is switched through the morphology switching structure of the image sensor; wherein the first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure and the pixel array corresponding to the second image frame starts to be exposed;
[0241] Controlling the pixel array of the image sensor to continue to perform line-by-line exposure;
[0242] At a second moment, the pixel array is controlled to stop exposure, and the microlens group corresponding to each pixel group is switched through the morphology switching structure; wherein the second moment is a moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure;
[0243] Controlling the pixel array of the image sensor to continue to perform row-by-row exposure until the exposure of the pixel array corresponding to the second image frame is completed;
[0244] Target image information or target phase information is obtained based on first data corresponding to the first image frame and second data corresponding to the second image frame.
[0245] In some embodiments of the present application, the first data and the second data include image information or phase information corresponding to each pixel group in the pixel array;
[0246] The processor 1510 is further configured to synthesize the image information in the first data and the image information in the second data to obtain target image information; or,
[0247] The phase information in the first data is synthesized with the phase information in the second data to obtain target phase information.
[0248] In the electronic device provided by the embodiment of the present application, in the process of controlling the pixel array to perform row-by-row exposure, the pixel array can be controlled to stop exposure at the first time when the first image frame is not completed and the second image frame starts to be exposed, and the morphology switching structure is controlled to switch the microlens group corresponding to each pixel group in the pixel array; then the pixel array is controlled to continue to be exposed, and the pixel array can be controlled to stop exposure again at the second moment when the first image frame is completed and the second image frame is not completed, and the morphology switching structure is controlled to switch the microlens group corresponding to each pixel group again to control the pixel array to continue to be exposed; so that each pixel group in the pixel array can output one type of information, such as image information, in the exposure period before the first moment and after the second moment, and output another type of information, such as phase information, in the exposure period after the first moment and before the second moment. Therefore, it is ensured that the first data and the second data include the image information or phase information output by each pixel unit in the pixel array. In this way, the target phase information for autofocus can be generated according to the phase information output by each pixel group, or the image information with higher clarity can be obtained according to the image information output by each pixel group, so that the exposure device can have both the ability to collect clear images and the ability to accurately focus.
[0249] It should be understood that in the embodiment of the present application, the input unit 1504 may include a graphics processing unit (GPU) 15041 and a microphone 15042. The graphics processing unit 15041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The sensor 1505 may include the components shown in FIG. Figure 8 The image sensor shown in any one of the items. The display unit 1506 may include a display panel 15061, and the display panel 15061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1507 includes a touch panel 15071 and at least one of other input devices 15072. The touch panel 15071 is also called a touch screen. The touch panel 15071 may include two parts: a touch detection device and a touch controller. Other input devices 15072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0250] The memory 1509 can be used to store software programs and various data. The memory 1509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1509 may include a volatile memory or a non-volatile memory, or the memory 1509 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0251] The processor 1510 may include one or more processing units; optionally, the processor 1510 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1510.
[0252] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned exposure method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0253] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0254] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned exposure method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0255] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0256] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned exposure method embodiment as described above, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0257] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0258] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0259] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image sensor, It is characterized in that include: A correspondingly arranged pixel array and microlens array, and a morphology switching structure connected to the microlens array; The pixel array includes a plurality of pixel groups, each pixel group includes N pixel units, and N is an integer greater than 1; The microlens array includes a plurality of microlens groups, each of which includes a first microlens or at least two second microlenses; wherein each pixel group is correspondingly provided with one first microlens or at least two second microlenses; The morphology switching structure is used to drive the microlens array to move, so as to control each of the pixel groups to switch from corresponding to the first microlens to corresponding to at least two of the second microlenses, or from corresponding to at least two of the second microlenses to corresponding to the first microlens, wherein when the pixel group corresponds to the first microlens, the pixel group is used to obtain phase information, and when the pixel group corresponds to at least two of the second microlenses, the pixel group is used to obtain image information.
2. The image sensor according to claim 1, It is characterized in that Each pixel group in the pixel groups in the same row corresponds to the first microlens; Alternatively, each pixel group in the pixel groups in the same row corresponds to at least two of the second micro lenses.
3. The image sensor according to claim 2, It is characterized in that In the case where the pixel group in the i-th row corresponds to the first microlens, the pixel group in the i+1-th row corresponds to at least two of the second microlenses, the pixel group in the i+2-th row corresponds to at least two of the second microlenses, and the pixel group in the i+3-th row corresponds to the first microlens; Alternatively, when the pixel group in the i-th row corresponds to the first microlens, the pixel group in the i+1-th row corresponds to at least two of the second microlenses, the pixel group in the i+2-th row corresponds to the first microlens, and the pixel group in the i+3-th row corresponds to at least two of the second microlenses; Alternatively, when the pixel group in the i-th row corresponds to the first microlens, the pixel group in the i+1-th row corresponds to the first microlens, the pixel group in the i+2-th row corresponds to at least two of the second microlenses, and the pixel group in the i+3-th row corresponds to at least two of the second microlenses; Alternatively, when the pixel group in the i-th row corresponds to at least two of the second microlenses, the pixel group in the i+1-th row corresponds to the first microlens, the pixel group in the i+2-th row corresponds to the first microlens, and the pixel group in the i+3-th row corresponds to at least two of the second microlenses; Wherein, i is a positive integer.
4. The image sensor according to claim 1, It is characterized in that The number of pixel units in the pixel group is 2, 4, 6, 9 or 16, and the color of the pixel units in each pixel group is the same.
5. The image sensor according to any one of claims 1 to 4, It is characterized in that The image sensor further comprises: an exposure reading control module and an exposure triggering control module; the exposure reading control module is connected to the pixel array, and the exposure triggering control module is respectively connected to the exposure reading control module, the pixel array and the morphology switching structure; The exposure reading control module is used to control the pixel array to perform row-by-row exposure to obtain a first image frame; The exposure trigger control module is used to: At a first moment, the pixel array is controlled to stop exposure, and the form switching structure is controlled to drive the microlens array to move, and after the microlens array has completed moving, the pixel array is controlled to continue to be exposed row by row; wherein the first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure, and the pixel array corresponding to the second image frame starts to be exposed; At a second moment, the pixel array is controlled to stop exposure, and the morphology switching structure is controlled to drive the microlens array to move, and after the microlens array has completed moving, the pixel array is controlled to continue to expose row by row; wherein the second moment is a moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure.
6. The image sensor according to claim 5, It is characterized in that The exposure reading control module is specifically used to control the pixel array to perform row-by-row exposure in an interlaced exposure mode.
7. The image sensor according to claim 5, It is characterized in that The exposure reading control module is further used to send first information to the exposure triggering control module, where the first information is used to indicate the first moment and the second moment; The exposure trigger control module is further used to determine the first moment and the second moment according to the first information.
8. The image sensor according to claim 7, It is characterized in that The first information includes any one of the following: The time information of the first moment and the time information of the second moment; An exposure parameter of the first image frame and an exposure parameter of the second image frame.
9. An exposure control method, It is characterized in that The method according to any one of claims 1 to 8, wherein the method comprises: Controlling the pixel array of the image sensor to perform row-by-row exposure to obtain a first image frame; Controlling the pixel array to stop exposure at a first moment, and switching the microlens group corresponding to each pixel group through the morphology switching structure of the image sensor; wherein the first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure and the pixel array corresponding to the second image frame starts to be exposed; Controlling the pixel array of the image sensor to continue to perform row-by-row exposure; At a second moment, the pixel array is controlled to stop exposure, and the microlens group corresponding to each pixel group is switched through the morphology switching structure; wherein the second moment is a moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure; Controlling the pixel array of the image sensor to continue to perform row-by-row exposure until the exposure of the pixel array corresponding to the second image frame is completed; Target image information or target phase information is obtained based on first data corresponding to the first image frame and second data corresponding to the second image frame.
10. The method according to claim 9, It is characterized in that The first data and the second data include image information or phase information corresponding to each pixel group in the pixel array; The obtaining target image information or target phase information based on first data corresponding to the first image frame and second data corresponding to the second image frame includes: synthesizing the image information in the first data and the image information in the second data to obtain the target image information; or, The phase information in the first data is synthesized with the phase information in the second data to obtain the target phase information.
11. An exposure device, It is characterized in that The exposure device comprises an image sensor as claimed in any one of claims 1 to 8, and the device further comprises a control module; The control module is used for: Controlling the pixel array of the image sensor to perform row-by-row exposure to obtain a first image frame; Controlling the pixel array to stop exposure at a first moment, and switching the microlens group corresponding to each pixel group through the morphology switching structure of the image sensor; wherein the first moment is the moment when the pixel array corresponding to the first image frame has not completed exposure and the pixel array corresponding to the second image frame starts to be exposed; Controlling the pixel array of the image sensor to continue to perform row-by-row exposure; At a second moment, the pixel array is controlled to stop exposure, and the microlens group corresponding to each pixel group is switched through the morphology switching structure; wherein the second moment is a moment when the pixel array corresponding to the first image frame has completed exposure, and the pixel array corresponding to the second image frame has not completed exposure; Controlling the pixel array of the image sensor to continue to perform row-by-row exposure until the exposure of the pixel array corresponding to the second image frame is completed; Target image information or target phase information is obtained based on first data corresponding to the first image frame and second data corresponding to the second image frame.
12. An electronic device, It is characterized in that The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the exposure method according to claim 9 or 10 are implemented.
Citation Information
Patent Citations
Image sensor, focusing control method, imaging device and mobile terminal
CN107135340A
Image sensor
CN115224060A
Hybrid image pixels for phase detection autofocus
CN117174723A
Optical device
JP2017207695A
Solid-state imaging device, method for manufacturing solid-state imaging device, and electronic apparatus
WO2023248693A1
