Imaging device and control method

Abstract

The imaging device (100) includes a plurality of microlenses (7), an imaging element including a plurality of pixels (2) arranged for each of the plurality of microlenses (7) and receiving light from the plurality of microlenses (7), and an image generating unit (36) that generates an image based on the light received by the plurality of pixels (2), each of the plurality of pixels (2) having a first portion including a central portion and a second portion surrounding the central portion, and the image generating unit (36) determines whether or not to combine a first image based on the light received by the first portion and a second image based on the light received by the second portion based on a predetermined condition. This makes it possible to determine whether or not to combine multiple images based on light received by different portions of the pixels in one shooting.

Description

Technical Field

[0001] The present disclosure relates to an imaging device and a control method.

Background Art

[0002] Conventionally, video cameras, digital cameras, etc. incorporating a solid-state imaging device are provided with a mechanical diaphragm mechanism for controlling the aperture in order to adjust the amount of light of a subject imaged on the solid-state imaging device or film.

[0003] For example, Patent Document 1 discloses a diaphragm mechanism in which a drive ring rotated by a step motor rotates a plurality of blades in the same direction at the same time to control the aperture diameter.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in an imaging device provided with a mechanical diaphragm mechanism as described in Patent Document 1, it is impossible to generate a plurality of images with different diaphragms in a single shooting in the first place. Therefore, it is not assumed to determine whether or not to synthesize images with different diaphragms in a single shooting.

[0006] Therefore, an object of the present disclosure is to provide an imaging technique capable of determining whether or not to synthesize a plurality of images based on light received by different portions of pixels in a single shooting.

Means for Solving the Problems

[0007] An imaging device according to an aspect of the present disclosure includes a plurality of microlenses, an image sensor provided for each of the plurality of microlenses and including a plurality of pixels that receive light from the plurality of microlenses, and an image generation unit that generates an image based on the light received by the plurality of pixels. Each of the plurality of pixels has a first portion including a central portion and a second portion surrounding the central portion. The image generation unit determines whether to synthesize a first image based on the light received by the first portion and a second image based on the light received by the second portion based on a predetermined condition.

[0008] A control method according to an aspect of the present disclosure is a control method executed by a processor included in an imaging device. The method includes generating an image based on light from a plurality of microlenses received by a plurality of pixels arranged for each of the plurality of microlenses included in the imaging device. Each of the plurality of pixels has a first portion including a central portion and a second portion surrounding the central portion. The generating step determines whether to synthesize a first image based on the light received by the first portion and a second image based on the light received by the second portion based on a predetermined condition.

Advantages of the Invention

[0009] According to the present disclosure, it is possible to determine whether to synthesize a plurality of images based on light received by different portions of pixels in one shooting.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11(A)

Figure 11(B)

[0011] Hereinafter, preferred embodiments of the present disclosure will be specifically described with reference to the accompanying drawings. Note that each of the embodiments described below is merely a specific example for carrying out the present disclosure and does not limitatively interpret the present disclosure. Also, for ease of understanding of the description, the same reference numerals are given to the same components in each drawing as much as possible, and redundant descriptions may be omitted.

[0012] FIG. 1 is a diagram showing an example of the configuration of an imaging device according to an embodiment of the present disclosure. The imaging device 100 illustratively includes an image sensor 10 (imaging element), an optical system 20, and a control unit 30.

[0013] The image sensor 10 is a device that receives light emitted from the subject S and converts the light and dark or the like into electrical information. The image sensor 10 includes, by way of example, a pixel group composed of a plurality of pixels 2, and a control circuit 1 that drives the pixel group, reads out data based on the optical signals accumulated in the pixel group, and controls to output the data to the outside of the image sensor 10.

[0014] The plurality of pixels 2 in the pixel group are arranged for each of the plurality of microlenses 7 described later, and receive light from the plurality of microlenses 7. The specific configuration of the image sensor 10 will be described with reference to FIG. 4. Note that the pixel group may be provided in the image sensor 10 as described above, or may be provided in the optical system 20.

[0015] The control unit 30 generates an image by analyzing data based on the data output from the image sensor 10. The specific configuration of the control unit 30 will be described with reference to FIG. 5.

[0016] The optical system 20 includes one or a plurality of devices that perform light condensation or the like on the light emitted from the subject S. The optical system 20 includes, by way of example, a main lens 6, a microlens 7, and a color filter 8.

[0017] The main lens 6 has a function as a photographing lens, for example. The main lens 6 has a central region LIA and a peripheral region LOA. The method of setting the central region LIA and the peripheral region LOA in the main lens 6 is arbitrary and can be appropriately set in consideration of the properties and arrangement of each component of the imaging device 100.

[0018] The microlens 7 is a condenser lens, for example. One or a plurality of microlenses 7 are arranged above or in front of the pixel group 2 and have a function of collecting desired light for each of the plurality of pixels included in the pixel group 2. Each microlens 7 corresponds to a pixel of the color filter 8 described later (for example, each of the plurality of pixels included in the pixel group 2).

[0019] The color filter 8 is associated with, for example, any one of the primary colors red, green, and blue. The color filter 8 may be a filter of complementary colors (for example, yellow, cyan, and magenta), and is selectively used depending on the application. The color filter 8 is, for example, of an on-chip type, but is not limited thereto, and may be of other forms such as a pasted type. Further, the color filter 8 may have a configuration separate from the microlens 7, or may be a part of the configuration of the microlens 7.

[0020] FIG. 2 is a diagram for explaining the light reception state in the main lens 6 viewed from the II direction of FIG. 1 according to an embodiment of the present disclosure. As shown in FIGS. 1 and 2, in the imaging device 100, the light emitted from the subject S passes through, for example, the central region LIA and the peripheral region LOA of the main lens 6, respectively, and enters the microlens 7.

[0021] FIG. 3 is a diagram for explaining the light reception state in the pixel viewed from the III direction of FIG. 1 according to an embodiment of the present disclosure. As shown in FIGS. 1 and 3, for example, each of the plurality of pixels 2 has a central sub-pixel 2a (first portion) including a central portion and a peripheral sub-pixel 2b (second portion) surrounding the central portion. The central sub-pixel 2a receives the optical signal (main signal PS) of the light passing through the central region LIA of the main lens 6. The peripheral sub-pixel 2b receives the optical signal (sub signal SS) of the light passing through the peripheral region LOA of the main lens 6.

[0022] FIG. 4 is a diagram showing an example of the configuration of an image sensor according to an embodiment of the present disclosure. The image sensor 10 is, for example, a CMOS image sensor or the like. The image sensor 10 illustratively includes a control circuit 1 shown in FIG. 1, a pixel group of a plurality of pixels 2 two-dimensionally arranged, signal lines 3, a readout circuit 4, and a digital signal processing unit (DSP) 5.

[0023] The configuration of the plurality of pixels 2 is arbitrary. For example, the plurality of pixels 2 may be grouped as one pixel group (unit pixel group) by collecting and grouping a plurality of single pixels. Also, as shown in FIG. 4, the plurality of pixels 2 may be grouped as one pixel group, for example, by grouping 4 (2×2) pixels. Further, the plurality of pixels 2 may use, for example, 3 (3×1) pixels, 8 (4×2) pixels, 9 (3×3) pixels, and 16 (4×4) pixels, etc. as the unit pixel group.

[0024] The plurality of pixels 2 are two-dimensionally arranged, and based on the control signal from the control circuit 1 and the control signal generated by the plurality of pixels 2 themselves, the optical signal brought to the image sensor 10 is accumulated and read out as data (electrical signal) based on the optical signal.

[0025] The read circuit 4 is transmitted with the electrical signals read from the plurality of pixels 2 via signal lines 3 (typically, column signal lines parallel to the column direction), and the electrical signals are analog-to-digital converted.

[0026] The digital signal processing unit (DSP) 5 processes the digital signals analog-to-digital converted by the read circuit 4. Then, the processed digital signals are transmitted via the data bus to the processor, memory, etc. of the imaging device.

[0027] Note that the DSP 5 is not limited to such an arrangement configuration. For example, the image sensor 10 may not include the DSP 5, and a configuration in which a subsequent processor (for example, the control unit 30) has the DSP may be used. Also, a configuration in which a part of the digital signal processing in image processing is processed by the DSP 5 included in the image sensor 10 and the DSPs included in subsequent processors, etc. may be used. In other words, the position of the DSP in the present disclosure is not limited to a specific position.

[0028] FIG. 5 is a diagram showing an example of the functional configuration of a control unit according to an embodiment of the present disclosure. As shown in FIG. 5, the control unit 30 (for example, a processor) functionally includes an analysis unit 32, a filter processing unit 34, and an image generation unit 36. Each of the above units of the control unit 30 can be realized, for example, by using a storage area such as a memory or a hard disk provided in the imaging device 100, or by the processor executing a program stored in the storage area.

[0029] The analysis unit 32 analyzes data based on the data output from the image sensor 10. For example, the analysis unit 32 analyzes the main signal PS or the first image (for example, the main image generated based on the main signal PS) to obtain information such as the depth of field and the sensitivity to light. The analysis unit 32 analyzes the sub-signal SS or the second image (for example, the sub-image generated based on the sub-signal SS) to obtain information such as whether the sub-signal SS includes a flare component and whether the sharpness is lost.

[0030] For example, when one or more images are generated, the analysis unit 32 may obtain and analyze the position information of each image and the position information of the pixels corresponding to each image. The analysis unit 32 may calculate the mutual relationship between a plurality of images to identify a portion with a large correlation (for example, a focused portion) or a small correlation (for example, an unfocused portion).

[0031] The filter processing unit 34 executes a filter process on the generated image based on the analysis result by the analysis unit 32. For example, the filter processing unit 34 may execute a predetermined spatial filter process on the second image generated based on the position information of the image obtained by the analysis unit 32.

[0032] The filter processing unit 34 may execute a predetermined low-pass filter process on a portion with a small correlation based on the mutual relationship between a plurality of images analyzed by the analysis unit 32.

[0033] The image generation unit 36 can generate one or more images based on the light received by the plurality of pixels 2. For example, based on a predetermined condition, the image generation unit 36 determines whether to synthesize a main image (first image) based on the light (main signal PS) received by the central sub-pixel 2a shown in FIGS. 1 and 3 and a sub-image (second image) based on the light (sub-signal SS) received by the peripheral sub-pixels 2b. Note that the "second image" includes, but is not limited to, a sub-image generated based only on the sub-signal SS, and may also include a sub-image generated based on the main signal PS and the sub-signal SS.

[0034] Here, the "predetermined condition" includes, but is not limited to, a condition related to the main image (first image). For example, when determining the synthesis of a plurality of images, as the "predetermined condition", a condition related to the sub-image (second image) may be referred to instead of or in addition to the condition related to the main image. Further, as the "predetermined condition", it may be set in advance for the imaging device 100 whether to synthesize the main image (first image) and the sub-image (second image) or not. Note that the "predetermined condition" may be fixed or may be appropriately changed based on the usage status of the user or the like.

[0035] Here, when classifying the main signal PS (main image), the sub-signal SS (sub-image), and the usage forms of the main signal PS and the sub-signal SS in the present embodiment, they are as follows (1) to (3). <(1) When the sub-signal SS (sub-image) is unnecessary> Regarding this usage form (1), the image generation unit 36 generates a main image based only on the main signal PS and uses the generated main image as the final image. The analysis unit 32 analyzes at least one of the depth of field and the sensitivity to light with respect to the main signal PS. For example, it is assumed that at least one of a depth equal to or greater than a predetermined threshold and a sensitivity equal to or less than a predetermined threshold is detected. The predetermined threshold related to the depth of field and the predetermined threshold related to the sensitivity are arbitrary values, which may be fixed values or values that can be appropriately changed according to the design of the imaging device 100 or the like.

[0036] Here, the predetermined conditions regarding the main image (first image) include, for example, at least one of whether the depth of field of the main signal PS (main image) is equal to or greater than a predetermined threshold value, and whether the sensitivity of the main signal PS (main image) to light is equal to or less than a predetermined threshold value. For example, when the depth of field of a certain main signal PS (main image) is equal to or greater than a predetermined threshold value, in order to satisfy this condition, the sub-image based on the sub-signal SS is not used, and only the main image based on the main signal PS is used (refer to steps S3 and S4 in FIG. 6 described later).

[0037] In this usage mode (1), the imaging device 100 can perform shooting with a high depth of field and low sensitivity to light using only the main signal PS (main image). In addition, the imaging device 100 can perform so-called electronic aperture processing, and a mechanical aperture mechanism as in the prior art is unnecessary.

[0038] <(2) When the sub-signal SS is used and the sub-signal SS includes, for example, optically unfavorable degradation, etc.> In the case of the above usage mode (1), while the depth of field increases, the optical signal used for the final image decreases, resulting in a trade-off where the signal-to-noise ratio (SNR) deteriorates. Especially when the luminance of the subject is low, the adverse effect of this trade-off becomes significant. Therefore, there is a desire to improve the image quality of the final image while eliminating or reducing the adverse effect of the trade-off using the sub-signal SS as described later. Note that the improvement of the image quality, etc. includes, for example, the improvement of at least one of the depth of field, SNR, MTF, and color reproducibility.

[0039] For example, the usage mode (2) may further include the following sub-classifications (i), (ii), and (iii). In the usage mode (2), the image generation unit 34 synthesizes the main image and the sub-image based on information regarding the sub-image (for example, a flare component of the sub-signal SS or a component related to non-ideal optical characteristics of the main lens 6, etc.).

[0040] <<(i) When the sub-signal SS includes a flare component>> When the sub-signal SS includes flare components due to unwanted reflections inside the camera optical system or the like, the analysis unit 32 detects the flare components. The image generation unit 34 reconstructs a sub-image based on the analysis result of the flare components. The image generation unit 34 can generate a final image by adding the reconstructed sub-image to the main image and improve the image quality and the like of the final image.

[0041] <<(ii) When sharpness loss occurs in the sub-signal SS>> When the sub-signal SS includes components related to non-ideal optical characteristics (for example, manufacturing constraints and manufacturing variations) of the main lens 6 or the like shown in FIG. 1 (for example, sharpness loss has occurred), the analysis unit 32 detects the components. The image generation unit 34 reconstructs a sub-image based on the analysis result of the components related to the non-ideal optical characteristics. The image generation unit 34 can generate a final image by adding the reconstructed sub-image to the main image and improve the image quality and the like of the final image.

[0042] <<(iii) Increasing the depth of field by reducing the trade-off of SNR degradation>> Under shooting conditions where a decrease in the SNR of the main signal PS is a concern, it is possible to recover the SNR by adding the sub-signal SS to the main signal PS after performing sharpness processing on the sub-signal SS. Note that, for example, the sharpness processing may include unsharp masking, deconvolution processing, optimization processing using the main signal PS or the defocus amount as reference information, and processing by a neural network.

[0043] The method for reconstructing the image is arbitrary. For example, the method for reconstructing the image includes a method that models optical characteristics and uses an analytical inverse function or an inverse function prepared as a look-up table. The method for reconstructing the image may include a method that models optical characteristics, separately calculates a point spread function (PSF), and performs deconvolution processing. The method for reconstructing the image may include a method that models after simplifying physical optical characteristics to a certain extent, performs regularization, normalization, or optimization, or uses AI technology (such as Deep Learning) to generate a final image.

[0044] <(3) When at least one of the main signal PS and the sub-signal SS is used as additional information to modify the main signal PS and the sub-signal SS> For example, the usage mode (3) includes a method for controlling the background blur of the image described with reference to FIG. 7.

[0045] According to this embodiment, the imaging device 100 includes a plurality of microlenses 7, an imaging element including a plurality of pixels 2 arranged for each of the plurality of microlenses 7 and receiving light from the plurality of microlenses 7, and an image generation unit 36 that generates an image based on the light received by the plurality of pixels 2. In the imaging device 100, each of the plurality of pixels 2 has a central sub-pixel 2a including a central portion and peripheral sub-pixels 2b surrounding the central portion. In the imaging device 100, the image generation unit 36 determines whether to synthesize a main image based on the light received by the central sub-pixel 2a and a sub-image based on the light received by the peripheral sub-pixels 2b based on a predetermined condition regarding the main image.

[0046] Therefore, it is possible to determine whether to synthesize a plurality of images based on light received by different portions of pixels in one shooting. Further, unlike conventional imaging devices, the imaging device 100 does not require a mechanical lens aperture function, and for example, in one shooting at the same time, it is possible to obtain images with a plurality of different aperture values by electronic processing. Furthermore, the imaging device 100 can perform image processing using the obtained images with a plurality of different aperture values, for example, after shooting, and can separately perform at least one of electronically changing the depth of field of view (for example, changing to deep or shallow) and changing the amount of incident light.

[0047] FIG. 6 is a flowchart showing an example of image generation processing according to an embodiment of the present disclosure. As shown in FIG. 6, the imaging device 100 receives the main signal PS at the central sub-pixel 2a shown in FIGS. 1 and 3 and receives the sub-signal SS at the peripheral sub-pixels 2b (step S1). The imaging device 100 generates a main image (first image) based on the main signal PS (step S2). The imaging device 100 determines whether the main image satisfies a predetermined condition (step S3). If the main image satisfies the predetermined condition (in the case of No), the process proceeds to step S4. The imaging device 100 sets the generated main image as the final image (step S4).

[0048] On the other hand, if the main image does not satisfy the predetermined condition (in the case of Yes in step S3), the process proceeds to step S5. The imaging device 100 generates a sub-image (second image) based on the sub-signal SS (step S5). The imaging device 100 generates a final image (third image) based on the generated main image and sub-image (step S6).

[0049] Note that the order of each step of the image generation process according to the embodiment is not limited to the above and can be changed as appropriate. For example, the generation of the sub-image based on the sub-signal SS (step S5) may be executed together with the generation of the main image based on the main signal PS in step S2. In this case, if it is "Yes" in step S3, step S5 is omitted and step S6 is executed. On the other hand, if it is "No" in step S3, the sub-image based on the generated sub-signal SS is not used, and only the main image is used as the final image.

[0050] FIG. 7 is a diagram for explaining an example of a process of electronically adjusting background blur according to an embodiment of the present disclosure. The analysis unit 32 shown in FIG. 5 analyzes, for example, a primary image based on the main signal PS and a sub-image based on the main signal PS and the sub-signal SS (a sub-image based on the Primary image and the Secondary image). The analysis unit 32 performs spatial frequency analysis on the primary image and the sub-image. For example, the analysis unit 32 calculates the correlation between the primary image and the sub-image to identify a portion with a large correlation (for example, a focused portion) or a small portion (for example, an unfocused portion).

[0051] The analysis unit 32 generates a defocus map based on the analysis results of the primary image and the sub-image by the analysis unit 32. Then, the filter processing unit 34 performs a low-pass filter process, for example, on a portion with a small correlation in the sub-image based on the defocus map generated by the analysis unit 32. In this way, the imaging device 100 can generate a final image with a shallow depth of field by synthesizing the primary image and the sub-image.

[0052] FIG. 8 is a diagram showing an example of the progress of light due to the wave nature of light according to an embodiment of the present disclosure. For example, when the sizes of the central sub-pixel 2a and the peripheral sub-pixels 2b are small, the ability to separate light between the central sub-pixel 2a and the peripheral sub-pixels 2b deteriorates due to the wave nature of light, and the efficiency of light collection in the sub-pixels may decrease.

[0053] Specifically, as shown in FIG. 8, for example, all the light incident on the boundary B between the central sub-pixel 2a and the peripheral sub-pixel 2b does not normally condense on either the central sub-pixel 2a or the peripheral sub-pixel 2b. In reality, due to the wave nature of light, it is split and condensed by both the central sub-pixel 2a and the peripheral sub-pixel 2b.

[0054] Therefore, in this embodiment, as shown in FIGS. 9 and 10, the imaging device 100 is configured such that the effective optical path length (first optical path length) with respect to the light ray LR from the light emitting surface S1 of the plurality of microlenses 7 to the photoelectric conversion surface S3 (first incident surface) of the light of the central sub-pixel 2a is different from the effective optical path length (second optical path length) with respect to the light ray LR from the light emitting surface S1 to the photoelectric conversion surface S5 (second incident surface) of the light of the peripheral sub-pixel 2b. According to this configuration, the light ray separation ability at the boundary between the central sub-pixel 2a and the peripheral sub-pixel 2b can be improved. Therefore, the light condensing efficiency in the sub-pixel can be improved.

[0055] As shown in FIG. 9, in the imaging device 100, a convex lens 9 is disposed between the light emitting surface S1 and the photoelectric conversion surface S3 of the light of the central sub-pixel 2a and the photoelectric conversion surface S5 of the light of the peripheral sub-pixel 2b (for example, the surface on the pixel 2 side of the color filter 8). Depending on the properties of the convex lens 9 (for example, shape and refractive index), the progress of light in the convex lens 9 (for example, the propagation direction) can be adjusted. Therefore, by using this convex lens 9, it is possible to make the effective optical path length from the light emitting surface S1 to the photoelectric conversion surface S3 different from the effective optical path length from the light emitting surface S1 to the photoelectric conversion surface S5. As long as the progress of light can be adjusted to improve the light ray separation ability at the boundary between the central sub-pixel 2a and the peripheral sub-pixel 2b, the properties of the convex lens 9 are arbitrary.

[0056] Note that the arrangement position of the convex lens 9 is arbitrary as long as it is between the light emitting surface S1 and the photoelectric conversion surface S3 of the light of the central sub-pixel 2a and the photoelectric conversion surface S5 of the light of the peripheral sub-pixel 2b. The convex lens 9 may be disposed on the light emitting surface S1 of the microlens 7 or between the color filter 8 and the pixel 2.

[0057] As shown in FIG. 10, in the imaging device 100, the pixel 2 (the central sub-pixel 2a and the peripheral sub-pixels 2b) is arranged such that the first distance from the emission surface S1 to the photoelectric conversion surface S3 is different from the second distance from the emission surface S1 to the photoelectric conversion surfaces S5 and S7. For example, in the imaging device 100, the pixel 2 (the central sub-pixel 2a and the peripheral sub-pixels 2b) is arranged such that the photoelectric conversion surface S3 and the photoelectric conversion surfaces S5 and S7 have different heights.

[0058] More specifically, the pixel 2 is arranged such that the photoelectric conversion surface S3 is closer to the emission surface S1 of the microlens 7 than the photoelectric conversion surfaces S5 and S7. Note that the photoelectric conversion surface S5 and the photoelectric conversion surface S7 may have the same height or different heights.

[0059] FIG. 11 is a diagram showing an example of the configuration of sub-pixels according to an embodiment of the present disclosure. For example, each of the plurality of pixels 2 in the unit pixel UP in the present embodiment is composed of sub-pixels corresponding to each color (for example, red, blue, and green). The configuration of the sub-pixels is arbitrary. For example, as shown in FIG. 11(A), each of the plurality of pixels 2 includes a central sub-pixel 2a and peripheral sub-pixels 2b.

[0060] As shown in FIG. 11(B), each of the plurality of pixels 2 may include a central sub-pixel 2a, a peripheral sub-pixel 2b, and a peripheral sub-pixel 2c. According to this configuration, while the central sub-pixel 2a condenses the light passing through the central region LIA of the main lens 6 shown in FIG. 1, it is possible to detect focus shift information with the left peripheral sub-pixel 2b and the right peripheral sub-pixel 2c.

[0061] Note that in each of the plurality of pixels 2, the peripheral sub-pixel may include three or more peripheral sub-pixels. Also, each of the plurality of pixels 2 may include a plurality of central sub-pixels in addition to one or more peripheral sub-pixels.

[0062] Note that the above embodiments are for facilitating the understanding of the present invention and are not intended to limit the interpretation of the present invention. The present invention can be changed / improved without departing from its gist, and equivalents thereof are also included in the present invention. Further, the present invention can form various disclosures by appropriate combination of a plurality of components disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components may be appropriately combined from different embodiments.

[0063] The imaging device 100 of the present disclosure is applicable to a digital camera and terminal devices such as a smartphone, a tablet terminal, and a laptop personal computer having a camera function.

Explanation of Reference Numerals

[0064] 1... Control circuit, 2... Pixel, 3... Signal line, 4... Readout circuit, 5... Digital signal processing unit (DSP), 6... Main lens, 7... Microlens, 8... Color filter, 10... Image sensor, 20... Optical system, 30... Control unit, 32... Analysis unit, 34... Filter processing unit, 36... Image generation unit, 100... Imaging device, S... Subject

Claims

1. A plurality of microlenses, an imaging device including a plurality of pixels arranged for each of the plurality of microlenses and receiving light from the plurality of microlenses, and an image generation unit that generates an image based on the light received by the plurality of pixels, wherein each of the plurality of pixels has a first portion including a central portion and a second portion surrounding the central portion, and the image generation unit determines whether to synthesize the first image based on the light received by the first portion and a second image based on the light received by the second portion based on a predetermined condition related to at least one of a depth of field and sensitivity to light in the first image based on the light received by the first portion. An imaging device.

2. The image generation unit synthesizes the first image and the second image using information related to the second image based on the predetermined condition. The imaging device according to claim 1.

3. A first optical path length from a light emission surface of the plurality of microlenses to a first light incident surface of the first portion is different from a second optical path length from the light emission surface to a second light incident surface of the second portion. The imaging device according to claim 1.

4. A convex lens is disposed between the light emission surface and the first light incident surface and the second light incident surface. The imaging device according to claim 3.

5. A first distance from the light emission surface to the first light incident surface is different from a second distance from the light emission surface to the second light incident surface. The imaging device according to claim 3.

6. A control method executed by a processor included in an imaging device, the method including: generating an image based on light from a plurality of microlenses received by a plurality of pixels arranged for each of the plurality of microlenses included in the imaging device, wherein each of the plurality of pixels has a first portion including a central portion and a second portion surrounding the central portion, and the generating step determines whether to synthesize the first image based on the light received by the first portion and a second image based on the light received by the second portion based on a predetermined condition related to at least one of a depth of field and sensitivity to light in the first image based on the light received by the first portion. A control method.

7. A terminal including the imaging device according to any one of claims 1 to 5. A terminal.

Images