Imaging device and processing unit

Abstract

This technology provides an improved convenience for imaging devices. [Solution] The imaging device comprises an image sensor having a plurality of pixels and a processing unit that processes the output of the image sensor. Each of the plurality of pixels has a plurality of light-receiving elements. Each of the plurality of light-receiving elements has a photoelectric conversion element whose current changes according to the amount of light detected by the light-receiving element. The plurality of pixels include a first pixel that detects visible light and near-infrared light, and a second pixel that has a lower detection sensitivity for visible light than the first pixel.

Description

Technical Field

[0001] The present disclosure relates to an imaging device.

Background Art

[0002] Patent Document 1 discloses an image sensor in which each pixel has a photoelectric conversion element.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is room for improvement in an imaging device including an image sensor.

[0005] An object of the present disclosure is to provide a technology capable of improving the convenience of an imaging device.

Means for Solving the Problems

[0006] One aspect of an imaging device includes an image sensor having a plurality of pixels and a processing unit that processes the output of the image sensor. Each of the plurality of pixels has a plurality of light receiving elements. Each of the plurality of light receiving elements has a photoelectric conversion element in which the flowing current changes according to the amount of light detected by the light receiving element. The plurality of pixels includes a first pixel that detects visible light and near-infrared light and a second pixel that has a lower detection sensitivity for visible light than the first pixel.

[0007] One aspect of the processing unit is the processing unit included in the above-described imaging device.

Effects of the Invention

[0008] The convenience of the imaging device can be improved.

Brief Description of the Drawings

[0009] [Figure 1] This is a schematic diagram showing an example of an imaging device. [Figure 2] This is a schematic diagram showing an example of an image sensor configuration. [Figure 3] This is a schematic diagram showing an example of the configuration of the processing unit. [Figure 4] This is a schematic diagram showing an example of the configuration of pixels and pixel-corresponding acquisition units. [Figure 5] This is a schematic diagram showing an example of the configuration of a light-receiving element. [Figure 6] This graph shows an example of the voltage-current characteristics of a photoelectric conversion element. [Figure 7] This is a schematic diagram illustrating an example of the operation of the processing unit. [Figure 8] This is a schematic diagram illustrating an example of an output correspondence value. [Figure 9] This is a schematic diagram showing an example of the structure above the third pixel. [Figure 10] This is a schematic diagram showing an example of the structure above the first pixel. [Figure 11] This is a schematic diagram showing an example of the structure above the second pixel. [Figure 12] This is a schematic diagram showing an example of the arrangement of the first, second, and third pixels. [Figure 13] This is a schematic diagram showing an example of the arrangement of multiple unit pixel groups. [Figure 14] This is a schematic diagram illustrating an example of a composite correspondence value. [Figure 15] This is a schematic diagram illustrating an example of image pixel values. [Figure 16] This is a schematic diagram showing an example of the configuration of the processing unit. [Figure 17] This is a schematic diagram showing an example of the configuration of a unit pixel group and part of the processing unit. [Figure 18] This is a schematic diagram showing an example of the configuration of a unit pixel group and part of the processing unit. [Modes for carrying out the invention]

[0010] FIG. 1 is a schematic diagram showing an example of an imaging device 1 that captures an image. The imaging device 1 can, for example, capture an image in response to a shooting instruction from a user and display the captured image. The imaging device 1 is also called, for example, a camera. The imaging device 1 can capture, for example, still images and moving images. The image captured by the imaging device 1 may be a color image or a grayscale image.

[0011] As shown in FIG. 1, the imaging device 1 includes, for example, an image sensor 2, a processing unit 3, a display unit 4, and an input unit 5 that receives an input from a user. The image sensor 2, the processing unit 3, the display unit 4, and the input unit 5 are, for example, housed in an exterior case (or housing or enclosure).

[0012] The image sensor 2 has a plurality of pixels that detect light. The imaging device 1 is provided with, for example, a lens, and the light passing through the lens is irradiated onto the plurality of pixels. The pixels included in the image sensor 2 can also be referred to as pixel circuits, for example.

[0013] The processing unit 3 processes the outputs of the plurality of pixels included in the image sensor 2. The processing unit 3 can also control the display unit 4. The processing unit 3 can be referred to as, for example, a control unit, a control device, or a processing device. The processing unit 3 can cause the display unit 4 to display an image based on the output of the image sensor 2, that is, the image captured by the imaging device 1.

[0014] The input unit 5 can receive various inputs from a user. The input unit 5 may include, for example, a plurality of operation buttons. The input unit 5 may also include a touch sensor that receives a touch operation from a user. The processing unit 3 can identify the content of the input received by the input unit 5 based on the output signal from the input unit 5.

[0015] The display unit 4 can display various types of information under the control of the processing unit 3. The display unit 4 has a display surface for displaying various types of information. The display unit 4 may be, for example, a liquid crystal display or an organic EL (electro-luminescence) display. Furthermore, if the input unit 5 is equipped with a touch sensor, the touch sensor and the display surface of the display unit 4 may constitute a touch panel display having display and touch detection functions. In this case, the input unit 5 can detect touch operations on the display surface of the display unit 4. The input unit 5 and the display unit 4 constitute a user interface.

[0016] Hereafter, when we simply refer to a pixel, we mean the pixels of the image sensor 2. Also, when we simply refer to an image, we mean the image based on the output of the image sensor 2 (in other words, the image captured by the imaging device 1).

[0017] <Example of image sensor configuration> Figure 2 is a schematic diagram showing an example of the configuration of the image sensor 2. As shown in Figure 2, the image sensor 2 comprises, for example, a plurality of pixels 201 arranged in a matrix. The image sensor 2 also comprises a plurality of row signal lines 202 that extend in the row direction (left-right direction in Figure 2) and are arranged in the column direction (up-down direction in Figure 2), and a plurality of column signal lines 203 that extend in the column direction and are arranged in the row direction. A single pixel 201 is located at the intersection where a row signal line 202 and a column signal line 203 intersect.

[0018] <Example of processing unit configuration> Figure 3 is a schematic diagram showing an example of the configuration of the processing unit 3. As shown in Figure 3, the processing unit 3 includes, for example, a control unit 30, a storage unit 31, an acquisition unit 32, an AD converter 33, and a voltage generation unit 35.

[0019] In this specification, the prefixes "1st," "2nd," "3rd," etc., "L" (where L is an integer greater than or equal to 1) at the beginning of element names are added for convenience to distinguish multiple elements from one another. Therefore, among multiple element names that have the same prefix other than the leading "L," the leading "L" may be swapped.

[0020] The control unit 30 can comprehensively manage the operation of the processing unit 3 by controlling other components of the processing unit 3. The control unit 30 can comprehensively manage the operation of the imaging device 1. The control unit 30 includes, for example, at least one processor. The control unit 30 may include, for example, a CPU (Central Processing Unit). The control unit 30 can also be called a control circuit. The processing unit 3 can also be called a computer device.

[0021] The memory unit 31 may include non-temporary recording media that can be read by the CPU of the control unit 30, such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory unit 31 stores, for example, a program 31a for controlling the processing unit 3. Various functions of the control unit 30 are realized, for example, by the CPU of the control unit 30 executing the program 31a in the memory unit 31.

[0022] The configuration of the control unit 30 is not limited to the example described above. For example, the control unit 30 may have multiple CPUs. The control unit 30 may also have at least one DSP (Digital Signal Processor). Furthermore, all or some of the functions of the control unit 30 may be implemented by hardware circuits that do not require software to implement those functions. In addition, the storage unit 31 may include a computer-readable non-temporary recording medium other than ROM and RAM. The storage unit 31 may include, for example, a small hard disk drive and an SSD (Solid State Drive).

[0023] The acquisition unit 32 acquires a value corresponding to the output of each of the multiple pixels 201 (also called an output-corresponding value). The acquisition unit 32 is implemented, for example, by a hardware circuit that does not require software to realize its function. The acquisition unit 32 can also be called an acquisition circuit. The acquisition unit 32 has multiple pixel-corresponding acquisition units 320, each corresponding to one of the multiple pixels 201. The pixel-corresponding acquisition unit 320 acquires the output-corresponding value of the corresponding pixel 201. The pixel-corresponding acquisition unit 320 can also be called a pixel-corresponding acquisition circuit.

[0024] Figure 4 is a schematic diagram showing an example of a configuration consisting of one pixel 201 and one pixel-corresponding acquisition unit 320 corresponding to that pixel 201. Hereafter, one pixel 201 and one pixel-corresponding acquisition unit 320 corresponding to that pixel 201 may be collectively referred to as the pixel processing unit 400. The pixel processing unit 400 comprises the pixel 201 and the pixel-corresponding acquisition unit 320 corresponding to that pixel 201. The pixel processing unit 400 can also be called, for example, a pixel processing circuit 400.

[0025] As shown in Figure 4, each pixel 201 is equipped with a light-receiving element 210 that detects light. The light-receiving element 210 has a photoelectric conversion element 211 whose current changes according to the amount of light detected by the light-receiving element 210. The photoelectric conversion element 211 is, for example, a diode; hereafter, the photoelectric conversion element 211 that is a diode may be referred to as diode 211.

[0026] Figure 5 is a schematic diagram showing an example of the configuration of the photodetector 210. The photodetector 210 is, for example, a semiconductor device. As shown in Figure 5, the photodetector 210 comprises, for example, a p-type semiconductor region 216 and an n-type semiconductor region 217. The p-type semiconductor region 216 functions as the anode of the diode 211, and the n-type semiconductor region 217 functions as the cathode of the diode 211. The n-type semiconductor region 217, which functions as the cathode, is supplied with a voltage VR output by the voltage generation unit 35. Voltage VR can be said to be the reverse voltage supplied to the diode 211. The configuration between the p-type semiconductor region 216 and the n-type semiconductor region 217 shown in Figure 5 represents a depletion layer.

[0027] When light is shone on pixel 201, the light is shone on the junction between the p-type semiconductor region 216 and the n-type semiconductor region 217 in the photodetector element 210 of pixel 201. As a result, electrons and holes are generated at the junction due to the photovoltaic effect. The holes move to the p-type semiconductor region 216, and the electrons move to the n-type semiconductor region 217. Consequently, the current flowing through diode 211 changes according to the amount of light shone on the junction, that is, the amount of light detected by the photodetector element 210. For example, the absolute value of the current I in diode 211 increases as the amount of light detected by the photodetector element 210 increases. Diode 211 can also be called a photodiode or light-receiving diode. Generally, the value of the current I is considered positive when moving from the anode to the cathode. Therefore, the value of the current I in diode 211 is negative.

[0028] Figure 6 is a schematic diagram showing an example of the voltage-current characteristics (also called IV characteristics) of diode 211. The horizontal axis of Figure 6 represents the forward voltage VF of diode 211, and the vertical axis of Figure 6 represents the current I.

[0029] Figure 6 shows curve 100 representing the IV characteristics when the photodetector 210 is not detecting light. Figure 6 also shows curve 101 representing the IV characteristics when the photodetector 210 is detecting a relatively small amount of light (first intensity). Furthermore, Figure 6 shows curve 102 representing the IV characteristics when the photodetector 210 is detecting a relatively large amount of light (second intensity). Hereafter, the curves representing the IV characteristics may be referred to as IV curves.

[0030] In this example, the voltage VR (in other words, the reverse voltage) is set so that a current I flows through the diode 211 when the photodetector 210 is not detecting light. In Figure 6, the negative value of the forward voltage VF is the value of the voltage VR. As shown in Figure 6, when the value of the voltage VR is V1, the value of the current I in the diode 211 of the photodetector 210 that is not detecting light is I1. That is, a current I1 flows through the photodetector 210 that is not detecting light. Also, when the value of the voltage VR is V1, the value of the current I in the diode 211 of the photodetector 210 that is detecting a relatively small first-intensity light is I2, which is smaller than I1. And when the value of the voltage VR is V1, the value of the current I in the diode 211 of the photodetector 210 that is detecting a relatively large second-intensity light is I3, which is smaller than both I1 and I2.

[0031] In this example, current flows through the light-receiving element 210, which is not detecting light. The current flowing through the light-receiving element 210 that is not detecting light is also called dark current. The current I in the diode 211 of the light-receiving element 210 that is not detecting light is the dark current. The current flowing through the light-receiving element 210 changes according to the amount of light detected by the light-receiving element 210. Specifically, the current flowing through the light-receiving element 210 increases as the amount of light detected by the light-receiving element 210 increases.

[0032] In this example, the current I flowing through diode 211 becomes the output of pixel 201. The output of pixel 201 changes according to the amount of light detected by that pixel 201.

[0033] Returning to Figure 4, the pixel-corresponding acquisition unit 320 is connected to the corresponding pixel 201. The pixel-corresponding acquisition unit 320 receives a current I, which is the output of the corresponding pixel 201. The pixel-corresponding acquisition unit 320 includes, for example, a first selection switch 322 and a second selection switch 323. The pixel-corresponding acquisition unit 320 also includes, for example, an exposure time setting switch 324, a reset switch 325, and a capacitor 327.

[0034] The on / off states of the first selection switch 322, the second selection switch 323, the exposure time setting switch 324, and the reset switch 325 are controlled by the control unit 30.

[0035] The first selection switch 322 and the second selection switch 323 may be composed of, for example, NMOS transistors. The first selection switch 322 and the second selection switch 323 provided in the pixel-dependent acquisition unit 320 are used to select the pixel 201 corresponding to the pixel-dependent acquisition unit 320. When the first selection switch 322 and the second selection switch 323 provided in the pixel-dependent acquisition unit 320 are turned ON, the pixel 201 corresponding to the pixel-dependent acquisition unit 320 is selected.

[0036] The on / off state of the first selection switch 322 is controlled by a control signal output from the control unit 30 to the row signal line 202. For example, when the control signal output to the row signal line 202 is at a high level, the first selection switch 322 is turned on, and when the control signal output to the row signal line 202 is at a low level, the first selection switch 322 is turned off. The on / off state of the second selection switch 323 is controlled by a control signal output from the control unit 30 to the column signal line 203. For example, when the control signal output to the column signal line 203 is at a high level, the second selection switch 323 is turned on, and when the control signal output to the column signal line 203 is at a low level, the second selection switch 323 is turned off.

[0037] One end of the first selection switch 322 is connected to the anode of the diode 211. The other end of the first selection switch 322 is connected to one end of the second selection switch 323. The other end of the second selection switch 323 is connected to one end of the exposure time setting switch 324.

[0038] The exposure time setting switch 324 is used to set the exposure time of pixel 201. The exposure time of pixel 201 is the time during which pixel 201 detects light, and can also be described as the time during which current I flows through the diode 211 of pixel 201.

[0039] The other end of the exposure time setting switch 324 is connected to one end of the reset switch 325 and one end of the capacitor 327. The other end of the reset switch 325 and the other end of the capacitor 327 are connected to ground potential. The on / off state of the exposure time setting switch 324 is controlled by the exposure setting signal es output from the control unit 30. For example, when the exposure setting signal es is at a high level, the exposure time setting switch 324 is in the ON state, and when the exposure setting signal es is at a low level, the exposure time setting switch 324 is in the OFF state. The exposure time setting switch 324 may be composed of, for example, an NMOS transistor.

[0040] The current I flowing through diode 211 causes charge to accumulate in capacitor 327. The reset switch 325 is used to discharge the charge accumulated in capacitor 327 and initialize capacitor 327.

[0041] The on / off state of the reset switch 325 is controlled by the reset signal rs output from the control unit 30. For example, when the reset signal rs is at a high level, the reset switch 325 is in the ON state, and when the reset signal rs is at a low level, the reset switch 325 is in the OFF state. The reset switch 325 may be composed of, for example, an NMOS transistor.

[0042] In the pixel processing unit 400 having the above configuration, when the reset switch 325 is in the off state and the first selection switch 322, the second selection switch 323, and the exposure time setting switch 324 are in the on state, the current I flowing through the light-receiving element 210 flows through the capacitor 327, and charge is accumulated in the capacitor 327. Then, the voltage at one end of the capacitor 327 changes according to the amount of charge accumulated in the capacitor 327. The value of the voltage at one end of the capacitor 327 becomes the output corresponding value corresponding to the output of the pixel 201. The output corresponding value of the pixel 201 is a value corresponding to the current I of the pixel 201. Since the current I can be said to be the detection result of the pixel 201, the output corresponding value can also be said to be a detection result value or detection result signal indicating the detection result of the pixel 201. The voltage at one end of the capacitor 327, i.e., the output corresponding value, is converted from analog format to digital format by the AD converter 33 of the processing unit 3, and then input to the control unit 30.

[0043] Figure 7 is a schematic diagram illustrating an example of the operation of the control unit 30. Figure 7 shows an example of the time variation of the reset signal rs and the exposure setting signal es output by the control unit 30. The control unit 30 controls multiple pixel processing units 400 to cause each pixel processing unit 400 to output an output corresponding value. The output corresponding value output from each pixel processing unit 400 is input to the control unit 30. Hereafter, a single pixel 201 of interest may be referred to as the "pixel of interest 201" in this explanation. Also, the pixel processing unit 400 equipped with the pixel of interest 201 may be referred to as the "pixel of interest processing unit 400".

[0044] When the control unit 30 causes the focus pixel processing unit 400 to output an output corresponding value, it first sets the first selection switch 322 and the second selection switch 323 of the focus pixel processing unit 400 to the ON state to select the focus pixel 201.

[0045] Next, as shown in Figure 7, the control unit 30 changes the reset signal rs and the exposure setting signal es from a low level to a high level, turning on the reset switch 325 and the exposure time setting switch 324. After a predetermined time has elapsed, the control unit 30 changes the reset signal rs to a low level, turning off the reset switch 325. This initializes the capacitor 327 and the state of the path through which the current I flows. If the exposure time setting switch 324 remains on and the reset switch 325 is turned off, the current I begins to flow into the capacitor 327, and the accumulation of charge in the capacitor 327 begins. After the accumulation of charge in the capacitor 327 has begun and a predetermined time has elapsed, the control unit 30 changes the exposure setting signal es from a high level to a low level, turning off the exposure time setting switch 324. As shown in Figure 7, the time from when the reset signal rs changes to a low level to when the exposure setting signal es changes to a low level is the exposure time T. During the exposure time T, a current I flows through the diode 211, and charge is accumulated in the capacitor 327 by this current I. After the exposure setting signal es changes to a low level, the AD converter 33 of the processing unit 3 converts the voltage at one end of the capacitor 327 from analog to digital and outputs it to the control unit 30 as an output corresponding value corresponding to the output of the pixel of interest 201. In this example, the exposure time T is fixed.

[0046] As described above, each pixel processing unit 400 is controlled by the control unit 30, allowing the control unit 30 to acquire output corresponding values ​​for multiple pixels 201. Since the current I changes according to the amount of light detected by the photodetector 210 (also called the detected light amount), the amount of charge accumulated in the capacitor 327 during exposure time T also changes according to the detected light amount. Therefore, the output corresponding value changes according to the detected light amount. Specifically, the larger the detected light amount, the larger the output corresponding value. The output corresponding value can also be said to be the output value of the pixel processing unit 400.

[0047] The AD converter 33 in the processing unit 3 has a resolution of, for example, 10 bits. Therefore, the minimum and maximum values ​​of the digital output corresponding values ​​that the control unit 30 receives from the pixel processing unit 400 are "0" and "1023", respectively. Note that the resolution of the AD converter 33 may be other than 10 bits. Hereafter, when simply referring to the output corresponding value, it means the digital output corresponding value output from the AD converter 33.

[0048] The voltage generation unit 35 in the processing unit 3 generates a voltage VR and supplies the generated voltage VR to the cathode of the diode 211 of each pixel 201. The voltage generation unit 35 may be configured as, for example, a step-down circuit that steps down the input voltage and outputs it. The voltage generation unit 35 can also be called, for example, a voltage generation circuit or a voltage generating circuit.

[0049] The voltage VR settings of the diodes 211 for multiple pixels 201 on the image sensor 2 are all the same. In this example, the voltage VR is fixed and does not change. The voltage VR and exposure time T can also be viewed as shooting conditions.

[0050] <About pixel types> In this example, the image sensor 2 has multiple types of pixels 201. For example, the image sensor 2 has three types of pixels 201: a first pixel 201a (see Figures 10 and 12 below), a second pixel 201b (see Figures 11 and 12 below), and a third pixel 201c (see Figures 9 and 12 below). For example, the image sensor 2 has multiple first pixels 201a, multiple second pixels 201b, and multiple third pixels 201c.

[0051] The first pixel 201a can detect visible light and near-infrared light. Specifically, the photodetector 210 of the first pixel 201a can detect visible light and near-infrared light. Here, visible light in this disclosure refers to light with a wavelength of 380 nm or more and less than 800 nm. Near-infrared light in this disclosure refers to light with a wavelength of 800 nm or more and less than or equal to 2500 nm. Hereafter, when simply referred to as light, it means light including visible light and near-infrared light.

[0052] In the first pixel 201a, the detection sensitivity to near-infrared light is lower than, for example, the detection sensitivity to visible light. In the first pixel 201a, the detection sensitivity to near-infrared light is, for example, 50% or more and 80% or less of the detection sensitivity to visible light.

[0053] When the first pixel 201a detects visible light and near-infrared light, the output corresponding value of the first pixel 201a (also called the first output corresponding value) includes a component corresponding to the dark current (dark current component), a component corresponding to the detection result of visible light (visible light detection component), and a component corresponding to the detection result of near-infrared light (near-infrared light detection component).

[0054] In this example, for the sake of explanation, the visible light detection component and the near-infrared light detection component included in the first output corresponding value may be represented as visible light detection component (medium) and near-infrared light detection component (small), respectively. The representations of visible light detection component (medium) and near-infrared light detection component (small) are solely for the sake of explanation, and it is not necessarily the case that the visible light detection component (medium) will be larger than the near-infrared light detection component (small) in the first output corresponding value. As is clear from the above explanation, the visible light detection component (medium) is determined by the amount of visible light contained in the light detected by the first pixel 201a, and the infrared light detection component (small) is determined by the amount of near-infrared light contained in the light detected by the first pixel 201a. Furthermore, the visible light detection component (medium) included in the first output corresponding value of one pixel 201 is not necessarily the same as the visible light detection component (medium) included in the first output corresponding value of another pixel 201, and the near-infrared light detection component (small) included in the first output corresponding value of one pixel 201 is not necessarily the same as the near-infrared light detection component (small) included in the first output corresponding value of another pixel 201. The same applies to the visible light detection component (small), visible light detection component (large), and near-infrared light detection component (medium) described later.

[0055] The second pixel 201b can detect visible light and near-infrared light. Its detection sensitivity for visible light is lower than that of the first pixel 201a, while its detection sensitivity for near-infrared light is about the same as that of the first pixel 201a. The ratio of the detection sensitivity of the second pixel 201b to that of the first pixel 201a for visible light is, for example, between 50% and 80%. The ratio of the detection sensitivity of the second pixel 201b to that of the first pixel 201a for near-infrared light is, for example, approximately 100%. In other words, the detection sensitivity of the second pixel 201b for near-infrared light is almost the same as that of the first pixel 201a for near-infrared light.

[0056] When the second pixel 201b detects visible light and near-infrared light, the output corresponding value of the second pixel 201b (also called the second output corresponding value) includes the dark current component, the visible light detection component, and the near-infrared light detection component, similar to the first output corresponding value.

[0057] When the first pixel 201a and the second pixel 201b detect the same light, the visible light detection component included in the second output corresponding value will be 50% or more and 80% or less of the visible light detection component included in the first output corresponding value, and the near-infrared light detection component included in the second output corresponding value will be approximately the same as the near-infrared light detection component included in the first output corresponding value. In addition, the dark current component included in the second output corresponding value will be approximately the same as the dark current component included in the first output corresponding value.

[0058] When the first pixel 201a and the second pixel 201b detect the same light, the ratio of the near-infrared light detection component to the visible light detection component in the second output corresponding value is greater than the ratio of the near-infrared light detection component to the visible light detection component in the first output corresponding value.

[0059] In this example, for the sake of explanation, the visible light detection component and the near-infrared light detection component included in the second output corresponding value may be represented as the visible light detection component (small) and the near-infrared light detection component (small), respectively.

[0060] The third pixel 201c has lower detection sensitivity for both visible light and near-infrared light than the first pixel 201a and the second pixel 201b. For example, the third pixel 201c can hardly detect visible light and near-infrared light. The ratio of the detection sensitivity of the third pixel 201c to the detection sensitivity of the first pixel 201a for visible light is, for example, 5% or less. Similarly, the ratio of the detection sensitivity of the third pixel 201c to the detection sensitivity of the first pixel 201a for near-infrared light is, for example, 5% or less. When light is shone on the third pixel 201c, the dark current component is dominant in the output corresponding value of the third pixel 201c (also called the third output corresponding value), and the proportion of the dark current component is, for example, 99% or more of the total.

[0061] Figure 8 is a schematic diagram summarizing the components included in the first, second, and third output corresponding values. Since the third output corresponding value contains almost no visible light detection components or near-infrared light detection components, only the dark current component is shown in the column for the third output corresponding value in Figure 8.

[0062] In this example, as shown in Figure 9, a plurality of wiring layers 220 covering the photodetector 210 (also called the third photodetector 210c) of the third pixel 201c are provided above it. Visible light and near-infrared light hardly penetrate the wiring layers 220. As a result, the third photodetector 210c is hardly irradiated with visible light and near-infrared light, and the third photodetector 210c can hardly detect visible light and near-infrared light. In other words, the third pixel 201c can hardly detect visible light and near-infrared light. The wiring layers 220 may be made of, for example, silver, or other materials.

[0063] Furthermore, in this example, as shown in Figure 10, a plurality of wiring layers 230 are provided above the light-receiving element 210 (also called the first light-receiving element 210a) of the first pixel 201a. The plurality of wiring layers 230 are provided with openings 235 that expose the first light-receiving element 210a. As a result, the first light-receiving element 210a is directly illuminated with visible light and near-infrared light, and the first light-receiving element 210a can detect visible light and near-infrared light. In other words, the first pixel 201a can detect visible light and near-infrared light. The wiring layers 230 may be made of, for example, aluminum, or other materials.

[0064] Furthermore, in this example, as shown in Figure 11, above the photodetector 210 (also called the second photodetector 210b) of the second pixel 201b, a plurality of wiring layers 240 covering the second photodetector 210b are provided. Visible light is transmitted through the wiring layers 240 while being attenuated, and near-infrared light is transmitted through the wiring layers 240 with almost no attenuation. As a result, the detection sensitivity of the second photodetector 210b for visible light is lower than that of the first photodetector 210a, and the detection sensitivity of near-infrared light is about the same as that of the first photodetector 210a. In other words, the detection sensitivity of the second pixel 201b for visible light is lower than that of the first pixel 201a, and the detection sensitivity of near-infrared light is about the same as that of the first pixel 201a. The wiring layers 240 may be made of, for example, aluminum, or other materials.

[0065] In this example, a unit pixel group 250 is formed by a first pixel 201a, a second pixel 201b, and a third pixel 201c, for obtaining one pixel value from among multiple pixel values ​​that make up a single image captured by the imaging device 1. The image sensor 2 includes multiple unit pixel groups 250 for obtaining multiple pixel values ​​that make up an image. The control unit 30 of the processing unit 3 generates a single pixel value by combining, for example, at least two output corresponding values ​​(i.e., a first output corresponding value, a second output corresponding value, and a third output corresponding value) of the first pixel 201a, second pixel 201b, and third pixel 201c that make up the unit pixel group 250.

[0066] The first pixel 201a, second pixel 201b, and third pixel 201c constituting the unit pixel group 250 are, for example, arranged in close proximity to each other. For example, as shown in Figure 12, the first pixel 201a, second pixel 201b, and third pixel 201c may be arranged consecutively along the row direction. Alternatively, the first pixel 201a, second pixel 201b, and third pixel 201c may be arranged consecutively along the column direction. In the image sensor 2, the multiple unit pixel groups 250 are arranged in a matrix, as shown in Figure 13.

[0067] The control unit 30 acquires a value (referred to as a composite value) for each unit pixel group 250 that corresponds to the combined output of at least two pixels 201, specifically the first pixel 201a, second pixel 201b, and third pixel 201c that constitute the unit pixel group 250. The composite value can also be said to be a value that corresponds to the combined current I of multiple pixels 201. The control unit 30 uses the multiple composite values ​​acquired for each of the multiple unit pixel groups 250 as, for example, multiple pixel values ​​that constitute an image.

[0068] The control unit 30 includes, for example, a first acquisition mode, a second acquisition mode, a third acquisition mode, and a fourth acquisition mode for acquiring the combined value. In the first acquisition mode, for example, a value corresponding to the combination of the output of the first pixel 201a and the output of the second pixel 201b (also called the first combined value) is acquired. The first combined value changes according to the output of the first pixel 201a and changes according to the output of the second pixel 201b. In other words, the first combined value changes according to the detection result of the first pixel 201a and changes according to the detection result of the second pixel 201b.

[0069] In the second acquisition mode, for example, a value corresponding to the combination of the output of the first pixel 201a and the output of the third pixel 201c (also called the second combined value) is acquired. The second combined value changes according to the output of the first pixel 201a and changes according to the output of the third pixel 201c. In other words, the second combined value changes according to the detection result of the first pixel 201a and changes according to the detection result of the third pixel 201c.

[0070] In the third acquisition mode, for example, a value corresponding to the combination of the output of the second pixel 201b and the output of the third pixel 201c (also called the third combined value) is acquired. The third combined value changes according to the output of the second pixel 201b and changes according to the output of the third pixel 201c. In other words, the third combined value changes according to the detection result of the second pixel 201b and changes according to the detection result of the third pixel 201c.

[0071] In the fourth acquisition mode, for example, a value corresponding to the combination of the outputs of the first pixel 201a, the second pixel 201b, and the third pixel 201c (also called the fourth combined value) is acquired. The fourth combined value changes according to the output of the first pixel 201a, the output of the second pixel 201b, and the output of the third pixel 201c. In other words, the fourth combined value changes according to the detection result of the first pixel 201a, the detection result of the second pixel 201b, and the detection result of the third pixel 201c.

[0072] In the first acquisition mode, the control unit 30 combines the output corresponding values ​​of the first pixel 201a and the second pixel 201b included in the unit pixel group 250 to obtain a first combined corresponding value corresponding to the combined output of the first pixel 201a and the second pixel 201b. The output corresponding value of the first pixel 201a is a value corresponding to the output of the first pixel 201a, and the output corresponding value of the second pixel 201b is a value corresponding to the output of the second pixel 201b. Therefore, the value obtained by combining the output corresponding values ​​of the first pixel 201a and the second pixel 201b can be said to be a value corresponding to the combined output of the first pixel 201a and the second pixel 201b.

[0073] In the first acquisition mode, the control unit 30 uses, for example, the value obtained by subtracting the second output corresponding value from the first output corresponding value as the first combined corresponding value. The first combined corresponding value can be said to be a value corresponding to the combined output obtained by subtracting the output of the second pixel 201b from the output of the first pixel 201a. In other words, the first combined corresponding value can be said to be a value corresponding to the combined current (also called differential current) obtained by subtracting the current I of the second pixel 201b from the current I of the first pixel 201a. In the first acquisition mode, the control unit 30 acquires the first combined corresponding value for each unit pixel group 250. In the first acquisition mode, the third output corresponding value is not used.

[0074] The imaging device 1 has a shooting mode (also called the first shooting mode) in which the control unit 30 operates in a first acquisition mode. In the first shooting mode, the control unit 30 uses multiple first composite corresponding values ​​acquired for each of the multiple unit pixel groups 250 as multiple pixel values ​​that constitute the image. The image captured by the imaging device 1 in the first shooting mode is composed of multiple first composite corresponding values ​​corresponding to each of the multiple unit pixel groups 250. Hereafter, the image captured in the first shooting mode will be called the first image. Also, each pixel value that constitutes the first image will be called the first image pixel value. In this example, the first composite corresponding value corresponding to the combination of the output of the first pixel 201a and the output of the second pixel 201b is adopted as the first image pixel value.

[0075] In the second acquisition mode, the control unit 30 acquires the second combined value for each unit pixel group 250 by combining the output corresponding values ​​of the first pixel 201a and the third pixel 201c included in the unit pixel group 250. This value corresponds to the combined output of the first pixel 201a and the third pixel 201c. For example, the control unit 30 subtracts the third output corresponding value from the first output corresponding value to obtain the second combined value. The second combined value can be said to be a value corresponding to the combined output obtained by subtracting the output of the third pixel 201c from the output of the first pixel 201a. In other words, the second combined value can be said to be a value corresponding to the combined current obtained by subtracting the current I of the third pixel 201c from the current I of the first pixel 201a. In the second acquisition mode, the control unit 30 acquires the second combined value for each unit pixel group 250. The second output corresponding value is not used in the second acquisition mode.

[0076] The imaging device 1 has a shooting mode (also called the second shooting mode) in which the control unit 30 operates in the second acquisition mode. In the second shooting mode, the control unit 30 uses multiple second composite corresponding values ​​acquired for each of the multiple unit pixel groups 250 as multiple pixel values ​​that constitute the image. The image captured by the imaging device 1 in the second shooting mode is composed of multiple second composite corresponding values ​​corresponding to each of the multiple unit pixel groups 250. Hereafter, the image captured in the second shooting mode will be called the second image. Also, each pixel value that constitutes the second image will be called the second image pixel value. In this example, the second composite corresponding value corresponding to the combination of the output of the first pixel 201a and the output of the third pixel 201c is adopted as the second image pixel value.

[0077] In the third acquisition mode, the control unit 30 acquires a third composite value for each unit pixel group 250 by combining the output corresponding values ​​of the second pixel 201b and the third pixel 201c included in that unit pixel group 250. This value corresponds to the combined output of the second pixel 201b and the third pixel 201c. For example, the control unit 30 subtracts the third output corresponding value from the second output corresponding value to determine the third composite value. The third composite value can be said to correspond to the combined output obtained by subtracting the output of the third pixel 201c from the output of the second pixel 201b. In other words, the third composite value can be said to correspond to the combined current obtained by subtracting the current I of the third pixel 201c from the current I of the second pixel 201b. In the third acquisition mode, the control unit 30 acquires a third composite value for each unit pixel group 250. In the third acquisition mode, the first output corresponding value is not used.

[0078] The imaging device 1 has a shooting mode (also called the third shooting mode) in which the control unit 30 operates in the third acquisition mode. In the third shooting mode, the control unit 30 uses multiple third composite corresponding values ​​acquired for each of the multiple unit pixel groups 250 as multiple pixel values ​​that constitute the image. The image captured by the imaging device 1 in the third shooting mode is composed of multiple third composite corresponding values ​​corresponding to each of the multiple unit pixel groups 250. Hereafter, the image captured in the third shooting mode will be called the third image. Also, each pixel value that constitutes the third image will be called the third image pixel value. In this example, the third composite corresponding value corresponding to the combination of the output of the second pixel 201b and the output of the third pixel 201c is adopted as the third image pixel value.

[0079] In the fourth acquisition mode, the control unit 30, for each unit pixel group 250, combines the output corresponding values ​​of the first pixel 201a, the second pixel 201b, and the third pixel 201c included in the unit pixel group 250 to obtain a fourth combined corresponding value corresponding to the combination of the output of the first pixel 201a, the output of the second pixel 201b, and the output of the third pixel 201c. For example, the control unit 30 takes the value obtained by adding the second output corresponding value to the first output corresponding value and subtracting twice the value of the third output corresponding value as the fourth combined corresponding value. The fourth combined corresponding value can be said to be a value corresponding to the combined output obtained by adding the output of the second pixel 201b to the output of the first pixel 201a and subtracting twice the output of the third pixel 201c. In other words, the fourth composite value is a value corresponding to the composite current obtained by adding the current I of the second pixel 201b to the current I of the first pixel 201a, and then subtracting twice the current I of the third pixel 201c. In the fourth acquisition mode, the control unit 30 acquires the fourth composite value for each unit pixel group 250.

[0080] The imaging device 1 has a shooting mode (also called the fourth shooting mode) in which the control unit 30 operates in the fourth acquisition mode. In the fourth shooting mode, the control unit 30 uses multiple fourth composite corresponding values ​​acquired for each of the multiple unit pixel groups 250 as multiple pixel values ​​that constitute the image. The image captured by the imaging device 1 in the fourth shooting mode is composed of multiple fourth composite corresponding values ​​corresponding to each of the multiple unit pixel groups 250. Hereafter, the image captured in the fourth shooting mode will be called the fourth image. Also, each pixel value that constitutes the fourth image will be called the fourth image pixel value. In this example, the fourth composite corresponding value corresponding to the combination of the output of the first pixel 201a, the output of the second pixel 201b, and the output of the third pixel 201c is adopted as the fourth image pixel value.

[0081] Figure 14 is a schematic diagram summarizing examples of the first composite correspondence value (in other words, the first image pixel value), the second composite correspondence value (in other words, the second image pixel value), the third composite correspondence value (in other words, the third image pixel value), and the fourth composite correspondence value (in other words, the fourth image pixel value). Hereafter, when there is no need to distinguish between the first image pixel value, the second image pixel value, the third image pixel value, and the fourth image pixel value, they will simply be referred to as image pixel values.

[0082] <Characteristics of each composite value> The dark current components included in the output values ​​of the multiple pixels 201 of the image sensor 2 are approximately the same. Furthermore, in the unit pixel group 250, since the first pixel 201a and the second pixel 201b are located close to each other, when the image sensor 2 detects light, the near-infrared light detection component in the first output value and the near-infrared light detection component in the second output value are approximately the same. Also, when the image sensor 2 detects light, the visible light detection component in the first output value is larger than the visible light detection component in the second output value.

[0083] The first image pixel value (in other words, the first composite corresponding value) is obtained by subtracting the second output corresponding value from the first output corresponding value. Therefore, when the image sensor 2 detects light, the first image pixel value contains almost no dark current component or near-infrared light detection component, and the visible light detection component is dominant in the first image pixel value. Furthermore, the visible light detection component included in the first image pixel value is smaller than the visible light detection component included in the first output corresponding value. In this example, for the sake of explanation, the visible light detection component included in the first image pixel value may be represented as the visible light detection component (small).

[0084] The second image pixel value (in other words, the second composite correspondence value) is obtained by subtracting the third output correspondence value, which is dominated by the dark current component, from the first output correspondence value. Therefore, the second image pixel value contains almost no dark current component and includes both the visible light detection component and the near-infrared light detection component. The visible light detection component included in the second image pixel value is about the same as the visible light detection component included in the first output correspondence value, and the near-infrared light detection component included in the second image pixel value is about the same as the near-infrared light detection component included in the first output correspondence value. In this example, for the sake of explanation, the visible light detection component and the near-infrared detection component included in the second image pixel value may be represented as the visible light detection component (medium) and the near-infrared light detection component (small), respectively.

[0085] The third image pixel value is obtained by subtracting the third output corresponding value from the second output corresponding value. Therefore, the third image pixel value contains almost no dark current component and includes both a visible light detection component and a near-infrared light detection component. The visible light detection component included in the third image pixel value is about the same as the visible light detection component included in the second output corresponding value, and the near-infrared light detection component included in the third image pixel value is about the same as the near-infrared light detection component included in the second output corresponding value. In this example, for the sake of explanation, the visible light detection component and the near-infrared detection component included in the third image pixel value may be represented as the visible light detection component (small) and the near-infrared light detection component (small), respectively.

[0086] The fourth image pixel value is obtained by adding the second output corresponding value to the first output corresponding value and then subtracting twice the value of the third output corresponding value. Therefore, the fourth image pixel value contains almost no dark current component and includes both a visible light detection component and a near-infrared light detection component. The visible light detection component included in the fourth image pixel value is larger than the visible light detection component included in the first output corresponding value, and the near-infrared light detection component included in the fourth image pixel value is larger than the near-infrared light detection components included in both the first and second output corresponding values. In this example, for the sake of explanation, the visible light detection component and the near-infrared detection component included in the fourth image pixel value may be represented as the visible light detection component (large) and the near-infrared light detection component (medium), respectively.

[0087] Figure 15 is a schematic diagram summarizing the components contained in the first, second, third, and fourth image pixel values. Since the first, second, third, and fourth image pixel values ​​contain almost no dark current components, the dark current components are not shown in the columns for the first, second, third, and fourth image pixel values ​​in Figure 15. Also, since the near-infrared light detection component is almost absent in the first image pixel value, the near-infrared light detection component is not shown in the column for the first image pixel value in Figure 15.

[0088] <Switching shooting modes> The control unit 30 of the processing unit 3 can, for example, switch the shooting mode of the imaging device 1 in response to instructions from the user. For example, when the input unit 5 receives an instruction input from the user to operate in the first shooting mode, the control unit 30 sets the operating mode of the imaging device 1 to the first shooting mode. When the input unit 5 receives an instruction input from the user to operate in the second shooting mode, the control unit 30 sets the operating mode of the imaging device 1 to the second shooting mode. When the input unit 5 receives an instruction input from the user to operate in the third shooting mode, the control unit 30 sets the operating mode of the imaging device 1 to the third shooting mode. And when the input unit 5 receives an instruction input from the user to operate in the fourth shooting mode, the control unit 30 sets the operating mode of the imaging device 1 to the fourth shooting mode.

[0089] In the first shooting mode, when the input unit 5 receives a shooting instruction from the user, the control unit 30 controls the acquisition unit 32 to acquire the output corresponding value for each pixel. The user's shooting instruction may be, for example, an operation on the shutter button included in the input unit 5.

[0090] The control unit 30 generates multiple first image pixel values ​​corresponding to multiple unit pixel groups 250 based on the multiple output corresponding values ​​acquired. The control unit 30 then displays a first image composed of the generated multiple first image pixel values ​​on the display unit 4. Since the visible light detection component is dominant in the first image pixel values, the first image can be said to be equivalent to a visible light image obtained when the image sensor detects only visible light. The first shooting mode can be said to be a shooting mode suitable for shooting in bright shooting environments, for example.

[0091] In the second shooting mode, when the input unit 5 receives a shooting instruction from the user, the control unit 30 controls the acquisition unit 32 to acquire the output corresponding value for each pixel. Based on the acquired output corresponding values, the control unit 30 generates a plurality of second image pixel values ​​corresponding to a plurality of unit pixel groups 250. The control unit 30 then displays the second image, composed of the generated plurality of second image pixel values, on the display unit 4.

[0092] When the image sensor 2 detects light, each pixel value of the second image that makes up the second image contains a certain amount of near-infrared light detection component. The second shooting mode can be said to be a shooting mode suitable for shooting in slightly dark shooting environments, for example.

[0093] In the fourth shooting mode, when the input unit 5 receives a shooting instruction from the user, the control unit 30 controls the acquisition unit 32 to acquire the output corresponding value for each pixel. Based on the acquired output corresponding values, the control unit 30 generates a plurality of fourth image pixel values ​​corresponding to a plurality of unit pixel groups 250. The control unit 30 then displays the fourth image, composed of the plurality of generated fourth image pixel values, on the display unit 4. When the image sensor 2 detects light, each fourth image pixel value constituting the fourth image contains a certain amount of near-infrared light detection component. If the light detected by the image sensor 2 in the fourth shooting mode is the same as the light detected by the image sensor 2 in the second shooting mode, the near-infrared light detection component included in the fourth image pixel value will be larger than the near-infrared light detection component included in the second image pixel value. The fourth shooting mode can be said to be a shooting mode suitable for shooting in, for example, a moderately dark shooting environment.

[0094] In the third shooting mode, when the input unit 5 receives a shooting instruction from the user, the control unit 30 controls the acquisition unit 32 to acquire the output corresponding value for each pixel. Based on the acquired output corresponding values, the control unit 30 generates multiple third image pixel values ​​corresponding to multiple unit pixel groups 250. The control unit 30 then displays the third image, composed of the generated multiple third image pixel values, on the display unit 4. When the image sensor 2 detects light, each third image pixel value constituting the third image contains a certain amount of near-infrared light detection component. If the light detected by the image sensor 2 in the third shooting mode and the light detected by the image sensor 2 in the fourth shooting mode are the same, the ratio of the near-infrared light detection component to the visible light detection component in the third image pixel value will be greater than the ratio of the near-infrared light detection component to the visible light detection component in the fourth image pixel value. The third shooting mode can be said to be a shooting mode suitable for shooting in very dark shooting environments, for example. The third image can be said to be closer to a near-infrared image obtained when the image sensor detects only near-infrared light, compared to the first, second, and fourth images.

[0095] The user may specify the shooting mode of the imaging device 1 according to the brightness of the shooting environment. For example, if the shooting environment is bright, the user may instruct the input unit 5 to operate in the first shooting mode. Also, for example, if the shooting environment is slightly dark, the user may instruct the input unit 5 to operate in the second shooting mode. Also, for example, if the shooting environment is moderately dark, the user may instruct the input unit 5 to operate in the fourth shooting mode. Also, for example, if the shooting environment is very dark, the user may instruct the input unit 5 to operate in the third shooting mode.

[0096] In the above example, the user specifies the shooting mode according to the brightness of the shooting environment, but the user may also specify the shooting mode based on other criteria. For example, given that water can absorb near-infrared light, if the user wants to photograph a water droplet with the imaging device 1, the user may instruct the input unit 5 to operate in a third shooting mode that produces a third image relatively close to a near-infrared image. Alternatively, if the user wants to check the contents of a bag, the user may instruct the input unit 5 to operate in a third shooting mode that produces a third image relatively close to a near-infrared image.

[0097] The imaging device 1 may automatically switch the shooting mode according to the brightness of the shooting environment. Since switching the shooting mode can also be viewed as switching the acquisition mode, the imaging device 1 may automatically switch the acquisition mode according to the brightness of the shooting environment.

[0098] For example, when the input unit 5 receives a shooting instruction from the user, the control unit 30 first sets the shooting mode to a first shooting mode in which a first image equivalent to a visible light image can be obtained. The control unit 30 then compares the maximum value (also called the maximum first image pixel value) among the multiple first image pixel values ​​that make up the first image captured in the first shooting mode with a first threshold value. The maximum first image pixel value indicates the brightness of the shooting environment. If the maximum first image pixel value is greater than the first threshold value, the control unit 30 determines that the shooting environment is bright and maintains the first shooting mode. The control unit 30 then displays the first image captured in the first shooting mode on the display unit 4.

[0099] On the other hand, if the maximum first image pixel value is less than or equal to the first threshold, the control unit 30 compares the maximum first image pixel value with a second threshold that is smaller than the first threshold. If the maximum first image pixel value is greater than the second threshold but less than or equal to the first threshold, the control unit 30 determines that the shooting environment is slightly dark and changes the shooting mode from the first shooting mode to the second shooting mode. The control unit 30 then displays the second image captured in the second shooting mode on the display unit 4. In this case, both the first and second images are captured in response to the user's shooting instruction, but the first image is discarded, for example, without being displayed on the display unit 4.

[0100] If the maximum first image pixel value is less than or equal to the second threshold, the control unit 30 compares the maximum first image pixel value with a third threshold that is smaller than the second threshold. If the maximum first image pixel value is greater than the third threshold but less than or equal to the second threshold, the control unit 30 determines that the shooting environment is moderately dark and changes the shooting mode from the first shooting mode to the fourth shooting mode. The control unit 30 then displays the fourth image, which was captured in the fourth shooting mode, on the display unit 4. In this case, both the first and fourth images are captured in response to the user's shooting instruction, but the first image is discarded, for example, without being displayed on the display unit 4.

[0101] If the maximum first image pixel value is below the third threshold, the control unit 30 determines that the shooting environment is quite dark and changes the shooting mode from the first shooting mode to the third shooting mode. The control unit 30 then displays the third image, which was captured in the third shooting mode, on the display unit 4. In this case, both the first and third images are captured in response to the user's shooting instruction, but the first image is not displayed on the display unit 4 and is, for example, discarded.

[0102] As described above, the multiple pixels 201 of the image sensor 2 include a first pixel 201a that detects visible light and near-infrared light, and pixels 201 such as the second pixel 201b and third pixel 201c that have a lower visible light detection sensitivity than the first pixel 201a. This allows the processing unit 3 to perform processing using the output of the first pixel 201a and the output of the pixels 201 that have a lower visible light detection sensitivity than the first pixel 201a. For example, the processing unit 3 can obtain a first combined value corresponding to the combination of the output of the second pixel 201b, which has a lower visible light detection sensitivity than the first pixel 201a, and the output of the first pixel 201a, and use the obtained first combined value as the pixel value of the image. Furthermore, the processing unit 3 can obtain a second combined value corresponding to the combination of the output of the third pixel 201c, which has a lower visible light detection sensitivity than the first pixel 201a, and the output of the first pixel 201a, and use the obtained second combined value as the pixel value of the image. Thus, the convenience of the imaging device 1 is improved.

[0103] Furthermore, as in the example above, if the image sensor 2 has multiple pixels 201, including a first pixel 201a, a second pixel 201b, and a third pixel 201c, the processing unit 3 can perform processing using the output of the first pixel 201a, the output of the second pixel 201b, and the output of the third pixel 201c. For example, the processing unit 3 can acquire not only the first and second composite values, but also a third composite value corresponding to the combination of the output of the second pixel 201b and the output of the third pixel 201c, and use the acquired third composite value as the pixel value of the image. In addition, the processing unit 3 can acquire a fourth composite value corresponding to the combination of the output of the first pixel 201a, the output of the second pixel 201b, and the output of the third pixel 201c, and use the acquired fourth composite value as the pixel value of the image. Thus, the convenience of the imaging device 1 is improved.

[0104] Furthermore, as can be understood from the above explanation, the effect of the dark current of pixel 201 is almost invisible in the first, second, third, and fourth images, so that high-quality images can be obtained in each shooting mode.

[0105] Furthermore, if the imaging device 1 automatically switches the shooting mode, in other words, if the imaging device 1 automatically switches the acquisition mode for use, the convenience of the imaging device 1 is improved.

[0106] The imaging device 1 may include a shooting mode in which composite corresponding values ​​are not acquired. For example, the imaging device 1 may include a fifth shooting mode in which the first output corresponding values ​​of a plurality of first pixels 201a are used as the multiple pixel values ​​that constitute the image. In the fifth shooting mode, an image composed of a plurality of first output corresponding values ​​(also called the fifth image) is obtained. The imaging device 1 may also include a sixth shooting mode in which the second output corresponding values ​​of a plurality of second pixels 201b are used as the multiple pixel values ​​that constitute the image. In the sixth shooting mode, an image composed of a plurality of second output corresponding values ​​(also called the sixth image) is obtained.

[0107] In the example above, each of the multiple unit pixel groups 250 has a third pixel 201c, but some of the multiple unit pixel groups 250 do not have a third pixel 201c. In other words, the multiple unit pixel groups 250 may include unit pixel groups 250 that have a third pixel 201c (also called the first unit pixel group 250) and unit pixel groups 250 that do not have a third pixel 201c (also called the second unit pixel group 250). In this case, the second image pixel value, third image pixel value, and fourth image pixel value corresponding to the second unit pixel group 250 may be generated, for example, by using the third output corresponding value of the third pixel 201c included in the first unit pixel group 250 that is closest to the second unit pixel group 250.

[0108] Furthermore, the image sensor 2 may have only one third pixel 201c. In this case, the single third pixel 201c may be located at a position separate from the multiple unit pixel groups 250. The second image pixel value, third image pixel value, and fourth image pixel value corresponding to each unit pixel group 250 may be generated using the third output corresponding value of the single third pixel 201c.

[0109] Furthermore, the variation in dark current among multiple pixels 201 located close to each other tends to be small. Therefore, as shown in the example in Figure 12, by having each unit pixel group 250 include a third pixel 201c, the dark current components included in the second image pixel value, third image pixel value, and fourth image pixel value can be made very small.

[0110] As the temperature of the imaging device 1 increases, the output of each pixel 201, i.e., the current I, increases, resulting in a larger output value for each pixel 201. Consequently, as the temperature of the imaging device 1 increases, the image pixel value increases. Therefore, even if the shooting environment remains constant, the image quality of the images captured by the imaging device 1 may change in response to changes in the temperature of the imaging device 1.

[0111] On the other hand, the dark current of pixel 201 changes depending on the temperature of the imaging device 1. Specifically, the dark current of pixel 201 increases as the temperature of the imaging device 1 increases. Since the third pixel 201c hardly detects visible light and near-infrared light, it can be said that the current I of the third pixel 201c is composed almost entirely of dark current. Therefore, the third pixel 201c can be used as a temperature sensor.

[0112] Therefore, the control unit 30 may correct images based on the output of the image sensor 2, such as the first image and the second image, based on the output of the third pixel 201c. This makes it less likely for the image quality to change in response to temperature changes in the imaging device 1. The control unit 30 displays the corrected image on the display unit 4.

[0113] For example, consider the case where each unit pixel group 250 includes a third pixel 201c, as shown in the example in Figure 12. In this case, the control unit 30 may correct the image composed of multiple image pixel values ​​by decreasing the corresponding image pixel value for each unit pixel group 250 as the third output corresponding value of the third pixel 201c included in that unit pixel group 250 increases. As a result, when the temperature of the imaging device 1 increases, the image pixel values ​​decrease, making it less likely for the image quality to change in response to temperature changes in the imaging device 1. Alternatively, the control unit 30 may calculate the average value of multiple third output corresponding values ​​and correct the image composed of multiple image pixel values ​​by decreasing the image pixel value as the average value increases. Furthermore, if the image sensor 2 includes one third pixel 201c, the image may be corrected by decreasing the image pixel value as the output corresponding value of that one third pixel 201c increases.

[0114] Furthermore, the control unit 30 may adjust the shooting conditions based on the output of the third pixel 201c. For example, the control unit 30 may adjust the exposure time T included in the shooting conditions, or adjust the voltage VR included in the shooting conditions, based on the third output corresponding value of the third pixel 201c. This makes it less likely for the image quality to change in response to temperature changes in the imaging device 1, similar to when the image is corrected based on the output of the third pixel 201c.

[0115] For example, consider the case where the image sensor 2 has multiple third pixels 201c, as shown in the example in Figure 12. In this case, the control unit 30 calculates the average value of multiple third output corresponding values ​​each time the input unit 5 receives a shooting instruction from the user. If the newly calculated average value in response to the shooting instruction from the user is different from the previous average value, the control unit 30 changes the exposure time T setting. The newly calculated average value represents the current temperature of the imaging device 1, and the previous average value represents the past temperature of the imaging device 1. If the newly calculated average value is greater than the previous average value, that is, if the temperature of the imaging device 1 is rising, the control unit 30 decreases the exposure time T setting. As a result, if the temperature of the imaging device 1 is rising, the value of each image pixel is reduced. On the other hand, if the newly calculated average value is smaller than the previous average value, that is, if the temperature of the imaging device 1 is falling, the control unit 30 increases the exposure time T setting. As a result, if the temperature of the imaging device 1 is falling, the value of each image pixel is increased. The control unit 30 displays the image obtained after changing the exposure time T setting on the display unit 4.

[0116] Furthermore, if the newly calculated average value differs from the previous average value, the control unit 30 may change the setting value of the voltage VR through the voltage generation unit 35. In this case, if the newly calculated average value is greater than the previous average value, the control unit 30 decreases the setting value of the voltage VR. When the setting value of the voltage VR decreases (in other words, when the reverse voltage of the diode 211 decreases), the current I of each pixel 201 decreases. Therefore, if the temperature of the imaging device 1 rises, the value of each image pixel decreases. On the other hand, if the newly calculated average value is less than the previous average value, the control unit 30 increases the setting value of the voltage VR. When the setting value of the voltage VR increases, the current I of each pixel 201 increases. Therefore, if the temperature of the imaging device 1 decreases, the value of each image pixel increases. The control unit 30 displays the image obtained after changing the setting value of the voltage VR on the display unit 4.

[0117] Furthermore, if the image sensor 2 has one third pixel 201c, the control unit 30 may use the third output corresponding value of that one third pixel 201c instead of the above average value to adjust the exposure time T, adjust the voltage VR, or adjust the voltage.

[0118] The imaging device 1 may function as a near-infrared light sensor (in other words, a near-infrared sensor) that detects near-infrared light. In this case, the control unit 30 may compare the first, second, and fourth composite values ​​and determine that near-infrared light has been detected when the third composite value, which has a large ratio of near-infrared light detection component to visible light detection component, is greater than or equal to a threshold value. In this case, for example, the imaging device 1 may have only a fifth shooting mode as a shooting mode, and the multiple pixels 201 may include only one set of pixels consisting of one second pixel 201b and one third pixel 201c, in addition to multiple first pixels 201a. The control unit 30 may then acquire a third composite value corresponding to the combination of the output of the one second pixel 201b and the output of the one third pixel 201c, and determine the detection of near-infrared light based on the acquired third composite value. In this case, the imaging device 1 may also detect near-infrared light emitted by a remote controller (abbreviated as remote control) for operating the imaging device 1.

[0119] In the example above, the processing unit 3 obtains a combined corresponding value corresponding to the combined output of multiple pixels 201 by combining the corresponding output values ​​of multiple pixels 201. However, the combined corresponding value may be obtained by other methods. Figure 16 is a schematic diagram showing an example of the configuration of the processing unit 3 (also called processing unit 3A) that obtains the combined corresponding value by a method different from that described above.

[0120] As shown in Figure 16, the processing unit 3A includes an acquisition unit 38 that acquires the first composite corresponding value, the second composite corresponding value, and the third composite corresponding value, instead of the acquisition unit 32. The acquisition unit 38 is implemented, for example, by a hardware circuit that does not require software to realize its function. The acquisition unit 38 can also be called an acquisition circuit.

[0121] In the first shooting mode (in other words, the first acquisition mode), the acquisition unit 38 acquires a first composite pixel value for each unit pixel group 250, corresponding to the combination of the outputs of the first pixel 201a and the second pixel 201b included in the unit pixel group 250. In the second shooting mode (in other words, the second acquisition mode), the acquisition unit 38 acquires a second composite pixel value for each unit pixel group 250, corresponding to the combination of the outputs of the first pixel 201a and the third pixel 201c included in the unit pixel group 250. In the third shooting mode (in other words, the third acquisition mode), the acquisition unit 38 acquires a third composite pixel value for each unit pixel group 250, corresponding to the combination of the outputs of the second pixel 201b and the third pixel 201c included in the unit pixel group 250. In the fourth shooting mode (in other words, the fourth acquisition mode), the acquisition unit 38 acquires a fourth composite pixel value for each unit pixel group 250, corresponding to the combination of the outputs of the first pixel 201a, second pixel 201b, and third pixel 201c included in the unit pixel group 250.

[0122] The acquisition unit 38 includes, for each of the multiple unit pixel groups 250, an output synthesis unit 385 and a unit pixel group corresponding acquisition unit 380 corresponding to the unit pixel group 250. Figure 17 is a schematic diagram showing an example of the configuration of the unit pixel group 250 and the output synthesis unit 385 and unit pixel group corresponding acquisition unit 380 corresponding to the unit pixel group 250.

[0123] The unit pixel group corresponding acquisition unit 380 has the same configuration as the pixel corresponding acquisition unit 320 described above. In this example, one unit pixel group 250 is arranged at the intersection where one row signal line 202 and one column signal line 203 intersect. The one row signal line 202 and the one column signal line 203 are connected to the unit pixel group corresponding acquisition unit 380 that corresponds to the unit pixel group 250.

[0124] The first selection switch 322 and the second selection switch 323 of the unit pixel group corresponding acquisition unit 380 are for selecting the unit pixel group 250. An output combining unit 385 is connected to one end of the first selection switch 322.

[0125] The output combining unit 385 combines the outputs of at least two pixels 201 of the first pixel 201a, second pixel 201b, and third pixel 201c that constitute the unit pixel group 250. Specifically, the output combining unit 385 combines the currents I of at least two pixels 201 of the first pixel 201a, second pixel 201b, and third pixel 201c that constitute the unit pixel group 250 to generate a combined current. The operation of the output combining unit 385 is controlled by the control unit 30. The combined current generated by the output combining unit 385 is output to one end of the first selection switch 322. The output combining unit 385 is implemented, for example, by a hardware circuit that does not require software to realize its function. The output combining unit 385 can also be called, for example, an output combining circuit. Hereafter, the currents I of the first pixel 201a, second pixel 201b, and third pixel 201c may be referred to as the first current I1, second current I2, and third current I3, respectively.

[0126] In the first shooting mode (in other words, the first acquisition mode), the output combining unit 385 generates a first combined current obtained by subtracting the second current I2 from the first current I1, and outputs it to the first selection switch 322. The first combined current can also be described as a combined output obtained by combining the output of the first pixel 201a and the output of the second pixel 201b.

[0127] In the second shooting mode (in other words, the second acquisition mode), the output combining unit 385 generates a second combined current obtained by subtracting the third current I3 from the first current I1, and outputs it to the first selection switch 322. The second combined current can also be described as a combined output obtained by combining the output of the first pixel 201a and the output of the third pixel 201c.

[0128] In the third shooting mode (or third acquisition mode), the output combining unit 385 generates a third combined current obtained by subtracting the third current I3 from the second current I2, and outputs it to the first selection switch 322. The third combined current can also be described as a combined output obtained by combining the output of the second pixel 201b and the output of the third pixel 201c.

[0129] In the fourth shooting mode (in other words, the fourth acquisition mode), the output combining unit 385 generates a fourth combined current by adding the second current I2 to the first current I1 and subtracting twice the current of the third current I3 from the result, and outputs this fourth combined current to the first selection switch 322. The fourth combined current can also be described as a combined output obtained by combining the output of the first pixel 201a, the output of the second pixel 201b, and the output of the third pixel 201c.

[0130] The output combining unit 385 can generate a first combined current, a second combined current, a third combined current, and a fourth combined current using, for example, an analog circuit including a current mirror circuit.

[0131] In the first shooting mode, when the input unit 5 receives a shooting command from the user, the control unit 30 turns on the reset switch 325 and the exposure time setting switch 324 of the unit pixel group corresponding acquisition unit 380. After a predetermined time has elapsed, the control unit 30 turns off the reset switch 325. This initializes the capacitor 327 and other components. If the exposure time setting switch 324 remains on and the reset switch 325 is turned off, the first combined current output by the output combined unit 385 begins to flow to the capacitor 327, and the accumulation of charge in the capacitor 327 begins. After the accumulation of charge in the capacitor 327 has begun and a predetermined time has elapsed, the control unit 30 turns off the exposure time setting switch 324. After the exposure time setting switch 324 is turned off, the AD converter 33 of the processing unit 3 converts the voltage at one end of the capacitor 327 from analog to digital and outputs it to the control unit 30. The voltage value at one end of the capacitor 327 is a value corresponding to the first combined current, that is, a first combined value corresponding to the combination of the output of the first pixel 201a and the output of the second pixel 201b, and this first combined value is input to the control unit 30.

[0132] As described above, the control unit 30 acquires a first composite correspondence value for each unit pixel group 250. Then, the control unit 30 uses the acquired multiple first composite correspondence values ​​as multiple first image pixel values ​​that constitute the first image.

[0133] In the second shooting mode, when the input unit 5 receives a shooting command from the user, the reset switch 325 and the exposure time setting switch 324 are controlled in the same way as in the first shooting mode, and the second composite current output by the output composite unit 385 flows to the capacitor 327. The voltage value at one end of the capacitor 327 becomes the second composite corresponding value corresponding to the composite of the output of the first pixel 201a and the output of the third pixel 201c. The control unit 30 uses the multiple second composite corresponding values ​​acquired by the multiple unit pixel group corresponding acquisition units 380 of the acquisition unit 38 as the multiple second image pixel values ​​that constitute the second image.

[0134] In the third shooting mode, when the input unit 5 receives a shooting command from the user, the reset switch 325 and the exposure time setting switch 324 are controlled in the same way as in the first shooting mode, and the third composite current output by the output composite unit 385 flows to the capacitor 327. The voltage value at one end of the capacitor 327 becomes the third composite corresponding value corresponding to the composite of the output of the first pixel 201a and the output of the third pixel 201c. The control unit 30 uses the multiple third composite corresponding values ​​acquired by the multiple unit pixel group corresponding acquisition units 380 of the acquisition unit 38 as the multiple third image pixel values ​​that constitute the third image.

[0135] In the fourth shooting mode, when the input unit 5 receives a shooting command from the user, the reset switch 325 and the exposure time setting switch 324 are controlled in the same way as in the first shooting mode, and the fourth composite current output by the output composite unit 385 flows to the capacitor 327. The voltage value at one end of the capacitor 327 becomes the fourth composite corresponding value corresponding to the composite of the output of the first pixel 201a, the output of the second pixel 201b, and the output of the third pixel 201c. The control unit 30 uses the multiple fourth composite corresponding values ​​acquired by the multiple unit pixel group corresponding acquisition units 380 of the acquisition unit 38 as the multiple fourth image pixel values ​​that constitute the fourth image.

[0136] The output combining unit 385 may output the first current I1 of the first pixel 201a directly to the first selection switch 322. For example, if the imaging device 1 has the fifth shooting mode described above, in the fifth shooting mode, the output combining unit 385 outputs the first current I1 directly to the first selection switch 322. In this case, the configuration shown in Figure 17 is equivalent to the configuration shown in Figure 4 described above, and the voltage value at one end of the capacitor 327 becomes the value corresponding to the first output.

[0137] Furthermore, the output combining unit 385 may output the second current I2 of the second pixel 201b directly to the first selection switch 322. For example, if the imaging device 1 has the sixth shooting mode described above, in the sixth shooting mode, the output combining unit 385 outputs the second current I2 directly to the first selection switch 322. In this case, the configuration shown in Figure 17 is equivalent to the configuration shown in Figure 4 described above, and the voltage value at one end of the capacitor 327 becomes the value corresponding to the second output.

[0138] Furthermore, it is not necessary for any one of the first pixel 201a, the second pixel 201b, and the third pixel 201c to be connected to the output combining unit 385. For example, consider the case where the third pixel 201c is not output to the output combining unit 385. In this case, as shown in Figure 18, the third pixel 201c is connected to the pixel correspondence acquisition unit 320 described above. The pixel correspondence acquisition unit 320 to which the third pixel 201c is connected outputs a third output correspondence value corresponding to the output of the third pixel 201c. When the output combining unit 385 to which the first pixel 201a and the second pixel 201b are connected outputs a first current I1, the unit pixel group correspondence acquisition unit 380 outputs a first output correspondence value corresponding to the output of the first pixel 201a. The control unit 30 can acquire a second combined correspondence value by combining the first output correspondence value from the unit pixel group correspondence acquisition unit 380 and the third output correspondence value from the pixel correspondence acquisition unit 320. Furthermore, when the output combining unit 385 outputs a second current I2, the unit pixel group corresponding acquisition unit 380 outputs a second output corresponding value corresponding to the output of the second pixel 201b. The control unit 30 can combine the second output corresponding value from the unit pixel group corresponding acquisition unit 380 and the third output corresponding value from the pixel corresponding acquisition unit 320 to obtain a third combined corresponding value. Also, when the output combining unit 385 outputs a combined current obtained by adding the first current I1 and the second current I2, the unit pixel group corresponding acquisition unit 380 outputs a combined corresponding value corresponding to the combination of the output of the first pixel 201a and the output of the second pixel 201b. The control unit 30 can combine the combined corresponding value output from the unit pixel group corresponding acquisition unit 380, corresponding to the combination of the output of the first pixel 201a and the output of the second pixel 201b, and the third output corresponding value from the pixel corresponding acquisition unit 320 to obtain a fourth combined corresponding value.

[0139] Even if the first pixel 201a is not output to the output combining unit 385, the processing unit 3 can similarly obtain each combining value. Similarly, even if the second pixel 201b is not output to the output combining unit 385, the processing unit 3 can similarly obtain each combining value.

[0140] The imaging device 1 does not have to have a first shooting mode, a second shooting mode, a third shooting mode, or a fourth shooting mode. The imaging device 1 may have at least two shooting modes from the first shooting mode, the second shooting mode, the third shooting mode, and the fourth shooting mode. In other words, the imaging device 1 may have at least two acquisition modes from the first acquisition mode, the second acquisition mode, the third acquisition mode, and the fourth acquisition mode. Alternatively, the imaging device 1 may have only one shooting mode from the first shooting mode, the second shooting mode, the third shooting mode, and the fourth shooting mode. In other words, the imaging device 1 may have only one acquisition mode from the first acquisition mode, the second acquisition mode, the third acquisition mode, and the fourth acquisition mode.

[0141] The functions of the elements disclosed herein may be implemented using circuit configurations or processing circuit configurations, including general-purpose processors, dedicated processors, integrated circuits, ASICs ("Application-Specific Integrated Circuits"), conventional circuit configurations, and / or combinations thereof, which are configured to perform the disclosed elements or programmed to perform the disclosed functions. A processor is considered a processing circuit configuration or circuit configuration if it includes transistors and other circuit configurations within it. In this disclosure, a circuit configuration, unit, or means is hardware that performs the listed functions or hardware programmed to perform such functions. Hardware may be any hardware disclosed herein or other known hardware programmed to perform the listed functions or configured to perform such functions. When hardware is a processor which may be considered a certain type of circuit configuration, a circuit configuration, means, or unit is a combination of hardware and software, software used to configure the hardware, and / or a processor.

[0142] As described above, the imaging apparatus has been explained in detail, but the above explanation is illustrative in all respects, and this disclosure is not limited thereto. Furthermore, the various examples described above can be combined and applied insofar as they do not contradict each other. And it is understood that countless examples not illustrated can be conceived without falling outside the scope of this disclosure.

[0143] This disclosure includes the following aspects:

[0144] The imaging device according to the first embodiment comprises an image sensor having a plurality of pixels and a processing unit that processes the output of the image sensor, each of the plurality of pixels having a plurality of light-receiving elements, each of the plurality of light-receiving elements having a photoelectric conversion element whose current changes according to the amount of light detected by the light-receiving element, and the plurality of pixels include a first pixel that detects visible light and near-infrared light and a second pixel that has a lower detection sensitivity for visible light than the first pixel.

[0145] The imaging device according to the second embodiment is an imaging device according to the first embodiment, wherein the second pixel detects visible light and near-infrared light, and has a lower detection sensitivity for visible light than the first pixel.

[0146] The imaging device according to the third embodiment is the imaging device according to the first embodiment, wherein the second pixel has lower detection sensitivity for visible light and near-infrared light than the first pixel.

[0147] The imaging device according to the fourth embodiment is the imaging device according to the second embodiment, wherein the plurality of pixels include a third pixel having lower detection sensitivity for visible light and near-infrared light than the first pixel and the second pixel, respectively.

[0148] The imaging device according to the fifth embodiment is an imaging device according to any one of the first to fourth embodiments, wherein the processing unit acquires a value corresponding to the combination of the output of the first pixel and the output of the second pixel.

[0149] The imaging device according to the sixth embodiment is an imaging device according to the fourth embodiment, wherein the processing unit has at least two acquisition modes: a first acquisition mode for acquiring a value corresponding to the combination of the output of the first pixel and the output of the second pixel; a second acquisition mode for acquiring a value corresponding to the combination of the output of the first pixel and the output of the third pixel; a third acquisition mode for acquiring a value corresponding to the combination of the output of the second pixel and the output of the third pixel; and a fourth acquisition mode for acquiring a value corresponding to the combination of the output of the first pixel, the output of the second pixel, and the output of the third pixel.

[0150] The imaging device according to the seventh embodiment is an imaging device according to the fifth or sixth embodiment, wherein the plurality of pixels include a plurality of first pixels that detect visible light and near-infrared light, and a plurality of second pixels that correspond to each of the plurality of first pixels and have a lower detection sensitivity for visible light than the plurality of first pixels, and the processing unit acquires a value for each of the plurality of first pixels corresponding to the combination of the output of the first pixel and the output of the second pixel corresponding to the first pixel.

[0151] The imaging device according to the eighth embodiment is the imaging device according to the seventh embodiment, wherein the processing unit takes the value acquired for each of the plurality of first pixels as the pixel value of the image.

[0152] The imaging device according to the ninth embodiment is the imaging device according to the sixth embodiment, wherein the processing unit switches between the at least two acquisition modes depending on the brightness of the shooting environment.

[0153] The imaging device according to the tenth embodiment is an imaging device according to the third embodiment, wherein the processing unit corrects the image based on the output of the image sensor based on the output of the second pixel.

[0154] The imaging device according to the eleventh embodiment is an imaging device according to the third embodiment, wherein the processing unit adjusts the shooting conditions of the imaging device based on the output of the second pixel.

[0155] The imaging device according to the twelfth embodiment is the imaging device according to the eleventh embodiment, wherein the imaging conditions adjusted by the processing unit include the voltage supplied to the photoelectric conversion element.

[0156] The imaging device according to the 13th embodiment is an imaging device according to the 11th or 12th embodiment, wherein the shooting conditions adjusted by the processing unit include exposure time.

[0157] The processing unit according to the 14th embodiment is a processing unit provided in an imaging device according to any one of the first to 13th embodiments. [Explanation of Symbols]

[0158] 1. Imaging device 2 Image Sensor 3, 3A Processing Unit 201 pixels 201a Pixel 1 201b 2nd pixel 201c 3rd pixel 210 Photodetector 211 Photoelectric conversion element I current I1 1st current I2 2nd current I3 3rd current

Claims

1. An image sensor having multiple pixels, A processing unit that processes the output of the image sensor and Equipped with, Each of the aforementioned multiple pixels has multiple light-receiving elements, Each of the aforementioned plurality of light-receiving elements has a photoelectric conversion element whose current changes according to the amount of light detected by the light-receiving element. The aforementioned plurality of pixels are A first pixel that detects visible light and near-infrared light, A second pixel having lower visible light detection sensitivity than the first pixel, An imaging device, including an imaging device.

2. The imaging apparatus according to claim 1, The imaging device has a second pixel that detects visible light and near-infrared light, and has a lower detection sensitivity for visible light than the first pixel.

3. The imaging apparatus according to claim 1, The imaging device wherein the second pixel has lower detection sensitivity for visible light and near-infrared light than the first pixel.

4. The imaging apparatus according to claim 2, An imaging device in which the plurality of pixels include a third pixel having lower detection sensitivity for visible light and near-infrared light than the first and second pixels, respectively.

5. The imaging apparatus according to claim 1, The processing unit is an imaging device that acquires a value corresponding to the combination of the output of the first pixel and the output of the second pixel.

6. The imaging apparatus according to claim 4, The aforementioned processing unit, A first acquisition mode that acquires a value corresponding to the combination of the output of the first pixel and the output of the second pixel, A second acquisition mode that acquires a value corresponding to the combination of the output of the first pixel and the output of the third pixel, A third acquisition mode that acquires a value corresponding to the combination of the output of the second pixel and the output of the third pixel, A fourth acquisition mode that acquires a value corresponding to the combination of the output of the first pixel, the output of the second pixel, and the output of the third pixel. An imaging device having at least two acquisition modes.

7. The imaging apparatus according to claim 5, The aforementioned plurality of pixels are Multiple first pixels that detect visible light and near-infrared light, A plurality of second pixels, each corresponding to the plurality of first pixels, and having a lower visible light detection sensitivity than the plurality of first pixels. Includes, The processing unit is an imaging device that acquires a value for each of the plurality of first pixels corresponding to the combination of the output of the first pixel and the output of the second pixel corresponding to the first pixel.

8. The imaging apparatus according to claim 7, The processing unit is an imaging device that uses the values ​​obtained for each of the plurality of first pixels as the pixel values ​​of the image.

9. The imaging device according to claim 6, The processing unit is an imaging device that switches between the at least two acquisition modes depending on the brightness of the shooting environment.

10. The imaging apparatus according to claim 3, The processing unit is an imaging device that corrects an image based on the output of the image sensor based on the output of the second pixel.

11. The imaging apparatus according to claim 3, The processing unit adjusts the shooting conditions of the imaging device based on the output of the second pixel.

12. The imaging apparatus according to claim 11, An imaging device in which the imaging conditions adjusted by the processing unit include the voltage supplied to the photoelectric conversion element.

13. The imaging apparatus according to claim 11, An imaging device in which the shooting conditions adjusted by the processing unit include exposure time.

14. A processing unit comprising an imaging device according to any one of claims 1 to 13.

Images