Control device, imaging device, control method thereof, and program

Abstract

This technology can reduce the amount of blur in images. [Solution] The control device controls an imaging device that generates an image based on a count value obtained by counting pulses output for each pixel according to the light reception frequency of a plurality of photoelectric conversion units that convert photons into electrical signals, and comprises detection means for detecting a subject in the image, and control means for acquiring information on exposure time when the count value obtained by counting pulses output from each pixel in the region of the subject exceeds a count threshold, and executing a modification process to change the imaging settings so that the number of pixels with matching exposure times increases.

Description

Technical Field

[0001] The present invention relates to a control device, an imaging device, a control method thereof, and a program.

Background Art

[0002] In recent years, an imaging device has been proposed that digitally counts the number of pulses corresponding to the reception frequency of photons incident on an avalanche photodiode (hereinafter, APD) and outputs the count value from each pixel to generate an image.

[0003] In Patent Document 1, an imaging device is disclosed that includes an APD, a detection unit that detects an avalanche current, a switch disposed between the APD and the detection unit, and a reset unit that applies a predetermined potential to the input unit of the switch to reset the node between the switch and the detection unit. The technique of Patent Document 1 pauses the counting of the number of photons when the count value reaches a predetermined threshold at an exposure time shorter than the maximum exposure time, calculates an estimated value of the count, and generates an image. Here, the estimated value of the count is a count value obtained from the count value at an exposure time shorter than the maximum exposure time and estimating the number of photons assuming that the APD is exposed for the length of the maximum exposure time.

[0004] Thus, by using the photoelectric conversion technique disclosed in Patent Document 1, the influence of readout noise can be eliminated and the number of photons can be digitally counted. Therefore, even in a dark place, a small amount of light can be detected, and an image with a very wide dynamic range can be captured.

[0005] Further, Patent Document 2 discloses a technique for specifying the movement of a subject region where the subject is imaged and setting the same shutter speed to be applied to the pixels within the subject region in an imaging device capable of setting an exposure time for each pixel.

[0006] Furthermore, Patent Document 3 discloses a technique for an imaging device in which exposure time can be set for each pixel, in which image recognition is performed and the exposure time is changed for the subject area according to the content of the image recognition. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2021-019281 [Patent Document 2] Japanese Patent Publication No. 2020-053960 [Patent Document 3] Japanese Patent Publication No. 2024-012828 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, the technology described in Patent Document 1 results in a large amount of blur in the image of the subject, especially moving subjects, when there is a large difference in brightness within the subject area and the exposure time differs for each pixel within the subject area.

[0009] Furthermore, in the technology described in Patent Document 2, it is necessary to set the optimal exposure time for each pixel according to the brightness within the subject area. Therefore, when there is a large difference in brightness within the subject area, it is not possible to set the optimal exposure time, resulting in a large amount of blurring of the subject in the image.

[0010] Furthermore, in the technology described in Patent Document 3, simply setting the exposure time according to the content of the image recognition does not allow for setting the optimal exposure time, resulting in a large amount of blurring of the subject in the image.

[0011] Therefore, the present invention provides a technology that can reduce the amount of blur in an image. [Means for solving the problem]

[0012] To solve this problem, for example, the control device of the present invention has the following configuration. That is, A control device that controls an imaging device that generates an image based on a count value obtained by counting pulses output for each pixel according to the light reception frequency of a plurality of photoelectric conversion units that convert photons into electrical signals. Detection means for detecting a subject of an image; Control means for acquiring information regarding an exposure time when a count value obtained by counting pulses output from each pixel in the region of the subject exceeds a count threshold, and executing a change process for changing imaging settings so that the number of pixels with the same exposure time increases. It is provided with.

Effect of the Invention

[0013] According to the present invention, it is possible to reduce the amount of blur of a subject in an image.

Brief Description of the Drawings

[0014] [Figure 1] A block diagram showing the overall configuration of the imaging device of the embodiment. [Figure 2] A diagram showing the configuration of the photoelectric conversion element of the embodiment. [Figure 3] A diagram showing a configuration example of the sensor substrate of the embodiment. [[ID=二十九]] [Figure 4] A diagram showing a configuration example of the circuit board of the embodiment. [Figure 5] A diagram showing an equivalent circuit of the pixel and the signal processing circuit of the embodiment. [Figure 6] A timing chart for explaining the control signal and the operation of the photoelectric conversion element of the embodiment. [Figure 7] A timing chart for explaining the exposure time of the embodiment. [Figure 8] A diagram showing the relationship between the exposure time for each pixel included in the photoelectric conversion element of the embodiment and the count value of the counter circuit. [Figure 9] A table showing exposure time information, which is an example of the data format output as the exposure time of the embodiment. [Figure 10] A diagram showing the relationship between the exposure time for each pixel included in the photoelectric conversion unit of the embodiment and the count value of the counter circuit. [Figure 11]Block diagram showing the functional configuration of the imaging device according to the embodiment. [Figure 12] Flowchart showing the setting process of the imaging device according to the embodiment.

Mode for Carrying Out the Invention

[0015] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant explanations are omitted.

[0016] <Embodiment> Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of the imaging device according to the embodiment. Referring to FIG. 1, the overall configuration of the imaging device 100 in the embodiment will be described.

[0017] As shown in FIG. 1, the imaging device 100 includes an imaging optical system 101, an image pickup device 102, a CPU 103, a video output driving unit 104, a frame memory 106, a ROM 107, a RAM 108, an operation unit 109, a display driving unit 110, and an internal bus 112. The CPU 103, the frame memory 106, the ROM 107, and the RAM 108 are a so-called computer and are an example of a control device.

[0018] The imaging optical system 101, the image pickup device 102, the CPU 103, the video output driving unit 104, the frame memory 106, the ROM 107, the RAM 108, the operation unit 109, and the display driving unit 110 are connected to the internal bus 112. Thereby, the imaging optical system 101, the image pickup device 102, the CPU 103, the video output driving unit 104, the frame memory 106, the ROM 107, the RAM 108, the operation unit 109, and the display driving unit 110 can transmit and receive data to and from each other via the internal bus 112.

[0019] The imaging optical system 101 is an optical component that includes an optical lens, an aperture, and a motor for driving the lens. Based on a control signal, the imaging optical system 101 operates the motor to move the lens, optically enlarge or reduce the image, and adjust the focal length, etc. Also, if it is desired to adjust the amount of incident light from the subject, the imaging optical system 101 adjusts the amount of light to achieve the desired brightness by controlling the aperture area of ​​the aperture. The imaging optical system 101 forms an image on the image sensor 102 by passing light from the subject through the lens.

[0020] The image sensor 102 includes a color filter, microlenses, and a photoelectric conversion element. The image sensor 102 converts the optical signal formed by the imaging optical system 101 into an electrical signal and outputs it.

[0021] CPU103 stands for Central Processing Unit and is a processor. CPU103 implements various functions of the imaging device 100. For example, CPU103 controls the imaging settings of the imaging device 100 and performs various image processing on the image output from the image sensor 102.

[0022] The imaging device 100 may have other processors such as an MPU (Micro Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Processing Unit), and QPU (Quantum Processing Unit) in place of or in addition to the CPU 103.

[0023] Some or all of the functions of the imaging device 100 are realized by one or more processors, including the CPU 103, reading programs stored in storage such as ROM 107, loading them into RAM 108, and executing them. For example, the CPU 103 reads a program and performs setting processing to set and change imaging settings, including the threshold exposure determination time and the frequency of recharge operations, which will be described later. In addition, some or all of the functions of the imaging device 100 may be realized by one or more circuits, such as an ASIC (Application Specific Integrated Circuit) and a PLD (Programmable Logic Device) including an FPGA (Field Programmable Gate Array).

[0024] The video output drive unit 104 outputs the image processed by the CPU 103 to the outside of the imaging device 100 via the video terminal 105.

[0025] The video terminal 105 is an interface that outputs video that can be viewed by the user. The video terminal 105 may have one or more interfaces, including SDI (Serial Digital Interface), HDMI (High Definition Multimedia Interface), and DisplayPort (registered trademark). This allows the imaging device 100 to display real-time video on an external display device or the like via the video terminal 105.

[0026] The frame memory 106 is a type of memory that temporarily stores video signals and other data. Based on instructions from the CPU 103 and other components, the frame memory 106 reads and transfers the video signals when needed. Because video signals contain a vast amount of data, the frame memory 106 requires high speed and high capacity. Therefore, the frame memory 106 may be RAM such as DDR4-SDRAM (Dual Data Rate 4 - Synchronous Dynamic RAM). The CPU 103 uses the frame memory 106 to perform various processes. For example, the CPU 103 can perform image processing using images from different time periods and extract only the necessary areas. Therefore, the frame memory 106 is an important element for image processing.

[0027] ROM107 stands for Read Only Memory and is a non-volatile memory element. ROM107 stores the computer program (also called the program) for operating the CPU103, as well as various adjustment parameters necessary for program execution. The imaging device 100 may have storage such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive) instead of, or in addition to, ROM107.

[0028] RAM108 stands for Random Access Memory and is a high-speed read and write memory. RAM108 may be slower and have a smaller capacity than frame memory 106.

[0029] The control unit 109 may be an input device that receives input from the user. For example, the control unit 109 receives user operations to change the setting menu of the imaging device 100. The control unit 109 may receive user operations from a physical interface such as a button, analog key, or switch, or from an external device via a USB interface, or from a web browser via a network.

[0030] The display driver unit 110 displays the image processed by the CPU 103 on the display unit 111.

[0031] The display unit 111 is a display device that allows the user to view images. The display unit 111 displays, for example, images processed by the CPU 103 and setting menus. This allows the user to check the operating status of the imaging device 100. The display unit 111 may be either an LCD (Liquid Crystal Display) or an organic EL (ElectroLuminescence) display device. The display unit 111 may be a small, low-power display device. The display unit 111 may also incorporate resistive and capacitive thin-film elements, such as those called touch panels.

[0032] Next, the photoelectric conversion element 200 of the image sensor 102 in this embodiment will be described using Figures 2 to 10.

[0033] Figure 2 shows an example of the configuration of a photoelectric conversion element 200 in an embodiment. The photoelectric conversion element 200 is an example of an image sensor 102. The photoelectric conversion element 200 has a sensor substrate 21 and a circuit board 23. The sensor substrate 21 and the circuit board 23 are stacked and electrically connected to each other. That is, the photoelectric conversion element 200 has a stacked structure, but is not limited to this structure. For example, the photoelectric conversion element 200 may have a so-called non-stacked structure in which the components included in the sensor substrate and the components included in the circuit board are arranged on a common semiconductor layer. The case in which the image sensor 102 has a stacked structure will be described below.

[0034] The sensor substrate 21 includes a pixel region 22 containing multiple pixels.

[0035] The circuit board 23 includes a circuit region 24 for processing signals detected in the pixel region 22.

[0036] Figure 3 shows an example of the configuration of the sensor substrate 21 of the embodiment. The pixel area 22 of the sensor substrate 21 includes a plurality of pixels 301 arranged two-dimensionally across multiple rows and columns.

[0037] Each pixel 301 includes a photoelectric conversion unit 302 containing an APD (Abalanche Photodiode). The photoelectric conversion unit 302 converts the received light into electricity and outputs the resulting charge as an electrical signal to the circuit board 23. The number of rows and columns of the pixel array forming the pixel region 22 is not particularly limited.

[0038] Figure 4 shows an example of the configuration of the circuit board 23 of the embodiment. The circuit board 23 includes a signal processing circuit 401, a readout circuit 402, a control unit 403, a horizontal scanning circuit unit 404, a signal line 405, and a vertical scanning circuit unit 406. The signal output from the photoelectric conversion unit 302 of the pixel is processed by the signal processing circuit 401.

[0039] The signal processing circuit 401 processes the charge photoelectrically converted by the photoelectric conversion unit 302 in Figure 3 as an electrical signal. The signal processing circuit 401 includes a counter and memory. The signal processing circuit 401 stores a digital value of the number of photons in its memory. The signal processing circuit 401 outputs the countered count value based on the control pulse received from the vertical scanning circuit unit 406.

[0040] The horizontal scanning circuit 404 receives control pulses supplied from the control unit 403 and supplies control pulses to each pixel column to sequentially select each column via the readout circuit 402 and the signal processing circuit 401. The control pulses of the horizontal scanning circuit 404 are pulses for reading out the pixel signal, which includes the count value of each pixel held in the memory of the signal processing circuit 401.

[0041] The signal line 405 outputs a pixel signal, including a count value, from the signal processing circuit 401 of the pixel at the intersection of the column selected by the horizontal scanning circuit unit 404 and the row selected by the vertical scanning circuit unit 406. The signal output to signal line 405 is then output to the outside of the photoelectric conversion element 200 via the output circuit 407. The pixel signal output to signal line 405 is then output to the outside of the photoelectric conversion element 200 via the readout circuit 402 and the output circuit 407.

[0042] The vertical scanning circuit unit 406 receives control pulses supplied from the control unit 403 and supplies control pulses to the signal processing circuit 401 of each pixel.

[0043] The readout circuit 402 reads out pixel signals, including the count value which is the output of the signal processing circuit 401, column by column via the signal line 405, based on the control pulse generated by the vertical scanning circuit unit 406. The readout circuit 402 has a shift register and an address decoder that connect multiple rows as a single unit. Therefore, the readout circuit 402 achieves high-speed readout by reading out pixel signals from multiple rows at once. In particular, in the case of an imaging device that digitally counts the number of photons incident on the APD and outputs the count value from the pixel as a photoelectrically converted digital signal, the operation of the counter circuit that digitally counts the number of photons takes time, so high-speed readout of pixel signals is achieved by reading out pixel signals from multiple rows simultaneously. In this embodiment, for example, the readout circuit 402 reads out pixel signals from pixels in the first row and pixel signals from pixels in the second row simultaneously, column by column.

[0044] The control unit 403 controls the photoelectric conversion element 200. The control unit 403 may be implemented as a function of the CPU 103 and may be an example of a control device. For example, the control unit 403 supplies control pulses to the vertical scanning circuit unit 406 and the horizontal scanning circuit unit 404 in order to selectively read out pixel signals from each pixel. The control unit 403 sets threshold information (count threshold) which serves as a judgment criterion to be compared with the count value, and the exposure time, which is a discrete timing for performing the judgment, to the count judgment circuit described later. The exposure time may be the timing at which the control pulses described above are output. Furthermore, if the exposure times of the pixels in the subject area do not match, the control unit 403 sets the threshold exposure judgment time, which indicates the timing for threshold judgment to determine whether the count value has exceeded the count threshold, and the frequency of recharge operations to the count judgment circuit.

[0045] As shown in Figures 3 and 4, multiple signal processing circuits 401 are arranged in the region that overlaps with the pixel region 22 in a plan view. Then, in a plan view, the vertical scanning circuit section 406, the horizontal scanning circuit section 404, the readout circuit 402, the output circuit 407, and the control unit 403 are arranged so as to overlap between the edge of the sensor substrate 21 and the edge of the pixel region 22. These circuit sections and control units constitute a non-pixel region that does not include pixels 301, and the sensor substrate 21 has a pixel region 22 and a non-pixel region arranged around the pixel region 22. Then, in a plan view, the vertical scanning circuit section 406, the horizontal scanning circuit section 404, the readout circuit 402, the output circuit 407, and the control unit 403 are arranged in the region that overlaps with the non-pixel region.

[0046] Note that the arrangement of the signal line 405, the readout circuit 402, and the output circuit 407 is not limited to Figure 4. For example, the signal line 405 may be arranged to extend in the row direction, and the readout circuit 402 may be placed at the end of the signal line 405. Also, the function of the signal processing circuit 401 does not necessarily have to be provided for each photoelectric conversion unit 302. For example, the signal processing circuit 401 may be associated with multiple photoelectric conversion units 302 and sequentially perform signal processing on the photoelectric conversion units 302. Figure 5 shows the equivalent circuits of pixel 301 and the signal processing circuit 401 corresponding to pixel 301, as seen in Figures 3 and 4.

[0047] The photoelectric conversion unit 302 of the pixel 301 has an APD501. The APD501 converts incident light photoelectrically, generating and outputting charge pairs corresponding to the incident light as electrical signals. The APD501 has two nodes. One of the two nodes of the APD501 is connected to a power line to which a drive voltage VL is supplied. The other node of the APD501 is connected to a power line to which a drive voltage VH, higher than voltage VL, is supplied. In Figure 5, one node of the APD501 is the anode. The other node of the APD501 is the cathode. A reverse bias voltage is supplied to the anode and cathode of the APD501 so that the APD501 performs avalanche multiplication operation. By supplying such a voltage, the charge generated by the incident light undergoes avalanche multiplication, and an avalanche current is generated. Note that the driving mode of the APD501 is classified into two types depending on the value of the reverse bias voltage used to operate the APD501. There are two modes: Geiger mode, in which the anode-cathode voltage difference is greater than the breakdown voltage, and linear mode, in which the anode-cathode voltage difference is near or below the breakdown voltage. An APD501 operating in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In the case of a SPAD, for example, the drive voltage VL is -30V and the drive voltage VH is 1V.

[0048] The signal processing circuit 401 includes a quench element 502, a waveform shaping unit 510, a counter circuit 511, a count determination circuit 512, a selection circuit 513, and drive lines 514, 515, 516, 517, and 518.

[0049] The quench element 502 is connected to a power line to which the drive voltage VH is supplied and to one of the nodes, either the anode or cathode, of the APD 501. The quench element 502 functions as a load circuit (quench circuit) when the signal is multiplied by avalanche multiplication. The quench element 502 suppresses the voltage supplied to the APD 501, thereby suppressing avalanche multiplication (quench operation). The quench element 502 also restores the voltage supplied to the APD 501 to the drive voltage VH by allowing current to flow to compensate for the voltage drop caused by the quench operation (recharge operation). In this embodiment, the quench element 502 is composed of a MOS (Metal Oxide Semiconductor) transistor. The on and off states of the quench element 502 are controlled by a control signal CLK connected to the gate of the quench element 502. The control signal CLK is controlled by the signal generation unit in the control unit 403.

[0050] The waveform shaping unit 510 shapes the cathode voltage change of the APD501 obtained via nodeA during photon detection and outputs a pulse signal from nodeB corresponding to the photon reception frequency. For example, an inverter circuit can be used as the waveform shaping unit 510. Figure 5 shows an example using one inverter as the waveform shaping unit 510, but a circuit with multiple inverters connected in series may be used, or other circuits that have a waveform shaping effect may be used.

[0051] The counter circuit 511 counts the pulse signal output from nodeB of the waveform shaping unit 510 and holds the count value. When the control pulse RES is supplied via the drive line 514, the signal held by the counter circuit 511 is reset. Furthermore, when the control pulse STOP is supplied via the drive line 517, the counter circuit 511 continues to hold the count value until the control pulse RES is supplied.

[0052] The count determination circuit 512 is supplied with the count value held by the counter circuit 511 via the drive line 516 and a control pulse signal from the control unit 403 via the drive line 518. When the count determination circuit 512 receives the control pulse signal, it compares the count value with a predetermined threshold, and if it determines that the count value exceeds the count threshold, it supplies a control pulse STOP to the counter circuit 511 via the drive line 517. Furthermore, the count determination circuit 512 outputs the determination result and the count value at the timing of the control pulse to the selection circuit 513. The timing chart of the control pulse will be described later with reference to Figure 7.

[0053] The selection circuit 513 receives a control pulse SEL from the vertical scanning circuit section 406 in Figure 4 via the drive line 515 (not shown in Figure 4) in Figure 5, which switches the electrical connection between the count determination circuit 512 and the signal line 405. The selection circuit 513 includes, for example, a buffer circuit for outputting a signal. The selection circuit 513 outputs the output signal output by the pixel count determination circuit 512 to the signal line 405.

[0054] Furthermore, switches such as transistors may be placed between the quench element 502 and the APD 501, and between the photoelectric conversion unit 302 and the signal processing circuit 401 to switch the electrical connections. Similarly, the supply of the drive voltage VH or drive voltage VL from the photoelectric conversion unit 302 to the APD 501 may be electrically switched using switches such as transistors.

[0055] Figure 6 is a timing chart illustrating the operation of the photoelectric conversion element 200. Figure 6 schematically shows the relationship between the control signal CLK, the voltage at node A and node B, and the count value in the photoelectric conversion element shown in Figure 5. The horizontal axis in Figure 6 represents time. The vertical axis represents the magnitude of the signal and voltage. The control signal CLK in Figure 6 is a signal that switches the MOS transistor constituting the quench element 502, which functions as a switch, on and off. The voltage at node A is the voltage at one node (also called the input terminal node) of the waveform shaping unit 510 connected to the APD 501. The voltage at node B is the voltage at the other node (also called the output terminal node) of the waveform shaping unit 510 connected to the counter circuit 511. The count value is the signal output from the counter circuit 511, and is the value obtained by counting the pulse signal output from the waveform shaping unit 510.

[0056] When the control signal CLK is high, the drive voltage VH is less likely to be supplied to the APD501. When the control signal CLK is low, the drive voltage VH is supplied to the APD501. A high level control signal CLK is, for example, 1V, and a low level control signal CLK is, for example, 0V. When the control signal CLK is high, the quench element 502 is off, and when the control signal CLK is low, the quench element 502 is on. The resistance value of the quench element 502 when the control signal CLK is high is higher than the resistance value of the quench element 502 when the control signal CLK is low. When the control signal CLK is high, even if avalanche multiplication occurs in the APD501, recharge operation is less likely to occur, so the voltage supplied to the APD501 is below the breakdown voltage of the APD501. Therefore, the avalanche multiplication operation in the APD501 stops.

[0057] At time t1, the control signal CLK changes from a high level to a low level, the quench element 502 turns on, and the recharge operation of the APD501 begins. As a result, the cathode voltage of the APD501 transitions to a high level. Then, the difference between the voltages applied to the anode and cathode of the APD501 becomes a state where avalanche multiplication is possible. The cathode voltage of the APD501 is the same as the voltage of nodeA of the waveform shaping unit 510. Therefore, when the cathode voltage of the APD501 transitions from a low level to a high level, the voltage of nodeA becomes greater than or equal to the judgment threshold at time t2. Here, the judgment threshold is a voltage value uniquely determined by the electrical characteristics of the waveform shaping unit 510. At this time, the pulse signal output from nodeB of the waveform shaping unit 510 inverts, changing from a high level to a low level. Once the APD501 has finished recharging, the voltage difference between the drive voltage VH and the drive voltage VL (= drive voltage VH - drive voltage VL) is applied to the APD501. Then, between time t2 and time t3, the control signal CLK becomes high level, and the quench element 502 turns off.

[0058] Next, at time t3, when a photon is incident on APD501, avalanche multiplication occurs in APD501, an avalanche multiplication current flows through the quench element 502, and the cathode voltage drops. In other words, the voltage at nodeA of the waveform shaping unit 510 drops. If the voltage at nodeA of the waveform shaping unit 510 falls below the judgment threshold while the voltage at nodeA of the waveform shaping unit 510 is dropping, the voltage at nodeB of the waveform shaping unit 510 changes from a low level to a high level. In other words, the portion of the output waveform at nodeA of the waveform shaping unit 510 that exceeds the judgment threshold is shaped by the waveform shaping unit 510, and a pulse signal is output from nodeB of the waveform shaping unit 510. The counter circuit 511 then counts the pulse signal output from the waveform shaping unit 510 and increases the count value of the output counter signal by 1 LSB (Least Significant Bit).

[0059] Although photons are incident on APD501 between time t3 and time t4, the quench element 502 is in the off state, and the voltage applied to APD501 is not a voltage difference that allows for avalanche multiplication, so the voltage level of nodeA of the waveform shaping unit 510 does not exceed the judgment threshold.

[0060] At time t4, the control signal CLK changes from a high level to a low level, and the quench element 502 turns on. Consequently, a current flows through nodeA of the waveform shaping unit 510 to compensate for the voltage drop from the drive voltage VL, and the voltage at nodeA of the waveform shaping unit 510 returns to its original voltage level. At time t5, the voltage at nodeA of the waveform shaping unit 510 becomes greater than or equal to the judgment threshold, so the pulse signal at nodeB of the waveform shaping unit 510 inverts, changing from a high level to a low level.

[0061] At time t6, nodeA of the waveform shaping unit 510 settles to its original voltage level, and the control signal CLK changes from a low level to a high level. In principle, it is sufficient that the period during which the control signal CLK is at a low level is longer than the period during which nodeA of the waveform shaping unit 510 transitions from a low level to a high level. In Figure 6, the period during which the control signal CLK is at a low level is set to be the same as the period during which nodeA of the waveform shaping unit 510 transitions from a low level to a high level. This allows the frequency of the control signal CLK to be set higher, thereby reducing the effect of the "nonlinear relationship between the number of output signals and the number of input signals," which will be discussed later. Subsequently, as explained from time t1 to time t6, the voltages of each node and signal line change in response to the control signal CLK and the incidence of photons.

[0062] However, when the frequency of the APD501's recharge operation is controlled by the control signal CLK, the relationship between the number of input signals and the number of output signals is not linear. In this case, the number of input signals refers to the number of photons incident on the APD501. The number of output signals refers to the count value of photons detected by the imaging device 100. In SPAD, when avalanche breakdown occurs, secondary photons are emitted, causing emission crosstalk with adjacent pixels. However, if the effect of emission crosstalk is ignored, the relationship between the number of input signals and the number of output signals can be theoretically derived. Specifically, when the number of input signals is Nph, the frequency of the control signal CLK (number of CLKs per unit time) is f, and the exposure time is T, the number of output signals Nct is described by the following equation (1).

[0063]

number

[0064] Figure 7 is a timing chart illustrating the exposure time. The control unit 403 sets multiple exposure times, which are the time from the start of exposure for each pixel until threshold determination, to T / (n to the power of (m-1)). These set multiple exposure times are an example of threshold exposure determination times. T represents the maximum exposure time within one frame. n is a positive integer. m is any integer such that m≧1, but Figure 7 shows the timing chart when m is set to 1≦m≦4. The threshold exposure determination times shown in Figure 7 are an example of initial values. A control pulse that becomes high level Hi is supplied to the drive line 518 at each exposure time determined by t=T / (n to the power of (m-1)). In this case, if the count value of the counter circuit 511 reaches a predetermined count threshold at the four exposure times T / (n to the power of (m-1)) when m=1, 2, 3, and 4, the photoelectric conversion element 200 switches from Geiger mode to linear mode, and the APD501 enters a dormant state. Once the APD501 enters a dormant state, the waveform shaping unit 510 does not output a pulse signal. Therefore, the counter circuit 511 maintains the count. The count determination circuit 512 then outputs T / (n^(m-1)), which represents the exposure time corresponding to the control pulse, and the count value to the selection circuit 513.

[0065] Figure 8 shows the relationship between the exposure time for each pixel 301 in the photoelectric conversion element 200 and the count value of the counter circuit 511. In Figure 8, as in Figure 7, the control unit 403 sets a predetermined exposure time T / (n to the power of (m-1)) for 1 ≤ m ≤ 4. When the upper limit of the count of the counter circuit 511 is Cmax, the count threshold is set to Cmax / n for reasons described later. In Figure 8, the count value of a certain pixel increases in proportion to time. As described in Figure 7, the count determination circuit 512 determines whether the count value exceeds the count threshold at the moment of T / (n to the power of (m-1)) in order of shortest exposure time. When the count value increases as shown in Figure 8, the count determination circuit 512 determines whether the exposure time exceeds T / n 3 T / n 2 In this case, the count value is determined not to exceed the count threshold. On the other hand, the count determination circuit 512 determines that the count value has exceeded the count threshold when the exposure time is T / n. Therefore, the count determination circuit 512 supplies a control pulse STOP to the counter circuit 511 to stop counting when T / n is reached. The count determination circuit 512 outputs the count value Cout at T / n and information regarding the exposure time T / n from the photoelectric conversion element 200.

[0066] Figure 9 is a table showing an example of the exposure time information Tcode, which is a data format output as exposure time. Here, the count determination circuit 512 may output the exposure time as time information directly, but as shown in Figure 9, it may also output the exposure time information Tcode corresponding to the exposure time when the count threshold is exceeded. The exposure time information Tcode may be, for example, a few bits (3 bits in this case).

[0067] The count estimate value (Cest) shown in Figure 8 is calculated as Cest = Cout × n. This count estimation method assumes that the count increases at the same rate during the exposure time from 0 to T / n and the non-exposure time from T / n to T. Therefore, if the upper count limit is reached before the count threshold is reached, the rate of increase during the exposure time from 0 to T / n cannot be correctly estimated, and the accuracy of the count estimation decreases. Thus, in this count estimation method, in order to maintain the accuracy of the count estimation, the count determination circuit 512 performs threshold determination before reaching the upper count limit (Cmax). As mentioned above, when the recharge frequency of the APD is controlled by the control signal CLK, the relationship between the number of output signals and the number of input signals is not linear, so linearity correction is necessary to further improve the accuracy of the count estimate value.

[0068] Figure 10 shows the relationship between the exposure time for each pixel 301 in the photoelectric conversion element 200 and the count value of the counter circuit 511. Figure 9 shows how the count increases when the count value becomes equal to the count threshold at each exposure time when the count determination circuit 512 performs threshold determination. As mentioned above, in this count estimation method, in order to maintain the accuracy of the count estimation, the count determination circuit 512 performs threshold determination before reaching the upper count limit (Cmax). Therefore, if the timing of threshold determination when the count increases at a rate that reaches Cmax at the moment of the maximum exposure time T is defined as t1, then it is necessary to determine the timing of threshold determination t2 when the count increases at a rate that reaches Cmax at the moment of t1. That is, when the count threshold is Cmax / n, the time until the count reaches Cmax / n when it increases at a rate that reaches Cmax at the moment of the maximum exposure time T is calculated as T × (1 / n), so t1 = T / n. Similarly, t2 = t1 × (1 / n) = T / n 2This is how it is determined. In this way, each exposure time for threshold determination is calculated as T / (n to the power of (m-1)). As shown in Figure 5, when the recharge frequency of the APD501 is controlled by the control signal CLK, the relationship between the number of output signals and the number of input signals is not linear, as shown in equation (1). Therefore, linearity correction is performed based on the estimated count calculated by the signal processing circuit 401. Specifically, linearity correction refers to determining the number of input signals Nph from the number of output signals Nct per exposure time using the following equation (2), where f is the frequency of the control CLK (number of CLKs per unit time) and T is the length of the exposure time.

[0069]

number

[0070] In this embodiment, the input signal number Nph derived by equation (2) is the number of photons per exposure time for threshold determination. Therefore, the number of photons in one frame exposure time is calculated as Nph × (n to the power of (m-1)).

[0071] Next, the functional configuration of the imaging device 100 in this embodiment will be described using Figure 11. Figure 11 is a block diagram showing the functional configuration of the imaging device 100 in this embodiment. In Figure 11, the imaging optical system 101, the image sensor 102, and the CPU 103 are connected to the internal bus 112 so that they can send and receive data from each other. The CPU 103 has the functions of an optical lens control unit 1100, an image sensor control unit 1101, an exposure control unit 1102, a subject detection unit 1103, and an image processing unit 1104. The image sensor control unit 1101 and the image processing unit 1104 are examples of control means. The CPU 103 may, for example, read a computer program stored in the ROM 107 and load it into the RAM 108 to realize the functions of the optical lens control unit 1100, the image sensor control unit 1101, the exposure control unit 1102, the subject detection unit 1103, and the image processing unit 1104. Note that the imaging device 100 may implement some or all of the functions of the optical lens control unit 1100, image sensor control unit 1101, exposure control unit 1102, subject detection unit 1103, and image processing unit 1104 using one or more circuits such as an ASIC. A description of a configuration similar to that in Figure 1 is omitted.

[0072] The optical lens control unit 1100 transmits control signals to the imaging optical system 101 to adjust the aperture, focus, zoom, and other parameters.

[0073] The image sensor control unit 1101 transmits a control signal to the image sensor 102 to set the exposure time of the image sensor, such as the shutter speed.

[0074] In this embodiment, the image sensor control unit 1101 changes the threshold exposure determination time and the frequency of the recharge operation if the exposure time for which the count value exceeds the count threshold does not coincide for multiple pixels in the subject area. Details of the modification process will be described later with reference to the flowchart.

[0075] The exposure control unit 1102 transmits control signals related to exposure, such as aperture, shutter speed, and gain, to the optical lens control unit 1100, the image sensor control unit 1101, and the image processing unit 1104, in accordance with the shooting settings of the imaging device 100. In addition, in the case of automatic exposure control, the exposure control unit 1102 detects the brightness of the image and automatically controls the aperture, shutter speed, gain, and other exposure-related settings to achieve the appropriate brightness.

[0076] The subject detection unit 1103 determines the presence or absence of a specific subject in the image captured by the image sensor 102 and in the image processed by the image processing unit 1104. Furthermore, if a subject is present, the subject detection unit 1103 detects the subject region, which is the area of ​​the subject within the image, and the coordinate position of the subject within the image. For example, the subject detection unit 1103 can detect people, cars, and boats as specific subjects. Specific subjects may be pre-set or may be set by user selection. The subject detection unit 1103 may detect multiple specific subjects.

[0077] The image processing unit 1104 performs various image processing on the image captured by the image sensor 102. Specifically, the image processing unit 1104 corrects the amount of light in the peripheral areas of the image caused by the characteristics of the imaging optical system 101, corrects sensitivity variations and defective pixels for each pixel of the image sensor 102, performs brightness-related corrections such as gain correction, color-related corrections such as white balance correction, and flicker correction. Furthermore, the image processing unit 1104 may generate strings of text and menu images to notify the user of the settings status of the imaging device 100, and superimpose these onto the image after various image processing has been performed. In addition to text information, the image processing unit 1104 may also superimpose images of shooting assist displays such as histograms, vectorscopes, waveform monitors, zebras, peaking, and false color.

[0078] Furthermore, the image processing unit 1104 in this embodiment determines whether the exposure time for which the count value exceeds the count threshold coincides for multiple pixels in the subject area. Based on the determination result of the image processing unit 1104, the image sensor control unit 1101 changes the threshold exposure determination time and the frequency of the recharge operation.

[0079] Figure 12 is a flowchart of the setting process performed by the imaging device 100. Next, the setting process performed by the imaging device 100 will be explained with reference to Figure 12. The setting process is the process of changing and setting the imaging settings. For example, if the exposure time in which the count value exceeds the count threshold does not match, the setting process includes changing and setting the time from the start of exposure until threshold determination is performed to determine whether the count value of the image sensor 102 has exceeded the count threshold (hereinafter also referred to as the threshold exposure determination time) in order to reduce the amount of blur of the subject (an example of the first setting process). Furthermore, if the exposure time in which the count value exceeds the count threshold does not match even after changing the threshold exposure determination time, the setting process includes changing and setting the frequency of the recharge operation (also referred to as the recharge frequency) (an example of the second setting process). The setting process in Figure 12 is started when the CPU 103 reads the program after the imaging device 100 is started up, and is executed repeatedly.

[0080] In the flowchart of Figure 12, first, in step S1200, the subject detection unit 1103 determines whether or not a subject has been detected. Specifically, the subject detection unit 1103 acquires the image data captured by the image sensor 102 and determines whether or not a specific subject is captured in the image. If the subject detection unit 1103 determines that a specific subject is captured, it proceeds to step S1201. If no specific subject is captured, it proceeds to the termination step and restarts the flowchart operation.

[0081] Next, in step S1201, the image processing unit 1104 acquires the exposure time information Tcode output by each pixel within the subject region, which is the area of ​​the subject in the image detected by the subject detection unit 1103. The exposure time information Tcode is information indicating the exposure time during which the count value exceeded the count threshold. The image processing unit 1104 then proceeds to step S1202.

[0082] Next, in step S1202, the image processing unit 1104 determines whether the exposure time information Tcode of each pixel in the acquired subject area matches, that is, whether the exposure times in which the count value exceeds the count threshold match. If the image processing unit 1104 determines that the exposure times match, it proceeds to the termination step without performing the process of changing the threshold exposure determination time and the frequency of the recharge operation, which will be described later. If the image processing unit 1104 determines that the exposure times do not match, it proceeds to step S1203.

[0083] The matching of exposure times (or exposure time information Tcode) here does not mean that the exposure times of all pixels within the acquired subject area are the same. For example, the image processing unit 1104 may determine that the exposure times are the same if the number of pixels with the most matching exposure times satisfies a predetermined condition. Specifically, the image processing unit 1104 determines that the exposure times are not the same and proceeds to step S1203 if the proportion of pixels with matching exposure times for all pixels within the subject area is smaller than a threshold proportion, and determines that they are the same otherwise. For example, the image processing unit 1104 calculates the mode of the acquired exposure times. The image processing unit 1104 may determine that the exposure times are not the same if the proportion of the number of exposure time information Tcodes with the most matching exposure times among the acquired exposure time information Tcodes is smaller than a predetermined threshold proportion (for example, 90%), and determines that they are the same otherwise.

[0084] Next, in step S1203, the image processing unit 1104 determines whether it is possible to match the exposure time or exposure time information Tcode of pixels in the subject area where the count value exceeds the count threshold by changing the threshold exposure determination time of the image sensor 102. If the image processing unit 1104 determines that it is possible to match the exposure time or exposure time information Tcode, it proceeds to step S1204. On the other hand, if the image processing unit 1104 determines that it is not possible to match the exposure times, it proceeds to step S1205.

[0085] Here, we will describe a method for determining whether it is possible to match the exposure time or exposure time information Tcode by changing the threshold exposure determination time. For example, the image processing unit 1104 gradually changes the exposure time for the image sensor 102 from the longest time to the shortest time that can be set for each frame period. The image processing unit 1104 acquires the exposure time information Tcode for each pixel that indicates the exposure time in which the count value exceeded the count threshold for each changed exposure time. The image processing unit 1104 may determine whether it is possible to match the exposure time by changing the threshold exposure determination time based on whether the acquired exposure time information Tcode matches. Note that, in determining whether it is possible to match the exposure time here, it is not necessary for all exposure times or exposure time information Tcodes to match, and the matching may be determined under the same conditions as in step S1202.

[0086] For example, the image processing unit 1104 may determine that the exposure times can be matched by changing the threshold exposure determination time if the number of pixels whose exposure time (or exposure time information Tcode) best matches the count value exceeding the count threshold among the gradually changing exposure times satisfies a predetermined condition (an example of the first condition). Specifically, if the proportion of pixels in the subject area whose exposure time best matches is smaller than the threshold proportion (the first condition is not met), the image processing unit 1104 determines that it is not possible to match the exposure times by changing the threshold exposure determination time and proceeds to step S1205. On the other hand, if the proportion of such pixels is equal to or greater than the threshold proportion, the image processing unit 1104 determines that it is possible to match the exposure times by changing the threshold exposure determination time and proceeds to S1204. For example, the image processing unit 1104 calculates the mode of the acquired exposure time information Tcode. The image processing unit 1104 may determine that the exposure time cannot be matched by changing the threshold exposure time if the ratio of the number of exposure time information Tcodes with the most mode among the acquired exposure time information Tcodes, i.e., the ratio of the number of pixels whose exposure time best matches, is smaller than a predetermined threshold ratio (for example, 90%), and otherwise determine that the exposure time can be matched by changing the threshold exposure time. Note that the threshold ratio used to determine the match in step S1203 may be different from the threshold ratio used in step S1202.

[0087] If the image processing unit 1104 determines that it is possible to match the exposure time or exposure time information Tcode of pixels within the subject area by changing the threshold exposure determination time, it stores the exposure time for which the exposure time can be matched as the new threshold exposure determination time in the RAM 108. On the other hand, if the image processing unit 1104 determines that it is not possible to match the exposure time of all pixels within the subject area, it stores the exposure time with the highest matching rate as the threshold exposure determination time in the RAM 108. The image processing unit 1104 may also store the exposure time information Tcode as the exposure time in the RAM 108.

[0088] Next, in step S1204, the image sensor control unit 1101 changes the threshold exposure determination time set for the image sensor 102 to a new threshold exposure determination time stored in the RAM 108. In other words, the image sensor control unit 1101 changes the threshold exposure determination time to the exposure time indicated by the exposure time information Tcode with the highest number of occurrences (in this case, the exposure time where all occurrences match). The threshold exposure determination time may include multiple exposure times, as shown in Figure 7. In this case, the image sensor control unit 1101 may set the threshold exposure determination time to a discrete set of multiple exposure times including the exposure time with the highest number of occurrences. After this, the image sensor control unit 1101 proceeds to the termination step.

[0089] Next, in step S1205, the image sensor control unit 1101 changes the threshold exposure determination time set for the image sensor 102 to a new threshold exposure determination time stored in the RAM 108. In other words, the image sensor control unit 1101 changes the threshold exposure determination time to the exposure time indicated by the most frequent exposure time information Tcode (in this case, the mode exposure time). Here, as in step S1204, the image sensor control unit 1101 may set the threshold exposure determination time to one of several discrete exposure times, including the most frequent exposure time. After this, the image sensor control unit 1101 proceeds to step S1206.

[0090] Next, in step S1206, the image sensor control unit 1101 changes the frequency of the APD501 recharge operation on the image sensor 102 and proceeds to step S1207. For example, the image sensor control unit 1101 may change the frequency of the recharge operation by changing the frequency of the control CLK.

[0091] Here, we will explain how to change the frequency of the control CLK that controls the recharge frequency of the APD501. For example, the image sensor control unit 1101 gradually changes the frequency of the control CLK, which can be set for each frame period, from the maximum frequency to the minimum frequency to the image sensor 102, and obtains exposure time information Tcode, which indicates the exposure time during which the count value exceeds the count threshold. The image sensor control unit 1101 may compare the acquired exposure time information Tcode and adopt the control CLK frequency with the highest proportion of pixels whose exposure time information Tcode, i.e., exposure time, matches, and set it as the new control CLK frequency.

[0092] Next, in step S1207, the image processing unit 1104 calculates and changes the correction parameters for linearity correction using the frequency of the control CLK set in step S1206 and the aforementioned equation (2). After this, the image processing unit 1104 proceeds to the termination step.

[0093] As described above, the imaging device 100 of the embodiment is an imaging device that generates an image based on a count value obtained by digitally counting the number of pulses according to the photon reception frequency. By performing the setting process shown in Figure 12, the device acquires exposure time information indicating the exposure time over which the count value exceeds the count threshold, and changes the imaging settings to increase the number of pixels that match this exposure time. As a result, the embodiment increases the number of pixels that match the exposure time over which the count value exceeds the count threshold, so even when capturing video or images with a very wide dynamic range, the amount of blur of moving subjects in the image can be reduced.

[0094] The imaging device 100 of this embodiment sets the threshold exposure determination time for determining whether the count value exceeds the count threshold as an imaging setting, and increases the number of pixels in the subject area whose exposure times match, thereby generating an image and reducing the amount of blur of the subject in the image.

[0095] The imaging device 100 of this embodiment can reduce the amount of motion blur of the subject in the image by changing the frequency of the APD recharge operation by changing the frequency of the control CLK of the image sensor 102, thereby changing the linearity of the number of output signals Nct with respect to the number of input signals Nph. In this embodiment, even if a threshold exposure determination time is set, the frequency of the recharge operation is changed if the proportion of pixels with matching exposure times is small. As a result, the embodiment can obtain the above-mentioned effects while reducing the increase in processing load by changing the frequency of the recharge operation when necessary.

[0096] In this embodiment, the threshold exposure determination time is changed when the proportion of pixels with matching exposure times is small, thereby reducing the increase in processing load while achieving the above-mentioned effects.

[0097] In this embodiment, the threshold exposure determination time is set to the exposure time with the highest number of occurrences, making it possible to more reliably match the exposure time where the count value exceeds the count threshold.

[0098] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.

[0099] Furthermore, the present invention also includes cases in which a software program that realizes the functions of the above-described embodiment is supplied directly from a recording medium or via wired / wireless communication to a system or device having a computer capable of executing the program, and the program is executed.

[0100] Therefore, in order to implement the functional processing of this embodiment using a computer, the program code supplied to and installed on the computer itself also realizes the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

[0101] In that case, the form of the program is irrelevant, as long as it possesses the functionality of a program, including object code, programs executed by an interpreter, and script data supplied to the OS.

[0102] The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory.

[0103] Another possible method for supplying the program is to store the computer program forming the present invention on a server on a computer network, and then have connected client computers download and run the computer program.

[0104] In the above-described embodiment, an example was given of setting the exposure time for determination and the frequency of the recharge operation, but other imaging settings may also be changed. For example, the CPU 103 or the image sensor control unit 1101 may change the count threshold in the setting process so that the number of pixels matching the exposure time in which the count value exceeds the count threshold increases.

[0105] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. Furthermore, the present invention can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0106] The disclosures herein include the following control devices, imaging devices, methods for controlling them, and programs. (Item 1) A control device for controlling an imaging device that generates an image based on a count value obtained by counting pulses output for each pixel according to the light reception frequency of multiple photoelectric conversion units that convert photons into electrical signals, A detection means for detecting the subject of an image, A control means that obtains information regarding the exposure time in which the count value obtained by counting the pulses output from each pixel in the region of the subject exceeds a count threshold, and performs a modification process to change the imaging settings so that the number of pixels matching the exposure time increases. A control device characterized by comprising: (Item 2) The control means executes a first modification process, as the modification process, which modifies the threshold exposure determination time, which is the time from the start of exposure until it is determined whether the count value exceeds the count threshold. The control device according to item 1, characterized in that it is a control device. (Item 3) The control means performs a second modification process, as the modification process, which changes the frequency of the recharge operation that supplies voltage to the photoelectric conversion unit. The control device according to item 2, characterized in that (Item 4) If, after executing the first modification process, the number of pixels whose exposure time best matches the first condition does not satisfy the first condition, the control means executes the second modification process. The control device according to item 3, characterized in that (Item 5) The first condition is that the proportion of pixels whose exposure time best matches is smaller than the first threshold proportion. The control device according to item 4, characterized in that it is a control device. (Item 6) The control means executes the modification process if the proportion of pixels with matching exposure times is smaller than the second threshold proportion. A control device according to any one of items 1 to 5, characterized in that it is a control device. (Item 7) The control means, in the first modification process, changes the threshold exposure determination time to the exposure time with the highest number of occurrences. The control device according to item 2, characterized in that (Item 8) The control device described in item 1, Multiple photoelectric conversion units that output electrical signals corresponding to the frequency of reception of incident light, A counting means that counts the number of pulses converted from the aforementioned electrical signal and outputs it as a count value, A recharge means for performing a recharge operation by applying voltage to the photoelectric conversion unit, A count determination means for determining whether the aforementioned count value has exceeded a count threshold, Equipped with, The control device generates an image based on the count value. An imaging device characterized by the following features. (Item 9) A control method for controlling an imaging device that generates an image based on a count value obtained by counting pulses output for each pixel according to the light reception frequency of multiple photoelectric conversion units that convert photons into electrical signals, Detect the subject in the image, The system obtains information regarding the exposure time when the count value obtained by counting the pulses output from each pixel in the region of the subject exceeds a count threshold, and performs a modification process to change the imaging settings so that the number of pixels matching the exposure time increases. A control method characterized by the following: (Item 10) A program to cause a computer to function as one of the control devices described in any one of items 1 through 7.

[0107] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0108] 100...Imaging device, 102...Image sensor, 103...CPU, 200...Photoelectric conversion element, 301...Pixel, 302...Photoelectric conversion unit, 403...Control unit, 501...APD, 1101...Image sensor control unit, 1103...Subject detection unit, 1104...Image processing unit.

Claims

1. A control device for controlling an imaging device that generates an image based on a count value obtained by counting pulses output for each pixel according to the light reception frequency of multiple photoelectric conversion units that convert photons into electrical signals, A detection means for detecting the subject of an image, A control means that obtains information regarding the exposure time in which the count value obtained by counting the pulses output from each pixel in the region of the subject exceeds a count threshold, and performs a modification process to change the imaging settings so that the number of pixels matching the exposure time increases. A control device characterized by comprising:

2. The control means executes a first modification process, as the modification process, which modifies the threshold exposure determination time, which is the time from the start of exposure until it is determined whether the count value exceeds the count threshold. The control device according to feature 1.

3. The control means performs a second modification process, as the modification process, which changes the frequency of the recharge operation that supplies voltage to the photoelectric conversion unit. The control device according to claim 2.

4. If the control means executes the first modification process but the number of pixels whose exposure time best matches does not satisfy the first condition, it executes the second modification process. The control device according to claim 3.

5. The first condition is that the proportion of pixels whose exposure time best matches is smaller than the first threshold proportion. The control device according to feature 4.

6. The control means executes the modification process if the proportion of pixels with matching exposure times is smaller than the second threshold proportion. The control device according to feature 1.

7. The control means, in the first modification process, changes the threshold exposure determination time to the exposure time with the highest number of occurrences. The control device according to claim 2.

8. The control device according to claim 1, Multiple photoelectric conversion units that output electrical signals corresponding to the frequency of reception of incident light, A counting means that counts the number of pulses converted from the aforementioned electrical signal and outputs it as a count value, A recharge means for performing a recharge operation by applying voltage to the photoelectric conversion unit, A count determination means for determining whether the aforementioned count value has exceeded a count threshold, Equipped with, The control device generates an image based on the count value. An imaging device characterized by the following features.

9. A control method for controlling an imaging device that generates an image based on a count value obtained by counting pulses output for each pixel according to the light reception frequency of multiple photoelectric conversion units that convert photons into electrical signals, Detect the subject in the image, The system obtains information regarding the exposure time when the count value obtained by counting the pulses output from each pixel in the region of the subject exceeds a count threshold, and performs a modification process to change the imaging settings so that the number of pixels matching the exposure time increases. A control method characterized by the following:

10. A program for causing a computer to function as one of the means of the control device described in any one of claims 1 to 7.

Images