Processing apparatus, imaging apparatus, processing method, and program

Abstract

To provide a processing apparatus that can acquire subject information while photographing one frame.SOLUTION: A processing apparatus processes signals from a plurality of pixels each including a counter that counts the number of photons based on light from a subject received through an optical system. The processing apparatus has: a first acquisition unit that acquires first signals based on count values from the counter at the start and the end of a first storage period, and second signals included in the same frame period as the first storage period and based on count values from the counter at the start and the end of a second storage period different from the first storage period; and a second acquisition unit that acquires the speed of the subject according to the first signals, the second signals, and the imaging magnification of the optical system.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a processing device. [Background technology]

[0002] Conventionally, there is known a technique for digitally measuring the number of photons arriving at an avalanche diode that causes avalanche doubling and outputting the count value as a digital signal. This technique is called a single photon avalanche diode (SPAD). Patent Document 1 discloses a configuration for acquiring multiple images whose accumulation periods overlap each other in a photoelectric conversion device having an avalanche photodiode (APD). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7223070 Summary of the Invention [Problem to be solved by the invention]

[0004] When SPAD is applied to an in-vehicle camera or a surveillance camera, it is desired to acquire information about the subject (for example, the speed of the subject) at the same time as capturing the image of the subject. The configuration of Patent Document 1 can acquire multiple images with overlapping image capturing periods, but cannot calculate the speed of the subject, etc.

[0005] An object of the present invention is to provide a processing device capable of acquiring subject information while one frame is being captured. [Means for solving the problem]

[0006] A processing device according to one aspect of the present invention is a processing device that processes signals from a plurality of pixels, each of which has a counter that counts photons based on light from a subject received through an optical system, and is characterized by having a first acquisition unit that acquires a first signal based on the count value of the counter at the start and end of a first accumulation period, and a second signal based on the count value of the counter at the start and end of a second accumulation period that is included in the same frame period as the first accumulation period but is different from the first accumulation period, and a second acquisition unit that acquires the velocity of the subject according to the first signal, the second signal, and the imaging magnification of the optical system. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a processing device capable of acquiring subject information while one frame is being captured. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a light detection system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of a read control unit according to the first embodiment. [Figure 3] FIG. 2 is a diagram showing a configuration of a pixel region of a sensor unit according to the first embodiment. [Figure 4] 4 is a timing chart showing the operation of the light detection unit. [Figure 5] 4 is a flowchart showing the operation of the photodetector of the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of an image of a plurality of divided frames. [Figure 7] FIG. 10 is a block diagram showing the configuration of a light detection system according to a second embodiment. [Figure 8] 10 is a flowchart showing the operation of the photodetector of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same components, and redundant explanations will be omitted. [Example]

[0010] FIG. 1 is a block diagram showing the configuration of a light detection system (imaging device) 1000 according to this embodiment. The light detection system 1000 includes a photographing lens (optical system) 11, a light detection unit 21, a memory unit 31, a readout control unit 41, a lens driving unit 51, a focus position detection unit (first detection unit) 52, a zoom position detection unit (second detection unit) 53, and an imaging magnification memory unit 61. The light detection system 1000 also includes a processing unit (processing device) 71, a control unit (first control unit) 81, an image reconstruction unit 91, a display unit 92, a recording unit 93, and a setting unit 94. Note that the functional blocks shown in FIG. 1 do not necessarily have to be built into the same housing. Alternatively, they may be configured as separate devices connected to each other via signal paths.

[0011] The light detection unit 21 includes an optical system 22 and a sensor unit (image pickup element) 23, and acquires an optical image of a subject formed by the optical system 22. The sensor unit 23 counts voltage pulses output from each pixel each time a photon is incident on the pixel (photo count). The obtained count value is stored in the memory unit 31. The optical system 22 may or may not be detachable from the light detection unit 21.

[0012] In the following description, the sensor unit 23 is a SPAD array in which a plurality of single photon avalanche diode pixels (hereinafter referred to as SPAD pixels) are arranged, but the present invention is not limited to this. The sensor unit 23 may have a configuration in which a plurality of pixels each having a photodiode that does not undergo avalanche doubling are arranged.

[0013] The light detection unit 21 can acquire data for first to Nth readout periods (accumulation times) (N: an integer equal to or greater than 2) included in a period equivalent to one frame. The readout control unit 41 outputs a signal based on the acquired data to the processing unit 71.

[0014] The lens driving unit 51 moves at least a part of the lens included in the photographing lens 11 based on information from the control unit 81. For example, it moves the lens to a focus position or to a zoom position. The focus position detection unit 52 acquires the focus position from the position information output from the lens driving unit 51. The zoom position detection unit 53 acquires the zoom position from the position information output from the lens driving unit 51. The imaging magnification storage unit 61 stores the imaging magnification of the imaging lens 11. In this embodiment, the imaging magnification storage unit 61 stores the relationship between the focus position, zoom position, and imaging magnification, calculates the imaging magnification using the information acquired from the focus position detection unit 52 and the zoom position detection unit 53, and outputs it to the processing unit 71.

[0015] The processing unit 71 includes a spatial information extraction unit (detection unit) 72, a traveling direction analysis unit (first acquisition unit) 73, and a speed calculation unit (second acquisition unit) 74. The spatial information extraction unit 72 recognizes (detects) the subject from the image. The traveling direction analysis unit 73 calculates (acquires) the traveling direction and movement amount (pix) of the subject in the sensor unit 23. The speed calculation unit 74 calculates (acquires) the actual speed of the subject using information from the traveling direction analysis unit 73 and the imaging magnification storage unit 61. The processing unit 71 may also include a focus position detection unit 52 and a zoom position detection unit 53. The processing unit 71 may also include an imaging magnification storage unit 61. The processing unit 71 may also include a control unit 81.

[0016] The control unit 81 controls the timing of imaging by the light detection unit 21. The control unit 81 also controls the lens driving unit 51 so that the imaging conditions set in the setting unit 94 are met. If the image size of the subject does not satisfy a predetermined condition (for example, if it is larger than the size of the imaging surface of the sensor unit 23), the control unit 81 controls the lens driving unit 51 to control the lens position (change the imaging magnification) so that the entire subject is imaged on the sensor unit 23. The control unit 81 also performs image reconstruction in the image reconstruction unit 91 based on information obtained from the processing unit 71. The control unit 81 also displays the information obtained from the processing unit 71 on the display unit 92 and records it in the recording unit 93.

[0017] The readout control unit 41 of this embodiment will be described below with reference to FIG. 2. FIG. 2 is a block diagram showing the configuration of the readout control unit 41 of this embodiment. A signal 411 output from the light detection unit 21 is input to a signal selection unit 412. The signal selection unit 412 outputs data from the first to N-1th readout periods to a first image acquisition unit 413, and outputs data from the Nth readout period to a second image acquisition unit 414. An image processing unit 415 calculates the difference between each pixel value of the i-th (i: natural number) and i+1-th readout periods of the data obtained from the first image acquisition unit 413. A memory 416 stores the output values ​​from the image processing unit 415 and the output values ​​(images) from the image acquisition unit 414. With this configuration, the output values ​​of the first to N-th readout periods and one frame of captured image can be recorded in the memory 416 in a period equivalent to one frame.

[0018] 3 is a diagram showing the configuration of the pixel region of the sensor unit 23. In the pixel region, a plurality of SPAD pixels 103 are repeatedly arranged two-dimensionally in the X and Y directions. Each SPAD pixel 103 includes a photoelectric conversion unit (avalanche diode) 201, a quenching element 202, a control unit 210, a counter / memory 211, and a readout unit 212.

[0019] A potential based on a potential VH higher than the voltage VL supplied to the anode is supplied to the cathode of the photoelectric conversion unit 201. A potential is supplied to the anode and cathode of the photoelectric conversion unit 201 so that a reverse bias is applied, which causes avalanche multiplication of photons incident on the photoelectric conversion unit 201. By performing photoelectric conversion with such a reverse bias potential supplied, the charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0020] When a reverse bias voltage is applied, if the potential difference between the anode and cathode is greater than the breakdown voltage, the avalanche diode operates in Geiger mode. An avalanche diode that uses Geiger mode operation to quickly detect weak signals at the single photon level is called a SPAD (Single Photon Avalanche Diode).

[0021] The control unit 210 determines whether or not to count the output signal from the photoelectric conversion unit 201. For example, the control unit 210 is a switch (gate circuit) provided between the photoelectric conversion unit 201 and the counter / memory 211. In this case, the gate of the control unit 210 is connected to the pulse line 124, and the control unit 210 is switched on and off in response to a signal based on a control signal from the control unit 81 input to the pulse line 124. The gates of the control unit 210 are controlled collectively for all columns. This collectively controls the start and end of photodetection for the SPAD pixels.

[0022] Furthermore, the control unit 210 may be configured with a logic circuit instead of a switch. For example, if an AND circuit is provided as the logic circuit, and the first input of the AND circuit is the output from the photoelectric conversion unit 201 and the second input is the signal on the pulse line 124, it becomes possible to switch whether or not to count the output signal from the photoelectric conversion unit 201.

[0023] Furthermore, the control unit 210 does not need to be provided between the photoelectric conversion unit 201 and the counter / memory 211, but may be a circuit that inputs a signal to switch the counter of the counter / memory 211 between operating and non-operating.

[0024] The counter / memory 211 counts the number of photons entering the photoelectric conversion unit 201 and stores the count as digital data (counter value). A reset line 213 is provided corresponding to each row, and when a control pulse is supplied to the reset line 213 from a vertical scanning circuit unit (not shown), the stored signal is reset.

[0025] The readout unit 212 is connected to the counter / memory 211 and a readout signal line 123. A control pulse is supplied to the readout unit 212 from the vertical scanning circuit unit via a control line 214. The readout unit 212 switches whether or not to output the count value of the counter / memory 211 to the readout signal line 123. The readout unit 212 includes, for example, a buffer circuit for outputting a signal.

[0026] The readout signal line 123 may be a signal line that outputs a signal from the photodetector 21 to the readout control unit 41, or a signal line that outputs a signal to a signal processing unit provided within the photodetector 21. Furthermore, the horizontal scanning circuit unit (not shown) and the vertical scanning circuit unit may be provided on the substrate on which the SPAD array is provided, or may be provided on a substrate different from the substrate on which the SPAD array is provided.

[0027] The imaging method by the control unit 81 will be described below with reference to FIG. 4. FIG. 4 is a timing chart showing the operation of the light detection unit 21. In this embodiment, photoelectric conversion is performed periodically, for example, at 120 full frames / second. In this embodiment, a frame having a length of, for example, 120 full frames / second is called a full frame, and a full frame divided into multiple parts is called a frame. In this embodiment, the explanation will be given using a full frame divided into four parts (2.08 ms), for example. Specifically, as shown in FIG. 4, a full frame is divided into frames (F) 1_1, 1_2, 1_3, and 1_4, each with an equal time (2.08 ms).

[0028] Note that frame 1_1 is a readout period from time T0, which is the start time of full frame 1, to time T1. Frame 1_2 is a readout period from time T0 to time T2. Frame 1_3 is a readout period from time T0 to time T3. Frame 1_4 is a readout period from time T0 to time T4.

[0029] At time T0, the counter / memory 211 is reset, and the reading unit 212 obtains the count values ​​C1_1, C1_2, C1_3, and C1_4 from the counter / memory 211 at times T1 to T4, respectively.

[0030] 5 is a flowchart showing the operation of the light detection unit 21 of this embodiment, and shows the operation when performing an imaging method and image processing. Note that the order of the steps in the flowchart of FIG. 5 is not limited to this.

[0031] In step S101, the control unit 81 performs initial setting of the photographing lens 11. The initial setting of the photographing lens 11 means, for example, outputting the zoom position and focus position to the lens driving unit 51. It is also possible to output information that has been set in advance or information used in the previous frame.

[0032] In step S102, the control unit 81 sets the frame sequence n (n: natural number) in the current full frame to 1.

[0033] In step S103, the control unit 81 obtains the image signal for the current frame from the read control unit 41.

[0034] In step S104, the control unit 81 adds 1 to the frame ordinal number n.

[0035] In step S105, the control unit 81 determines whether the sequence number n is smaller than the division number N of a full frame (N: natural number, 4 in this embodiment). If the control unit 81 determines that the sequence number n of the frame is smaller than the division number N, it executes the process of step S106, and if it determines that the sequence number n is not smaller than the division number N, it executes the process of step S103. In this embodiment, for example, in the cases of frames 1_1, 1_2, and 1_3, the process of step S106 is executed, and in the case of frame 1_4, the process moves to the next frame F2_1.

[0036] In step S106, the control unit 81 determines whether or not the subject has been recognized by the spatial information extraction unit 72. Specifically, using information such as the position of the subject, the recognition accuracy of the subject, and the type of the subject, it is determined whether or not the entire area of ​​the subject has been detected by the sensor unit 23. If the control unit 81 determines that the subject has been recognized, it executes the process of step S107, and if it determines that the subject has not been recognized, it executes the process of step S101 and changes the imaging magnification.

[0037] In step S107, the traveling direction analysis unit 73 calculates (acquires) the difference between the count values ​​(first signal, second signal). For example, the difference between frames 1_2 and 1_1 is calculated, and the amount of movement of the subject on the imaging surface of the sensor unit 23 is calculated.

[0038] In step S108, the speed calculation unit 74 acquires the imaging magnification at the time of shooting based on the information acquired from the imaging magnification storage unit 61.

[0039] In step S109, the control unit 81 calculates (obtains) the subject speed using the difference in the count values ​​obtained in step S107 and the imaging magnification obtained in step S108.

[0040] An example of a method for calculating the subject speed will now be described. The following description is merely an example, and the present invention is not limited to this.

[0041] The conditions are as follows:

[0042] Subject size (L): 2000mm x 1000mm Number of pixels of the sensor unit 23 (w): 20 pix x 15 pix Imaging magnification (β): -0.0031 Full frame rate: 120 full frames / s Frame rate: 480 frames / s In this case, the subject size L' (=w / (-β)) per pixel of the sensor unit 23 is calculated to be 3 mm.

[0043] Furthermore, when the amount of movement of the object in the sensor unit 23 is d, the speed of the object is v (= d × L' / (1 / 480) × 60 × 60 / 10 6 ) is calculated as 6km / h.

[0044] 6 is a diagram showing an example of images (count values) of multiple divided frames. The traveling direction analysis unit 73 calculates the traveling direction using signals based on the count values ​​at the start and end of each of the multiple frames. When the count values ​​at times T1 to T4 are C1_1, C1_2, C1_3, and C1_4, respectively, the differences D1_1, D1_2, D1_3, and D1_4 between the count values ​​of the multiple frames are expressed by the following equations.

[0045] D1_1=C1_1-0 D1_2=C1_2-C1_1 D1_3=C1_3-C1_2 D1_4=C1_4-C1_3 The differences E1_1, E1_2, E1_3, and E1_4 of the differences D1_1, D1_2, D1_3, and D1_4 are respectively expressed by the following equations.

[0046] E1_1=D1_1-0 E1_2=D1_2-D1_1 E1_3=D1_3-D1_2 E1_4=D1_4-D1_3 The speed calculation unit 74 can calculate the moving speed of the subject on the imaging surface of the sensor unit 23 using the differences E1_1, E1_2, E1_3, and E1_4.

[0047] Conventionally, it was not possible to recognize a subject until one frame had elapsed, but in this embodiment, it is possible to quickly recognize a subject by using output values ​​from the first to Nth readout periods within one frame period. Also, it is possible to calculate the subject speed using the output values ​​from the first to Nth readout periods. In other words, with the configuration of this embodiment, it is possible to acquire subject information while one frame is being captured.

[0048] Furthermore, images acquired in full frame have a long accumulation time and can improve contrast, so they can be used as images for display.

[0049] The spatial information extraction section 72 may recognize the subject only from the signal acquired in full frame.

[0050] In this embodiment, when the illuminance of the subject is Le (lux), it is preferable to satisfy the following conditional expression (1):

[0051] 0.00 <Le<1.00 (1) Condition (1) defines the subject illuminance. If the upper limit of condition (1) is exceeded, the relationship between the number of incident photons and the counted value will no longer be linear, which is undesirable.

[0052] It is more preferable that the numerical range of conditional expression (1) be the numerical range of conditional expression (1a) below.

[0053] 0.00 <Le<0.25 (1a) It is more preferable that the numerical range of conditional expression (1) be the numerical range of the following conditional expression (1b).

[0054] 0.00 <Le<0.10 (1b) In each embodiment, it is preferable to satisfy the following conditional expression (2).

[0055] 0.0 <m / fT<1.0 (2) Here, m is the average number of incident photons per pixel, and fT is the clock frequency. Conditional formula (2) defines the average number of incident photons m per pixel relative to the clock frequency fT. The count value n per pixel is defined by the following formula:

[0056] n=fT×(1-Exp(-m / fT)) The average number of incident photons per pixel, m, is defined by the following formula:

[0057] m=S×Lf×t Lf=(R×T / 4F^2)×Le Here, S is the sensor sensitivity, Lf is the sensor surface illuminance, t is the exposure time, R is the reflectance of the subject, T is the lens transmittance, and F is the lens Fno. According to the above relationship, the smaller the value of the average number of incident photons per pixel m relative to the clock frequency fT, the more linearity can be satisfied.

[0058] Exceeding the upper limit of conditional expression (2) is undesirable because the relationship between the number of incident photons and the counted value is no longer linear. Falling below the lower limit of conditional expression (2) is undesirable because the measurement error increases due to the influence of noise components. Regarding the determination of linearity, the determination criteria are not limited to counting the number of photons, and may be based on the illuminance of the subject and the luminance value of the image.

[0059] It is more preferable that the numerical range of conditional expression (2) be the numerical range of conditional expression (2a) below.

[0060] 0.01 <m / fT<0.80 (2a) It is more preferable that the numerical range of conditional expression (2) be the numerical range of the following conditional expression (2b).

[0061] 0.05 <m / fT<0.50 (2b) In each embodiment, it is preferable to satisfy the following conditional expressions (3) and (4).

[0062] Le<0.50 (3) 3 <m<500 (4) Conditions (3) and (4) define the subject illumination and the average number of incident photons per pixel, respectively. Even if the subject illumination is low, as long as the detectable light quantity satisfies conditions (3) and (4), the camera can be used in the dark. It is not desirable to fall below the lower limit of condition (4) because the influence of noise components increases.

[0063] It is more preferable that the numerical ranges of the conditional expressions (3) and (4) be set to the numerical ranges of the following conditional expressions (3a) and (4a), respectively.

[0064] Le<0.25 (3a) 10 <m<300 (4a) It is more preferable that the numerical ranges of the conditional expressions (3) and (4) be set to the numerical ranges of the following conditional expressions (3b) and (4b), respectively.

[0065] Le<0.10 (3b) 100 <m<250 (4b) [Example]

[0066] 7 is a block diagram showing the configuration of a light detection system (imaging device) 1001 of this embodiment. The light detection system 1001 includes a photographing lens (optical system) 11, a light detection unit 21, a storage unit 31, a readout control unit 41, a lens driving unit 51, a light source 55, a timing control unit (second control unit) 56, a time information acquisition unit 57, and an imaging magnification storage unit 61. The light detection system 1001 also includes a processing unit (processing device) 71, a control unit (first control unit) 81, an image reconstruction unit 91, a display unit 92, a recording unit 93, and a setting unit 94. In this embodiment, only the configuration different from that of embodiment 1 will be described, and a description of the common configuration will be omitted.

[0067] The light source 55 emits light whose light intensity changes over time (for example, pulsed light or light modulated with a sine wave). The light emitted by the light source 55 may be pulsed laser light of a specific wavelength or LED pulsed light. The subject is illuminated by the light source 55, and the light reflected from the subject is acquired by the light detection unit 21. The light source 55 may be disposed independently, or may be fixed to a device such as an HMD (head mounted display) or a car.

[0068] The control unit 81 transmits a light emission control signal to the timing control unit 56, and the timing control unit 56 causes the light source 55 to emit light in accordance with the light emission control signal. In this embodiment, the light source 55 emits a light beam of a predetermined wavelength with a width of, for example, several picoseconds to several tens of nanoseconds.

[0069] The light detection unit 21 receives light rays (reflected light) reflected from the subject. The light detection unit 21 acquires an optical image of the subject formed by the optical system 22. The sensor unit 23 counts voltage pulses output from each pixel each time a photon is incident on the pixel (photo count). In this embodiment, the light detection unit 21 uses a TOF (Time of Flight) method.

[0070] During one light receiving operation, the sensor unit 23 is exposed for a predetermined exposure time. The sensor unit 23 receives an exposure control signal transmitted from the timing control unit 56 and is exposed in accordance with the exposure control signal. During this time, the light source 55 emits light at least once with a predetermined pulse width, and the sensor unit 23 receives the light.

[0071] The timing control unit 56 controls the light source 55 and the light detection unit 21 so that the light detection unit 21 receives reflected light at the timing of irradiation by the light source 55. The timing control unit 56 may be provided within the processing unit 71.

[0072] The time information acquisition unit 57 acquires, for each frame, a set of data including light intensity distribution information on the XY plane and time information indicating the time when the light intensity distribution information was acquired.

[0073] Pulsed light emitted from the light source 55 is irradiated onto the subject, and the reflected light reaches the photodetector 21, where it is detected (photocounted). The photodetector 21 has a plurality of SPAD pixels arranged in an array, and the photodetection timing of the plurality of SPADs arranged in each row is controlled collectively for all pixels. Because the photodetector 21 is a SPAD array in which SPAD pixels are arranged two-dimensionally, it can acquire, for each frame, a set of data comprising light intensity distribution information in the XY plane and time information indicating the time when the light intensity distribution information was acquired. This makes it possible to acquire information regarding x, y, and t.

[0074] 8 is a flowchart showing the operation of the light detection unit 21 of this embodiment, and shows the operation when performing an imaging method and image processing. Note that the order of the steps in the flowchart of FIG. 8 is not limited to this.

[0075] The processing from step S201 to step S207 is the same as the processing from step S101 to step S107 in FIG. 5, and therefore the description thereof will be omitted.

[0076] In step S208, the control unit 81 calculates (acquires) subject distance information using the time information acquired from the time information acquisition unit 57.

[0077] In step S209, the control unit 81 acquires the imaging magnification at the time of shooting based on the subject information calculated in step S208 and the information acquired from the imaging magnification storage unit 61.

[0078] In step S210, the control unit 81 calculates (obtains) the subject speed using the difference between the count values ​​obtained in step S207 and the imaging magnification obtained in step S209.

[0079] As described above, according to the configuration of this embodiment, it is possible to acquire subject information while one frame is being photographed. [Other Examples] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0080] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) A processing device that processes signals from a plurality of pixels, each of which includes a counter that counts photons based on light from an object received through an optical system, a first acquisition unit that acquires a first signal based on a count value of the counter at the start and end of a first accumulation period, and a second signal based on a count value of the counter at the start and end of a second accumulation period that is included in the same frame period as the first accumulation period and is different from the first accumulation period; a second acquisition unit that acquires a velocity of the subject in accordance with the first signal, the second signal, and an imaging magnification of the optical system. (Configuration 2) a detection unit that detects the subject from an image; and a first control unit that changes the imaging magnification when the size of the subject in the image detected by the detection unit does not satisfy a predetermined condition. (Configuration 3) 3. The processing device according to configuration 1 or 2, wherein the imaging magnification is a value according to a focus position of the optical system. (Configuration 4) 3. The processing device according to configuration 1 or 2, wherein the imaging magnification is a value corresponding to information based on the timing at which a light source unit emits light and the timing at which the photons are counted. (Configuration 5) 5. The processing device according to configuration 4, further comprising a second control unit that controls the timing at which the light source unit emits light and the timing at which the photons are counted. (Claim 6) 6. The processing device according to claim 1, further comprising a storage unit that stores the imaging magnification. (Configuration 7) 7. The processing device according to any one of configurations 1 to 6, further comprising a first position detection unit that detects a focus position of the optical system. (Configuration 8) 8. The processing device according to any one of configurations 1 to 7, further comprising a second position detection unit that detects a zoom position of the optical system. (Configuration 9) When the illuminance of the subject is Le (lux), Le<1.00 9. The processing apparatus according to any one of configurations 1 to 8, wherein the following condition is satisfied: (Configuration 10) When the average number of incident photons for each of the plurality of pixels is m and the number of clocks is fT, 0.00 <m / fT<1.00 10. The processing apparatus according to any one of configurations 1 to 9, wherein the following condition is satisfied: (Configuration 11) When the illuminance of the subject is Le (lux) and the average number of incident photons for each of the plurality of pixels is m, Le<0.5 3 <m<500 11. The processing apparatus according to any one of configurations 1 to 10, wherein the following condition is satisfied: (Configuration 12) A processing device according to any one of configurations 1 to 11; and a plurality of pixels each including a counter that counts photons based on light from a subject received via an optical system. (Configuration 13) 13. The imaging device according to configuration 12, wherein each of the plurality of pixels includes a memory for storing the count value of the counter. (Method 1) A processing method for processing signals from a plurality of pixels, each of which has a counter that counts photons based on light from an object received through an optical system, comprising: acquiring a first signal based on the count value of the counter at the start and end of a first accumulation period, and a second signal based on the count value of the counter at the start and end of a second accumulation period that is included in the same frame period as the first accumulation period and is different from the first accumulation period; and acquiring the velocity of the subject according to the first signal, the second signal, and an imaging magnification of the optical system. (Configuration 14) A program that causes a computer to execute the processing method described in Method 1.

[0081] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0082] 11. Photographic lens (optical system) 71 Processing equipment 73 Direction analysis unit (first acquisition unit) 74 Speed ​​calculation unit (second acquisition unit) 103 pixels 211 Counter

Claims

1. A processing device that processes signals from a plurality of pixels, each of which includes a counter that counts photons based on light from an object received through an optical system, a first acquisition unit that acquires a first signal based on count values ​​of the counter at the start and end of a first accumulation period, and a second signal based on count values ​​of the counter at the start and end of a second accumulation period that is included in the same frame period as the first accumulation period and is different from the first accumulation period; a second acquisition unit (74) that acquires the velocity of the subject in accordance with the first signal, the second signal, and the imaging magnification of the optical system.

2. a detection unit that detects the subject from an image; 2. The processing device according to claim 1, further comprising: a first control unit that changes the imaging magnification when the size of the subject in the image detected by the detection unit does not satisfy a predetermined condition.

3. 3. The processing device according to claim 1, wherein the imaging magnification is a value corresponding to a focus position of the optical system.

4. 3. The processing device according to claim 1, wherein the imaging magnification is a value corresponding to information based on timing at which a light source unit emits light and timing at which the photons are counted.

5. 5. The processing device according to claim 4, further comprising a second control unit that controls the timing at which the light source unit emits light and the timing at which the photons are counted.

6. 3. The processing device according to claim 1, further comprising a storage unit for storing the imaging magnification.

7. 3. The processing apparatus according to claim 1, further comprising a first position detection unit that detects a focus position of the optical system.

8. 3. The processing device according to claim 1, further comprising a second position detection unit that detects a zoom position of the optical system.

9. When the illuminance of the subject is Le (lux), Le<1.00 3. The processing apparatus according to claim 1, wherein the following condition is satisfied:

10. When the average number of incident photons for each of the plurality of pixels is m and the number of clocks is fT, 0.00<m / fT<1.00 3. The processing apparatus according to claim 1, wherein the following condition is satisfied:

11. When the illuminance of the subject is Le (lux) and the average number of incident photons for each of the plurality of pixels is m, Le<0.5 3<m<500 3. The processing apparatus according to claim 1, wherein the following condition is satisfied:

12. The processing device according to claim 1 or 2; and a plurality of pixels each including a counter that counts photons based on light from a subject received via an optical system.

13. 13. The imaging device according to claim 12, wherein each of the plurality of pixels includes a memory that stores the count value of the counter.

14. A processing method for processing signals from a plurality of pixels, each of which has a counter that counts photons based on light from an object received through an optical system, comprising: acquiring a first signal based on the count value of the counter at the start and end of a first accumulation period, and a second signal based on the count value of the counter at the start and end of a second accumulation period that is included in the same frame period as the first accumulation period and is different from the first accumulation period; and acquiring a velocity of the subject in accordance with the first signal, the second signal, and an imaging magnification of the optical system.

15. A program causing a computer to execute the processing method according to claim 14.

Images