Photoelectric conversion device, mobile device, photoelectric conversion method, storage medium, and computer program product
By setting short and long accumulation periods in the photoelectric conversion device and outputting signals between corresponding periods, the problem of low recognition rate of fast moving objects under low illumination is solved, and the image effect of fast recognition and high contrast is achieved.
Patent Information
- Application Number
- CN202410679677.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-30
- Filing Date
- 2024-05-29
- Publication Date
- 2024-12-03
AI Technical Summary
The existing photoelectric conversion device is difficult to quickly identify rapidly moving objects under low illumination conditions, resulting in a decrease in recognition rate, especially in vehicle-mounted cameras, where the subject is blurred due to the accumulated period.
A photoelectric conversion device is designed to use a counter to output a signal by setting a short first accumulation period and a long second accumulation period in a full frame period and outputting a signal between the end of the first accumulation period and the end of the second accumulation period. Memory generates signals to achieve fast image recognition.
Image recognition after the shortest full frame quarter length is achieved, which improves the recognition speed of fast moving obstacles, reduces subject blur, and enhances image contrast at low illumination.
Smart Images

Figure CN119071660A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a photoelectric conversion device, a movable device, a photoelectric conversion method and a storage medium. Background Art
[0002] In recent years, a photoelectric conversion device has been developed that digitally counts the number of photons incident on an avalanche photodiode (APD) and outputs the count value from a pixel as a photoelectrically converted digital signal.
[0003] Japanese Patent No. 7223070 discloses a configuration of a photoelectric conversion device including an APD, which can output a plurality of images whose accumulation periods overlap with each other, thus enabling continuous imaging even under low illumination.
[0004] However, in the configuration disclosed in Japanese Patent No. 7223070, for example, when an imaging device for a vehicle-mounted camera installed in a mobile device is assumed, recognition processing is performed frame by frame in a normal sensor drive, and thus, for example, recognition processing can only be performed every 33.3 ms in the case of 30 fps. Therefore, in the vehicle-mounted camera, if an object jumps in immediately after the frame has changed, recognition processing cannot be performed until the current frame ends.
[0005] Many on-vehicle cameras have an accumulation period equal to or greater than a predetermined period (11ms) to suppress flickering due to traffic signals, and especially in low illumination, bright images are captured by extending the accumulation period. However, due to the long accumulation period, subject blur occurs in fast-moving objects, thereby reducing their recognition rate. Summary of the invention
[0006] According to one aspect of the present invention, a photoelectric conversion device is provided, which includes: a plurality of pixels, each of which includes: a sensor portion, which is configured to emit pulses corresponding to incident photons, a counter, which is configured to count the number of pulses, and a memory, which is configured to store the count value of the counter; one or more memories for storing instructions; and one or more processors for executing the instructions to generate a signal based on the difference in the count value of the counter between the start timing and the end timing of the accumulation period, and output the signal generated in the first accumulation period between the end of the first accumulation period and the end of the second accumulation period, the first accumulation period and the second accumulation period being included in the full frame period, and the first accumulation period being shorter than the second accumulation period.
[0007] Further features of the present invention will become apparent from the following description of embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 is a diagram showing an example of the configuration of a photoelectric conversion device according to an embodiment of the present invention.
[0009] Figure 2 is a diagram showing an example of the configuration of the sensor board 11 .
[0010] Figure 3 2 is a diagram showing an example of the configuration of the circuit board 21 .
[0011] Figure 4 It is shown Figure 2 and Figure 3 1 is a diagram of an equivalent circuit of a pixel 101 and a signal processing circuit 103 corresponding to the pixel 101.
[0012] Figure 5 is a diagram schematically showing the relationship between the operation of the APD 201 and the output signal.
[0013] Figure 6 is a functional block diagram of a photoelectric conversion apparatus 600 and a movable device 700 according to the present embodiment.
[0014] Figure 7 is a diagram illustrating a photoelectric conversion method performed by the camera control unit 605 according to the present embodiment.
[0015] Figure 8 is a diagram showing an example of divided images of a plurality of frames.
[0016] Fig. 9 is a diagram showing the relationship between the memory circuit and the buffer according to the present embodiment.
[0017] Fig.10 : is a flowchart specifically showing a driving example of the photoelectric conversion device according to the present embodiment.
[0018] Fig.11 yes Fig.10 A subsequent flowchart of the flowchart shown in . DETAILED DESCRIPTION
[0019] Hereinafter, with reference to the accompanying drawings, an advantageous mode of the present invention will be described using embodiments. In the various drawings, the same reference numerals are applied to the same members or elements, and repeated descriptions will be omitted or simplified.
[0020] Figure 1 1 is a diagram showing an example of a configuration of a photoelectric conversion device according to an embodiment of the present invention. In the following description, the photoelectric conversion device 100 is, for example, a photoelectric conversion device having a so-called stacked structure in which two boards including a sensor board 11 and a circuit board 21 are stacked and electrically connected.
[0021] However, a so-called non-stacked structure may be adopted in which the elements included in the sensor board and the elements included in the circuit board are arranged in a common semiconductor layer. The sensor board 11 includes a pixel area 12. The circuit board 21 includes a circuit area 22 for processing a signal detected from the pixel area 12.
[0022] Figure 2 1 is a diagram showing an example of the configuration of the sensor panel 11. The pixel region 12 of the sensor panel 11 includes a plurality of pixels 101 arranged two-dimensionally in the directions of rows and columns. Each pixel 101 includes a photoelectric conversion element 102 including an avalanche photodiode (hereinafter referred to as APD).
[0023] Here, the photoelectric conversion element 102 functions as a sensor portion for emitting a pulse corresponding to the photon reception frequency. The number of rows and the number of columns in the pixel array constituting the pixel region 12 are not particularly limited.
[0024] Figure 3 2 is a diagram showing an example of the configuration of the circuit board 21. The circuit board 21 includes a circuit board for processing Figure 2 A signal processing circuit 103 for the charge photoelectrically converted by the corresponding photoelectric conversion element 102, a reading circuit 112, a control pulse generator 115, a horizontal scanning circuit 111, a vertical signal line 113, a vertical scanning circuit 110 and an output circuit 114.
[0025] The vertical scanning circuit 110 receives the control pulse supplied from the control pulse generator 115 and sequentially supplies the control pulse to a plurality of pixels arranged in the row direction. A logic circuit such as a shift register or an address decoder is used for the vertical scanning circuit 110 .
[0026] The signal output from the photoelectric conversion element 102 of each pixel is processed by the corresponding signal processing circuit 103. The signal processing circuit 103 is provided with a counter or a memory, etc., and the digital value is stored in the memory. The horizontal scanning circuit 111 inputs a control pulse for sequentially selecting a column to the signal processing circuit 103 to read the signal from the memory of the pixel storing the digital signal.
[0027] The signal from the signal processing circuit 103 of the pixel in the row selected by the vertical scanning circuit 110 is output to the vertical signal line 113. The signal output to the vertical signal line 113 is output to the outside of the photoelectric conversion device 100 via the reading circuit 112 and the output circuit 114. A plurality of buffers connected to the vertical signal line 113 are incorporated in the reading circuit 112.
[0028] like Figure 2 and Figure 3As shown, in a plan view, a plurality of signal processing circuits 103 are arranged in a region overlapping with the pixel region 12. In a plan view, a vertical scanning circuit 110, a horizontal scanning circuit 111, a reading circuit 112, an output circuit 114, and a control pulse generator 115 are arranged to overlap between one end of the sensor panel 11 and one end of the pixel region 12.
[0029] In other words, the sensor panel 11 includes a pixel area 12 and a non-pixel area arranged around the pixel area 12. In a plan view, a vertical scanning circuit 110, a horizontal scanning circuit 111, a reading circuit 112, an output circuit 114, and a control pulse generator 115 are arranged in an area overlapping the non-pixel area.
[0030] The arrangement of the vertical signal lines 113 and the arrangement of the readout circuits 112 and the output circuits 114 are not limited to Figure 3 For example, the vertical signal line 113 may be arranged to extend in the row direction, and the reading circuit 112 may be arranged in front of the extending direction of the vertical signal line 113. The signal processing circuit 103 does not necessarily have to be provided for each photoelectric conversion element, and there may be a configuration in which one signal processing circuit is shared by a plurality of photoelectric conversion elements and sequential signal processing is performed on the one signal processing circuit.
[0031] Figure 4 It is shown Figure 2 and Figure 3 1 is a diagram of an equivalent circuit of a pixel 101 and a signal processing circuit 103 corresponding to the pixel 101.
[0032] The APD 201 included in each photoelectric conversion element 102 generates a charge pair corresponding to the incident light by photoelectric conversion. One of the two nodes of the APD 201 is connected to a power supply line for supplying a driving voltage VL (first voltage). The other of the two nodes of the APD 201 is connected to a power supply line for supplying a driving voltage VH (second voltage) higher than the voltage VL.
[0033] exist Figure 4 In the embodiment, one node of the APD 201 is an anode and the other node of the APD 201 is a cathode. The anode and cathode of the APD 201 are supplied with a reverse bias voltage, which is used to enable the APD 201 to perform an avalanche multiplication operation. By supplying the reverse bias voltage, the charge generated by the incident light causes avalanche multiplication and generates an avalanche current.
[0034] When a reverse bias voltage is supplied, there is a Geiger mode and a linear mode, wherein in the Geiger mode, the voltage difference between the anode and the cathode is greater than the breakdown voltage, and in the linear mode, the voltage difference between the anode and the cathode is a voltage difference close to the breakdown voltage or equal to or less than the breakdown voltage. An APD operating in Geiger mode is called a SPAD. For example, in a SPAD, the voltage VL (first voltage) is -30V, and the voltage VH (second voltage) is 1V.
[0035] The signal processing circuit 103 includes a quenching element 202, a waveform shaping section 210, a counter circuit 211, and a memory circuit 212. The quenching element 202 is connected to a power supply line for supplying a driving voltage VH and one node of an anode and a cathode of the APD 201.
[0036] The quenching element 202 serves as a load circuit (quenching circuit) when a signal is multiplied by avalanche multiplication, and has a function of suppressing the voltage supplied to the APD 201 to suppress avalanche multiplication (quenching operation). The quenching element 202 also has a function of returning the voltage supplied to the APD 201 to the driving voltage VH by causing a current corresponding to the voltage drop caused by the quenching operation to flow through the quenching element 202 (recharging operation).
[0037] exist Figure 4 , an example is shown in which the signal processing circuit 103 includes a waveform shaping section 210 , a counter circuit 211 , and a memory circuit 212 in addition to the quenching element 202 .
[0038] The waveform shaping section 210 shapes the voltage change of the cathode of the APD 201 acquired when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping section 210. Figure 4 In the embodiment, one inverter is used as the waveform shaping section 210, but a circuit in which a plurality of inverters are connected in series may be used, or other circuits having a waveform shaping effect may be used.
[0039] The counter circuit 211 counts the number of pulses output from the waveform shaping section 210 and stores the count value. When the control pulse RES is supplied via the drive line 213, the signal stored in the counter circuit 211 is reset. Here, the counter circuit 211 generates a signal based on the difference in the count value between the start timing and the end timing of the accumulation period.
[0040] via Figure 4 The driving line 214 ( Figure 3 (not shown) from Figure 3The vertical scanning circuit 110 shown supplies a control pulse SEL to the memory circuit 212 to switch between electrical connection and disconnection between the counter circuit 211 and the vertical signal line 113. The memory circuit 212 serves as a memory for temporarily storing a count value of the counter, and outputs an output signal from the counter circuit 211 of the corresponding pixel to the vertical signal line 113.
[0041] A switch such as a transistor may be provided between the quenching element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing circuit 103 to switch the electrical connection therebetween. Similarly, a switch such as a transistor may be used to electrically switch the supply of the voltage VH or the voltage VL to the photoelectric conversion element 102.
[0042] Figure 5 2 is a diagram schematically showing the relationship between the operation and output signal of the APD 201. The input side of the waveform shaping section 210 is defined as a node A, and the output side of the waveform shaping section 210 is defined as a node B. From time t0 to time t1, a potential difference of VH-VL is applied to the APD 201. When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows in the quenching element 202, and the voltage of the node A drops.
[0043] When the voltage drop increases and the potential difference applied to the APD 201 decreases, the avalanche multiplication of the APD 201 stops at time t2, and the voltage level of the node A does not drop by a predetermined value or more.
[0044] Thereafter, between time t2 and time t3, a current for compensating the voltage drop from the voltage VL flows in the node A, and the node A is corrected to the original potential level at time t3. At this time, the portion of the output waveform exceeding the predetermined threshold value in the node A is shaped by the waveform shaping section 210, and the pulse signal is output to the node B.
[0045] The photoelectric conversion apparatus 600 and the movable device 700 according to the present embodiment will be described below. Figure 6 1 is a functional block diagram of the photoelectric conversion device 600 and the mobile device 700 according to the present embodiment. The photoelectric conversion device 600 and the mobile device 700 are implemented by causing a computer (not shown) included in the photoelectric conversion device 600 and the mobile device 700 to execute a computer program stored in a memory as a storage medium not shown. Figure 6 Some functional boxes are shown.
[0046] However, part or all of these functional blocks may be realized by hardware. An application-specific circuit (ASIC) or a processor (such as a reconfigurable processor or DSP, etc.) or the like may be used as the hardware. Figure 6The functional blocks shown in FIG. 1 may not be incorporated in the same housing and may be constituted by different devices connected to each other via signal lines.
[0047] The photoelectric conversion device 600 includes the above-mentioned Figures 1 to 5 The photoelectric conversion device 100 described above includes an imaging optical system 601, an image processing unit 603, a recognition unit 604, a camera control unit 605, a storage unit 606, and a communication unit 607. The photoelectric conversion device 100 includes Figures 1 to 5 An avalanche photodiode for photoelectric conversion of an optical image is described.
[0048] The photoelectric conversion device according to the present embodiment is mounted in the movable device 700, and the camera unit including the set of the imaging optical system 601 and the photoelectric conversion device 100 is configured to capture at least one of the front view, the rear view, and the side view of the movable device, for example. A plurality of camera units may be provided in the movable device 700.
[0049] The image processing unit 603 performs image processing such as black level correction, gamma curve adjustment, noise reduction, digital gain adjustment, demosaicing or data compression on the image signal acquired by the photoelectric conversion device 100, and generates a final image signal. When the photoelectric conversion device 100 includes an on-chip color filter such as RGB, it is preferred that the image processing unit 603 performs processing such as white balance correction or color conversion.
[0050] The output of the image processing unit 603 is supplied to the recognition unit 604, the camera control unit 605, and the electronic control unit (ECU) 701 of the movable device 700. The recognition unit 604 recognizes nearby persons or nearby vehicles, etc. by performing image recognition based on the image signal, and issues a warning as necessary.
[0051] The present embodiment is based on an example in which the movable device 700 is a car, but the movable device may be any of an aircraft, a subway train, a ship, a drone, an AGV, and a robot as long as it is movable.
[0052] The camera control unit 605 includes a CPU as a computer and a memory storing a computer program, and controls the configuration of the photoelectric conversion apparatus 600 by causing the CPU to execute the computer program stored in the memory.
[0053] The camera control unit 605 controls the length of the exposure period of each frame of the photoelectric conversion apparatus 100 , the timing of the control signal CLK, and the like, for example, via a control pulse generator of the photoelectric conversion apparatus 100 .
[0054] The storage unit 606 includes a recording medium such as a memory card or a hard disk, and can store and read image signals. The communication unit 607 includes a wireless or wired interface, outputs the generated image signal to the outside of the photoelectric conversion device 600, and receives various types of signals from the outside.
[0055] The ECU 701 includes a CPU as a computer and a memory storing a computer program, and controls the configuration of the movable device 700 by causing the CPU to execute the computer program stored in the memory.
[0056] The output of the ECU 701 is supplied to a vehicle control unit 702 and a display unit 703. The vehicle control unit 702 serves as a mobile control unit configured to perform driving, stopping, and direction control of a vehicle as a movable device, etc., based on the output of the ECU 701. The display unit 703 includes a display device such as a liquid crystal device or an organic EL display, and is mounted in the movable device 700.
[0057] The display unit 703 displays an image acquired by the photoelectric conversion device 100 or various types of information related to the running state of the vehicle and the like to the driver of the movable device 700 using, for example, a GUI based on the output of the ECU 701 .
[0058] Figure 6 The image processing unit 603 and the recognition unit 604 shown may not be installed in the mobile device 700, but may be set in an external terminal that is set separately from the mobile device 700 and is used to remotely control the mobile device 700 or monitor the travel of the mobile device.
[0059] Figure 7 1 is a diagram showing a photoelectric conversion method performed by the camera control unit 605 according to the present embodiment. In the present embodiment, the photoelectric conversion is driven periodically at, for example, 30 full frames / second. A frame corresponding to one vertical period of 33.3 ms is referred to as a full frame, and a quarter of a full frame is referred to as a frame.
[0060] That is to say, Figure 7 As shown, the entire frame 1 from time T0 to time T4 is divided into frames 1_1, 1_2, 1_3, and 1_4 having equal periods (8.33 ms).
[0061] Frame 1_1 includes an accumulation period (photoelectric conversion period) from start time T0 to time T1 in full frame 1, and frame 1_2 includes an accumulation period from time T0 to time T2. Frame 1_3 includes an accumulation period from time T0 to time T3, and frame 1_4 includes an accumulation period from time T0 to time T4.
[0062] The counter circuit 211 is reset at time T0 , and count values C1_1 , C1_2 , C1_3 , and C1_4 are acquired from the counter circuit 211 at time T1 to time T4 .
[0063] The count values C1_1, C1_2, C1_3, and C1_4 are temporarily stored in the memory circuit 212. The signals corresponding to one row temporarily stored in the memory circuit 212 are sequentially output from the photoelectric conversion devices via the buffer of the reading circuit 112.
[0064] In this way, according to the present embodiment, the signal accumulated in the period of frame 1_1 is read from time T1 to time T2, and the signal is quickly processed by the recognition unit 604. Therefore, image recognition can be quickly performed. Similarly, from time T2 to time T3, from time T3 to T4, and from T4 to T1, the signal accumulated in the period of frame 1_2, frame 1_3, and frame 1_4 is sequentially read, and image recognition can be repeatedly performed on the signal.
[0065] Figure 8 is a diagram showing an example of images of a plurality of frames after segmentation. Figure 8 As shown, the image of frame 1_1 is dark because the accumulation period is short, but the image of frame 1_1 has a small subject blur for the person rushing out. On the other hand, the accumulation periods of frame 1_2, frame 1_3, and frame 1_4 are getting longer in this order, so subject blur is likely to occur in this order. Blurring is less likely to occur in a stopped vehicle or a white line, and as the accumulation period becomes longer, the contrast is more likely to increase.
[0066] In this way, in the present embodiment, the first accumulation period and the second accumulation period are set in the full frame, the first accumulation period is shorter than the second accumulation period, and control is performed so that the signal generated in the first accumulation period is output between the end of the first accumulation period and the end of the second accumulation period.
[0067] In this embodiment, the first accumulation period overlaps with the second accumulation period, and the first accumulation period and the second accumulation period start at the same time. The end of the second accumulation period coincides with the end of the full frame, and the second accumulation period is an integer multiple of the first accumulation period.
[0068] Here, the second accumulation period does not necessarily have to be an integral multiple of the first accumulation period as long as the second accumulation period is longer than the first accumulation period (the first accumulation period is shorter than the second accumulation period) and the end of the second accumulation period is later than the end of the first accumulation period.
[0069] That is, an image with a short accumulation period and an image with a long accumulation period are generated, the end timing of the short accumulation period is set to be earlier than the end timing of the long accumulation period, and immediately after the end of the short accumulation period, the image is output to the recognition unit of the subsequent stage. Then, the subject is recognized based on at least the signal generated in the first accumulation period. The recognition unit 604 recognizes the subject based on at least the signal generated in the first accumulation period.
[0070] Therefore, in the related art, image recognition can be performed only after a full frame has passed, but in the present embodiment, image recognition can be performed after a length of at least a quarter of the full frame has passed. For example, when the movable device is moving quickly, obstacles can be quickly recognized. Therefore, rapid braking can be made possible. Alternatively, obstacles can also be avoided in advance.
[0071] An image with a long accumulation period can enhance contrast and thus can be used as a display image. That is, an image with a short accumulation period is suitable for rapid recognition of a subject, and an image with a long accumulation period is suitable for displaying an image. In this way, the display device according to the present embodiment displays a signal generated at least in the second accumulation period as an image.
[0072] In this embodiment, since APD is used, unlike CMOS sensor, the accumulated charge does not deteriorate due to reading, so the accumulation period can overlap. Since there is no read noise, the original signal does not deteriorate even if multiple readings are performed with one accumulation.
[0073] Fig. 9 is a diagram showing the relationship between the memory circuit and the buffer according to the present embodiment. Fig. 9 In Figure 3 The memory circuits 212 in the signal processing circuit 103 shown in the figure are arranged in N rows and M columns, and the memory circuits are referred to as memory 1-1 To Storage N-M . Fig. 9 Buffer 1 to Buffer M instruct Figure 3 The buffer included in the read circuit 112 in. Fig. 9 The output circuit 114 and Figure 3 Corresponding to the output circuit 114 in.
[0074] Fig.10 is a flowchart showing details of a driving example of the photoelectric conversion device according to the present embodiment, and Fig.11 yes Fig.10 The following flowchart is a flowchart of the flowchart of the camera control unit 605. The computer program stored in the memory is executed by the CPU or the like as a computer in the camera control unit 605 to sequentially perform the following steps. Fig.10 and Fig.11 the operations of the steps in the flowchart shown
[0075] In Fig.10 step S101, set i = 1. Then, in step S102, output the count value Count in the counter circuit 211 at time T i to the memory circuit 212. At this time, output the count value to all memory circuits simultaneously. This operation corresponds to Figure 7 the operation at time T1 in
[0076] Then, in step S103, set j = 1. In step S104, output the Fig. 9 count value Count in the memory circuit j-k in j-k-i to the buffer. At this time, in columns 1 to M, output the count value to the buffer simultaneously. This operation corresponds to outputting the count value of Fig. 9 the first row in
[0077] Then, in step S105, set k = 1. In step S106, output the count value Count of the k buffer j-k-i to the output circuit 114. This operation corresponds to reading the signal of the buffer in the leftmost column from the output circuit Fig. 9 in
[0078] Then, the operation process advances via A to Fig.11 step S107, and in step S107, judge whether k < M is satisfied. When the judgment result is "yes", in step S108, increase k by 1 through k = k + 1, and the operation process returns to step S106 via B to perform the operation of step S106. This operation corresponds to reading the signal of the buffer in the second leftmost column from the output circuit Fig. 9 in
[0079] When the judgment result of step S107 is "no" (i.e., when k = M is satisfied), this means that all signals of the M - th column buffer in Fig. 9 have been read from the output circuit, and then in step S109, judge whether j < N is satisfied. When the judgment result of step S109 is "yes", in step S110, increase j by 1 through j = j + 1, and then the operation process returns to step S104 via C. This corresponds to the operation of starting to read the next row
[0080] When the judgment result of step S109 is "No", it means that the reading of all rows has been completed, and thus it is judged in step S111 whether i<4 is satisfied. When the judgment result of step S111 is "Yes", i is increased by 1 by i=i+1, and the operation flow returns to step S102 via D. This operation corresponds to the operation of starting the reading at the next time T2.
[0081] When the judgment result of step S111 is "No", it means that the reading at time T4 has been completed, so the counter circuit 211 is reset by the reset signal in step S113. Figure 7 This corresponds to the operation of resetting the counter circuit 211 at time T4 in . In this way, the signals accumulated in the photoelectric conversion apparatus 100 can be sequentially read.
[0082] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
[0083] For example, in the aforementioned embodiment, accumulation in a full frame period as short as one quarter is performed, but the length of the shortest accumulation period may be changed to, for example, 1 / 5 of a full frame period or 1 / 3 of a full frame period according to the recognition accuracy of the recognition unit 604. Alternatively, the length of the shortest accumulation period may be changed according to the brightness of the subject.
[0084] Even if the reading period is set to one-quarter of the full-frame period, the reading period can be set to one-quarter of the full-frame period depending on the brightness of the subject or the image recognition accuracy. Figure 7 The counter circuit is reset in the accumulation period of frame 1_1 in FIG. As a result, substantially, the accumulation period can be set shorter than a quarter of the full frame period.
[0085] Alternatively, you can Figure 7 The counter circuit is temporarily reset at time T1 in the example. As a result, the count value read at time T4 can be adjusted.
[0086] In addition, as part or all of the control according to the present embodiment, a computer program for realizing the functions of the above-described embodiments may be supplied to the photoelectric conversion device, etc., via a network or various storage media. Then, the computer (or CPU or MPU, etc.) of the photoelectric conversion device, etc. may be configured to read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention.
[0087] Furthermore, for example, the present invention includes those realized using at least one processor or circuit configured to perform the functions of the above-described embodiments. A plurality of processors may be used to perform distributed processing.
[0088] This application claims the benefit of Japanese Patent Application No. 2023-088746, filed May 30, 2023, which is hereby incorporated by reference herein in its entirety.
Claims
1. A photoelectric conversion device, comprising: A plurality of pixels, each comprising: a sensor portion configured to emit pulses corresponding to incident photons, a counter configured to count the number of pulses, and a memory configured to store a count value of the counter; and A control component configured to: generating a signal based on a difference in count value of the counter between a start timing and an end timing of an accumulation period, and A signal generated in a first accumulation period is output between an end of a first accumulation period and an end of a second accumulation period, the first accumulation period and the second accumulation period being included in a full frame period, the first accumulation period being shorter than the second accumulation period.
2. The photoelectric conversion device according to claim 1, wherein The first accumulation period and the second accumulation period overlap with each other.
3. The photoelectric conversion device according to claim 2, wherein: The first accumulation period and the second accumulation period start simultaneously.
4. The photoelectric conversion device according to claim 1, wherein An end timing of the second accumulation period coincides with an end timing of the full frame period. 5 . The photoelectric conversion apparatus according to claim 1 , further comprising a recognition section configured to recognize a subject based on at least a signal generated in the first accumulation period.
6. The photoelectric conversion device according to claim 5, wherein: The identification section additionally identifies the subject based on the signal generated in the second accumulation period. 7 . The photoelectric conversion apparatus according to claim 1 , further comprising a display section configured to display at least the signal generated in the second accumulation period as an image.
8. The photoelectric conversion device according to claim 1, wherein The sensor portion includes an avalanche photodiode.
9. A movable device comprising: A plurality of pixels, each comprising: a sensor portion configured to emit pulses corresponding to incident photons, a counter configured to count the number of pulses, and a memory configured to store the count value of the counter; A control component configured to: generating a signal based on a difference in count value of the counter between a start timing and an end timing of an accumulation period, and outputting a signal generated in a first accumulation period between an end of a first accumulation period and an end of a second accumulation period, the first accumulation period and the second accumulation period being included in a full frame period, the first accumulation period being shorter than the second accumulation period; and A movement control component is configured to control the operation of the movable device.
10. A photoelectric conversion method for performing photoelectric conversion in a plurality of pixels, each of the plurality of pixels including a sensor portion configured to emit a pulse corresponding to an incident photon, a counter configured to count the number of the pulses, and a memory configured to store a count value of the counter, the photoelectric conversion method comprising: generating a signal based on a difference in count value of the counter between a start timing and an end timing of an accumulation period, and A signal generated in a first accumulation period is output between an end of a first accumulation period and an end of a second accumulation period, the first accumulation period and the second accumulation period being included in a full frame period, the first accumulation period being shorter than the second accumulation period.
11. A non-transitory computer-readable storage medium configured to store a computer program, the computer program including instructions for executing a photoelectric conversion process for performing photoelectric conversion in a plurality of pixels, the plurality of pixels each including a sensor portion configured to emit a pulse corresponding to an incident photon, a counter configured to count the number of the pulses, and a memory configured to store a count value of the counter, the photoelectric conversion process comprising: generating a signal based on a difference in count value of the counter between a start timing and an end timing of an accumulation period, and A signal generated in a first accumulation period is output between an end of a first accumulation period and an end of a second accumulation period, the first accumulation period and the second accumulation period being included in a full frame period, the first accumulation period being shorter than the second accumulation period.
12. A computer program product, comprising a computer program, the computer program comprising instructions for executing a photoelectric conversion process for performing photoelectric conversion in a plurality of pixels, the plurality of pixels each comprising a sensor portion configured to emit a pulse corresponding to an incident photon, a counter configured to count the number of the pulses, and a memory configured to store a count value of the counter, the photoelectric conversion process comprising: generating a signal based on a difference in count value of the counter between a start timing and an end timing of an accumulation period, and A signal generated in a first accumulation period is output between an end of a first accumulation period and an end of a second accumulation period, the first accumulation period and the second accumulation period being included in a full frame period, the first accumulation period being shorter than the second accumulation period.
