Solid-state imaging device

Abstract

A solid-state imaging device 1A comprises a light receiving unit 10A, a readout unit 20, a selection unit 30, a counter group 40A, a processing unit 50, and a control unit 60A. The light receiving unit 10A comprises N pixels 12A. The counter group 40A comprises M counters. Each pixel 12A detects an event and outputs an event signal. Based on a correspondence relationship between each pixel, each event type, and the counter, the selection unit 30 selects a plurality of counters corresponding to the pixel outputting the event signal and the event type, and inputs the event signal into the selected plurality of counters. In this way, an event-driven solid-state imaging device is realized, which can reduce both power consumption and signal transmission bandwidth.

Description

Technical field

[0001] The present invention relates to a solid-state imaging device. State of the art

[0002] A solid-state imaging device generally comprises a light receiving unit in which a plurality of pixels are arranged, each of which outputs a signal in response to an incident light, and based on the output signal from each of the plurality of pixels it is possible to capture an image of the light incident on the light receiving unit.

[0003] An imaging method is also known that is performed using compressive sensing (Patent Document 1, Non-Patent Document 1). In the imaging method described above, for each of a plurality of mask patterns (compression patterns), several pixels are selected from the plurality of pixels in the light receiving unit according to the mask pattern, and a sum of the signals output by each of the selected multiple pixels is obtained. By performing the necessary processing based on the signal sums obtained for each of the multiple mask patterns, it is possible to reconstruct the image incident on the light receiving unit.

[0004] The aforementioned method requires sequentially switching through numerous mask patterns to obtain the sum of the output signals. Therefore, the time needed to acquire the data required for image reconstruction is lengthy, and if the image changes rapidly, motion blur can occur. To avoid this, the frame rate must be increased, which in turn increases power consumption.

[0005] An event-driven solid-state imaging device is also known (Patent Documents 2 and 3, Non-Patent Document 2). In the aforementioned solid-state imaging device, each of the multiple pixels arranged in the light-receiving unit outputs an event signal indicating the detection of an event (for example, a change in the incident light intensity) when an event is detected. Each of the multiple pixels does not output the event signal if the event is not detected.

[0006] In the aforementioned solid-state imaging device, only the pixel in which the event is detected asynchronously outputs a position signal, indicating the pixel's position, and the event signal to a processing unit in a subsequent stage. This signal is derived from the number of pixels arranged in the light receiving unit. Furthermore, the processing unit, which receives the position and event signals from the pixel, can track the changes in the image resulting from the incident light. The aforementioned solid-state imaging device is expected to reduce power consumption depending on the image acquisition target. Citation list for patent literature Patent document 1: Published Japanese patent application with publication number 2022-112364 Patent document 2: published Japanese patent application with publication number 2022-548199 Patent document 3: published Japanese patent application with publication number 2021-524705 Non-patented literature Non-Patent Document 1: L. Jacques et al., “CMOS compressed imaging by Random Convolution,” Proceedings: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing (2009) Non-patent document 2: Thomas Finateu et al., “A 1280×720 Back-Illuminated Stacked Temporal Contrast Event-Based Vision Sensor with 4.86 µm Pixels, 1.066GEPS Readout, Programmable Event-Rate Controller and Compressive Data-Formatting Pipeline,” ISSCC 2020 / SESSION 5 / IMAGERS AND ToF SENSORS / 5.10 (2020) Summary of the invention: Technical problem

[0007] In conventional event-driven solid-state imaging devices, when each of the multiple pixels detects the event, the position signal and the event signal are transmitted to the processing unit of the subsequent stage, requiring a large bandwidth for signal transmission.

[0008] The aim of one embodiment is to provide an event-driven solid-state imaging device that can reduce both power consumption and signal transmission bandwidth. Solution to the problem

[0009] One embodiment is a solid-state imaging device. The solid-state imaging device comprises (1) a light receiving unit in which N pixels are arranged, each N pixel serving to detect K types of events based on the incident light and to output an event signal indicating an event detection; (2) a counter group with M counters, each serving to count an event detection count based on the input of the event signal (where 1 < M < NK); and (3) a selection unit for selecting, based on a correspondence relationship between each pixel from the N pixels, each event type from the K types of events, and a plurality of counters from the M counters, wherein the plurality of counters corresponds to the pixel that outputs the event signal and the event type, and for injecting the event signal into the selected plurality of counters. Advantageous effects of the invention

[0010] According to the solid-state imaging device of the embodiment, it is possible to provide an event-driven type solid-state imaging device that can reduce both power consumption and signal transmission bandwidth. Brief description of the drawings [ Fig. 1] Fig. Figure 1 is a diagram illustrating a configuration of a solid-state imaging device 1A according to a first embodiment. [ Fig. 2] Fig. Figure 2 is a diagram illustrating a configuration of each pixel 12A of the solid-state imaging device 1A according to the first embodiment. [ Fig. 3] Fig. Figure 3 is a diagram describing a frame and a subframe. [ Fig. 4] Fig. Figure 4 is a diagram illustrating a configuration of a solid-state imaging device 1B according to a second embodiment. [ Fig. 5] Fig. Figure 5 is a diagram illustrating a configuration of each area 11B of the solid-state imaging device 1B according to the second embodiment. [ Fig. 6] Fig. Figure 6 is a diagram illustrating a configuration of each pixel 12B of the solid-state imaging device 1B according to the second embodiment. [ Fig. 7] Fig. Figure 7 is a diagram illustrating a configuration of each pixel 12C of a solid-state imaging device according to a third embodiment. Description of embodiments

[0011] The following describes in detail embodiments of a solid-state imaging device with reference to the accompanying drawings. In the description of the drawings, identical elements are designated with the same reference numerals, and redundant descriptions are omitted. The present invention is not limited to these examples, and the claims, their equivalents, and any modifications within the scope are to be understood as falling within the scope of the present invention.

[0012] Fig. Figure 1 is a diagram illustrating a configuration of a solid-state imaging device 1A according to a first embodiment. The solid-state imaging device 1A comprises a light receiving unit 10A, a readout unit 20, a selection unit 30, a counter group 40A, a processing unit 50, and a control unit 60A. The light receiving unit 10A comprises N pixels 12A arranged two-dimensionally (in the Fig. (Rows and 10 columns). Furthermore, the N pixels 12A can be arranged one-dimensionally. The counter group 40A comprises M counters. The light receiving unit 10A, the readout unit 20, the selection unit 30, the counter group 40A, and the processing unit 50 each operate under the control of the control unit 60A.

[0013] Fig. Figure 2 is a diagram illustrating the configuration of each of the pixels 12A of the solid-state imaging device 1A according to the first embodiment. This diagram describes the relationship between each of the pixels 12A and the readout unit 20 and the control unit 60A, in addition to describing the configuration of each of the pixels 12A.

[0014] The N Pixel 12A have a common configuration. Each Pixel 12A comprises a photodiode 81, an IV conversion unit 82, an amplifier 83, a comparator unit 84, and an output unit 85A. The photodiode 81 generates charges in response to incident light and outputs a current signal to the IV conversion unit 82. The magnitude of the current signal output by the photodiode 81 (the amount of charge generated per unit of time) corresponds to the intensity of the light incident on the photodiode 81.

[0015] The IV conversion unit 82 inputs the current signal output by the photodiode 81 and outputs a voltage signal with a value corresponding to the input current value to the amplifier 83. The amplifier 83 inputs the voltage signal output by the IV conversion unit 82, amplifies the input voltage signal, and outputs the amplified voltage signal to the comparator unit 84.

[0016] The comparator 84 inputs the voltage signal output by the amplifier 83 and compares the magnitudes of the input voltage value V. in and a reference voltage value V ref In the case where the input voltage value V in is greater than a value obtained by adding a first threshold value V th1 to the reference voltage value V ref is obtained (V ref + V th1 < V in The comparison unit 84 determines that a positive event occurs. If the input voltage value V in is smaller than a value obtained by subtracting a second threshold V th2 from the reference voltage value V ref is obtained (V in < V ref - V th2The comparison unit 84 determines that a negative event has occurred. When the comparison unit 84 detects that either the positive or negative event has occurred, an event signal indicating the occurrence of the event is sent to the output unit 85A.

[0017] The reference voltage value V ref In the comparator unit 84, the reference voltage value V is reset by a command from the control unit 60A at the time the solid-state imaging device 1A is switched on and updated for each predefined image period. ref The reference voltage value V is updated in the comparison unit 84 when the event signal is output to the output unit 85A. This update changes the input signal value Vin, which is set at that time, to the new reference voltage value V. ref determined.

[0018] When the event signal is received by the comparator unit 84, the output unit 85A sets a Y-Req line connected to the pixel to high and notifies the control unit 60A of an event signal output request. When a notification of an event signal output acknowledgment is received from the control unit 60A via a Y-Ack line, the output unit 85A outputs the event signal to the readout unit 20 via a positive or negative event line and sets the Y-Req line to high-Z. By using the above configuration, even if event signal output requests are issued simultaneously by a large number of pixels, adjustment of the event signal output acknowledgment can be made, thus allowing the event signal to be read out while avoiding an event signal output collision.

[0019] For each row of the plurality of pixels 12A from the N pixels 12A arranged two-dimensionally in the light receiving unit 10A, a Y-Req line and a Y-Ack line are provided. For each column of the plurality of pixels 12A from the N pixels 12A arranged two-dimensionally in the light receiving unit 10A, a positive event line and a negative event line are provided.

[0020] Back to Fig. The explanation continues below. The readout unit 20 receives the output signals from the pixels 12A in the row that receives the notification of the event signal output confirmation from the control unit 60A, from the N pixels 12A that are arranged two-dimensionally in the light receiving unit 10A, via the positive event line or the negative event line. In addition, the readout unit 20 identifies the pixel 12A that is outputting the event signal from among the pixels and outputs the position information in the X direction of the pixel and the event signal to the control unit 60A.

[0021] Based on the position information in the X direction of the pixel and the event signal received by the readout unit 20, as well as the position information in the Y direction of the pixel received by the light receiver unit 10A via the Y-Req line, the control unit 60A can obtain the XY position information (a pixel address) of the pixel that detects the event and also the type of event detected in the pixel (whether it is a plus event or a minus event).

[0022] The selection unit 30 selects, based on a correspondence relationship between each pixel from the N pixels 12A in the light receiving unit 10A and each event information, as well as a plurality of counters from the M counters in the counter group 40A, the plurality of counters corresponding to the pixel outputting the event signal and the event type, and inputs the event signal into the selected plurality of counters. The selection unit 30 can be configured to include a decoding circuit provided for selecting the appropriate plurality of counters based on the XY position information (the pixel address) of each pixel and the event type.

[0023] In the present embodiment, the two event types, plus event and minus event, are provided as event types, and thus the selection unit 30 selects the appropriate plurality of counters based on the XY position information (the pixel address) of each pixel and the event type (whether it is the plus event or the minus event). In other words, each counter contained in the counter group 40A inputs the event signal of the corresponding event type from the corresponding pixel. Fig. For example, 1 represents the pixels that correspond to a specific counter contained in counter group 40A, marked by hatching.

[0024] The mapping relationship between the pixel, the event type, and the counter can be predefined as an optimal combination and can be further defined using a Hadamard matrix or similar method. Each counter can be provided according to the pixels in a specific area of ​​the light receiving unit 10A. Furthermore, each event type of each pixel must correspond to a multitude of counters.

[0025] Each counter in counter group 40A counts the event detection count based on the event signal from the corresponding pixel. Each counter in counter group 40A outputs the count value to processing unit 50 at the end of each frame period and then resets the count value.

[0026] The processing unit 50 reconstructs a change in the image of the light incident on the light receiving unit 10A based on the correspondence relationship between the pixel, the event type, and the counter in the selection unit 30 and the count value of each counter in the counter group 40A. In this case, the processing unit 50 can be configured, for example, using a digital circuit or the like, in which an image reconstruction algorithm using compressive sampling is implemented.

[0027] Furthermore, the processing unit 50 can be configured to perform inference using a machine learning unit, based on the correspondence between the pixel, the event type, and the counter in the selection unit 30 and the count value of each counter in the counter group 40A. Through the above inference, it may be possible to reconstruct or evaluate the change in the image of the light incident on the light receiving unit 10A, and furthermore, it may be possible to determine the state of the imaging target (for example, the presence of a fallen person in the image or the like) from the evaluation result. The machine learning unit can be configured to include a computer chip in which an inference model, such as a neural network or reservoir computing, is implemented.

[0028] In the case where the processing unit 50 reconstructs the change in the image of the light incident on the light receiving unit 10A using compression scanning technology, the processing unit 50 performs the following processing.

[0029] If the measurement data (the M-counter values) are represented by a vector y, the image to be reconstructed is represented by a vector x, and a sensor matrix is ​​set on Φ, the relationship between these is represented by the following formula (1). An element y m of the measurement data vector y (the following formula (2)) is the count value y m of the m-th numerator from the M numerators. An element x n,kThe image vector x (the following formula (3)) represents the number of occurrences of the k-th event type out of the K event types in the n-th pixel, where the position of each of the N two-dimensionally arranged pixels is represented by a one-dimensional variable n. In the present embodiment, K = 2. [Formula 1] y=Φx [Formula 2] y=(y1,y1,⋯,yM) [Formula 3] x=(x1,1,x1,2,⋯,x1,K,x2,1,⋯,xN,K)

[0030] An element Φ m,n,k The acquisition matrix Φ (the following formula (4)) takes on different values ​​depending on whether the m-th counter, the n-th pixel, and the k-th event in the selection unit 30 have a correspondence relationship. For example, if the m-th counter, the n-th pixel, and the k-th event have a correspondence relationship, it will be set to Φ m,n,k≠ 0. However, if the m-th counter, the n-th pixel, and the k-th event have no correspondence, it is set to Φ. m,n,k = set to 0. [Formula 4] Φ=(ϕ1,1,1ϕ1,1,2⋯ϕ1,1,Kϕ1,2,1⋯ϕ1,N,Kϕ2,1,1ϕ2,1,2⋯ϕ2,1, Kϕ2,2,1⋯ϕ2,N,K⋮⋮⋱⋮⋮⋱⋮ϕM,1,1ϕM,1,2⋯ϕM,1,KϕM,2,1⋯ϕM,N,K)

[0031] With respect to the number of elements NK of the image vector x to be reconstructed, the number of elements M of the measurement data vector y is small, and there is a relationship 1 < M < NK. The reconstruction of the image vector x is an underdetermined problem. Therefore, the formula to be solved becomes an optimization problem, represented by the following formula (5) or formula (6). [Formula 5] minx∈RNK12‖Φx−y‖22+λ‖x‖0 [Formula 6] minx∈RNK12‖Φx−y‖22+λ‖x‖1

[0032] Solving the optimization problem mentioned above allows the reconstruction of the image vector x. In the case of an imaging goal where events occur in only a small number of pixels, sparsity is assumed, thus enabling the optimization problem to be solved. Furthermore, the matrix to be obtained can be transformed, for example, by total variation or similar methods to make it sparse.

[0033] For example, if the number of pixels N is set to 1024 × 1024 = 1,048,576, the number of event types K is set to 2, and the number of counters that have a correspondence relationship with respect to each pixel and each event type is set to 16, the total number of counters M can be set to 262,144. In a conventional event-driven solid-state imaging device, it is necessary to transmit information about the K event types for each of the N pixels to the processing unit.

[0034] On the other hand, in the solid-state imaging device of the present embodiment, it is sufficient to transmit information about the counter values ​​of the M (= 262,144) counters to the processing unit, thereby reducing the signal transmission bandwidth compared to the conventional device. Furthermore, the solid-state imaging device of the present embodiment is of an event-driven type, so power consumption can be reduced. If the image acquisition object is sparse, the number of counters M can be even lower.

[0035] It is also possible to add further event information. For example, the 60A control unit can set a predetermined fixed period in a frame and further divide this into smaller subframes using a timer. Fig. Figure 3 is a diagram describing the frame and the subframe. In this diagram, a positive event is indicated by an upward arrow and a negative event by a downward arrow.

[0036] As shown in this diagram, if each frame is divided into three subframes, there are two types of events (the plus event and the minus event) for each of the three subframes, so the number of event types K for each frame is 6. The image vector x to be reconstructed becomes a vector containing subframe information for each pixel, and the temporal resolution is improved.

[0037] Next, a solid-state imaging device 1B of a second embodiment is described with reference to the Fig. 4 to Fig. Section 6 describes the main differences compared to the configuration of the solid-state imaging device 1A according to the first embodiment.

[0038] Fig. Figure 4 is a diagram illustrating a configuration of a solid-state imaging device 1B according to a second embodiment. The solid-state imaging device 1B comprises a light receiving unit 10B, a processing unit 50, and a control unit 60B. The light receiving unit 10B is divided into a plurality of areas 11B, which are arranged two-dimensionally (in the Fig. (Rows and 10 columns). Furthermore, the multitude of areas 11B can be arranged one-dimensionally. The light receiving unit 10B and the processing unit 50 each operate under the control of the control unit 60B.

[0039] Fig. Figure 5 is a diagram illustrating a configuration of each of the areas 11B of the solid-state imaging device 1B according to the second embodiment. The area 11B comprises a plurality of pixels 12B arranged two-dimensionally (in the Fig. Rows and 8 columns). In addition, the pixels 12B can be arranged one-dimensionally. For each area 11B, a counter group 40B with a multitude of counters 41 is provided.

[0040] In this embodiment, the selection unit comprises a fixed wiring line 30B for electrically connecting each pixel to a plurality of counters corresponding to the pixel. In this diagram, the pixels connected to each counter 41 in the counter group 40B by the fixed wiring line are indicated by hatching.

[0041] The mapping between the pixel, the event type, and the counter via the fixed wiring line 30B can be predefined as an optimal combination and can be further defined using a Hadamard matrix or the like. The number of fixed wiring lines 30B corresponds to the number of counters 41. For example, to avoid a collision of the event signal output between pixels 12B, a function for prioritizing adjacent top, left, and left-top pixels, and the like, can be added to each area 11B.

[0042] Fig. Figure 6 is a diagram illustrating a configuration of each of the pixels 12B of the solid-state imaging device 1B according to the second embodiment. Pixel 12B comprises the photodiode 81, the IV conversion unit 82, the amplifier 83, the comparator unit 84, and an output unit 85B. In comparison to the configuration of pixel 12A of the first embodiment, shown in Figure 6, the configuration of pixel 12B in the second embodiment is as follows: Fig. As shown in 2, pixel 12B of the second embodiment differs, which is shown in Fig. 6 is shown by the fact that the pixel includes output unit 85B instead of output unit 85A.

[0043] Output unit 85B does not need to send the event signal output request to control unit 60B via the Y-Req line, nor does it need to receive the event signal output acknowledgment notification via the Y-Ack line. The counter value of each area 11B is reset upon power-up, and the counters of all areas are read sequentially for each frame using either a rolling shutter or a global shutter configuration, after which the reset is performed again.

[0044] In the case where the processing unit 50 reconstructs the change in the image of the light incident on the light receiving unit 10B using compression sampling, it suffices to solve the optimization problem using formulas (1) to (6) above for the entire light receiving unit 10B, for each of the number of areas 11B, or for any combination of the number of areas 11B. For example, the number N of pixels 12B contained in each area 11B can be set to 8 × 8 = 64, the number of event types K can be set to 2, and the number of counters that have a correspondence relationship with respect to each pixel and each event type in each area 11B can be set to 16. Furthermore, the number of areas 11B can be set to 128 × 128.

[0045] In the solid-state imaging device of the present embodiment, it is sufficient to transmit information about the 262,144 counter values ​​to the processing unit, thus reducing the signal transmission bandwidth compared to the conventional device. Furthermore, the solid-state imaging device of the present embodiment is of an event-driven type, which reduces power consumption. If the imaging target is sparser, the number of counters M can be even lower.

[0046] Next, a solid-state imaging device of a third embodiment is described. The main differences compared to the configuration of the solid-state imaging device 1B according to the second embodiment are described.

[0047] In the solid-state imaging device 1A of the first embodiment and the solid-state imaging device 1B of the second embodiment, described above, each pixel detects the increase or decrease in the incident light intensity as an event. In the solid-state imaging device according to the third embodiment, however, each pixel detects an increase in the amount of light charge accumulated in a capacitor contained within the pixel as an event. Compared to the configuration of the solid-state imaging device 1B of the second embodiment, the solid-state imaging device of the third embodiment differs in the configuration of each pixel.

[0048] Fig.Figure 7 is a diagram illustrating the configuration of each pixel 12C of the solid-state imaging device according to the third embodiment. The pixel 12C comprises a photodiode 91, a transistor 92, a capacitor 93, a comparator 94, and an output 95. An anode of the photodiode 91 is connected to a ground terminal. A cathode of the photodiode 91 is connected via the transistor 92 to a power supply terminal, via the capacitor 93 to the ground terminal, and further connected to one of two inputs of the comparator 94.

[0049] The reference unit 94 gives a potential V c The potential of capacitor 93 (the cathode potential of photodiode 91) is fed into one input and outputs a threshold potential V. th at the other input and compares the magnitudes of the two potentials mentioned above V c and V th Is the potential V chigher than the threshold potential V th , the comparator unit 94 sets the potential of an output terminal to level L, and the potential V c equal to or lower than the threshold potential V th The comparator unit 94 sets the potential of the output terminal to level H. The output terminal of the comparator unit 94 is connected to the output unit 95 and also to a gate of transistor 92.

[0050] In pixel 12C, which has the configuration described above, the accumulated charge in capacitor 93 is reset by a command from the control unit at the time of power-up and is also updated for each predefined image period. The negative charges generated in photodiode 91 in response to the incident light are accumulated in capacitor 93.

[0051] Due to the accumulation of the negative charges described above, the potential V decreases c gradually decreases until it finally falls below the threshold potential V th falls. If the potential V c below the threshold potential V th When the voltage drops, the output terminal of comparator 94 changes from a low level to a high level, thereby switching transistor 92 into the ON state. When transistor 92 is switched into the ON state, the amount of charge accumulated in capacitor 93 is reset, and the output terminal of comparator 94 returns from a high level to a low level.

[0052] This means that each time a specific amount of charge is generated in the photodiode 91, a pulse is output from the output terminal of the comparator unit 94. The pulse generation frequency corresponds to the intensity of the incident light. The aforementioned pulse is output to the counter via the output unit 95 as an event signal indicating the increase in the amount of light charge accumulated in the capacitor 93. In the present embodiment, the number of event types K is set to 1.

[0053] In the case where the processing unit reconstructs the image of the light incident on the light receiving unit using compression sampling, it suffices to solve the optimization problem using the above formulas (1) to (6) for the entire light receiving unit, for each of the multiple areas, or for any combination of the multiple areas. For example, the number N of pixels contained in each area 12C can be set to 8 × 8 = 64, the number of event types K can be set to 1, and the number of counters that have a correspondence relationship with respect to each pixel in each area can be set to 8. Furthermore, the number of areas can be set to 128 × 128.

[0054] In the solid-state imaging device of the present embodiment, it is sufficient to transmit information on 131,072 counter values ​​to the processing unit, thus reducing the signal transmission bandwidth compared to the conventional device. Furthermore, the solid-state imaging device of the present embodiment is of an event-driven type, which reduces power consumption. If the imaging target is sparser, the number of counters M can be even lower.

[0055] The solid-state imaging device is not limited to the embodiments and configuration examples described above; various modifications are possible. For example, the event detected by each pixel can be defined as any event. Each pixel can detect the event related to the intensity of the light incident on the pixel, and additionally, the event related to a polarization state or wavelength of the light. Each pixel can detect the event related to the amount of charge accumulated in the pixel, and additionally, the event related to a current or voltage signal in the pixel.

[0056] The processing unit need not necessarily be located within the solid-state imaging device and can be a hardware device comprising a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system-on-a-chip (SoC), or the like, connected to the solid-state imaging device. Alternatively, the processing unit can be a cloud device connected to the solid-state imaging device via a network.

[0057] The solid-state imaging device of a first aspect according to the above embodiment comprises (1) a light receiving unit in which N pixels are arranged, wherein each of the N pixels serves to detect K types of events based on the incident light and to output an event signal indicating an event detection; (2) a counter group with M counters, each serving to count an event detection count based on the input of the event signal (where 1 < M < NK); and (3) a selection unit for selecting, based on a correspondence relationship between each pixel from the N pixels, each event type from the K types of events, and a plurality of counters from the M counters, wherein the plurality of counters corresponds to the pixel that outputs the event signal and the event type, and for injecting the event signal into the selected plurality of counters.

[0058] In the solid-state imaging device of a second aspect, the selection unit in the configuration of the first aspect may include a circuit for selecting the plurality of counters corresponding to each pixel based on position information of the pixel.

[0059] In the solid-state imaging device of a third aspect, the selection unit in the configuration of the first aspect may include a fixed wiring line for electrically connecting each pixel and the plurality of counters corresponding to the pixel.

[0060] In the solid-state imaging device of a fourth aspect, in the configuration of any of the first to third aspects, the light receiving unit can be divided into a multitude of areas, and the counter group can be provided for each of the multitude of areas.

[0061] In the solid-state imaging device of a fifth aspect, in the configuration of any of the first to fourth aspects, each of the N pixels can capture the event associated with the light incident on the pixel.

[0062] In the solid-state imaging device of a sixth aspect, in the configuration of any of the first to fifth aspects, each of the N pixels can detect the event with respect to an electrical signal in the pixel.

[0063] In the solid-state imaging device of a seventh aspect, the device in the configuration of any of the first to sixth aspects may further include a processing unit for reconstructing an image of the light incident on the light receiving unit or a change of the image using a compression scanning technique based on the correspondence relationship between the pixel and the counter in the selection unit and a count value of each of the M counters.

[0064] In the solid-state imaging device of an eighth aspect, the device in the configuration of any of the first to sixth aspects may further include a processing unit for performing inference using a machine learning unit based on the correspondence relationship between the pixel and the counter in the selection unit and a count value of each of the M counters. Industrial applicability

[0065] The embodiments can be used as an event-driven solid-state imaging device that can reduce both power consumption and signal transmission bandwidth. List of reference symbols

[0066] 1A, 1B - Solid-state imaging device, 10A, 10B - Light receiving unit, 11B - Area, 12A - 12C - Pixel, 20 - Readout unit, 30 - Selection unit, 40A, 40B - Counter group, 41 - Counter, 50 - Processing unit, 60A, 60B - Control unit, 81 - Photodiode, 82 - IV conversion unit, 83 - Amplifier, 84 - Comparator unit, 85A, 85B - Output unit, 91 - Photodiode, 92 - Transistor, 93 - Capacitor, 94 - Comparator unit, 95 - Output unit, 30B - Fixed wiring line. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2022-112364

[0006] JP 2022-548199

[0006] JP 2021-524705

[0006] Cited non-patent literature

[0000] L. Jacques et al., “CMOS compressed imaging by Random Convolution,” Proceedings: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing (2009)

[0006] Thomas Finateu et al., „A 1280×720 Back-Illuminated Stacked Temporal Contrast Event-Based Vision Sensor with 4.86 µm Pixels, 1.066GEPS Readout, Programmable Event-Rate Controller and Compressive Data-Formatting Pipeline“, ISSCC 2020 / SESSION 5 / IMAGERS AND ToF SENSORS / 5.10 (2020)

[0006]

Claims

Solid-state imaging device comprising: a light receiving unit in which N pixels are arranged, each of the N pixels serving to detect K types of events based on an incident light and to output an event signal indicating an event detection; a counter group comprising M counters, each serving to count an event detection count based on an input of the event signal (where 1 < M < NK); and a selection unit for selecting each event type from the K types of events and a plurality of counters from the M counters based on a correspondence relationship between each pixel from the N pixels, the plurality of counters corresponding to the pixel that outputs the event signal and the event type, and for feeding the event signal into the selected plurality of counters. Solid-state imaging device according to claim 1, wherein the selection unit comprises a circuit for selecting the plurality of counters corresponding to each pixel based on position information of the pixel. Solid-state imaging device according to claim 1, wherein the selection unit comprises a fixed wiring line for electrically connecting each pixel and the plurality of counters corresponding to the pixel. Solid-state imaging device according to one of claims 1 to 3, wherein the light receiving unit is divided into a plurality of areas and the counter group is provided for each of the plurality of areas. Solid-state imaging device according to any one of claims 1 to 4, wherein each of the N pixels captures the event associated with the light incident on the pixel. Solid-state imaging device according to any one of claims 1 to 5, wherein each of the N pixels captures the event that is associated with an electrical signal in the pixel. Solid-state imaging device according to any one of claims 1 to 6, further comprising a processing unit for reconstructing an image of the light incident on the light receiving unit or of a change in the image using a compression scanning technique, based on the correspondence relationship between the pixel and the counter in the selection unit and a count value of each of the M counters. Solid-state imaging device according to any one of claims 1 to 6, further comprising a processing unit for performing inference using a machine learning unit based on the correspondence relationship between the pixel and the counter in the selection unit and a count value of each of the M counters.

Images