Solid-state imaging apparatus and electronic device

Abstract

The present technology relates to a solid-state imaging apparatus and an electronic device in which a sensitivity difference between gradation pixels due to color mixing can be suppressed. The solid-state imaging apparatus is provided with a pixel array unit in which a plurality of first pixels for outputting a luminance signal having a gradation level corresponding to the amount of incident light, and a plurality of second pixels, different from the first pixels, for detecting information are regularly arrayed. The second pixels are adjacent to each other by the same amount around each of the first pixels at least in a region excluding an end portion of the pixel array unit. The present technology can be applied to image sensors, for example.

Description

Solid-state Imaging Device and Electronic Apparatus

[0001] The present technology relates to a solid-state imaging device and an electronic apparatus, and particularly to a solid-state imaging device and an electronic apparatus configured to suppress a sensitivity difference between pixels caused by color mixing.

[0002] Conventionally, there has been proposed a solid-state imaging device arranged in a pixel array unit such that grayscale pixels that output luminance signals at grayscale levels corresponding to the amount of incident light and event pixels that detect, as an event, that a luminance change exceeds a predetermined threshold are mixed (see, for example, Patent Document 1).

[0003] International Publication No. WO 2023 / 106232

[0004] Generally, an event pixel is provided with an optical filter that transmits white light including wavelength bands of R (red), G (green), and B (blue) (hereinafter referred to as a W filter), or is not provided with an optical filter. Therefore, white light incident on an event pixel leaks from inter-pixel light shielding portions arranged around, or is reflected in a lower layer of a photodiode, thereby causing color mixing into adjacent grayscale pixels. When color mixing from event pixels is not uniform among grayscale pixels, an output difference in luminance signals caused by color mixing, that is, a sensitivity difference occurs.

[0005] The present technology has been made in view of such circumstances, and is intended to suppress a sensitivity difference between grayscale pixels caused by color mixing.

[0006] A solid-state imaging device according to a first aspect of the present technology includes a pixel array unit in which a plurality of first pixels that output luminance signals at grayscale levels corresponding to the amount of incident light and a plurality of second pixels that detect information different from that of the first pixels are regularly arranged, and in at least a region excluding an end portion of the pixel array unit, an amount of adjacent second pixels around each of the first pixels is uniform.

[0007] The electronic device in the second aspect of this technology includes a pixel array portion in which a plurality of first pixels that output a luminance signal of a gradation level corresponding to the amount of incident light, and a plurality of second pixels that detect information different from that of the first pixels are arranged in a regular manner, wherein in the region excluding the edges of the pixel array portion, the amount of adjacent second pixels around each first pixel is the same.

[0008] In the first or second aspect of this technology, in a region excluding the edges of a pixel array portion in which a plurality of first pixels that output a luminance signal of a gradation level corresponding to the amount of incident light, and a plurality of second pixels that detect information different from that of the first pixels are regularly arranged, the amount of adjacent second pixels is similar around each first pixel.

[0009] This figure shows an example of a conventional pixel layout. This figure shows an example of a conventional pixel layout. This is a block diagram showing an example configuration of an imaging device according to an embodiment of the present technology. This is a block diagram showing an example configuration of the solid-state imaging device of Figure 3. This figure shows a first example of a pixel layout. This is a diagram of the pixel layout of Figure 5 rotated 180 degrees. This is a circuit diagram showing an example configuration of a grayscale pixel unit constituting the pixel layout of Figure 5. This is a block diagram showing an example configuration of an event pixel unit constituting the pixel layout of Figure 5. This is a circuit diagram showing a detailed example configuration of the address event detection circuit of Figure 8. This is a circuit diagram showing a modified current-voltage conversion circuit of the address event detection circuit of Figure 8. This is a circuit diagram showing an example circuit configuration of a unit block constituting the pixel layout of Figure 5. This is a circuit diagram showing an example circuit configuration of an event pixel of a unit block constituting the pixel layout of Figure 5. This is a cross-sectional view showing a first example configuration of a cross-section of a solid-state imaging device having the pixel layout of Figure 5. This is a cross-sectional view showing a first example configuration of a cross-section of a solid-state imaging device having the pixel layout of Figure 5. This is a cross-sectional view showing a second example configuration of a cross-section of a solid-state imaging device having the pixel layout of Figure 5. This is a cross-sectional view showing a second example configuration of a cross-section of a solid-state imaging device having the pixel layout of Figure 5. This is a cross-sectional view showing a third example configuration of a cross-section of a solid-state imaging device having the pixel layout of Figure 5. This is a cross-sectional view showing a third configuration example of a solid-state imaging device having the pixel layout of Figure 5. This is a cross-sectional view showing a fourth diagram showing a second example of a pixel layout. This is a circuit diagram showing an example of the configuration of a grayscale pixel unit constituting the unit unit of Figure 21. This is a diagram showing a third example of a pixel layout. This is a diagram of the pixel layout of Figure 23 rotated 180 degrees. This is a circuit diagram showing an example of the configuration of a grayscale pixel unit constituting the pixel layout of Figure 23. This is a circuit diagram showing an example of the configuration of an event pixel unit constituting the pixel layout of Figure 23. This is a circuit diagram showing an example of the circuit configuration of a unit block constituting the pixel layout of Figure 23. This is a diagram showing a fourth example of a pixel layout. This is a circuit diagram showing an example of the configuration of a grayscale pixel unit constituting the pixel layout of Figure 28. This is a diagram showing a fifth example of a pixel layout.This is a circuit diagram showing an example of the circuit configuration of a unit block constituting the pixel layout of Figure 30. This is a cross-sectional view showing an example of the cross-sectional configuration of a solid-state imaging device having the pixel layout of Figure 30. This is a cross-sectional view showing an example of the cross-sectional configuration of a solid-state imaging device having the pixel layout of Figure 30. This is a diagram showing a sixth example of the pixel layout. This is a diagram showing a seventh example of the pixel layout. This is a diagram showing an eighth example of the pixel layout. This is a diagram showing a ninth example of the pixel layout. This is a diagram showing a tenth example of the pixel layout. This is a diagram showing an eleventh example of the pixel layout. This is a diagram showing a twelfth example of the pixel layout. This is a diagram showing a thirteenth example of the pixel layout. This is a diagram of the pixel layout of Figure 41 rotated 180 degrees. This is a circuit diagram showing an example of the configuration of a grayscale pixel unit constituting the pixel layout of Figure 41. This is a circuit diagram showing an example of the circuit configuration of a unit block constituting the pixel layout of Figure 41. This is a cross-sectional view showing an example of the cross-sectional configuration of a solid-state imaging device having diagram showing a fourteenth example of the pixel layout. This figure shows an example of the 15th pixel layout. This figure shows an example of the 16th pixel layout. This figure illustrates an example of image sensor usage. This block diagram shows an example of a schematic configuration of a vehicle control system. This is an explanatory diagram showing an example of the installation location of the external information detection unit and the imaging unit.

[0010] The following describes the embodiments for implementing this technology. The explanation will proceed in the following order: 0. Background of this technology 1. First embodiment 2. Second embodiment 3. Third embodiment 4. Fourth embodiment 5. Fifth embodiment 6. Sixth embodiment 7. Seventh embodiment 8. Eighth embodiment 9. Ninth embodiment 10. Tenth embodiment 11. Eleventh embodiment 12. Twelfth embodiment 13. Modifications 14. Examples of image sensor use 15. Examples of application to mobile objects 16. Others

[0011] <<0. Background of this Technology>> First, the background of this technology will be explained with reference to Figures 1 and 2.

[0012] Figure 1 shows an example of the configuration of a unit region UAa, which is the smallest unit of pixel layout in the pixel array section of a solid-state imaging device.

[0013] The unit region UAa is a block consisting of 16 pixels arranged in a 4x4 grid. Within the unit region UAa, subblocks consisting of 2x2 pixels are arranged in a 2x2 grid.

[0014] In the upper left subblock, the first column contains grayscale pixels (hereinafter referred to as R pixels) equipped with a color filter (hereinafter referred to as R filter), which is an optical filter that transmits light in the red wavelength band, and the second column contains event pixels.

[0015] In the upper right subblock, grayscale pixels (hereinafter referred to as G pixels) equipped with a color filter (hereinafter referred to as G filter), which is an optical filter that transmits light in the green wavelength band, are arranged in a 2x2 grid.

[0016] In the lower left subblock, the G pixels are arranged in a 2x2 grid.

[0017] In this specification, Gr pixels, which are G pixels located in the same row as R pixels, and Gb pixels, which are G pixels located in the same row as B pixels, are not distinguished and are simply referred to as G pixels.

[0018] In the lower right subblock, event pixels are arranged in the first column, and grayscale pixels (hereinafter referred to as B pixels) equipped with a color filter (hereinafter referred to as a B filter), which is an optical filter that transmits light in the blue wavelength band, are arranged in the second column.

[0019] As mentioned above, the event pixel either has a W filter or does not have an optical filter. Therefore, the event pixel can detect events for all colors: R, G, and B. Furthermore, if the event pixel does not have an optical filter, it can also detect events for IR light.

[0020] Here, within the unit region UAa, the grayscale pixels are arranged asymmetrically, with some grayscale pixels adjacent to the event pixel and others not. Therefore, the change in the luminance signal due to color mixing from the event signal differs depending on the grayscale pixel.

[0021] For example, in unit region UAa, the G pixels in the third column are adjacent to the event pixel, while the G pixels in the fourth column are not. Therefore, the color mixing from the event pixel is greater in the G pixels in the third column than in the G pixels in the fourth column, resulting in a sensitivity difference between the G pixels in the third and fourth columns. Also, in unit region UAa, the G pixels in the second column are adjacent to the event pixel, while the G pixels in the first column are not. Therefore, the color mixing from the event pixel is greater in the G pixels in the second column than in the G pixels in the first column, resulting in a sensitivity difference between the G pixels in the first and second columns.

[0022] Furthermore, because grayscale pixels and event pixels are mixed within the unit region UBa, grayscale pixels are lost, and when processing the image as a normal Bayer array, it is necessary to interpolate the color information of the missing pixels.

[0023] For example, in the second column of the unit region UAa, an event pixel causes a loss of an R pixel, and the R (red) luminance signal of the missing pixel needs to be interpolated from surrounding grayscale pixels. However, while G pixels are located in the second column of the unit region UAa, R pixels are not, which reduces the interpolation accuracy of the R luminance signal and makes false colors more likely to occur.

[0024] Similarly, in the third column of the unit region UAa, a B pixel is missing due to an event pixel, and the B (blue) luminance signal of the missing pixel needs to be interpolated from the surrounding grayscale pixels. However, although a G pixel is located in the third column of the unit region UAa, a B pixel is not located there, which reduces the interpolation accuracy of the B luminance signal and makes false colors more likely to occur.

[0025] Furthermore, event pixels are missing in the first and fourth columns of the unit region UAa, meaning that event information indicating brightness changes is not acquired. Consequently, a spatial bias in event pixels occurs within the pixel array, potentially reducing the resolution of event detection.

[0026] Figure 2 shows an example of the configuration of a unit region UAb, which is the smallest unit of pixel layout in the pixel array section of a solid-state imaging device.

[0027] The unit region UAb is a block consisting of 16 pixels arranged in a 4x4 grid. Within the unit region UAb, subblocks consisting of 2x2 pixels are arranged in a 2x2 grid.

[0028] In the upper left subblock, the R pixel is placed in the upper left, the W pixel is placed in the lower left, and the event pixel is placed in the second column.

[0029] In the upper right subblock, the G pixels are arranged diagonally, and the W pixels are arranged diagonally opposite to the G pixels.

[0030] In the lower left subblock, the G pixels are arranged diagonally, and the W pixels are arranged diagonally opposite to the G pixels.

[0031] In the lower right subblock, the R pixels are arranged diagonally, and the W pixels are arranged diagonally opposite to the R pixels.

[0032] Here, the G pixel in the third column of unit region UAa is surrounded on all four sides by event pixels and W pixels, and an event pixel is adjacent to it diagonally to the lower left. On the other hand, the G pixel in the fourth column of unit region UAa is surrounded on all four sides by W pixels, but is adjacent to a color pixel diagonally. Therefore, the color mixing from event pixels and W pixels is greater in the G pixel in the third column than in the G pixel in the fourth column, resulting in a sensitivity difference between the G pixels in the third and fourth columns.

[0033] Furthermore, R information is missing in the second column and second row of the unit region UAa. Consequently, false color is likely to cause a decrease in image quality after interpolation of the R pixels.

[0034] Furthermore, event pixels are missing in the first, third, and fourth columns, third row, and fourth row of the unit region UAa. Consequently, spatial bias occurs in event pixels within the pixel array, potentially reducing the resolution of event detection.

[0035] In contrast, this technology suppresses sensitivity differences between grayscale pixels due to color mixing. For example, this technology improves the interpolation accuracy of the luminance signal of grayscale pixels. For example, this technology suppresses spatial bias of event pixels within a pixel array.

[0036] <<1. First Embodiment>> Next, a first embodiment of the present technology will be described with reference to Figures 3 to 20.

[0037] <Example of configuration of imaging device 101> Figure 3 is a block diagram showing an example of the configuration of an imaging device 101 according to an embodiment of this technology.

[0038] The imaging device 101 comprises an optical system 111, a solid-state imaging device 112, a control unit 113, and a data processing unit 114. Examples of imaging devices 101 include cameras mounted on industrial robots and in-vehicle cameras.

[0039] The optical system 111 collects light from the subject and directs it into the solid-state imaging device 112. The solid-state imaging device 112 generates a luminance signal with a gradation level corresponding to the amount of incident light received through the optical system 111 and outputs it to the data processing unit 114. The solid-state imaging device 112 also detects when the luminance change due to the incident light exceeds a predetermined threshold as an event, generates an event signal, and outputs it to the data processing unit 114. More specifically, the solid-state imaging device 112 detects as an event whether or not a change exceeding a predetermined threshold has occurred in the photocurrent corresponding to the luminance of the incident light. In the following description, the event signal may be referred to as event data.

[0040] The control unit 113 is composed of, for example, a CPU (Central Processing Unit) and controls the solid-state imaging device 112. For example, the control unit 113 instructs the solid-state imaging device 112 on the operating mode and the start and end of imaging.

[0041] The data processing unit 114 is configured by, for example, an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), a microprocessor, or the like, and includes a data generation unit 121 and a recording unit 122. The data generation unit 121 performs predetermined data processing using a luminance signal supplied from the solid-state imaging device 112, predetermined data processing using an event signal supplied from the solid-state imaging device 112, and the like. The data generation unit 121 outputs processed data, which is a result of the data processing, to an external device. Further, the data generation unit 121 may output the luminance signal and the event signal supplied from the solid-state imaging device 112 to an external device as they are.

[0042] The recording unit 122 is configured by, for example, a flash memory, a DRAM (Dynamic Random Access Memory), an SRAM (Static Random Access Memory), or the like. The recording unit 122 records the luminance signal, the event signal, or the processed data supplied from the data generation unit 121.

[0043] Note that the data processing unit 114 may be incorporated and configured as a part of the solid-state imaging device 112.

[0044] <Configuration Example of Solid-State Imaging Device 112> FIG. 4 is a block diagram showing a schematic configuration example of the solid-state imaging device 112.

[0045] The solid-state imaging device 112 includes a pixel array unit 141, a driving unit 142, an arbiter (arbitration unit) 143, an event signal processing unit 144, and a luminance signal processing unit 145.

[0046] A plurality of pixels PX are arranged in the pixel array unit 141. Here, the horizontal direction of the pixel array unit 141 in which the plurality of pixels PX are arranged is referred to as a "row", and the vertical direction is referred to as a "column".

[0047] In the pixels PX arranged in the pixel array unit 141, there are grayscale pixels PXG that generate a luminance signal of a grayscale level corresponding to the light amount of incident light, and event pixels PXE that detect a change in luminance of incident light as an event and generate an event signal. In other words, in the pixel array unit 141, as the pixels PX, grayscale pixels PXG and event pixels PXE are mixed and arranged in a two-dimensional grid pattern, respectively.

[0048] The grayscale pixel PXG includes a photodiode 161 serving as a photoelectric conversion element, and a readout circuit 162.

[0049] The readout circuit 162 reads out an analog luminance signal based on the amount of charge accumulated by photoelectric conversion of the photodiode 161 under the control of the driving unit 142, and supplies the analog luminance signal to the luminance signal processing unit 145.

[0050] It should be noted that the readout circuit 162 may be provided for each pixel, or all or part of the readout circuit 162 may be shared by a plurality of grayscale pixels PXG.

[0051] The event pixel PXE includes a photodiode 171 serving as a photoelectric conversion element that has a smaller capacitance than the photodiode 161, and an address event detection circuit 172. The photodiode 171 causes a photocurrent as an electrical signal to flow by photoelectrically converting incident light.

[0052] The address event detection circuit 172 reads out an event signal representing the occurrence of an event based on a change in photocurrent generated by photoelectric conversion of the photodiode 171 under the control of the arbiter 143, and supplies the event signal to the event signal processing unit 144. Specifically, the address event detection circuit 172 detects a change in the photocurrent generated by photoelectric conversion of the photodiode 171 as an event when the change exceeds a predetermined threshold. When an event is detected, the address event detection circuit 172 outputs a request for output of an event signal representing the occurrence of the event to the arbiter 143. When a response representing permission to output the event signal is obtained as a result of arbitration by the arbiter 143, the address event detection circuit 172 supplies the event signal to the event signal processing unit 144.

[0053] The address event detection circuit 172 may be provided for each pixel, or all or part of the address event detection circuit 172 may be shared by multiple event pixels PXE.

[0054] The readout circuit 162 for the grayscale pixel PXG and the address event detection circuit 172 for the event pixel PXE operate independently, enabling asynchronous signal readout. Therefore, the timing of signal readout differs between the grayscale pixel PXG and the event pixel PXE, allowing the event pixel PXE to be readout faster (at a higher frame rate) than the grayscale pixel PXG.

[0055] The drive unit 142 drives each grayscale pixel PXG of the pixel array unit 141 by supplying control signals to each grayscale pixel PXG.

[0056] The arbiter 143 mediates requests from event pixels PXEs in the pixel array unit 141 and returns a response to the event pixel PXE that sent the request indicating whether to permit or deny the output of an event signal. An event pixel PXE that has received a permit response from the arbiter 143 can output an event signal to the event signal processing unit 144. The event signals are transferred to the event signal processing unit 144 row by row, and the event signals of event pixel PXEs in the same row that have not generated an event are discarded by the event signal processing unit 144. The arbiter 143 supplies a reset signal to the event pixel PXE to reset event detection.

[0057] The event signal processing unit 144 performs the necessary processing on the event signals output by each event pixel PXE of the pixel array unit 141 and supplies them to the data processing unit 114 (Figure 3).

[0058] Here, the change in photocurrent generated at the event pixel PXE can also be considered as a change in the amount of light incident on the event pixel PXE. Therefore, an event can also be described as a change in the amount of light at the event pixel PXE (a change in the amount of light exceeding a threshold).

[0059] The event data allows us to identify at least the location information (coordinates, etc.) representing the position of the event pixel PXE where the light intensity change occurred. In addition, the event data allows us to identify the polarity (positive or negative) of the light intensity change.

[0060] For the sequence of event data output by the event pixel PXE at the time an event occurs, time information representing the (relative) time the event occurred can be identified as long as the intervals between event data are maintained at the time the event occurred. However, if the intervals between event data are no longer maintained at the time the event occurred, such as when the event data is stored in memory, the time information is lost. Therefore, before the intervals between event data are no longer maintained at the time the event occurred, time information representing the (relative) time the event occurred, such as a timestamp, is added to the event data. The process of adding time information to event data may be performed by the address event detection circuit 172, or by the event signal processing unit 144 or the data processing unit 114, as long as it is done before the intervals between event data are no longer maintained at the time the event occurred.

[0061] The luminance signal processing unit 145 has, for example, a single-slope type ADC (AD Converter) (not shown) corresponding to a row of grayscale pixels PXG. In each ADC, the luminance signal processing unit 145 performs AD conversion on the analog luminance signal of the grayscale pixels PXG in the corresponding row and supplies it to the data processing unit 114 (Figure 3). In addition to AD conversion of the luminance signal, the luminance signal processing unit 145 can also perform CDS (Correlated Double Sampling).

[0062] <Example of Pixel Layout> Figure 5 shows a part of an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0063] In this example, in the pixel array section 141, the unit regions UA1 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0064] The unit region UA1 is composed of eight pixels PX, consisting of four grayscale pixels PXG and four event pixels PXE.

[0065] Each PXG (grayscale pixel) has an octagonal shape, formed by cutting out the four corners of a square at a 45° angle to the row and column directions. Each PXG is connected to an adjacent PXG by either its row or column edge, and is arranged in a grid (matrix) pattern in both the row and column directions.

[0066] The grayscale pixels (PXG) are arranged in a grid according to the Bayer array. That is, within a unit region UA1, R pixels (which are grayscale pixels PXG equipped with an R filter) and B pixels (which are grayscale pixels PXG equipped with a B filter) are arranged diagonally. Within the unit region UA1, G pixels (which are grayscale pixels PXG equipped with a G filter) are arranged diagonally opposite to the R and B pixels.

[0067] Event pixels (PXE) are positioned in the gaps between tone pixels (PXG). Specifically, an event pixel (PXE) is located in a region enclosed by the diagonal edges of four adjacent tone pixels (PXG), and is offset from the tone pixels (PXG) by half the pitch between them in both the row and column directions. Therefore, the tone pixels (PXG) and event pixels (PXE) are arranged in a grid (matrix) pattern in both the row and column directions, each offset from the other by half a pitch. Furthermore, an event pixel (PXE) has a shape that is a square tilted at 45° with respect to the row and column directions.

[0068] Hereafter, as shown in Figure 5, the larger pixel PX within a unit area will be referred to as the large pixel, as shown in the grayscale pixel PXG, and the smaller pixel PX within a unit area will be referred to as the small pixel, as shown in the event pixel PXE. Hereafter, the R pixel, G pixel, and B pixel will be collectively referred to as the color pixel.

[0069] Figure 6 shows a portion of the pixel layout from Figure 5 rotated 180 degrees.

[0070] This pixel layout has approximate point symmetry. That is, even if the pixel layout is rotated 180 degrees, the area excluding the edges of the pixel array portion 141 will have a layout in which the unit region UA1 is repeatedly arranged in the row and column directions.

[0071] Furthermore, in this pixel layout, event pixels (PXEs) are adjacent to each diagonal edge of each color pixel. This ensures that the amount of adjacent event pixels (PXEs) around each color pixel is the same. In other words, in two dimensions, the total length of adjacent event pixels (PXEs) around each color pixel is the same, and in three dimensions, the total area of ​​adjacent event pixels (PXEs) around each color pixel is the same. Note that within a range of slight errors such as manufacturing tolerances, the amount of adjacent event pixels (PXEs) is considered to be the same.

[0072] This suppresses the occurrence of sensitivity differences between color pixels due to color mixing from event pixel PXEs. In other words, since the color mixing from event pixel PXEs is similar for each color pixel, it suppresses the occurrence of output differences in the luminance signals of the gradation levels output from each color pixel.

[0073] Furthermore, there are no missing color pixels in the Bayer array of each unit region UA1. This makes it easier to interpolate the luminance signal at the location where an event pixel PXE is located from surrounding color pixels, improving the interpolation accuracy of the luminance signal and suppressing the occurrence of false colors.

[0074] Furthermore, the event pixels (PXEs) are arranged at equal intervals. This allows for the acquisition of event signals uniformly without spatial bias within the pixel array 141, improving the resolution of event detection.

[0075] Furthermore, the pixel size of color pixels and event pixel PXEs can be adjusted, for example, by adjusting the length of the edges in the row and column directions of the color pixels. This allows for flexible changes to the pixel size of color pixels and event pixel PXEs depending on the product's application.

[0076] <Example of circuit configuration of grayscale pixel unit PUG1> Figure 7 shows an example of the configuration of grayscale pixel unit PUG1 that constitutes the circuit of grayscale pixel PXG.

[0077] The grayscale pixel unit PUG1 constitutes the unit pixel of the grayscale pixel PXG. The grayscale pixel unit PUG1 includes a photodiode 161 and a readout circuit 162.

[0078] The readout circuit 162 comprises a transfer transistor TG, a floating diffusion region FD, an additional capacitance subFD, a switching transistor FDG, a reset transistor RST, an amplification transistor AMP, and a selection transistor SEL. Each pixel transistor of the transfer transistor TG, switching transistor FDG, reset transistor RST, amplification transistor AMP, and selection transistor SEL is composed of an N-type MOS transistor (MOS FET).

[0079] The photodiode 161 generates and stores an electric charge (signal charge) corresponding to the amount of light it receives. The anode terminal of the photodiode 161 is grounded, and the cathode terminal is connected to the floating diffusion region FD via a transfer transistor TG.

[0080] When the transfer transistor TG is turned on by the transfer drive signal supplied to the gate electrode, it reads the charge generated by the photodiode 161 and transfers it to the floating diffusion region FD. The floating diffusion region FD holds the charge read from the photodiode 161.

[0081] The switching transistor FDG switches the connection between the floating diffusion region FD and the additional capacitance subFD on and off according to the capacitance switching signal supplied to the gate electrode, thereby switching the conversion efficiency. Specifically, the drive unit 142 turns on the switching transistor FDG when the amount of incident light is high, connecting the floating diffusion region FD and the additional capacitance subFD. This allows more charge to be accumulated at high illumination levels. On the other hand, when the amount of incident light is low, the drive unit 142 turns off the switching transistor FDG, disconnecting the additional capacitance subFD from the floating diffusion region FD. This increases the conversion efficiency.

[0082] When the reset transistor RST is turned on by the reset drive signal supplied to its gate electrode, the charge stored in the additional capacitor subFD is discharged to the drain (power supply VDD), resetting the potential of the additional capacitor subFD. When the reset transistor RST is turned on, the switching transistor FDG is also turned on simultaneously, and the floating diffusion region FD is also reset.

[0083] The additional capacitance subFD consists of a diffusion layer (high-concentration N-type semiconductor region) that serves as both the drain region of the switching transistor FDG and the source region of the reset transistor RST.

[0084] The amplification transistor AMP outputs a pixel signal corresponding to the potential of the floating diffusion region FD. Specifically, the amplification transistor AMP forms a source follower circuit with a load MOS transistor (not shown) which acts as a constant current source connected via the vertical signal line VSL, and a pixel signal indicating a level corresponding to the charge accumulated in the floating diffusion region FD is output from the amplification transistor AMP to the brightness signal processing unit 145 (Figure 4) via the selection transistor SEL.

[0085] The selection transistor SEL is turned on when a grayscale pixel PXG is selected by a selection drive signal supplied to the gate electrode, and outputs the signal generated by the grayscale pixel PXG as a pixel signal to the brightness signal processing unit 145 via the vertical signal line VSL.

[0086] The signal lines through which the transfer drive signal, capacity switching signal, selection drive signal, and reset drive signal are transmitted are connected to the drive unit 142 in Figure 4.

[0087] <Example of the configuration of the event pixel unit PUE1a> Figure 8 shows an example of the configuration of the event pixel unit PUE1a that constitutes the circuit of the event pixel PXE.

[0088] The event pixel unit PUE1a constitutes the unit pixel of the event pixel PXE. The grayscale pixel unit PUG1a includes a photodiode 171 and an address event detection circuit 172.

[0089] The address event detection circuit 172 includes a current-voltage conversion circuit 201, a buffer 202, a subtractor 203, a quantizer 204, and a transfer circuit 205.

[0090] The current-voltage conversion circuit 201 converts the photocurrent from the corresponding photodiode 171 into a voltage signal. The current-voltage conversion circuit 201 generates a voltage signal corresponding to the logarithmic value of the photocurrent (hereinafter also referred to as a logarithmic converted voltage signal as appropriate) and outputs it to the buffer 202.

[0091] Buffer 202 buffers the voltage signal from the current-voltage conversion circuit 201 and outputs it to the subtractor 203. This buffer 202 ensures noise isolation from the switching operation of the subsequent stage and improves the driving force that drives the subsequent stage. Note that this buffer 202 can be omitted.

[0092] The subtractor 203 reduces the level of the voltage signal from the buffer 202 according to the reset signal from the arbiter 143. The subtractor 203 outputs the reduced voltage signal to the quantizer 204.

[0093] The quantizer 204 quantizes the voltage signal from the subtractor 203 into a digital signal and supplies it to the transfer circuit 205 as event data.

[0094] The transfer circuit 205 transfers (outputs) the event data to the event signal processing unit 144. That is, the transfer circuit 205 supplies a request to the arbiter 143 requesting the output of event data. When the transfer circuit 205 receives a response from the arbiter 143 indicating permission to output the event data in response to the request, it transfers the event data to the event signal processing unit 144.

[0095] <Detailed Configuration Example of Address Event Detection Circuit 172> Figure 9 shows the detailed configuration of the current-voltage conversion circuit 201, subtractor 203, and quantizer 204 of the address event detection circuit 172 shown in Figure 8.

[0096] The current-voltage conversion circuit 201 is composed of transistors 211 to 213. For example, N-type MOS transistors (MOS FETs) can be used as transistors 211 and 213, and for example, a P-type MOS transistor (MOS FET) can be used as transistor 212.

[0097] The photodiode 171 receives incident light, performs photoelectric conversion, and generates and transmits a photocurrent as an electrical signal. The current-voltage conversion circuit 201 converts the photocurrent from the photodiode 171 into a voltage (hereinafter also called photovoltage) VLOG corresponding to the logarithm of the photocurrent and outputs it to the buffer 202.

[0098] The source of transistor 211 is connected to the gate of transistor 213, and a photocurrent from the photodiode 171 flows at the connection point between the source of transistor 211 and the gate of transistor 213. The drain of transistor 211 is connected to the power supply VDD, and its gate is connected to the drain of transistor 213.

[0099] The source of transistor 212 is connected to the power supply VDD, and its drain is connected to the connection point between the gate of transistor 211 and the drain of transistor 213. A predetermined bias voltage Vbias is applied to the gate of transistor 212. The source of transistor 213 is grounded.

[0100] The drain of transistor 211 is connected to the power supply VDD side, and it is a source follower. A photodiode 171 is connected to the source of the source follower transistor 211, and as a result, a photocurrent flows through transistor 211 (from drain to source) due to the charge generated by the photoelectric conversion of the photodiode 171. Transistor 211 operates in the subthreshold region, and an optical voltage VLOG corresponding to the logarithm of the photocurrent flowing through transistor 211 appears at the gate of transistor 211.

[0101] As described above, the current-voltage conversion circuit 201 consists of a transistor 211 as a logarithmic transistor (LOG transistor) that converts the photocurrent flowing through the photodiode 171 into an optical voltage corresponding to the logarithm of the photocurrent, a transistor 212 as a constant current source, and a transistor 213 as an amplifying transistor that amplifies the optical voltage.

[0102] The optical voltage VLOG is output to the subtractor 203 via the buffer 202 from the connection point between the gate of transistor 211 and the drain of transistor 213.

[0103] The subtractor 203 calculates the difference between the current optical voltage VLOG from the current-voltage conversion circuit 201 and the optical voltage at a timing that is only a small amount of time different from the current one, and outputs a difference signal Vdiff corresponding to that difference.

[0104] The subtractor 203 comprises a capacitor 221, an operational amplifier 222, a capacitor 223, and a switch 224. The quantizer 204 comprises comparators 231 and 232.

[0105] One end of capacitor 221 is connected to the output of buffer 202, and the other end is connected to the input terminal of operational amplifier 222. Therefore, the optical voltage VLOG is input to the (inverting) input terminal of operational amplifier 222 via capacitor 221.

[0106] The output terminal of the operational amplifier 222 is connected to the non-inverting input terminal (+) of comparators 231 and 232 of the quantizer 204.

[0107] One end of the capacitor 223 is connected to the input terminal of the operational amplifier 222, and the other end is connected to the output terminal of the operational amplifier 222.

[0108] Switch 224 is connected to capacitor 223 to switch the connection across the capacitor 223 on and off. Switch 224 switches the connection across the capacitor 223 on and off in accordance with a reset signal from arbiter 143.

[0109] The capacitor 223 and the switch 224 constitute a switched capacitor. When the switch 224, which is off, is temporarily turned on and then turned off again, the capacitor 223 is discharged and reset to a state where it can store new charge.

[0110] When switch 224 is turned on, the photovoltage VLOG on the photodiode 171 side of capacitor 221 is denoted as Vinit, and the capacitance of capacitor 221 is denoted as C1. The input terminal of the operational amplifier 222 is virtual ground, and the charge Qinit accumulated in capacitor 221 when switch 224 is on is expressed by equation (1).

[0111] Qinit=C1×Vinit...(1)

[0112] Furthermore, when switch 224 is ON, the terminals of capacitor 223 are short-circuited, so the charge stored in capacitor 223 becomes zero.

[0113] Subsequently, if we denote the photovoltage VLOG on the photodiode 171 side of capacitor 221 when switch 224 is turned off as Vafter, then the charge Qafter accumulated in capacitor 221 when switch 224 is turned off is expressed by equation (2).

[0114] Qafter=C1×Vafter...(2)

[0115] If we denote the capacitance of capacitor 223 as C2, the charge Q2 stored in capacitor 223 can be expressed by equation (3) using the difference signal Vdiff, which is the output voltage of operational amplifier 222.

[0116] Q2=-C2×Vdiff...(3)

[0117] Since the total charge amount, which is the sum of the charge of capacitor 221 and the charge of capacitor 223, does not change before and after switch 224 is turned off, equation (4) holds true.

[0118] Qinit=Qafter+Q2...(4)

[0119] Substituting equations (1) through (3) into equation (4) yields equation (5).

[0120] Vdiff=-(C1 / C2)×(Vafter-Vinit)...(5)

[0121] According to equation (5), the subtractor 203 subtracts the optical voltages Vafter and Vinit, that is, it calculates the difference signal Vdiff, which corresponds to the difference between the optical voltages Vafter and Vinit (Vafter - Vinit). According to equation (5), the subtraction gain of the subtractor 203 is C1 / C2. Therefore, the subtractor 203 outputs a voltage obtained by multiplying the change in the optical voltage VLOG after the reset of capacitor 223 by C1 / C2 as the difference signal Vdiff.

[0122] The subtractor 203 outputs a difference signal Vdiff when the switch 224 is turned on or off by the reset signal output by the arbiter 143.

[0123] The difference signal Vdiff output from the subtractor 203 is supplied to the non-inverting input terminals (+) of comparators 231 and 232 of the quantizer 204.

[0124] The comparator 231 compares the difference signal Vdiff from the subtractor 203 with the positive threshold Vrefp input to the inverting input terminal (-). The comparator 231 outputs a detection signal DET(+) at either a High (H) or Low (L) level, indicating whether or not the positive threshold Vrefp has been exceeded, to the transfer circuit 205 as event data obtained by quantizing the difference signal Vdiff.

[0125] The comparator 232 compares the difference signal Vdiff from the subtractor 203 with the negative threshold Vrefn input to the inverting input terminal (-). The comparator 232 outputs a detection signal DET(-) at either a High (H) or Low (L) level, indicating whether or not the negative threshold Vrefn has been exceeded, to the transfer circuit 205 as event data obtained by quantizing the difference signal Vdiff.

[0126] <Other detailed configuration examples of current-voltage conversion circuits> In the current-voltage conversion circuit 201 shown in Figure 9, a single loop circuit is provided, consisting of a logarithmic transistor 211 and an amplifying transistor 213. However, with only one loop circuit, there is a risk that the conversion gain when converting current to voltage will be insufficient. Therefore, a configuration example of a current-voltage conversion circuit 201 with a two-stage loop circuit will be described.

[0127] In other words, Figure 10 is a circuit diagram showing another example configuration of the current-voltage conversion circuit 201.

[0128] The current-voltage conversion circuit 201 in Figure 10 differs from the current-voltage conversion circuit 201 in Figure 6 in that it further includes transistors 214 and 215. Transistors 214 and 215 are, for example, N-type MOS transistors (MOS FETs).

[0129] Transistors 214 and 211 are connected in series between the power supply VDD and the photodiode 171, while transistors 212, 215, and 213 are connected in series between the power supply VDD and the ground terminal. The gate of transistor 211 is connected to the connection point between transistors 215 and 213, and the gate of transistor 214 is connected to the connection point between transistors 212 and 215.

[0130] On the other hand, the gate of transistor 213 is connected to the connection point between photodiode 171 and transistor 211, similar to the current-voltage conversion circuit 201 in Figure 9. The gate of transistor 215 is connected to the connection point between transistor 211 and transistor 214. Also, the connection point between transistor 212 and transistor 215 is connected to buffer 202 in Figure 9.

[0131] As described above, since the loop circuit consisting of transistor 211 as a logarithmic transistor and transistor 213 as an amplifying transistor is connected in two stages, and the loop circuit consisting of transistor 214 as a logarithmic transistor and transistor 215 as an amplifying transistor is connected in two stages, the conversion gain can be doubled compared to the case where there is only one loop circuit.

[0132] <Example of circuit configuration of unit region UA1> Figure 11 shows an example of the circuit configuration of unit region UA1 in Figure 5.

[0133] Note that some of the symbols have been omitted due to space limitations in the drawings. Also, in the event pixel unit PUE1a, only the current-voltage conversion circuit 201, which is part of the address event detection circuit 172, is shown. The same applies to subsequent drawings.

[0134] Each color pixel (grayscale pixel PXG) in the unit region UA1 uses the grayscale pixel unit PUG1 shown in Figure 7. Grayscale pixel units PUG1 arranged in the same row share the vertical signal line VSL.

[0135] In the unit region UA1, one of the event pixel PXEs uses the event pixel unit PUE1a shown in Figure 9, while the other event pixel PXEs use the event pixel unit PUE1b.

[0136] The event pixel unit PUE1b comprises only a photodiode 171 and does not have an address event detection circuit 172. The cathode of the photodiode 171 of event pixel unit PUE1b is connected to the cathode of the photodiode 171 of event pixel unit PUE1a. The address event detection circuit 172 of event pixel unit PUE1a is shared by the four event pixel PXEs within the unit region UA1.

[0137] As a result, multiple event pixels PXE within the unit region UA1 are binned, and the event signals of the multiple event pixels PXE are summed. This increases the output value of the event signal, making it easier to detect events. Even when using binned event pixels PXE within the unit region UA1, event signals can be acquired uniformly within the pixel array 141 without spatial bias.

[0138] Alternatively, event pixels PXE from multiple unit regions UA1 may be binned and used for event detection.

[0139] For example, Figure 12 shows an example of binning event pixels in four unit regions UA1 via transistor BIN.

[0140] Note that the illustration of the grayscale pixels (PXG) is omitted in Figure 12.

[0141] As a result, multiple event pixels PXE within multiple unit regions UA1 are binned, further increasing the output value of the event signal and making it easier to detect events. Even when using event pixels PXE within multiple unit regions UA1 by binning, event signals can be acquired uniformly within the pixel array 141 without spatial bias.

[0142] <First Configuration Example of Cross-Section of Solid-State Imaging Device 112> Figures 13 and 14 show a first configuration example of a part of the cross-section of the solid-state imaging device 112. Figure 13 shows an example of a cross-section of an arrangement of R pixels, event pixels PXE, and B pixels arranged diagonally in a unit region UA1. Figure 14 shows an example of a cross-section of an arrangement of G pixels, event pixels PXE, and G pixels arranged diagonally in a unit region UA1.

[0143] Note that in Figures 13 and 14, the event pixel PXE is labeled as the EV pixel. This is also the case in subsequent figures.

[0144] In the solid-state imaging device 112, the on-chip lens layer 301, optical filter layer 302, semiconductor substrate 303, wiring layer 304, and support substrate 305 are stacked in order from the incident light side.

[0145] In the R pixel, a microlens 321L, an R filter 322R, an interlayer insulating film 323, a pinning film 324 having a negative fixed charge, a photodiode 161, and a P-type pinning region 325L are stacked on top of the wiring layer 304. The photodiode 161 is formed by a P-well region and an n-type impurity region formed inside it. An inter-pixel light-shielding portion 326 is arranged around the photodiode 161 to prevent light leakage into adjacent pixels PX.

[0146] In the B pixel, a microlens 321L, a B filter 322B, an interlayer insulating film 323, a pinning film 324, a photodiode 161, and a P-type pinning region 325L are stacked on top of the wiring layer 304. An inter-pixel light shielding portion 326 is positioned around the photodiode 161 to prevent light leakage into adjacent pixels PX.

[0147] In the G pixel, a microlens 321L, a G filter 322G, an interlayer insulating film 323, a pinning film 324, a photodiode 161, and a P-type pinning region 325L are stacked on top of the wiring layer 304. An inter-pixel light shielding portion 326 is positioned around the photodiode 161 to prevent light leakage into adjacent pixels PX.

[0148] In the event pixel PXE, a microlens 321S smaller in size than the microlens 321L, an interlayer insulating film 323, a pinning film 324, a photodiode 171, and a P-type pinning region 325S smaller in size than the pinning region 325L are stacked on top of the wiring layer 304. The photodiode 171 is formed by a P-well region and an n-type impurity region formed inside it. No optical filter is provided in the event pixel PXE. An inter-pixel light shielding portion 326 is arranged around the photodiode 171 to prevent light leakage into adjacent pixels PX.

[0149] Hereafter, when it is not necessary to distinguish between the microlens 321L and the microlens 321S individually, they will simply be referred to as the microlens 321. Hereafter, when it is not necessary to distinguish between the pinning region 325L and the pinning region 325S individually, they will simply be referred to as the pinning region 325.

[0150] In this way, by using pixels without optical filters as event pixels (PXEs), incident light can be converted into photoelectric light without attenuation and detected as an event signal.

[0151] <Second Configuration Example of Cross-Section of Solid-State Imaging Device 112> Figures 15 and 16 show a second configuration example of a part of the cross-section of the solid-state imaging device 112. Figure 15, similar to Figure 13, shows an example of a cross-section of an arrangement of R pixels, event pixels PXE, and B pixels arranged diagonally in a unit region UA1. Figure 16, similar to Figure 14, shows an example of a cross-section of an arrangement of G pixels, event pixels PXE, and G pixels arranged diagonally in a unit region UA1.

[0152] For clarity, some of the reference numerals for parts similar to those in Figures 13 and 14 have been omitted. This practice is also observed in subsequent figures.

[0153] Comparing the second example of the cross-sectional configuration of the solid-state imaging device 112 with the first example of the configuration in Figures 13 and 14, the difference is that an optical filter is provided on the event pixel PXE.

[0154] Specifically, in event pixels PXE adjacent to an R pixel in a predetermined direction, an R filter 341R is provided in the optical filter layer 302. Note that adjacent R filters 322R and R filter 341R may be integrated.

[0155] The difference is that in event pixels PXE adjacent to B pixels in a predetermined direction, a B filter 341B is provided in the optical filter layer 302. Furthermore, adjacent B filters 322B and 341B may be integrated.

[0156] The difference is that in event pixels PXE adjacent to a G pixel in a predetermined direction, a G filter 341G is provided in the optical filter layer 302. Furthermore, adjacent G filters 322G and 341G may be integrated.

[0157] In this way, by using an optical filter of the same color as the adjacent color pixel in the event pixel PXE, it is possible to detect events based on light of the same wavelength as the adjacent color pixel.

[0158] <Third Configuration Example of Cross-Section of Solid-State Imaging Device 112> Figures 17 and 18 show a third configuration example of a part of the cross-section of the solid-state imaging device 112. Figure 17, similar to Figure 13, shows an example of a cross-section of an arrangement of R pixels, event pixels PXE, and B pixels arranged diagonally in a unit region UA1. Figure 18, similar to Figure 14, shows an example of a cross-section of an arrangement of G pixels, event pixels PXE, and G pixels arranged diagonally in a unit region UA1.

[0159] Comparing the third configuration example of the cross-section of the solid-state imaging device 112 with the second configuration example in Figures 15 and 16, the configuration of the optical filter of the event pixel PXE is different. Specifically, in each event pixel PXE, a W filter 341W is provided in the optical filter layer 302.

[0160] In this way, by using the W filter 341W in the event pixel PXE, the incident light can be photoelectrically converted without attenuation and detected as an event signal.

[0161] <Fourth Configuration Example of Cross-Section of Solid-State Imaging Device 112> Figures 19 and 20 show a fourth configuration example of a part of the cross-section of the solid-state imaging device 112. Figure 19, similar to Figure 13, shows an example of a cross-section of an arrangement of R pixels, event pixels PXE, and B pixels arranged diagonally in a unit region UA1. Figure 20, similar to Figure 14, shows an example of a cross-section of an arrangement of G pixels, event pixels PXE, and G pixels arranged diagonally in a unit region UA1.

[0162] Comparing the fourth configuration example of the cross-section of the solid-state imaging device 112 with the second configuration example in Figures 15 and 16, the configuration of the optical filter of the event pixel PXE is different. Specifically, each event pixel PXE is provided with an optical filter (hereinafter referred to as an IR filter) 341IR that transmits infrared light.

[0163] In this way, by using the IR filter 341IR in the event pixel PXE, events can be detected based on infrared light. The solid-state imaging device 112 using the IR filter 341IR in the event pixel PXE can be applied, for example, to sensing systems and ranging systems.

[0164] <<2. Second Embodiment>> Next, a second embodiment of the present technology will be described with reference to Figures 21 and 22.

[0165] Comparing the second embodiment of this technology with the first embodiment, the pixel layout is different.

[0166] <Example of Pixel Layout> Figure 21 shows an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0167] In this example, in the pixel array section 141, the unit regions UA2 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0168] Comparing unit region UA2 with unit region UA1 in Figure 5, the difference is that each color pixel is composed of phase difference pixels. That is, the photodiodes 161 are separated to the left and right (in the row direction) so that the phase difference characteristics can be detected in each color pixel.

[0169] This pixel layout has similar characteristics to the pixel layout in Figure 5. For example, the pixel layout has approximate point symmetry. The amount of adjacent event pixels (PXEs) around each color pixel is the same. There are no missing color pixels in the Bayer array of each unit region UA2. The event pixels (PXEs) are arranged at equal intervals.

[0170] Furthermore, it is also possible to configure only some of the color pixels to be composed of phase-difference pixels.

[0171] <Example of circuit configuration of grayscale pixel unit PUG2> Figure 22 shows an example of the circuit configuration of grayscale pixel unit PUG2 used in the unit region UA2 of Figure 21.

[0172] Note that the same reference numerals are used for the parts corresponding to the grayscale pixel unit PUG1 in Figure 7, and their explanations are omitted as appropriate.

[0173] Compared to the grayscale pixel unit PUG1, the grayscale pixel unit PUG2 differs in that it has two sets of photodiodes 161 and transfer transistors TG.

[0174] Each photodiode 161 corresponds to the two photodiodes of each color pixel, which is a phase difference pixel.

[0175] Each photodiode 161 has its anode terminal grounded and its cathode terminal connected to the floating diffusion region FD via a transfer transistor TG.

[0176] When each transfer transistor TG is turned on by a transfer drive signal supplied to its gate electrode, it reads the charge generated by each photodiode 161 and transfers it to the floating diffusion region FD.

[0177] In this way, the readout circuit 162 is shared by the two photodiodes 161.

[0178] Furthermore, in the circuit configuration example shown in Figure 11, the circuit of the unit region UA2 is realized by using the grayscale pixel unit PUG2 instead of the grayscale pixel unit PUG1.

[0179] <<3. Third Embodiment>> Next, a third embodiment of the present technology will be described with reference to Figures 23 to 27.

[0180] Comparing the third embodiment of this technology with the first embodiment, the pixel layout is different.

[0181] <Example of Pixel Layout> Figure 23 shows an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0182] In this example, in the pixel array section 141, the unit regions UA3 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0183] The unit region UA3 consists of four R pixels, eight G pixels, four B pixels, and sixteen event pixels PXE.

[0184] Each color pixel has the same shape as the large pixels in Figure 5. Each event pixel PXE has the same shape as the small pixels in Figure 5.

[0185] In the unit region UA3, subblocks composed of R pixels, subblocks composed of G pixels, and subblocks composed of B pixels are arranged in a grid according to the Bayer array. In the R pixel subblocks, the R pixels are arranged in a 2x2 grid. In each G pixel subblock, the G pixels are arranged in a 2x2 grid. In the B pixel subblocks, the B pixels are arranged in a 2x2 grid.

[0186] Each event pixel PXE is positioned in the gaps between each color pixel, similar to the event pixel PXE in Figure 5, and is arranged in a 4x4 grid.

[0187] This pixel layout has the same characteristics as the pixel layout in Figure 5.

[0188] For example, the pixel layout has approximate point symmetry. Specifically, Figure 24 shows a portion of the pixel layout of Figure 23 rotated by 180 degrees. Even when the pixel layout is rotated by 180 degrees, the area excluding the edges of the pixel array portion 141 remains a layout in which the unit region UA3 is repeatedly arranged in the row and column directions.

[0189] Furthermore, for example, the number of adjacent event pixels (PXEs) is similar around each color pixel. There are no missing color pixels in each unit region (UA3). Event pixels (PXEs) are arranged at equal intervals.

[0190] <Example of circuit configuration of grayscale pixel unit PUG3> Figure 25 shows an example of the circuit configuration of grayscale pixel unit PUG3 used in the unit region UA3 of Figure 23.

[0191] Note that the same reference numerals are used for the parts corresponding to the grayscale pixel unit PUG1 in Figure 7, and their explanations are omitted as appropriate.

[0192] Compared to the grayscale pixel unit PUG1, the grayscale pixel unit PUG3 differs in that it has four sets of photodiodes 161 and transfer transistors TG.

[0193] Each photodiode 161 corresponds to a photodiode 161 of one of the four color pixels within a subblock of the unit region UA3.

[0194] Each photodiode 161 has its anode terminal grounded and its cathode terminal connected to the floating diffusion region FD via a transfer transistor TG.

[0195] When each transfer transistor TG is turned on by a transfer drive signal supplied to its gate electrode, it reads the charge generated by each photodiode 161 and transfers it to the floating diffusion region FD.

[0196] In this way, the readout circuit 162 is shared by the photodiodes 161 of the four grayscale pixels PXG within the subblock.

[0197] This allows multiple color pixels within a subblock of the unit region UA3 to be binned, thereby increasing the output value of the luminance signal.

[0198] <Example of circuit configuration of event pixel unit PUE2a> Figure 26 shows an example of the circuit configuration of event pixel unit PUE2 used in the unit region UA3 of Figure 23.

[0199] Note that the subtractor 203, quantizer 204, and transfer circuit 205 are the same as those in the event pixel unit PUE1a in Figures 8 and 9, and their illustrations are omitted. Also, the same reference numerals are used for parts corresponding to those in Figure 9, and their explanations are omitted as appropriate.

[0200] The event pixel unit PUE2a differs from the event pixel unit PUE1a in that it has four photodiodes 171.

[0201] Each photodiode 171 corresponds to the photodiode 171 of the four event pixel PXE within the subblock of the unit region UA3.

[0202] The cathode of each photodiode 171 is connected to the connection point between the source of transistor 211 and the gate of transistor 213 in the current-voltage conversion circuit 201. The photocurrent of each photodiode 171 flows to the connection point between the source of transistor 211 and the gate of transistor 213. An optical voltage VLOG corresponding to the logarithm of the photocurrent flowing through transistor 211 appears at the gate of transistor 211 and is output to the subtractor 203 via the buffer 202 from the connection point between the gate of transistor 211 and the drain of transistor 213. In other words, the optical voltage VLOG corresponding to the sum of the photocurrents of the four photodiodes 171 is output to the subtractor 203.

[0203] In this way, the current-voltage conversion circuit 201 (which includes the address event detection circuit 172) is shared by the photodiodes 171 of the four event pixels PXE within the subblock.

[0204] <Example of circuit configuration of unit region UA3> Figure 27 shows an example of the circuit configuration of unit region UA3 in Figure 23.

[0205] Note that some of the reference numerals have been omitted due to space limitations in the drawing. Also, in the event pixel unit PUE2a, only the current-voltage conversion circuit 201, which is part of the address event detection circuit 172, is shown.

[0206] The subblocks of color pixels for each color in the unit region UA3 use the grayscale pixel units PUG2 shown in Figure 25. Grayscale pixel units PUG2 arranged in the same column share the vertical signal line VSL.

[0207] In one of the four subblocks of the event pixel PXE in the unit region UA3, the event pixel unit PUE2a shown in Figure 26 is used, and in the subblocks of the other event pixel PXEs, the event pixel unit PUE2b is used.

[0208] The event pixel unit PUE2b comprises only four photodiodes 171 and does not have an address event detection circuit 172. The cathode of each photodiode 171 in the event pixel unit PUE2b is connected to the cathode of each photodiode 171 in the event pixel unit PUE2a. The address event detection circuit 172 of the event pixel unit PUE2a is shared by the event pixels PXE within the unit region UA2.

[0209] As a result, multiple event pixels PXE within the unit region UA3 are binned, increasing the output value of the event signal and making it easier to detect events. Even when using binned event pixels PXE within the unit region UA3, event signals can be acquired uniformly within the pixel array 141 without spatial bias.

[0210] <<4. Fourth Embodiment>> Next, a fourth embodiment of the present technology will be described with reference to Figures 28 and 29.

[0211] Comparing the fourth embodiment of this technology with the third embodiment, the pixel layout is different.

[0212] <Example of Pixel Layout> Figure 28 shows an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0213] In this example, in the pixel array section 141, the unit regions UA4 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0214] Comparing unit region UA4 with unit region UA3 in Figure 23, the difference is that each color pixel is composed of phase difference pixels.

[0215] This pixel layout has similar characteristics to the pixel layout in Figure 23. For example, the pixel layout has approximate point symmetry. The amount of adjacent event pixels (PXE) around each color pixel is the same. There are no missing color pixels in each unit region (UA4). The event pixels (PXE) are arranged at equal intervals.

[0216] Furthermore, it is also possible to configure only some of the color pixels to be composed of phase-difference pixels.

[0217] <Example of circuit configuration of grayscale pixel unit PUG4> Figure 29 shows an example of the circuit configuration of grayscale pixel unit PUG4 used in the unit region UA4 of Figure 28.

[0218] Note that the same reference numerals are used for the parts corresponding to the grayscale pixel unit PUG1 in Figure 7, and their explanations are omitted as appropriate.

[0219] Compared to the grayscale pixel unit PUG1, the grayscale pixel unit PUG4 differs in that it has eight sets of photodiodes 161 and transfer transistors TG.

[0220] Each photodiode 161 corresponds to one of the eight photodiodes 161 of the four grayscale pixels PXG within the subblock of the unit region UA4.

[0221] Each photodiode 161 has its anode terminal grounded and its cathode terminal connected to the floating diffusion region FD via a transfer transistor TG.

[0222] When each transfer transistor TG is turned on by a transfer drive signal supplied to its gate electrode, it reads the charge generated by each photodiode 161 and transfers it to the floating diffusion region FD.

[0223] In this way, the readout circuit 162 is shared by the eight photodiodes 161 of the four grayscale pixels PXG within the subblock.

[0224] Furthermore, in the circuit configuration example shown in Figure 27, the circuit for the unit region UA4 is constructed by using the grayscale pixel unit PUG4 instead of the grayscale pixel unit PUG3.

[0225] <<5. Fifth Embodiment>> Next, a fifth embodiment of the present technology will be described with reference to Figures 30 to 33.

[0226] Comparing the fifth embodiment with the first embodiment, the difference is that instead of event pixels PXE, grayscale pixels PXG equipped with a W filter (hereinafter referred to as W pixels) are arranged.

[0227] <Example of Pixel Layout> Figure 30 shows an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0228] In this example, in the pixel array section 141, the unit regions UA5 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0229] Comparing unit region UA5 with unit region UA1 in Figure 5, the difference is that each event pixel PXE has been replaced with a W pixel.

[0230] The W pixels acquire luminance signals with gradation levels corresponding to each wavelength band.

[0231] Similar to the pixel layout in Figure 5, this pixel layout also exhibits approximate point symmetry. There are no missing color pixels in each unit region UA5.

[0232] Furthermore, in this pixel layout, the amount of W pixels adjacent to each color pixel is similar. This suppresses the occurrence of sensitivity differences between color pixels due to color mixing from W pixels. In other words, the color mixing from W pixels is similar for each color pixel, and differences in the output brightness signals of the gradation levels output from each color pixel are suppressed.

[0233] Furthermore, because the W pixels are arranged at equal intervals, the white luminance signal can be acquired uniformly within the pixel array 141 without spatial bias.

[0234] <Example of circuit configuration of unit region UA5> Figure 31 shows an example of the circuit configuration of unit region UA5 in Figure 30.

[0235] Note that some of the reference numerals have been omitted due to space limitations in the drawing. Also, in Figure 31, the photodiodes 161 that constitute the grayscale pixels PXG are distinguished into photodiodes 161L and photodiodes 161S based on their light-receiving area. Photodiode 161L has a larger light-receiving area than photodiode 161S.

[0236] The grayscale pixel units PUG1a and PUG1b have the same circuitry as the grayscale pixel unit PUG1 shown in Figure 7.

[0237] The grayscale pixel unit PUG1a is equipped with a photodiode 161L and is used for large color pixels. The grayscale pixel unit PUG1b is equipped with a photodiode 161S and is used for small W pixels.

[0238] The grayscale pixel units PUG1a located in the same column share the vertical signal line VSL, and the grayscale pixel units PUG1b located in the same column also share the vertical signal line VSL.

[0239] <Examples of Cross-Sectional Configuration of Solid-State Imaging Device 112> Figures 32 and 33 show examples of partial cross-sectional configurations of a solid-state imaging device 112 equipped with a unit region UA5. Figure 32 shows an example of a cross-section of an arrangement of R pixels, W pixels, and B pixels arranged diagonally in the unit region UA5. Figure 33 shows an example of a cross-section of an arrangement of G pixels, W pixels, and G pixels arranged diagonally in the unit region UA5.

[0240] Note that the same reference numerals are used for parts corresponding to Figures 13, 14, 17, and 18, and their explanations are omitted as appropriate.

[0241] The cross-sectional configuration of the solid-state imaging device 112 in Figures 32 and 33 is substantially the same as the cross-sectional configuration of the solid-state imaging device 112 in Figures 17 and 18. The only difference is that in the W pixels, a photodiode 161S is used instead of a photodiode 171.

[0242] <<6. Sixth Embodiment>> Next, a sixth embodiment of the present technology will be described with reference to Figure 34.

[0243] Comparing the sixth embodiment of this technology with the fifth embodiment, the pixel layout is different.

[0244] <Example of Pixel Layout> Figure 34 shows an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0245] In this example, in the pixel array section 141, the unit regions UA6 enclosed by the dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0246] Comparing unit region UA6 with unit region UA5 in Figure 30, the difference is that each W pixel is composed of phase difference pixels. That is, the photodiodes 161S are separated to the left and right (in the row direction) so that the phase difference characteristics can be detected in each W pixel.

[0247] For example, by using binning for each phase difference pixel, the signal amount of the phase difference signal indicating the detection result of the phase difference characteristic can be increased, and a different type of signal from the R, G, and B grayscale pixels PXG can be detected. This increases the added value of the solid-state imaging device 112.

[0248] This pixel layout has similar characteristics to the pixel layout in Figure 30. For example, the pixel layout has approximate point symmetry. The amount of adjacent W pixels is the same around each color pixel. There are no missing color pixels in each unit region UA6. The W pixels are arranged at equal intervals.

[0249] Furthermore, it is also possible to have only some of the W pixels composed of phase-difference pixels.

[0250] Furthermore, pixels other than event pixels (PXEs) and phase-difference pixels that detect information different from that of color pixels may be used as small pixels.

[0251] For example, pixels equipped with narrowband filters that detect wavelengths narrower than those of color pixels may be used in at least some of the small pixels. This allows for the realization of multispectral sensors or hyperspectral sensors using small pixels.

[0252] For example, pixels equipped with SPAD (Single Photon Avalanche Diode) may be used for at least some of the smaller pixels.

[0253] <<7. Seventh Embodiment>> Next, a ninth embodiment of the present technology will be described with reference to Figures 35 to 37.

[0254] Comparing the seventh embodiment of this technology with the third embodiment described above, with reference to Figure 23, etc., the pixel layout is different.

[0255] <Examples of Pixel Layouts> Figures 35 to 37 show examples of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0256] In the example shown in Figure 35, in the pixel array section 141, the unit regions UA7 enclosed by the dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0257] Comparing unit region UA7 with unit region UA3 in Figure 23, some event pixels (PXE) are replaced with color pixels that are grayscale pixels (PXG). Specifically, in each subblock of a color, only the small pixels located in the center of the large 2x2 color pixels (hereinafter referred to as the first color pixels) are composed of event pixels (PXE), while the other small pixels are composed of color pixels of the same color (hereinafter referred to as the second color pixels).

[0258] In the example shown in Figure 36, in the pixel array section 141, the unit regions UA8 enclosed by the dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0259] Comparing unit region UA8 with unit region UA3 in Figure 23, it can be seen that some event pixels (PXEs) have been replaced by second color pixels. Specifically, in each subblock of a color, the first row of small pixels consists of second color pixels of the same color as the first color pixels, and the second row of small pixels consists of event pixels (PXEs).

[0260] In the example shown in Figure 37, in the pixel array section 141, the unit regions UA9 enclosed by the dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0261] Comparing unit region UA9 with unit region UA3 in Figure 23, we see that some event pixels (PXEs) have been replaced by second color pixels. Specifically, in each color subblock, the first row of small pixels consists of second color pixels of the same color as the first color pixels, and the second row of small pixels consists of event pixels (PXEs).

[0262] The pixel layouts in Figures 35 to 37 share some of the same characteristics as the pixel layout in Figure 23.

[0263] For example, the pixel layout has approximate point symmetry.

[0264] Furthermore, the amount of proximity to the event pixel PXE is similar around each first color pixel. That is, in the pixel layout of Figure 35, one of the diagonal edges of each first color pixel is adjacent to the event pixel PXE. In the pixel layout of Figure 36, two of the diagonal edges of each first color pixel that are aligned in the column direction are adjacent to the event pixel PXE. In the pixel layout of Figure 37, two of the diagonal edges of each first color pixel that are aligned in the row direction are adjacent to the event pixel PXE.

[0265] In the pixel layouts of Figures 35 to 37, the positions of adjacent event pixels (PXE) around each first color pixel are rotationally symmetrical. That is, by rotating each first color pixel and its adjacent pixels in a direction parallel to the light-receiving surface, the positions of adjacent event pixels (PXE) between each first color pixel coincide. For example, in the pixel layout of Figure 35, one of the diagonal edges of each first color pixel is adjacent to an event pixel (PXE), so by rotating each first color pixel and its adjacent pixels in a direction parallel to the light-receiving surface, the positions of adjacent event pixels (PXE) between each first color pixel coincide.

[0266] In the pixel layout shown in Figure 5 above, event pixels (PXE or W) are adjacent to all four diagonal sides of each color pixel. Therefore, it can be said that the positions adjacent to event pixels (PEX or W) around each color pixel are rotationally symmetrical.

[0267] Furthermore, in unit regions UA7 to UA9, there are no missing first color pixels.

[0268] <<8. Eighth Embodiment>> Next, an eighth embodiment of the present technology will be described with reference to Figures 38 to 40.

[0269] The eighth embodiment of this technology is compared with the seventh embodiment in terms of the pixel layout.

[0270] <Examples of Pixel Layouts> Figures 38 to 40 show examples of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0271] In the example shown in Figure 38, in the pixel array section 141, the unit regions UA10 enclosed by the dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0272] Comparing unit region UA10 with unit region UA7 in Figure 35, the difference is that the first color pixels are composed of phase difference pixels.

[0273] In the example shown in Figure 39, in the pixel array section 141, the unit regions UA11 enclosed by the dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0274] Comparing unit region UA11 with unit region UA8 in Figure 36, the difference is that the first color pixels are composed of phase difference pixels.

[0275] In the example shown in Figure 40, in the pixel array section 141, the unit regions UA12 enclosed by the dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0276] Comparing unit region UA12 with unit region UA7 in Figure 37, the difference is that the first color pixels are composed of phase difference pixels.

[0277] The pixel layouts in Figures 38 to 40 have the same characteristics as the pixel layouts in Figures 35 to 37. For example, the pixel layout has approximate point symmetry. The amount of adjacent event pixels PXE is the same around each first color pixel. In each first color pixel, the positions of adjacent event pixels PEX are rotationally symmetric. In unit regions UA10 to UA12, there are no missing first color pixels.

[0278] <<9. The Ninth Embodiment>> Next, the ninth embodiment of the present technology will be described with reference to Figures 41 to 48.

[0279] Comparing the ninth embodiment of this technology with the first embodiment, the pixel layout is different.

[0280] <Example of Pixel Layout Configuration> Figure 41 shows a part of an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0281] In this example, in the pixel array section 141, the unit regions UA13 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0282] In the unit region UA13, a 2x2 subblock containing R pixels, a 2x2 subblock containing G pixels, and a 2x2 subblock containing B pixels are arranged in a grid according to the Bayer array.

[0283] In the subblock of the R pixels, the R pixels are arranged diagonally. Also, in the subblock of the R pixels, the event pixels PXE and W pixels are arranged diagonally opposite to the R pixels.

[0284] In the subblock of the G pixels, the G pixels are arranged diagonally. Also, in the subblock of the G pixels, the event pixels (PXE) and W pixels are arranged diagonally opposite to the G pixels.

[0285] In the subblock of B pixels, the B pixels are arranged diagonally. Also, in the subblock of B pixels, the event pixels PXE and W pixels are arranged diagonally opposite to the B pixels.

[0286] In this way, in each subblock of a color, the color pixels, event pixels (PXE), and W pixels are arranged in a similar layout.

[0287] Furthermore, as will be described later, event pixels (PXE) and W pixels do not have optical filters. Hereafter, pixels without optical filters will be referred to as filterless pixels.

[0288] Figure 42 shows a portion of the pixel layout from Figure 41 rotated 180 degrees.

[0289] If we consider the filterless pixels as pixels of the same type, this pixel layout has approximately point symmetry. That is, even if the pixel layout is rotated 180 degrees, the area excluding the edges of the pixel array portion 141 will have a layout in which the unit region UA13 is repeatedly arranged in the row and column directions.

[0290] Furthermore, in this pixel layout, the amount of adjacent filterless pixels around each color pixel is similar in the region excluding the edges of the pixel array 141. That is, in the region excluding the edges of the pixel array 141, filterless pixels are adjacent to all four sides of each rectangular color pixel. As a result, in the region excluding the edges of the pixel array 141, differences in sensitivity among the color pixels due to color mixing from the filterless pixels are suppressed. In other words, the color mixing from the filterless pixels is similar for each color pixel, and differences in the output of the luminance signal of the gradation level output from each color pixel are suppressed.

[0291] Furthermore, because the event pixels PXE are arranged at equal intervals, event signals can be acquired uniformly within the pixel array 141 without spatial bias, thereby improving the resolution of event detection.

[0292] <Example of circuit configuration of grayscale pixel unit PUG5> Figure 43 shows an example of the circuit configuration of grayscale pixel unit PUG5 used in the unit region UA13 of Figure 41.

[0293] Note that the same reference numerals are used for the parts corresponding to the grayscale pixel unit PUG1 in Figure 7, and their explanations are omitted as appropriate.

[0294] Compared to the grayscale pixel unit PUG1, the grayscale pixel unit PUG5 differs in that it has three sets of photodiodes 161 and transfer transistors TG.

[0295] Each photodiode 161 corresponds to the photodiode 161 of the color pixel and W pixel of the subblock of the unit region UA13, respectively.

[0296] Each photodiode 161 has its anode terminal grounded and its cathode terminal connected to the floating diffusion region FD via a transfer transistor TG.

[0297] When each transfer transistor TG is turned on by a transfer drive signal supplied to its gate electrode, it reads the charge generated by each photodiode 161 and transfers it to the floating diffusion region FD.

[0298] In this way, the readout circuit 162 is shared by the three photodiodes 161.

[0299] <Example of circuit configuration of unit region UA13> Figure 44 shows an example of circuit configuration of unit region UA13.

[0300] Note that the same reference numerals are used for the parts corresponding to the circuit of the unit region UA1 in Figure 11, and their explanations are omitted as appropriate.

[0301] A grayscale pixel unit PUG5 is used for the color pixels and W pixels of each subblock of the unit region UA13. Grayscale pixel units PUG5 arranged in the same column share the vertical signal line VSL.

[0302] One of the event pixel PXEs in the unit region UA13 uses the event pixel unit PUE1a shown in Figure 9, while the other event pixel PXEs use the event pixel unit PUE1b shown in Figure 11.

[0303] As a result, multiple event pixels PXE within the unit region UA13 are binned, increasing the output value of the event signal and making it easier to detect events. Even when using binned event pixels PXE within the unit region UA13, event signals can be acquired uniformly within the pixel array 141 without spatial bias.

[0304] <Example of Cross-sectional Configuration of Solid-State Imaging Device 112> Figures 45 to 48 show examples of a part of the cross-sectional configuration of a solid-state imaging device 112 having the pixel layout of Figure 41. Figures 45 to 48 show examples of the cross-sections of the first to fourth rows of the unit region UA13 in Figure 44, respectively.

[0305] In the solid-state imaging device 112, the on-chip lens layer 401, optical filter layer 402, semiconductor substrate 403, wiring layer 404, and support substrate 405 are stacked in order from the incident light side. The on-chip lens layer 401, optical filter layer 402, semiconductor substrate 403, wiring layer 404, and support substrate 405 are the same as the on-chip lens layer 301, optical filter layer 302, semiconductor substrate 303, wiring layer 304, and support substrate 305 in Figure 13.

[0306] In the R pixel, a microlens 421, an R filter 422R, an interlayer insulating film 423, a pinning film 424 having a negative fixed charge, a photodiode 161, and a P-type pinning region 425 are stacked on top of the wiring layer 404.

[0307] In the G pixel, a microlens 421, a G filter 422G, an interlayer insulating film 423, a pinning film 424, a photodiode 161, and a pinning region 425 are stacked on top of the wiring layer 404.

[0308] In the B pixel, a microlens 421, a B filter 422B, an interlayer insulating film 423, a pinning film 424, a photodiode 161, and a pinning region 425 are stacked on top of the wiring layer 404.

[0309] In the W pixel, a microlens 421, an interlayer insulating film 423, a pinning film 424, a photodiode 161, and a pinning region 425 are stacked on top of the wiring layer 404. The W pixel does not have an optical filter.

[0310] In the event pixel PXE, a microlens 421, an interlayer insulating film 423, a pinning film 424, a photodiode 171, and a pinning region 425 are stacked on top of the wiring layer 404. The event pixel PXE does not have an optical filter.

[0311] <<10. Tenth Embodiment>> Next, a tenth embodiment of the present technology will be described with reference to Figure 49.

[0312] Comparing the tenth embodiment of this technology with the ninth embodiment, the pixel layout is different.

[0313] <Example of Pixel Layout Configuration> Figure 49 shows a part of an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0314] In this example, in the pixel array section 141, the unit regions UA14 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0315] Comparing unit region UA14 with unit region UA13 in Figure 41, the difference is that each color pixel is composed of phase difference pixels.

[0316] This pixel layout has similar characteristics to the pixel layout in Figure 41. That is, the pixel layout has approximately point symmetry. In the region excluding the edges of the pixel array 141, the amount of adjacent filterless pixels around each color pixel is similar, and the positions of adjacent filterless pixels are rotationally symmetric. Event pixels PXE are arranged at equal intervals.

[0317] Furthermore, it is also possible to configure only some of the color pixels to be composed of phase-difference pixels.

[0318] <<11. Eleventh Embodiment>> Next, an eleventh embodiment of the present technology will be described with reference to Figure 50.

[0319] Comparing the eleventh embodiment of this technology with the ninth embodiment, the pixel layout is different.

[0320] <Example of Pixel Layout Configuration> Figure 50 shows a part of an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0321] In this example, in the pixel array section 141, the unit regions UA15 enclosed by dotted lines are repeatedly arranged in the row direction (horizontal direction) and the column direction (vertical direction).

[0322] Comparing unit region UA15 with unit region UA13 in Figure 41, the arrangement of W pixels and event pixels PXE differs in each subblock of each color.

[0323] Specifically, the pixel arrangement of the R pixel subblock is the same as the pixel arrangement of the R pixel subblock in Figure 41. In the G pixel subblock, the event pixel PXE is replaced by the W pixel. In the B pixel subblock, the arrangement of the W pixel and the event pixel PXE is reversed.

[0324] Similar to the pixel layout in Figure 41, this pixel layout also exhibits approximate point symmetry. Similar to the pixel layout in Figure 41, in the region excluding the edges of the pixel array 141, the amount of adjacent filterless pixels around each color pixel is similar, and the positions of adjacent filterless pixels are rotationally symmetrical.

[0325] <<12. Twelfth Embodiment>> Next, a twelfth embodiment of the present technology will be described with reference to Figure 51.

[0326] Comparing the twelfth embodiment of this technology with the ninth embodiment, the pixel layout is different.

[0327] <Example of Pixel Layout Configuration> Figure 51 shows a part of an example of the layout of each pixel PX in the pixel array section 141 of Figure 4.

[0328] In this example, color pixels and event pixels (PXEs) are arranged alternately in odd-numbered rows. Specifically, a unit region UA16 consisting of an R pixel, an event pixel (PXE), a G pixel, an event pixel (PXE), a B pixel, and an event pixel (PXE) arranged in the row direction is repeated in the row direction in each odd-numbered row.

[0329] In even-numbered rows, color pixels and W pixels are arranged alternately. Specifically, a unit region UA17 in which R pixels, W pixels, G pixels, W pixels, B pixels, and W pixels are arranged in the row direction is repeated in the row direction in each even-numbered row.

[0330] Furthermore, unit regions UA16 and UA17 are arranged such that color pixels of the same color are aligned diagonally.

[0331] Similar to the pixel layout in Figure 41, in the region excluding the edges of the pixel array 141, the amount of adjacent filterless pixels around each color pixel is the same, and the positions of adjacent filterless pixels are rotationally symmetric. Similar to the pixel layout in Figure 41, event pixels PXE are arranged at equal intervals.

[0332] <<13. Modifications>> Modifications of the embodiments of the present technology described above will be explained below.

[0333] For example, in the pixel layout shown in Figure 5, a large pixel may have an octagonal shape, where the four corners of a square are cut out at angles different from 45° to the row and column directions. For example, a large pixel may have an octagonal shape, where the four corners of a rectangle are cut out. Accordingly, for example, a small pixel may be a rhombus other than a square.

[0334] The "region excluding the edges of the pixel array 141" mentioned above is the region excluding the area of ​​1 to n pixels from the edges of the pixel array 141, and the value of n varies depending on the pixel layout.

[0335] <<14. Examples of Image Sensor Use>> Figure 52 shows an example of use when the above-mentioned solid-state imaging device 112 is used as an image sensor.

[0336] The solid-state imaging device 112 described above can be used as an image sensor in various cases where light such as visible light, infrared light, ultraviolet light, and X-rays is sensed, for example, as follows.

[0337] - Devices that capture images for viewing purposes, such as digital cameras and portable devices with camera functions. - Devices used for traffic purposes, such as in-vehicle sensors that capture images of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping and recognition of the driver's condition, surveillance cameras that monitor moving vehicles and roads, and distance measuring sensors that measure distances between vehicles. - Devices used in home appliances such as TVs, refrigerators, and air conditioners that capture user gestures and allow device operation according to those gestures. - Devices used for medical and healthcare purposes, such as endoscopes and devices that perform angiography using infrared light reception. - Devices used for security purposes, such as surveillance cameras for crime prevention and cameras for person recognition. - Devices used for beauty purposes, such as skin measuring devices that capture images of skin and microscopes that capture images of the scalp. - Devices used for sports purposes, such as action cameras and wearable cameras for sports use. - Devices used for agriculture, such as cameras that monitor the condition of fields and crops.

[0338] <<15. Examples of Application to Mobile Devices>> The technology relating to this disclosure (this technology) can be applied to various products. For example, the technology relating to this disclosure may be implemented as a device mounted on any type of mobile device such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0339] Figure 53 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0340] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 53, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0341] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0342] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0343] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0344] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0345] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0346] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0347] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0348] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0349] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 53, the output devices include an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0350] Figure 54 shows an example of the installation position of the imaging unit 12031.

[0351] In Figure 54, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0352] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0353] Figure 54 shows an example of the imaging ranges of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0354] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0355] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0356] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0357] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0358] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to the imaging unit 12031 of the configuration described above. Specifically, the imaging device 101 in Figure 3 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, it becomes possible, for example, to detect obstacles more quickly. <<16. Others>>

[0359] The components (blocks) of the apparatus illustrated in this specification are functional conceptual blocks, and the actual apparatus does not need to have the illustrated configuration. That is, the apparatus can have any configuration in which the functions of the illustrated components are divided and / or integrated into any unit, for example, a configuration having one block in which the functions of all components are integrated.

[0360] The embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0361] <Examples of configuration combinations> This technology can also be configured as follows:

[0362] (1) A solid-state imaging device comprising a pixel array portion having a plurality of first pixels that output a luminance signal of a gradation level corresponding to the amount of incident light, and a plurality of second pixels that detect information different from that of the first pixels, arranged in a regular manner, wherein in the region excluding the ends of the pixel array portion, the amount of adjacent second pixels around each first pixel is the same. (2) The solid-state imaging device according to (1), wherein each of the first pixels is a pixel that detects visible light in a predetermined wavelength band. (3) The solid-state imaging device according to (2), wherein each of the second pixels is equipped with an optical filter that transmits white light, or is not equipped with an optical filter. (4) The solid-state imaging device according to (3), wherein at least a portion of the second pixels are event pixels that detect changes in luminance as events. (5) The solid-state imaging device according to (3) or (4), wherein at least a portion of the second pixels are pixels that output a luminance signal of a gradation level corresponding to the amount of incident light which is white light. (6) The solid-state imaging apparatus according to any one of (1) to (5), wherein, in the region excluding the ends of the pixel array, the positions of adjacent second pixels around each first pixel are rotationally symmetric. (7) The solid-state imaging apparatus according to (1) to (6), wherein at least a portion of the second pixels are event pixels that detect brightness changes as events. (8) The solid-state imaging apparatus according to (7), wherein the event pixels are arranged at equal intervals. (9) The solid-state imaging apparatus according to (1) to (8), wherein the first pixels and the second pixels are of different sizes. (10) The solid-state imaging apparatus according to (9), wherein the first pixels are arranged in a grid pattern, and the second pixels are arranged in a grid pattern in the gaps between the first pixels. (11) The solid-state imaging apparatus according to (10), wherein the first pixels are octagons with the four corners of a rectangle cut off, and the second pixels are arranged in a region enclosed by the diagonal sides of four adjacent first pixels. (12) The solid-state imaging apparatus according to (10) or (11), wherein at least a portion of at least one of the second pixels is a pixel capable of detecting a phase difference.(13) The solid-state imaging apparatus according to (10) or (11), wherein at least a portion of the second pixels are pixels that detect light in a narrower wavelength band than the first pixels. (14) The solid-state imaging apparatus according to (10) or (11), wherein at least a portion of the second pixels are equipped with SPAD (Single Photon Avalanche Diode). (15) The solid-state imaging apparatus according to any one of (1) to (14), wherein the first pixels and the second pixels have different timings for reading out signals. (16) The solid-state imaging apparatus according to (15), comprising a first readout circuit for reading out the signal of the first pixel and a second readout circuit for reading out the signal of the second pixel asynchronously with respect to the first readout circuit. (17) The solid-state imaging apparatus according to (16), wherein the second readout circuit is capable of summing and reading out the signals of a plurality of the second pixels. (18) A solid-state imaging device according to any one of (1) to (17), wherein at least a portion of the first pixels are pixels capable of detecting a phase difference. (19) An electronic device comprising a pixel array portion having a plurality of first pixels that output a luminance signal of a gradation level corresponding to the amount of incident light, and a plurality of second pixels that detect information different from that of the first pixels, arranged in a regular manner, wherein in a region excluding the ends of the pixel array portion, the amount of adjacent second pixels around each first pixel is similar.

[0363] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0364] 101 Imaging device, 112 Solid-state imaging device, 113 Control unit, 114 Data processing unit, 121 Data generation unit, 141 Pixel array unit, 142 Drive unit, 143 Arbiter, 144 Event signal processing unit, 145 Brightness signal processing unit, PXG Grayscale pixels, PXE Event pixels, UA1 to UA16 Unit areas, PUG1 to PUG5 Grayscale pixel units, PUE1a to PUE2b Event pixel units

Claims

1. A solid-state imaging device comprising a pixel array section having a plurality of first pixels that output luminance signals of a gradation level corresponding to the amount of incident light, and a plurality of second pixels that detect information different from that of the first pixels, arranged in a regular pattern, wherein, in a region excluding the edges of the pixel array section, the amount of adjacent second pixels around each first pixel is similar.

2. The solid-state imaging apparatus according to claim 1, wherein each of the first pixels is a pixel that detects visible light in a predetermined wavelength band.

3. The solid-state imaging apparatus according to claim 2, wherein each of the second pixels is equipped with an optical filter that transmits white light, or is not equipped with an optical filter.

4. The solid-state imaging device according to claim 3, wherein at least a portion of the second pixels is an event pixel that detects a change in brightness as an event.

5. The solid-state imaging apparatus according to claim 3, wherein at least a portion of the second pixels is a pixel that outputs a luminance signal of a gradation level corresponding to the amount of incident light which is white light.

6. The solid-state imaging apparatus according to claim 1, wherein, in a region excluding the edges of the pixel array, the positions of adjacent second pixels around each first pixel are rotationally symmetrical.

7. The solid-state imaging apparatus according to claim 1, wherein at least a portion of the second pixels is an event pixel that detects a change in brightness as an event.

8. The solid-state imaging apparatus according to claim 7, wherein the event pixels are arranged at equal intervals.

9. The solid-state imaging apparatus according to claim 1, wherein the sizes of the first pixel and the second pixel are different.

10. The solid-state imaging apparatus according to claim 9, wherein the first pixels are arranged in a grid pattern, and the second pixels are arranged in a grid pattern in the gaps between the first pixels.

11. The solid-state imaging apparatus according to claim 10, wherein the first pixel is an octagon formed by cutting off the four corners of a rectangle, and the second pixel is located in a region enclosed by the diagonal sides of four adjacent first pixels.

12. The solid-state imaging apparatus according to claim 10, wherein at least a portion of at least one of the second pixels is a pixel capable of detecting a phase difference.

13. The solid-state imaging apparatus according to claim 10, wherein at least a portion of the second pixel is a pixel that detects light in a narrower wavelength band than the first pixel.

14. The solid-state imaging apparatus according to claim 10, wherein at least a portion of the second pixels is a SPAD (Single Photon Avalanche Diode).

15. The solid-state imaging apparatus according to claim 1, wherein the first pixel and the second pixel have different timings for reading out signals.

16. The solid-state imaging apparatus according to claim 15, further comprising a first readout circuit for reading the signal of the first pixel, and a second readout circuit for reading the signal of the second pixel asynchronously with respect to the first readout circuit.

17. The solid-state imaging apparatus according to claim 16, wherein the second readout circuit is capable of summing and reading out the signals of a plurality of the second pixels.

18. The solid-state imaging apparatus according to claim 1, wherein at least a portion of the first pixels are pixels capable of detecting a phase difference.

19. An electronic device comprising a pixel array portion having a plurality of first pixels that output a luminance signal of a gradation level corresponding to the amount of incident light, and a plurality of second pixels that detect information different from that of the first pixels, arranged in a regular manner, wherein in a region excluding the edges of the pixel array portion, the amount of adjacent second pixels around each first pixel is similar.

Images