Image sensor

Abstract

The present invention improves robustness with respect to an imaging environment in an image sensor that detects and processes an edge.　A vertical scanning circuit according to the present invention performs: control for driving, in a first exposure mode, a pixel array unit in which a plurality of pixels are arranged, and outputting a pair of pixel signals as a pair of first pixel signals; and control for driving the pixel array unit in a second exposure mode and outputting a pair of pixel signals as a pair of second pixel signals. An edge determination circuit performs a process for determining whether an edge is present on the basis of the pair of first pixel signals, and a process for determining whether an edge is present on the basis of the pair of second pixel signals. A synthesis unit synthesizes signals from the edge determination circuit.

Description

Image sensor

[0001] This technology relates to image sensors. More specifically, it relates to image sensors that can determine the presence or absence of edges.

[0002] Conventionally, various processes such as edge detection and object recognition have been performed on image signals in image sensors. For example, an image sensor has been proposed in which a vertical scanning circuit selects a pair of pixels at both ends of a line segment that does not belong to either a row or a column and whose angle with the row is different from 45 degrees, and outputs a pair of pixel signals (see, for example, Patent Document 1). Edge detection processing is performed on the output pixel signals. Furthermore, object recognition processing can also be performed on the edge image.

[0003] International Publication No. 2023 / 286279

[0004] In the conventional technology described above, the vertical scanning circuit attempts to reduce the sample-and-hold circuit by selecting a pair of pixels at both ends of a predetermined line segment that does not belong to either a row or a column and whose angle with the row is different from 45 degrees. However, in the above-described image sensor, when detecting edges in imaging environments with large differences in brightness, the exposure time may be too short, resulting in insufficient signal, or the exposure time may be too long, resulting in signal saturation. These signal deficiencies or saturations can cause edges to disappear, which may result in a decrease in the robustness of the image sensor to the imaging environment.

[0005] This technology was developed in light of these circumstances, and aims to improve robustness to the imaging environment in image sensors that detect and process edges.

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an image sensor comprising: a vertical scanning circuit that controls a pixel array section, which has multiple pixels arranged in a sequence, to drive in a first exposure mode to output a pair of pixel signals as a pair of first pixel signals, and a control that drives the pixel array section in a second exposure mode to output a pair of pixel signals as a pair of second pixel signals; an edge detection circuit that processes to determine the presence or absence of an edge based on the pair of first pixel signals and the presence or absence of an edge based on the pair of second pixel signals; and a combining unit that combines the signals from the edge detection circuit. This results in improved robustness to the imaging environment.

[0007] Furthermore, in this first aspect, the exposure time of pixels driven in the first exposure mode may differ from the exposure time of pixels driven in the second exposure mode. This results in the effect that edges do not disappear even when there is a difference in brightness.

[0008] Furthermore, in this first aspect, the sensitivity of the first pixel driven in the first exposure mode may be different from the sensitivity of the second pixel driven in the second exposure mode. This results in the effect that edges do not disappear even when there is a difference in brightness.

[0009] Furthermore, in this first aspect, at least one of the first pixel signal and the second pixel signal may be a signal obtained by adding up multiple pixel signals. This results in an increase in the signal amount of the first exposure mode.

[0010] Furthermore, in this first aspect, the vertical scanning circuit may start exposure in the second exposure mode when exposure in the first exposure mode ends. This allows for the reading of pixel signals with different exposure times from the same pixel.

[0011] Furthermore, in this first aspect, the layout may be such that the first pixels and the second pixels are not mixed in each row within the pixel array. This eliminates the need to drive each window row in both the first and second exposure modes.

[0012] Also, in this first aspect, the pair of pixels may be arranged in a layout in which the first pixel and the second pixel are not mixed in each column within the pixel array unit. This brings about an effect that it is not necessary to hold the first determination result.

[0013] Also, in this first aspect, the edge determination circuit includes a gain control circuit that performs control to increase or decrease at least one of the pair of pixel signals by a predetermined gain as gain control, an offset control circuit that performs control to superimpose a predetermined offset on at least one of the pair of pixel signals as offset control, and a comparison circuit that compares the pair of pixel signals on which the gain control and the offset control have been performed and outputs a comparison result. This brings about an effect that the detection accuracy of the edge is improved.

[0014] Also, in this first aspect, it may further include an analog-to-digital converter that converts a pixel signal into a digital signal, and the analog-to-digital converter and the gain control circuit may share the comparison circuit. This brings about an effect that the circuit scale is reduced.

[0015] Also, in this first aspect, the gain control circuit includes a first gain control circuit that increases or decreases one of the pair of pixel signals by the gain, and a second gain control circuit that increases or decreases the other of the pair of pixel signals by the gain. The comparison circuit includes a first comparison circuit that compares the signal increased or decreased by the first gain control circuit with the other signal, and a second comparison circuit that compares the signal increased or decreased by the second gain control circuit with the one signal. This brings about an effect that information indicating the direction of the edge can be output.

[0016] Also, in this first aspect, the combining unit may sequentially generate and combine first and second determination results indicating the presence or absence of an edge based on the respective comparison results of the first and second comparison circuits. This brings about an effect that the determination results of each exposure mode are compressed.

[0017] Also, in this first aspect, the combining unit may include a NAND gate that outputs, as an input signal, the logical product of the negations of the respective comparison results of the first and second comparison circuits, an OR gate that outputs the logical sum of the input signal and the feedback signal, and an AND gate that outputs the logical product of the output signal of the OR gate and the enable signal and feeds it back as the feedback signal. This brings about the effect of compressing the determination results of each exposure mode.

[0018] Also, in this first aspect, the combining unit may include a first AND gate that outputs, as an input signal, the logical product of the comparison results of the first and second comparison circuits, a second AND gate that outputs the logical product of the input signal and the feedback signal, and an OR gate that outputs the logical sum of the output signal of the second AND gate and the enable signal and feeds it back as the feedback signal. This brings about the effect of compressing the determination results of each exposure mode.

[0019] Also, in this first aspect, the combining unit may include an AND gate that outputs, as an input signal, the logical product of the comparison results of the first and second comparison circuits, a first NAND gate that outputs the logical product of the negations of the input signal and the feedback signal, and a second NAND gate that outputs the logical product of the negations of the output signal of the first NAND gate and the enable signal and feeds it back as the feedback signal. This brings about the effect of compressing the determination results of each exposure mode.

[0020] Also, in this first aspect, the combining unit may include a NAND gate that outputs, as an input signal, the logical product of the negations of the comparison results of the first and second comparison circuits, a latch circuit that holds the input signal according to the enable signal and outputs the held value, and an OR gate that outputs the logical sum of the output signal of the latch circuit and the input signal. This brings about the effect of compressing the determination results of each exposure mode.

[0021] Furthermore, in this first aspect, the edge determination circuit may further include a logic circuit that combines the comparison results of the first and second comparison circuits in the first exposure mode and outputs them as the first determination result, and combines the comparison results of the first and second comparison circuits in the second exposure mode and outputs them as the second determination result. This has the effect of reducing the amount of data in the determination result.

[0022] Furthermore, in this first aspect, the logic circuit and the synthesis unit may be arranged for each row of the pixel array. This has the effect of reducing the size of the subsequent circuits.

[0023] Furthermore, in this first aspect, the vertical scanning circuit drives a pair of pixels within the window of the pixel array to output the pair of pixel signals, and the pair of pixels may be located at both ends of a predetermined line segment that does not correspond to either a row or a column of the pixel array and has an angle with respect to the row that is different from 45 degrees. This eliminates the need for a sample-and-hold circuit.

[0024] Furthermore, in this first aspect, two adjacent windows may overlap in some areas. This results in a higher resolution of the edge image.

[0025] Furthermore, in this first aspect, the vertical scanning circuit may further control the pixel array to drive the pixel array in a third exposure mode to output a pair of pixel signals as a pair of third pixel signals, the edge determination circuit may further perform a process to determine the presence or absence of an edge based on the pair of third pixel signals and output a third determination result, and the synthesis unit may synthesize the first, second, and third determination results. This results in improved edge detection accuracy.

[0026] Furthermore, in this first aspect, the vertical scanning circuit may output a signal obtained by adding the pixel signal corresponding to the second exposure mode and the first pixel signal as the second pixel signal. This results in an improvement in readout speed.

[0027] Furthermore, in this first aspect, the plurality of pixels may include a first pixel and a second pixel that share an FD (Floating Diffusion), and the vertical scanning circuit may drive the first pixel in the first exposure mode and the second pixel in the second exposure mode. This results in an improved read speed in the FD-sharing structure.

[0028] Furthermore, in this first aspect, the plurality of pixels may include a first pixel having a first FD and a second pixel having a second FD, and the vertical scanning circuit may drive the first pixel in the first exposure mode and the second pixel in the second exposure mode while connecting the first FD and the second FD. This results in an improved read speed when the FD is not shared.

[0029] Furthermore, in this first aspect, an addition circuit may be provided that outputs the summation average of the first pixel signal and the second pixel signal. This provides the effect that the pixel signals of each exposure mode can be added together even in a configuration that does not share a floppy disk.

[0030] Furthermore, in this first aspect, a sample-and-hold circuit may be provided that samples and adds the first pixel signal and the second pixel signal. This provides the ability to add the pixel signals of each exposure mode even in a configuration where the floppy disk is not shared.

[0031] Furthermore, in this first aspect, the system may further include a line memory that holds a predetermined number of first determination results from the edge determination circuit, and the combining unit may combine the predetermined number of first determination results held in the line memory with a predetermined number of second determination results from the edge determination circuit. This results in a reduction in circuit size.

[0032] Furthermore, the second aspect of this technology is an image sensor comprising: a vertical scanning circuit that controls a pixel array section, which has multiple pixels arranged in a sequence, to drive in a first exposure mode to output a pair of pixel signals as a pair of first pixel signals, and a pixel array section that controls to drive in a second exposure mode to output a pair of pixel signals as a pair of second pixel signals; an edge detection circuit that processes to determine the presence or absence of an edge based on the pair of first pixel signals and to determine the presence or absence of an edge based on the pair of second pixel signals; a recognition unit that processes to recognize a predetermined object based on the signals from the edge detection circuit and to output first and second recognition results; and a processing result synthesis unit that synthesizes the result of processing the first recognition result and the result of processing the second recognition result. This results in improved robustness to the imaging environment.

[0033] This is a block diagram showing an example configuration of an imaging device in the first embodiment of this technology. This is a block diagram showing an example configuration of an image sensor in the first embodiment of this technology. This is a circuit diagram showing an example configuration of a vertical scanning circuit and an FD (Floating Diffusion) shared block in the first embodiment of this technology. This is an example of a cross-sectional view of a back-illuminated image sensor in the first embodiment of this technology. This is an example of a cross-sectional view of a front-illuminated image sensor in the first embodiment of this technology. This is a block diagram showing an example configuration of a column signal processing circuit in the first embodiment of this technology. This is a circuit diagram showing an example configuration of an offset control circuit and a gain control circuit in the first embodiment of this technology. This is a circuit diagram showing an example configuration of a comparison circuit in the first embodiment of this technology. This is a diagram showing an example of the control of the vertical scanning circuit and the column signal processing circuit when detecting an edge in the first exposure mode in the first embodiment of this technology. This is a diagram showing an example of the control of the vertical scanning circuit and the column signal processing circuit when detecting an edge in the second exposure mode in the first embodiment of this technology. This is a block diagram showing an example configuration of a signal processing circuit in the first embodiment of this technology. This is a plan view showing an example of a pixel driven when reading out the long-stored signal of the first window row in the first embodiment of this technology. This is a plan view showing an example of a pixel driven to read out a short signal from the first window row in the first embodiment of this technology. This is a plan view showing an example of a pixel driven when reading out a long signal from the second window row in the first embodiment of this technology. This is a plan view showing an example of a pixel driven when reading out a short signal from the second window row in the first embodiment of this technology. This is a timing chart showing an example of the exposure and readout operation of the image sensor in the first embodiment of this technology. This is a timing chart showing the details of the readout operation of the long signal from the image sensor in the first embodiment of this technology. This is a diagram showing an example of the imaging environment in the first embodiment of this technology. This is a diagram showing an example of an edge image in the first embodiment of this technology. This is a diagram for comparing the edge determination methods of a comparative example and the first embodiment of this technology.This is a flowchart illustrating an example of the operation of an image sensor in the first embodiment of this technology. This is a diagram illustrating an example of the control of a vertical scanning circuit when detecting an edge in the first exposure mode in the second embodiment of this technology. This is a timing chart showing an example of the exposure and readout operation of an image sensor in the second embodiment of this technology. This is a timing chart showing an example of the exposure and readout operation of an image sensor in the third embodiment of this technology. This is a diagram for explaining the processing of an edge determination circuit in the fourth embodiment of this technology. This is a block diagram illustrating an example of an edge determination circuit with a reduced window edge synthesis section in the fourth embodiment of this technology. This is a plan view showing an example of a pixel driven when reading the first window row in the fifth embodiment of this technology. This is a plan view showing an example of a pixel driven when reading the second window row in the fifth embodiment of this technology. This is a plan view showing an example of a pixel driven when reading the first window row in the first modified example of the fifth embodiment of this technology. This is a plan view showing an example of a pixel driven when reading the second window row in the first modified example of the fifth embodiment of this technology. This is a plan view showing an example of a pixel driven when reading the third window row in the first modified example of the fifth embodiment of this technology. This figure shows an example of the layout when adding two pixels in the first modified example of the fifth embodiment of this technology. This figure shows an example of the operation of the edge detection mode of the signal processing circuit in the first modified example of the fifth embodiment of this technology. This is a plan view showing an example of pixels driven when reading the first window row in the second modified example of the fifth embodiment of this technology. This is a plan view showing an example of pixels driven when reading the second window row in the second modified example of the fifth embodiment of this technology. This is a block diagram showing an example of the configuration of the signal processing circuit in the second modified example of the fifth embodiment of this technology. This figure shows an example of pixels driven in the edge determination mode in the sixth embodiment of this technology. This figure shows an example of pixels in a unit block in the sixth embodiment of this technology. This figure shows an example of pixels in a unit block in a modified example of the sixth embodiment of this technology.This figure shows an example of the operation of the edge detection mode of the signal processing circuit in a modified version of the sixth embodiment of this technology. This is a circuit diagram showing an example configuration of the vertical scanning circuit in the seventh embodiment of this technology. This is a block diagram showing an example configuration of the column signal processing circuit in the eighth embodiment of this technology. This is a block diagram showing an example configuration of the column signal processing circuit in the ninth embodiment of this technology. This is a block diagram showing an example configuration of the counter in the ninth embodiment of this technology. This is a timing chart showing an example of the operation of the imaging mode of the image sensor in the ninth embodiment of this technology. This is a block diagram showing an example configuration of the signal processing circuit in the ninth embodiment of this technology. This is a block diagram showing an example configuration of the column signal processing circuit in a modified version of the ninth embodiment of this technology. This is a block diagram showing an example configuration of the signal processing circuit in the tenth embodiment of this technology. This is a figure showing an example of the stacked structure of the image sensor in the eleventh embodiment of this technology. This is a figure showing another example of the stacked structure of the image sensor in the eleventh embodiment of this technology. This is a timing chart showing an example of the readout operation of the image sensor in the twelfth embodiment of this technology. This is a circuit diagram showing an example configuration of a pixel in the thirteenth embodiment of this technology. This is a timing chart showing an example of the image sensor readout operation in the thirteenth embodiment of this technology. This is a circuit diagram showing an example configuration of the pixel array unit and column signal processing circuit in the fourteenth embodiment of this technology. This is a circuit diagram showing an example configuration of the adder circuit in the fourteenth embodiment of this technology. This is a timing chart showing an example of the image sensor readout operation in the fourteenth embodiment of this technology. This is a circuit diagram showing an example configuration of the pixel array unit and column signal processing circuit in the fifteenth embodiment of this technology. This is a circuit diagram showing an example configuration of the sample-and-hold circuit in the fifteenth embodiment of this technology. This is a timing chart showing an example of the image sensor readout operation in the fifteenth embodiment of this technology. This is a block diagram showing an example configuration of the column signal processing circuit in the sixteenth embodiment of this technology. This is an example of a circuit diagram of a logic gate in the sixteenth embodiment of this technology.This is a truth table showing an example of the operation of the synthesis unit in the 16th embodiment of this technology. This is a timing chart showing an example of the operation of the synthesis unit in the 16th embodiment of this technology. This is a circuit diagram showing an example of the configuration of the synthesis unit in the first modified example of the 16th embodiment of this technology. This is a truth table showing an example of the operation of the synthesis unit in the first modified example of the 16th embodiment of this technology. This is a timing chart showing an example of the operation of the synthesis unit in the first modified example of the 16th embodiment of this technology. This is a circuit diagram showing an example of the configuration of the synthesis unit in the second modified example of the 16th embodiment of this technology. This is a truth table showing an example of the operation of the synthesis unit in the second modified example of the 16th embodiment of this technology. This is a timing chart showing an example of the operation of the synthesis unit in the second modified example of the 16th embodiment of this technology. This is a circuit diagram showing an example of the configuration of the synthesis unit in the third modified example of the 16th embodiment of this technology. This is an example of a circuit diagram of a latch circuit in the third modified example of the 16th embodiment of this technology. This is a block diagram showing an example of the configuration of an edge determination circuit in the 17th embodiment of this technology. This is a block diagram showing an example of the configuration of a signal processing circuit in the 17th embodiment of this technology. This is a block diagram showing an example of a schematic configuration of a vehicle control system. This is an explanatory diagram showing an example of the installation location of the external information detection unit and the imaging unit.

[0034] The following describes embodiments for implementing this technology (hereinafter referred to as embodiments). The description will proceed in the following order: 1. First embodiment (an example of combining two judgment results with different exposure modes) 2. Second embodiment (an example of combining the judgment results of a first exposure mode and a second exposure mode that perform pixel addition) 3. Third embodiment (an example of combining the judgment results of two exposure modes whose exposure periods do not overlap) 4. Fourth embodiment (an example of combining two judgment results with different exposure modes so that information indicating direction is not lost) 5. Fifth embodiment (an example of combining two judgment results with different exposure modes without mixing pixels with different exposure modes in the same row) 6. Sixth embodiment (an example of combining the judgment results of a first and second exposure mode that perform pixel addition) 7. Seventh embodiment (an example of combining two judgment results with different exposure modes by changing the voltage of the driver's ground terminal) 8. Eighth embodiment (an example of combining two judgment results with different exposure modes by reducing the circuit size) 9. 10. Embodiment 9 (Example of providing a circuit for combining two judgment results with different exposure modes in each column) 11. Embodiment 11 (Example of combining two recognition results with different exposure modes in a stacked structure) 12. Embodiment 12 (Example of reading out the pixel signal corresponding to the second exposure mode by adding it to the pixel signal of the first exposure mode) 13. Embodiment 13 (Example of driving in the first and second exposure modes while connecting FDs) 14. Embodiment 14 (Example of calculating the average of the pixel signals output in two exposure modes) 15. Embodiment 15 (Example of sampling and adding the pixel signals output in two exposure modes) 16. Embodiment 16 (Example of combining two judgment results with different exposure modes) 17. Embodiment 17 (Example of combining using line memory) 18. Application examples to mobile devices

[0035] <1. First Embodiment> [Example of Imaging Device Configuration] Figure 1 is a block diagram showing an example configuration of an imaging device 100 in the first embodiment of this technology. This imaging device 100 is a device for capturing image data and comprises an optical unit 110, an image sensor 200, and a DSP (Digital Signal Processing) circuit 120. Furthermore, the imaging device 100 comprises a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. Examples of imaging devices 100 include digital cameras such as digital still cameras, as well as smartphones, personal computers, and in-vehicle cameras that have imaging capabilities.

[0036] The optical unit 110 collects light from the subject and guides it to the image sensor 200. The image sensor 200 generates image data by photoelectric conversion in synchronization with the vertical synchronization signal. Here, the vertical synchronization signal is a periodic signal of a predetermined frequency that indicates the timing of imaging. The image sensor 200 supplies the generated image data to the DSP circuit 120 via the signal line 209.

[0037] The DSP circuit 120 performs predetermined signal processing on the image data from the image sensor 200. The DSP circuit 120 outputs the processed image data to a frame memory 160 or the like via the bus 150.

[0038] The display unit 130 displays image data. The display unit 130 could be, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The operation unit 140 generates operation signals according to user input.

[0039] Bus 150 is a common path for the optical unit 110, image sensor 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, and power supply unit 180 to exchange data with each other.

[0040] The frame memory 160 stores image data. The storage unit 170 stores various data, including image data. The power supply unit 180 supplies power to the image sensor 200, the DSP circuit 120, the display unit 130, and other components.

[0041] [Example of Image Sensor Configuration] Figure 2 is a block diagram showing an example configuration of an image sensor 200 in the first embodiment of this technology. This image sensor 200 includes a vertical scanning circuit 210, a DAC (Digital to Analog Converter) 220, a timing control circuit 230, and a pixel array unit 240. Furthermore, the image sensor 200 includes a column signal processing circuit 300, a horizontal scanning circuit 270, and a signal processing circuit 280. These circuits are arranged, for example, on a single semiconductor chip.

[0042] The pixel array section 240 has multiple pixels arranged in a two-dimensional grid. These pixels include R (Red) pixels that receive red visible light through a color filter, G (Green) pixels that receive green visible light through a color filter, and B (Blue) pixels that receive blue visible light through a color filter. The G pixels include Gr pixels arranged in the row of R pixels and Gb pixels arranged in the row of B pixels. The color filters for each pixel are arranged, for example, in a Bayer array. Note that the arrangement of the color filters is not limited to a Bayer array, but may also be a quad-Bayer array or the like.

[0043] Furthermore, the pixel array section 240 is divided into multiple FD sharing blocks 250. Multiple pixels that share an FD are arranged in the FD sharing block 250. For example, four pixels are arranged in a 2x2 arrangement in the FD sharing block 250. Note that the number of pixels that share an FD is not limited to four pixels. Alternatively, pixels that do not share an FD, but each having its own FD, can be arranged in the pixel array section 240.

[0044] Hereafter, when we simply refer to them as "rows" and "columns," we mean the rows and columns of pixels. The rows and columns of the FD shared block 250 are referred to as "FD rows" and "FD columns."

[0045] The timing control circuit 230 controls the operating timing of the vertical scanning circuit 210, the column signal processing circuit 300, etc., in synchronization with the vertical synchronization signal.

[0046] Furthermore, the timing control circuit 230 receives a mode signal MODE that indicates the operating mode of the image sensor 200. The operating modes include an edge detection mode that detects the presence or absence of edges, and an imaging mode that captures image data without detecting the presence or absence of edges. The timing control circuit 230 controls each circuit based on these operating modes. The image sensor 200 can also capture image data while detecting the presence or absence of edges.

[0047] The vertical scanning circuit 210 sequentially selects and drives the FD rows and outputs analog pixel signals. The DAC 220 generates a reference signal by DA (Digital to Analog) conversion and supplies it to the column signal processing circuit 300. For example, a sawtooth-shaped ramp signal is used as the reference signal. Each pixel generates a pixel signal by photoelectric conversion according to the control of the vertical scanning circuit 210 and outputs that pixel signal to the column signal processing circuit 300.

[0048] The column signal processing circuit 300 performs signal processing on the pixel signal for each column, such as CDS (Correlated Double Sampling) processing and AD (Analog to Digital) conversion processing. In imaging mode, the column signal processing circuit 300 supplies the image data consisting of the digital signals after signal processing to the signal processing circuit 280, according to the control of the horizontal scanning circuit 270.

[0049] Furthermore, in edge detection mode, the column signal processing circuit 300 performs a process to determine the presence or absence of an edge and supplies the determination result to the signal processing circuit 280.

[0050] The horizontal scanning circuit 270 sequentially selects FD columns and outputs digital signals sequentially to the column signal processing circuit 300.

[0051] The signal processing circuit 280 performs various signal processing on the image data as needed. In imaging mode, the signal processing circuit 280 performs image processing such as defect correction and supplies the processed image data to the DSP circuit 120. In edge detection mode, the signal processing circuit 280 acquires the edge determination result and performs processing such as object recognition. The signal processing circuit 280 then supplies the processed data to the DSP circuit 120.

[0052] Figure 3 is a circuit diagram showing one example configuration of the vertical scanning circuit 210 and FD shared block 250 in the first embodiment of this technology.

[0053] The FD shared block 250 comprises transfer transistors 251, 252, 256, and 257, photoelectric conversion elements 253, 254, 255, and 258, a reset transistor 259, an amplification transistor 260, a selection transistor 261, and an FD 262.

[0054] Photoelectric conversion elements 253, 254, 255, and 258 convert received light into electric charge and store it. Photoelectric conversion element 253 is connected to transfer transistor 251, and photoelectric conversion element 254 is connected to transfer transistor 252. Photoelectric conversion element 255 is connected to transfer transistor 256, and photoelectric conversion element 258 is connected to transfer transistor 257.

[0055] Transfer transistors 251, 252, 256, and 257 transfer the accumulated charge to the FD 262. For example, n-type MOS (Metal-Oxide-Semiconductor) transistors are used as transfer transistors 251, 252, 256, and 257. The gates of these transistors are connected to the vertical scanning circuit 210 and the transfer signals TRG_Gb, TRG_B, TRG_R, and TRG_Gr are input. The drains of transfer transistors 251, 252, 256, and 257 are commonly connected to the FD 262, and their sources are connected to the photoelectric conversion elements 253, 254, 255, and 258, respectively.

[0056] FD262 stores the transferred charge and generates a voltage corresponding to the amount of charge. This FD262 is connected to the transfer transistors 251, 252, 256, and 257, the reset transistor 259, and the amplification transistor 260.

[0057] The reset transistor 259 resets the charge amounts stored in the FD 262 and the photoelectric conversion elements 253, 254, 255, and 258 to their initial values. For example, an n-type MOS transistor is used as the reset transistor 259. The gate of this reset transistor 259 is connected to the vertical scanning circuit 210 to which the reset signal RST is input, the source is connected to the power supply, and the drain is connected to the FD 262.

[0058] The amplification transistor 260 forms a source follower circuit and outputs a voltage corresponding to the voltage of FD 262. For example, an n-type MOS transistor is used as the amplification transistor 260. The gate of this amplification transistor 260 is connected to FD 262, the source is connected to the power supply, and the drain is connected to the selection transistor 261.

[0059] When an FD row is selected, the selection transistor 261 supplies the pixel signal SIG, which is the voltage output by the amplification transistor 260, to the column signal processing circuit 300 via the vertical signal line VSL. For example, an n-type MOS transistor is used as the selection transistor 261. The gate of this selection transistor 261 is connected to the vertical scanning circuit 210 and the selection signal SEL is input to it, the source is connected to the amplification transistor 260, and the drain is connected to the vertical signal line VSL.

[0060] The vertical scanning circuit 210 also includes drivers 211 to 216. Driver 211 supplies the reset signal RST. Drivers 212, 213, 214, and 215 supply the transfer signals TRG_Gb, TRG_B, TRG_R, and TRG_Gr. Driver 216 supplies the selection signal SEL.

[0061] For example, driver 211 supplies a signal of the power supply voltage VDD or the ground voltage GND as a reset signal RST. The remaining drivers 212 to 216 supply a signal of the power supply voltage VDD or the negative voltage VEE as a transfer signal or selection signal. The power supply voltage VDD is higher than the ground voltage GND, for example, 2.9 volts (V), and the ground voltage GND is, for example, 0 volts (V). The negative voltage VEE is lower than the ground voltage GND, for example, -1.2 volts (V).

[0062] Figure 4 is an example of a cross-sectional view of an image sensor 200 in the first embodiment of this technology. This image sensor 200 includes a substrate 530 and a wiring layer 520. An on-chip lens 511, a color filter 512, and a photoelectric conversion element 253 are arranged for each pixel. With the side into which light is incident facing upwards, the color filter 512 is positioned below the on-chip lens 511, and the photoelectric conversion element 253 is positioned below that.

[0063] Furthermore, a wiring layer 520 is formed on the upper surface of the substrate 530, and transistors such as the aforementioned transfer transistor 252 are formed within the wiring layer 520. Photoelectric conversion elements 253 for each pixel are arranged on top of the wiring layer 520. The structure illustrated in the figure is called a back-illuminated type.

[0064] As illustrated in Figure 5, a front-illuminated type can also be used instead of a back-illuminated type. In this front-illuminated type, a wiring layer 520 is formed below the color filter 512, and a photoelectric conversion element 253 is placed below that.

[0065] [Example of Column Signal Processing Circuit Configuration] Figure 6 is a block diagram showing an example configuration of a column signal processing circuit 300 in the first embodiment of this technology. This column signal processing circuit 300 includes a pixel signal selection unit 310, an edge determination circuit 320, and an AD conversion unit 330.

[0066] The pixel signal selection unit 310 is equipped with a load MOS current source and a selector for each vertical signal line VSL. For example, load MOS current sources 311 to 314 and selectors 315 to 318 are arranged for vertical signal lines VSL0 to VSL3.

[0067] The load MOS current sources 311 to 314 supply load MOS current to the corresponding vertical signal line VSL.

[0068] Selectors 315 to 318 select one of the vertical signal lines VSL0 to VSL3 according to the control of the timing control circuit 230 and connect to the edge detection circuit 320. For example, selector 315 selects vertical signal line VSL3 which transmits pixel signal SIG3, selector 316 selects vertical signal line VSL2 which transmits pixel signal SIG2, selector 317 selects vertical signal line VSL1 which transmits pixel signal SIG1, and selector 318 selects vertical signal line VSL0 which transmits pixel signal SIG0.

[0069] The edge detection circuit 320 determines the presence or absence of an edge based on at least one pair of pixel signals. The presence or absence of an edge is determined for each area called a "window" within the pixel array section 240. The edge detection circuit 320 includes a gain control circuit, an offset control circuit, and a comparison circuit for each vertical signal line VSL, and a window edge synthesis section 450 for each window. For example, for vertical signal lines VSL0 to VSL3, gain control circuits 410, 451, 452, and 453, offset control circuits 420, 454, 455, and 456, and comparison circuits 430, 457, 458, and 459 are arranged.

[0070] Gain control circuits 410, 451, 452, and 453 perform gain control by increasing or decreasing the pixel signal by a predetermined gain. Gain control circuit 410 increases or decreases the pixel signal SIG3 from selector 315 by a gain, and gain control circuit 451 increases or decreases the pixel signal SIG2 from selector 316 by a gain. Gain control circuit 452 increases or decreases the pixel signal SIG1 from selector 317 by a gain, and gain control circuit 453 increases or decreases the pixel signal SIG0 from selector 318 by a gain. Gain control circuits 410 and 453 are examples of the first and second gain control circuits described in the claims.

[0071] Offset control circuits 420, 454, 455, and 456 perform offset control by superimposing a predetermined offset (such as an offset voltage) onto the pixel signal. Offset control circuit 420 superimposes the offset onto pixel signal SIG3, offset control circuit 454 superimposes the offset onto pixel signal SIG2, offset control circuit 455 superimposes the offset onto pixel signal SIG1, and offset control circuit 456 superimposes the offset onto pixel signal SIG0.

[0072] Gain control circuits 410 and 453 increase or decrease one of the pixel signals SIG3 and SIG0 by gain, but it is also possible to increase or decrease both of them. The same applies to gain control circuits 451 and 452. Offset control circuits 420 and 456 superimpose an offset on one of the pixel signals SIG3 and SIG0, but it is also possible to superimpose an offset on both of them. The same applies to offset control circuits 454 and 455.

[0073] Comparison circuits 430, 457, 458, and 459 compare a pair of pixel signals that have undergone gain control and offset control in edge detection mode. Comparison circuit 430 compares pixel signal SIG3 and pixel signal SIG0, which have undergone gain control and offset control, and supplies the comparison result VCO0 to the window edge merging unit 450. Comparison circuit 457 compares pixel signal SIG2 and pixel signal SIG1, which have undergone gain control and offset control, and supplies the comparison result VCO1 to the window edge merging unit 450. Comparison circuit 458 compares pixel signal SIG1 and pixel signal SIG2, which have undergone gain control and offset control, and supplies the comparison result VCO2 to the window edge merging unit 450. Comparison circuit 459 compares pixel signal SIG0 and pixel signal SIG3, which have undergone gain control and offset control, and supplies the comparison result VCO3 to the window edge merging unit 450.

[0074] The comparison circuits 430 and 459 are examples of the first and second comparison circuits described in the claims.

[0075] Furthermore, in imaging mode, the timing control circuit 230 stops the comparison circuits 430, 457, 458, and 459.

[0076] The window edge synthesis unit 450 synthesizes VCO3 from the comparison result VCO0 and outputs the determination result Ew, which indicates the presence or absence of an edge, to the signal processing circuit 280.

[0077] For example, when the following equation holds, the low-level comparison result VCO0 is output: SIG0 < SIG3 * g - c ... Equation 1 In the above equation, "*" indicates multiplication. "g" indicates the gain value, and "-c" indicates the offset value. The same applies to subsequent equations.

[0078] On the other hand, when the following equation holds, a high-level comparison result VCO0 is output: SIG0 > SIG3 * g - c ... Equation 2

[0079] Furthermore, it is assumed that a high-level comparison result VCO3 is output when the following equation is true: SIG3 > SIG0 * g - c ... Equation 3

[0080] On the other hand, the low-level comparison result VCO3 is output when the following equation holds: SIG3 ≤ SIG0 * g - c ... Equation 4

[0081] If both equation 1 and equation 3 are true, then the following equation holds: SIG0 << SIG3 ... Equation 5 In the above equation, "<<" indicates the degree to which an edge is judged to exist.

[0082] On the other hand, if both equations 2 and 4 are true, the following equation holds: SIG0 >> SIG3 ... Equation 6 In the above equation, ">>" indicates the degree of inequality to which an edge is judged to exist.

[0083] If the comparison results show that VCO0 and VCO3 have different values, then either equation 5 or equation 6 is true. Equations 5 and 6 indicate that the ratio of one pixel signal SIG0 and SIG3 to the other is greater than or equal to a certain value. In this case, an edge is determined to exist.

[0084] On the other hand, if the comparison results VCO0 and VCO3 are the same value, neither equation 5 nor equation 6 holds true. In this case, it is determined that there is no edge. Note that if g is less than "1", it is impossible for both comparison results VCO0 and VCO3 to be "0" in the calculation. Also, if g is "1" or greater, it is impossible for both comparison results VCO0 and VCO3 to be "1" in the calculation.

[0085] Next, we will explain the advantages of a control method that compares one of a pair of pixel signals, SIG0 and SIG3, by increasing or decreasing it using a gain g.

[0086] The pixel that generates the pixel signal SIG0 receives light reflected from the surface of object A with reflectivity Ra, and the pixel that generates the pixel signal SIG3 receives light reflected from the surface of object B with reflectivity Rb. The amount of ambient light is AL, and the gain g is less than 1.

[0087] Here, we consider a first comparative example in which the presence or absence of an edge is detected by comparing the difference between pixel signals SIG0 and SIG3 with a threshold Th. In this first comparative example, an edge is determined to exist if the following equation holds: SIG0 - SIG3 = Ra*AL - Rb*AL > Th ... Equation 7

[0088] Equation 7 can be transformed into the following equation: (Ra - Rb) > Th / AL ... Equation 8

[0089] As illustrated in Equations 7 and 8, in the first comparison example, the edge detection result may change depending on the ambient light intensity AL. This reduces the accuracy of edge detection. For example, the smaller the light intensity AL, the smaller the difference, making edge detection difficult. While changing the threshold according to the ambient light intensity can suppress the decrease in detection accuracy, accurately measuring the light intensity and fine-tuning the threshold accordingly is difficult.

[0090] Furthermore, we consider a second comparative example in which the presence or absence of an edge is detected by comparing the difference between the logarithms of the respective pixel signals SIG0 and SIG3 with a threshold Th. In this second comparative example, an edge is determined to exist if the following equation holds: log(SIG0) - log(SIG3) = log(Ra*AL / Rb*AL) = log(Ra / Rb) > Th ...Equation 9

[0091] As illustrated in Equation 9, the second comparative example can suppress the effects of ambient light, but it requires logarithmic calculations and is difficult to implement.

[0092] In contrast to these comparative examples, the influence of ambient light can be easily suppressed by introducing a method that determines the threshold based on the signal rather than on ambient light.

[0093] Then, the same edge determination is performed on the comparison results VCO1 and VCO2. The edge determination results for comparison results VCO0 and VCO3 are set to Ex, and the determination results for comparison results VCO1 and VCO2 are set to Ey. The window edge blending unit 450 calculates the logical OR of these Ex and Ey, blends them into a 1-bit determination result Ew, and outputs it to the signal processing circuit 280.

[0094] For example, a four-input NAND gate that outputs the negative logical AND of comparison results VCO0, VCO1, VCO2, and VCO3 as the determination result Ew is used as the window edge synthesis unit 450. Note that the window edge synthesis unit 450 is an example of the logic circuit described in the claims.

[0095] The AD conversion unit 330 converts analog pixel signals into digital signals for each FD row in the imaging mode. This AD conversion unit 330 is equipped with an ADC for each vertical signal line VSL. For example, ADCs 331, 332, 333, and 334 are arranged for vertical signal lines VSL0 to VSL3.

[0096] ADCs 331, 332, 333, and 334 convert the pixel signals of the corresponding FD rows into digital signals CNT and output them to the signal processing circuit 280. For example, single-slope type ADCs consisting of a comparator circuit and a counter are used as these ADCs. The comparator circuit within the ADC compares the pixel signal with the reference signal REF from the DAC 220, and the counter counts for the period until the comparison result is reversed, outputting a digital signal CNT indicating the count value.

[0097] In edge detection mode, the timing control circuit 230 stops ADCs 331, 332, 333, and 334.

[0098] [Example Configuration of Offset Control Circuit and Gain Control Circuit] Figure 7 is a circuit diagram showing an example configuration of an offset control circuit 420 and a gain control circuit 410 in the first embodiment of this technology. The offset control circuit 420 includes, for example, switches 421, 422 and 423 and a capacitive element 425. The gain control circuit 410 includes, for example, switches 411, 412, 413 and 414 and capacitive elements 417 and 418. The circuit configurations of the offset control circuits 454, 455 and 456 are the same as those of the offset control circuit 420, and the circuit configurations of the gain control circuits 451, 452 and 43 are the same as those of the gain control circuit 410.

[0099] In the offset control circuit 420, the switch 421 opens and closes the path between the selector 315 and one end of the capacitive element 425 according to the control signal SW3a from the timing control circuit 230. The other end of the capacitive element 425 is connected to the non-inverting input terminal (+) of the comparator circuit 430.

[0100] Switch 422 opens and closes the path between the ground node of the ground voltage GND and one end of the capacitive element 425 according to the control signal SW3b from the timing control circuit 230. Switch 423 opens and closes the path between the input node to which the offset voltage Vofs is input as an offset and one end of the capacitive element 425 according to the control signal SW3c from the timing control circuit 230.

[0101] In edge detection mode, the timing control circuit 230 controls, for example, the DAC 220 to generate a constant offset voltage Vofs. Alternatively, a separate circuit for generating the offset voltage Vofs can be provided instead of the DAC 220.

[0102] Furthermore, the switch 422 can be eliminated in the offset control circuit 420. In this case, the DAC 220 only needs to generate the ground voltage GND when one end of the capacitive element 425 is grounded.

[0103] In the gain control circuit 410, switch 411 opens and closes the path between selector 315 and one end of capacitive element 417 according to control signal SW0a from timing control circuit 230. Switch 412 opens and closes the path between the ground node and one end of capacitive element 417 according to control signal SW0b from timing control circuit 230. The other end of this capacitive element 417 is connected to the non-inverting input terminal (+) of comparator circuit 430.

[0104] Switch 413 opens and closes the path between the selector 315 and one end of the capacitive element 418 according to the control signal SW1a from the timing control circuit 230. Switch 414 opens and closes the path between the ground node and one end of the capacitive element 418 according to the control signal SW1b from the timing control circuit 230. The other end of this capacitive element 418 is connected to the non-inverting input terminal (+) of the comparator circuit 430.

[0105] Furthermore, the non-inverting input terminal (+) of the comparison circuit 430 is connected to the pixel signal Diff after offset and gain control. ｐоThe following is input. The pixel signal SIG0 from the vertical signal line VSL0 is input to the inverting input terminal (-) of the comparison circuit 430 via the capacitive element 405, Diff ｎо It will be entered as follows.

[0106] [Example of Comparator Circuit Configuration] Figure 8 is a circuit diagram showing an example configuration of a comparator circuit 430 in the first embodiment of this technology. This comparator circuit 430 comprises pMOS transistors 431 to 435, nMOS transistors 436 to 440, and capacitive elements 441 to 443.

[0107] The pMOS transistors 431, 432, and 435 are connected in parallel to the power supply. The gate of pMOS transistor 431 is connected to its drain and to the gate of pMOS transistor 432.

[0108] The drain of nMOS transistor 436 is connected to pMOS transistor 431, and its source is connected to the common node. The gate of nMOS transistor 436 is connected to the offset control circuit 420 via capacitive element 441. The drain of nMOS transistor 437 is connected to pMOS transistor 432, and its source is connected to the common node. The gate of nMOS transistor 437 is connected to capacitive element 405 via capacitive element 442.

[0109] The nMOS transistor 438 is inserted between the common node and the ground terminal, and a predetermined bias voltage Vbias3 is input to its gate.

[0110] The pMOS transistor 433 shorts the drain and source of the nMOS transistor 436 when the inverted signal XAZPL is high level. The pMOS transistor 434 shorts the drain and source of the nMOS transistor 437 when the inverted signal XAZPR is high level. The inverted signal XAZPL is the inverted auto-zero signal AZPL from the timing control circuit 230, and the inverted signal XAZPR is the inverted auto-zero signal AZPR from the timing control circuit 230.

[0111] The connection point between pMOS transistor 432 and nMOS transistor 437 is connected to the gate of pMOS transistor 435.

[0112] The nMOS transistor 440 is inserted between the pMOS transistor 435 and the ground terminal. The comparison result VCO0 is output from the connection point of the pMOS transistor 435 and the nMOS transistor 440. The nMOS transistor 439 and the capacitive element 443 are connected in series between the connection point of the pMOS transistor 435 and the nMOS transistor 440 and the ground terminal. In addition, the auto-zero signal AZ is input to the gate of the nMOS transistor 439, and the connection point of the nMOS transistor 439 and the capacitive element 443 is connected to the gate of the nMOS transistor 440.

[0113] In the circuit illustrated in the figure, the comparison circuit 430 compares the pixel signal SIG3*g-c and the pixel signal SIG0. The timing control circuit 230 also shifts the input timing of the auto-zero signal AZPL, which initializes one of the positive and negative signals, and the auto-zero signal AZPR, which initializes the other. By setting the input timing of the auto-zero signal AZPL to be different from that of the auto-zero signal AZPR, the offset of the comparison circuit 430 can be adjusted.

[0114] The configurations of the comparison circuits 457, 458, and 459 are the same as those of the comparison circuit 430.

[0115] With the configurations illustrated in Figures 6, 7, and 8, the image sensor 200 can detect edges for each window. However, when detecting edges in imaging environments with large differences in brightness, the exposure time may be too short, resulting in insufficient signal, or the exposure time may be too long, leading to signal saturation.

[0116] To solve this problem, the vertical scanning circuit 210 in the first embodiment drives multiple pixels in the first and second exposure modes in edge detection mode. For example, the exposure time of the first exposure mode is longer than the exposure time of the second exposure mode. The first and second exposure modes will be described with reference to Figures 9 and 10.

[0117] FIG. 9 is a diagram showing an example of control of a vertical scanning circuit 210 and a column signal processing circuit 300 when detecting an edge in a first exposure mode of the present technology. An area of 8 rows × 8 columns in the pixel array unit 240 is taken as one window 241. The vertical scanning circuit 210 selects two pairs of Gr pixels for each window 241. One pair of the two pairs of Gr pixels does not correspond to either a row or a column and is arranged at both ends of a line segment whose angle with the row is different from 45 degrees. Similarly, the remaining pair of Gr pixels also does not correspond to either a row or a column and is arranged at both ends of a line segment whose angle with the row is different from 45 degrees. Also, each of these two pairs of Gr pixels has different rows from each other and different columns from each other. The line segment with arrows at both ends in the figure is an example of a line segment whose angle with the row is different from 45 degrees.

[0118] According to the pattern illustrated in the figure, a pair of pixel signals are read from different vertical signal lines. Therefore, a sample-and-hold circuit is not required in the edge determination circuit 320.

[0119] Pixel signals generated in the first exposure mode with a long exposure time are defined as "long storage signals" SIG0 Ｌ , SIG1 Ｌ , SIG2 Ｌ and SIG3 Ｌ . As gains and offsets for these long storage signals, g Ｌ and -c Ｌ are set.

[0120] Comparison circuits 430, 457, 458, and 459 compare the long storage signals subjected to offset control and gain control, and output the comparison results as VCO0 Ｌ , VCO1 Ｌ , VCO2 Ｌ and VCO3 Ｌ . The window edge synthesis unit 450 synthesizes those comparison results and outputs the obtained determination result as Ew_L.

[0121] Figure 10 shows an example of the control of the vertical scanning circuit 210 and the column signal processing circuit 300 when detecting edges in the second exposure mode in the first embodiment of the present technology. In the second exposure mode, the vertical scanning circuit 210 selects two pairs of Gb pixels for each window. These pixels do not belong to either a row or a column and are placed at both ends of a line segment that has an angle of less than 45 degrees with respect to a row.

[0122] The pixel signal generated in the second exposure mode with a shorter exposure time is called a "short-acting signal" (SIG0). Ｓ SIG1 Ｓ SIG2 Ｓ and SIG3 Ｓ Let g be the gain and offset for these short signal values. Ｓ and -c Ｓ This will be set.

[0123] Comparison circuits 430, 457, 458, and 459 compare long-storage signals that have undergone offset control and gain control, and the comparison result is VCO0 Ｓ , VCO1 Ｓ VCO2 Ｓ and VCO3 Ｓ The window edge blending unit 450 blends these comparison results and outputs the resulting judgment result as Ew_S.

[0124] As illustrated in Figures 9 and 10, Gr pixels driven in the first exposure mode and Gb pixels driven in the second exposure mode do not coexist within each row. Therefore, within a row, it is sufficient to drive them in either the first or second exposure mode, and there is no need to separate the transfer control.

[0125] The vertical scanning circuit 210 changes the exposure time in the first and second exposure modes, but is not limited to this control. The vertical scanning circuit 210 can also drive pixels with different sensitivities in the first and second exposure modes. When driving pixels with different sensitivities, the color of the color filter of the pixels driven in the first exposure mode may be different from the color of the color filter of the pixels driven in the second exposure mode.

[0126] [Example of Signal Processing Circuit Configuration] Figure 11 is a block diagram showing an example configuration of a signal processing circuit 280 in the first embodiment of this technology. This signal processing circuit 280 includes an edge determination and holding unit 281, an HDR edge synthesis unit 282, a recognition unit 283, and an image processing unit 284.

[0127] As described above, in edge detection mode, the vertical scanning circuit 210 drives the Gr pixels in the pixel array 240 in the first exposure mode to output two pairs of pixel signals as two pairs of long-storage signals for each window. In addition, the vertical scanning circuit 210 drives the Gb pixels in the pixel array 240 in the second exposure mode to output two pairs of pixel signals as two pairs of short-storage signals for each window. From the viewpoint of preventing charge overflow, it is preferable to read out the long-storage signals before the short-storage signals.

[0128] The long-sequenced signal is an example of the first pixel signal described in the claims, and the short-sequenced signal is an example of the second pixel signal described in the claims. Furthermore, the Gr pixel is an example of the first pixel described in the claims, and the Gb pixel is an example of the second pixel described in the claims.

[0129] The edge detection circuit 320 performs, for each window, the following processes: determining the presence or absence of an edge based on a long signal and outputting a determination result Ew_L; and determining the presence or absence of an edge based on a short signal and outputting a determination result Ew_S. The determination result Ew_L is an example of the first determination result described in the claims, and the determination result Ew_S is an example of the second determination result described in the claims.

[0130] The edge determination and retention unit 281 retains either the determination result Ew_L or Ew_S for each window. For example, if the long storage signal is read first, the determination result Ew_L is retained.

[0131] The HDR edge synthesis unit 282 synthesizes the judgment results Ew_L and Ew_S for each window. For example, when the judgment result Ew_S is output, the HDR edge synthesis unit 282 reads the judgment result Ew_L from the edge judgment holding unit 281. Then, the HDR edge synthesis unit 282 synthesizes these judgment results into a 1-bit judgment result E_HDR by, for example, calculating the logical OR of the judgment results Ew_L and Ew_S for each window, and outputs it to the recognition unit 283. Note that the HDR edge synthesis unit 282 is an example of a synthesis unit described in the claims.

[0132] The recognition unit 283 performs a process to recognize a predetermined object in the edge image obtained by arranging the judgment results E_HDR of each window. The recognition unit 283 outputs the recognition result to the DSP circuit 120.

[0133] Furthermore, the image processing unit 284 performs various image processing operations on the image data, which consists of an array of digital signal CNTs, in the imaging mode, such as defect correction, demosaicing, and white balance correction. The image processing unit 284 outputs the processed image data to the DSP circuit 120.

[0134] Furthermore, at least a portion of the processing within the signal processing circuit 280 can be performed by the subsequent DSP circuit 120.

[0135] [Example of Image Sensor Operation] Next, the scanning method of the vertical scanning circuit 210 will be explained with reference to Figures 12 to 15. Hereinafter, a collection of multiple windows arranged in the row direction will be referred to as a "window row".

[0136] Figure 12 is a plan view showing an example of a Gr pixel driven when reading the long-storage signal of the first window row in the first embodiment of this technology. The area enclosed by the dashed line indicates the leftmost window within the window row to be driven. The area enclosed by the dotted line indicates the window to the right of the window enclosed by the dashed line. As illustrated in the figure, the leftmost window does not have an area that overlaps with the window to its right. In this way, multiple windows are arranged in the row direction such that adjacent windows do not overlap with each other.

[0137] The vertical scanning circuit 210 drives the first window row in a first exposure mode with a long exposure time and outputs two pairs of long exposure signals for each window. The edge detection circuit 320 generates and outputs a 1-bit detection result Ew_L for each window.

[0138] Figure 13 is a plan view showing an example of a Gb pixel driven when reading out a short signal from the first window row in the first embodiment of this technology.

[0139] The vertical scanning circuit 210 drives the first window row in a second exposure mode with a shorter exposure time, and outputs two pairs of short-pass signals for each window. The edge detection circuit 320 generates and outputs a 1-bit detection result Ew_S for each window.

[0140] Figure 14 is a plan view showing an example of a Gr pixel driven when reading the long stored signal of the second window row in the first embodiment of this technology. Here, multiple windows are arranged so that adjacent windows do not overlap in the column direction. Therefore, the vertical scanning circuit 210 only needs to slide the window to be driven in the column direction by the window size. If the window is 8 rows x 8 columns, it will be slid by 8 rows.

[0141] The vertical scanning circuit 210 drives the second window row in the first exposure mode and outputs two pairs of long-pass signals for each window. The edge detection circuit 320 generates and outputs a 1-bit detection result Ew_SL for each window.

[0142] Figure 15 is a plan view showing an example of a Gb pixel driven when reading out a short signal from the second window row in the first embodiment of this technology.

[0143] The vertical scanning circuit 210 drives the second window row in the second exposure mode and outputs two pairs of short signaling signals for each window. The edge detection circuit 320 generates and outputs a 1-bit detection result Ew_S for each window.

[0144] The third and subsequent window rows are driven in the same manner. As illustrated in Figures 12 to 15, the vertical scanning circuit 210 sequentially selects window rows and then sequentially controls the Gr pixels within that window row to be driven in a first exposure mode and the Gb pixels to be driven in a second exposure mode.

[0145] Figure 16 is a timing chart showing an example of the exposure and readout operation of the image sensor 200 in the first embodiment of this technology. In the figure, 'a' indicates the exposure and readout timing for each window row. In the figure, the vertical axis of 'a' indicates the row address of the window row, and the horizontal axis indicates time. Also, in the figure, 'b' indicates the control of exposure and readout for the first window row.

[0146] As illustrated in figure a, the vertical motion signal VSYNC falls at timing T0 and falls at timing T5. Within this period from timing T0 to T5, one frame of edge image is read out.

[0147] Immediately following timing T0, at timing T1 and beyond, the vertical scanning circuit 210 sequentially selects window rows, drives the Gr pixels, and starts exposure. In the figure, the solid diagonal lines in a indicate the timing of exposure start for each Gr pixel in each window row.

[0148] Then, after timing T2, the vertical scanning circuit 210 sequentially selects window rows, drives the Gb pixels, and starts exposure. The dashed line a in the figure indicates the timing of exposure start for each Gb pixel in each window row.

[0149] After timing T3, the vertical scanning circuit 210 sequentially selects window rows, terminates the exposure of Gr pixels and Gb pixels, and outputs their pixel signals sequentially. The thick line a in the figure indicates the timing of exposure termination and readout for each window row. The period from timing T1 to T3 represents the exposure period EX_Gr of the first exposure mode of the first window row. The period from timing T2 to T3 represents the exposure period EX_Gb of the second exposure mode of the first window row. Since the pixels driven in the first and second exposure modes are different, a portion of the exposure periods of each mode can be overlapped.

[0150] As illustrated in figure b, the vertical scanning circuit 210 sets the reset signal RST to a high level and supplies a high-level transfer signal TRG_Gr from timing T1 through the pulse period. More precisely, exposure in the first exposure mode begins at the end of this transfer.

[0151] Then, the vertical scanning circuit 210 keeps the reset signal RST at a high level and supplies a high-level transfer signal TRG_Gb over the pulse period from timing T2. Strictly speaking, exposure in the second exposure mode begins at the end of this transfer.

[0152] Then, the vertical scanning circuit 210 sets the selection signal SEL to a high level at timing T3, and immediately afterwards sets the reset signal RST to a low level. Then, from timing T21, the vertical scanning circuit 210 supplies a high-level transfer signal TRG_Gr for the duration of the pulse. Strictly speaking, exposure in the first exposure mode ends at the end of this transfer.

[0153] The vertical scanning circuit 210 then supplies a high-level reset signal RST for a pulse period starting at timing T22, and immediately after that, supplies a high-level transfer signal TRG_Gb for a pulse period starting at timing T23. Strictly speaking, exposure in the second exposure mode ends at the end of this transfer. Then, the vertical scanning circuit 210 lowers the selection signal SEL to a low level at timing T4.

[0154] Figure 17 is a timing chart showing the details of the readout operation of the long stored signal of the image sensor 200 in the first embodiment of this technology.

[0155] The vertical scanning circuit 210 transmits a high-level selection signal SEL to the selected window row during the period from timing T3 to T4. Immediately after timing T3, the vertical scanning circuit 210 also lowers the reset signal RST to a low level.

[0156] Furthermore, the timing control circuit 230 transmits a high-level auto-zero signal AZ from timing T3 over the pulse period to perform auto-zero on the comparator circuit 430 and other components. The timing control circuit 230 also sets the control signals SW0a and SW1b to a high level and the control signals SW0b and SW1a to a low level. This reset level is also called the P phase.

[0157] The vertical scanning circuit 210 then transmits a high-level transfer signal TRG_Gr to the selected row from timing T21 over the pulse period. As a result, signal charge is transferred within the pixel, and the level of the vertical signal line becomes a signal level corresponding to the exposure amount. This signal level is also called the D phase. The difference between the reset level and the signal level corresponds to the net pixel signal (SIG0 or SIG3).

[0158] Furthermore, the timing control circuit 230 sets the control signal SW3a to a low level. Also, at timing T22 immediately after the transfer, the timing control circuit 230 sets the control signal SW3b from a high level to a low level, while simultaneously setting the control signal SW3c from a low level to a high level.

[0159] As a result, among the input signals to the comparison circuit 430, the pixel signal Diff on the vertical signal line VSL3 side ｐо An offset is superimposed on it. Also, among the input signals to the comparison circuit 459, the pixel signal Diff on the vertical signal line VSL0 side ｐｅ An offset is superimposed. In the figure, the dashed line represents the pixel signal Diff on the vertical signal line VSL3 side. ｐо and Diff ｎｅ The waveform is shown. The superposition of these offsets can suppress false detection of edges in low light conditions.

[0160] Under high illumination conditions, the difference between the pair of pixel signals being compared becomes large. Therefore, the comparison results VCO0 and VCO3, respectively, will have values ​​corresponding to the relative magnitudes of the pixel signals SIG0 and SIG3. For example, if the relative magnitudes given in Equation 5 are true, both Equations 1 and 2 are true, and a low-level comparison result VCO0 and a high-level comparison result VCO3 are output. In this case, an edge is determined to exist. In this way, the image sensor 200 can detect edges with high accuracy even under high illumination conditions.

[0161] Figure 18 shows an example of the imaging environment in the first embodiment of this technology. The area enclosed by the thick line in the figure indicates the imaging range. Subjects 711 and 712 are assumed to be located within this imaging range. Furthermore, the area around subject 711 is assumed to be bright, and the area around subject 712 is assumed to be dark.

[0162] Figure 19 shows an example of an edge image in the first embodiment of this technology. Figure a shows an example of edge image 701 in which the judgment result Ew_S of the shorter exposure time is arranged. Figure b shows an example of edge image 702 in which the judgment result Ew_L of the longer exposure time is arranged. Figure c shows an example of edge image 703 in which the combined judgment result E_HDR is arranged.

[0163] In edge detection mode, only the combined edge image 703 is output, and the pre-combination edge images 702 and 703 are not output. However, to illustrate the effect of the combination, the pre-combination edge images are also shown as examples.

[0164] If image compositing is not performed, an insufficient exposure time will result in insufficient signal, causing the edges of the subject 712 in dark areas to disappear, as exemplified in Figure a. Conversely, if image compositing is not performed, an insufficient exposure time will result in signal saturation, causing the edges of the subject 711 in bright areas to disappear, as exemplified in Figure b. Such edge loss may reduce the accuracy of object recognition.

[0165] In contrast, when the judgment result Ew_S (for the shorter exposure time) and the judgment result Ew_L (for the longer exposure time) are combined, both edges of subjects 711 and 712 are obtained, as illustrated in figure c. As a result, edges do not disappear even if there is a difference in brightness in the imaging environment, improving robustness to the imaging environment.

[0166] Figure 20 is a diagram for comparing the edge determination methods of a comparative example and the first embodiment of the present technology. In the figure, the vertical axes of a and b represent the level of the pixel signal SIG3, and the horizontal axis represents the level of the pixel signal SIG0.

[0167] In the figure, the gray area labeled 'a' indicates the region where an edge is determined to exist in the comparative example without superimposed offset, while the white area labeled 'a' indicates the region where an edge is determined to be absent in the comparative example. The slope of the boundary of each region corresponds to a value corresponding to the gain.

[0168] In the figure, the gray area b indicates the region where an edge is determined to exist in the first embodiment where an offset is superimposed, and the white area b indicates the region where an edge is determined to be absent in the first embodiment.

[0169] As illustrated in Figure a, in the comparative example, when the illumination is low and the difference between pixel signals SIG0 and SIG3 is small, the white area becomes very narrow. Therefore, even if a signal is actually within the white area and has no edges, noise may cause it to be mistakenly detected as being in the gray area (i.e., having edges) beyond the boundary.

[0170] In contrast, as illustrated in figure b, the first embodiment superimposes an offset, resulting in a wider white area even at low illumination levels. This suppresses false detections caused by noise.

[0171] Figure 21 is a flowchart showing an example of the operation of the image sensor 200 in the first embodiment of this technology.

[0172] The image sensor 200 determines whether or not the edge detection mode is set (step S901). If the edge detection mode is set (step S901: Yes), the image sensor 200 selects the window row to be read (step S902) and reads the long-storage signal in the first exposure mode (step S903). Then, the image sensor 200 determines whether or not there is an edge for each window based on the long-storage signal (step S904) and stores the determination result Ew_L for one row (step S905).

[0173] Then, the image sensor 200 reads out the short-pass signal in the second exposure mode (step S906), and based on the short-pass signal, determines whether there is an edge for each window and generates a determination result Ew_S (step S907). The image sensor 200 then combines the determination results Ew_L and Ew_S for each window to generate a determination result E_HDR (step S908).

[0174] The image sensor 200 then determines whether all window rows have been selected (step S909). If there are any unselected window rows (step S909: No), the image sensor 200 repeatedly executes steps S902 onward.

[0175] On the other hand, if all rows are selected (step S909: Yes), the image sensor 200 performs object recognition on the edge image containing the judgment result E_HDR (step S910).

[0176] Furthermore, if an imaging mode is set (step S901: No), the image sensor 200 captures image data and performs various image processing (step S911). After step S910 or S911, the image sensor 200 returns to step S901.

[0177] Thus, according to the first embodiment of this technology, since the HDR edge synthesis unit 282 synthesizes the determination result of the first exposure mode and the determination result of the second exposure mode, robustness to the imaging environment can be improved.

[0178] <2. Second Embodiment> In the first embodiment described above, the vertical scanning circuit 210 drove the Gr pixels in the first exposure mode to output a long-storage signal, but the signal amount may be insufficient with only Gr pixels. The image sensor 200 in this second embodiment differs from the first embodiment in that it drives multiple pixels in the first exposure mode to output an added signal obtained by adding the pixels together.

[0179] Figure 22 shows an example of the control of the vertical scanning circuit when detecting an edge in the first exposure mode in a second embodiment of this technology. In this second embodiment, the vertical scanning circuit 210 drives multiple pixels in the first exposure mode with a long exposure time and outputs a signal obtained by adding the pixel signals of those pixels. For example, R pixels, Gr pixels, and B pixels that share an FD are driven in the first exposure mode. Each of the two pairs of long-accumulation signals is a signal obtained by adding the respective pixel signals of the three pixels. In the second exposure mode with a short exposure time, only the Gb pixels are driven, as in the first embodiment.

[0180] Figure 23 is a timing chart showing an example of the exposure and readout operation of the image sensor 200 in a second embodiment of this technology. The control of the image sensor 200 in this second embodiment differs from the first embodiment in that the transfer signals TRG_R, TRG_Gr, and TRG_B are supplied simultaneously at timings T1 and T21. As a result, three pixels are added together, and the signal amount of the long-storage signal can be increased compared to when pixel addition is not performed.

[0181] Thus, according to the second embodiment of this technology, the vertical scanning circuit 210 drives multiple pixels in the first exposure mode and outputs an added signal obtained by pixel summing, thereby increasing the signal amount of the long-storage signal.

[0182] <3. Third Embodiment> In the second embodiment described above, the vertical scanning circuit 210 drove different pixels in the first and second exposure modes, but the control is not limited to this. The image sensor 200 in this third embodiment differs from the second embodiment in that it drives the same pixel in each of the first and second exposure modes.

[0183] Figure 24 is a timing chart showing an example of the exposure and readout operation of the image sensor 200 in a third embodiment of this technology. In the figure, 'a' indicates the exposure and readout timing for each window row. In the figure, the vertical axis of 'a' indicates the row address of the window row, and the horizontal axis indicates time. In the figure, 'b' indicates the control of exposure and readout for the first window row.

[0184] As illustrated in Figure a, after timing T1, the vertical scanning circuit 210 sequentially selects window rows and drives the four pixels sharing the FD in the first exposure mode, initiating exposure. The solid diagonal lines in Figure a indicate the timing of the start of exposure in the first exposure mode for each window row. For each window, FD sharing blocks 250 at both ends of a predetermined line segment that does not correspond to either a row or a column and whose angle with the row is different from 45 degrees are selected. Then, an added signal obtained by adding the four pixel signals is read out for each FD sharing block 250.

[0185] Then, after timing T3, the vertical scanning circuit 210 sequentially selects window rows to end exposure in the first exposure mode and outputs summation signals sequentially. Furthermore, for each window row, the vertical scanning circuit 210 drives the same four pixels as in the first exposure mode in the second exposure mode and outputs summation signals. In the figure, the dashed line a indicates the timing of the end of exposure in the first exposure mode and the start of exposure in the second exposure mode for each window row. The thick line indicates the timing of the end of exposure in the second exposure mode for each window row.

[0186] As illustrated in figure b, the vertical scanning circuit 210 supplies high-level transfer signals TRG_R, TRG_Gr, TRG_Gb, and TRG_B from timing T1 through the pulse period. At the end of this transfer, exposure in the first exposure mode begins.

[0187] Then, the vertical scanning circuit 210 sets the selection signal SEL to a high level at timing T3, and immediately afterwards sets the reset signal RST to a low level. Then, from timing T21, the vertical scanning circuit 210 supplies high-level transfer signals TRG_R, TRG_Gr, TRG_Gb, and TRG_B over the pulse period. At the end of this transfer, the vertical scanning circuit 210 concludes that the exposure of the first exposure mode has ended and the exposure of the second exposure mode has begun.

[0188] The vertical scanning circuit 210 then supplies a high-level reset signal RST from timing T22 for the duration of the pulse. The vertical scanning circuit 210 then supplies high-level transfer signals TRG_R, TRG_Gr, TRG_Gb, and TRG_B from timing T23 for the duration of the pulse. At the end of this transfer, exposure in the second exposure mode is completed.

[0189] As illustrated in figure b, the control for transferring signal charge in the first exposure mode can also be used to initiate exposure in the second exposure mode. This allows the image sensor 200 to read out both long-storage and short-storage signals from the same pixel.

[0190] In the diagram, the image sensor 200 performs pixel binning, but it is also possible to read out the pixel signals without performing pixel binning.

[0191] Thus, according to the third embodiment of this technology, the vertical scanning circuit 210 starts exposure in the second exposure mode when exposure in the first exposure mode ends, so that long-storage signals and short-storage signals can be read from the same pixel.

[0192] <4. Fourth Embodiment> In the first embodiment described above, the edge determination circuit 320 combined the comparison results of 4 bits for each window into a 1-bit determination result, but this control loses information indicating the direction of the edge. The image sensor 200 in this fourth embodiment differs from the first embodiment in that it outputs a 2-bit or 4-bit determination result for each window.

[0193] Figure 25 is a diagram illustrating the processing of the edge determination circuit 320 in the fourth embodiment of this technology.

[0194] As illustrated in figure a, when the long-cycle signal is read out, the comparison circuits 430, 457, 458 and 459 read the comparison result into VCO0 Ｌ , VCO1 Ｌ VCO2 Ｌ and VCO3 Ｌ Output as follows. The window edge blending unit 450 compares the results VCO0 Ｌ and VCO3 Ｌ By calculating the logical OR of these, they are combined into a 1-bit judgment result Ew_Lx. Furthermore, the window edge blending unit 450 calculates the comparison result VCO1 Ｌ and VCO2 Ｌ By performing a logical OR operation, these are combined into a 1-bit judgment result Ew_Ly. These 2-bit judgment results are output for each window.

[0195] Furthermore, as illustrated in figure b, when a short signal is read out, the comparison circuits 430, 457, 458, and 459 read the comparison result into VCO0 Ｓ , VCO1 Ｓ VCO2 Ｓ and VCO3 Ｓ Output as follows. The window edge blending unit 450 compares the results VCO0 Ｓ and VCO3 Ｓ By calculating the logical OR of these, they are combined into a 1-bit judgment result Ew_Sx. The window edge blending unit 450 also combines the comparison result VCO1 Ｓ and VCO2 Ｓ By performing a logical OR operation, these are combined into a 1-bit judgment result Ew_Sy. These 2-bit judgment results are output for each window.

[0196] The subsequent HDR edge blending unit 282 blends the judgment results Ew_Lx and Ew_Lx and outputs them as E_HDRx. The HDR edge blending unit 282 also blends the judgment results Ew_Ly and Ew_Ly and outputs them as E_HDRy.

[0197] As illustrated in figures a and b, the edge determination circuit 320 in the fourth embodiment outputs a 2-bit determination result for each window. This prevents the loss of information indicating the direction of the edge.

[0198] Furthermore, as illustrated in Figures 26 a and b, the window edge merging section 450 can also be reduced.

[0199] In this case, as illustrated in figure a, when the long-storage signal is read out, the comparison circuits 430, 457, 458 and 459 read out the comparison result VCO0 Ｌ , VCO1 Ｌ VCO2 Ｌ and VCO3 Ｌ Each of these will be output as the result of the judgment.

[0200] Furthermore, as illustrated in figure b, when a short signal is read out, the comparison circuits 430, 457, 458, and 459 perform the comparison result VCO0 Ｓ , VCO1 Ｓ VCO2 Ｓ and VCO3 Ｓ Each of these will be output as the result of the judgment.

[0201] As illustrated in figures a and b, if the comparison results are not combined for each window, a 4-bit judgment result (in other words, comparison result) is output for each window in both the first and second modes. The subsequent HDR edge synthesis unit 282 is VCO0 Ｌ and VCO0 Ｓ The results are combined and output as the judgment result E_HDR0, and VCO1 Ｌ and VCO1 Ｓ The HDR edge synthesis unit 282 combines the VCO2 and outputs the result as E_HDR1. Ｌ and VCO2 Ｓ The results are combined and output as the judgment result E_HDR2, and VCO3 Ｌ and VCO3 Ｓ The results are combined and output as the judgment result E_HDR3.

[0202] Furthermore, the first, second, and third embodiments can be applied to the fourth embodiment.

[0203] Thus, according to the fourth embodiment of this technology, the HDR edge synthesis unit 282 outputs a 2-bit or 4-bit determination result for each window, thereby preventing the loss of information indicating the direction of the edge.

[0204] <5. Fifth Embodiment> In the first embodiment described above, the vertical scanning circuit 210 selected each window so that two adjacent windows did not overlap with each other. However, in this configuration, the resolution of the edge image may be insufficient. The image sensor 200 in this fifth embodiment differs from the first embodiment in that it selects each window so that a portion of two adjacent windows overlap.

[0205] Figure 27 is a plan view showing an example of pixels driven when reading the first window row in the fifth embodiment of this technology. In the figure, the area enclosed by the dashed line represents the leftmost window of the first window row, and the area enclosed by the dotted line represents the window to the right of the leftmost window. As illustrated in the figure, parts of two adjacent windows in the row direction overlap.

[0206] Figure 28 is a plan view showing an example of pixels driven when reading the second window row in the fifth embodiment of this technology. Here, it is assumed that multiple windows are arranged such that parts of two adjacent windows overlap in the column direction. The vertical scanning circuit 210 slides the window to be driven in the column direction with a slide width smaller than the window size. If the window is 8 rows x 8 columns, for example, it is slid by 4 rows.

[0207] As illustrated in Figures 27 and 28, the vertical scanning circuit 210 selects each window such that a portion of two adjacent windows overlaps in the row and column directions. This improves the resolution of the edge image compared to the first embodiment.

[0208] Furthermore, the first, second, third, and fourth embodiments can be applied to the fifth embodiment.

[0209] Thus, according to the fifth embodiment of this technology, the vertical scanning circuit 210 selects each window such that a portion of two adjacent windows overlap, thereby improving the resolution of the edge image.

[0210] [First Modification] In the fifth embodiment described above, the vertical scanning circuit 210 outputs a pixel signal without pixel summing in each exposure mode, but the control is not limited to this. The image sensor 200 in this first modification of the fifth embodiment differs from the fifth embodiment in that it sums pixels in each exposure mode.

[0211] Figure 29 is a plan view showing an example of pixels driven when reading the first window row in the first modification of the fifth embodiment of the present technology. In the first modification of the fifth embodiment, as in the fifth embodiment, parts of two adjacent windows overlap.

[0212] However, in the first and second exposure modes, four pixels that share the FD are driven, and an added signal obtained by adding these pixels together is read out. Also, four pixels driven in the first exposure mode are arranged in the 2n (where n is an integer) + 1th FD row, and four pixels driven in the second exposure mode are arranged in the 2nth FD row. In the figure, light gray pixels indicate pixels driven in the first exposure mode with a longer exposure time, and dark gray pixels indicate pixels driven in the second exposure mode with a shorter exposure time. In this layout as well, since pixels driven in the first exposure mode and pixels driven in the second exposure mode are not mixed within a row, there is no need to separate the transfer control for each exposure mode.

[0213] As illustrated in the figure, pixels are driven in the first exposure mode in the first window row, and two pairs of summing signals are read out. Pixel summing is performed, for example, by FD summing similar to the control illustrated in Figure 23. However, it is not limited to FD summing; source follower summing or summing using a sample-and-hold circuit can also be performed.

[0214] Figure 30 is a plan view showing an example of pixels driven when reading the second window row in the first modification of the fifth embodiment of this technology. The vertical scanning circuit 210 slides the window to be driven in the column direction with a slide width smaller than the window size. If the window is 16 rows x 16 columns, for example, it is slid by 8 rows.

[0215] As illustrated in the figure, in the second window row, the pixels are driven in a second exposure mode with a shorter exposure time, and two pairs of summation signals are read out.

[0216] Figure 31 is a plan view showing an example of pixels driven when reading the third window row in the first modification of the fifth embodiment of the present technology. The vertical scanning circuit 210 slides the window to be driven in the column direction with a slide width smaller than the window size.

[0217] As illustrated in the figure, in the third window row, the pixels are driven in the first exposure mode with a longer exposure time, and two pairs of summing signals are read out.

[0218] Similarly, the vertical scanning circuit 210 switches the exposure mode each time the window is slid in the column direction. This eliminates the need to drive both the first and second exposure modes for each window row.

[0219] As illustrated in Figure 32, the image sensor 200 can also add two pixels. The number of pixels to be added is not limited to two or four pixels.

[0220] Figure 33 shows an example of the operation of the edge detection mode of the signal processing circuit 280 in the first modified example of the fifth embodiment of the present technology.

[0221] As illustrated in figure a, when the first window row is driven in a first exposure mode with a long exposure time, a determination result Ew_L1 is generated for each window and held in the edge determination holding unit 281.

[0222] Then, as illustrated in figure b, when the second window row is driven in a second exposure mode with a shorter exposure time, a determination result Ew_S2 is generated for each window. The HDR edge synthesis unit 282 reads the determination result Ew_L1 from the edge determination holding unit 281, synthesizes it with the determination result Ew_S2, and outputs it as the determination result E_HDR1.

[0223] Then, as illustrated in figure c, the HDR edge synthesis unit 282 updates the determination result Ew_L1 of the edge determination and holding unit 281 with the determination result Ew_S2.

[0224] Then, as illustrated in figure d, when the third window row is driven in the first exposure mode with a longer exposure time, a determination result Ew_L3 is generated for each window. The HDR edge synthesis unit 282 reads the determination result Ew_S2 from the edge determination holding unit 281, synthesizes it with the determination result Ew_L3, and outputs it as the determination result E_HDR2.

[0225] Similarly, the edge determination and holding unit 281 alternately holds the determination result of the first exposure mode and the determination result of the second exposure mode.

[0226] Thus, according to the first modification of the fifth embodiment of this technology, the vertical scanning circuit 210 performs pixel addition in each exposure mode in a layout where parts of two adjacent windows overlap. This eliminates the need to drive each window row in both the first and second exposure modes.

[0227] [Second Modification] In the fifth embodiment described above, pixels driven in the first exposure mode and pixels driven in the second exposure mode were mixed and arranged in each row, but the layout is not limited to this. The image sensor 200 in this second modification of the fifth embodiment differs from the fifth embodiment in that pixels driven in the first exposure mode and pixels driven in the second exposure mode are not mixed and arranged in each row.

[0228] Figure 34 is a plan view showing an example of pixels driven when reading the first window row in a second modification of the fifth embodiment of the present technology. In the second modification of the fifth embodiment, as in the fifth embodiment, parts of two adjacent windows overlap.

[0229] However, in the first and second exposure modes, two pixels sharing the FD are driven, and an added signal obtained by adding these pixels together is read out. Furthermore, pixels driven in each exposure mode are selected so that pixels driven in the first exposure mode and pixels driven in the second exposure mode do not mix in each column. In the figure, light gray pixels indicate pixels driven in the first exposure mode with a longer exposure time, and dark gray pixels indicate pixels driven in the second exposure mode with a shorter exposure time. Note that the number of pixels to be added is not limited to two.

[0230] The vertical scanning circuit 210 drives the pixels in the 2n+1th window in the first window row in the first exposure mode, and simultaneously drives the pixels in the 2nth window in the second exposure mode.

[0231] Note that in the layout shown in the figure, pixels driven by the first exposure mode and pixels driven by the second exposure mode are mixed in the same row. Therefore, it is necessary to separate the transfer control for each exposure mode, and it should be noted that this increases the number of signal lines that transmit the transfer signals.

[0232] Figure 35 is a plan view showing an example of pixels driven when reading the second window row in a second modification of the fifth embodiment of the present technology. The vertical scanning circuit 210 slides the window to be driven in the column direction with a slide width smaller than the window size.

[0233] The vertical scanning circuit 210 drives the pixels in the 2n+1th window in the second window row in the first exposure mode, and simultaneously drives the pixels in the 2nth window in the second exposure mode.

[0234] Figure 36 is a block diagram showing an example configuration of a signal processing circuit 280 in a second modification of the fifth embodiment of the present technology. This signal processing circuit 280 of the second modification of the fifth embodiment differs from the fifth embodiment in that an edge determination and holding unit 281 is not provided.

[0235] As described above, the vertical scanning circuit 210 drives the pixels in the 2n+1th window in the first exposure mode and simultaneously drives the pixels in the 2nth window in the second exposure mode for each window row. Therefore, the determination result Ew_L for the first exposure mode and the determination result Ew_S for the second exposure mode are output simultaneously. Consequently, as illustrated in the figure, the edge determination and holding unit 281 can be reduced in the signal processing circuit 280.

[0236] Thus, according to the first modification of the fifth embodiment of this technology, since pixels driven in the first exposure mode and pixels driven in the second exposure mode are not mixed and arranged in the row, the edge determination and holding unit 281 can be reduced.

[0237] <6. Sixth Embodiment> In the first embodiment described above, the vertical scanning circuit 210 output a pixel signal without pixel summing in each exposure mode, but with only one pixel, the signal amount may be insufficient. The image sensor 200 in this sixth embodiment differs from the first embodiment in that it drives multiple pixels in each of the first and second exposure modes and outputs an added signal with pixel summing.

[0238] Figure 37 shows an example of a pixel driven in edge detection mode in the sixth embodiment of this technology. In the sixth embodiment, the color filters are arranged in a quad Bayer array. In addition, unit blocks 242 of a predetermined size are placed at both ends of each of the two line segments within the window.

[0239] Figure 38 shows an example of pixels within a unit block 242 in a sixth embodiment of the present technology. Multiple pixels are arranged in this unit block 242. For example, 16 pixels are arranged in a 4x4 grid. Half of these pixels, 8 pixels, are driven in a first exposure mode, and the remaining 8 pixels are driven in a second exposure mode.

[0240] For example, in a quad Bayer array, two of four adjacent R pixels are driven in a first exposure mode, and the remaining two pixels are driven in a second exposure mode. The same applies to Gr, Gb, and B pixels. In the figure, light gray pixels indicate pixels driven in a first exposure mode with a longer exposure time, and dark gray pixels indicate pixels driven in a second exposure mode with a shorter exposure time.

[0241] Eight pixels driven in the first exposure mode are added together, and two pairs of added signals (in other words, long-accumulation signals) are read out for each window. Similarly, eight pixels driven in the second exposure mode are added together, and two pairs of added signals (in other words, short-accumulation signals) are read out for each window. This allows for an increase in the signal strength of both the long-accumulation and short-accumulation signals compared to when pixel addition is not performed.

[0242] Furthermore, the fourth embodiment, which outputs in 2 bits or 4 bits, and the fifth embodiment, in which parts of two adjacent windows overlap, can be applied to the sixth embodiment.

[0243] Thus, according to the sixth embodiment of this technology, the vertical scanning circuit 210 drives multiple pixels in each of the first and second exposure modes and outputs an added signal obtained by pixel summing, thereby increasing the signal amounts of both the long-accumulation signal and the short-accumulation signal.

[0244] [Modified Version] In the sixth embodiment described above, the image sensor 200 switched the exposure time in two stages, but the configuration is not limited to this. The modified version of the sixth embodiment differs from the sixth embodiment in that the image sensor 200 switches the exposure time in three stages.

[0245] Figure 39 shows an example of a pixel within a unit block 242 in a modified example of the sixth embodiment of the present technology. In the first modified example of this sixth embodiment, the vertical scanning circuit 210 uses three exposure modes with different exposure times.

[0246] For example, in a quad Bayer array, one of the four R pixels is driven in the first exposure mode, two pixels are driven in the second exposure mode, and the remaining pixel is driven in the third exposure mode. The same applies to Gr, Gb, and B pixels. In the figure, pixels labeled L are those driven in the first exposure mode, which has the longest exposure time, pixels labeled M are those driven in the second exposure mode, which has the second longest exposure time, and pixels labeled S are those driven in the third exposure mode, which has the shortest exposure time.

[0247] Figure 40 shows an example of the operation of the edge detection mode of the signal processing circuit 280 in a modified example of the sixth embodiment of the present technology.

[0248] As illustrated in figure a, when the determination result Ew_L of the first exposure mode is output for each window, the edge determination and holding unit 281 holds those determination results Ew_L.

[0249] Then, as illustrated in figure b, when the determination result Ew_M of the second exposure mode is output for each window, the HDR edge compositing unit 282 reads the determination result Ew_L from the edge determination holding unit 281. The HDR edge compositing unit 282 then combines the determination result Ew_L with the determination result Ew_M to generate the determination result E_LM. The HDR edge compositing unit 282 then updates the determination result Ew_L in the edge determination holding unit 281 with the determination result E_LM.

[0250] Then, as illustrated in figure c, when the determination result Ew_S for the third exposure mode is output for each window, the HDR edge synthesis unit 282 reads the determination result E_LM from the edge determination holding unit 281. The HDR edge synthesis unit 282 then synthesizes the determination result E_LM with the determination result Ew_S and outputs it as the determination result E_HDR. In this way, by switching the exposure time in three stages and synthesizing the respective determination results, the edge detection accuracy can be improved compared to the case where the exposure time is switched in two stages.

[0251] The image sensor 200 switches the exposure time in three stages, but it is also possible to have four or more exposure modes and switch the exposure time in four or more stages. In addition, the vertical scanning circuit 210 can drive pixels with different sensitivities in each of the three or more exposure modes.

[0252] Thus, according to the modified sixth embodiment of this technology, the vertical scanning circuit 210 is driven in three exposure modes, and the HDR edge synthesis unit 282 synthesizes the respective determination results, thereby improving the edge detection accuracy.

[0253] <7. Seventh Embodiment> In the first embodiment described above, the driver in the vertical scanning circuit 210 supplied a transfer signal of the power supply voltage VDD or negative voltage VEE, but it is preferable to further reduce power consumption. The image sensor 200 in this seventh embodiment differs from the first embodiment in that it supplies a transfer signal of the power supply voltage VDD or ground voltage GND to pixels driven in a second exposure mode with a short exposure time.

[0254] Figure 41 is a circuit diagram showing one example configuration of the vertical scanning circuit 210 in the seventh embodiment of this technology. In this seventh embodiment, the driver 212 in the vertical scanning circuit 210 supplies a transfer signal TRG_Gb of the power supply voltage VDD or the ground voltage GND to the Gb pixels driven in the second exposure mode with a short exposure time. On the other hand, the driver 215 supplies a transfer signal TRG_Gr of the power supply voltage VDD or the negative voltage VEE to the Gr pixels, similar to the first embodiment.

[0255] Drivers 215 and 212 are examples of the first and second drivers described in the claims.

[0256] Generally, dark current increases with longer exposure times. To suppress this dark current, for Gr pixels with long exposure times, the low level of their transfer signal TRG_Gr must be set to a negative voltage VEE. On the other hand, for Gb pixels with short exposure times, the dark current is small, so setting the low level of their transfer signal TRG_Gb to the ground voltage GND does not have any adverse effects. This reduces the load on the charge pump circuit supplying the negative voltage VEE, thereby reducing power consumption.

[0257] Furthermore, each of the first to sixth embodiments can be applied to the seventh embodiment.

[0258] Thus, according to the seventh embodiment of this technology, power consumption can be reduced because the driver 212 supplies the transfer signal TRG_Gb of the power supply voltage VDD or the ground voltage GND.

[0259] <8. Eighth Embodiment> In the first embodiment described above, the comparison circuit, such as the comparison circuit 430, was provided outside the ADC, but it is preferable to further reduce the circuit size. The image sensor 200 in this eighth embodiment differs from the first embodiment in that the edge determination circuit 320 and the ADC share a comparison circuit.

[0260] Figure 42 is a block diagram showing an example configuration of a column signal processing circuit 300 in the eighth embodiment of this technology. In this eighth embodiment, the edge determination circuit 320 is further equipped with two switches and one selector for each vertical signal line VSL. For example, switches 461 to 468 and selectors 471 to 474 are provided for vertical signal lines VSL0 to VSL3. In addition, the AD conversion unit 330 is equipped with a counter for each vertical signal line VSL. For example, counters 335 to 338 are provided for vertical signal lines VSL0 to VSL3. The comparison circuit 430 and counter 335 function as a single-slope type ADC. The same applies to the comparison circuits 457, 458, 459 and counters 336, 337, 338.

[0261] Switches 461, 462, 463, and 464 open and close the path between the DAC 220, which supplies the reference signal REF, and the comparison circuits 430, 457, 458, and 459, according to the control of the timing control circuit 230. Switches 465, 466, 467, and 468 open and close the path between the pixel signal selection unit 310 and the gain control circuits 410, 451, 452, and 453, according to the control of the timing control circuit 230.

[0262] The timing control circuit 230 controls switches 461 to 464 to the open state and switches 465 to 468 to the closed state in edge detection mode. In imaging mode, the timing control circuit 230 controls switches 461 to 464 to the closed state and switches 465 to 468 to the open state.

[0263] Selectors 471, 472, 473, and 474 switch the output destination of the comparison results of the comparison circuits 430, 457, 458, and 459 according to the control of the timing control circuit 230. Selectors 471 to 474 output the comparison result VCO to the window edge synthesis unit 450 in edge detection mode and to the corresponding counter in imaging mode.

[0264] As illustrated in the figure, the comparison circuit can be shared between the ADC and the edge detection circuit 320. This reduces the circuit size compared to when the circuits are not shared.

[0265] Furthermore, each of the first to seventh embodiments can be applied to the eighth embodiment.

[0266] As described above, according to the eighth embodiment of this technology, since the ADC and the edge detection circuit 320 share a comparison circuit, the circuit size can be reduced compared to the case where they are not shared.

[0267] <9. Ninth Embodiment> In the eighth embodiment described above, a window edge synthesis unit 450 was arranged for each window, and an HDR edge synthesis unit 282 was arranged in the subsequent signal processing circuit 280, but the configuration is not limited to this. The image sensor 200 in this ninth embodiment differs from the eighth embodiment in that a window edge synthesis unit and an HDR edge synthesis unit are arranged for each FD row.

[0268] Figure 43 is a block diagram showing an example configuration of a column signal processing circuit 300 in the ninth embodiment of this technology. In this ninth embodiment, selectors 471 to 474 are not arranged. Also, window edge merging units are arranged for each FD column. For example, window edge merging units 450-1 to 450-4 are arranged for vertical signal lines VSL0 to VSL3. For example, NAND gates are used as these window edge merging units.

[0269] Furthermore, the AD conversion unit 330 has counters such as counters 600, 601, 602, and 603 arranged for each FD column. Each of the window edge blending units 450-1 to 450-4 outputs a judgment result Ew to the corresponding counter.

[0270] Figure 44 is a block diagram showing one example configuration of a counter 600 in the ninth embodiment of the present technology. This counter 600 comprises an HDR edge synthesis unit 610 and a counter circuit 620. The HDR edge synthesis unit 610 comprises a selector 611, a latch circuit 612, an OR (logical disjunction) gate 613, and a switch 614. The circuit configurations of counters 601, 602, and 603 are the same as those of counter 600.

[0271] The selector 611 selects either the comparison result VCO or the determination result Ew according to the selection signal SELe from the timing control circuit 230, and outputs it to the latch circuit 612 and the OR gate 613.

[0272] The timing control circuit 230 controls the selector 611 to select the judgment result Ew in edge detection mode. In addition, the timing control circuit 230 controls the selector 611 to select the comparison result VCO0 in imaging mode.

[0273] The latch circuit 612 holds the signal from the selector 611 according to the enable signal EN from the timing control circuit 230. When the enable signal EN is high, the latch circuit 612 updates the held value based on the signal from the selector 611. On the other hand, when the enable signal EN is low, the latch circuit 612 latches the held value. The latch circuit 612 also outputs the signal of the held value to the OR gate 613. This output signal from the latch circuit 612 is also output to the counter circuit 620 as a control signal Lout.

[0274] In imaging mode, the timing control circuit 230 supplies an enable signal EN pulse when the AD conversion to the reset level is completed, and updates the held value based on the comparison result VCO0.

[0275] Furthermore, in edge detection mode, when the determination result Ew_L for the first exposure mode with a longer exposure time is output, the timing control circuit 230 supplies a pulse of the enable signal EN and updates the held value based on the determination result Ew_L. On the other hand, when the determination result Ew_S for the second exposure mode with a shorter exposure time is output, the timing control circuit 230 lowers the enable signal EN to a low level and latches the held value.

[0276] The OR gate 613 outputs the logical OR of the signal from the latch circuit 612 and the signal from the selector 611 as the determination result E_HDR to the signal processing circuit 280 via the switch 614.

[0277] Switch 614 opens and closes the path between the output terminal of OR gate 613 and the signal processing circuit 280 according to the control signal STB from the timing control circuit 230. The timing control circuit 230 controls all counter switches 614 to the open state in imaging mode. In edge detection mode, the timing control circuit 230 closes only the switch 614 of one of the counters 600, 601, 602, and 603 for each window, and opens the rest. This control allows the unnecessary HDR edge synthesis unit 610 to be put into standby mode, thereby reducing power consumption.

[0278] The counter circuit 620 counts the count value for a period of time until the comparison result VCO0 reverses, provided that the control signal Lout from the latch circuit 612 is at its initial value (e.g., low level) in imaging mode. However, if the control signal Lout from the latch circuit 612 is updated to a value different from the initial value (e.g., high level), the counter circuit 620 continues counting until the count value reaches its maximum value. The counter circuit 620 then outputs a digital signal CNT indicating the count value to the signal processing circuit 280.

[0279] Figure 45 is a timing chart showing an example of the operation of the imaging mode of the image sensor in the ninth embodiment of this technology. Here, it is known that when very high-intensity light is incident on a pixel, the brightness decreases and the pixel becomes black due to charge leakage from the photoelectric conversion element within the pixel during the reset operation. This phenomenon is called sunspot activity. Figure a shows an example of operation at an illuminance level that does not cause sunspots, and figure b shows an example of operation at an illuminance level that causes sunspots. The dashed lines in figures a and b indicate the level of the pixel signal SIG. This level includes the reset level when the pixel is initialized (i.e., P phase) and the signal level when the signal charge is transferred (i.e., D phase).

[0280] As illustrated in figure a, the reference signal REF gradually decreases during the P-phase conversion period from timing T11 to T13. Assume that at timing T12 within this period, the reset level becomes lower than the reference signal REF. At this time, the comparison result VCO0 inverts from a high level to a low level. The counter circuit 620 counts down the count value (CNT) over the period from timing T11 to timing T12.

[0281] Furthermore, during the pulse period from just before timing T13 to timing T13, the timing control circuit 230 supplies a high-level enable signal EN to the latch circuit 612, and updates the held value based on the comparison result VCO0. In the case of illuminance that does not cause sunspots, the reset level is higher than a predetermined threshold at this point, so a low-level Lout is held in the latch circuit 612.

[0282] Then, the reference signal REF and the comparison result VCO0 are initialized, and during the D-phase conversion period from timing T14 to T16, the reference signal REF gradually decreases. Assume that at timing T15 within this period, the reset level becomes lower than the reference signal REF. At this time, the comparison result VCO0 inverts from a high level to a low level. The counter circuit 620 increments the count value over the period from timing T14 to timing T15 and outputs the digital signal CNT of that value.

[0283] As illustrated in figure a, at illuminance levels that do not produce sunspots, the reset level is higher than a predetermined threshold, so the latch circuit 612 holds a low level, and the counter circuit 620 counts until the comparison result VCO0 inverts during D-phase conversion.

[0284] On the other hand, as illustrated in figure b, in the case of high illuminance that causes sunspots, the reset level becomes lower than the threshold at timing T13. Therefore, the latch circuit 612 holds a high level of Lout. When Lout is at a high level, the counter circuit 620 counts until the count value reaches its maximum, regardless of the comparison result VCO0. In the figure, the comparison result VCO0 is reversed at timing T15, but the counter circuit 620 continues counting until timing T16, when the D phase period has elapsed. This control prevents the occurrence of sunspots.

[0285] As described above, a latch circuit 612 in the counter 600 is used to prevent sunspots. In the ninth embodiment, the HDR edge synthesis unit 610 can be realized simply by adding a selector 611, an OR gate 613, and a switch 614 to the latch circuit 612.

[0286] Figure 46 is a block diagram showing an example configuration of the signal processing circuit 280 in the ninth embodiment of this technology. The signal processing circuit 280 of this ninth embodiment differs from the eighth embodiment in that the edge determination and holding unit 281 and the HDR edge synthesis unit 282 have been reduced.

[0287] By arranging a window edge blending unit 450-1 and an HDR edge blending unit 610 for each FD row, the edge determination and holding unit 281 and the HDR edge blending unit 282 can be reduced from the subsequent circuitry, as illustrated in the figure.

[0288] Furthermore, each of the first to seventh embodiments can be applied to the ninth embodiment.

[0289] Thus, according to the ninth embodiment of this technology, since a window edge synthesis unit 450-1 and an HDR edge synthesis unit 610 are arranged for each FD row, the edge determination and holding unit 281 and the HDR edge synthesis unit 282 can be reduced from the subsequent circuitry.

[0290] [Modification] In the ninth embodiment described above, in edge detection mode, a comparison circuit 430, etc., compared a pair of signals, but it is preferable to further reduce power consumption. The image sensor 200 in this modification of the ninth embodiment differs from the first embodiment in that it stops the comparison circuit 430, etc., based on the held determination result Ew_L.

[0291] Figure 47 is a block diagram showing an example configuration of a column signal processing circuit 300 in a modified version of the ninth embodiment of this technology. This modified version of the ninth embodiment differs from the ninth embodiment in that it further includes an inverter 340.

[0292] For example, the gray counters 601, 602, and 603 in the figure are assumed to be stopped in accordance with the control signal STB. The operating counter 600 outputs the judgment result Ew_L, which was held in the first exposure mode with a long exposure time, to the inverter 340. The inverter 340 inverts the judgment result Ew_L and supplies it as a bias voltage Vbias3 to the comparator circuits 430, 457, 458, and 459. Note that the inverter 340 is an example of a logic gate as described in the claims.

[0293] For example, if there is an edge, the judgment results Ew_L and Ew_S will be high levels, and they will be combined by the logical OR of the judgment results Ew_L and Ew_S. In this case, if the judgment result Ew_L of the first exposure mode is high level, the combined judgment result E_HDR will be high level regardless of the value of the judgment result Ew_S of the second exposure mode. Therefore, if the judgment result Ew_L is high level, there is no need to perform a comparison in the second exposure mode. Accordingly, when the judgment result Ew_L is high level, the inverter 340 outputs a low-level bias voltage Vbias3 and stops the comparison circuits 430, 457, 458 and 459. This makes it possible to reduce power consumption compared to the ninth embodiment.

[0294] Thus, according to a modified version of the ninth embodiment of this technology, the inverter 340 stops the comparison circuits 430, 457, 458, and 459 based on the determination result Ew_L, thereby reducing power consumption.

[0295] <10. Tenth Embodiment> In the first embodiment described above, the signal processing circuit 280 combined the determination results of the first and second exposure modes and performed object recognition on the edge image obtained by arranging the combined determination results E_HDR. However, the invention is not limited to this process. The image sensor 200 in this tenth embodiment differs from the first embodiment in that it performs moving average calculations and threshold processing on the recognition results of the first exposure mode and the recognition results of the second exposure mode, respectively, and then combines the results of these processes.

[0296] Figure 48 is a block diagram showing one example configuration of a signal processing circuit 280 in the tenth embodiment of this technology. In this tenth embodiment, the signal processing circuit 280 does not include an edge determination and holding unit 281 and an HDR edge synthesis unit 282. The signal processing circuit 280 further includes confidence value moving average calculation units 285 and 286, threshold processing units 287 and 288, and a processing result synthesis unit 289.

[0297] The recognition unit 283 performs object recognition on the edge image in which the determination results Ew_L of the first exposure mode are arranged, and generates a confidence value Ct_L as a result of the recognition. This confidence value Ct_L is output to the confidence value moving average calculation unit 285.

[0298] Furthermore, the recognition unit 283 performs object recognition on the edge image in which the determination results Ew_L of the second exposure mode are arranged, and generates a confidence value Ct_S as a result of the recognition. This confidence value Ct_S is output to the confidence value moving average calculation unit 286.

[0299] Here, the edge image of the first exposure mode and the edge image of the second exposure mode are assumed to be generated periodically in synchronization with the vertical synchronization signal.

[0300] The confidence value moving average calculation unit 285 stores the confidence values ​​Ct_L obtained within a certain period and calculates their moving average Ave_L. This moving average Ave_L is output to the threshold processing unit 287.

[0301] The confidence value moving average calculation unit 286 stores the confidence values ​​Ct_S obtained within a certain period and calculates their moving average Ave_S. This moving average Ave_S is output to the threshold processing unit 288.

[0302] The thresholding unit 287 performs thresholding by comparing the moving average Ave_L with a predetermined threshold. The result of this thresholding is output to the processing result synthesis unit 289.

[0303] The thresholding unit 288 performs thresholding by comparing the moving average Ave_S with a predetermined threshold. The result of this thresholding is output to the processing result synthesis unit 289.

[0304] The processing result synthesis unit 289 synthesizes the processing results of the threshold processing units 287 and 288 by calculating their respective logical OR values. The processing result synthesis unit 289 outputs the synthesized value as the final recognition result to the DSP circuit 120.

[0305] As illustrated in the figure, the recognition accuracy can be improved by the processing result synthesis unit 289 synthesizing the result of processing the determination result of the first exposure mode and the result of processing the determination result of the second exposure mode.

[0306] Figure 49 is a diagram illustrating the signal processing circuit in the tenth embodiment of this technology. The confidence value C1_L for the first exposure mode and the confidence value C1_S for the second exposure mode are output in order. Then, the confidence value C2_L for the first exposure mode and the confidence value C2_S for the second exposure mode are output in order. Then, the confidence value C3_L for the first exposure mode and the confidence value C3_S for the second exposure mode are output in order.

[0307] The average of the confidence values ​​C1_L, C2_L, and C3_L is calculated as the moving average Ave_L, and the average of the confidence values ​​C1_S, C2_S, and C3_S is calculated as the moving average Ave_S.

[0308] The threshold processing units 287 and 288 perform threshold processing on their moving averages, and the processing result synthesis unit 289 synthesizes these processing results.

[0309] Furthermore, each of the second to ninth embodiments can be applied to the tenth embodiment.

[0310] Thus, according to the tenth embodiment of this technology, the processing result synthesis unit 289 synthesizes the result of processing the determination result of the first exposure mode and the result of processing the determination result of the second exposure mode, thereby improving recognition accuracy.

[0311] <11. Eleventh Embodiment> In the first embodiment described above, the circuit was arranged on a single semiconductor chip, but this configuration may make it difficult to miniaturize the pixels. The image sensor 200 in this eleventh embodiment differs from the first embodiment in that the circuit is distributed and arranged on multiple stacked semiconductor chips.

[0312] Figure 50 shows an example of a stacked structure of an image sensor 200 in the eleventh embodiment of this technology. The image sensor 200 in this eleventh embodiment comprises a circuit chip 202 and a pixel chip 201 stacked on the circuit chip 202. These chips are electrically connected via connection points such as vias. In addition to vias, connections can also be made by Cu-Cu junctions or bumps.

[0313] The pixel chip 201 includes a pixel array section 240, a north column signal processing circuit 301, and a south column signal processing circuit 302.

[0314] The northern column signal processing circuit 301 reads pixel signals and edge detection results from half of the pixels (odd-numbered rows, etc.) in the pixel array section 240, while the southern column signal processing circuit 302 reads pixel signals and other data from the remaining half. Various peripheral circuits, such as the signal processing circuit 280, are arranged on the circuit chip 202.

[0315] As illustrated in Figure 51, it is also possible to place only the pixel array section 240 on the pixel chip 201 and the remaining circuits on the circuit chip 202.

[0316] As illustrated in Figures 50 and 51, miniaturization of pixels becomes easier by distributing the circuits across the stacked pixel chips 201 and circuit chips 202. It is also possible to stack three or more semiconductor chips and arrange the circuits on them.

[0317] Furthermore, each of the second to tenth embodiments can be applied to the eleventh embodiment.

[0318] Thus, according to the eleventh embodiment of this technology, since the circuits are distributed and arranged on the stacked pixel chip 201 and circuit chip 202, pixel miniaturization becomes easier.

[0319] <12. Twelfth Embodiment> In the first embodiment described above, the image sensor 200 reads out the pixel signal of the Gr pixel in the first exposure mode, and then reads out the pixel signal of the Gb pixel in the second exposure mode. However, it is preferable to improve the readout speed. The image sensor 200 in this twelfth embodiment differs from the first embodiment in that it adds the respective pixel signals of the Gr pixel and the Gb pixel in the second exposure mode.

[0320] In the twelfth embodiment, the same FD sharing structure as illustrated in Figure 3 is used.

[0321] Figure 52 is a timing chart showing an example of the readout operation of the image sensor 200 in the twelfth embodiment of this technology. The vertical scanning circuit 210 sets the selection signal SEL to a high level at timing T3, and immediately thereafter sets the reset signal RST and the auto-zero signal AZ to low levels in sequence.

[0322] The vertical scanning circuit 210 then supplies a high-level transfer signal TRG_Gr from timing T21 through the pulse period. This reads out the D phase of the first exposure mode. The solid curve in the figure represents the pixel signal Diff on the vertical signal line VSL0 side. ｎо The waveform is shown, and the dashed line represents the pixel signal Diff on the vertical signal line VSL3 side. ｐо The waveform is shown.

[0323] The vertical scanning circuit 210 then supplies a high-level transfer signal TRG_Gb from timing T23 for the duration of the pulse. In the twelfth embodiment, unlike the first embodiment illustrated in Figure 16b, the reset signal RST pulse at timing T22 is not supplied, and auto-zero is not performed at that time. This control allows the D0, which is the D phase of the first exposure mode, to be read out, and then the D0 and the D phase corresponding to the second exposure mode to be added together and read out as D1 (in other words, output). Note that D0 is an example of the first pixel signal described in the claims, and D1 is an example of the second pixel signal described in the claims.

[0324] In the first embodiment, pixel addition was not performed when reading the D phase of the second exposure mode, so it was necessary to perform auto-zero again after reading D0. In contrast, in the twelfth embodiment, the pixel signal corresponding to the second exposure mode is added to the pixel signal of the first exposure mode and read out. Therefore, the second auto-zero is unnecessary, and the readout speed can be improved.

[0325] Furthermore, each of the second to eleventh embodiments can be applied to the twelfth embodiment.

[0326] Thus, according to the twelfth embodiment of this technology, the pixel signal corresponding to the second exposure mode is read out by adding it to the pixel signal of the first exposure mode, thereby improving the readout speed.

[0327] <13. Thirteenth Embodiment> In the twelfth embodiment described above, multiple pixels shared a single disk disk (FD), but the circuit configuration is not limited to this. The image sensor 200 in the thirteenth embodiment differs from the first embodiment in that it has an FD for each pixel, connects the FDs to each other, and is driven in the first and second exposure modes.

[0328] Figure 53 is a circuit diagram showing an example configuration of pixels 710 and 720 in the thirteenth embodiment of this technology. Pixel 710 comprises a photoelectric conversion element 711, a transfer transistor 712, a connection transistor 713, an FD 714, a reset transistor 715, an amplification transistor 716, and a selection transistor 717. Pixel 720 is a pixel adjacent to pixel 710 and comprises a photoelectric conversion element 721, a transfer transistor 722, a connection transistor 723, an FD 724, a reset transistor 725, an amplification transistor 726, and a selection transistor 727. For example, nMOS transistors are used as transistors in pixels 710 and 720.

[0329] The photoelectric conversion elements 711 and 721 convert received light into electric charge and store it. The transfer transistor 712 transfers charge from the photoelectric conversion element 711 to the FD 714 according to the transfer signal TRGa from the vertical scanning circuit 210. The transfer transistor 722 transfers charge from the photoelectric conversion element 721 to the FD 724 according to the transfer signal TRGb from the vertical scanning circuit 210.

[0330] The reset transistor 715 initializes the photoelectric conversion elements 711 and FD714 according to the reset signal RST from the vertical scanning circuit 210. The reset transistor 725 initializes the photoelectric conversion elements 721 and FD724 according to the reset signal RST.

[0331] The amplification transistor 716 forms a source follower circuit and outputs a voltage corresponding to the voltage of FD 714. The amplification transistor 726 forms a source follower circuit and outputs a voltage corresponding to the voltage of FD 724. The selection transistors 717 and 727 are driven simultaneously by the selection signal SEL from the vertical scanning circuit 210. These selection transistors supply the pixel signal SIG, which is the voltage output by the amplification transistors 716 and 726, to the column signal processing circuit 300 via the vertical signal line VSL.

[0332] The connecting transistors 713 and 723 connect FD714 and FD724 according to the control signal ALSEN.

[0333] Figure 54 is a timing chart showing an example of the readout operation of the image sensor 200 in the thirteenth embodiment of this technology. The timing control circuit 230 raises the control signal ALSEN to a high level during the readout period from timing T3 to T4, when the selection signal SEL is at a high level. As a result, the FDs of adjacent pixels 710 and 720 are connected.

[0334] The control methods for SEL, RST, TRGa, TRGb, and AZ in the same figure are the same as the control methods for SEL, RST, TRG_Gr, TRG_Gb, and AZ in Figure 52.

[0335] As illustrated in Figure 54, the vertical scanning circuit 210 can drive pixel 710 in a first exposure mode and pixel 720 in a second exposure mode, while connecting the respective FDs of adjacent pixels 710 and 720. This allows the pixel signals of the first exposure mode and the second exposure mode to be added together, even in a configuration where the FDs are not shared.

[0336] Furthermore, each of the second to eleventh embodiments can be applied to the twelfth embodiment.

[0337] Thus, according to the thirteenth embodiment of this technology, the vertical scanning circuit 210 drives each pixel in the first and second exposure modes while connecting the respective FDs of the two pixels. Therefore, even in a configuration where the FDs are not shared, the pixel signals of each exposure mode can be added together.

[0338] <14. Fourteenth Embodiment> In the thirteenth embodiment described above, the pixel signals were added by connecting two FDs, but the method of addition is not limited to this. The image sensor 200 in this fourteenth embodiment differs from the thirteenth embodiment in that it adds the pixel signals using a capacitive element in the preceding stage of the comparison circuit.

[0339] Figure 55 is a circuit diagram showing an example configuration of the pixel array unit 240 and column signal processing circuit 300 in the 14th embodiment of this technology. The pixel array unit 240 has multiple pixels arranged in it, such as pixels 710, 720, 730, and 740. For example, pixels 710 and 720 are arranged in one column, and pixels 730 and 740 are arranged in another column. The circuit configuration of pixels 710 and 720 in the 14th embodiment is the same as in the 13th embodiment, except that the connecting transistors 713 and 723 are not arranged. The circuit configuration of pixels 730 and 740 is the same as that of pixels 710 and 720.

[0340] Furthermore, two vertical signal lines VSL are wired to each row of the pixel array 240. Pixel 710 is connected to vertical signal line VSL0-a, and pixel 720 is connected to vertical signal line VSL0-b. Pixel 730 is connected to vertical signal line VSL3-a, and pixel 740 is connected to vertical signal line VSL3-b.

[0341] In the 14th embodiment, the pixel signal selection unit 310 in the column signal processing circuit 300 further comprises adder circuits 750 and 760. In addition, load MOS current sources 311-1 and 311-2 are provided instead of load MOS current source 311, and load MOS current sources 314-1 and 314-2 are provided instead of load MOS current source 314.

[0342] A load MOS current source 311-1 is connected to the vertical signal line VSL0-a, and a load MOS current source 311-2 is connected to the vertical signal line VSL0-b. A load MOS current source 314-1 is connected to the vertical signal line VSL3-a, and a load MOS current source 314-2 is connected to the vertical signal line VSL3-b.

[0343] Furthermore, the adder circuit 750 is connected to the vertical signal lines VSL0-a and VSL0-b, and the two pixel signals are input to the adder circuit 750 via these lines. The adder circuit 750 calculates the summation average of these pixel signals and outputs it to the comparator circuit 430 as the pixel signal SIG0 via the signal line 319-0.

[0344] The summing circuit 760 is connected to the vertical signal lines VSL3-a and VSL3-b, through which the two pixel signals are input to the summing circuit 760. The summing circuit 760 calculates the summed average of these pixel signals and outputs it to the comparator circuit 459 as the pixel signal SIG3 via the signal line 319-3.

[0345] Figure 56 is a circuit diagram showing one example configuration of an adder circuit 750 in the fourteenth embodiment of this technology. This adder circuit 750 includes switches 751 and 752 and capacitive elements 753 and 754. The circuit configuration of the adder circuit 760 is the same as that of the adder circuit 750.

[0346] Switch 751 opens and closes the path between the vertical signal line VSL0-a and one end of the capacitive element 753 according to the control signal SWa from the timing control circuit 230. Switch 752 opens and closes the path between the vertical signal line VSL0-b and one end of the capacitive element 754 according to the control signal SWb from the timing control circuit 230. The other ends of the capacitive elements 753 and 754 are connected in common to the signal line 319-0.

[0347] Figure 57 is a timing chart showing an example of the image sensor readout operation in the 14th embodiment of this technology. At timing T3, the timing control circuit 230 sets the control signals SWa and Swb to a high level.

[0348] Then, the timing control circuit 230 sets the control signals SWa and SWb to a low level at timing T20, immediately before the charge transfer within the pixel 710. The timing control circuit 230 supplies a high-level transfer signal TRGa during the pulse period from timing T21 to T22. Also, the timing control circuit 230 sets the control signal SWa to a high level at timing T22.

[0349] The vertical scanning circuit 210 then supplies a high-level transfer signal TRGb during the pulse period from timing T23 to T24. The timing control circuit 230 sets the control signal SWb to a high level at timing T24 and sets the control signals SWa and SWb to a low level at timing T4.

[0350] With the control shown in the figure, the adder circuit 750 (not shown) holds D0 output in the first exposure mode in the capacitive element 753 (not shown) and holds D1 output in the second exposure mode in the capacitive element 754 (not shown). Since these capacitive elements are connected in parallel, the summation average of D0 and D1 is output from the adder circuit 750 as SIG0. Note that D0 is an example of the first pixel signal described in the claims, and D1 is an example of the second pixel signal described in the claims.

[0351] Furthermore, each of the second to eleventh embodiments can be applied to the fourteenth embodiment.

[0352] Thus, according to the 14th embodiment of this technology, since the summing circuit 750 calculates the average of the pixel signals output in each exposure mode, the pixel signals of each exposure mode can be added even in a configuration where the FD is not shared.

[0353] <15. Fifteenth Embodiment> In the thirteenth embodiment described above, pixel signals were added by connecting two floppy disks, but the method of addition is not limited to this. The image sensor 200 in this fifteenth embodiment differs from the thirteenth embodiment in that it adds pixel signals using a sample-and-hold circuit.

[0354] Figure 58 is a circuit diagram showing an example configuration of the pixel array unit 240 and column signal processing circuit 300 in the 15th embodiment of this technology. The configuration of the pixel array unit 240 in the 15th embodiment is the same as that of the 14th embodiment illustrated in Figure 55.

[0355] Furthermore, the pixel signal selection unit 310 in the column signal processing circuit 300 in the 15th embodiment further comprises sample-and-hold circuits 770 and 780. In addition, load MOS current sources 311-1 and 311-2 are provided instead of load MOS current source 311, and load MOS current sources 314-1 and 314-2 are provided instead of load MOS current source 314.

[0356] A load MOS current source 311-1 is connected to the vertical signal line VSL0-a, and a load MOS current source 311-2 is connected to the vertical signal line VSL0-b. A load MOS current source 314-1 is connected to the vertical signal line VSL3-a, and a load MOS current source 314-2 is connected to the vertical signal line VSL3-b.

[0357] Furthermore, the sample-and-hold circuit 770 is connected to the vertical signal lines VSL0-a and VSL0-b, and the two pixel signals are input to the sample-and-hold circuit 770 via these lines. The sample-and-hold circuit 770 samples these pixel signals, adds them together, and outputs the pixel signal SIG0 to the comparator circuit 430 via the signal line 319-0.

[0358] The sample-and-hold circuit 780 is connected to vertical signal lines VSL3-a and VSL3-b, through which two pixel signals are input to the sample-and-hold circuit 780. The sample-and-hold circuit 780 samples these pixel signals, adds them together, and outputs the pixel signal SIG3 to the comparator circuit 459 via signal line 319-3.

[0359] Figure 59 is a circuit diagram showing one example configuration of a sample-and-hold circuit 770 in the fifteenth embodiment of this technology. This sample-and-hold circuit 770 comprises switches 771, 772, 773, 774, 775, and 776, and capacitive elements 777 and 778. The circuit configuration of the sample-and-hold circuit 780 is the same as that of the sample-and-hold circuit 770.

[0360] Switch 771 opens and closes the path between the vertical signal line VSL0-a and one end of the capacitive element 777 according to the control signal SWa from the timing control circuit 230. Switch 772 opens and closes the path between the vertical signal line VSL0-b and one end of the capacitive element 778 according to the control signal SWb from the timing control circuit 230. The other ends of the capacitive elements 777 and 778 are grounded.

[0361] Switch 773 opens and closes the path between the connection node of switch 771 and capacitive element 777 and the power supply voltage according to the control signal rSWa from the timing control circuit 230. Switch 774 opens and closes the path between the connection node of switch 772 and capacitive element 778 and the power supply voltage according to the control signal rSWb from the timing control circuit 230.

[0362] Switch 775 opens and closes the path between the connection node of switch 771 and capacitive element 777 and signal line 319-0 according to the control signal adSWa from the timing control circuit 230. Switch 776 opens and closes the path between the connection node of switch 772 and capacitive element 778 and signal line 319-0 according to the control signal adSWb from the timing control circuit 230.

[0363] Figure 60 is a timing chart showing an example of the image sensor readout operation in the 15th embodiment of this technology. The timing control circuit 230 sets the control signals rSWa, rSWb, SWa, and SWb to a high level at timing T3. Then, at timing T20, when the reset signal RST becomes low, the timing control circuit 230 sets the control signals rSWa and rSWb to a low level.

[0364] The timing control circuit 230 then lowers the control signals SWa and SWb to a low level at timing T21, and supplies high-level control signals adSWa and adSWb over the period from timing T21 to T22. The vertical scanning circuit 210 then supplies a high-level transfer signal TRGa over the pulse period from timing T22 to T23.

[0365] The timing control circuit 230 then supplies a high-level control signal SWa during the period from timing T23 to T24, and a high-level control signal adSWa over the period from timing T24 to T25. The vertical scanning circuit 210 then supplies a high-level transfer signal TRGb over the pulse period from timing T25 to T26.

[0366] The timing control circuit 230 then supplies a high-level control signal SWb during the period from timing T26 to T27, and supplies a high-level control signal adSWb over the period from timing T27 to T4.

[0367] With the control shown in the figure, the sample-and-hold circuit 770 samples D0 corresponding to the first exposure mode and D1 corresponding to the second exposure mode, adds them together, and outputs them as the pixel signal SIG0. D0 is an example of the first pixel signal described in the claims, and D1 is an example of the second pixel signal described in the claims.

[0368] Furthermore, each of the second to eleventh embodiments can be applied to the fifteenth embodiment.

[0369] Thus, according to the 15th embodiment of this technology, since the sample-and-hold circuit 770 samples and adds the pixel signals output in each exposure mode, the pixel signals of each exposure mode can be added even in a configuration where the floppy disk is not shared.

[0370] <16. Sixteenth Embodiment> In the first embodiment described above, the HDR edge synthesis unit 282 synthesized (in other words, compressed) the determination results Ew_L and Ew_S, which had different exposure times, into one bit, but the details of the compression circuit were not described. The image sensor 200 in this sixteenth embodiment differs from the first embodiment in that a specific example of the compression circuit is described.

[0371] Figure 61 is a block diagram showing one example configuration of the column signal processing circuit 300 in the sixteenth embodiment of this technology.

[0372] The edge determination circuit 320 in the 16th embodiment differs from that in the first embodiment in that the window edge synthesis unit 450 is not provided.

[0373] The column signal processing circuit 300 in the 16th embodiment differs from the first embodiment in that it further includes a plurality of synthesis units such as synthesis units 790 and 800. These synthesis units are arranged every two columns. Furthermore, the subsequent signal processing circuit 280 (not shown) does not include an edge determination and holding unit 281 and an HDR edge synthesis unit 282, and the outputs of each synthesis unit are output to the recognition unit 283.

[0374] The synthesis unit 790 determines the presence or absence of edges based on the comparison results VCO0 and VCO3 of the first exposure mode, determines the presence or absence of edges based on the comparison results VCO0 and VCO3 of the second exposure mode, and synthesizes (compresses) these determination results. This synthesis unit 790 includes, for example, a NAND gate 791, an OR gate 792, and an AND (logical conjunction) gate 793. The configuration of the synthesis unit 800 is the same as that of the synthesis unit 790, except that it determines the presence or absence of edges based on the comparison results VCO1 and VCO2.

[0375] The NAND gate 791 outputs the negative logical AND of the comparison results VCO0 and VCO3 as the input signal IN to the OR gate 792. This input signal IN indicates the presence or absence of an edge. Furthermore, the NAND gate 791 is implemented using four transistors, as illustrated in Figure 62a.

[0376] The OR gate 792 outputs the logical OR of the feedback signal FB from the AND gate 793 and the input signal IN to the AND gate 793. The AND gate 793 outputs the logical AND of the enable signal EN from the timing control circuit 230 and the output signal of the OR gate 792 as the output signal OUT to the recognition unit 283. When the synthesis unit 790 is enabled, the logical value "1" is set to the enable signal EN, and when the synthesis unit 790 is disabled, the logical value "0" is set to the enable signal EN. Furthermore, both the OR gate 792 and the AND gate 793 are implemented by six transistors, as illustrated in Figures b and c of 62.

[0377] In the circuit illustrated in Figure 61, the NAND gate 791 determines the presence or absence of an edge and outputs IN as the result. The determination result for the first exposure mode (IN) and the determination result for the second exposure mode (IN) are output in sequence, and the enable signal EN is set to a logical value of "1" (i.e., enable) within the period during which they are output.

[0378] The OR gate 792 and AND gate 793 hold the determination result of the first exposure mode as a feedback signal FB, compress this signal and the determination result of the second exposure mode (IN) into one bit, and output it as OUT.

[0379] Figure 63 is a truth table showing an example of the operation of the synthesis unit 790 in the sixteenth embodiment of this technology. In the figure, a shows the truth table of the NAND gate 791 in the synthesis unit 790, and b shows the truth tables of the OR gate 792 and the AND gate 793.

[0380] As illustrated in figure a, when both comparison results VCO0 and VCO3 have a logical value of "1", the NAND gate 791 outputs an input signal IN with a logical value of "0". This value indicates that there is no edge. Also, when at least one of the comparison results VCO0 and VCO3 has a logical value of "0", the NAND gate 791 outputs an input signal IN with a logical value of "1". This value indicates that an edge has been detected.

[0381] As illustrated in figure b, when the enable signal EN is logically "1" (enable) and at least one of the input signal IN and the feedback signal FB is logically "1", an output signal OUT with a logical value of "1" is output. This value indicates that an edge was detected in either the first or second exposure mode. Also, when the enable signal EN is logically "1" and both the input signal IN and the feedback signal FB are logically "0", an output signal OUT with a logical value of "0" is output. This value indicates that there are no edges in either the first or second exposure mode. Furthermore, when the enable signal EN is logically "0" (disabled), an output signal OUT with a logical value of "0" is output regardless of the values ​​of the input signal IN and the feedback signal FB.

[0382] Figure 64 is a timing chart showing an example of the operation of the synthesis unit 790 in the 16th embodiment of this technology. In the figure, a shows the operation when there are no edges in the first exposure mode and edges are detected in the second exposure mode. In the figure, b shows the operation when edges are detected in both the first and second exposure modes.

[0383] As illustrated in figure a, the timing control circuit 230 sets the enable signal EN to a high level at timing T30 and to a low level at timing T33. Also, during the period from timing T30 to T31, the pixels are driven in the first exposure mode, and the NAND gate 791 outputs a low-level input signal IN indicating the absence of edges. The feedback signal FB at this time is, for example, set to a low level.

[0384] Then, within the period from timing T31 to T33, the pixels are driven in the second exposure mode, and the NAND gate 791 outputs a high-level input signal IN indicating that an edge has been detected. The feedback signal FB, which indicates the determination result of the first exposure mode, is low level. Based on these, the AND gate 793 outputs a high-level output signal OUT within the period from timing T32 to T33.

[0385] Furthermore, as illustrated in figure b, when an edge is detected in the first exposure mode, the NAND gate 791 outputs a high-level input signal IN at timing T31. Then, the AND gate 793 outputs a high-level output signal OUT at timing T32 or later.

[0386] Then, within the period from timing T33 to T34, the pixels are driven in the second exposure mode, and the NAND gate 791 outputs a high-level input signal IN indicating the presence of an edge. Also, the feedback signal FB indicating the determination result of the first exposure mode is high-level. Based on these, the AND gate 793 keeps the output signal OUT high-level within the period from timing T33 to T34.

[0387] As illustrated in figures a and b, the pixels are driven sequentially in the first and second exposure modes during the period when the enable signal EN is at a high level. The compositing unit 790 sequentially generates the determination result (IN) for the first exposure mode and the determination result (IN) for the second exposure mode during that period, and outputs an output signal OUT which is a 1-bit composite (compressed) version of these.

[0388] Furthermore, each of the second to eleventh embodiments can be applied to the sixteenth embodiment.

[0389] Thus, according to the sixteenth embodiment of this technology, the synthesis unit 790 sequentially generates the determination results for the first and second exposure modes and synthesizes (compresses) them, thereby reducing the amount of data.

[0390] [First Modification] In the sixteenth embodiment described above, a NAND gate 791, an OR gate 792, and an AND gate 793 were arranged in the combining unit 790, but the circuit configuration is not limited to this. The image sensor 200 in this first modification of the sixteenth embodiment differs from the sixteenth embodiment in that two AND gates and an OR gate are arranged in the combining unit 790.

[0391] Figure 65 is a circuit diagram showing one example configuration of the synthesis unit 790 in a first modified example of the sixteenth embodiment of this technology. The synthesis unit 790 in this first modified example of the sixteenth embodiment comprises AND gates 794 and 795 and an OR gate 796.

[0392] The AND gate 794 outputs the logical AND of the comparison results VCO0 and VCO3 as the input signal IN to the AND gate 795. The AND gate 795 outputs the logical AND of the feedback signal FB and the input signal IN to the OR gate 796. The OR gate 796 outputs the logical OR of the inverted signal XEN (obtained by inverting the enable signal EN) and the signal from the AND gate 795 as the output signal OUT, and also feeds this back to the AND gate 795 as the feedback signal FB. Note that the AND gates 794 and 795 are examples of the first AND gate and the second AND gate described in the claims.

[0393] Figure 66 is a truth table showing an example of the operation of the synthesis unit 790 in a first modified example of the sixteenth embodiment of the present technology. In the figure, a shows the truth table of the AND gate 794 in the synthesis unit 790, and b shows the truth tables of the AND gate 795 and the OR gate 796.

[0394] As illustrated in figure a, when both comparison results VCO0 and VCO3 have a logical value of "1", the AND gate 794 outputs an input signal IN with a logical value of "1". This value indicates that there is no edge. Also, when at least one of the comparison results VCO0 and VCO3 has a logical value of "0", the AND gate 794 outputs an input signal IN with a logical value of "0". This value indicates that an edge has been detected.

[0395] As illustrated in figure b, when the inversion signal XEN has a logical value of "0" (enable) and at least one of the input signal IN and the feedback signal FB has a logical value of "0", an output signal OUT with a logical value of "0" is output. This value indicates that an edge was detected in either the first or second exposure mode. Also, when the inversion signal XEN has a logical value of "0" and both the input signal IN and the feedback signal FB have a logical value of "1", an output signal OUT with a logical value of "1" is output. This value indicates that there are no edges in either the first or second exposure mode. Furthermore, when the inversion signal XEN has a logical value of "1" (disable), an output signal OUT with a logical value of "1" is output regardless of the values ​​of the input signal IN and the feedback signal FB.

[0396] Figure 67 is a timing chart showing an example of the operation of the composite unit 790 in a first modified example of the 16th embodiment of the present technology. In the figure, a shows the operation when an edge is detected in the first exposure mode and there is no edge in the second exposure mode. In the figure, b shows the operation when there is no edge in both the first and second exposure modes.

[0397] As illustrated in figure a, the timing control circuit 230 sets the inverted signal XEN to a low level at timing T30 and to a high level at timing T34. During the period from timing T30 to T32, the pixels are driven in the first exposure mode, and the AND gate 794 outputs a low-level input signal IN indicating that an edge has been detected. The feedback signal FB at this time is, for example, set to a low level. The OR gate 796 then outputs a high-level output signal OUT from timing T31 onward.

[0398] Then, within the period from timing T32 to T34, the pixels are driven in the second exposure mode, and the AND gate 794 outputs a high-level input signal IN indicating that there are no edges. Also, the feedback signal FB indicating the determination result of the first exposure mode is low level. Based on these, the OR gate 796 lowers the output signal OUT to a low level at timing T33.

[0399] Furthermore, as illustrated in figure b, if there are no edges in the first exposure mode, the AND gate 794 outputs a high-level input signal IN at timing T31. Then, the OR gate 796 outputs a high-level output signal OUT at timing T32 or later.

[0400] Then, within the period from timing T33 to T34, the pixels are driven in the second exposure mode, and the AND gate 794 outputs a high-level input signal IN indicating that there are no edges. Also, the feedback signal FB indicating the result of the first exposure mode determination is high level. Based on these, the OR gate 796 keeps the output signal OUT high level after timing T33.

[0401] Thus, according to the first modification of the sixteenth embodiment of this technology, the determination results of each exposure mode can be combined (compressed) by a circuit using AND gates 794 and 795 and OR gate 796.

[0402] [Second Modification] In the sixteenth embodiment described above, a NAND gate 791, an OR gate 792, and an AND gate 793 were arranged in the combining unit 790, but the circuit configuration is not limited to this. The image sensor 200 in this second modification of the sixteenth embodiment differs from the sixteenth embodiment in that an AND gate and two NAND gates are arranged in the combining unit 790.

[0403] Figure 68 is a circuit diagram showing one example configuration of the synthesis unit 790 in a second modification of the sixteenth embodiment of the present technology. The synthesis unit 790 in this second modification of the sixteenth embodiment comprises an AND gate 794 and NAND gates 797 and 798. Each of these NAND gates is realized by, for example, four transistors. In the sixteenth embodiment, a total of 16 transistors were required, but in the configuration shown in the figure, a total of 12 transistors are sufficient.

[0404] The AND gate 794 outputs the logical AND of the comparison results VCO0 and VCO3 as the input signal IN to the NAND gate 797. The NAND gate 797 outputs the negative logical AND of the feedback signal FB and the input signal IN to the NAND gate 798. The NAND gate 798 outputs the negative logical AND of the enable signal EN and the signal from the NAND gate 797 as the output signal OUT, and also feeds this back to the NAND gate 797 as the feedback signal FB. Note that the NAND gates 797 and 798 are examples of the first NAND gate and the second NAND gate described in the claims.

[0405] Figure 69 is a truth table showing an example of the operation of the synthesis unit 790 in a second modification of the sixteenth embodiment of the present technology. In the figure, a shows the truth table of the AND gate 794 in the synthesis unit 790, and b shows the truth tables of the NAND gates 797 and 798.

[0406] As illustrated in figure a, when both comparison results VCO0 and VCO3 have a logical value of "1", the AND gate 794 outputs an input signal IN with a logical value of "1". Also, when at least one of the comparison results VCO0 and VCO3 has a logical value of "0", the AND gate 794 outputs an input signal IN with a logical value of "0".

[0407] As illustrated in figure b, when the enable signal EN is a logical value of "1" (enable) and at least one of the input signal IN and the feedback signal FB is a logical value of "0", an output signal OUT with a logical value of "0" is output. This value indicates that an edge was detected in either the first or second exposure mode. Also, when the enable signal EN is a logical value of "1" and both the input signal IN and the feedback signal FB are logical values ​​of "1", an output signal OUT with a logical value of "1" is output. This value indicates that there are no edges in either the first or second exposure mode. Furthermore, when the enable signal EN is a logical value of "0" (disable), an output signal OUT with a logical value of "1" is output regardless of the values ​​of the input signal IN and the feedback signal FB.

[0408] Figure 70 is a timing chart showing an example of the operation of the composite unit 790 in a second modification of the 16th embodiment of the present technology. In the figure, a shows the operation when there are no edges in the first exposure mode and edges are detected in the second exposure mode. In the figure, b shows the operation when there are no edges in both the first and second exposure modes.

[0409] As illustrated in figure a, the timing control circuit 230 sets the enable signal EN to a high level at timing T30 and to a low level at timing T33. Also, during the period from timing T30 to T31, the pixels are driven in the first exposure mode, and the AND gate 794 outputs a high-level input signal IN indicating the absence of edges. The feedback signal FB at this time is, for example, set to a low level.

[0410] Then, within the period from timing T31 to T33, the pixels are driven in the second exposure mode, and the AND gate 794 outputs a low-level input signal IN indicating the presence of an edge. Also, the feedback signal FB indicating the determination result of the first exposure mode is high level. Based on these, the AND gate 793 outputs a low-level output signal OUT within the period from timing T32 to T33.

[0411] Furthermore, as illustrated in figure b, if there are no edges in the first exposure mode, the AND gate 794 outputs a high-level input signal IN at timing T31. Then, the NAND gate 798 outputs a high-level output signal OUT at timing T32 or later.

[0412] Then, during the period from timing T33 to T34, the pixels are driven in the second exposure mode, and the AND gate 794 outputs a high-level input signal IN indicating that there are no edges. Also, the feedback signal FB indicating the result of the first exposure mode determination is high. Based on these, the NAND gate 798 keeps the output signal OUT at a high level after timing T33.

[0413] Thus, according to the second modification of the sixteenth embodiment of this technology, the determination results of each exposure mode can be combined (compressed) by a circuit using an AND gate 794 and NAND gates 797 and 798.

[0414] [Third Modification] In the sixteenth embodiment described above, a NAND gate 791, an OR gate 792, and an AND gate 793 were arranged in the combining section 790, but the circuit configuration is not limited to this. The image sensor 200 in this third modification of the sixteenth embodiment differs from the sixteenth embodiment in that a NAND gate, a latch circuit, and an OR gate are arranged in the combining section 790.

[0415] Figure 71 is a circuit diagram showing one example configuration of the synthesis unit 790 in a third modified example of the sixteenth embodiment of this technology. The synthesis unit 790 in this third modified example of the sixteenth embodiment includes a NAND gate 791, a latch circuit 799, and an OR gate 796.

[0416] The NAND gate 791 outputs the negative logical AND of the comparison results VCO0 and VCO3 as the input signal IN to the input terminal D of the latch circuit 799 and the OR gate 796. This input signal IN indicates the presence or absence of an edge.

[0417] The latch circuit 799 holds the input signal IN according to the enable signal EN and outputs the held value. For example, a D latch is used as the latch circuit 799. The enable signal EN is input to the gate terminal G of this latch circuit 799. The output terminal Q of the latch circuit 799 is connected to the input terminal of the OR gate 796. When the enable signal EN is high level, the latch circuit 799 allows the input signal IN to pass through as is. On the other hand, when the enable signal EN is low level, the latch circuit 799 holds the value of the previous IN and outputs it from the output terminal Q. The latch circuit 799 can be implemented using, for example, 14 transistors, as illustrated in Figure 72.

[0418] The OR gate 796 outputs the logical OR of the input signal IN and the output signal of the latch circuit 799 as the output signal OUT.

[0419] The timing control circuit 230 sets the enable signal EN to a high level during the period of the first exposure mode and to a low level during the period of the second exposure mode. As a result, the latch circuit 799 holds the input signal IN of the first exposure mode, and the OR gate 796 can combine the held value and the input signal IN of the second exposure mode.

[0420] Thus, according to the third modification of the sixteenth embodiment of the present technology, the circuit using the NAND gate 791, the latch circuit 799, and the OR gate 796 can combine (compress) the determination results of each exposure mode.

[0421] <17. The Seventeenth Embodiment> In the above-described sixteenth embodiment, the combining unit is provided every two columns in the column signal processing circuit 300 for combining. However, it is also possible to perform the combining at the subsequent stage of the column signal processing circuit 300. The image sensor 200 in this seventeenth embodiment is different from the sixteenth embodiment in that the combining is performed using the line memory at the subsequent stage of the column signal processing circuit 300.

[0422] FIG. 73 is a block diagram showing a configuration example of the edge determination circuit 320 in the seventeenth embodiment of the present technology. The edge determination circuit 320 in the seventeenth embodiment is different from the sixteenth embodiment in that it includes a plurality of NAND gates such as the NAND gates 791 and 801 instead of the combining units 790 and 800. These NAND gates are arranged every two columns. If the number of columns is M (M is an integer), M / 2 NAND gates are arranged.

[0423] The NAND gate 791 outputs the negative logical product of the comparison results VCO0 and VCO3 as the input signal IN to the signal processing circuit 280. This input signal IN indicates the presence or absence of an edge. The NAND gate 801 outputs the negative logical product of the comparison results VCO1 and VCO2 as the input signal IN to the signal processing circuit 280. If the number of columns is M, M / 2 input signals IN (in other words, M / 2 bits) are output.

[0424] Figure 74 is a block diagram showing one example configuration of a signal processing circuit 280 in the 17th embodiment of this technology. The signal processing circuit 280 in this 17th embodiment includes a line memory 291, an OR processing circuit 292, a recognition unit 283, and an image processing unit 284.

[0425] The line memory 291 holds M / 2 input signals IN (determination results). This line memory 291 holds M / 2 input signals IN over the duration of the first exposure mode, which has a long exposure time, and outputs them as IN_L to the OR processing circuit 292.

[0426] The OR processing circuit 292 receives M / 2 INs corresponding to the second exposure mode with a shorter exposure time as IN_S, and calculates the logical OR of these IN_Ss and M / 2 IN_Ls. The OR processing circuit 292 calculates the logical OR digit by digit and outputs M / 2 output signals OUT to the recognition unit 283. As a result, IN_S and IN_L are combined (compressed). Note that the OR processing circuit 292 is an example of the combining unit described in the claims.

[0427] Furthermore, each of the second to eleventh embodiments can be applied to the seventeenth embodiment.

[0428] Thus, according to the 17th embodiment of this technology, since synthesis (compression) is performed using the line memory 291 in the subsequent stage of the column signal processing circuit 300, the circuit size of the column signal processing circuit 300 can be reduced.

[0429] <18. Examples of Application to Mobile Devices> The technology relating to this disclosure (this technology) can be applied to various products. For example, the technology relating to this disclosure may be realized as a device mounted on any type of mobile device such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0430] Figure 75 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0431] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 75, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0432] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0433] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0434] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0435] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0436] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0437] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0438] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0439] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0440] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 75, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0441] Figure 76 shows an example of the installation position of the imaging unit 12031.

[0442] In Figure 76, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0443] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0444] Figure 76 shows an example of the imaging ranges of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0445] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0446] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0447] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0448] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0449] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to, for example, the imaging unit 12031 of the configuration described above. Specifically, the imaging device 100 in Figure 1 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, it becomes possible to improve robustness to the imaging environment and improve the reliability of the system.

[0450] The embodiments described above are merely examples of how to realize this technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of this technology that bear the same name. However, this technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology.

[0451] The effects described herein are merely illustrative and not limited to those described herein, and other effects may also occur.

[0452] Furthermore, this technology can also take the following configurations: (1) An image sensor comprising: a vertical scanning circuit that controls a pixel array section having a plurality of pixels arranged in a first exposure mode to output a pair of pixel signals as a pair of first pixel signals, and controls the pixel array section to drive a pair of pixel signals as a pair of second pixel signals; an edge detection circuit that performs a process to determine the presence or absence of an edge based on the pair of first pixel signals and a process to determine the presence or absence of an edge based on the pair of second pixel signals; and a synthesis section that synthesizes the signals from the edge detection circuit. (2) The image sensor according to (1), wherein the exposure time of the pixels driven in the first exposure mode is different from the exposure time of the pixels driven in the second exposure mode. (3) The image sensor according to (1), wherein the sensitivity of the first pixels driven in the first exposure mode is different from the sensitivity of the second pixels driven in the second exposure mode. (4) The image sensor according to any one of (1) to (3), wherein at least one of the first pixel signals and the second pixel signals is a signal obtained by adding a plurality of pixel signals. (5) The vertical scanning circuit drives a pair of first pixels in a first exposure mode and drives a pair of second pixels different from the pair of first pixels in a second exposure mode, according to any one of (1) to (4). (6) The image sensor according to (5), wherein the row in the pixel array is not composed of a mixture of first pixels and second pixels. (7) The image sensor according to (5), wherein the column in the pixel array is not composed of a mixture of first pixels and second pixels. (8) The image sensor according to any one of (1) to (7), wherein the edge determination circuit comprises a gain control circuit that controls at least one of the pair of pixel signals to increase or decrease by a predetermined gain as gain control, an offset control circuit that controls at least one of the pair of pixel signals to superimpose a predetermined offset as offset control, and a comparison circuit that compares the pair of pixel signals subjected to the gain control and the offset control and outputs a comparison result.(9) The image sensor according to (8), further comprising an analog-to-digital converter that converts pixel signals into digital signals, wherein the analog-to-digital converter and the gain control circuit share the comparison circuit. (10) The image sensor according to (8) or (9), wherein the gain control circuit comprises a first gain control circuit that increases or decreases one of the pair of pixel signals by the gain, and a second gain control circuit that increases or decreases the other of the pair of pixel signals by the gain, and the comparison circuit comprises a first comparison circuit that compares the signal increased or decreased by the first gain control circuit with the other signal, and a second comparison circuit that compares the signal increased or decreased by the second gain control circuit with the one of the signals. (11) The image sensor according to (10), wherein the combining unit sequentially generates and combines first and second determination results indicating the presence or absence of edges based on the comparison results of the first and second comparison circuits, respectively. (12) The image sensor according to (11), wherein the combining unit comprises a NAND gate that outputs the negative logical AND of the comparison results of the first and second comparison circuits as an input signal, an OR gate that outputs the logical OR of the input signal and the feedback signal, and an AND gate that outputs the logical AND of the output signal of the OR gate and the enable signal and feeds it back as the feedback signal. (13) The image sensor according to (11), wherein the combining unit comprises a first AND gate that outputs the logical AND of the comparison results of the first and second comparison circuits as an input signal, a second AND gate that outputs the logical AND of the input signal and the feedback signal, and an OR gate that outputs the logical OR of the output signal of the second AND gate and the enable signal and feeds it back as the feedback signal. (14) The image sensor according to (11), wherein the combining unit comprises an AND gate that outputs the logical AND of the comparison results of the first and second comparison circuits as an input signal; a first NAND gate that outputs the negative logical AND of the input signal and the feedback signal; and a second NAND gate that outputs the negative logical AND of the output signal of the first NAND gate and the enable signal and feeds it back as the feedback signal.(15) The image sensor according to (11), wherein the combining unit comprises a NAND gate that outputs the negative logical AND of the comparison results of the first and second comparison circuits as an input signal, a latch circuit that holds the input signal according to an enable signal and outputs a held value, and an OR gate that outputs the logical OR of the output signal of the latch circuit and the input signal. (16) The image sensor according to (10), further comprising a logic circuit that combines the comparison results of the first and second comparison circuits in the first exposure mode and outputs the result as a first determination, and combines the comparison results of the first and second comparison circuits in the second exposure mode and outputs the result as a second determination. (17) The image sensor according to (16), wherein the logic circuit and the combining unit are arranged in each row of the pixel array. (18) The vertical scanning circuit drives a pair of pixels in the window of the pixel array to output the pair of pixel signals, the pair of pixels does not correspond to either a row or a column of the pixel array and is arranged at both ends of a predetermined line segment that has an angle of less than 45 degrees with respect to the row, according to any one of (1) to (20) above. (19) The image sensor according to (18) above, wherein two adjacent windows overlap in some area. (20) The vertical scanning circuit further controls the pixel array to drive the pair of pixel signals as a pair of third pixel signals in a third exposure mode, the edge determination circuit further performs a process to determine the presence or absence of an edge based on the pair of third pixel signals and output a third determination result, and the synthesis unit synthesizes the first, second and third determination results, according to any one of (16) to (19) above. (21) The image sensor according to (1), wherein the vertical scanning circuit outputs a signal obtained by adding the pixel signal corresponding to the second exposure mode and the first pixel signal as the second pixel signal. (22) The image sensor according to (21), wherein the plurality of pixels include a first pixel and a second pixel that share an FD (Floating Diffusion), and the vertical scanning circuit drives the first pixel in the first exposure mode and drives the second pixel in the second exposure mode.(23) The image sensor according to (22), wherein the plurality of pixels include a first pixel having a first FD and a second pixel having a second FD, and the vertical scanning circuit drives the first pixel in a first exposure mode and drives the second pixel in a second exposure mode while connecting the first FD and the second FD. (24) The image sensor according to (1), further comprising an adder circuit that outputs an average of the sum of the first pixel signal and the second pixel signal. (25) The image sensor according to (1), further comprising a sample-and-hold circuit that samples and adds the first pixel signal and the second pixel signal. (26) The image sensor according to (1), further comprising a line memory that holds a predetermined number of first determination results from the edge determination circuit, and the combining unit combines a predetermined number of first determination results held in the line memory with a predetermined number of second determination results from the edge determination circuit. (27) An image sensor comprising: a vertical scanning circuit that controls a pixel array section having a plurality of pixels arranged in a first exposure mode to output a pair of pixel signals as a pair of first pixel signals, and controls the pixel array section to drive a pair of pixel signals as a pair of second pixel signals; an edge detection circuit that performs a process to determine the presence or absence of an edge based on the pair of first pixel signals and a process to determine the presence or absence of an edge based on the pair of second pixel signals; a recognition unit that performs a process to recognize a predetermined object based on a signal from the edge detection circuit and output first and second recognition results; and a processing result synthesis unit that synthesizes the result of processing the first recognition result and the result of processing the second recognition result.

[0453] 100 Imaging device 110 Optical unit 120 DSP circuit 130 Display unit 140 Operation unit 150 Bus 160 Frame memory 170 Storage unit 180 Power supply unit 200 Image sensor 201 Pixel chip 202 Circuit chip 210 Vertical scanning circuit 211-216 Driver 220 DAC 230 Timing control circuit 240 Pixel array unit 241 Window 242 Unit block 250 FD shared block 251, 252, 256, 257, 712, 722 Transfer transistors 253, 254, 255, 258, 711, 721 Photoelectric conversion element 259, 715, 725 Reset transistor 260, 716, 726 Amplifier transistor 261, 717, 727 Selection transistor 262, 714, 724 FD 270 Horizontal scanning circuit 280 Signal processing circuit 281 Edge determination and holding unit 282, 610 HDR edge synthesis unit 283 Recognition unit 284 Image processing unit 285, 286 Confidence value moving average calculation unit 287, 288 Threshold processing unit 289 Processing result synthesis unit 291 Line memory 292 OR processing circuit 300 Column signal processing circuit 301 North column signal processing circuit 302 South column signal processing circuit 310 Pixel signal selection unit 311-314, 311-1, 311-2, 314-1, 314-2 Load MOS current source 315-318, 471-474, 611 Selector 320 Edge determination circuit 330 AD conversion unit 331-334 ADC 335-338, 600-603 Counter 340 Inverter 405, 417, 418, 425, 441-443, 753, 754, 777, 778 Capacitive element 410, 451-453 Gain control circuit 411-414, 421-423, 461-468, 614, 751, 752, 771-776 Switch 420, 454-456 Offset control circuit 430, 457-459 Comparator circuit 431-435 pMOS transistor 436-440 nMOS transistor 450, 450-1 to 450-4 Window edge composite section 511 On-chip lens 512 Color filter 520 Wiring layer 530 Substrate612 Latch circuit 613, 792, 796 OR gate 620 Counter circuit 710, 720, 730, 740 Pixel 713, 723 Connecting transistor 750, 760 Adder circuit 770, 780 Sample and hold circuit 790, 800 Combination unit 791, 797, 798, 801 NAND gate 793, 794, 795 AND gate 799 Latch circuit 12031 Imaging unit

Claims

1. An image sensor comprising: a vertical scanning circuit that controls a pixel array section having multiple pixels arranged in a sequence to output a pair of pixel signals as a pair of first pixel signals, and controls the pixel array section to drive a pair of pixel signals as a pair of second pixel signals; an edge detection circuit that performs a process to determine the presence or absence of an edge based on the pair of first pixel signals and a process to determine the presence or absence of an edge based on the pair of second pixel signals; and a combining unit that combines the signals from the edge detection circuit.

2. The image sensor according to claim 1, wherein the exposure time of a pixel driven in the first exposure mode is different from the exposure time of a pixel driven in the second exposure mode.

3. The image sensor according to claim 1, wherein the sensitivity of the first pixel driven in the first exposure mode is different from the sensitivity of the second pixel driven in the second exposure mode.

4. The image sensor according to claim 1, wherein at least one of the first pixel signal and the second pixel signal is a signal obtained by adding up a plurality of pixel signals.

5. The image sensor according to claim 1, wherein the vertical scanning circuit drives a pair of first pixels in a first exposure mode and drives a pair of second pixels different from the pair of first pixels in a second exposure mode.

6. The image sensor according to claim 5, wherein the first pixels and the second pixels are not mixed in each row within the pixel array.

7. The image sensor according to claim 5, wherein the first pixels and the second pixels are not mixed in each of the rows within the pixel array.

8. The image sensor according to claim 1, wherein the edge determination circuit comprises a gain control circuit that performs gain control by increasing or decreasing at least one of the pair of pixel signals by a predetermined gain; an offset control circuit that performs offset control by superimposing a predetermined offset on at least one of the pair of pixel signals; and a comparison circuit that compares the pair of pixel signals on which the gain control and the offset control have been performed and outputs a comparison result.

9. The image sensor according to claim 8, further comprising an analog-to-digital converter that converts pixel signals into digital signals, wherein the analog-to-digital converter and the gain control circuit share the comparison circuit.

10. The image sensor according to claim 8, wherein the gain control circuit comprises a first gain control circuit for increasing or decreasing one of the pair of pixel signals by the gain, and a second gain control circuit for increasing or decreasing the other of the pair of pixel signals by the gain, and the comparison circuit comprises a first comparison circuit for comparing the signal increased or decreased by the first gain control circuit with the other signal, and a second comparison circuit for comparing the signal increased or decreased by the second gain control circuit with the one of the signals.

11. The image sensor according to claim 10, wherein the synthesis unit sequentially generates and synthesizes first and second determination results indicating the presence or absence of an edge based on the comparison results of the first and second comparison circuits, respectively.

12. The image sensor according to claim 11, wherein the combining unit comprises a NAND gate that outputs the negative logical AND of the comparison results of the first and second comparison circuits as an input signal; an OR gate that outputs the logical OR of the input signal and the feedback signal; and an AND gate that outputs the logical AND of the output signal of the OR gate and the enable signal and feeds it back as the feedback signal.

13. The image sensor according to claim 11, wherein the combining unit comprises: a first AND gate that outputs the logical AND of the comparison results of the first and second comparison circuits as an input signal; a second AND gate that outputs the logical AND of the input signal and the feedback signal; and an OR gate that outputs the logical OR of the output signal of the second AND gate and the enable signal and feeds it back as the feedback signal.

14. The image sensor according to claim 11, wherein the combining unit comprises an AND gate that outputs the logical AND of the comparison results of the first and second comparison circuits as an input signal; a first NAND gate that outputs the negative logical AND of the input signal and the feedback signal; and a second NAND gate that outputs the negative logical AND of the output signal of the first NAND gate and the enable signal and feeds it back as the feedback signal.

15. The image sensor according to claim 11, wherein the combining unit comprises a NAND gate that outputs the negative logical AND of the comparison results of the first and second comparison circuits as an input signal, a latch circuit that holds the input signal according to an enable signal and outputs a held value, and an OR gate that outputs the logical OR of the output signal of the latch circuit and the input signal.

16. The image sensor according to claim 10, further comprising a logic circuit that combines the comparison results of the first and second comparison circuits in the first exposure mode and outputs a first determination result, and combines the comparison results of the first and second comparison circuits in the second exposure mode and outputs a second determination result.

17. The image sensor according to claim 16, wherein the logic circuit and the synthesis unit are arranged in each row of the pixel array unit.

18. The image sensor according to claim 1, wherein the vertical scanning circuit drives a pair of pixels within the window of the pixel array to output the pair of pixel signals, and the pair of pixels does not correspond to either a row or a column of the pixel array and is arranged at both ends of a predetermined line segment that has an angle of less than 45 degrees with respect to the row.

19. The image sensor according to claim 18, wherein two adjacent windows overlap in some areas.

20. The vertical scanning circuit further controls the pixel array to drive in a third exposure mode to output a pair of pixel signals as a pair of third pixel signals; the edge determination circuit further performs a process to determine the presence or absence of an edge based on the pair of third pixel signals and output a third determination result; and the synthesis unit synthesizes the first, second and third determination results, as described in claim 16.

21. The image sensor according to claim 1, wherein the vertical scanning circuit outputs a signal obtained by adding the pixel signal corresponding to the second exposure mode and the first pixel signal as the second pixel signal.

22. The image sensor according to claim 21, wherein the plurality of pixels include a first pixel and a second pixel that share an FD (Floating Diffusion), and the vertical scanning circuit drives the first pixel in the first exposure mode and drives the second pixel in the second exposure mode.

23. The image sensor according to claim 21, wherein the plurality of pixels include a first pixel having a first FD and a second pixel having a second FD, and the vertical scanning circuit drives the first pixel in a first exposure mode and drives the second pixel in a second exposure mode while connecting the first FD and the second FD.

24. The image sensor according to claim 1, further comprising an adder circuit that outputs an average of the sum of the first pixel signal and the second pixel signal.

25. The image sensor according to claim 1, further comprising a sample-and-hold circuit for sampling and adding the first pixel signal and the second pixel signal.

26. The image sensor according to claim 1, further comprising a line memory that holds a predetermined number of first determination results from the edge determination circuit, wherein the combining unit combines a predetermined number of the first determination results held in the line memory with a predetermined number of second determination results from the edge determination circuit.

27. An image sensor comprising: a vertical scanning circuit that controls a pixel array section having multiple pixels arranged in a row to drive in a first exposure mode to output a pair of pixel signals as a pair of first pixel signals, and controls the pixel array section to drive in a second exposure mode to output a pair of pixel signals as a pair of second pixel signals; an edge detection circuit that performs a process to determine the presence or absence of an edge based on the pair of first pixel signals and a process to determine the presence or absence of an edge based on the pair of second pixel signals; a recognition unit that performs a process to recognize a predetermined object based on a signal from the edge detection circuit and output first and second recognition results; and a processing result synthesis unit that synthesizes the result of processing the first recognition result and the result of processing the second recognition result.

Images