Image sensor and control method for image sensor

Abstract

The present invention improves the precision of edge detection in an image sensor for detecting the presence or absence of an edge in each pixel pair.　This image sensor comprises: a vertical scanning circuit; and an edge detection circuit. In the image sensor, the vertical scanning circuit outputs pixel signals by sequentially driving first and second pixel pairs, and third and fourth pixel pairs that have an inter-pixel distance different from that of the first and second pixel pairs. The edge detection circuit detects the presence or absence of an edge on the basis of pixel signals of the first, second, third, and fourth pixel pairs.

Description

Image sensor and method for controlling image sensor

[0001] The present technology relates to an image sensor, and more particularly to an image sensor that detects the presence or absence of an edge, and a method for controlling the image sensor.

[0002] Conventionally, image sensors have performed various signal processing such as pixel addition and CDS (Correlated Double Sampling) processing. For example, an image sensor has been proposed that drives each of multiple pixel pairs and detects the presence or absence of an edge for each pixel pair (see, for example, Patent Document 1). In this image sensor, a gain circuit increases or decreases the voltage of one of a pair of vertical signal lines using a relative gain, and these voltages are compared using one of a pair of comparators. Also, a gain circuit increases or decreases the voltage of the other of the pair of vertical signal lines using a relative gain, and these voltages are compared using the other of the pair of comparators.

[0003] International Publication No. 2021 / 090538

[0004] The above-mentioned conventional technology attempts to detect the presence or absence of an edge without relying on ambient light by increasing or decreasing the voltage of a pair of vertical signal lines using a gain and comparing the results. However, this conventional technology can sometimes have difficulty detecting edges. This is because the appropriate pixel pair pattern varies depending on the distance to the subject, the angle, the contrast of the subject, and other factors.

[0005] This technology was developed in light of these circumstances, and aims to improve the accuracy of edge detection in image sensors that detect the presence or absence of an edge for each pixel pair.

[0006] The present technology has been made to solve the above-mentioned problems, and a first aspect thereof relates to an image sensor including: a vertical scanning circuit that sequentially drives first and second pixel pairs and third and fourth pixel pairs having an inter-pixel distance different from that of the first and second pixel pairs to output pixel signals; and an edge detection circuit that detects the presence or absence of an edge based on the pixel signals of the first, second, third, and fourth pixel pairs, and a control method thereof, thereby improving the edge detection accuracy.

[0007] In this first aspect, the angle formed by the line segment connecting a pair of pixels included in each of the first, second, third, and fourth pixel pairs and the line on which the pixels are arranged may be a value that does not correspond to any of 0 degrees, 45 degrees, and 90 degrees, thereby providing an effect that both pixels of a pixel pair are not connected to the vertical signal line.

[0008] In addition, in this first aspect, one of a pair of pixels included in each of the first, second, third, and fourth pixel pairs may be connected to one of first and second selection lines wired for each row, and the other of the pair of pixels may be connected to the other of the first and second selection lines, thereby preventing a plurality of pixel signals from flowing through the vertical signal lines.

[0009] In addition, in this first aspect, one of a pair of pixels included in each of the first, second, third, and fourth pixel pairs may be connected to one of first and second transfer lines wired for each row, and the other of the pair of pixels may be connected to the other of the first and second transfer lines, thereby preventing a plurality of pixel signals from flowing through the vertical signal lines.

[0010] In addition, in this first aspect, the line segment connecting the first pixel pair and the line segment connecting the second pixel pair may be perpendicular to each other, and the line segment connecting the third pixel pair and the line segment connecting the fourth pixel pair may be perpendicular to each other, thereby improving detection accuracy.

[0011] In this first aspect, a read start address of a first window including the first and second pixel pairs may be different from a read start address of a second window including the third and fourth pixel pairs, thereby providing an effect that reading is performed from different addresses for each window.

[0012] In addition, in this first aspect, the pattern that is line-symmetric to the pattern of the third and fourth pixel pairs may be similar to the pattern of the first and second pixel pairs, thereby improving detection accuracy.

[0013] In the first aspect, the charge accumulation period of the first and second pixel pairs may be different from the charge accumulation period of the third and fourth pixel pairs, thereby improving detection accuracy.

[0014] In this first aspect, each of the first, second, third, and fourth pixel pairs may output a pair of pixel signals, and the edge detection circuit may include a level control circuit that increases or decreases the signal level of one of the pair of pixel signals by a predetermined gain, and a comparison circuit that compares the pair of pixel signals, one of which has its signal level increased or decreased, and outputs a comparison result. This brings about the effect of being able to determine the presence or absence of an edge regardless of the amount of light.

[0015] In this first aspect, the level control circuit may increase or decrease the signal level of one of the pair of pixel signals using the gain, and may apply a predetermined offset voltage to the other of the pair of pixel signals, thereby suppressing erroneous edge detection.

[0016] In addition, in the first aspect, the pixel array unit may further include a pair of shared blocks, each of which includes a plurality of pixels that share a floating diffusion layer, and each of the first, second, third, and fourth pixel pairs may include an addition pixel obtained by adding together the plurality of pixels arranged in one of the pair of shared blocks and an addition pixel obtained by adding together the plurality of pixels arranged in the other of the pair of shared blocks. This provides the effect of improving edge detection accuracy even in a low-illumination environment.

[0017] In addition, in this first aspect, the arrangement of pixels in the pixel array unit may be a Bayer arrangement, and each of the pair of shared blocks may include an R pixel that receives red light, a Gr pixel and a Gb pixel that receive green light, and a B pixel that receives blue light, and the sum pixel may be a pixel obtained by adding the R pixel, at least one of the Gr pixel and the Gb pixel, and the B pixel. This brings about the effect of detecting edges in the Bayer arrangement.

[0018] In this first aspect, the pixel array unit may have a quad-bayer arrangement, four pixels of the same color are arranged in each of the pair of shared blocks, and the sum pixel may be a pixel obtained by adding at least two of the four pixels. This brings about the effect of detecting an edge in the quad-bayer arrangement.

[0019] In addition, in this first aspect, the image sensor may further include a pixel array unit including the first, second, third, and fourth pixel pairs, and a signal processing unit that processes a detection result of the edge detection circuit, wherein the pixel array unit and the edge detection circuit are disposed on a predetermined pixel chip, and the signal processing unit is disposed on a predetermined circuit chip. This brings about the effect of facilitating miniaturization.

[0020] In addition, in the first aspect, the pixel array unit may further include a pixel array unit including the first, second, third, and fourth pixel pairs, the pixel array unit being disposed on a predetermined pixel chip, and the edge detection circuit being disposed on a predetermined circuit chip. This brings about the effect of facilitating miniaturization.

[0021] According to a second aspect of the present technology, there is provided an image sensor including: a pixel array unit in which R pixels that receive red light, Gr pixels that are arranged in a row of the R pixels and receive green light, B pixels that receive blue light, and Gb pixels that are arranged in a row of the B pixels and receive green light, a vertical scanning circuit that sequentially drives two pairs of the Gr pixels and two pairs of the Gb pixels to output pixel signals, and an edge detection circuit that detects the presence or absence of an edge based on the pixel signals of each of the pairs, thereby improving the edge detection accuracy.

[0022] 1 is a block diagram showing an example of a configuration of an imaging device according to a first embodiment of the present technology. FIG. 2 is a block diagram showing an example of a configuration of an image sensor according to the first embodiment of the present technology. FIG. 3 is a circuit diagram showing an example of a configuration of a pixel according to the first embodiment of the present technology. FIG. 4 is an example of a cross-sectional view of a back-illuminated image sensor according to the first embodiment of the present technology. FIG. 5 is an example of a cross-sectional view of a front-illuminated image sensor according to the first embodiment of the present technology. FIG. 6 is a block diagram showing an example of a configuration of a column signal processing circuit according to the first embodiment of the present technology. FIG. 7 is a block diagram showing an example of a configuration of a load MOS current source circuit and a column ADC according to the first embodiment of the present technology. FIG. 8 is a block diagram showing an example of a configuration of an edge detection circuit according to the first embodiment of the present technology. FIG. 9 is a circuit diagram showing an example of a configuration of a gain circuit and a gain offset circuit according to the first embodiment of the present technology. FIG. 10 is a diagram showing an example of an operation of an edge determination circuit according to the first embodiment of the present technology. FIG. 11 is a diagram for explaining a method of determining an edge according to the first embodiment of the present technology. FIG. 12 is a diagram showing an example of a method of driving a first row according to the first embodiment of the present technology. FIG. 13 is a diagram showing an example of a method of driving a second row according to the first embodiment of the present technology. FIG. 14 is a flowchart showing an example of an operation of an image sensor according to the first embodiment of the present technology. FIG. 10 is a timing chart showing an example of the operation of an image sensor in a modified example of the first embodiment of the present technology. FIG. 11 is a diagram showing an example of a pixel array unit in a second embodiment of the present technology. FIG. 12 is a circuit diagram showing an example of a configuration of an FD sharing block in the second embodiment of the present technology. FIG. 13 is a diagram showing an example of pixel pairs in a second window in the second embodiment of the present technology. FIG. 14 is a diagram showing an example of a pixel array unit in a Quad Bayer arrangement in the second embodiment of the present technology. FIG. 15 is a diagram showing an example of a pixel array unit in which phase difference pixel pairs are arranged in the second embodiment of the present technology. FIG. 16 is a diagram showing an example of a window in a third embodiment of the present technology. FIG. 17 is a diagram showing an example of a stacked structure of an image sensor in a fourth embodiment of the present technology. FIG. 18 is a diagram showing another example of the stacked structure of an image sensor in the fourth embodiment of the present technology. FIG. 19 is a block diagram showing an example of a schematic configuration of a vehicle control system.FIG. 2 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit.

[0023] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be given in the following order: 1. First embodiment (an example of driving two windows of different sizes) 2. Second embodiment (an example of driving two windows of different sizes while adding pixels) 3. Third embodiment (an example of driving two windows of the same size but different patterns) 4. Fourth embodiment (an example of driving two windows of different sizes in a stacked structure) 5. Example of application to a moving body

[0024] 1 is a block diagram showing an example of the configuration of an imaging device 100 according to a first embodiment of the present technology. The imaging device 100 is a device for capturing image data, and includes an optical unit 110, an image sensor 200, and a DSP (Digital Signal Processing) circuit 120. The imaging device 100 further includes a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. Examples of the imaging device 100 include digital cameras such as digital still cameras, as well as smartphones, personal computers, and in-vehicle cameras that have an imaging function.

[0025] The optical unit 110 collects light from a subject and guides it to the image sensor 200. The image sensor 200 generates image data by photoelectric conversion in synchronization with a vertical synchronization signal. The vertical synchronization signal is a periodic signal with a predetermined frequency that indicates the timing of imaging. The image sensor 200 supplies the generated image data to the DSP circuit 120 via a signal line 209.

[0026] The DSP circuit 120 performs predetermined signal processing on the image data from the image sensor 200. The DSP circuit 120 outputs the processed image data to a frame memory 160 or the like via a bus 150.

[0027] The display unit 130 displays image data. For example, a liquid crystal panel or an organic EL (Electro Luminescence) panel is assumed as the display unit 130. The operation unit 140 generates an operation signal in accordance with a user's operation.

[0028] The bus 150 is a common path for the optical unit 110, image sensor 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, and power supply unit 180 to exchange data with one another.

[0029] The frame memory 160 holds image data. The storage unit 170 stores various data such as image data. The power supply unit 180 supplies power to the image sensor 200, the DSP circuit 120, the display unit 130, and the like.

[0030] 2 is a block diagram showing an example of the configuration of an image sensor 200 according to the first embodiment of the present technology. The image sensor 200 includes a vertical scanning circuit 210, a digital-to-analog converter (DAC) 220, a timing control circuit 230, and a pixel array unit 240. The image sensor 200 further includes a column signal processing circuit 300, a horizontal scanning circuit 280, a signal processing unit 290, and a memory 295. These circuits are arranged on a single semiconductor chip, for example.

[0031] In the pixel array section 240, a plurality of pixels 250 are arranged in a two-dimensional lattice pattern.

[0032] The timing control circuit 230 controls the operation timing of the vertical scanning circuit 210, the column signal processing circuit 300, etc. in synchronization with the vertical synchronization signal.

[0033] The timing control circuit 230 also receives a mode signal MODE that indicates the operation mode of the image sensor 200. The operation modes include an edge detection mode that detects the presence or absence of edges, and an imaging mode that only captures image data without detecting the presence or absence of edges. The timing control circuit 230 controls each circuit based on the operation mode. Note that the image sensor 200 can also capture image data while detecting the presence or absence of edges.

[0034] The vertical scanning circuit 210 sequentially selects and drives rows to output analog pixel signals. The DAC 220 generates a reference signal by DA (Digital to Analog) conversion and supplies it to the column signal processing circuit 300. For example, a sawtooth ramp signal is used as the reference signal. The pixels 250 generate pixel signals by photoelectric conversion under the control of the vertical scanning circuit 210. Each pixel 250 outputs a pixel signal to the column signal processing circuit 300.

[0035] The column signal processing circuit 300 performs signal processing such as CDS (Correlated Double Sampling) processing and AD (Analog to Digital) conversion processing on pixel signals for each column. The column signal processing circuit 300 supplies image data consisting of digital signals after signal processing to the signal processing unit 290 under the control of the horizontal scanning circuit 280.

[0036] The horizontal scanning circuit 280 selects the columns in order and causes the column signal processing circuit 300 to output digital signals in order.

[0037] The signal processing unit 290 performs various signal processing on image data as necessary. In imaging mode, the signal processing unit 290 performs image processing such as defect correction and supplies the processed image data to the DSP circuit 120. In edge detection mode, the signal processing unit 290 acquires edge determination results for each pixel and performs processing such as feature extraction. Furthermore, the signal processing unit 290 can also perform object detection and recognition such as person detection, face detection, and face recognition based on the extracted features. The signal processing unit 290 then supplies the processed data to the DSP circuit 120.

[0038] The memory 295 is used by the signal processing unit 290 to temporarily store data.

[0039] 3 is a circuit diagram showing a configuration example of a pixel 250 according to the first embodiment of the present technology. In the pixel array section 240, nine lines, namely, a reset line 219-R, transfer lines 219-Ta, 219-Tb, 219-Tc, and 219-Td, and selection lines 219-Sa, 219-Sb, 219-Sc, and 219-Sd, are wired in the horizontal direction for each row.

[0040] The reset line 219-R transmits a reset signal RST from the vertical scanning circuit 210. The transfer lines 219-Ta, 219-Tb, 219-Tc, and 219-Td transmit transfer signals TRGa, TRGb, TRGc, and TRGd from the vertical scanning circuit 210. The selection lines 219-Sa, 219-Sb, 219-Sc, and 219-Sd transmit selection signals SELa, SELb, SELc, and SELd from the vertical scanning circuit 210.

[0041] The pixels 250 in the 4nth column (n is an integer equal to or greater than 0) are connected to a reset line 219-R, a transfer line 219-Ta, and a selection line 219-Sa. The pixels 250 in the 4n+1th column are connected to a reset line 219-R, a transfer line 219-Tb, and a selection line 219-Sb. The pixels 250 in the 4n+2th column are connected to a reset line 219-R, a transfer line 219-Tc, and a selection line 219-Sc. The pixels 250 in the 4n+3th column are connected to a reset line 219-R, a transfer line 219-Td, and a selection line 219-Sd.

[0042] The pixel 250 includes a photoelectric conversion element 251, a transfer transistor 252, a reset transistor 253, a floating diffusion layer 254, an amplification transistor 255, and a selection transistor 256. The pixel 250 illustrated in the figure is connected to a transfer line 219-Ta and a selection line 219-Sa.

[0043] For example, nMOS (n-channel Metal Oxide Semiconductor) transistors are used as various transistors in the pixel 250, such as the transfer transistor 252.

[0044] The photoelectric conversion element 251 generates electric charges by photoelectric conversion. The transfer transistor 252 transfers the electric charges from the photoelectric conversion element 251 to the floating diffusion layer 254 in accordance with a transfer signal TRGa from the vertical scanning circuit 210.

[0045] The reset transistor 253 extracts electric charges from at least one of the photoelectric conversion element 251 and the floating diffusion layer 254 in accordance with a reset signal RST from the vertical scanning circuit 210 to initialize the same.

[0046] Furthermore, a vertical signal line VSL is wired for each column within the pixel array section 240. The vertical signal line VSL for the n-th column will hereinafter be referred to as VSLn.

[0047] The floating diffusion layer 254 accumulates electric charges and converts the amount of electric charges into a voltage. The amplification transistor 255 forms a source follower circuit and outputs a voltage corresponding to the voltage of the floating diffusion layer 254 to the selection transistor 256. The selection transistor 256 supplies a pixel signal of a voltage from the amplification transistor 255 to a vertical signal line VSL in accordance with a selection signal SELa from the vertical scanning circuit 210.

[0048] 4 is an example of a cross-sectional view of the image sensor 200 according to the first embodiment of the present technology. The image sensor 200 includes a substrate 430 and a wiring layer 420. An on-chip lens 411, a color filter 412, and a photoelectric conversion element 251 are arranged for each pixel. With the light incident side facing up, the color filter 412 is arranged below the on-chip lens 411, and the photoelectric conversion element 251 is arranged below the color filter 412.

[0049] Furthermore, a wiring layer 420 is formed on the upper surface of the substrate 430, and transistors such as the transfer transistor 252 described above are formed in the wiring layer 420. A photoelectric conversion element 251 of each pixel is disposed on top of the wiring layer 420. The structure illustrated in the figure is called a back-illuminated type.

[0050] 5, a front-side illumination type may be used instead of the back-side illumination type. In this front-side illumination type, a wiring layer 420 is formed below the color filter 412, and a photoelectric conversion element 251 is disposed below that.

[0051] 6 is a block diagram showing an example of the configuration of a column signal processing circuit 300 according to the first embodiment of the present technology. The column signal processing circuit 300 includes a load MOS current source circuit 310, a column ADC (Analog to Digital Converter) 320, and an edge detection circuit 330. Details of each circuit will be described later.

[0052] [Configuration Example of Column ADC] FIG. 7 is a block diagram showing a configuration example of the load MOS current source circuit 310 and the column ADC 320 according to the first embodiment of the present technology.

[0053] A load MOS current source 311 is arranged for each vertical signal line VSL in the load MOS current source circuit 310. The load MOS current source 311 supplies a load MOS current to the corresponding vertical signal line VSL.

[0054] An ADC 321 is arranged for each vertical signal line VSL in the column ADC 320. The ADC 321 is connected to the corresponding vertical signal line VSL and the DAC 220, and receives the pixel signal ΔVSL from the pixel and the ramp signal RMP from the DAC 220 as input.

[0055] The ADC 321 converts analog pixel signals of the corresponding columns into digital signals under the control of the timing control circuit 230. The ADC 321 supplies the digital signals to the DSP circuit 120 under the control of the horizontal scanning circuit 280. For example, a single-slope ADC including a comparator and a counter is used as the ADC 321. Note that a successive approximation ADC including a comparator, a successive approximation logic circuit, and a register can also be used.

[0056] 8 is a block diagram showing an example of the configuration of the edge detection circuit 330 according to the first embodiment of the present technology. Here, in the edge detection mode, the vertical scanning circuit 210 drives two pixel pairs within a window to output pixel signals. This window is a rectangular area including two pixel pairs that are targets for edge detection, and each pixel within the pixel pair is arranged along the periphery of the window.

[0057] Furthermore, the line segment connecting one of the two pixel pairs intersects with the line segment connecting the other. In the figure, the line segments with arrows at both ends are line segments connecting pixel pairs. Furthermore, it is preferable that the angle between these line segments and the line (row or column) on which the pixels are arranged is not 0 degrees, 45 degrees, or 90 degrees. By setting the angle between the line segment and the line to a value that is not 0 degrees, 45 degrees, or 90 degrees, it is possible to prevent two pixels to be detected from being connected to a single vertical signal line VSL.

[0058] The window size is, for example, 4 rows x 4 columns. To satisfy the condition of the angle between the line and the line segment, for example, a pixel pair consisting of the pixel at coordinate (1, 0) and the pixel at coordinate (2, 3) and a pixel pair consisting of the pixel at coordinate (3, 1) and the pixel at coordinate (0, 2) are driven as targets for edge detection.

[0059] The vertical scanning circuit 210 simultaneously drives multiple windows arranged in the row direction. The distance between the representative positions (such as the upper left pixel) of two adjacent windows is called the "stride." In the figure, the area surrounded by a thick line corresponds to a window, and the area surrounded by a dashed line corresponds to a window adjacent to the thick-lined window. As illustrated in the figure, the stride is four pixels.

[0060] The vertical scanning circuit 210 also sets and drives two windows of different sizes in sequence. Only one of the two windows is shown in the figure. An example of the other window will be described later.

[0061] The edge detection circuit 330 includes an edge detection unit 340 for each pixel pair, which detects the edge of that pixel pair. Each edge detection unit 340 is connected to a pair of vertical signal lines VSL corresponding to the pixel pair to be detected. For example, an edge detection unit 340 that detects a pixel pair consisting of a pixel at coordinates (1, 0) and a pixel at coordinates (2, 3) is connected to vertical signal lines VSL0 and VSL3. An edge detection unit 340 that detects a pixel pair consisting of a pixel at coordinates (3, 1) and a pixel at coordinates (0, 2) is connected to vertical signal lines VSL1 and VSL2. These edge detection units 340 output edge detection results EG to the signal processing unit 290.

[0062] 9 is a block diagram showing an example of the configuration of the edge detection unit 340 according to the first embodiment of the present technology. The edge detection unit 340 includes a level control circuit 350, a comparison circuit 380, and an edge determination circuit 390. The diagram shows circuits connected to vertical signal lines VSL1 and VSL2.

[0063] Here, the voltage of the vertical signal line VSL when the pixel 250 is initialized is called a reset level or P phase. The voltage of the vertical signal line VSL when the charge in the pixel 250 is transferred is called a signal level or D phase. The difference between the P phase and the D phase corresponds to a net pixel signal ΔVSL excluding the noise component at the time of initialization.

[0064] The level control circuit 350 includes gain circuits 360-1 and 360-2, and gain / offset circuits 370-1 and 370-2. The comparison circuit 380 includes comparators 381 and 382.

[0065] The gain circuit 360-1 is connected to the vertical signal line VSL2. The gain circuit 360-1 applies a predetermined gain g １ The signal is increased or decreased by the input voltage Vout and supplied to the inverting input terminal (-) of the comparator 381.

[0066] The gain circuit 360-2 is connected to the vertical signal line VSL1. The gain circuit 360-2 converts the pixel signal ΔVSL1 into a signal g １ Gain g different from ２ The signal is increased or decreased by the value g and supplied to the inverting input terminal (-) of the comparator 382. １ and g ２ is a real number.

[0067] The gain / offset circuit 370-1 is connected to the vertical signal line VSL1. The gain circuit 370-1 applies a gain g １ The voltage is increased or decreased by , and a predetermined offset voltage C is applied as needed, and supplied to the non-inverting input terminal (+) of the comparator 381.

[0068] The gain / offset circuit 370-2 is connected to the vertical signal line VSL2. The gain circuit 370-2 applies a gain g ２ The voltage is increased or decreased by , and an offset voltage C is applied as needed, and supplied to the non-inverting input terminal (+) of the comparator 382.

[0069] g １ / g ２is the relative gain G, the above-described configuration allows the level control circuit 350 to increase or decrease one of the pair of pixel signals ΔVSL1 and ΔVSL2 using the relative gain G, and to apply an offset voltage C to the other pixel signal as needed. Specifically, the level control circuit 350 increases or decreases the pixel signal ΔVSL2 using the relative gain G, and applies the offset voltage C to the pixel signal ΔVSL1 as needed, and supplies the pixel signal ΔVSL1 to the comparator 381. The level control circuit 350 also increases or decreases the pixel signal ΔVSL1 using the relative gain G, and applies the offset voltage C to the pixel signal ΔVSL2 as needed, and supplies the pixel signal ΔVSL2 to the comparator 382.

[0070] The comparator 381 compares the pair of pixel signals after the increase or decrease, and supplies the comparison result CMP1 to the edge determination circuit 390. The comparator 382 compares the pair of pixel signals after the increase or decrease, and supplies the comparison result CMP2 to the edge determination circuit 390.

[0071] The edge determination circuit 390 determines whether or not an edge exists based on the comparison results CMP1 and CMP2, and outputs the determination result to the signal processing unit 290 as an edge detection result EG.

[0072] Although the level control circuit 350 performs both the increase / decrease by gain and the application of an offset voltage, it is also possible to perform only the increase / decrease by gain without applying an offset voltage.

[0073] 10 is a circuit diagram showing an example configuration of a gain circuit 360-1 and a gain / offset circuit 370-1 according to the first embodiment of the present technology. The gain circuit 360-1 includes selectors 361, 362, 363, and 364, and capacitors 365, 366, 367, and 368. The gain / offset circuit 370-1 includes selectors 371, 372, 373, and 374, and capacitors 375, 376, 377, and 378. The circuit configurations of the gain circuit 360-2 and the gain / offset circuit 370-2 are similar to those of the gain circuit 360-1 and the gain / offset circuit 370-1.

[0074] The gain circuit 360-1 receives a 4-bit control signal diff2 from the timing control circuit 230. Within the gain circuit 360-1, a selector 361 switches the connection destination of one end of a capacitor 365 between a ground terminal and a vertical signal line VSL2 in accordance with the first bit of the control signal diff2. A selector 362 switches the connection destination of one end of a capacitor 366 between a ground terminal and a vertical signal line VSL2 in accordance with the second bit of the control signal diff2.

[0075] The selector 363 switches the connection destination of one end of the capacitor 367 between the ground terminal and the vertical signal line VSL2 in accordance with the third bit of the control signal diff2. The selector 364 switches the connection destination of one end of the capacitor 368 between the ground terminal and the vertical signal line VSL2 in accordance with the fourth bit of the control signal diff2.

[0076] The other ends of the capacitors 365 to 368 are commonly connected to the inverting input terminal (−) of a comparator 381 .

[0077] The gain / offset circuit 370-1 receives a 4-bit control signal diff1 and an offset voltage C from the timing control circuit 230. Within the gain / offset circuit 370-1, a selector 371 switches the connection destination of one end of a capacitor 375 between the offset voltage C and the vertical signal line VSL1 in accordance with the first bit of the control signal diff1. A selector 372 switches the connection destination of one end of a capacitor 376 between the ground terminal and the vertical signal line VSL1 in accordance with the second bit of the control signal diff1.

[0078] The selector 373 switches the connection destination of one end of the capacitor 377 between the ground terminal and the vertical signal line VSL1 in accordance with the third bit of the control signal diff1. The selector 374 switches the connection destination of one end of the capacitor 378 between the ground terminal and the vertical signal line VSL1 in accordance with the fourth bit of the control signal diff1.

[0079] The other ends of the capacitors 375 to 378 are commonly connected to the non-inverting input terminal (+) of a comparator 381 .

[0080] With the circuit configuration illustrated in the figure, the timing control circuit 230 controls the gain g １ The timing control circuit 230 controls whether or not to apply the offset voltage C by the control signal diff2, and also controls the gain g ２ can be controlled in eight stages.

[0081] 11 is a diagram illustrating an example of the operation of the edge determination circuit 390 according to the first embodiment of the present technology. The edge determination circuit 390 determines that an edge exists if the comparison results CMP1 and CMP2 have different values. On the other hand, the edge determination circuit 390 determines that an edge does not exist if the comparison results CMP1 and CMP2 have the same value. For ease of explanation, it is assumed in the diagram that the offset voltage C is not applied.

[0082] For example, it is assumed that a low-level comparison result CMP1 is output when the following equation is true: ΔVSL1<ΔVSL2*G Equation 1 In the above equation, "*" indicates multiplication.

[0083] On the other hand, it is assumed that a high-level comparison result CMP1 is output when the following equation is satisfied: ΔVSL1>ΔVSL2*G (Equation 2)

[0084] Also, it is assumed that a high-level comparison result CMP2 is output when the following equation is satisfied: ΔVSL2>ΔVSL1*G (Equation 3)

[0085] On the other hand, it is assumed that a low-level comparison result CMP2 is output when the following equation is satisfied: ΔVSL2≦ΔVSL1*G (Equation 4)

[0086] When both Equation 1 and Equation 3 are true, the following equation is true: ΔVSL1<<ΔVSL2 Equation 5 In the above equation, "<<" indicates the magnitude relationship of the degree to which an edge is determined to exist.

[0087] On the other hand, when both Equation 2 and Equation 4 are true, the following equation is true: ΔVSL1>>ΔVSL2 Equation 6 In the above equation, “>>” indicates the magnitude relationship of the degree to which an edge is determined to exist.

[0088] When the comparison results CMP1 and CMP2 are different values, either of Equation 5 or Equation 6 holds. These Equations 5 and 6 indicate that the ratio of one of the pixel signals ΔVSL1 and ΔVSL2 to the other is equal to or greater than a certain value. In this case, it is determined that an edge exists.

[0089] On the other hand, if the comparison results CMP1 and CMP2 are the same value, neither Equation 5 nor Equation 6 holds. In this case, it is determined that there is no edge. Note that, when G is less than "1", the calculations show that both the comparison results CMP1 and CMP2 are never "0". Also, when G is "1" or greater, the calculations show that both the comparison results CMP1 and CMP2 are never "1". However, because these cases may occur due to the influence of noise, edge determination results are set even for cases where the calculations show that the results do not hold.

[0090] Next, the advantages of the control in which one of the pair of pixel signals ΔVSL1 and ΔVSL2 is increased or decreased by the relative gain G and compared will be described.

[0091] The pixel that generates the pixel signal ΔVSL1 receives light that is ambient light reflected from the surface of object A with reflectance Ra, and the pixel that generates the pixel signal ΔVSL2 receives light that is ambient light reflected from the surface of object B with reflectance Rb. The amount of ambient light is AL, and the relative gain G is less than 1.

[0092] Here, a first comparative example is considered in which the difference between pixel signals ΔVSL1 and ΔVSL2 is compared with a threshold value Th to detect the presence or absence of an edge. In this first comparative example, it is determined that an edge exists when the following formula is satisfied: ΔVSL1-ΔVSL2=Ra*AL-Rb*AL>Th (Formula 7)

[0093] Equation 7 can be transformed into the following equation: (Ra-Rb)>Th / AL Equation 8

[0094] As exemplified by Equation 7 and Equation 8, in the comparative example, the edge determination result may change depending on the amount of ambient light AL. This reduces the accuracy of edge detection. For example, the smaller the amount of ambient light AL, the smaller the difference, making edge detection more difficult. Changing the threshold value depending on the amount of ambient light can suppress the reduction in detection accuracy, but it is difficult to accurately measure the amount of light and fine-tune the threshold value accordingly.

[0095] Also, consider a second comparative example in which the difference between the logarithmic values ​​of the pixel signals ΔVSL1 and ΔVSL2 is compared with a threshold value Th to detect the presence or absence of an edge. In this second comparative example, it is determined that an edge exists when the following equation is satisfied: log(ΔVSL1) - log(ΔVSL2) = log(Ra * AL / Rb * AL) = log(Ra / Rb) > Th (Equation 9)

[0096] As shown in Equation 9, the second comparative example can suppress the influence of ambient light, but requires logarithmic calculations and is difficult to implement.

[0097] In contrast to these comparative examples, by introducing a method of determining a threshold value based on a signal rather than based on ambient light, the influence of ambient light can be easily suppressed.

[0098] It is assumed that the magnitude relationship between the pixel signals after attenuation remains unchanged whether only the pixel signal ΔVSL2 is attenuated by the relative gain G or whether only the pixel signal ΔVSL1 is attenuated by the relative gain G. In this case, the comparison results CMP1 and CMP2 will have different values ​​due to the connection relationship of the input terminals of the comparators 381 and 382. In this case, the following equation holds: |Ra*AL-Rb*AL|>G*max(Ra, Rb)*AL ...Equation 10 In the above equation, max() is a function that returns the maximum value of the multiple input values ​​in (). Note that if the relative gain G is greater than 1, the function min(), which returns the minimum value, is used instead of max().

[0099] When Ra is greater than Rb, Equation 10 can be transformed into the following equation: (Ra-Rb)*AL>G*Ra*AL Equation 11

[0100] Equation 11 can be transformed into the following equation: (1-G)*Ra*AL-Rb*AL>0 Equation 12

[0101] Furthermore, if 1-G is g, then Equation 12 can be transformed into the following equation: g*(Ra*AL)-(Rb*AL)>0 Equation 13

[0102] Dividing both sides of Equation 13 by the amount of light AL gives the following equation: g*Ra-Rb>0 Equation 14

[0103] That is, when the comparison results CMP1 and CMP2 are different values, Equation 14 holds. Equation 14 indicates that the ratio between pixel signals ΔVSL1 and ΔVSL2 is equal to or greater than a certain value (g in the above equation). In other words, the comparison results CMP0 and CMP1 are values ​​indicating that the ratio between pixel signals ΔVSL1 and ΔVSL2 is equal to or greater than a certain value. Because Equation 14 does not include a term that includes the light amount AL, the image sensor 200 can determine the presence or absence of an edge without depending on the light amount AL. This allows for improved edge detection accuracy compared to the first comparative example.

[0104] When the illuminance is low and ΔVSL1 and ΔVSL2 are both low, noise can cause the comparison results CMP0 and CMP2 to have different values, resulting in an erroneous edge detection. The edge detection unit 340 can suppress such erroneous edge detection by applying the above-mentioned offset voltage C.

[0105] 12 is a diagram for explaining an edge determination method according to the first embodiment of the present technology. In the diagram, the vertical axis represents the pixel signal ΔVSL2, and the horizontal axis represents the pixel signal ΔVSL1. The thick line represents a threshold value for determining the edge.

[0106] Taking the offset voltage C into consideration, the edge detection unit 340 detects an edge when the following determination formula is satisfied: |ΔVSL1−ΔVSL2|>G*max(ΔVSL1, ΔVSL2)+C (Formula 15)

[0107] If 1-G is g, the following equation can be obtained from equation 15: |ΔVSL1-ΔVSL2|>(1-g)*max(ΔVSL1, ΔVSL2)+C ...Equation 16

[0108] The gray area in the figure indicates the area where Equation 16 holds and edges are detected.

[0109] [Example of Image Sensor Operation] Next, operation in edge detection mode will be described with reference to Figures 13 and 14. In edge detection mode, the vertical scanning circuit 210 sequentially drives two windows of different sizes and patterns. One of these windows is referred to as the "first window" and the other is referred to as the "second window." The charge accumulation periods of these windows are assumed to be the same.

[0110] In Figure 13, "a" shows an example of the first row of the first window driven by the vertical scanning circuit 210. The size of the first window is, for example, 8 rows x 8 columns, with a stripe of 4 pixels. The bold frame corresponds to the leftmost first window, and the dashed dotted line indicates the first window adjacent to the bold-framed first window. The leftmost column address of the leftmost first window is defined as the "read start address." The address indicated by the arrow in the figure indicates the read start address.

[0111] In the first window on the left, for example, a pixel pair consisting of a pixel at coordinates (5,0) and a pixel at coordinates (2,7) and a pixel pair consisting of a pixel at coordinates (7,5) and a pixel at coordinates (0,2) are driven.

[0112] As described above, transfer lines 219-Ta, 219-Tb, 219-Tc, and 219-Td and selection lines 219-Sa, 219-Sb, 219-Sc, and 219-Sd are wired for each row.

[0113] One of each pixel pair is connected to one of selection lines 219-Sa, 219-Sb, 219-Sc, 219-Sd, and 219-Td, and the other of the pixel pair is connected to a selection line different from the selection line. For example, the pixel at coordinates (5, 0) is connected to selection line 219-Sa on the fifth row, and the pixel at coordinates (2, 7) is connected to selection line 219-Sd on the second row. The same applies to the transfer lines.

[0114] The vertical scanning circuit 210 supplies a selection signal SELc to the selection line 219-Sc of row 0 and a selection signal SELd to the selection line 219-Sd of row 2. At the same time, the vertical scanning circuit 210 supplies a selection signal SELa to the selection line 219-Sa of row 5 and a selection signal SELb to the selection line 219-Sb of row 7. The same applies to the transfer lines.

[0115] If there is only one selection line per row, and the vertical scanning circuit 210 simultaneously supplies the selection signal SEL to the selection lines of rows 0, 2, 5, and 7, four pixel signals will flow through one vertical signal line VSL.

[0116] However, in the first embodiment, four selection lines are wired for each row, and one pixel and the other pixel of a pixel pair are connected to different selection lines, so that it is possible to prevent multiple pixel signals from flowing through the vertical signal line VSL when four rows are selected simultaneously.

[0117] It is also possible to wire two selection lines and two transfer lines for each row. In this case, for example, the vertical scanning circuit 210 may halve the cycle for selecting the selection signal compared to when four lines are wired for each row, and drive one of the two pixel pairs first, and then drive the other of the two pixel pairs.

[0118] After the control shown in a in the figure, the vertical scanning circuit 210 drives the first row of the second window as shown in b in the figure. The size of the second window is, for example, 4 rows x 4 columns, with a stripe of 4 pixels. The bold frame corresponds to the leftmost second window, and the dashed dotted line indicates the second window adjacent to the bold-framed second window. The leftmost column address of the leftmost second window corresponds to the read start address.

[0119] In the first window on the left, for example, a pixel pair consisting of the pixel at coordinate (1, 2) and the pixel at coordinate (2, 5) and a pixel pair consisting of the pixel at coordinate (3, 3) and the pixel at coordinate (0, 4) are driven.

[0120] As shown in a and b in the figure, the first window and the second window are different in size, i.e., the inter-pixel distance between a pixel pair in the first window is different from the inter-pixel distance between a pixel pair in the second window.

[0121] As shown in Fig. 1A and 1B, the vertical scanning circuit 210 drives two pixel pairs in a first window, and then drives two pixel pairs in a second window having a different inter-pixel distance to output pixel signals. The edge detection circuit 330 detects the presence or absence of an edge for each pixel pair based on the pixel signals.

[0122] The appropriate window size and pixel pair pattern differ depending on the distance to the subject, the angle between the direction to the subject and the optical axis, the contrast of the subject, etc. For this reason, while a configuration with a fixed window size and pixel pair pattern may be able to detect the edge of a certain subject, it may be difficult to detect the edge of a subject with different characteristics.

[0123] However, the vertical scanning circuit 210 of the first embodiment sequentially drives two windows of different sizes and patterns as described above, and therefore can detect edges even when subjects with different characteristics are mixed, thereby improving detection accuracy.

[0124] The two pixel pairs in the first window are an example of the first and second pixel pairs set forth in the claims, and the two pixel pairs in the second window are an example of the third and fourth pixel pairs set forth in the claims.

[0125] Also, as mentioned above, it is preferable that the read start address of the first window is different from the read start address of the second window.

[0126] As shown in a and b in the figure, it is preferable that the pattern that is line-symmetric to the pattern of pixel pairs in the first window is similar to the pattern of pixel pairs in the second window.

[0127] Next, the vertical scanning circuit 210 drives the second row of the first window, as illustrated in Fig. 14a, where the stride between the first and second rows is, for example, four pixels.

[0128] Then, as shown in FIG. 1B, the vertical scanning circuit 210 drives the second row of the second window. The stride between the first and second rows is, for example, four pixels. Similarly, the first and second windows are alternately driven for the third and subsequent rows.

[0129] 15 is a flowchart showing an example of the operation of the image sensor 200 according to the first embodiment of the present technology. This operation is started, for example, when the image sensor 200 is powered on.

[0130] The image sensor 200 determines whether the operation mode is set to the edge detection mode (step S901). If the operation mode is set to the edge detection mode (step S901: Yes), the vertical scanning circuit 210 in the image sensor 200 selects and drives a row of the first window (step S902), and the edge detection circuit 330 detects the presence or absence of an edge for each pixel pair in the row (step S903).

[0131] Then, the vertical scanning circuit 210 selects and drives a row in the second window (step S904), and the edge detection circuit 330 detects the presence or absence of an edge for each pixel pair in that row (step S905).

[0132] The vertical scanning circuit 210 determines whether all rows have been driven (step S906). If not all rows have been driven (step S906: No), the image sensor 200 changes the row to be driven and repeats step S902 and subsequent steps. On the other hand, if all rows have been driven (step S906: Yes), the image sensor 200 repeats step S901 and subsequent steps.

[0133] If the edge detection mode is not set (step S901: No), the image sensor 200 captures image data (step S907) and repeatedly executes step S901 and subsequent steps.

[0134] As described above, according to the first embodiment of the present technology, the vertical scanning circuit 210 drives two sets of pixel pairs in the first window, and then drives two sets of pixel pairs in the second window having a different inter-pixel distance from the first set of pixel pairs, thereby improving detection accuracy.

[0135] In the first embodiment described above, the charge accumulation periods of the first and second windows are the same, but the charge accumulation periods can be controlled to be different values. Image sensor 200 in this modification of the first embodiment differs from the first embodiment in that the charge accumulation periods of the first and second windows are different.

[0136] 16 is a timing chart showing an example of the operation of the image sensor according to the modification of the first embodiment of the present technology, in which the vertical axis indicates the row address and the horizontal axis indicates time.

[0137] In the modification of the first embodiment, the vertical scanning circuit 210 transmits a reset signal RST and a transfer signal TRG to the pixel pair in the first window at timing T1, causing them to start accumulating charges.

[0138] Then, at timing T2, the vertical scanning circuit 210 transmits a reset signal RST and a transfer signal TRG to the pixel pairs in the second window, causing them to start accumulating charges.

[0139] Then, the vertical scanning circuit 210 supplies a selection signal SEL at timing T3 to start reading out the detection results of the first and second windows.

[0140] In the example shown in the figure, the period from timing T1 to T3 corresponds to the charge accumulation period of the first window, and the period from timing T2 to T3 corresponds to the charge accumulation period of the second window.

[0141] As shown in the figure, by setting the charge accumulation period to a different value for each window, even when multiple subjects with different brightness levels are mixed, their edges can be properly detected, further improving detection accuracy.

[0142] As described above, according to the modification of the first embodiment of the present technology, the charge accumulation period of the first window is different from that of the second window, so that it is possible to further improve the accuracy of edge detection.

[0143] 2. Second Embodiment In the first embodiment described above, the vertical scanning circuit 210 sequentially drives the first window and the second window, which have different inter-pixel distances, but each pixel in a pixel pair within a window may be obtained by pixel addition of two or more pixels. The image sensor 200 in this second embodiment differs from the first embodiment in that each pixel in a pixel pair is obtained by pixel addition.

[0144] 17 is a diagram showing an example of a pixel array unit 240 according to a second embodiment of the present technology. In the second embodiment, the pixel array unit 240 has R (Red) pixels, G pixels, and B (Blue) pixels arranged in a Bayer array. The R pixels receive red visible light, the G pixels receive green visible light, and the B pixels receive blue visible light. The G pixels include Gr pixels arranged in rows of R pixels and Gb pixels arranged in rows of B pixels.

[0145] The pixel array unit 240 is divided into a plurality of FD shared blocks, such as FD shared blocks 260-1, 260-2, 260-3, and 260-4. Each FD shared block has an R pixel, a Gr pixel, a Gb pixel, and a B pixel arranged in two rows and two columns. These four pixels share a floating diffusion layer. The black squares in the figure indicate the floating diffusion layer. The FD shared blocks 260-1 and 260-2 are an example of a pair of shared blocks described in the claims.

[0146] 18 is a circuit diagram showing a configuration example of an FD shared block 260-1 according to the second embodiment of the present technology. The FD shared block 260-1 includes photoelectric conversion elements 261, 262, 263, and 264, and transfer transistors 265, 266, 267, and 268. The FD shared block 260-1 further includes a reset transistor 253, a floating diffusion layer 254, an amplification transistor 255, and a selection transistor 256. For example, nMOS transistors are used as the transfer transistors 265, 266, 267, and 268. The circuit configurations of the FD shared blocks 260-2, 260-3, and 260-4 are the same as that of the FD shared block 260-1.

[0147] The transfer transistor 265 transfers charges from the photoelectric conversion element 261 to the floating diffusion layer 254 in accordance with a transfer signal TRGa1 from the vertical scanning circuit 210. The transfer transistor 266 transfers charges from the photoelectric conversion element 262 to the floating diffusion layer 254 in accordance with a transfer signal TRGa2 from the vertical scanning circuit 210. The transfer transistor 267 transfers charges from the photoelectric conversion element 263 to the floating diffusion layer 254 in accordance with a transfer signal TRGa3 from the vertical scanning circuit 210. The transfer transistor 268 transfers charges from the photoelectric conversion element 264 to the floating diffusion layer 254 in accordance with a transfer signal TRGa4 from the vertical scanning circuit 210.

[0148] The connection configuration of the reset transistor 253, the floating diffusion layer 254, the amplification transistor 255, and the selection transistor 256 in the FD shared block 260-1 is the same as that in the first embodiment.

[0149] In addition, in the pixel array unit 240, 16 transfer lines and four selection lines are wired for each row of the FD shared block. Four transfer lines transmitting transfer signals TRGa1, TRGa2, TRGa3, and TRGa4 and a selection line transmitting a selection signal SELa are connected to 4n columns of the FD shared block. Four transfer lines transmitting transfer signals TRGb1, TRGb2, TRGb3, and TRGb-4 and a selection line transmitting a selection signal SELb are connected to 4n+1 columns of the FD shared block. Four transfer lines transmitting transfer signals TRGc1, TRGc2, TRGc3, and TRGc4 and a selection line transmitting a selection signal SELc are connected to 4n+2 columns of the FD shared block. Four transfer lines for transmitting transfer signals TRGd1, TRGd2, TRGd3, and TRGd4 and a selection line for transmitting a selection signal SELd are connected to the 4n+3 columns of the FD sharing block.

[0150] FIG. 19 is a diagram showing an example of pixel pairs in the second window in the second embodiment of the present technology. The vertical scanning circuit 210 simultaneously supplies transfer signals TRGa1, TRGa2, TRGa3, and TRGa4 to the FD sharing block 260-1 at the timing of charge transfer. As a result, the R pixel, Gr pixel, Gb pixel, and B pixel in the FD sharing block 260-1 are added together. Similarly, four pixels are added together in each of the FD sharing blocks 260-2, 260-3, and 260-4. The new pixel generated by pixel addition is referred to as the "added pixel." The gray areas in the figure indicate the added pixels.

[0151] As shown in the figure, pixel addition results in two pairs of pixel summations. The downstream edge detection unit 340 detects the presence or absence of an edge based on the pixel signals of each pixel summation pair. Because the pixel summations are brighter than before addition, edge detection accuracy can be improved even in low-light environments.

[0152] Note that the vertical scanning circuit 210 adds together an R pixel, a Gr pixel, a Gb pixel, and a B pixel for each FD shared block, but it is also possible to add together three pixels: an R pixel, a Gr pixel or a Gb pixel, and a B pixel.

[0153] Although only the second window is shown in the figure, two pairs of added pixels are formed in the first window as well, and an edge is detected for each pair.

[0154] Pixels can also be arranged in a quad-bayer arrangement, as shown in FIG. 20A. In this quad-bayer arrangement, four pixels of the same color are arranged in an FD shared block. For each FD shared block, the vertical scanning circuit 210 adds two, three, or four pixels in that block. Pixels marked with an x ​​in FIG. 20A indicate pixels that are not added.

[0155] In the figure, b is an example of a pair of added pixels when pixel addition is performed in a.

[0156] 21A, a phase difference pixel pair for detecting a phase difference can be arranged in the pixel array unit 240. The phase difference pixel pair is a pair of pixels that share the on-chip lens 411 and the floating diffusion layer, and is used to detect a phase difference. The phase difference pixel pair surrounded by the dotted line in a in the same drawing is subjected to pixel addition.

[0157] In the figure, b is an example of a pair of added pixels when pixel addition is performed in a.

[0158] It should be noted that the modified example of the first embodiment can be applied to the second embodiment.

[0159] As described above, according to the second embodiment of the present technology, the edge detection unit 340 detects the presence or absence of an edge based on the pixel signals of each pair of summed pixels, thereby improving the accuracy of edge detection even in a low-illumination environment.

[0160] 3. Third Embodiment In the first embodiment described above, the vertical scanning circuit 210 drives the first and second windows having different sizes and patterns, but this is not limiting. The image sensor 200 in this third embodiment differs from the first embodiment in that it drives the first and second windows having the same size but different patterns.

[0161] 22 is a diagram showing an example of a window according to the third embodiment of the present technology, in which "a" in the figure shows an example of a first window, and "b" in the figure shows an example of a second window.

[0162] As shown in FIG. 1A, the pixel array unit 240 has R, Gr, Gb, and B pixels arranged in a Bayer pattern. The vertical scanning circuit 210 drives two Gr pixel pairs in a first window of 7 rows and 7 columns. For example, the Gr pixel pair at coordinates (4, 1) and (2, 7) and the Gr pixel pair at coordinates (6, 5) and (0, 3) are driven. The edge detection unit 340 detects the presence or absence of an edge based on the pixel signals of each Gr pixel pair.

[0163] As shown in FIG. 1B, the vertical scanning circuit 210 drives two sets of Gb pixel pairs in a second window of 7 rows and 7 columns, which has a different pattern from that of the first window. For example, the Gb pixel pair at coordinates (3, 0) and (5, 6) and the Gr pixel pair at coordinates (7, 2) and (1, 4) are driven. The edge detection unit 340 detects the presence or absence of an edge based on the pixel signals of each Gb pixel pair.

[0164] As described above, according to the third embodiment of the present technology, the vertical scanning circuit 210 drives two sets of Gr pixel pairs in a first window, and then drives two sets of Gb pixel pairs in a second window having a different pattern, thereby improving detection accuracy.

[0165] 4. Fourth Embodiment In the first embodiment described above, the circuits are arranged on a single semiconductor chip, but this configuration may make it difficult to miniaturize pixels. Image sensor 200 in this fourth embodiment differs from the first embodiment in that the circuits are distributed across multiple stacked semiconductor chips.

[0166] 23 is a diagram showing an example of a stacked structure of an image sensor 200 according to a fourth embodiment of the present technology. The image sensor 200 according to the fourth embodiment includes a circuit chip 202 and a pixel chip 201 stacked on the circuit chip 202. These chips are electrically connected via connecting portions such as vias. Note that, in addition to vias, they can also be connected by Cu-Cu bonding or bumps.

[0167] The pixel chip 201 includes a pixel array unit 240, a north column signal processing circuit 301, and a south column signal processing circuit 302.

[0168] The north column signal processing circuit 301 reads pixel signals and edge determination results from half of the pixels (such as odd-numbered columns) in the pixel array unit 240, and the south column signal processing circuit 302 reads pixel signals and the like from the remaining half. Various peripheral circuits such as a signal processing unit 290 are arranged on the circuit chip 202.

[0169] As shown in FIG. 24, it is also possible to arrange only the pixel array section 240 on the pixel chip 201 and arrange the remaining circuits on the circuit chip 202.

[0170] 23 and 24, miniaturization of pixels is facilitated by distributing circuits among stacked pixel chips 201 and circuit chips 202. It is also possible to stack three or more semiconductor chips and arrange circuits on them.

[0171] The fourth embodiment can be applied to the modified example of the first embodiment and the second and third embodiments.

[0172] In this way, according to the fourth embodiment of the present technology, since the circuits are distributed and arranged on the stacked pixel chip 201 and circuit chip 202, it becomes easy to miniaturize the pixels.

[0173] 5. Application Examples to Mobile Bodies The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0174] FIG. 25 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0175] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 25, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0176] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0177] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0178] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0179] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0180] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0181] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0182] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0183] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0184] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of Fig. 25, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0185] FIG. 26 is a diagram showing an example of the installation position of the imaging unit 12031.

[0186] In FIG. 26, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0187] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0188] 26 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0189] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0190] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which runs autonomously without relying on driver operation.

[0191] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0192] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0193] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 of the above-described configuration. Specifically, the imaging device 100 of FIG. 1 can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, it is possible to increase the accuracy of edge detection and improve the reliability of the vehicle control system.

[0194] Note that the above-described embodiment shows an example for realizing the present technology, and the matters in the embodiment and the matters specifying the invention in the claims correspond to each other. Similarly, the matters specifying the invention in the claims and the matters in the embodiment of the present technology having the same name correspond to each other. However, the present technology is not limited to the embodiment, and can be realized by applying various modifications to the embodiment within the scope of the gist thereof.

[0195] The effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0196] The present technology may also be configured as follows: (1) An image sensor comprising: a vertical scanning circuit that sequentially drives first and second pixel pairs and third and fourth pixel pairs, the inter-pixel distance of which is different from that of the first and second pixel pairs, to output pixel signals; and an edge detection circuit that detects the presence or absence of an edge based on the pixel signals of the first, second, third, and fourth pixel pairs. (2) The image sensor according to (1), in which an angle formed by a line segment connecting a pair of pixels included in each of the first, second, third, and fourth pixel pairs and a line on which the pixels are arranged is not 0 degrees, 45 degrees, or 90 degrees. (3) The image sensor according to (1) or (2), in which one of a pair of pixels included in each of the first, second, third, and fourth pixel pairs is connected to one of first and second selection lines wired for each row, and the other of the pair of pixels is connected to the other of the first and second selection lines. (4) The image sensor according to any one of (1) to (3), wherein one of a pair of pixels included in each of the first, second, third, and fourth pixel pairs is connected to one of first and second transfer lines wired for each row, and the other of the pair of pixels is connected to the other of the first and second transfer lines. (5) The image sensor according to any one of (1) to (4), wherein a line segment connecting the first pixel pair and a line segment connecting the second pixel pair are orthogonal, and a line segment connecting the third pixel pair and a line segment connecting the fourth pixel pair are orthogonal. (6) The image sensor according to any one of (1) to (5), wherein a readout start address of a first window including the first and second pixel pairs is different from a readout start address of a second window including the third and fourth pixel pairs. (7) The image sensor according to any one of (1) to (6), wherein a pattern that is linearly symmetrical to the pattern of the third and fourth pixel pairs is similar to the pattern of the first and second pixel pairs. (8) The image sensor according to any one of (1) to (7), wherein the charge accumulation period of the first and second pixel pairs is different from the charge accumulation period of the third and fourth pixel pairs.(9) The image sensor according to any one of (1) to (8), wherein each of the first, second, third, and fourth pixel pairs outputs a pair of pixel signals, and the edge detection circuit comprises: a level control circuit that increases or decreases a signal level of one of the pair of pixel signals by a predetermined gain, and a comparison circuit that compares the pair of pixel signals whose signal level has been increased or decreased and outputs a comparison result. (10) The image sensor according to (9), wherein the level control circuit increases or decreases the signal level of one of the pair of pixel signals by the gain and applies a predetermined offset voltage to the other of the pair of pixel signals. (11) The image sensor according to any one of (1) to (10), further comprising a pixel array unit including a pair of shared blocks in each of which a plurality of pixels sharing a floating diffusion layer are arranged, and each of the first, second, third, and fourth pixel pairs includes an addition pixel obtained by adding together a plurality of pixels arranged in one of the pair of shared blocks and a addition pixel obtained by adding together a plurality of pixels arranged in the other of the pair of shared blocks. (12) The image sensor according to (11), wherein the arrangement of pixels in the pixel array unit is a Bayer arrangement, and each of the pair of shared blocks includes an R pixel that receives red light, a Gr pixel and a Gb pixel that receive green light, and a B pixel that receives blue light, and the sum pixel is a pixel obtained by adding the R pixel, at least one of the Gr pixel and the Gb pixel, and the B pixel. (13) The image sensor according to (11), wherein the arrangement of pixels in the pixel array unit is a Quad Bayer arrangement, and each of the pair of shared blocks includes four pixels of the same color, and the sum pixel is a pixel obtained by adding at least two of the four pixels. (14) The image sensor according to any one of (1) to (13), further comprising: a pixel array section including the first, second, third, and fourth pixel pairs; and a signal processing section that processes the detection results of the edge detection circuit, wherein the pixel array section and the edge detection circuit are arranged on a predetermined pixel chip, and the signal processing section is arranged on a predetermined circuit chip.(15) The image sensor according to any one of (1) to (13), further comprising a pixel array unit including the first, second, third, and fourth pixel pairs, the pixel array unit being disposed on a predetermined pixel chip, and the edge detection circuit being disposed on a predetermined circuit chip. (16) An image sensor comprising: a pixel array unit in which R pixels that receive red light, Gr pixels that are arranged in a row of the R pixels and receive green light, B pixels that receive blue light, and Gb pixels that are arranged in a row of the B pixels and receive green light, a vertical scanning circuit that sequentially drives two pairs of the Gr pixels and two pairs of the Gb pixels to output pixel signals, and an edge detection circuit that detects the presence or absence of an edge based on the pixel signals of each of the pairs. (17) A control method for an image sensor, comprising: a vertical scanning step of sequentially driving first and second pixel pairs and third and fourth pixel pairs having an inter-pixel distance different from that of the first and second pixel pairs to output pixel signals; and an edge detection step of detecting the presence or absence of an edge based on the pixel signals of the first, second, third, and fourth pixel pairs.

[0197] 100 Imaging device 110 Optical unit 120 DSP circuit 130 Display unit 140 Operation unit 150 Bus 160 Frame memory 170 Storage unit 180 Power supply unit 200 Image sensor 201 Pixel chip 202 Circuit chip 210 Vertical scanning circuit 220 DAC 230 Timing control circuit 240 Pixel array unit 250 Pixel 251, 261 to 264 Photoelectric conversion element 252, 265 to 268 Transfer transistor 253 Reset transistor 254 Floating diffusion layer 255 Amplification transistor 256 Selection transistor 260-1, 260-2, 260-3, 260-4 FD shared block 280 Horizontal scanning circuit 290 Signal processing unit 295 Memory 300 Column signal processing circuit 301 North column signal processing circuit 302 South column signal processing circuit 310 Load MOS current source circuit 311 Load MOS current source 320 Column ADC 321 ADC 330 Edge detection circuit 340 Edge detection unit 350 Level control circuit 360-1, 360-2 Gain circuit 361 to 364, 371 to 374 Selector 365 to 368, 375 to 378 Capacitor 370-1, 370-2 Gain / offset circuit 380 Comparison circuit 381, 382 Comparator 390 Edge determination circuit 411 On-chip lens 412 Color filter 420 Wiring layer 430 Substrate 12031 Imaging unit

Claims

1. An image sensor comprising: a vertical scanning circuit that sequentially drives first and second pixel pairs and third and fourth pixel pairs having a different inter-pixel distance from the first and second pixel pairs to output pixel signals; and an edge detection circuit that detects the presence or absence of an edge based on the pixel signals of each of the first, second, third, and fourth pixel pairs.

2. The image sensor according to claim 1, wherein the angle formed by the line segment connecting a pair of pixels included in each of the first, second, third and fourth pixel pairs and the line on which the pixels are arranged is not 0 degrees, 45 degrees or 90 degrees.

3. The image sensor according to claim 1, wherein one of a pair of pixels included in each of the first, second, third and fourth pixel pairs is connected to one of first and second selection lines wired for each row, and the other of the pair of pixels is connected to the other of the first and second selection lines.

4. The image sensor according to claim 1, wherein one of a pair of pixels included in each of the first, second, third and fourth pixel pairs is connected to one of first and second transfer lines wired for each row, and the other of the pair of pixels is connected to the other of the first and second transfer lines.

5. The image sensor according to claim 1, wherein the line segment connecting the first pixel pair and the line segment connecting the second pixel pair are perpendicular to each other, and the line segment connecting the third pixel pair and the line segment connecting the fourth pixel pair are perpendicular to each other.

6. The image sensor according to claim 1, wherein a readout start address of a first window including said first and second pixel pairs is different from a readout start address of a second window including said third and fourth pixel pairs.

7. The image sensor according to claim 1, wherein the pattern that is line-symmetric to the pattern of the third and fourth pixel pairs is similar to the pattern of the first and second pixel pairs.

8. The image sensor according to claim 1, wherein the charge accumulation periods of the first and second pixel pairs are different from the charge accumulation periods of the third and fourth pixel pairs.

9. The image sensor according to claim 1, wherein each of the first, second, third and fourth pixel pairs outputs a pair of pixel signals, and the edge detection circuit comprises: a level control circuit that increases or decreases the signal level of one of the pair of pixel signals by a predetermined gain; and a comparison circuit that compares the pair of pixel signals whose signal level has been increased or decreased, and outputs the comparison result.

10. The image sensor according to claim 9, wherein the level control circuit increases or decreases the signal level of one of the pair of pixel signals using the gain, and applies a predetermined offset voltage to the other of the pair of pixel signals.

11. The image sensor according to claim 1, further comprising a pixel array section including a pair of shared blocks in each of which a plurality of pixels sharing a floating diffusion layer are arranged, and each of the first, second, third and fourth pixel pairs includes an addition pixel obtained by adding together a plurality of pixels arranged in one of the pair of shared blocks and a addition pixel obtained by adding together a plurality of pixels arranged in the other of the pair of shared blocks.

12. The image sensor of claim 11, wherein the arrangement of pixels in the pixel array section is a Bayer arrangement, and each of the pair of shared blocks is arranged with an R pixel that receives red light, a Gr pixel and a Gb pixel that receive green light, and a B pixel that receives blue light, and the sum pixel is a pixel obtained by adding the R pixel, at least one of the Gr pixel and the Gb pixel, and the B pixel.

13. The image sensor according to claim 11, wherein the arrangement of pixels in the pixel array section is a Quad Bayer arrangement, four pixels of the same color are arranged in each of the pair of shared blocks, and the sum pixel is a pixel obtained by adding together at least two of the four pixels.

14. The image sensor according to claim 1, further comprising: a pixel array section including the first, second, third, and fourth pixel pairs; and a signal processing section that processes the detection results of the edge detection circuit, wherein the pixel array section and the edge detection circuit are disposed on a predetermined pixel chip, and the signal processing section is disposed on a predetermined circuit chip.

15. The image sensor according to claim 1, further comprising a pixel array section including the first, second, third and fourth pixel pairs, the pixel array section being disposed on a predetermined pixel chip, and the edge detection circuit being disposed on a predetermined circuit chip.

16. An image sensor comprising: a pixel array section in which R pixels that receive red light, Gr pixels that are arranged in rows of the R pixels and receive green light, B pixels that receive blue light, and Gb pixels that are arranged in rows of the B pixels and receive green light; a vertical scanning circuit that sequentially drives two pairs of Gr pixels and two pairs of Gb pixels to output pixel signals; and an edge detection circuit that detects the presence or absence of an edge based on the pixel signals of each of the pairs.

17. A control method for an image sensor comprising: a vertical scanning procedure for sequentially driving first and second pixel pairs and third and fourth pixel pairs having an inter-pixel distance different from that of the first and second pixel pairs to output pixel signals; and an edge detection procedure for detecting the presence or absence of an edge based on the pixel signals of each of the first, second, third, and fourth pixel pairs.

Images