Photoelectric conversion device and imaging system
The photoelectric conversion device efficiently processes pixel data by dividing the pixel section into regions with separate processing circuits, addressing inefficiencies in circuit size and data handling.
Patent Information
- Application Number
- JP2020152620
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-09-11
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2040-09-11
AI Technical Summary
Existing photoelectric conversion devices face inefficiencies in processing pixel data without increasing circuit scale, particularly when signal processing circuits are divided horizontally, leading to underutilization and increased circuit size for handling multiple types of data.
The device is configured with a pixel section divided into regions, utilizing first and second processing circuits with memories and a data exchange circuit to efficiently process and output signals without increasing circuit scale, allowing for separate handling of different types of pixel data.
This configuration enables efficient processing of pixel data without enlarging the circuit, facilitating simultaneous output of multiple types of signals through independent processing circuits.
Smart Images

Figure 0007583562000001 
Figure 0007583562000002 
Figure 0007583562000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a photoelectric conversion device and an imaging system. [Background technology]
[0002] In a stacked photoelectric conversion device formed by stacking a pixel substrate on which a pixel array is provided and a signal processing substrate on which a signal processing circuit is provided, the signal processing circuit on the signal processing substrate may be divided and arranged in a plurality of regions. For example, for performance improvement and convenience in manufacturing, a vertical scanning circuit may be arranged in a portion overlapping with the center of the pixel array. In such a case, the column circuit section and the data processing section are divided in two in the horizontal direction and arranged on the signal processing substrate. Patent Document 1 describes a solid-state imaging device in which the signal processing circuit is divided in two and arranged on a logic substrate. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2015-216334 A Summary of the Invention [Problem to be solved by the invention]
[0004] When the column circuits corresponding to each column of the pixel array are divided in the horizontal direction, horizontal scanning is performed for each divided area to output data. When combining data divided in the horizontal direction, it is possible to combine the data in a data processing unit of one of the signal processing circuits, as described in Patent Document 1, for example.
[0005] However, when the signal processing circuit is configured in this way, the other of the divided signal processing units is not used. Also, when multiple types of data that are independent of each other are read from the pixel array and processed by the data processing unit, it is necessary to implement a circuit for processing the multiple types of data in one of the signal processing circuits, which may increase the circuit size.
[0006] An object of the present invention is to provide a photoelectric conversion device and an imaging system that can efficiently output pixel data without increasing the circuit scale. [Means for solving the problem]
[0007] According to one aspect of the present invention, there is provided a pixel section in which a plurality of pixels, each having a photoelectric conversion section, are arranged in a plurality of rows and a plurality of columns, and a processing circuit for processing signals output from the pixel section, the pixel section having a first region and a second region defined by columns, and configured to output a plurality of types of signals including a first signal and a second signal from each of the columns belonging to the first region and the second region, the processing circuit having a first processing circuit including a first memory and a third memory, a second processing circuit including a second memory and a fourth memory, and a data exchange circuit, the first memory being configured to store the first signal and the second signal of each column output from the first region, and the second memory being configured to read and write signals from the second region and a data exchange circuit. the data exchange circuit is configured to read out the first signal stored in the first memory and the first signal stored in the second memory and store them in the third memory, and to read out the second signal stored in the first memory and the second signal stored in the second memory and store them in the fourth memory, and the processing circuit is configured to output the first signal of the first region and the second region stored in the third memory from the first processing circuit, and to output the second signal of the first region and the second region stored in the fourth memory from the second processing circuit. Effect of the Invention
[0008] According to the present invention, it is possible to realize efficient processing of pixel data without increasing the circuit scale. [Brief description of the drawings]
[0009] [Figure 1]1 is a block diagram showing a schematic configuration of an imaging device according to a first embodiment of the present invention. [Diagram 2] 1 is a circuit diagram showing an example of the configuration of a pixel in an imaging device according to a first embodiment of the present invention. [Diagram 3] 1 is a perspective view illustrating a schematic configuration of an imaging device according to a first embodiment of the present invention. [Figure 4] 2 is a diagram showing the connection relationship between a pixel section and a vertical scanning section in the image pickup device according to the first embodiment of the present invention. FIG. [Diagram 5] 1 is a diagram showing the connection relationship between a pixel section, an AD conversion section, and a horizontal scanning section in an image pickup device according to a first embodiment of the present invention. [Figure 6] 1 is a block diagram showing an example of the configuration of a processing circuit in an imaging device according to a first embodiment of the present invention. [Figure 7] FIG. 11 is a block diagram showing an example of the configuration of a processing circuit in an imaging device according to a second embodiment of the present invention. [Figure 8] FIG. 11 is a circuit diagram showing an example of the configuration of a pixel in an imaging device according to a third embodiment of the present invention. [Figure 9] FIG. 11 is a schematic top view of a pixel in an imaging device according to a third embodiment of the present invention. [Figure 10] 13 is a diagram showing the connection relationship between a pixel section, a vertical scanning section, an AD conversion section, and a horizontal scanning section in an imaging device according to a third embodiment of the present invention. FIG. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of a processing circuit in an imaging device according to a third embodiment of the present invention. [Figure 12] FIG. 13 is a plan view (part 1) showing an example of the layout of each functional block on a signal processing board of an imaging device according to a fourth embodiment of the present invention. [Figure 13] FIG. 13 is a plan view (part 2) showing an example of the layout of each functional block on a signal processing board of an imaging device according to a fourth embodiment of the present invention. [Figure 14] FIG. 13 is a plan view (part 3) showing an example of the layout of each functional block on a signal processing board of an imaging device according to a fourth embodiment of the present invention. [Figure 15]FIG. 13 is a block diagram showing a schematic configuration of an imaging system according to a fifth embodiment of the present invention. [Figure 16] FIG. 13 is a diagram showing an example of the configuration of an imaging system and a moving object according to a sixth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] [First embodiment] An imaging device according to a first embodiment of the present invention will be described with reference to Figs. 1 to 6. Fig. 1 is a block diagram showing a schematic configuration of the imaging device according to this embodiment. Fig. 2 is a circuit diagram showing an example of the configuration of a pixel in the imaging device according to this embodiment. Fig. 3 is a perspective view showing the imaging device according to this embodiment. Fig. 4 is a diagram showing the connection relationship between a pixel section and a vertical scanning section in the imaging device according to this embodiment. Fig. 5 is a diagram showing the connection relationship between the pixel section, an AD conversion section, and a horizontal scanning section in the imaging device according to this embodiment. Fig. 6 is a block diagram showing an example of the configuration of a processing circuit in the imaging device according to this embodiment.
[0011] An imaging device 10 according to this embodiment may be configured with a processing device (processor) 1, a control unit 2, a vertical scanning unit 3, a pixel unit 4, an AD conversion unit (analog-to-digital conversion unit) 5, a horizontal scanning unit 6, a data processing unit 7, and an output unit 8, as shown in Fig. 1 for example. The pixel unit 4 is connected to the vertical scanning unit 3 and the AD conversion unit 5. The AD conversion unit 5 is connected to the horizontal scanning unit 6. The horizontal scanning unit 6 is connected to the data processing unit 7. The data processing unit 7 is connected to the output unit 8. The vertical scanning unit 3, the AD conversion unit 5, the horizontal scanning unit 6, and the data processing unit 7 are connected to the control unit 2. The processing device 1 is connected to the control unit 2.
[0012] The processing device 1 is, for example, a CPU (Central Processing Unit), and controls the overall operation of the imaging device 10. The processing device 1 outputs control signals such as a synchronization signal and setting signals such as an operation mode to the control unit 2. Note that the processing device 1 may be provided within an imaging system in which the imaging device 10 is mounted, that is, outside the imaging device 10.
[0013] The control unit 2 is a control circuit that receives control signals and setting signals from the processing device 1 and outputs control signals to the vertical scanning unit 3, AD conversion unit 5, horizontal scanning unit 6 and data processing unit 7 to control their operation and timing.
[0014] In the pixel section 4, a plurality of pixels P, each including a photoelectric conversion section, are two-dimensionally arranged in a plurality of rows and a plurality of columns. Each of the plurality of pixels P is configured to output a pixel signal according to the amount of light incident on the photoelectric conversion section. FIG. 1 shows the pixel section 4 composed of pixels P arranged in a matrix of m columns x n rows, and the coordinates represented by (column number, row number) are added to the reference symbol P of each pixel. In this specification, the direction in which each row extends (row direction) is defined as the horizontal direction, and the direction in which each column extends (column direction) is defined as the vertical direction. In addition, the row number of the first row is the first row, and the column number of the first column is the first column.
[0015] The vertical scanning unit 3 is a control circuit that operates upon receiving a signal from the control unit 2 and drives a plurality of pixels P constituting the pixel unit 4 on a row-by-row basis. The operations performed by the vertical scanning unit 3 on a plurality of pixels P constituting the pixel unit 4 include readout scanning and shutter scanning. The shutter scanning refers to an operation of resetting the photoelectric conversion unit for some or all of the pixels P of the pixel unit 4 sequentially on a row-by-row basis and starting exposure. The readout scanning refers to an operation of outputting a signal based on the charge accumulated in the photoelectric conversion unit from the pixels P of some or all of the rows of the pixel unit 4 sequentially on a row-by-row basis. The vertical scanning unit 3 supplies a control signal to the pixels P on a row-by-row basis via a control line (not shown) arranged on each row of the pixel array. The vertical scanning unit 3 may be configured using a shift register or an address decoder.
[0016] The AD conversion unit 5 may include an amplifier circuit, an analog-to-digital conversion (AD conversion) circuit, and a memory, all of which are not shown. The AD conversion unit 5 performs amplification and AD conversion on pixel signals, which are analog signals output from the pixels P of the pixel unit 4 via vertical output lines, all of which are not shown, and stores the pixel signals in the memory as digital data.
[0017] The horizontal scanning unit 6 is a control circuit that operates upon receiving a signal from the control unit 2 and transfers the pixel signals converted into digital data by the AD conversion unit 5 sequentially for each column to the data processing unit 7. The horizontal scanning unit 6 can be configured using a shift register and an address decoder.
[0018] The data processing unit 7 is a functional block that performs predetermined processing on the digital data transferred from the horizontal scanning unit 6. A specific configuration of the data processing unit 7 and specific processing in the data processing unit 7 will be described later.
[0019] The output unit 8 is an output circuit that digitally processes the signal transferred from the data processing unit 7 and outputs the signal as serial data to the outside of the imaging device 10. The output unit 8 may be configured to include a digital processing unit, a parallel-serial conversion circuit, and an output circuit such as LVDS (Low Voltage Differential Signaling).
[0020] In this specification, among these functional blocks constituting the imaging device 10, the control unit 2, AD conversion unit 5, horizontal scanning unit 6, data processing unit 7 and output unit 8 may be collectively referred to as a processing circuit.
[0021] Fig. 2 is a circuit diagram showing a configuration example of a pixel P. Fig. 2 shows a pixel P(M,N) arranged in the Mth column and Nth row of the pixel section 4 as an example, but the same is true for the other pixels P. Note that M is an integer not less than 1 and not more than m, and N is an integer not less than 1 and not more than n.
[0022] As shown in FIG. 2, each pixel P may be composed of a photoelectric conversion unit PD, a transfer transistor M1, a reset transistor M2, an amplification transistor M3, and a selection transistor M4.
[0023] The photoelectric conversion unit PD may be composed of a photoelectric conversion element such as a photodiode or a photogate. Here, it is assumed that the photoelectric conversion unit PD is composed of a photodiode. The photodiode constituting the photoelectric conversion unit PD has an anode connected to a ground node and a cathode connected to a source of a transfer transistor M1. The drain of the transfer transistor M1 is connected to a source of a reset transistor M2 and a gate of an amplification transistor M3. A connection node between the drain of the transfer transistor M1, the source of the reset transistor M2, and the gate of the amplification transistor M3 is a so-called floating diffusion unit FD. The drain of the reset transistor M2 and the drain of the amplification transistor M3 are connected to a power supply voltage node (voltage VCC). The source of the amplification transistor M3 is connected to the drain of a selection transistor M4. The source of the selection transistor M4 is connected to a vertical output line Vline(M).
[0024] In the case of the pixel configuration shown in FIG. 2, each of the control lines arranged in each row of the pixel section 4 includes a transfer gate signal line, a reset signal line, and a selection signal line. For example, the transfer gate signal line of the Nth row is connected to the gate of the transfer transistor M1 of the pixel P of the Nth row, and supplies the control signal PTX(N) output from the vertical scanning unit 3 to the gate of the transfer transistor M1 of these pixels P. The reset signal line of the Nth row is connected to the gate of the reset transistor M2 of the pixel P of the Nth row, and supplies the control signal PRES(N) output from the vertical scanning unit 3 to the gate of the reset transistor M2 of these pixels P. The selection signal line of the Nth row is connected to the gate of the selection transistor M4 of the pixel P of the Nth row, and supplies the control signal PSEL(N) output from the vertical scanning unit 3 to the gate of the selection transistor M4 of these pixels P. A common control signal is supplied from the vertical scanning unit 3 to the pixels P in the same row.
[0025] When each transistor is an N-channel transistor, when a high-level control signal is supplied from the vertical scanning unit 3, the corresponding transistor is turned on, and when a low-level control signal is supplied from the vertical scanning unit 3, the corresponding transistor is turned off. Here, the high level corresponds to a logical value of "1," and the low level corresponds to a logical value of "0." Note that each transistor constituting a pixel P may be an N-channel transistor, but may also be a P-channel transistor.
[0026] The photoelectric conversion unit PD converts incident light into an electric charge corresponding to the amount of the light (photoelectric conversion) and accumulates the generated electric charge. When the transfer transistor M1 is turned on (conductive state), it transfers the electric charge held by the photoelectric conversion unit PD to the floating diffusion unit FD. The floating diffusion unit FD has a capacitance component, and holds the electric charge transferred from the photoelectric conversion unit PD, and sets the floating diffusion unit FD to a potential corresponding to the amount of the electric charge by charge-voltage conversion using the capacitance. The amplification transistor M3 has a configuration in which a voltage VCC is supplied to the drain, and a bias current is supplied to the source from a current source (not shown) via the vertical output line Vline and the selection transistor M4, forming a source follower circuit with the gate as an input node. As a result, the amplification transistor M3 outputs a signal based on the electric potential of the floating diffusion unit FD to the vertical output line Vline via the selection transistor M4. When the reset transistor M2 is turned on (conductive state), it resets the floating diffusion unit FD to a potential corresponding to the power supply voltage. Moreover, by turning on (conducting state) the transfer transistor M1 simultaneously with the reset transistor M2, the photoelectric conversion unit PD can be reset to a potential corresponding to the voltage VCC. The selection transistor M4 switches the connection between the amplification transistor M3 and the vertical output line Vline.
[0027] The imaging device 10 according to the present embodiment may be configured as a stacked sensor in which a pixel substrate 11 and a signal processing substrate 12 are stacked, as shown in Fig. 3 for example. Of the functional blocks shown in Fig. 1, for example, the pixel unit 4 may be arranged on the pixel substrate 11. Also, of the functional blocks shown in Fig. 1, for example, the control unit 2, vertical scanning unit 3, AD conversion unit 5, horizontal scanning unit 6, data processing unit 7, and output unit 8 may be arranged on the signal processing substrate 12.
[0028] The vertical scanning unit 3 is divided into three blocks (vertical scanning units 3L, 3C, and 3R) and arranged on a signal processing board 12. A processing circuit 13 is arranged between the vertical scanning unit 3L and the vertical scanning unit 3C, and a processing circuit 14 is arranged between the vertical scanning unit 3C and the vertical scanning unit 3R. The processing circuit 13 is provided with a processing circuit for processing signals of pixels P arranged in a first region defined by columns among the pixels P arranged in the pixel unit 4. The processing circuit 14 is provided with a processing circuit for processing signals of pixels P arranged in a second region defined by columns among the pixels P arranged in the pixel unit 4. Here, the range from the 1st column to the m / 2th column is the first region, and the range from the m / 2+1th column to the mth column is the second region. Each of the control unit 2, the AD conversion unit 5, the horizontal scanning unit 6, the data processing unit 7, and the output unit 8 is divided into two blocks and assigned to the processing circuit 13 and the processing circuit 14. The processing circuit 13 and the processing circuit 14 are arranged apart from each other.
[0029] As shown in FIG. 4, each row of the pixel array constituting the pixel unit 4 is provided with a signal line to which the control signal PRES is supplied, a signal line to which the control signal PTX is supplied, and a signal line to which the control signal PSEL is supplied. These signal lines are connected to a plurality of pixels P arranged in the corresponding row. For example, the signal line arranged in the (12i+1)th row is connected to the pixels P(1,12i+1), ..., P(j,12i+1), P(j+1,12i+1), P(j+2,12i+1), ..., P(m,12i+1). Each of these signal lines is connected to the vertical scanning unit 3 arranged on the signal processing board 12 via an electrical connection between the pixel board 11 and the signal processing board 12. Note that i is an integer of 0 or more with the row number ranging from 1 to n. Blocks of 12 rows, similar to those in the (12i+1)th row to the (12i+12)th row, are repeatedly arranged in the first row to the nth row.
[0030] In addition, as shown in FIG. 4, a predetermined number of vertical output lines Vline are provided in each column of the pixel array constituting the pixel unit 4. In FIG. 4, it is assumed that eight vertical output lines Vline are provided in each column. For example, eight vertical output lines Vline, namely, vertical output lines Vline1(j), Vline2(j), Vline3(j), Vline4(j), Vline5(j), Vline6(j), Vline7(j), and Vline8(j), are provided in the j-th column. The number of vertical output lines Vline provided in each column is not limited to eight, and can be increased or decreased as appropriate. Here, j is an integer of 1 or more such that the column number is in the range of 1 to m.
[0031] The vertical output line Vline1 of each column is connected to a pixel P in the (12i+1)th row and a pixel P in the (12i+3)th row. The vertical output line Vline2 of each column is connected to a pixel P in the (12i+4)th row and a pixel P in the (12i+6)th row. The vertical output line Vline3 of each column is connected to a pixel P in the (12i+7)th row and a pixel P in the (12i+9)th row. The vertical output line Vline4 of each column is connected to a pixel P in the (12i+10)th row and a pixel P in the (12i+12)th row. The vertical output line Vline5 of each column is connected to a pixel P in the (12i+5)th row. The vertical output line Vline6 of each column is connected to a pixel P in the (12i+2)th row. The pixel P in the (12i+11)th row is connected to the vertical output line Vline7 of each column, and the pixel P in the (12i+8)th row is connected to the vertical output line Vline8 of each column.
[0032] In response to control signals PRES, PTX, and PSEL supplied from the vertical scanning unit 3, pixel data obtained by adding together pixel signals read from a plurality of pixels P arranged in the vertical direction is read out to the vertical output lines Vline1, Vline2, Vline3, and Vline4. That is, pixel data obtained by adding together the pixel signal of pixel P in the (12i+1)th row and the pixel signal of pixel P in the (12i+3)th row is read out to the vertical output line Vline1. Pixel data obtained by adding together the pixel signal of pixel P in the (12i+4)th row and the pixel signal of pixel P in the (12i+6)th row is read out to the vertical output line Vline3. Pixel data obtained by adding together the pixel signal of pixel P in the (12i+7)th row and the pixel signal of pixel P in the (12i+9)th row is read out to the vertical output line Vline3. Pixel data obtained by adding the pixel signal of pixel P in the (12i+10)th row and the pixel signal of pixel P in the (12i+12)th row is read out to the vertical output line Vline4.
[0033] Moreover, non-added pixel data thinned out in the vertical direction is read out to the vertical output lines Vline5, Vline6, Vline7, and Vline8 in response to the control signals PRES, PTX, and PSEL supplied from the vertical scanning unit 3. That is, pixel data of pixel P in the (12i+5)th row is read out to the vertical output line Vline5. Pixel data of pixel P in the (12i+2)th row is read out to the vertical output line Vline6. Pixel data of pixel P in the (12i+11)th row is read out to the vertical output line Vline7. Pixel data of pixel P in the (12i+8)th row is read out to the vertical output line Vline8.
[0034] That is, the imaging device 10 of this embodiment is configured to be able to read two types of pixel data: pixel data obtained by adding pixel signals read from a plurality of pixels P aligned in the vertical direction, and non-added pixel data thinned out in the vertical direction. In other words, the imaging device 10 of this embodiment is configured to output a plurality of types of signals including at least a first signal and a second signal from each column of the pixel section 4. Here, the first signal is pixel data obtained by adding pixel signals read from a first number (two) of pixels P aligned in the vertical direction. The second signal is pixel data obtained by adding pixel signals read from a second number (one) of pixels P aligned in the vertical direction that is different from the first number.
[0035] As shown in Fig. 5, each of the vertical output lines Vline is connected to an AD conversion unit 5 provided on the signal processing substrate 12 via an electrical connection between the pixel substrate 11 and the signal processing substrate 12. The AD conversion unit 5 has a plurality of (m) column circuit groups 51 corresponding to each column of the pixel array constituting the pixel unit 4. Each of the column circuit groups 51 has a plurality of (8 in the example of Fig. 5) column circuits 52 corresponding to the number of vertical output lines Vline arranged in the corresponding column. In other words, the AD conversion unit 5 has m x 8 column circuits 52 for a pixel array of n rows and m columns.
[0036] As shown in FIG. 5, the column circuit group 51 of the jth column is composed of column circuits 52(j,1), 52(j,2), 52(j,3), 52(j,4), 52(j,5), 52(j,6), 52(j,7), and 52(j,8). The column circuit 52(j,1) is connected to the vertical output line Vline1(j). The column circuit 52(j,2) is connected to the vertical output line Vline2(j). The column circuit 52(j,3) is connected to the vertical output line Vline3(j). The column circuit 52(j,4) is connected to the vertical output line Vline4(j). The column circuit 52(j,5) is connected to the vertical output line Vline5(j). The column circuit 52(j,6) is connected to the vertical output line Vline6(j). The column circuit 52(j,7) is connected to the vertical output line Vline7(j). The column circuit 52(j,8) is connected to a vertical output line Vline8(j).
[0037] As described above, the AD conversion unit 5 is divided and arranged in the processing circuit 13 and the processing circuit 14. In this embodiment, of the m×8 column circuits 52 constituting the AD conversion unit 5, half are arranged in the processing circuit 13, and the remaining half are arranged in the processing circuit 14. The vertical output lines Vline of each of the first column to the (m / 2)th column are connected to the column circuits 52 arranged in the processing circuit 13. The vertical output lines Vline of each of the (m / 2+1)th column to the mth column are connected to the column circuits 52 arranged in the processing circuit 14.
[0038] The horizontal scanning unit 6 has a plurality of horizontal scanning circuits 61 (eight in the example of FIG. 5) corresponding to the number of vertical output lines Vline and column circuits 52 arranged in each column of the pixel array constituting the pixel unit 4. The column circuits 52 belonging to the column circuit group 51 of the same column are connected to different horizontal scanning circuits 61. For example, as shown in FIG. 5, the horizontal scanning unit 6 is composed of horizontal scanning circuits 61(1), 61(2), 61(3), 61(4), 61(5), 61(6), 61(7), and 61(8). The column circuit (j,1) of the jth column is connected to the horizontal scanning circuit 61(1). The column circuit (j,2) of each column is connected to the horizontal scanning circuit 61(2). The column circuit (j,3) of each column is connected to the horizontal scanning circuit 61(3). The column circuit (j,4) of each column is connected to the horizontal scanning circuit 61(4). The column circuit (j,5) of each column is connected to the horizontal scanning circuit 61(5). The column circuit (j,6) of each column is connected to a horizontal scanning circuit 61(6). The column circuit (j,7) of each column is connected to a horizontal scanning circuit 61(7). The column circuit (j,8) of each column is connected to a horizontal scanning circuit 61(8).
[0039] As described above, the horizontal scanning unit 6 is divided into the processing circuits 13 and 14. The horizontal scanning circuit 61 arranged in the processing circuit 13 sequentially horizontally transfers digital data of pixels P from the 1st column to the (m / 2)th column, and outputs it to the data processing unit 7 of the processing circuit 13. The horizontal scanning circuit 61 arranged in the processing circuit 14 sequentially horizontally transfers digital data of pixels P from the (m / 2+1)th column to the mth column, and outputs it to the data processing unit 7 of the processing circuit 14.
[0040] Next, an example of the configuration of the processing circuit 13 and the processing circuit 14 in the image pickup device 10 according to the present embodiment will be described in more detail with reference to FIG.
[0041] The processing circuit 13 is composed of an AD conversion unit 5L, a horizontal scanning unit 6L, a data processing unit 7L, and an output unit 8L. The processing circuit 14 is composed of an AD conversion unit 5R, a horizontal scanning unit 6R, a data processing unit 7R, and an output unit 8R. The AD conversion units 5L, 5R and the horizontal scanning units 6L, 6R are as described with reference to Figs. 4 and 5.
[0042] The data processing unit 7L may be composed of an input data storage unit 71L, a calculation data storage unit 72L, a calculation data input unit 73L, and a calculation unit 74L. Each of the input data storage unit 71L and the calculation data storage unit 72L has a plurality of memories corresponding to the number of horizontal scanning circuits 61L of the horizontal scanning unit 6L. For example, as shown in FIG. 6, the input data storage unit 71L has eight memories 71L1, 71L2, 71L3, 71L4, 71L5, 71L6, 71L7, and 71L8. Also, as shown in FIG. 6, the calculation data storage unit 72L has eight memories 72L1, 72L2, 72L3, 72L4, 72L5, 72L6, 72L7, and 72L8.
[0043] Similarly, the data processing unit 7R may be composed of an input data storage unit 71R, a calculation data storage unit 72R, a calculation data input unit 73R, and a calculation unit 74R. Each of the input data storage unit 71R and the calculation data storage unit 72R has a number of memories corresponding to the number of horizontal scanning circuits 61R of the horizontal scanning unit 6R. For example, as shown in FIG. 6, the input data storage unit 71R has eight memories 71R1, 71R2, 71R3, 71R4, 71R5, 71R6, 71R7, and 71R8. Also, as shown in FIG. 6, the calculation data storage unit 72R has eight memories 72R1, 72R2, 72R3, 72R4, 72R5, 72R6, 72R7, and 72R8.
[0044] The horizontal scanning circuits 61(1) to 61(8) of the horizontal scanning unit 6L are connected to the memories 71L1 to 71L8 of the input data storage unit 71L, respectively. The horizontal scanning circuits 61(1) to 61(8) of the horizontal scanning unit 6R are connected to the memories 71R1 to 71R8 of the input data storage unit 71R, respectively.
[0045] The input data storage units 71L, 71R and the calculation data storage units 72L, 72R are connected via a data exchange circuit 9. Specifically, the memories 71L1 to 71L4 of the input data storage unit 71L are connected to the memories 72L1 to 72L4 of the calculation data storage unit 72L. The memories 71L5 to 71L8 of the input data storage unit 71L are connected to the memories 72R1 to 72R4 of the calculation data storage unit 72R. The memories 71R1 to 71R4 of the input data storage unit 71R are connected to the memories 72L5 to 72L8 of the calculation data storage unit 72L. The memories 71R5 to 71R8 of the input data storage unit 71R are connected to the memories 72R5 to 72R8 of the calculation data storage unit 72R. The data exchange circuit 9 may be configured, for example, by a bus. In this case, data transfer between memories is possible by serial transfer, for example, on a pixel-by-pixel basis.
[0046] The calculation data input unit 73L is connected to the calculation unit 74L. The calculation unit 74L is connected to the output unit 8L. Similarly, the calculation data input unit 73R is connected to the calculation unit 74R. The calculation unit 74R is connected to the output unit 8R.
[0047] Next, the operation of the processing circuits 13 and 14 will be described with reference to FIG. The pixel data read from the vertical output line Vline1 of each column from the 1st column to the (m / 2)th column is sequentially transferred by the horizontal scanning circuit 61(1) of the horizontal scanning unit 6L and stored in the memory 71L1 of the input data storage unit 71L. This data is referred to as data D1L.
[0048] The pixel data read from the vertical output line Vline2 of each of the first to (m / 2) columns is sequentially transferred by the horizontal scanning circuit 61(2) of the horizontal scanning unit 6L and stored in the memory 71L2 of the input data storage unit 71L. This data is referred to as data D2L.
[0049] The pixel data read from the vertical output line Vline3 of each of the first to (m / 2)th columns is sequentially transferred by the horizontal scanning circuit 61(3) of the horizontal scanning unit 6L and stored in the memory 71L3 of the input data storage unit 71L. This data is referred to as data D3L.
[0050] The pixel data read from the vertical output line Vline4 of each column from the 1st column to the (m / 2)th column is sequentially transferred by the horizontal scanning circuit 61(4) of the horizontal scanning unit 6L and stored in the memory 71L4 of the input data storage unit 71L. This data is referred to as data D4L.
[0051] The pixel data read from the vertical output line Vline5 of each column from the 1st column to the (m / 2)th column is sequentially transferred by the horizontal scanning circuit 61(5) of the horizontal scanning unit 6L and stored in the memory 71L5 of the input data storage unit 71L. This data is referred to as data D5L.
[0052] The pixel data read from the vertical output line Vline6 of each column from the 1st column to the (m / 2)th column is sequentially transferred by the horizontal scanning circuit 61(6) of the horizontal scanning unit 6L and stored in the memory 71L6 of the input data storage unit 71L. This data is referred to as data D6L.
[0053] The pixel data read from the vertical output line Vline7 of each column from the 1st column to the (m / 2)th column is sequentially transferred by the horizontal scanning circuit 61(7) of the horizontal scanning unit 6L and stored in the memory 71L7 of the input data storage unit 71L. This data is referred to as data D7L.
[0054] The pixel data read from the vertical output line Vline8 of each column from the 1st column to the (m / 2)th column is sequentially transferred by the horizontal scanning circuit 61(8) of the horizontal scanning unit 6L and stored in the memory 71L8 of the input data storage unit 71L. This data is referred to as data D8L.
[0055] Similarly, pixel data read from the vertical output line Vline1 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(1) of the horizontal scanning unit 6R and stored in the memory 71R1 of the input data storage unit 71R. This data is referred to as data D1R.
[0056] The pixel data read from the vertical output line Vline2 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(2) of the horizontal scanning unit 6R and stored in the memory 71R2 of the input data storage unit 71R. This data is referred to as data D2R.
[0057] The pixel data read from the vertical output line Vline3 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(3) of the horizontal scanning unit 6R and stored in the memory 71R3 of the input data storage unit 71R. This data is referred to as data D3R.
[0058] The pixel data read from the vertical output line Vline4 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(4) of the horizontal scanning unit 6R and stored in the memory 71R4 of the input data storage unit 71R. This data is referred to as data D4R.
[0059] The pixel data read from the vertical output line Vline5 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(5) of the horizontal scanning unit 6R and stored in the memory 71R5 of the input data storage unit 71R. This data is referred to as data D5R.
[0060] The pixel data read from the vertical output line Vline6 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(6) of the horizontal scanning unit 6R and stored in the memory 71R6 of the input data storage unit 71R. This data is referred to as data D6R.
[0061] The pixel data read from the vertical output line Vline7 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(7) of the horizontal scanning unit 6R and stored in the memory 71R7 of the input data storage unit 71R. This data is referred to as data D7R.
[0062] The pixel data read from the vertical output line Vline8 of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61(8) of the horizontal scanning unit 6R and stored in the memory 71R8 of the input data storage unit 71R. This data is referred to as data D8R.
[0063] Of the data stored in the input data storage unit 71L, data D1L-D4L stored in memories 71L1-71L4 are transferred to memories 72L1-72L4, respectively, of the calculation data storage unit 72L. Of the data stored in the input data storage unit 71L, data D5L-D8L stored in memories 71L5-71L8 are transferred to memories 72R1-72R4, respectively, of the calculation data storage unit 72R.
[0064] Among the data stored in the input data storage unit 71R, the data D1R-D4R stored in memories 71R1-71R4 are transferred to memories 72L5-72L8, respectively, of the calculation data storage unit 72L. Among the data stored in the input data storage unit 71R, the data D5R-D8R stored in memories 71R5-71R8 are transferred to memories 72R5-72R8, respectively, of the calculation data storage unit 72R.
[0065] That is, in this embodiment, the data D5L, D6L, D7L, and D8L are transferred from the processing circuit 13 to the processing circuit 14, and the data D1R, D2R, D3R, and D4R are transferred from the processing circuit 14 to the processing circuit 13.
[0066] As a result, the calculation data storage unit 72L stores pixel data (data D1L, D2L, D3L, D4L, D1R, D2R, D3R, D4R) from the first column to the m-th column corresponding to the vertical output lines Vline1, Vline2, Vline3, Vline4. Also, the calculation data storage unit 72R stores pixel data (data D5L, D6L, D7L, D8L, D5R, D6R, D7R, D8R) from the first column to the m-th column corresponding to the vertical output lines Vline5, Vline6, Vline7, Vline8.
[0067] The pixel data corresponding to the vertical output lines Vline1, Vline2, Vline3, and Vline4 stored in the calculation data storage unit 72L in this way is pixel data obtained by vertical pixel addition. On the other hand, the pixel data corresponding to the vertical output lines Vline5, Vline6, Vline7, and Vline8 stored in the calculation data storage unit 72R in this way is non-added pixel data thinned out in the vertical direction. In other words, data corresponding to two different pixel readout methods can be stored separately in the calculation data storage unit 72L and the calculation data storage unit 72R.
[0068] The data stored in the calculation data storage unit 72L is read out by the calculation data input unit 73L, then processed by the calculation unit 74L, and output from the output unit 8L. Similarly, the data stored in the calculation data storage unit 72R is read out by the calculation data input unit 73R, then processed by the calculation unit 74R, and output from the output unit 8R. Since the calculation units 74L and 74R are independent of each other, different calculation processes can be performed simultaneously in each. This makes it possible to use all the divided data processing units 7L and 7R to separate and simultaneously output data corresponding to two different pixel readout methods.
[0069] In this way, according to this embodiment, it is possible to realize efficient processing of pixel data without increasing the circuit scale.
[0070] [Second embodiment] An imaging device according to a second embodiment of the present invention will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of the configuration of a processing circuit in the imaging device according to this embodiment. The same components as those in the imaging device according to the first embodiment are given the same reference numerals, and their description will be omitted or simplified.
[0071] In the first embodiment, the data is divided and output according to two types of pixel readout methods, vertical addition and non-addition. In the present embodiment, an example is described in which pixel data of R / Gr rows and pixel data of Gb / B rows are separated and output in an imaging device having a pixel array of a Bayer array. The R / Gr rows are rows in which pixels (R pixels) having red color filters and pixels (G pixels) having green color filters are alternately arranged. The Gb / B rows are rows in which G pixels and pixels (B pixels) having blue color filters are alternately arranged. The R / Gr rows and the Gb / B rows are alternately arranged. For example, in the pixel array of FIG. 1, pixel P(1,1) is an R pixel, pixel P(2,1) is a G pixel, pixel P(1,2) is a G pixel, and pixel P(2,2) is a B pixel. Pixels P in adjacent rows in the same column are pixels of different colors.
[0072] The imaging device according to this embodiment is similar to the imaging device 10 according to the first embodiment except for the configuration of the data processing units 7L and 7R. The imaging device according to this embodiment will be described below, focusing on the differences from the imaging device according to the first embodiment.
[0073] The data processing unit 7L in the imaging device according to this embodiment is composed of an input data storage unit 71L, a calculation data storage unit 72L, a calculation data input unit 73L, and a calculation unit 74L, as in the first embodiment. The data processing unit 7R in the imaging device according to this embodiment is composed of an input data storage unit 71R, a calculation data storage unit 72R, a calculation data input unit 73R, and a calculation unit 74R, as in the first embodiment. The data processing units 7L and 7R in this embodiment differ from the first embodiment in the connection between the input data storage units 71L and 71R and the calculation data storage units 72L and 72R in the data exchange circuit 9.
[0074] That is, in this embodiment, the memories 71L1, 71L3, 71L5, and 71L7 of the input data storage unit 71L are connected to the memories 72L1, 72L3, 72L5, and 72L7 of the calculation data storage unit 72L. The memories 71L2, 71L4, 71L6, and 71L8 of the input data storage unit 71L are connected to the memories 72R1, 72R3, 72R5, and 72R7 of the calculation data storage unit 72R. The memories 71R1, 71R3, 71R5, and 71R7 of the input data storage unit 71R are connected to the memories 72L2, 72L4, 72L6, and 72L8 of the calculation data storage unit 72L. The memories 71R2, 71R4, 71R6, and 71R8 of the input data storage unit 71R are connected to the memories 72R2, 72R4, 72R6, and 72R8 of the calculation data storage unit 72R.
[0075] As a result, the data D1L, D3L, D5L, and D7L stored in the memories 71L1, 71L3, 71L5, and 71L7 of the input data storage unit 71L are transferred to the memories 72L1, 72L3, 72L5, and 72L7 of the calculation data storage unit 72L, respectively. The data D2L, D4L, D6L, and D8L stored in the memories 71L2, 71L4, 71L6, and 71L8 of the input data storage unit 71L are transferred to the memories 72R1, 72R3, 72R5, and 72R7 of the calculation data storage unit 72R, respectively.
[0076] Furthermore, the data D1R, D3R, D5R, and D7R stored in memories 71R1, 71R3, 71R5, and 71R7 of the input data storage unit 71R are transferred to memories 72L2, 72L4, 72L6, and 72L8, respectively, of the calculation data storage unit 72L. The data D2R, D4R, D6R, and D8R stored in memories 71R2, 71R4, 71R6, and 71R8 of the input data storage unit 71R are transferred to memories 72R2, 72R4, 72R6, and 72R8, respectively, of the calculation data storage unit 72R.
[0077] That is, in this embodiment, the data D2L, D4L, D6L, and D8L are transferred from the processing circuit 13 to the processing circuit 14, and the data D1R, D3R, D5R, and D7R are transferred from the processing circuit 14 to the processing circuit 13.
[0078] As a result, the calculation data storage unit 72L stores pixel data (data D1L, D3L, D5L, D7L, D1R, D3R, D5R, D7R) from the first column to the m-th column corresponding to the vertical output lines Vline1, Vline3, Vline5, Vline7. Also, the calculation data storage unit 72R stores pixel data (data D2L, D4L, D6L, D8L, D2R, D4R, D6R, D8R) from the first column to the m-th column corresponding to the vertical output lines Vline2, Vline4, Vline6, Vline8.
[0079] In the pixel array of Fig. 4, the R / Gr rows are, for example, rows (12i+1), (12i+3), (12i+5), (12i+7), (12i+9), and (12i+11). The Gb / B rows are, for example, rows (12i+2), (12i+4), (12i+6), (12i+8), (12i+10), and (12i+12). Therefore, the data output to the vertical output lines Vline1, Vline3, Vline5, and Vline7 is the data of the R / Gr rows, and the data output to the vertical output lines Vline2, Vline4, Vline6, and Vline8 is the data of the Gb / B rows.
[0080] By configuring the data processing unit 7 in this way, it is possible to store the R / Gr row data in the calculation data storage unit 72L and the Gb / B row data in the calculation data storage unit 72R. This makes it possible to separate the R / Gr row data and the Gb / B row data and output them simultaneously.
[0081] The data stored in the calculation data storage unit 72L is read out by the calculation data input unit 73L, then processed by the calculation unit 74L, and output from the output unit 8L. Similarly, the data stored in the calculation data storage unit 72R is read out by the calculation data input unit 73R, then processed by the calculation unit 74R, and output from the output unit 8R. Since the calculation units 74L and 74R are independent of each other, different calculation processes can be performed simultaneously in each. This makes it possible to use all the divided data processing units 7L and 7R to separate and simultaneously output data corresponding to two different pixel readout methods.
[0082] In this way, according to this embodiment, it is possible to realize efficient processing of pixel data without increasing the circuit scale.
[0083] [Third embodiment] An imaging device according to a third embodiment of the present invention will be described with reference to Figs. 8 to 11. Fig. 8 is a circuit diagram showing an example of the configuration of a pixel in the imaging device according to this embodiment. Fig. 9 is a schematic top view of a pixel in the imaging device according to this embodiment. Fig. 10 is a diagram showing the connection relationship between a pixel section, a vertical scanning section, an AD conversion section, and a horizontal scanning section in the imaging device according to this embodiment. Fig. 11 is a block diagram showing an example of the configuration of a processing circuit in the imaging device according to this embodiment. Components similar to those in the imaging devices according to the first and second embodiments are given the same reference numerals, and descriptions thereof will be omitted or simplified.
[0084] In this embodiment, an example will be shown in which, in an imaging apparatus equipped with an image plane phase difference AF function, image generation data and focus detection data are separated and output simultaneously.
[0085] The imaging device 10 according to this embodiment has a similar schematic configuration to the imaging device according to the first embodiment, except for a different circuit configuration of the pixel P. The imaging device according to this embodiment will be described below, focusing on the differences from the imaging device according to the first embodiment.
[0086] As shown in FIG. 8, each of the pixels P in the imaging device 10 according to this embodiment can be composed of photoelectric conversion units PDA, PDB, transfer transistors M1A, M1B, a reset transistor M2, an amplification transistor M3, and a selection transistor M4.
[0087] The photoelectric conversion units PDA and PDB may be composed of photoelectric conversion elements such as photodiodes and photogates. Here, it is assumed that the photoelectric conversion units PDA and PDB are composed of photodiodes. The photodiode constituting the photoelectric conversion unit PDA has an anode connected to a ground node and a cathode connected to the source of the transfer transistor M1A. The photodiode constituting the photoelectric conversion unit PDB has an anode connected to a ground node and a cathode connected to the source of the transfer transistor M1B. The drain of the transfer transistor M1A and the drain of the transfer transistor M1B are connected to the source of the reset transistor M2 and the gate of the amplification transistor M3. The connection node between the drain of the transfer transistor M1A, the drain of the transfer transistor M1B, the source of the reset transistor M2, and the gate of the amplification transistor M3 is a floating diffusion unit FD. The drain of the reset transistor M2 and the drain of the amplification transistor M3 are connected to a power supply voltage node (voltage VCC). The source of the amplification transistor M3 is connected to the drain of the selection transistor M4. The source of the selection transistor M4 is connected to the vertical output line Vline.
[0088] 8, each of the control lines arranged in each row of the pixel unit 4 includes a transfer gate signal line connected to the gate of the transfer transistor M1A and a transfer gate signal line connected to the gate of the transfer transistor M1B. In addition, the control line in each row further includes a reset signal line connected to the gate of the reset transistor M2 and a selection signal line connected to the gate of the selection transistor M4.
[0089] The pixels P arranged in the Nth row are supplied with control signals PTXA(N), PTXB(N), PRES(N), and PSEL(N) from the vertical scanning unit 3. The control signal PTXA(N) is a control signal supplied to the gate of the transfer transistor M1A of the pixel P in the Nth row. The control signal PTXB(N) is a control signal supplied to the gate of the transfer transistor M1B of the pixel P in the Nth row. The control signal PRES(N) is a control signal supplied to the gate of the reset transistor M2 of the pixel P in the Nth row. The control signal PSEL(N) is a control signal supplied to the gate of the selection transistor M4 of the pixel P in the Nth row. A common control signal is supplied from the vertical scanning unit 3 to the pixels P in the same row.
[0090] When each transistor is an N-type transistor, when a high-level control signal is supplied from the vertical scanning unit 3, the corresponding transistor is turned on and when a low-level control signal is supplied from the vertical scanning unit 3, the corresponding transistor is turned off. Here, a high level corresponds to a logical value of "1" and a low level corresponds to a logical value of "0." Note that each transistor constituting a pixel P may be an N-channel transistor, but may also be a P-channel transistor.
[0091] The photoelectric conversion units PDA and PDB convert the incident light into an electric charge corresponding to the amount of the light (photoelectric conversion) and accumulate the generated electric charge. The transfer transistor M1A transfers the electric charge held by the photoelectric conversion unit PDA to the floating diffusion unit FD by being turned on (conductive state). The transfer transistor M1B transfers the electric charge held by the photoelectric conversion unit PDB to the floating diffusion unit FD by being turned on (conductive state). The floating diffusion unit FD has a capacitance component, and holds the electric charge transferred from the photoelectric conversion units PDA and PDB, and sets the electric potential of the floating diffusion unit FD to an electric potential corresponding to the amount of the electric charge by charge-voltage conversion by the capacitance. The amplifier transistor M3 is configured such that a power supply voltage VCC is supplied to the drain, and a bias current is supplied to the source from a current source (not shown) via the vertical output line Vline and the selection transistor M4, forming a source follower circuit with the gate as an input node. As a result, the amplifier transistor M3 outputs a signal based on the electric potential of the floating diffusion unit FD to the vertical output line Vline via the selection transistor M4. The reset transistor M2 resets the floating diffusion unit FD to a potential corresponding to the power supply voltage by being turned on (conductive state). In addition, by turning on (conductive state) the transfer transistors M1A and M1B simultaneously with the reset transistor M2, the photoelectric conversion units PDA and PDB can be reset to a potential corresponding to the power supply voltage. The selection transistor M4 switches the connection between the amplification transistor M3 and the vertical output line Vline.
[0092] In addition, the photoelectric conversion unit PDA and the photoelectric conversion unit PDB constituting one pixel P share one microlens ML, for example, as shown in Fig. 9. In other words, the photoelectric conversion unit PDA and the photoelectric conversion unit PDB are configured to receive light that has passed through different pupil regions of light that has entered the imaging optical system. With this configuration, a signal based on the charge generated by the photoelectric conversion unit PDA and a signal based on the charge generated by the photoelectric conversion unit PDB can be used as signals for focus detection.
[0093] In the pixel P of this embodiment, the charge of the photoelectric conversion unit PDA and the charge of the photoelectric conversion unit PDB are transferred separately. First, the charge of the photoelectric conversion unit PDA is transferred to the floating diffusion unit FD by the transfer transistor M1A, and the signal output from the pixel P is AD converted by the AD conversion unit 5. Next, the charge of the photoelectric conversion unit PDB is transferred to the floating diffusion unit FD by the transfer transistor M1B, and the signal output from the pixel P is AD converted by the AD conversion unit 5. In other words, the data readout in the imaging device of this embodiment is configured to obtain two types of data for one exposure.
[0094] 10, a signal line to which the control signal PRES is supplied, a signal line to which the control signal PTXA is supplied, a signal line to which the control signal PTXB is supplied, and a signal line to which the control signal PSEL is supplied are provided in each row of the pixel array constituting the pixel unit 4. These signal lines are connected to a plurality of pixels P arranged in the corresponding row. For example, the signal line arranged in the i-th row is connected to the pixels P(1,i), ..., P(j,i), P(j+1,i), ..., P(m,i). Each of these signal lines is connected to the vertical scanning unit 3 provided on the signal processing board 12 via an electrical connection part between the pixel substrate 11 and the signal processing board 12.
[0095] 10, one vertical output line Vline is provided in each column of the pixel array constituting the pixel unit 4. For example, a vertical output line Vline(j) is provided in the j-th column. All pixels P arranged in the j-th column are connected to the vertical output line Vline(j). Pixel data is read from the pixels P in the corresponding column to the vertical output line Vline in response to control signals PRES, PTXA, PTXB, and PSEL supplied from the vertical scanning unit 3. Each vertical output line Vline is connected to the AD conversion unit 5 provided on the signal processing board 12 via an electrical connection between the pixel substrate 11 and the signal processing board 12.
[0096] The AD conversion unit 5 has a plurality of (m) column circuits 52 corresponding to each column of the pixel array of n rows and m columns constituting the pixel unit 4. The vertical output line Vline of each column is connected to the column circuit 52 of the corresponding column. For example, the vertical output line Vline(j) of the jth column is connected to the column circuit 52(j) corresponding to the jth column. The horizontal scanning unit 6 has a horizontal scanning circuit 61. The column circuits 52 of each column are connected to the horizontal scanning circuit 61. The horizontal scanning circuit 61 is connected to the data processing unit 7.
[0097] As described above, the AD conversion unit 5, the horizontal scanning unit 6, the data processing unit 7, and the output unit 8 are each divided and arranged in the processing circuit 13 and the processing circuit 14. Of the m×8 column circuits 52 constituting the AD conversion unit 5, half are arranged in the processing circuit 13, and the remaining half are arranged in the processing circuit 14. The vertical output lines Vline of each of the first column to the (m / 2)th column are connected to the column circuits 52 arranged in the processing circuit 13. The vertical output lines Vline of each of the (m / 2+1)th column to the mth column are connected to the column circuits 52 arranged in the processing circuit 14. The horizontal scanning circuit 61 arranged in the processing circuit 13 sequentially horizontally transfers digital data of the pixels P from the first column to the (m / 2)th column, and outputs it to the data processing unit 7 of the processing circuit 13. The horizontal scanning circuit 61 disposed in the processing circuit 14 sequentially transfers digital data of pixels P from the (m / 2+1)th column to the mth column horizontally, and outputs the data to the data processing unit 7 of the processing circuit 14.
[0098] In this embodiment, two types of data are obtained for one exposure of one pixel P. In the readout operation of the pixel P, first, the charge of the photoelectric conversion unit PDA is transferred to the floating diffusion unit FD by the transfer transistor M1A, and the signal output from the pixel P is AD converted by the AD conversion unit 5. The digital data generated in this manner is called data A. Next, the charge of the photoelectric conversion unit PDB is transferred to the floating diffusion unit FD by the transfer transistor M1B, and the signal output from the pixel P is AD converted by the AD conversion unit 5. The digital data generated in this manner is called data B. The data A may be a signal corresponding to the amount of charge generated in the photoelectric conversion unit PDA during a predetermined exposure period. The data B may be a signal corresponding to the amount of charge generated in the photoelectric conversion unit PDA and the photoelectric conversion unit PDB during a predetermined exposure period.
[0099] Next, an example of the configuration of the processing circuit 13 and the processing circuit 14 in the image pickup device according to the present embodiment will be described in more detail with reference to FIG.
[0100] The processing circuit 13 is composed of an AD conversion unit 5L, a horizontal scanning unit 6L, a data processing unit 7L, and an output unit 8L. The processing circuit 14 is composed of an AD conversion unit 5R, a horizontal scanning unit 6R, a data processing unit 7R, and an output unit 8R. The AD conversion units 5L, 5R and the horizontal scanning units 6L, 6R are as described with reference to FIG. 10.
[0101] The data processing unit 7L may be composed of an input data storage unit 71L, a calculation data storage unit 72L, a calculation data input unit 73L, and a calculation unit 74L. Each of the input data storage unit 71L and the calculation data storage unit 72L has a number of memories according to the type of data (here, two types) output by the pixel P. For example, the input data storage unit 71L has two memories 71L1 and 71L2. Also, the calculation data storage unit 72L has two memories 72L1 and 72L2.
[0102] Similarly, the data processing unit 7R may be composed of an input data storage unit 71R, a calculation data storage unit 72R, a calculation data input unit 73R, and a calculation unit 74R. Each of the input data storage unit 71R and the calculation data storage unit 72R has a number of memories corresponding to the type of data (here, two types) output by the pixel P. For example, the input data storage unit 71R has two memories 71R1 and 71R2. Also, the calculation data storage unit 72R has two memories 72R1 and 72R2.
[0103] The horizontal scanning circuit 61 of the horizontal scanning unit 6L is connected to the memory 71L1 and memory 71L2 of the input data storage unit 71L. The horizontal scanning circuit 61 of the horizontal scanning unit 6R is connected to the memory 71R1 and memory 71R2 of the input data storage unit 71R.
[0104] The memory 71L1 of the input data storage unit 71L is connected to the memory 72L1 of the calculation data storage unit 72L. The memory 71L2 of the input data storage unit 71L is connected to the memory 72R1 of the calculation data storage unit 72R. The memory 71R1 of the input data storage unit 71R is connected to the memory 72L2 of the calculation data storage unit 72L. The memory 71R2 of the input data storage unit 71R is connected to the memory 72R2 of the calculation data storage unit 72R.
[0105] The calculation data input unit 73L is connected to the calculation unit 74L. The calculation unit 74L is connected to the output unit 8L. Similarly, the calculation data input unit 73R is connected to the calculation unit 74R. The calculation unit 74R is connected to the output unit 8R.
[0106] Next, the operation of the processing circuits 13 and 14 will be described with reference to FIG. Data A read from the vertical output lines Vline of each of the first to (m / 2)th columns is sequentially transferred by the horizontal scanning circuit 61 of the horizontal scanning unit 6L and stored in the memory 71L1 of the input data storage unit 71L. This data is referred to as data AL.
[0107] Data B read from the vertical output lines Vline of each of the first to (m / 2)th columns is sequentially transferred by the horizontal scanning circuit 61 of the horizontal scanning unit 6L and stored in the memory 71L2 of the input data storage unit 71L. This data is referred to as data BL.
[0108] Similarly, data A read from the vertical output lines Vline of each of the (m / 2+1)th to mth columns is sequentially transferred by the horizontal scanning circuit 61 of the horizontal scanning unit 6R and stored in the memory 71R1 of the input data storage unit 71R. This data is referred to as data AR.
[0109] Data B read from the vertical output lines Vline of each column from the (m / 2+1)th column to the mth column is sequentially transferred by the horizontal scanning circuit 61 of the horizontal scanning unit 6R and stored in the memory 71R2 of the input data storage unit 71R. This data is referred to as data BR.
[0110] Of the data stored in the input data storage unit 71L, the data AL stored in the memory 71L1 is transferred to the memory 72L1 of the calculation data storage unit 72L. Of the data stored in the input data storage unit 71L, the data BL stored in the memory 71L2 is transferred to the memory 72R1 of the calculation data storage unit 72R.
[0111] Among the data stored in the input data storage unit 71R, the data AR stored in the memory 71R1 is transferred to the memory 72L2 of the calculation data storage unit 72L. Among the data stored in the input data storage unit 71R, the data BR stored in the memory 71R2 is transferred to the memory 72R2 of the calculation data storage unit 72R.
[0112] That is, in this embodiment, the data BL is transferred from the processing circuit 13 to the processing circuit 14, and the data AR is transferred from the processing circuit 14 to the processing circuit 13.
[0113] As a result, the calculation data storage unit 72L stores data A (data AL, AR) read from pixels P from the first column to the m-th column, and the calculation data storage unit 72R stores data B (data BL, BR) read from pixels P from the first column to the m-th column.
[0114] The pixel data stored in this manner in the calculation data storage unit 72L is data A (data AL, AR) output from the pixels P of each column. On the other hand, the pixel data stored in this manner in the calculation data storage unit 72R is data B (data BL, BR) output from the pixels P of each column. In other words, two different types of pixel data can be stored separately in the calculation data storage unit 72L and the calculation data storage unit 72R.
[0115] The data stored in the calculation data storage unit 72L is read out by the calculation data input unit 73L, then processed by the calculation unit 74L, and output from the output unit 8L. Similarly, the data stored in the calculation data storage unit 72R is read out by the calculation data input unit 73R, then processed by the calculation unit 74R, and output from the output unit 8R. Since the calculation units 74L and 74R are independent of each other, different calculation processes can be performed simultaneously in each. This makes it possible to separate data corresponding to two different pixel readout methods and output them simultaneously.
[0116] The data B output from the output unit 8L in this way can be used as image generation data. The data A and data B output from the output units 8L and 8R in this way can be used as focus detection data.
[0117] In this way, according to this embodiment, it is possible to realize efficient processing of pixel data without increasing the circuit scale.
[0118] [Fourth embodiment] An imaging device according to a fourth embodiment of the present invention will be described with reference to Figures 12 to 14. Components similar to those of the imaging devices according to the first to third embodiments are given the same reference numerals, and descriptions thereof will be omitted or simplified.
[0119] In this embodiment, an example layout of each functional block on the signal processing board 12 is shown. Figures 12 to 14 are plan views showing an example layout of each functional block on the signal processing board of the imaging device according to this embodiment. The layout example described in this embodiment is applicable to any of the first to third embodiments.
[0120] 12 is an example in which a control unit 2L, an AD conversion unit 5L, a horizontal scanning unit 6L, a data processing unit 7L, and an output unit 8L are arranged in a processing circuit 13, and a control unit 2R, an AD conversion unit 5R, a horizontal scanning unit 6R, a data processing unit 7R, and an output unit 8R are arranged in the processing circuit 14. The control unit 2L, the AD conversion unit 5L, the horizontal scanning unit 6L, the data processing unit 7L, and the output unit 8L are arranged in this order along the vertical direction between the vertical scanning unit 3C and the vertical scanning unit 3L. Similarly, the control unit 2R, the AD conversion unit 5R, the horizontal scanning unit 6R, the data processing unit 7R, and the output unit 8R are arranged in this order along the column direction between the vertical scanning unit 3C and the vertical scanning unit 3R.
[0121] In this layout, the data stored in the data processing unit 7L and the data stored in the data processing unit 7R can be transferred in the horizontal direction as shown by the arrows in Fig. 12. Moreover, the data output from the output units 8L and 8R can be output from one side of the signal processing board 12 along the horizontal direction.
[0122] FIG. 13 is an example in which one control unit 2L or 2R and one data processing unit 7L or 7R are arranged in each of the processing circuits 13 and 14, and two AD conversion units 5L or 5R, horizontal scanning units 6L or 6R, and output units 8L or 8R are arranged in each of the processing circuits 13 and 14. The AD conversion unit 5L, the horizontal scanning unit 6L, the control unit 2L, the data processing unit 7L and the output unit 8L, the horizontal scanning unit 6L, and the AD conversion unit 5L are arranged in this order along the vertical direction between the vertical scanning unit 3C and the vertical scanning unit 3L. Also, the output unit 8L, the control unit 2L, the data processing unit 7L, and the output unit 8L are arranged in this order along the horizontal direction. Similarly, the AD conversion unit 5R, the horizontal scanning unit 6R, the control unit 2R, the data processing unit 7R and the output unit 8R, the horizontal scanning unit 6R, and the AD conversion unit 5R are arranged in this order along the vertical direction between the vertical scanning unit 3C and the vertical scanning unit 3R. Moreover, the output section 8R, the control section 2R, the data processing section 7R, and the output section 8R are arranged in this order along the horizontal direction.
[0123] In this layout, in each of the processing circuits 13 and 14, two AD conversion units 5 are arranged in two physically separated locations, and AD converted data is collected in a data processing unit 7 arranged in the center of each area. Moreover, the data stored in the data processing unit 7L and the data stored in the data processing unit 7R can be transferred in the horizontal direction, as shown by the arrows in Fig. 13. Moreover, the data output from the output units 8L and 8R can be output from the two sides of the signal processing board 12 along the vertical direction.
[0124] 14 is an example in which one control unit 2L or 2R, two AD conversion units 5L or 5R, two horizontal scanning units 6L or 6R, two data processing units 7L or 7R, and two output units 8L or 8R are arranged in each of the processing circuit 13 and the processing circuit 14. The output unit 8L, the data processing unit 7L, the horizontal scanning unit 6L, the AD conversion unit 5L, the control unit 2L, the AD conversion unit 5L, the horizontal scanning unit 6L, the data processing unit 7L, and the output unit 8L are arranged in this order along the vertical direction between the vertical scanning unit 3C and the vertical scanning unit 3L. Similarly, the output unit 8R, the data processing unit 7R, the horizontal scanning unit 6R, the AD conversion unit 5R, the control unit 2R, the AD conversion unit 5R, the horizontal scanning unit 6R, the data processing unit 7R, and the output unit 8R are arranged in this order along the vertical direction between the vertical scanning unit 3C and the vertical scanning unit 3R.
[0125] In this layout, the data stored in the data processing unit 7L and the data stored in the data processing unit 7R can be transferred in the horizontal direction as shown by the arrows in Fig. 14. Moreover, the data output from the output units 8L and 8R can be output from the two sides of the signal processing board 12 along the horizontal direction.
[0126] [Fifth embodiment] An imaging system according to a fifth embodiment of the present invention will be described with reference to Fig. 15. Fig. 15 is a block diagram showing a schematic configuration of the imaging system according to this embodiment.
[0127] The imaging device 10 described in the first to fourth embodiments can be applied to various imaging systems. Examples of the applicable imaging systems include digital still cameras, digital camcorders, security cameras, copiers, fax machines, mobile phones, car-mounted cameras, and observation satellites. Camera modules equipped with an optical system such as a lens and an imaging device are also included in the imaging system. FIG. 15 illustrates a block diagram of a digital still camera as an example of these.
[0128] 15 includes an imaging device 201, a lens 202 that forms an optical image of a subject on the imaging device 201, an aperture 204 that varies the amount of light passing through the lens 202, and a barrier 206 that protects the lens 202. The lens 202 and the aperture 204 form an optical system that focuses light on the imaging device 201. The imaging device 201 is the imaging device 10 described in any one of the first to fourth embodiments, and converts the optical image formed by the lens 202 into image data.
[0129] The imaging system 200 also includes a signal processing unit 208 that processes an output signal output from the imaging device 201. The signal processing unit 208 generates image data from a digital signal output from the imaging device 201. The signal processing unit 208 also performs various corrections and compression as necessary to output image data. The imaging device 201 may include an AD conversion unit that generates a digital signal to be processed by the signal processing unit 208. The AD conversion unit may be formed in a semiconductor layer (semiconductor substrate) in which a photoelectric conversion unit of the imaging device 201 is formed, or may be formed in a semiconductor substrate different from the semiconductor layer in which the photoelectric conversion unit of the imaging device 201 is formed. The signal processing unit 208 may also be formed in the same semiconductor substrate as the imaging device 201.
[0130] The imaging system 200 further includes a memory unit 210 for temporarily storing image data, and an external interface unit (external I / F unit) 212 for communicating with an external computer or the like. The imaging system 200 further includes a recording medium 214 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 216 for recording or reading out on the recording medium 214. The recording medium 214 may be built into the imaging system 200, or may be removable.
[0131] Furthermore, the imaging system 200 has an overall control / calculation unit 218 that performs various calculations and controls the entire digital still camera, and a timing generation unit 220 that outputs various timing signals to the imaging device 201 and the signal processing unit 208. Here, the timing signals and the like may be input from outside, and the imaging system 200 only needs to have at least the imaging device 201 and the signal processing unit 208 that processes the output signal output from the imaging device 201.
[0132] The imaging device 201 outputs an imaging signal to the signal processing unit 208. The signal processing unit 208 performs predetermined signal processing on the imaging signal output from the imaging device 201, and outputs image data. The signal processing unit 208 generates an image using the imaging signal.
[0133] In this way, according to this embodiment, it is possible to realize an imaging system to which the imaging device 10 according to the first to fourth embodiments is applied.
[0134] [Sixth embodiment] An imaging system and a moving object according to a sixth embodiment of the present invention will be described with reference to Fig. 16. Fig. 16 is a diagram showing the configuration of an imaging system and a moving object according to this embodiment.
[0135] FIG. 16(a) shows an example of an imaging system related to an in-vehicle camera. The imaging system 300 has an imaging device 310. The imaging device 310 is the imaging device 10 described in any one of the first to fourth embodiments. The imaging system 300 has an image processing unit 312 that performs image processing on a plurality of image data acquired by the imaging device 310, and a parallax acquisition unit 314 that calculates parallax (phase difference of parallax images) from the plurality of image data acquired by the imaging system 300. The imaging system 300 also has a distance acquisition unit 316 that calculates a distance to an object based on the calculated parallax, and a collision determination unit 318 that determines whether or not there is a possibility of a collision based on the calculated distance. Here, the parallax acquisition unit 314 and the distance acquisition unit 316 are examples of distance information acquisition means that acquire distance information to an object. That is, the distance information is information related to the parallax, the defocus amount, the distance to the object, and the like. The collision determination unit 318 may determine the possibility of a collision using any of these distance information. The distance information acquisition means may be realized by dedicated hardware, a software module, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a combination of these.
[0136] The imaging system 300 is connected to a vehicle information acquisition device 320, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The imaging system 300 is also connected to a control ECU 330, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 318. The imaging system 300 is also connected to an alarm device 340 that issues an alarm to the driver based on the judgment result of the collision judgment unit 318. For example, when the judgment result of the collision judgment unit 318 indicates that there is a high possibility of a collision, the control ECU 330 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 340 warns the user by sounding an alarm, displaying alarm information on the screen of a car navigation system, etc., and vibrating the seat belt or steering wheel.
[0137] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the imaging system 300. Fig. 16(b) shows an imaging system for imaging the area in front of the vehicle (imaging range 350). A vehicle information acquisition device 320 sends instructions to the imaging system 300 or imaging device 310. This configuration can further improve the accuracy of distance measurement.
[0138] Although the above describes an example of control to prevent collision with other vehicles, the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to prevent deviation from lanes, etc. Furthermore, the imaging system is not limited to vehicles such as the vehicle itself, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the present invention can be applied not only to moving bodies, but also to devices that use object recognition widely, such as intelligent transport systems (ITS).
[0139] [Modified embodiment] The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, an example in which part of the configuration of any one of the embodiments is added to another embodiment, or an example in which part of the configuration of another embodiment is substituted therefor, is also an embodiment of the present invention.
[0140] In addition, in the above embodiment, an example of application to a stacked type imaging device in which a pixel substrate and a signal processing substrate are stacked has been described; however, the present invention is not necessarily limited to stacked type imaging devices, and can be widely applied to imaging devices in which a signal processing circuit is divided into multiple regions.
[0141] In the above embodiment, the signal processing circuit is divided into two regions having different corresponding columns, and data is exchanged between these regions, but the manner of dividing the signal processing circuit is not limited to this. For example, the signal processing circuit may be divided into three or more regions having different corresponding columns, and data may be exchanged between these regions. In addition to the division in the horizontal direction, the signal processing circuit may be divided into two or more regions having different corresponding rows.
[0142] In addition, in the above first to fourth embodiments, a device intended to obtain an image, that is, an imaging device, is exemplified, but the application examples of the present invention are not necessarily limited to imaging devices. For example, when applied to a device whose main purpose is distance measurement as described in the above sixth embodiment, it is not necessarily necessary to output an image. In such a case, the device can be said to be a photoelectric conversion device that converts optical information into a predetermined electrical signal. An imaging device is one type of photoelectric conversion device.
[0143] Furthermore, the imaging systems shown in the fifth and sixth embodiments are examples of imaging systems to which the photoelectric conversion device of the present invention can be applied, and imaging systems to which the photoelectric conversion device of the present invention can be applied are not limited to the configurations shown in Figures 15 and 16.
[0144] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0145] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]
[0146] 1... Processing equipment 2...Control section 3,3L,3C,3R…Vertical scanning section 4. Pixel section 5,5L,5R…AD conversion section 6,6L,6R...Horizontal scanning section 7, 7L, 7R...Data processing section 8, 8L, 8R...Output section 10...Imaging device 11...Pixel substrate 12...Signal processing board 13, 14...Processing circuit
Claims
1. a pixel section in which a plurality of pixels each having a photoelectric conversion unit are arranged in a plurality of rows and a plurality of columns; A processing circuit that processes a signal output from the pixel unit, the pixel unit has a first region and a second region defined by columns, and is configured to output a plurality of types of signals including a first signal and a second signal from each of the columns belonging to the first region and the second region; The processing circuit includes a first processing circuit including a first memory and a third memory, a second processing circuit including a second memory and a fourth memory, and a data exchange circuit; the first memory is configured to store the first signal and the second signal for each column output from the first region; the second memory is configured to store the first signal and the second signal for each column output from the second area; the data exchange circuit is configured to read the first signal stored in the first memory and the first signal stored in the second memory, and store them in the third memory, and to read the second signal stored in the first memory and the second signal stored in the second memory, and to store them in the fourth memory; The processing circuit is configured to output the first signals of the first area and the second area stored in the third memory from the first processing circuit, and to output the second signals of the first area and the second area stored in the fourth memory from the second processing circuit. A photoelectric conversion device comprising:
2. The processing circuit is configured to simultaneously output the first signals of the first region and the second region stored in the third memory and the second signals of the first region and the second region stored in the fourth memory.
2. The photoelectric conversion device according to claim 1.
3. the first signal is a signal obtained by adding up signals of a first number of pixels; The second signal is a signal obtained by adding together signals of a second number of pixels different from the first number.
3. The photoelectric conversion device according to claim 1 or 2.
4. A plurality of output lines including a first output line and a second output line are arranged in each column of the pixel section, and the first signal is output to the first output line, and the second signal is output to the second output line.
4. The photoelectric conversion device according to claim 3.
5. the first signal is a signal output from a pixel belonging to a first row, The second signal is a signal output from a pixel belonging to a second row different from the first row.
3. The photoelectric conversion device according to claim 1 or 2.
6. The first row and the second row are adjacent rows.
6. The photoelectric conversion device according to claim 5.
7. the first signal is a signal output from a pixel of a first color; The second signal is a signal output from a pixel of a second color different from the first color.
7. The photoelectric conversion device according to claim 5 or 6.
8. Each of the plurality of pixels has a first photoelectric conversion unit and a second photoelectric conversion unit, the first signal is a signal for focus detection based on charges generated in the first photoelectric conversion unit, The second signal is a signal for generating an image based on charges generated in the first photoelectric conversion unit and the second photoelectric conversion unit.
3. The photoelectric conversion device according to claim 1 or 2.
9. The processing circuit further includes an analog-to-digital conversion unit, The first signal and the second signal are digital signals converted from analog signals by the analog-to-digital converter.
9. The photoelectric conversion device according to claim 1,
10. The first processing circuit and the second processing circuit are disposed apart from each other.
10. The photoelectric conversion device according to claim 1,
11. A vertical scanning unit that drives the plurality of pixels on a row-by-row basis is disposed between the first processing circuit and the second processing circuit.
11. The photoelectric conversion device according to claim 1,
12. A pixel substrate on which the pixel unit is provided and a signal processing substrate on which the processing circuit is provided are laminated.
12. The photoelectric conversion device according to claim 1,
13. The photoelectric conversion device according to claim 1 , a signal processing unit that processes a signal output from the photoelectric conversion device; An imaging system comprising:
14. A mobile object, The photoelectric conversion device according to claim 8 ; a distance information acquiring unit that acquires distance information to an object from a parallax image based on the first signal and the second signal; a control means for controlling the moving object based on the distance information; A moving object comprising:
Citation Information
Patent Citations
Imaging apparatus
JP2004350265A
Solid state imaging device
JP2005277513A
Electronic camera
JP2006109374A
Solid state image pickup device, manufacturing method of solid state image pickup device, and electronic apparatus
JP2015216334A
Imaging apparatus and imaging system
JP2018139374A
