Photoelectric conversion device, photoelectric conversion system and mobile body
Patent Information
- Application Number
- JP2020157865
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-09-18
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2040-09-18
AI Technical Summary
【0007】 本発明によれば、1フレーム期間に複数の走査を行う構成において、より簡便な制御を実現し得る光電変換装置、光電変換システム及び移動体を提供することができる。
Smart Images

Figure 0007587380000001 
Figure 0007587380000002 
Figure 0007587380000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a photoelectric conversion device, a photoelectric conversion system, and a moving object. [Background technology]
[0002] Patent Document 1 discloses a CMOS image sensor that scans a two-dimensional pixel array by pixel row by pixel row using a rolling shutter. Patent Document 1 also discloses, as an embodiment, an example in which the readout operation of a distance measurement signal used for AF control is performed during a blank period of the readout operation of an image to be captured. In this example, AF control can be performed during the image capture operation. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2004-140479 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, in a complex scanning method in which multiple scans are performed in one frame period as disclosed in Patent Document 1, communication of information related to scan settings becomes complicated, and it may be difficult to secure sufficient communication time.
[0005] Therefore, an object of the present invention is to provide a photoelectric conversion device, a photoelectric conversion system, and a moving body that can realize simpler control in a configuration in which multiple scans are performed in one frame period. [Means for solving the problem]
[0006] According to one aspect of the present invention, there is provided a method for manufacturing a pixel array including the steps of: a plurality of pixels arranged in a plurality of rows; a scanning unit that performs first and second scanning for sequentially outputting signals from the plurality of pixels for each row; and a scanning unit that performs the first scanning and the second scanning in response to an external input. Readout operation of multiple rows of pixels ina first storage unit for storing a first setting value indicating a setting of the second scanning Readout operation of multiple rows of pixels in and a second memory unit that stores a second setting value indicating a setting of, wherein the scanning unit performs the first scanning based on the first setting value and the second scanning based on the second setting value in one frame period, and the storage of the first setting value by the first memory unit and the storage of the second setting value by the second memory unit are both performed prior to the start of the first scanning and the start of the second scanning. Effect of the Invention
[0007] According to the present invention, it is possible to provide a photoelectric conversion device, a photoelectric conversion system, and a moving body that can realize simpler control in a configuration in which multiple scans are performed in one frame period. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of a photoelectric conversion device according to a first embodiment. [Diagram 2] 2 is an equivalent circuit diagram showing a configuration example of a pixel according to the first embodiment. FIG. [Diagram 3] 2 is a block diagram showing an example of the configuration of a storage unit group according to the first embodiment; FIG. [Figure 4] 4 is a timing chart showing a method of driving one row of pixels according to the first embodiment. [Diagram 5] 4 is a timing chart showing a method of driving a pixel unit according to the first embodiment. [Figure 6] FIG. 13 is a block diagram showing an example of the configuration of a storage unit group according to a second embodiment. [Figure 7] 10 is a timing chart showing a method of driving a pixel unit according to the second embodiment. [Figure 8] FIG. 11 is a block diagram showing a schematic configuration of an imaging system according to a third embodiment. [Figure 9] FIG. 13 is a diagram illustrating an example of the configuration of an imaging system and a moving object according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The same elements or corresponding elements in multiple drawings are denoted by the same reference numerals, and the description thereof may be omitted or simplified.
[0010] [First embodiment] 1 is a block diagram showing an example of the configuration of a photoelectric conversion device 1 according to this embodiment. The photoelectric conversion device 1 has a communication unit 11, a memory group 12, a control unit 13, a vertical scanning unit 14, a pixel unit 15, a column circuit 16, a horizontal scanning unit 17, a data processing unit 18, and a signal output unit 19. In this embodiment, an example of a solid-state imaging device, which is a type of photoelectric conversion device 1 to which the present invention can be applied, is described, but the present invention is not limited to this. Examples of photoelectric conversion devices to which the present invention can be applied include solid-state imaging devices, focus detection devices, distance measuring devices, and TOF (Time-Of-Flight) cameras.
[0011] In the pixel section 15, a plurality of pixels P are arranged across a plurality of rows and a plurality of columns. FIG. 1 shows pixels P(1,1) to P(n,m) arranged two-dimensionally in n rows and m columns. Here, in the notation of pixel P(X,Y), X represents the row number and Y represents the column number. The row number of the first row is the first row, and the column number of the first column is the first column. In this specification, when a pixel P arranged at a specific position in the pixel section 15 is indicated, it is expressed together with the row number and column number, such as "pixel P(n,m)", and when the position in the pixel section 15 is not specified, it is expressed simply as "pixel P".
[0012] The vertical scanning unit 14 (scanning unit) is connected to the pixels P via control lines provided for each row. The vertical scanning unit 14 is a circuit unit that supplies control signals for driving readout circuits in the pixels P to the pixels P via the control lines. The vertical scanning unit 14 operates upon receiving a signal from the control unit 13, and performs vertical scanning to sequentially output signals from the pixels P of the pixel unit 15. The vertical scanning includes a shutter operation and a readout operation. The shutter operation in this vertical scanning refers to an operation of sequentially resetting and releasing the reset of the photoelectric conversion elements for some or all of the rows of the pixel unit 15 on a row-by-row basis, thereby starting exposure (accumulation of electric charge). The readout operation in this vertical scanning refers to an operation of sequentially outputting pixel signals based on electric charge accumulated in the photoelectric conversion elements for some or all of the rows of the pixel unit 15 on a row-by-row basis.
[0013] The column circuit 16 includes an amplifier circuit, an analog-to-digital conversion (AD conversion) circuit, a memory, etc. The column circuit 16 amplifies pixel signals output from the pixels P via output lines arranged corresponding to each column, performs AD conversion, and stores the signals in the memory as digital pixel signals.
[0014] The horizontal scanning unit 17 operates upon receiving a signal from the control unit 13, and sequentially outputs a control signal to the memory of each column of the column circuit 16. Upon receiving a control signal from the horizontal scanning unit 17, the column circuit 16 outputs the digital pixel signal held in the memory of the corresponding column to the data processing unit 18.
[0015] The data processing unit 18 is a digital circuit that performs predetermined digital signal processing, parallel-serial conversion, etc. The processed data is output to a signal output unit 19. The signal output unit 19 includes an external interface such as an LVDS (Low Voltage Differential Signaling) circuit. The signal output unit 19 outputs the acquired digital pixel signal to the outside of the photoelectric conversion device 1 as serial data.
[0016] The control unit 13 is a circuit unit for supplying control signals to the vertical scanning unit 14, the column circuit 16, the horizontal scanning unit 17, etc., for controlling the operations and operation timings performed by these units.
[0017] The CPU 2 is an arithmetic processing circuit that controls the operation of the photoelectric conversion device 1. The CPU 2 can be provided, for example, in a photoelectric conversion system in which the photoelectric conversion device 1 is mounted, that is, outside the photoelectric conversion device 1. The CPU 2 transmits a synchronization signal to the control unit 13. The CPU 2 also transmits setting values indicating settings such as scanning timing to the communication unit 11. The setting values received by the communication unit 11 are stored in the memory unit group 12 and then supplied to the control unit 13.
[0018] FIG. 2 is an equivalent circuit diagram showing a configuration example of a pixel P according to this embodiment. The pixel P includes photoelectric conversion units 151a and 151b, transfer transistors 152a and 152b, a reset transistor 153, an amplification transistor 154, and a selection transistor 155. In addition, one microlens is provided corresponding to one pixel P. In other words, the two photoelectric conversion units 151a and 151b share one microlens. As a result, the photoelectric conversion units 151a and 151b are arranged to receive light that has passed through different exit pupils of the optical system. Note that FIG. 2 shows one pixel P arranged in the kth column of the multiple pixels P as an example, but the same applies to the other pixels P. Note that "k" in parentheses in the names of the control signals shown in FIG. 2 indicates the row number. That is, k is an integer between 1 and n.
[0019] The photoelectric conversion units 151a and 151b are photoelectric conversion elements, for example, photodiodes. The anodes of the photodiodes constituting the photoelectric conversion units 151a and 151b are connected to a ground node. The cathodes of the photodiodes constituting the photoelectric conversion units 151a and 151b are connected to the sources of the transfer transistors 152a and 152b, respectively. The drains of the transfer transistors 152a and 152b are connected to the source of the reset transistor 153 and the gate of the amplification transistor 154. A connection node of the drains of the transfer transistors 152a and 152b, the source of the reset transistor 153, and the gate of the amplification transistor 154 is a so-called floating diffusion FD. The floating diffusion FD includes a capacitance component, functions as a charge holding unit, and constitutes a charge-voltage conversion unit made of this capacitance component. The drain of the reset transistor 153 and the drain of the amplification transistor 154 are connected to a power supply node (voltage VDD). The source of the amplification transistor 154 is connected to the drain of the selection transistor 155. The source of the selection transistor 155 is connected to an output line 156. A current source (not shown) is connected to the output line 156. Note that the names of the source and drain of a transistor may differ depending on the conductivity type or the function of interest of the transistor, and the above-mentioned source and drain may be called by the opposite names.
[0020] 2, the control lines for each row include two transfer gate signal lines, a reset signal line, and a selection signal line. The two transfer gate signal lines are connected to the gates of the transfer transistors 152a and 152b of the pixels P belonging to the corresponding rows, and supply the control signals PTXA and PTXB supplied from the vertical scanning unit 14 to the gates of the transfer transistors 152a and 152b. For example, the pixel P in the kth row is supplied with the control signals PTXA(k) and PTXB(k) via the two transfer gate signal lines in the kth row.
[0021] The reset signal line is connected to the gate of the reset transistor 153 of the pixel P belonging to the corresponding row, and supplies a control signal PRES supplied from the vertical scanning unit 14 to the gate of the reset transistor 153. For example, a control signal PRES(k) is supplied to a pixel P in the kth row via the kth row reset signal line.
[0022] The selection signal line is connected to the gates of the selection transistors 155 of the pixels P belonging to the corresponding row, and supplies a control signal PSEL supplied from the vertical scanning unit 14 to the gates of the selection transistors 155. For example, a control signal PSEL(k) is supplied to the pixel P in the kth row via the selection signal line in the kth row.
[0023] When each transistor of pixel P is configured as an N-channel transistor, the corresponding transistor turns on when a high-level control signal is supplied from vertical scanning unit 14, and the corresponding transistor turns off when a low-level control signal is supplied from vertical scanning unit 14. Here, a high level corresponds to a logical value "1" and a low level corresponds to a logical value "0." Note that each transistor of pixel P may be configured as a P-channel transistor. When each transistor of pixel P is configured as a P-channel transistor, the signal level of the control signal that drives each transistor is opposite to that in the case of N-channel transistors.
[0024] When an optical image of a subject is incident on the pixel unit 15, the photoelectric conversion units 151a and 151b of each pixel P convert the incident light into an electric charge in an amount corresponding to the amount of light (photoelectric conversion) and accumulate the generated electric charge. When the transfer transistors 152a and 152b are turned on, they transfer the electric charges accumulated in the photoelectric conversion units 151a and 151b, respectively, to the floating diffusion FD.
[0025] The floating diffusion FD holds the charge transferred from the photoelectric conversion units 151a and 151b, and converts the charge to a voltage according to the amount of charge transferred from the photoelectric conversion units 151a and 151b through charge-voltage conversion by its capacitance component. The amplifier transistor 154 is configured such that a voltage VDD is supplied to its drain and a bias current is supplied to its source via a selection transistor 155, forming an amplifier unit (source follower circuit) with the gate as an input node. As a result, the amplifier transistor 154 outputs a pixel signal based on the voltage of the floating diffusion FD to an output line 156 via the selection transistor 155.
[0026] The reset transistor 153, when turned on, resets the floating diffusion FD to a voltage corresponding to the voltage VDD. In addition, the reset transistor 153 and the transfer transistors 152a and 152b are turned on simultaneously, so that the photoelectric conversion units 151a and 151b can be reset to a voltage corresponding to the voltage VDD.
[0027] 3 is a block diagram showing an example of the configuration of the memory group 12 according to this embodiment. The memory group 12 has a memory M1 (third memory), a memory M2 (first memory), a memory S1 (fourth memory), a memory S2 (second memory), and a selection unit SL. The memory units M1, M2, S1, and S2 are storage elements that store information including setting values related to scanning. The selection unit SL is a selection circuit that selects and outputs one of a plurality of input signals according to an input selection flag.
[0028] In response to an output signal from CPU2, communication unit 11 outputs to storage units M1 and S1 control signals for controlling data input / output in these storage units M1 and S1, and data (DATA IN) relating to setting values indicating settings such as scanning timing. Storage units M1 and S1 can each store setting values for different scans performed within one frame period. Here, the setting value stored in storage unit M1 is referred to as data M1, and the setting value stored in storage unit S1 is referred to as data S1.
[0029] Storage unit M1 outputs data M1 to storage unit M2, and storage unit S1 outputs data S1 to storage unit S2. Storage unit M2 updates the setting value stored in storage unit M2 by storing data M1 input from storage unit M1 at a timing based on an update flag input from control unit 13. Storage unit S2 updates the setting value stored in storage unit S2 by storing data S1 input from storage unit S1 at a timing based on an update flag input from control unit 13. Here, the setting value stored in storage unit M2 is referred to as data M2, and the setting value stored in storage unit S2 is referred to as data S2.
[0030] The storage units M2 and S2 output the data M2 and S2, respectively, to the selection unit SL. The selection unit SL selectively validates one of the data M2 and the data S2 based on a selection flag input from the control unit 13. The selection unit SL outputs the selected data to the control unit 13 as data OUT.
[0031] Fig. 4(a) is a timing chart showing an example of a method for driving a row of pixels P according to the first embodiment. Fig. 4(b) is a timing chart showing another example of a method for driving a row of pixels P according to the first embodiment. Figs. 4(a) and 4(b) show an outline of the shutter operation and readout operation in each row of the pixel section 15. Note that Figs. 4(a) and 4(b) only show control signals related to the kth row of pixels P.
[0032] First, a first example of a driving method will be described with reference to Fig. 4(a). This example is a driving method in which a pixel signal (image A) based only on the charge generated by one photoelectric conversion unit 151a and a pixel signal (image A+B) based on the charge generated by both the photoelectric conversion unit 151a and the photoelectric conversion unit 151b are successively read out.
[0033] In the shutter operation, the control signals PRES(k), PTXA(k), and PTXB(k) are controlled to a high level, and the transfer transistors 152a, 152b and the reset transistor 153 are simultaneously turned on. This connects the photoelectric conversion units 151a and 151b to a power supply node via the transfer transistors 152a, 152b and the reset transistor 153, and resets the photoelectric conversion units 151a and 151b to a potential corresponding to the voltage VDD. After this, the timing at which the control signals PTXA(k) and PTXB(k) are controlled to a low level to release the reset of the photoelectric conversion units 151a and 151b corresponds to the start time of the charge accumulation period in the photoelectric conversion units 151a and 151b.
[0034] In the readout operation of the A image, the control signal PRES(k) is controlled to a low level, and the control signals PSEL(k) and PTXA(k) are controlled to a high level, and with the reset transistor 153 in an off state, the transfer transistor 152a and the selection transistor 155 are simultaneously turned on. This causes the charge accumulated in the photoelectric conversion unit 151a to be transferred to the floating diffusion FD. The amplification transistor 154 outputs a pixel signal of the A image according to the potential of the floating diffusion FD to the output line 156 via the selection transistor 155. In this way, the A image is read out based on the charge generated in the photoelectric conversion unit 151a.
[0035] Following the readout operation of the A image, the readout operation of the A+B image is performed. In the readout operation of the A+B image, the control signal PRES(k) is controlled to a low level, and the control signals PSEL(k), PTXA(k), and PTXB(k) are controlled to a high level. At this time, the reset transistor 153 is in an off state, and the transfer transistors 152a, 152b and the selection transistor 155 are simultaneously turned on. As a result, the charge accumulated in the photoelectric conversion unit 151a and the charge accumulated in the photoelectric conversion unit 151b are transferred to the floating diffusion FD. The amplification transistor 154 outputs a pixel signal of the A+B image according to the potential of the floating diffusion FD to the output line 156 via the selection transistor 155. In this way, the A+B image is read out based on the charge generated in the photoelectric conversion units 151a and 151b.
[0036] By subtracting the pixel signal of the A image from the pixel signal of the A+B image, a pixel signal (B image) based on the charge generated in the photoelectric conversion unit 151b can be obtained. The pixel signals of the A image and the B image obtained in this manner can be used for distance measurement purposes such as focus detection. The result of focus detection can be used for the focusing operation of the imaging system. Furthermore, the pixel signals of the A+B images can be used to generate an image. In this way, in this example, pixel signals for distance measurement and pixel signals for images can be obtained.
[0037] Next, a second example of the driving method will be described with reference to FIG. 4(b). This example is a driving method in which only pixel signals (A+B images) based on charges generated by both the photoelectric conversion unit 151a and the photoelectric conversion unit 151b are read out. In this example, the readout operation of the A image is omitted compared to the first example shown in FIG. 4(a). Other points are generally similar to FIG. 4(b), so the description will be omitted. In this example, it is not possible to obtain pixel signals of the A and B images, but pixel signals for an image can be obtained by a simpler driving method.
[0038] In addition, in Fig. 4(a), after acquiring the pixel signal of the image A, the pixel signal of the image B may be acquired instead of the image A+B. Even with this driving method, it is possible to acquire pixel signals for distance measurement. Also, by acquiring the pixel signals of the image A and the pixel signals of the image B and then adding them, it is possible to acquire pixel signals of the image A+B.
[0039] Although not explicitly shown in Figures 4(a) and 4(b), the N signal may be read out immediately after the control signal PRES(k) is set to low level, i.e., before the control signals PTXA(k) and PTXB(k) are set to high level. The N signal is a signal that corresponds to the potential of the floating diffusion after reset is released, and indicates the level of reset noise that may occur due to reset. The effect of reset noise can be reduced by performing correlated double sampling processing in which the N signal is subtracted from the pixel signals of the A image, the B image, or the A+B image.
[0040] FIG. 5 is a timing chart showing a driving method of the pixel unit 15 according to this embodiment. In this embodiment, an example will be described in which a pixel signal for an image (A+B image) is obtained from a pixel P in the (2k-1)th row by the first vertical scanning (first scanning), and a pixel signal for distance measurement (A image and A+B image) is obtained from a pixel P in the (10k-8)th row by the second vertical scanning (second scanning). That is, the driving method of each row in the first vertical scanning is, for example, as shown in FIG. 4(b), and the driving method of each row in the second vertical scanning is, for example, as shown in FIG. 4(a). Note that the type of vertical scanning, the number of the row to be read out, the ratio of the rows to be read out, and the like are merely examples and are not limited to those shown as examples.
[0041] The upper part of Fig. 5 shows the timing at which the vertical synchronization signal VD and the horizontal synchronization signal HD are input. The boxes in "Setting value communication" in the middle part of Fig. 5 show the setting values input from the communication unit 11 to the memory unit group 12. The boxes in "Memory unit M1" to "Memory unit S2" in the middle part of Fig. 5 show the setting values stored in the corresponding memory units. The boxes in "Selection unit" in the middle part of Fig. 5 show the setting values output from the selection unit SL. The lower part of Fig. 5 shows the timing of the shutter operation and readout operation for the pixels in each pixel row. The boxes in each pixel row show the period between the shutter operation and the readout operation.
[0042] First, a description will be given of the operation of the first frame (from time T1 to time T2) shown in Fig. 5. The first frame shows an example in which only the first vertical scanning is performed, and the second vertical scanning is not performed.
[0043] At time T1, the readout operation of the first vertical scan is started in synchronization with the vertical synchronization signal VD. Note that it is assumed that the shutter operation has been performed in advance in the frame before the first frame, that is, in the period before time T1.
[0044] During the period from time T1 to time T1m1, a readout operation of pixel signals of the A+B images for the image is performed from pixels P in the first row in synchronization with the horizontal synchronization signal HD. Next, during the period from time T1m1 to time T1m2, a readout operation of pixel signals of the A+B images for the image is performed from pixels P in the third row in synchronization with the horizontal synchronization signal HD. Similarly, a readout operation of pixels P in the (2k-1)th row is sequentially performed in synchronization with the horizontal synchronization signal HD.
[0045] Also, in the first frame, the shutter operation of the first vertical scan starts at time T1ms1. This is to start charge accumulation for the readout operation of the second frame. At time T1ms1, a shutter operation is performed in synchronization with the horizontal synchronization signal HD to release the reset of the pixels P in the first row. Next, at time T1ms2, a shutter operation is performed in synchronization with the horizontal synchronization signal HD to release the reset of the pixels P in the third row. Similarly, the shutter operation of the pixels P in the (2k-1)th row is sequentially performed in synchronization with the horizontal synchronization signal HD.
[0046] Next, the operation of the second frame (time T2 to time T3) shown in Fig. 5 will be described. The second frame shows an example in which one first vertical scan and four second vertical scans are performed. Note that before both the first vertical scan and the second vertical scan, the M setting 2 is stored in the memory unit M2, which is the first memory unit, and the S setting 2 is stored in the memory unit S2, which is the second memory unit.
[0047] At time T2, a first vertical scanning readout operation is started in synchronization with the vertical synchronization signal VD. During the period from time T2 to time T2m1, a readout operation of pixel signals of the A+B images for images is performed from the pixel P in the first row in synchronization with the horizontal synchronization signal HD. Similarly, readout operations of the pixels P in the (2k-1)th row are sequentially performed in synchronization with the horizontal synchronization signal HD.
[0048] Also, at time T2ss3, a shutter operation is started in synchronization with the horizontal synchronization signal HD to release the reset of the pixels P in the first row. This is to start charge accumulation for the readout operation of the third frame. At time T2ss3, a shutter operation is performed in synchronization with the horizontal synchronization signal HD to release the reset of the pixels P in the first row. Similarly, the shutter operation of the pixels P in the (2k-1)th row is sequentially performed in synchronization with the horizontal synchronization signal HD.
[0049] As described above, in this example, the first vertical scanning in the second frame is generally similar to the first vertical scanning in the first frame. However, the setting contents of the first vertical scanning may be different between frames.
[0050] In the second frame, four second vertical scans are performed in parallel with the first vertical scan described above. The first second vertical scan is started at time T2ss1. At time T2ss1, a shutter operation is performed to release the reset of the pixels P in the second row in synchronization with the horizontal synchronization signal HD. During the period from time T2s1 to time T2s11, a read operation of the pixel signals of the A image for distance measurement is performed from the pixels P in the second row in synchronization with the horizontal synchronization signal HD. Thereafter, during the period from time T2s11 to time T2s12, a read operation of the pixel signals of the A+B images for distance measurement and imaging is performed from the pixels P in the second row in synchronization with the horizontal synchronization signal HD. As described above, the pixel signals of the B image can be calculated from the pixel signals of the A+B images and the pixel signals of the A image, and the pixel signals of the A image and the B image can be used for distance measurement.
[0051] At a time two cycles of the horizontal synchronization signal HD after time T2ss1, a shutter operation is performed on the pixels in the 12th row. Thereafter, a similar readout operation of two pixel signals is performed on the pixels P in the 12th row. Similarly, a shutter operation and a readout operation of two pixel signals are sequentially performed on the pixels P in the (10k-8)th row every two cycles of the horizontal synchronization signal HD.
[0052] The second second vertical scan starts from time T2ss2, the third second vertical scan starts from time T2ss3, and the fourth second vertical scan starts from time T2ss4. The contents of the second to fourth second vertical scans are similar to those of the first second vertical scan, so a description thereof will be omitted.
[0053] Next, a description will be given of the operation of the third frame (from time T3 to time T4) shown in Fig. 5. The third frame is an example in which only the first vertical scan is performed and the second vertical scan is not performed, and is similar to the first frame, so a description will be omitted.
[0054] Next, the operation of the memory unit group 12 regarding the setting of the readout operation in the above-mentioned first vertical scanning and second vertical scanning will be described with reference to Fig. 5. Here, "M setting #" (# is a natural number) in Fig. 5 means the setting value (first setting value) of the first vertical scanning, and "S setting #" (# is a natural number) means the setting value (second setting value) of the second vertical scanning.
[0055] At time T1 of the first frame, M setting 1 is stored in memory units M1 and M2, and S setting 1 is stored in memory units S1 and S2. Here, selection unit SL selects M setting 1 stored in memory unit M2 and outputs it to control unit 13. As a result, the readout operation of the first vertical scan, which starts at time T1, is performed based on M setting 1.
[0056] Between time T1 and time T2, i.e., during the period of the first frame, the CPU 2 transmits an M setting 2, which is a setting value for the first vertical scanning in the second frame, to the communication unit 11. Immediately after the communication unit 11 receives the M setting 2, the M setting 2 is stored in the memory unit M1. Thereafter, the CPU 2 transmits an S setting 2, which is a setting value for the second vertical scanning in the second frame, to the communication unit 11 during the period of the first frame. Immediately after the communication unit 11 receives the S setting 2, the S setting 2 is stored in the memory unit S1.
[0057] After that, at time T2, the M setting 2 stored in memory unit M1 is stored in memory unit M2, and the S setting 2 stored in memory unit S1 is stored in memory unit S2. At this time, the output of selection unit SL also changes from M setting 1 to M setting 2. As a result, the M setting 2 is applied to the readout operation of the first vertical scan that starts at time T2.
[0058] Next, at time T2s1, the signal selection by the selection unit SL is switched, and the output of the selection unit SL changes from the M setting 2 to the S setting 2. As a result, the S setting 2 is applied to the readout operation of the second vertical scanning that starts at time T2s1.
[0059] Between time T2 and time T3, i.e., during the second frame period, CPU 2 transmits M setting 3, which is the setting value for the first vertical scanning in the third frame, to communication unit 11. Immediately after communication unit 11 receives M setting 3, M setting 3 is stored in memory unit M1. Thereafter, CPU 2 transmits S setting 3, which is the setting value for the second vertical scanning in the third frame, to communication unit 11 during the second frame period. Immediately after communication unit 11 receives S setting 3, S setting 3 is stored in memory unit S1.
[0060] After that, at time T3, the M setting 3 stored in memory unit M1 is stored in memory unit M2, and the S setting 3 stored in memory unit S1 is stored in memory unit S2. Furthermore, in this example, since only the first vertical scanning is performed in the third frame, the signal selected by selection unit SL is switched at time T3. Therefore, the output of selection unit SL changes from S setting 2 to M setting 3. As a result, M setting 3 is applied to the readout operation of the first vertical scanning that starts at time T3.
[0061] In the scanning method of performing two scanning operations, the first vertical scanning and the second vertical scanning, in one frame period as in the present embodiment, it is necessary to switch the setting value at high speed in accordance with the timing of the scanning, which may complicate the control of the scanning. Therefore, for example, it may be difficult to ensure the communication time of the setting value. In contrast, in the present embodiment, the memory group 12 has a memory M2 that stores the setting value of the first vertical scanning and a memory S2 that stores the setting value of the second vertical scanning. This allows the memory group 12 to store both the setting value of the first vertical scanning and the setting value of the second vertical scanning at the same time. Therefore, these setting values can be communicated in advance prior to the first vertical scanning and the second vertical scanning, which simplifies the control of the scanning. As described above, according to the present embodiment, a photoelectric conversion device 1 that can realize simpler control in a configuration in which multiple scanning operations are performed in one frame period is provided.
[0062] In this embodiment, the storage group 12 further includes a storage unit M1 that temporarily stores the setting value of the first vertical scan input from the outside, and a storage unit S1 that temporarily stores the setting value of the second vertical scan input from the outside. This allows the timing at which the setting value is reflected to be delayed from the timing of communication, so that more communication time can be secured. This configuration also allows the time at which the setting value of the first vertical scan is input to the storage unit M1 and the time at which the setting value of the second vertical scan is input to the storage unit S1 to be earlier than the start of the frame period to which these setting values are applied. This allows the communication of the setting value to be completed before the start of the frame period in which the setting value is used, making it easier to secure the communication time. In addition, the time at which the setting value stored in the storage unit M1 is output to the storage unit M2 and the time at which the setting value stored in the storage unit S1 is output to the storage unit S2 are the start times of the frames. This allows the setting value to be enabled in accordance with the start of the frame.
[0063] In this embodiment, the memory group 12 further includes a selection unit SL that selectively outputs the setting value stored in the memory unit M2 and the setting value stored in the memory unit S2. This allows the timing at which either the setting value for the first vertical scanning or the setting value for the second vertical scanning is enabled to be shifted from the timing at which the memory units M2 and S2 store the setting values, thereby ensuring more communication time. This configuration also allows either the setting value for the first vertical scanning or the setting value for the second vertical scanning to be enabled at a timing after the start time of the frame period. This allows the setting value to be changed immediately before scanning, improving the flexibility of scanning.
[0064] The above-mentioned M setting # and S setting # may include not only the vertical scanning setting values but also setting values related to gain settings, data correction settings, etc. This makes it possible to apply different gains between the first vertical scanning and the second vertical scanning, for example, setting the gain of the first vertical scanning to 1x and the gain of the second vertical scanning to 4x.
[0065] In addition, in the example of the driving method shown in Fig. 5, an example has been described in which a pixel signal for an image is acquired in the first vertical scanning, and a pixel signal for distance measurement is acquired in the second vertical scanning, but this is not limited to this. For example, a pixel signal for distance measurement may be acquired in the first vertical scanning, and a pixel signal for an image may be acquired in the second vertical scanning. Also, a pixel signal for an image may be acquired in both the first vertical scanning and the second vertical scanning, and a pixel signal for distance measurement may be acquired in both the first vertical scanning and the second vertical scanning.
[0066] In addition, although the description of the setting values in the driving method shown in FIG. 5 only concerns the read operation, the same processing can also be applied to the setting values of the shutter operation.
[0067] [Second embodiment] In the first embodiment, the same setting value is used in the four second vertical scans of the second frame. In contrast, in the present embodiment, an example in which the setting value is changed in each of the four second vertical scans is described. Descriptions of elements common to the first embodiment may be simplified or omitted.
[0068] 6 is a block diagram showing an example of the configuration of the storage unit group 12 according to this embodiment. The storage unit group 12 has storage units M1, M2, S11, S12, S21, S22, S31, S32, S41, and S42 and a selection unit SL. That is, in this embodiment, the storage unit S1 of the first embodiment is replaced with four storage units S11, S21, S31, and S41, and the storage unit S2 of the first embodiment is replaced with four storage units S12, S22, S32, and S42.
[0069] The setting values stored in the memory units S11, S12, S21, S22, S31, S32, S41, and S42 are referred to as data S11, S12, S21, S22, S31, S32, S41, and S42, respectively. The selection unit SL selectively enables one of the data M2, S12, S22, S32, and S42 based on a selection flag input from the control unit 13. The selection unit SL outputs the selected data to the control unit 13 as data OUT.
[0070] FIG. 7 is a timing chart showing a method of driving the pixel unit 15 according to the present embodiment. The difference from the first embodiment is the operation of the memory unit group 12 regarding the setting of the read operation, and the timing of the shutter operation and the read operation is the same as in the first embodiment. Only the differences from the first embodiment will be described below. "S1 setting #", "S2 setting #", "S3 setting #" and "S4 setting #" in FIG. 7 respectively mean the setting values of the first to fourth second vertical scans in the second frame.
[0071] At time T1 of the first frame, M setting 1 is stored in storage units M1 and M2. Furthermore, S1 setting 1 is stored in storage units S11 and S12, and S2 setting 1 is stored in storage units S21 and S22. Furthermore, S3 setting 1 is stored in storage units S31 and S32, and S4 setting 1 is stored in storage units S41 and S42. Here, selection unit SL selects M setting 1 stored in storage unit M2 and outputs it to control unit 13. As a result, the readout operation of the first vertical scan, which starts from time T1, is performed based on M setting 1.
[0072] Between time T1 and time T2, that is, during the period of the first frame, the CPU2 transmits the M setting 2, which is the setting value of the first vertical scan in the second frame, to the communication unit 11. When the communication unit 11 receives the M setting 2, the M setting 2 is immediately stored in the memory unit M1. After that, the CPU2 transmits the S1 setting 2, which is the setting value of the first second vertical scan in the second frame, to the communication unit 11 during the period of the first frame. When the communication unit 11 receives the S1 setting 2, the S1 setting 2 is immediately stored in the memory unit S11. After that, in the same manner, the CPU2 sequentially transmits the S2 setting 2, S3 setting 2, and S4 setting 2, which are the setting values of the second to fourth second vertical scans, to the communication unit 11. The S2 setting 2, S3 setting 2, and S4 setting 2 are stored in the memory units S21, S31, and S41, respectively.
[0073] After that, at time T2, M setting 2 stored in memory unit M1 is stored in memory unit M2. Also, S1 setting 2, S2 setting 2, S3 setting 2, and S4 setting 2 stored in memory units S11, S21, S31, and S41 are stored in memory units S12, S22, S32, and S42, respectively. At this time, the output of selection unit SL also changes from M setting 1 to M setting 2. As a result, M setting 2 is applied to the readout operation of the first vertical scan that starts at time T2.
[0074] Next, at time T2s1, the signal selection by the selection unit SL is switched, and the output of the selection unit SL changes from the M setting 2 to the S1 setting 2. As a result, the S1 setting 2 is applied to the second readout operation of the second vertical scanning that starts at time T2s1.
[0075] Next, at time T2s2, the signal selection by the selection unit SL is switched, and the output of the selection unit SL changes from S1 setting 2 to S2 setting 2. As a result, the S2 setting 2 is applied to the second readout operation of the second vertical scanning that starts at time T2s2.
[0076] Thereafter, at times T2s3 and T2s4, the selection of signals is switched in a similar manner by the selection unit SL, and S3 setting 2 and S4 setting 2 are applied to the readout operations of the third and fourth second vertical scans, respectively. In this manner, in the first to fourth second vertical scans, the selection unit SL sequentially enables S1 setting 2 to S4 setting 2. As a result, different setting values are applied in the four second vertical scans.
[0077] Between time T2 and time T3, that is, during the period of the second frame, the CPU 2 transmits the M setting 3, which is the setting value of the first vertical scan in the third frame, to the communication unit 11. When the communication unit 11 receives the M setting 3, the M setting 3 is immediately stored in the memory unit M1. After that, the CPU 2 transmits the S1 setting 3, which is the setting value of the first second vertical scan in the third frame, to the communication unit 11 during the period of the second frame. When the communication unit 11 receives the S1 setting 3, the S1 setting 3 is immediately stored in the memory unit S11. After that, in the same manner, the CPU 2 sequentially transmits the S2 setting 3, S3 setting 3, and S4 setting 3, which are the setting values of the second to fourth second vertical scans, to the communication unit 11. The S2 setting 3, S3 setting 3, and S4 setting 3 are stored in the memory units S21, S31, and S41, respectively.
[0078] After that, at time T3, the M setting 3 stored in the memory unit M1 is stored in the memory unit M2. Also, the S1 setting 3, S2 setting 3, S3 setting 3, and S4 setting 3 stored in the memory units S11, S21, S31, and S41 are stored in the memory units S12, S22, S32, and S42, respectively. Furthermore, in this example, since only the first vertical scanning is performed in the third frame, the signal selected by the selection unit SL is switched at time T3. Therefore, the output of the selection unit SL changes from the S4 setting 2 to the M setting 3. As a result, the M setting 3 is applied to the readout operation of the first vertical scanning that starts at time T3.
[0079] In this embodiment, the memory group 12 has four sets of memory units that store the setting values of the second vertical scan. This allows a different setting value to be set for each of the four second vertical scans, improving the degree of freedom in setting the second vertical scan.
[0080] The number of storage units that store the setting values of the second vertical scan is not limited to four. The number of storage units can be changed as appropriate according to the design, and can be set according to the maximum number of second vertical scans per frame period, for example. For example, if the maximum number of second vertical scans is N (N is an integer equal to or greater than 2), different setting values can be applied in each second vertical scan by providing N sets of storage units that store the setting values of the second vertical scan. Specifically, the storage units S11, S21, S31, and S41 in FIG. 6 are replaced with N storage units (N fourth storage units), and the storage units S12, S22, S32, and S42 are replaced with N storage units (N second storage units), thereby realizing the second vertical scan based on the setting values N times.
[0081] Note that the above-mentioned M setting #, S1 setting #, S2 setting #, S3 setting #, and S4 setting # may include not only vertical scanning setting values but also setting values related to gain settings, data correction settings, etc., as in the first embodiment. This makes it possible to apply different gains between multiple second vertical scans, for example, setting the gain of the first second vertical scan to 1x and the gain of the second second vertical scan to 2x.
[0082] [Third embodiment] An imaging system according to a third embodiment of the present invention will be described with reference to Fig. 8. Fig. 8 is a block diagram showing a schematic configuration of the imaging system according to this embodiment. The photoelectric conversion device 1 shown in Fig. 8 is the photoelectric conversion device 1 described in the first or second embodiment. That is, the imaging system 500 according to this embodiment is an example of a photoelectric conversion system to which the photoelectric conversion device 1 described in the first or second embodiment can be applied.
[0083] The imaging system 500 according to this embodiment is not particularly limited, but can be applied to, for example, digital still cameras, digital camcorders, camera heads, copiers, fax machines, mobile phones, vehicle-mounted cameras, observation satellites, and the like.
[0084] 8, the imaging system 500 has a photoelectric conversion device 1, a lens 502, an aperture 504, a barrier 506, a signal processing unit 508, a timing generating unit 520, and an overall control / calculation unit 518. The imaging system 500 also has a memory unit 510, a recording medium control I / F unit 516, and an external I / F unit 512.
[0085] The lens 502 is for forming an optical image of a subject on the pixel unit 15 of the photoelectric conversion device 1. The diaphragm 504 is for varying the amount of light that passes through the lens 502. The barrier 506 is for protecting the lens 502. The photoelectric conversion device 1 is the photoelectric conversion device 1 described in the previous embodiment, and outputs a signal based on the optical image formed by the lens 502 to a signal processing unit 508. The signals output from the photoelectric conversion device 1 may include an A signal, an A+B signal, a B signal, etc., read out from a pixel P.
[0086] The signal processing unit 508 performs desired processing, correction, data compression, and the like on the signal output from the photoelectric conversion device 1. The processing performed by the signal processing unit 508 includes generation of corrected image data, processing of acquiring distance information to a subject based on the A signal and the B signal, and the like. The signal processing unit 508 may be mounted on the same board as the photoelectric conversion device 1, or may be mounted on a different board. Also, some of the functions of the signal processing unit 508 may be mounted on the same board as the photoelectric conversion device 1, and other parts of the functions of the signal processing unit 508 may be mounted on a different board.
[0087] The timing generating unit 520 is for outputting various timing signals to the photoelectric conversion device 1 and the signal processing unit 508. The overall control / calculation unit 518 is a control unit that manages the overall driving and calculation processing of the imaging system 500. The CPU 2 in the first or second embodiment may correspond to one or both of the overall control / calculation unit 518 and the timing generating unit 520. Here, control signals such as timing signals may be input from outside the imaging system 500, and the imaging system 500 only needs to have at least the photoelectric conversion device 1 and the signal processing unit 508 that processes signals output from the photoelectric conversion device 1.
[0088] The memory unit 510 is a frame memory unit for temporarily storing image data. The recording medium control I / F unit 516 is an interface unit for recording to the recording medium 514 or reading from the recording medium 514. The external I / F unit 512 is an interface unit for communicating with an external computer or the like. The recording medium 514 is a removable recording medium such as a semiconductor memory for recording or reading imaging data.
[0089] In this manner, by configuring the imaging system 500 to which the photoelectric conversion device 1 according to the first or second embodiment is applied, it is possible to realize the imaging system 500 that can realize simpler control.
[0090] [Fourth embodiment] 9(a) and 9(b) are diagrams showing the configuration of an imaging system 600 and a moving body according to this embodiment. FIG. 9(a) shows an example of an imaging system 600 related to an on-vehicle camera. The imaging system 600 has the photoelectric conversion device 1 described in either the first or second embodiment described above. The imaging system 600 has an image processing unit 612 that performs image processing on a plurality of image data acquired by the photoelectric conversion device 1, and a parallax calculation unit 614 that calculates parallax (phase difference of parallax images) from the plurality of image data acquired by the imaging system 600. The imaging system 600 also has a distance measurement unit 616 that calculates a distance to an object based on the calculated parallax, and a collision determination unit 618 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax calculation unit 614 and the distance measurement unit 616 are an example of distance information acquisition means that acquires distance information to an object. That is, the distance information is information on the parallax, the defocus amount, the distance to the object, and the like. The collision determination unit 618 may determine the possibility of a collision using any of these pieces of distance information. The distance information acquisition means may be realized by dedicated hardware or a software module. In addition, it may be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a combination of these.
[0091] The imaging system 600 is connected to a vehicle information acquisition device 620, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The imaging system 600 is also connected to a control ECU 630, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 618. That is, the control ECU 630 is an example of a moving body control means that controls a moving body based on distance information. The imaging system 600 is also connected to an alarm device 640 that issues an alarm to the driver based on the judgment result of the collision judgment unit 618. For example, when the judgment result of the collision judgment unit 618 indicates that there is a high possibility of a collision, the control ECU 630 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 640 warns the user by sounding an alarm, displaying alarm information on a screen of a car navigation system, etc., vibrating a seat belt or steering wheel, etc.
[0092] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are captured by the imaging system 600. Fig. 9(b) shows the configuration of the imaging system 600 when capturing an image of the area in front of the vehicle (imaging range 650). The vehicle information acquisition device 620 sends an instruction to operate the imaging system 600 to perform imaging.
[0093] In this manner, by configuring an imaging system 600 and a moving body to which the photoelectric conversion device 1 according to the first or second embodiment is applied, a moving body that can be controlled more simply can be realized.
[0094] In the above description, an example of control to avoid collision with other vehicles has been described, but the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to avoid going out of a lane, etc. Furthermore, the imaging system is not limited to vehicles such as the vehicle itself, but can be applied to moving bodies (moving devices) such as ships, aircraft, industrial robots, etc. In addition, the imaging system can be applied not only to moving bodies, but also to a wide range of devices that use object recognition, such as an intelligent transport system (ITS).
[0095] [Modified embodiment] The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, it should be understood that the present invention can be applied to an embodiment in which a part of the configuration of any of the embodiments is added to another embodiment, or an embodiment in which a part of the configuration of any of the embodiments is replaced with a part of the configuration of another embodiment.
[0096] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0097] The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]
[0098] 1 Photoelectric conversion device 14 Vertical scanning section M2, S2 storage section P pixels
Claims
1. a plurality of pixels arranged in a plurality of rows; a scanning unit that performs a first scan and a second scan to sequentially output signals from the plurality of pixels row by row; a first storage unit configured to store a first setting value indicating a setting of a read operation of the plurality of rows of pixels in the first scan in response to an external input; a second storage unit configured to store a second setting value indicating a setting of a read operation of the plurality of rows of pixels in the second scan in response to an external input; having the scanning unit performs the first scanning based on the first set value and the second scanning based on the second set value in one frame period; The first setting value is stored by the first storage unit, and the second setting value is stored by the second storage unit prior to the start of the first scan and the start of the second scan. A photoelectric conversion device comprising:
2. Further comprising a third storage unit and a fourth storage unit, the third storage unit stores the first setting value input from an external device and outputs the first setting value to the first storage unit; The fourth storage unit stores the second setting value input from an external device and outputs the second setting value to the second storage unit.
2. The photoelectric conversion device according to claim 1.
3. The time when the first setting value is input to the third storage unit is before the time when the one frame period starts.
3. The photoelectric conversion device according to claim 2.
4. The time when the second setting value is input to the fourth storage unit is before the time when the one frame period starts.
4. The photoelectric conversion device according to claim 2 or 3.
5. The time when the first setting value is output from the third storage unit to the first storage unit is the time when the one frame period starts.
5. The photoelectric conversion device according to claim 2, wherein the first and second electrodes are arranged in a first direction.
6. The time when the second setting value is output from the fourth storage unit to the second storage unit is the time when the one frame period starts.
6. The photoelectric conversion device according to claim 2, wherein the first and second electrodes are arranged in a first direction.
7. The device further includes a selection unit that selectively enables one of the first setting value stored in the first storage unit and the second setting value stored in the second storage unit.
7. The photoelectric conversion device according to claim 1, wherein the first and second electrodes are arranged in a first direction.
8. The time when the selection unit enables the second setting value is after the time when the one frame period starts.
8. The photoelectric conversion device according to claim 7.
9. The scanning unit performs the second scanning N times (N is an integer equal to or greater than 2) in one frame period.
9. The photoelectric conversion device according to claim 1,
10. N second storage units each storing N second setting values indicating settings for N second scans are provided.
10. The photoelectric conversion device according to claim 9.
11. Further, N fourth storage units are provided corresponding to the N second storage units, Each of the N fourth storage units stores the second setting value input from the outside and outputs it to the corresponding second storage unit. The photoelectric conversion device according to claim 10 .
12. The apparatus further includes a selection unit that selectively enables one of the first setting value stored in the first storage unit and the N second setting values stored in the N second storage units.
12. The photoelectric conversion device according to claim 10 or 11.
13. The selection unit sequentially enables the N second setting values corresponding to the N second scans.
13. The photoelectric conversion device according to claim 12.
14. During a period in which signals are output from pixels in a first row of the plurality of rows by the first scanning, signals are output from pixels in a second row, which is different from the first row of the plurality of rows, by the second scanning.
14. The photoelectric conversion device according to claim 1,
15. The length of the period from the start to the end of the first scan is different from the length of the period from the start to the end of the second scan.
15. The photoelectric conversion device according to claim 1,
16. The photoelectric conversion device according to any one of claims 1 to 15, a signal processing unit that processes a signal output from the photoelectric conversion device; A photoelectric conversion system comprising:
17. A mobile object, The photoelectric conversion device according to any one of claims 1 to 15, a distance information acquiring means for acquiring distance information to an object from a parallax image based on a signal from the photoelectric conversion device; A control means for controlling the moving object based on the distance information; A moving object comprising:
Citation Information
Patent Citations
Solid-state imaging apparatus, camera apparatus, and control method thereof
JP2004140479A
Drive circuit of solid-state imaging device, method and imaging system
JP2008035425A
Imaging apparatus, imaging system, driving method of imaging apparatus
JP2017103537A
Imaging device, imaging system, mobile body and drive method of imaging device
JP2018032976A
