Photoelectric conversion device, photoelectric conversion system, transport device, and signal processing device

Abstract

Disclosed are a photoelectric conversion device, a photoelectric conversion system, a transport device, and a signal processing device. A photoelectric conversion device including a pixel array and a signal processor is provided. The pixel array is configured to be operable in a driving mode using different signal readout methods. The signal processor includes a selector configured to select a first pixel group and a second pixel group from a region of the pixel array that has been designated for generating a correction value based on a driving mode set for each pixel in the driving mode, a correction value generator configured to generate the correction value in accordance with a first representative value based on signals read out from the first pixel group and a second representative value based on signals read out from the second pixel group, and a corrector configured to correct signals read out from the pixel array based on the correction value.

Description

TECHNICAL FIELD

[0001] The present application relates to a photoelectric conversion device, a photoelectric conversion system, a transport device, and a signal processing device. BACKGROUND

[0002] In recent years, the multifunctionality of image capturing devices installed on image capturing systems such as digital still cameras or digital video cameras is developing. Japanese Patent Publication No. 2020-167542 describes an image capturing device that can drive pixels in different modes depending on the pixel row at the time of reading out a signal and output focus detection data in addition to capturing image data. SUMMARY

[0003] If each pixel is driven in different modes depending on the pixel at the time of reading out a signal from a pixel array, the correction amount used to perform correction processing on the read signal can be different for each driving mode. It is desirable that the correction processing can cope with various driving modes.

[0004] Some embodiments of the present application provide a technique that facilitates improving the accuracy of correction processing.

[0005] According to some embodiments, a photoelectric conversion device is provided, the photoelectric conversion device including a pixel array in which a plurality of pixels are arranged in a matrix, and a signal processor configured to process a signal read out from the pixel array, wherein the plurality of pixels are configured to be operable in a plurality of driving modes using different signal readout methods, and the signal processor includes: a selection circuit configured to select, from an area of the pixel array that has been designated for generating a correction value, a first group of pixels and a second group of pixels among the plurality of pixels based on a driving mode set for each pixel among the plurality of driving modes; a correction value generation circuit configured to generate a correction value in accordance with a first representative value based on a signal read out from the first group of pixels and a second representative value based on a signal read out from the second group of pixels; and a correction circuit configured to correct a signal read out from the pixel array based on the correction value.

[0006] According to some other embodiments, there is provided a signal processing apparatus for processing a signal read out from a pixel array in which a plurality of pixels capable of operating in a plurality of driving modes using different signal readout methods are arranged in a matrix, the signal processing apparatus including: a selection circuit configured to select, from an area of the pixel array of the plurality of pixels that has been designated for generating a correction value, a first group of pixels and a second group of pixels among the plurality of pixels based on a driving mode set for each pixel among the plurality of driving modes; a correction value generation circuit configured to generate a correction value in accordance with a first representative value based on a signal read out from the first group of pixels and a second representative value based on a signal read out from the second group of pixels; and a correction circuit configured to correct the signal read out from the pixel array based on the correction value.

[0007] Other features of the present application will become apparent from the following description of example embodiments (with reference to the accompanying drawings). BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a block diagram showing an example of an arrangement of a photoelectric conversion apparatus according to an embodiment;

[0009] Figure 2 is a circuit diagram showing an example of an arrangement of a pixel of the photoelectric conversion apparatus shown in Figure 1

[0010] Figure 3 is a timing chart showing an example of a readout operation of the photoelectric conversion apparatus shown in Figure 1

[0011] Figure 4 is a timing chart showing an example of a readout operation of the photoelectric conversion apparatus shown in Figure 1

[0012] Figure 5 is a view showing an overview of an output signal of the photoelectric conversion apparatus shown in Figure 1

[0013] Figure 6 is a block diagram showing an example of an arrangement of a corrector of the photoelectric conversion apparatus shown in Figure 1

[0014] Figure 7 is a view showing an example of a position of a group of pixels for generating a correction value in the photoelectric conversion apparatus shown in Figure 1

[0015] Figure 8 is a block diagram showing an example of an arrangement of a correction value obtainer of the photoelectric conversion apparatus shown in Figure 1 ​​​​​​​

[0016] Figure 9 is a view showing an example of an arrangement of a region controller of the photoelectric conversion device shown in Figure 1

[0017] Figure 10 is a view showing an example of an arrangement of a correction applicator of the photoelectric conversion device shown in Figure 1

[0018] Figure 11A and Figure 11B are each a table showing an example of a setting of a correction process of the photoelectric conversion device shown in Figure 1

[0019] Figure 12 is a view showing an example of an arrangement of a correction value obtainer of the photoelectric conversion device shown in Figure 1

[0020] Figure 13A and Figure 13B are each a view showing an example of a position of a pixel group for generating a correction value in the photoelectric conversion device shown in Figure 1

[0021] Figure 14 is a block diagram showing an example of a configuration of an image capturing system incorporated in the photoelectric conversion device shown in Figure 1

[0022] Figure 15A is a block diagram showing an example of a configuration of a photoelectric conversion system incorporated in the photoelectric conversion device shown in Figure 1

[0023] Figure 15B is a view showing an example of a configuration of a transport device of the photoelectric conversion system shown in Figure 15A DETAILED DESCRIPTION

[0024] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the application of claims. A plurality of features are described in the embodiments, but the application does not necessarily need all such features, and a plurality of such features can be appropriately combined. Furthermore, in the drawings, the same reference numerals are assigned to the same or similar configurations, and redundant description thereof is omitted.

[0025] A photoelectric conversion device according to an embodiment of the present application will be described with reference to Figures 1-15B Figure 1 and Figure 2 The structure of the photoelectric conversion device according to this embodiment will be described with reference to Figure 1 ​​​​​​​​​is a block diagram showing a schematic arrangement of the photoelectric conversion device 100 according to this embodiment. Figure 2 is a circuit diagram showing an example of an arrangement of the pixels 102 of the photoelectric conversion device 100. As shown in Figure 1 The photoelectric conversion device 100 includes a pixel array 101, a vertical scanning circuit 103, a controller 104, a readout circuit 105, an AD conversion circuit 106, a memory 107, a horizontal scanning circuit 108, and a processor 109, as shown in

[0026] In the pixel array 101, a plurality of pixels 102 are arranged in an array of a plurality of rows and a plurality of columns. Figure 1 The pixel array 101 is shown, which includes the pixels 102 arranged in (m+1) columns of the 0th column to the mth column and (n+1) rows of the 0th row to the nth row. For each pixel 102, a number P(m,n) is described, which is obtained by adding a coordinate representing a corresponding column number and row number to the number P. Each pixel 102 includes a plurality of photoelectric converters. More specifically, in this embodiment, each pixel 102 includes two photoelectric converters 112A and 112B. The photoelectric conversion device 100 can detect a phase difference by comparing a signal output from the photoelectric converter 112A with a signal output from the photoelectric converter 112B. In this embodiment, each pixel 102 includes two photoelectric converters 112A and 112B, but can include three or more photoelectric converters 112.

[0027] In the pixel array 101, a row selection line 110 is arranged for each row. Each row selection line 110 is connected to the (m+1) pixels 102 arranged in a corresponding row. The row selection line 110 is connected to the vertical scanning circuit 103.

[0028] In the pixel array 101, a vertical output line 111 is arranged for each column. Each vertical output line 111 is connected to the (n+1) pixels 102 arranged in a corresponding column. The vertical output line 111 is connected to the readout circuit 105.

[0029] The vertical scanning circuit 103 is a control circuit for performing an operation of providing a driving signal to the pixels 102 of the pixel array 101 row by row (vertical scanning). In a row selected by the vertical scanning circuit 103, signals are simultaneously output from the (m+1) pixels 102 included in the selected row via the corresponding vertical output line 111, respectively. The signals (analog signals) output from the pixel array 101 row by row are input to the readout circuit 105.

[0030] The readout circuit 105 is a circuit that performs predetermined processing on an analog signal read out from the pixel array 101. The readout circuit 105 can include a plurality of signal storage circuits and a plurality of amplification circuits corresponding to the columns (vertical output lines 111) of the pixel array 101, respectively. For example, the readout circuit 105 amplifies an analog signal output from the vertical output line 111 of each column of the pixel array 101 by the amplification circuit of the corresponding column, and stores the amplified signal in the signal storage circuit of the corresponding column.

[0031] The AD conversion circuit 106 is a circuit that converts a signal of each column output from the readout circuit 105 from an analog signal to a digital signal. The memory 107 is a signal storage circuit that temporarily stores a digital signal of each column AD-converted by the AD conversion circuit 106.

[0032] The horizontal scan circuit 108 is a circuit that supplies a control signal for outputting a digital pixel signal stored in the column memory of each column of the memory 107 to the processor 109 to the memory 107. That is, a digital signal stored in the column memory of a column for which address designation has been performed by the horizontal scan circuit 108 is sequentially read out from the memory 107 and transmitted to the processor 109.

[0033] The processor 109 performs predetermined processing on a digital signal read out from the memory 107. The processing performed by the processor 109 includes CDS (Correlated Double Sampling) processing and correction processing (to be described later). Furthermore, the processor 109 includes an external interface such as LVDS (Low Voltage Differential Signaling), and outputs a processed digital signal to the outside of the photoelectric conversion device 100.

[0034] The controller 104 (hereinafter, also referred to as a timing generator) is a control circuit that supplies control signals for controlling operations and timings thereof to the vertical scan circuit 103, the readout circuit 105, the AD conversion circuit 106, the memory 107, the horizontal scan circuit 108, and the processor 109. At least some of these control signals can be supplied from the outside of the photoelectric conversion device 100. Furthermore, the controller 104 can be provided with setting information of the photoelectric conversion device 100 by communication from the outside. In this case, the controller 104 controls the vertical scan circuit 103, the readout circuit 105, the AD conversion circuit 106, the memory 107, the horizontal scan circuit 108, and the processor 109 on the basis of the setting information input from the outside. The function of the controller 104 can be implemented when a CPU or an MPU reads out a program and executes it, or can be implemented by a circuit for implementing the function of an ASIC, an FPGA, or the like.

[0035] The signal obtained from photoelectric converter 112A—that is, the signal corresponding to the charge generated by photoelectric converter 112A—will hereinafter be referred to as image signal A. The signal obtained from photoelectric converter 112B—that is, the signal corresponding to the charge generated by photoelectric converter 112B—will hereinafter be referred to as image signal B. Image signals A and B can be used, for example, as focus detection signals. The signal obtained by synthesizing image signals A and B—that is, the signal corresponding to the charge obtained by synthesizing the charge generated by photoelectric converter 112A and the charge generated by photoelectric converter 112B—will hereinafter be referred to as image signal A+B. Image signal A+B can be used, for example, as a capture image signal.

[0036] Among the multiple rows forming the pixel array 101, the row containing pixels 102 that read out both image signal A and image signal A+B will hereinafter be referred to as a focus detection data row. Among the multiple rows forming the pixel array 101, the row containing pixels 102 that read out only image signal A+B will hereinafter be referred to as a capture image data row. In the photoelectric conversion apparatus 100 according to this embodiment, the focus detection data row and the capture image data row are appropriately set in the pixel array 101 according to image capture parameters (image capture conditions), etc.

[0037] Each pixel 102 can be, for example, Figure 2 The circuit formation shown in the figure. Figure 2 As shown, each pixel 102 may include two photodiodes PDA and PDB, transmission transistors M1A and M1B, a reset transistor M2, an amplification transistor M3, and a selection transistor M4.

[0038] Two photodiodes, PDA and PDB, correspond to the aforementioned photoelectric converters 112A and 112B, respectively. The photodiodes PDA and PDB of a single pixel 102 share a microlens (not shown) and are configured to receive light that has passed through a pupil region different from the exit pupil of the imaging lens. This can be achieved using signals based on the charge generated by the photodiode PDA and signals based on the charge generated by the photodiode PDB as signals for focus detection. Additionally, a signal based on the total charge obtained by adding the charges generated by the photodiodes PDA and PDB can be used as a signal for image acquisition.

[0039] The photodiode PDA has an anode connected to a ground node (GND) and a cathode connected to a source of the transfer transistor MIA. The photodiode PDB has an anode connected to the ground node (GND) and a cathode connected to a source of the transfer transistor MIB. The drains of the transfer transistors MIA and MIB are connected to a source of the reset transistor M2 and a gate of the amplification transistor M3. A connection node of the drains of the transfer transistors MIA and MIB, the source of the reset transistor M2, and the gate of the amplification transistor M3 is a so-called floating diffusion FD. The floating diffusion FD includes a capacitive component, and forms a charge-voltage converter by the capacitive component while functioning as a charge storage device. The drain of the reset transistor M2 and the drain of the amplification transistor M3 are connected to a power node (voltage VDD). The source of the amplification transistor M3 is connected to a drain of the selection transistor M4. The source of the selection transistor M4 is connected to the vertical output line 111. The source and the drain at the two main terminals of the transistor can be changed depending on the conductive type of the transistor, the function of interest, and the like, and the above-mentioned source and drain can be called by the opposite names.

[0040] The row selection line 110 of each row includes a signal line connected to a gate of the transfer transistor MIA, a signal line connected to a gate of the transfer transistor MIB, a signal line connected to a gate of the reset transistor M2, and a signal line connected to a gate of the selection transistor M4. The vertical scanning circuit 103 supplies a control signal PTXA to the signal line connected to the gate of the transfer transistor MIA. The vertical scanning circuit 103 supplies a control signal PTKB to the signal line connected to the gate of the transfer transistor MIB. The vertical scanning circuit 103 supplies a control signal PRES to the signal line connected to the gate of the reset transistor M2. The vertical scanning circuit 103 supplies a control signal PSEL to the signal line connected to the gate of the selection transistor M4. If each transistor is formed by an N-type transistor, the corresponding transistor is turned on (set to an on state) when the vertical scanning circuit 103 supplies a high-level control signal, and is turned off (set to an off state) when the vertical scanning circuit 103 supplies a low-level control signal.

[0041] If light enters the pixel array 101, each of the photodiodes PDA and PDB of each pixel 102 generates (photoelectrically converts) an amount of electric charge corresponding to the amount of light of the incident light, and accumulates the generated electric charge. The transfer transistors M1A and M1B transfer the electric charges of the photodiodes PDA and PDB to the floating diffusion FD, respectively, by performing an ON operation. The floating diffusion FD stores the electric charges transferred from each of the photodiodes PDA and PDB. When an A image signal is read out, the electric charge photoelectrically converted by the photodiode PDA is transferred to the floating diffusion FD via the transfer transistor M1A. When a B image signal is read out, the electric charge photoelectrically converted by the photodiode PDB is transferred to the floating diffusion FD via the transfer transistor M1B. When an A+B image signal is read out, the total electric charge photoelectrically converted by the photodiodes PDA and PDB is transferred to the floating diffusion FD via the transfer transistors M1A and M1B. This sets the floating diffusion FD to a voltage corresponding to the amount of electric charge transferred from the photodiodes PDA and PDB by charge-voltage conversion with a capacitive component.

[0042] The amplification transistor M3 is configured so that the voltage VDD is supplied to the source and a bias current is supplied to the drain from a current source (not shown) via the selection transistor M4, and forms an amplifier (source follower circuit) with the gate as an input node. This causes the amplification transistor M3 to output a signal based on the voltage of the floating diffusion FD to the vertical output line 111 via the selection transistor M4. The reset transistor M2 resets the floating diffusion FD to a voltage corresponding to the voltage VDD by performing an ON operation.

[0043] The transfer transistors M1A and M1B, the reset transistor M2, and the selection transistor M4 of the pixel 102 are controlled row by row by the control signals PTXA, PTXB, PRES, and PSEL supplied from the vertical scanning circuit 103. The pixel signals of the plurality of pixels 102 connected to a row (selected row) selected by the control signal PSEL are simultaneously output to the vertical output line 111 of each column.

[0044] Next, the operation of the pixel array 101 of the image capturing device according to this embodiment will be described with reference to Figure 3 and Figure 4 Figure 3 is a timing chart at the time of reading out signals from the pixels 102 belonging to the image data row. Figure 4 is a timing chart at the time of reading out signals from the pixels 102 belonging to the focus detection data row. As an example, Figure 3 and Figure 4 ​Each shows a readout operation when the vertical scanning circuit 103 selects the pixel 102 of the nth row. As described above, each transistor of each pixel 102 turns on when the vertical scanning circuit 103 supplies a high-level control signal, and turns off when the vertical scanning circuit 103 supplies a low-level control signal.

[0045] The readout operation for capturing an image data row will be described with reference to Figure 3 Figure 3 The horizontal synchronization signal SYNC, the control signals PSEL(n), PRES(n), PTXA(n), and PTXB(n), the AD conversion period, and the horizontal scanning pulse signal are shown. In a driving mode in which an A+B image signal corresponding to the charge obtained by synthesizing the charges generated by the photodiodes PDA and PDB is read out, data is read out from the capturing image data row. On the other hand, in reading out data from the capturing image data row, a driving mode in which an A image signal corresponding to the charge generated by only the photodiode PDA or a B image signal corresponding to the charge generated by only the photodiode PDB is read out is not performed.

[0046] At time t301, the horizontal synchronization signal SYNC supplied from the controller 104 to the vertical scanning circuit 103 transitions from a low level to a high level. When the horizontal synchronization signal SYNC rises, the vertical scanning circuit 103 controls the control signal PSEL(n) of the nth row from a low level to a high level. This turns on the selection transistor M4 of each pixel 102 belonging to the nth row, and each pixel 102 can output a signal to the vertical output line 111 via the selection transistor M4. That is, the nth row is selected by the control signal PSEL(n) from the vertical scanning circuit 103.

[0047] Next, at time t302, the vertical scanning circuit 103 controls the control signal PRES(n) of the selected row (that is, the nth row) from a low level to a high level. This turns on the reset transistor M2 of each pixel 102 belonging to the nth row, and the floating diffusion FD is reset to a potential corresponding to the voltage VDD.

[0048] Subsequently, at time t303, the vertical scanning circuit 103 controls the control signal PRES(n) of the nth row from a high level to a low level. This turns off the reset transistor M2 of each pixel 102 belonging to the nth row, and the reset of the floating diffusion FD is cancelled. At this time, since the selection transistor M4 of each pixel 102 belonging to the nth row remains in the on state, an output signal corresponding to the potential of the gate of the amplification transistor M3 when the reset of the floating diffusion FD is cancelled is output to the vertical output line 111. Hereinafter, the pixel signal of the reset level output from each pixel 102 will be referred to as an N signal (noise signal).

[0049] ​Next, the period from time t304 to time t305 is a period in which AD conversion processing is performed on the N signal output to the vertical output line 111. The N signal output to the vertical output line 111 is read by the readout circuit 105 and converted into a digital signal by the AD conversion circuit 106. The digital signal of the N signal obtained by the AD conversion circuit 106 is stored in the memory 107. The operation performed during the period from time t304 to time t305, that is, the operation of converting the N signal into a digital signal will be hereinafter referred to as N conversion.

[0050] Next, at time t306, the vertical scanning circuit 103 controls each of the control signals PTXA(n) and PTXB(n) of the nth row from the low level to the high level. This turns on the transfer transistors MIA and MIB of each pixel 102 belonging to the nth row, and the charges accumulated in the photodiodes PDA and PDB are transferred to the floating diffusion FD. This sets the potential of the floating diffusion FD, that is, the potential of the gate of the amplification transistor M3 to a potential corresponding to the amount of charges transferred from the photodiodes PDA and PDB. At this time, since the selection transistor M4 of each pixel 102 belonging to the nth row remains in the on state, the A+B image signal as a signal corresponding to the total amount of charges generated by the photodiodes PDA and PDB is output to the vertical output line 111.

[0051] Subsequently, at time t307, the vertical scanning circuit 103 controls each of the control signals PTXA(n) and PTXB(n) of the nth row from the high level to the low level. This turns off the transfer transistors MIA and MIB of each pixel 102 belonging to the nth row. Even after the transfer transistors MIA and MIB are turned off, the A+B image signal is continuously output to the vertical output line 111.

[0052] Next, the period from time t308 to time t309 is a period in which AD conversion processing is performed on the A+B image signal output to the vertical output line 111. The A+B image signal output to the vertical output line 111 is read by the readout circuit 105 and converted into a digital signal by the AD conversion circuit 106. The digital signal of the A+B image signal obtained by the AD conversion circuit 106 is stored in a storage region different from the storage region in which the N signal is stored in the memory 107. The operation performed during the period from time t308 to time t309, that is, the operation of converting the A+B image signal into a digital signal will be hereinafter referred to as A+B conversion.

[0053] The period from time t310 to time t311 is a period in which the N signal and the A+B image signal stored in the memory 107 are transferred to the processor 109. The horizontal scanning circuit 108 outputs a horizontal scanning pulse signal to the memory 107. The memory 107 transfers the N signal and the A+B image signal corresponding to an address (column) selected by the horizontal scanning pulse signal to the processor 109. When this operation is repeatedly performed from the 0th column to the mth column while changing the address selected by the horizontal scanning pulse signal (horizontal scanning is performed), the N signal and the A+B image signal of one row as a read target row can be read out. The processor 109 performs a process of subtracting the N signal corresponding to the A+B image signal from the A+B image signal, thereby suppressing noise to be superimposed on the A+B image signal.

[0054] Thereafter, the vertical scanning circuit 103 cancels selection of the nth row by controlling the control signal PSEL(n) of the nth row from high level to low level, completes the read operation for one row of the captured image data row, and shifts to the read operation of the next row.

[0055] Next, the read operation for the focus detection data row will be described with reference to Figure 4 Figure 4 The horizontal synchronization signal SYNC, the control signals PSEL(n), PRES(n), PTXA(n), and PTXB(n), the AD conversion period, and the horizontal scanning pulse signal are shown.

[0056] In the drive mode in which the A image signal corresponding to the charge generated by the photodiode PDA and the A+B image signal corresponding to the synthetic charge obtained by synthesizing the charges generated by the photodiodes PDA and PDB are read out, data is read out from the focus detection data row.

[0057] At time t401, the horizontal synchronization signal SYNC supplied from the controller 104 to the vertical scanning circuit 103 transitions from low level to high level. When the horizontal synchronization signal SYNC rises, the vertical scanning circuit 103 controls the control signal PSEL(n) of the nth row from low level to high level. This turns on the selection transistor M4 of each pixel 102 belonging to the nth row, thereby selecting the nth row.

[0058] Next, at time t402, the vertical scanning circuit 103 controls the control signal PRES(n) of the nth row from low level to high level. This turns on the reset transistor M2 of each pixel 102 belonging to the nth row, and the floating diffusion FD is reset to a potential corresponding to the voltage VDD.

[0059] ​Subsequently, at time t403, the vertical scanning circuit 103 controls the control signal PRES(n) of the nth row from the high level to the low level. This causes the reset transistor M2 of each pixel 102 belonging to the nth row to be turned off, and the reset of the floating diffusion FD is cancelled. At this time, since the selection transistor M4 of each pixel 102 belonging to the nth row remains in the on state, the pixel signal (N signal) of the reset level is output to the vertical output line 111.

[0060] Next, the period from time t404 to time t405 is a period in which AD conversion processing (N conversion) is performed on the N signal output to the vertical output line 111. The N signal output to the vertical output line 111 is read by the read circuit 105 and converted into a digital signal by the AD conversion circuit 106. The digital signal of the N signal obtained by the AD conversion circuit 106 is stored in the memory 107.

[0061] Next, at time t406, the vertical scanning circuit 103 controls the control signal PTXA(n) of the nth row from the low level to the high level. This causes the transfer transistor MIA of each pixel 102 belonging to the nth row to be turned on, and the charge accumulated in the photodiode PDA is transferred to the floating diffusion FD. This sets the potential of the floating diffusion FD, that is, the potential of the gate of the amplification transistor M3 to a potential corresponding to the amount of charge transferred from the photodiode PDA. At this time, since the selection transistor M4 of each pixel 102 belonging to the nth row remains in the on state, the A image signal as the pixel signal corresponding to the amount of charge generated by the photodiode PDA is output to the vertical output line 111.

[0062] Subsequently, at time t407, the vertical scanning circuit 103 controls the control signal PTXA(n) of the nth row from the high level to the low level. This causes the transfer transistor MIA of each pixel 102 belonging to the nth row to be turned off. Even after the transfer transistor MIA is turned off, the A image signal is continuously output to the vertical output line 111.

[0063] Next, the period from time t408 to time t409 is a period in which AD conversion processing is performed on the A image signal output to the vertical output line 111. The A image signal output to the vertical output line 111 is read by the read circuit 105 and converted into a digital signal by the AD conversion circuit 106. The digital signal of the A image signal obtained by the AD conversion circuit 106 is stored in a storage region different from the storage region in which the N signal is stored in the memory 107. The operation performed during the period from time t408 to time t409, that is, the operation of converting the A image signal into a digital signal will be referred to as A conversion hereinafter.

[0064] The period from time t410 to time t411 is a period in which the N signal and the A image signal stored in the memory 107 are transferred to the processor 109. The horizontal scanning circuit 108 outputs a horizontal scanning pulse signal to the memory 107. The memory 107 transfers the N signal and the A image signal corresponding to the address (column) selected by the horizontal scanning pulse signal to the processor 109. When this operation is repeatedly performed from the 0th column to the mth column while changing the address selected by the horizontal scanning pulse signal (horizontal scanning is performed), the N signal and the A image signal of one row as a read target row can be read out.

[0065] At time t412, the horizontal synchronization signal SYNC supplied from the controller 104 to the vertical scanning circuit 103 again transitions from the low level to the high level. At this time, the selection transistor M4 of each pixel 102 belonging to the nth row remains in the on state, and the state in which the nth row is selected is maintained.

[0066] During the period from time t413 to time t414, the vertical scanning circuit 103 maintains the control signal PRES(n) of the nth row at the low level. That is, during the period from time t413 to time t414, neither the reset of the floating diffusion FD nor the N conversion is performed. During the period from time t406 to time t415, the charge transferred from the photodiode PDA is stored in the floating diffusion FD.

[0067] Next, at time t415, the vertical scanning circuit 103 controls each of the control signals PTXA(n) and PTXB(n) of the nth row from the low level to the high level. This turns on the transfer transistors M1A and M1B of each pixel 102 belonging to the nth row, and the charge accumulated in the photodiode PDB is transferred to the floating diffusion FD. This adds the charge transferred from the photodiode PDB to the charge transferred from the photodiode PDA in the floating diffusion FD. The potential of the floating diffusion FD, that is, the potential of the gate of the amplification transistor M3 is set to a potential corresponding to the total amount of the charges transferred from the photodiodes PDA and PDB. At this time, since the selection transistor M4 of each pixel 102 belonging to the nth row remains in the on state, the A+B image signal as a pixel signal corresponding to the total amount of the charges generated by the photodiodes PDA and PDB is output to the vertical output line 111.

[0068] Next, at time t416, the vertical scanning circuit 103 controls each of the control signals PTXA(n) and PTXB(n) of the nth row from the high level to the low level. This turns off the transfer transistors M1A and M1B of each pixel 102 belonging to the nth row. Even after the transfer transistors M1A and M1B are turned off, the A+B image signal is continuously output to the vertical output line 111.

[0069] The period from time t417 to time t418 is a period in which AD conversion processing (A+B conversion) is performed on the A+B image signal output to the vertical output line 111. The A+B image signal output to the vertical output line 111 is read by the read circuit 105 and converted into a digital signal by the AD conversion circuit 106. The digital signal of the A+B image signal obtained by the AD conversion circuit 106 can be stored in the same storage region of the memory 107 as the storage region of the A image signal.

[0070] The period from time t419 to time t420 is a period in which the N signal and the A+B image signal stored in the memory 107 are transferred to the processor 109. The horizontal scanning circuit 108 outputs a horizontal scanning pulse signal to the memory 107. The memory 107 transfers the N signal and the A+B image signal corresponding to the address (column) selected by the horizontal scanning pulse signal to the processor 109. When this operation is repeatedly performed from the 0th column to the mth column while changing the address selected by the horizontal scanning pulse signal (horizontal scanning is performed), the N signal and the A+B image signal of one row as a read target row can be read.

[0071] As described above, the A image signal of one row as a read target row, the N signal corresponding to the A image signal, the A+B image signal, and the N signal corresponding to the A+B image signal can be read.

[0072] The processor 109 performs processing of subtracting the N signal corresponding to the A image signal from the A image signal, thereby suppressing noise to be superimposed on the A image signal. Further, the processor 109 performs processing of subtracting the N signal corresponding to the A+B image signal from the A+B image signal, thereby suppressing noise to be superimposed on the A+B image signal. By subtracting the A image signal from the A+B image signal, the B image signal required to perform focus detection can be obtained.

[0073] Thereafter, the vertical scanning circuit 103 cancels the selection of the nth row by controlling the control signal PSEL(n) of the nth row from the high level to the low level, completes the read operation for one row of the focus detection data row, and shifts to the read operation of the next row.

[0074] When the pixel signal is read from the pixel 102 of the captured image data row, as shown in Figure 3 , the A+B image signal is read. On the other hand, when the pixel signal is read from the pixel 102 of the focus detection data row, as shown in Figure 4 , the A image signal and the A+B image signal are read from the same row.

[0075] Figure 5 is a view for conceptually illustrating the read operation in the photoelectric conversion device 100 according to the embodiment.Figure 5 An example of the arrangement of the pixel array 101 is shown on the left side. Figure 5 A conceptual diagram showing signals output from the photoelectric conversion device 100 in the output order from the left side while starting a new line at the reference horizontal synchronization signal SYNC is shown on the right side. Figure 5 The width in the horizontal direction on the right side corresponds to the length of one horizontal period defined by the interval of the horizontal synchronization signal SYNC.

[0076] The plurality of pixels 102 forming the pixel array 101 includes light-receiving pixels and optical black (OB) pixels, in each of which the photodiodes PDA and PDB are not shaded in the light-receiving pixels, and in each of which the photodiodes PDA and PDB are shaded in the optical black pixels. In the pixel array 101, the region in which the light-receiving pixels are arranged is a light-receiving pixel region 502, and the region in which the OB pixels are arranged is a reference pixel region (also referred to as an optical black region) 501. Referring to Figure 5 , a dot pattern is applied to the reference pixel region 501 so as to visually easily distinguish the reference pixel region 501 from the light-receiving pixel region 502. An arbitrary region can be provided in the reference pixel region 501, and the provided region can be used to generate a correction value (clamp value) to be used in correction processing of output data for the light-receiving pixels. This region will be referred to hereinafter as a clamp value acquisition region.

[0077] In the reference pixel region 501, a region in contact with the upper side of the pixel array 101 will be referred to hereinafter as a VOB pixel region 506, and a region in contact with the left side of the pixel array 101 will be referred to hereinafter as an HOB pixel region 507. The VOB pixel region 506 is a region including OB pixels arranged in a different row from the light-receiving pixels in the light-receiving pixel region 502. The HOB pixel region 507 is a region including OB pixels arranged in a different column from the light-receiving pixels in the light-receiving pixel region 502. In one example, a clamp value acquisition region is provided in some rows of the VOB pixel region 506.

[0078] On the right side of Figure 5 , a detailed view of the pixel signals read out from the pixel array 101 is shown. Referring to Figure 5 , in order to visually easily distinguish the signals read out from the focus detection data lines from the signals read out from the captured image data lines, the signals read out from the focus detection data lines are shaded. For example, according to the timing shown in Figure 3 , during the horizontal period 503, A+B image signals are output from a given captured image data line of the pixel array 101. Further, according to the timing shown in Figure 4From a given focus detection data row of the pixel array 101, the A image signal is output during the horizontal period 504 and the A+B image signal is output during the horizontal period 505, as shown in FIG. 6. As described above, in the readout operation of the photoelectric conversion device 100 according to this embodiment, the focus detection data is discretely output while the captured image data is output.

[0079] Next, the signal processing performed by the processor 109 will be described. The processor 109 first performs processing of subtracting the N signal corresponding to the A+B image signal from the A+B image signal. This suppresses noise from the A+B image signal. Next, the A+B image signal from which noise has been suppressed is subjected to general correction processing (which will be described later). Thereafter, processing of subtracting the clamp value generated from the reference pixel region 501 is performed. The processing of subtracting the clamp value is correction processing of setting the black level in accordance with the reference level.

[0080] Figure 6 is a block diagram showing an example of the arrangement of a corrector 600 arranged in the processor 109 for performing general correction processing in the photoelectric conversion device 100 according to this embodiment. The corrector 600 including a completer 601, a correction value obtainer 602, and a correction applier 603 performs general correction processing on a signal input to the corrector 600. The functions of the processor 109 including the corrector 600 can be implemented when a CPU or an MPU reads out a program and executes it, or can be implemented by a circuit for implementing the functions of an ASIC, an FPGA, or the like.

[0081] In order to prevent a signal having an abnormal value due to a failure or a burst noise (hereinafter, will be referred to as defective data) from affecting the general correction processing (which will be described later), the completer 601 replaces the value of the defective data with a reference value. For example, the completer 601 can detect signal data exceeding a predetermined threshold among the signals output from the pixel array 101 and input to the corrector 600 as defective data. For example, the average value or the median value within a predetermined region can be used as the reference value.

[0082] The correction value obtainer 602 generates a correction value in accordance with a signal output from a region of the pixel array 101 that has been designated for generating the correction value. As Figure 6 As shown in FIG. 6, a plurality of correction value obtainers 602a and 602b can be arranged in the corrector 600. If a plurality of correction value obtainers 602a and 602b are arranged in the corrector 600, each of the plurality of correction value obtainers 602a and 602b can individually obtain a correction value. The correction applier 603 corrects a signal read out from the pixel array 101 based on the correction value generated by the correction value obtainer 602.

[0083] Next, a case where each pixel 102 is driven in the drive mode in which the captured image data (A+B image signal) is output and the drive mode in which the focus detection data (A image signal and A+B image signal) is output will be exemplified. Figure 7 is a view showing an example of the regions Pos and Neg of the pixel array 101 that have been designated for generating the correction value and the region Cor of the pixel array that has been designated for applying the correction value. As Figure 7 indicated in (a) of

[0084] The correction value obtainer 602 will be described in detail with reference to Figure 8 The correction value obtainer 602 includes the region controller 701, the representative value generators 702 and 703, and the correction value generator 704. Based on the drive mode set for each pixel among the plurality of drive modes, the region controller 701 selects, from the regions Pos and Neg of the pixel array 101 that have been designated for generating the correction value, a group of pixels 102 whose signals are to be used for generating the correction value.

[0085] The representative value generator 702 generates a representative value of the region Pos based on the signals read out from the group of pixels selected from the region Pos. The representative value generator 703 generates a representative value of the region Neg based on the signals read out from the group of pixels selected from the region Neg. Each of the representative values of the regions Pos and Neg can be, for example, an average value or a median value of the signals read out from the group of pixels selected from each of the regions Pos and Neg.

[0086] The correction value generator 704 generates a correction value from the representative values of the regions Pos and Neg. For example, the correction value generator 704 can calculate a difference between the representative values of the regions Pos and Neg as the correction value. Alternatively, for example, the correction value generator 704 can generate the correction value by performing adjustment calculation (for example, by multiplying a difference between the representative values of the regions Pos and Neg by a predetermined coefficient). The representative value generators 702 and 703 and the correction value generator 704 function as a correction value generation circuit in cooperation with each other, which generates the representative value of the region Pos from the group of pixels selected for generating the representative value of the region Pos, generates the representative value of the region Neg from the group of pixels selected for generating the representative value of the region Neg, and generates the correction value from the representative values of the regions Pos and Neg.

[0087] The correction value generator 704 can include a storage device 705 that stores a plurality of correction values generated in a plurality of frames, respectively. The photoelectric conversion device 100 can be configured to read out a signal from the pixel array 101 within a continuous frame. This allows the corrector 600 to apply a correction value generated in the same frame among the plurality of correction values or a correction value generated in a frame before one or more frames among the plurality of correction values to the signal read out from the pixel array 101. As described above, the storage device 705 functions as a storage device that stores a plurality of correction values generated in a plurality of frames, respectively.

[0088] For example, as shown in FIG. 6B, the correction value acquirer 602 can include two storage devices 705 that each store a correction value, and one of a correction value acquired from a signal of a current frame and a correction value acquired from a signal of an immediately preceding frame can be applied to a signal output from the pixel array 101. For example, based on a setting of an image capturing parameter or the like, a correction value acquired in a certain frame can be selected as a correction value to be output. Figure 8

[0089] The area controller 701 will be described with reference to FIG. 7. The area controller 701 selects a group of pixels for generating a representative value of the area Pos from the area Pos by combining a position setting and a flag setting. Similarly, the area controller 701 selects a group of pixels for generating a representative value of the area Neg from the area Neg by combining a position setting and a flag setting. Figure 9

[0090] The position setting determines an area based on a coordinate position in the pixel array 101 of the pixel 102 that output a signal. For example, a rectangular area is specified in the pixel array 101 (a start point and an end point in the vertical and horizontal directions are specified), and it is determined that a signal output from a pixel 102 in the area is a signal output from the area Pos or Neg.

[0091] ​​The flag setting determines whether an input flag (metadata) other than the signal output from the pixel 102 matches a predetermined value. For example, the area controller 701 selects a signal whose flag matches a predetermined value as the pixel group for generating each of the representative values of the areas Pos and Neg. The flag is used to identify a characteristic of the signal such as a driving mode of the pixel 102. For example, the flag can be a signal indicating whether the signal is an A image signal or an A+B image signal. Alternatively, for example, the flag can be a signal indicating whether the signal is a signal of the pixel 102 of a line read out in the focus detection driving mode or a signal of the pixel 102 of a line read out in the captured image driving mode. For example, the flag can be a signal for identifying a gain in each driving mode, for example, different gains can be output for one pixel output. For example, the flag can be a signal indicating whether the signal is a pixel output from the VOB pixel region 506, the HOB pixel region 507, or the light-receiving pixel region 502. Examples of the flag are a signal indicating whether the current time is in a vertical blanking interval and a frame number. Further, in addition to the corresponding pixel, a driving mode of a surrounding pixel (for example, whether the pixel is adjacent to a focus detection pixel) can be included as the flag.

[0092] By combining the position setting and the flag setting, the area controller 701 finally determines the pixel 102 that forms the pixel group for generating each of the representative values of the areas Pos and Neg among the plurality of pixels 102 arranged in the pixel array 101. For example, the position of the area Pos of the pixel array 101 is determined by the position setting, and by the flag setting, the pixel group for generating the representative value of the area Pos formed by the pixels 102 that have read out the signal in the captured image driving mode is selected from the area Pos. As described above, the area controller 701 functions as a selection circuit that selects the pixel group for generating the representative value of the area Pos from the area Pos of the pixel array 101 designated for generating the correction value based on the driving mode set for each pixel 102 among the plurality of driving modes.

[0093] In addition to the position setting and the flag setting, the area controller 701 can select the pixel group for generating each of the representative values of the areas Pos and Neg by combining a pattern setting. As described above, the area controller 701 functions as a selection circuit that selects the pixel group for generating the representative value of the area Pos from the area Pos of the pixel array 101 designated for generating the correction value based on the driving mode set for each pixel 102 among the plurality of driving modes. Figure 5 As shown on the right, in the pixel array 101, the pixels 102 driven in each of the plurality of driving modes can be arranged to have a periodic pattern. The pattern setting selects the pixel group for generating each of the representative values of the areas Pos and Neg based on a predetermined periodic pattern. For example, flexible settings can be made using the pattern setting. For example, in the case of the pixel array 101 in which the pixels 102 driven in each of the plurality of driving modes are arranged to have a periodic pattern, the area controller 701 can select the pixel group for generating each of the representative values of the areas Pos and Neg based on the periodic pattern. Figure 5In the style shown, the rows before and after the pixel row of the readout signal in focus detection driven mode can be selected as the pixel group for each of the representative values ​​of the generated region Pos and Neg.

[0094] Next, we will refer to Figure 10 The correction applicator 603 is described below. The correction applicator 603 includes a region controller 801 and an applicator 802. The region controller 801 may have the same arrangement as the region controller 701. Based on a driving mode set for each pixel 102 among multiple driving modes, the region controller 801 selects a group of pixels from a specified region Cor of the pixel array 101 to which the correction value generated by the correction value acquirer 602 will be applied. As described above, by combining position settings, flag settings, style settings, etc., the region controller 801 selects the group of pixels formed by pixels arranged in region Cor to which the correction value will be applied.

[0095] The applicator 802 applies a correction value to each of the signals output from the pixels 102 forming the pixel group selected by the region controller 801. For example, the applicator 802 can perform correction using arithmetic operations, such as adding, subtracting, multiplying, or dividing the correction values ​​generated by the correction value acquirer 602, for each signal output from the pixel 102 selected by the region controller 801. For example, the correction value generator 704 generates a correction value based on the difference between the representative values ​​of region Pos and Neg. In this case, the applicator 802 can add the correction value to the signal read from each pixel 102 selected by the region controller 801. Alternatively, for example, the correction value generator 704 generates a correction value based on the ratio between the representative values ​​of region Pos and Neg. In this case, the applicator 802 can correct the gain component by multiplying by the correction value. As described above, the correction applicator 603 serves as a correction circuit for correcting the signals read from the pixel array 101 based on the correction value.

[0096] The applicator 802 of the correction applicator 603 does not need to apply the correction value to the signal read from pixels other than the pixel group selected by the region controller 801. For example, in the signals read from the pixel array 101, signals read from pixels other than the pixel group selected by the region controller 801 can be output without undergoing general correction processing.

[0097] If as Figure 6 If multiple correction value acquirers 602 are present as shown, then multiple applicators 802a and 802b can be used as follows: Figure 10The locations shown are arranged corresponding to the correction value acquirer 602. The result of applying correction values ​​1 and 2 sequentially by multiple applicators 802a and 802b is output from the correction applicator 603. In this case, the pixel group to which the correction value is applied can be set individually for each of the correction values ​​1 and 2. If correction is performed for all signals output from the pixel array 101, the area controller 801 is not required.

[0098] Figure 11A and Figure 11B Examples of general correction processing in the corrector 600 according to this embodiment are shown respectively. For example, consider the following case: a correction value is obtained from the difference between focus detection data and captured image data in VOB pixel region 506, and this correction value is used to correct the focus detection data in light receiving pixel region 502. In this case, as Figure 11A As shown, the VOB pixel region 506 is set to a position setting as the pixel group selected for generating the representative value of region Pos, and the signal read out in the capture image driving mode is set to a flag setting. This selects the pixel group formed by pixels 102 arranged in the VOB pixel region 506 and the capture image data row as the pixel group for generating the representative value of region Pos. Furthermore, the VOB pixel region 506 is set to a position setting as the pixel group selected for generating the representative value of region Neg, and the signal read out in the focus detection driving mode is set to a flag setting. This selects the pixel group formed by pixels 102 arranged in the VOB pixel region 506 and the focus detection data row as the pixel group for generating the representative value of region Neg. As described above, the pixel group for generating the representative value of region Pos and the pixel group for generating the representative value of region Neg can be formed by pixels 102 arranged in the reference pixel region 501 (optical black region) from among a plurality of pixels 102.

[0099] As a selection of the pixel group for applying correction values ​​generated based on representative values ​​of region Pos and Neg, the light-receiving pixel region 502 is set to a position setting, and the signal read out in focus detection drive mode is set to a flag setting. This selects the pixel group formed by pixels 102 arranged in the light-receiving pixel region 502 and the focus detection data row as the pixel group to which the correction values ​​will be applied. Furthermore, if as... Figure 8 As shown, the correction value acquirer 602 includes two storage devices 705, each storing a correction value. The correction value acquired from the signal generated in the same frame can be applied to each of the signals read from the pixel group formed by the pixels 102 arranged in the light receiving pixel area 502 and the focus detection data row.

[0100] For example, the vertical black spot can be large, and the offset amount between the VOB pixel region 506 and the light-receiving pixel region 502 can be different. In this case, as shown in FIG. 6B, the HOB pixel region 507 is set to the position setting, and the signal read out in the focus detection drive mode is set to the flag setting as the selection of the pixel group for generating the representative value of the region Pos. This selects the pixel group formed by the pixels 102 arranged in the HOB pixel region 507 and the focus detection data row as the pixel group for generating the representative value of the region Pos. Further, the HOB pixel region 507 is set to the position setting, and the signal read out in the capture image drive mode is set to the flag setting as the selection of the pixel group for generating the representative value of the region Neg. This selects the pixel group formed by the pixels 102 arranged in the HOB pixel region 507 and the capture image data row as the pixel group for generating the representative value of the region Neg. Figure 11B

[0101] As the selection of the pixel group to which the correction value generated in accordance with the representative values of the regions Pos and Neg is to be applied, the pixel array 101 is set to the position setting and the signal read out in the focus detection drive mode is set to the flag setting. This selects the pixel group formed by the pixels 102 arranged in the pixel array 101 and the focus detection data row as the pixel group to which the correction value is to be applied. If, as shown in FIG. 6B, the correction value acquirer 602 includes two storage devices 705 each storing the correction value, the correction value acquired from the signal of the immediately preceding frame can be applied to each of the signals read out from the pixel group formed by the pixels 102 arranged in the light-receiving pixel region 502 and the focus detection data row. Figure 8

[0102] As described above, the photoelectric conversion device 100 including the corrector 600 for performing the general correction processing according to this embodiment can flexibly control the position setting of the regions Pos and Neg to generate the representative values of the regions Pos and Neg. This can realize the general correction processing that can cope with various drive modes and the arrangement pattern of the pixel rows operated in the respective drive modes.

[0103] Next, a method of applying the above-described general correction processing to a case where the correction values are not uniform in the pixel array 101 will be described. In the photoelectric conversion device 100, if the correction values are ideally measured directly at all coordinate positions, the accuracy of the correction processing is high. However, since the calculation cost for acquiring the correction values is high, the correction values at each coordinate position (interpolated correction values) are obtained by interpolation from the correction values measured discretely.

[0104] Figure 12 As shown in FIG. 7B, the HOB pixel region 507 is set to the position setting, and the signal read out in the focus detection drive mode is set to the flag setting as the selection of the pixel group for generating the representative value of the region Pos. This selects the pixel group formed by the pixels 102 arranged in the HOB pixel region 507 and the focus detection data row as the pixel group for generating the representative value of the region Pos. Further, the HOB pixel region 507 is set to the position setting, and the signal read out in the capture image drive mode is set to the flag setting as the selection of the pixel group for generating the representative value of the region Neg. This selects the pixel group formed by the pixels 102 arranged in the HOB pixel region 507 and the capture image data row as the pixel group for generating the representative value of the region Neg. Figure 8 ​​The arrangement of the correction value acquirer 1002, which is a variation of the correction value acquirer 602 illustrated in FIG. 6, will be described. The correction value acquirer 1002 is different from the correction value acquirer 602 illustrated in FIG. 6 in that Figure 8 The correction value acquirer 1002 illustrated in FIG. 10 is different from the correction value acquirer 602 illustrated in FIG. 6 in that Figure 12 The correction value acquirer 1002 illustrated in FIG. 10 is different from the correction value acquirer 602 illustrated in FIG. 6 in that

[0105] In addition, the operation of the photoelectric conversion device 100 including the correction value acquirer 1002 will be described by assuming that the frame in which the correction value is generated and the frame in which the correction value is applied are different frames. For example, the region controller 701, the representative value generators 702 and 703, and the correction value generator 704 operate in the frame in which the correction value is generated, and the interpolator 1005 operates in the frame in which the correction value is applied to the signal read out from the pixel array 101 by the correction applicator 603. As the frame in which the correction value is generated, a frame in which the signal is read out from the pixel array 101 in a state in which light is shielded or a state in which the control signals PTXA and PTXB are fixed at the low level can be used. For example, the correction value can be generated using the correction value generation frame in which the signal is read out from the pixel array 101 in a state in which light is shielded or a state in which the control signals PTXA and PTXB are fixed at the low level.

[0106] First, the operation in the frame in which the correction value is generated will be described. The region controller 701 sets a plurality of regions Pos and a plurality of regions Neg in the position setting, and determines that the signal output from the pixels 102 in the plurality of regions is the signal output from the regions Pos and Neg. For example, the regions Pos and Neg can be set in the nine regions 1 to 9 illustrated in FIG. 6 in one frame. The first regions Pos and Neg are set in the region 1 illustrated in FIG. 6, the second regions Pos and Neg are set in the region 2 illustrated in FIG. 6, and the ninth regions Pos and Neg are set in the region 9 illustrated in FIG. 6. In addition, the regions Pos and Neg are set in the regions 3 to 8 illustrated in FIG. 6. Figure 13A The first regions Pos and Neg are set in the region 1 illustrated in FIG. 6, the second regions Pos and Neg are set in the region 2 illustrated in FIG. 6, and the ninth regions Pos and Neg are set in the region 9 illustrated in FIG. 6. In addition, the regions Pos and Neg are set in the regions 3 to 8 illustrated in FIG. 6. Figure 13A The first regions Pos and Neg are set in the region 1 illustrated in FIG. 6, the second regions Pos and Neg are set in the region 2 illustrated in FIG. 6, and the ninth regions Pos and Neg are set in the region 9 illustrated in FIG. 6. In addition, the regions Pos and Neg are set in the regions 3 to 8 illustrated in FIG. 6. Figure 13A The first regions Pos and Neg are set in the region 1 illustrated in FIG. 6, the second regions Pos and Neg are set in the region 2 illustrated in FIG. 6, and the ninth regions Pos and Neg are set in the region 9 illustrated in FIG. 6. In addition, the regions Pos and Neg are set in the regions 3 to 8 illustrated in FIG. 6. Figure 13A The first regions Pos and Neg are set in the region 1 illustrated in FIG. 6, the second regions Pos and Neg are set in the region 2 illustrated in FIG. 6, and the ninth regions Pos and Neg are set in the region 9 illustrated in FIG. 6. In addition, the regions Pos and Neg are set in the regions 3 to 8 illustrated in FIG. 6. Figure 13A The first regions Pos and Neg are set in the region 1 illustrated in FIG. 6, the second regions Pos and Neg are set in the region 2 illustrated in FIG. 6, and the ninth regions Pos and Neg are set in the region 9 illustrated in FIG. 6. In addition, the regions Pos and Neg are set in the regions 3 to 8 illustrated in FIG. 6.

[0107] Next, the representative value generator 702 generates the representative values of the region Pos and the region Neg of each of the regions 1 to 9. Using the representative values of the region Pos and the region Neg of each of the regions 1 to 9, the correction value generator 704 generates the correction values of each of the regions 1 to 9 from the representative values of the region Pos and the region Neg of each of the regions 1 to 9. For example, as described above, the correction value generator 704 generates the difference or the ratio between the representative values of the region Pos and the region Neg as the correction values. This generates a plurality of correction values corresponding to the plurality of regions 1 to 9 illustrated in FIG. 8 in the frame in which the correction values are generated. Figure 13A

[0108] Next, the operation in the frame in which the correction values are applied will be described. The interpolator 1005 functions as an interpolation circuit that generates an interpolated correction value corresponding to the position of each of the plurality of pixels 102 in the pixel array 101 on the basis of the positions of the plurality of regions 1 to 9 in the pixel array 101 and the plurality of correction values. For example, on the basis of the positions of the centers of gravity of the regions 1 to 9 in the pixel array 101 and the correction values of the regions 1 to 9, and the coordinate positions of each of the pixels 102, the interpolator 1005 obtains the interpolated correction value of each of the pixels 102 through an interpolation process. As illustrated in, for example, FIG. 9, the interpolation process can obtain the correction values through bilinear interpolation from the positions of the centers of gravity C2, C3, C5, and C6 of the four regions 2, 3, 5, and 6 surrounding the coordinate position P of the pixel 102 of interest. The number of regions and the interpolation method at the time of performing the interpolation process are not limited to the above number of regions and the above interpolation method. The interpolator 1005 can generate the interpolated correction value from the correction values of arbitrary regions through predetermined curve approximation. Alternatively, the interpolator 1005 can generate the interpolated correction value using an extrapolation method in addition to the interpolation method illustrated in, for example, FIG. 9. Figure 13B Figure 13B

[0109] After generating the interpolated correction value corresponding to each of the pixels 102 arranged in the pixel array 101, the correction applier 603 performs correction by applying the interpolated correction value to the signal read out from the pixel array 101. In the frame in which the correction values are applied, new correction values to be used for correction in the next frame can be generated and acquired.

[0110] As described above, even if the correction values are not uniform in the plane of the pixel array 101, the accuracy of the correction process can be improved. Also in this case, the photoelectric conversion device 100 according to the embodiment including the corrector 600 for performing a general correction process can flexibly control the position settings of the region Pos and the region Neg to generate the representative values of the region Pos and the region Neg. This can realize a general correction process that can cope with various driving modes and the arrangement pattern of the pixel rows operating in each driving mode.

[0111] ​​​In each of the above-described embodiments, the case where the signal is read out from the pixel 102 arranged in the pixel array 101 in the captured image driving mode in which the A+B image signal is acquired and the focus detection driving mode in which the A image signal and the A+B image signal are acquired has been described. However, the application of the general correction processing is not limited to this, and the general correction processing can be applied to the case where a plurality of pixels 102 arranged in the pixel array 101 are configured to be operable in a plurality of driving modes using different signal readout methods. For example, in order to expand the dynamic range of an image obtained by the photoelectric conversion device 100, the gain at the time of reading out the signal can be changed depending on the pixel row of the pixel array 101, or the exposure time (accumulation time) can be changed depending on the pixel row of the pixel array 101. Further, for example, in order to improve the readout speed of the photoelectric conversion device 100, the signal can be read out simultaneously from a plurality of pixel rows, or the pixel row in which the signal is read out and the pixel row in which the signal is not read out can be mixed. If a plurality of driving modes are provided to read out the signal from the pixel 102, as described above, the positions of the regions Pos and Neg can be freely designated by arranging the region controller 701 in the corrector 600. This copes with various driving modes, and thus the correction processing can be accurately performed.

[0112] Further, depending on the type of the driving mode and the request for the precision of the correction processing, the region controller 701 can select one pixel group from one region in the pixel array 101 and generate a correction value based on the driving mode set for each pixel in a plurality of driving modes. Even in this case, the region for generating the correction value can be freely designated, and the correction processing can be accurately performed according to various driving modes.

[0113] Now, an application example of the photoelectric conversion device 100 according to this embodiment will be described. Figure 14 is a block diagram showing a schematic configuration of a photoelectric conversion system 1400 in which the photoelectric conversion device 100 is incorporated.

[0114] The photoelectric conversion device 100 of the above-described embodiment can be applied to various photoelectric conversion systems. Examples of the photoelectric conversion system to which the photoelectric conversion device is applicable are a digital still camera, a digital camcorder, a surveillance camera, a copier, a facsimile device, a mobile phone, a car-mounted camera, and an observation satellite. A camera module including an optical system such as a lens and the photoelectric conversion device is also included in the photoelectric conversion system. Figure 14 A block diagram of a digital still camera as an example of these is shown.

[0115] Figure 14The photoelectric conversion system 1400 illustrated in FIG. 12 includes the photoelectric conversion device 100, a lens 1402 that forms an optical image of an object on a photoelectric converter (e.g., the pixel array 101, e.g., the light-receiving pixel region 502) of the photoelectric conversion device 100, an aperture 1403 configured to change the amount of light passing through the lens 1402, and a shutter 1401 configured to protect the lens 1402. The lens 1402 and the aperture 1403 form an optical system that converges light to the photoelectric conversion device 100. The photoelectric conversion device 100 converts the optical image formed by the lens 1402 into an electric signal.

[0116] The photoelectric conversion system 1400 further includes a signal processor 1407 that is an image generator configured to generate an image by processing an output signal output from the photoelectric conversion device 100. The signal processor 1407 performs operations for various corrections and compressions as needed, thereby outputting image data. The signal processor 1407 can be formed on a semiconductor substrate provided with the photoelectric conversion device 100, or can be formed on a semiconductor substrate different from the photoelectric conversion device 100. In addition, the photoelectric conversion device 100 and the signal processor 1407 can be formed on the same semiconductor substrate.

[0117] The photoelectric conversion system 1400 further includes a memory unit 1410 configured to temporarily store image data, and an external interface unit (external I / F unit) 1413 configured to communicate with an external computer or the like. Furthermore, the photoelectric conversion system 1400 includes a recording medium 1412 such as a semiconductor memory configured to record or read image capture data, and a recording medium control interface unit (recording medium control I / F unit) 1411 configured to perform recording or reading on the recording medium 1412. Note that the recording medium 1412 can be built into the photoelectric conversion system 1400 or can be detachable.

[0118] Furthermore, the photoelectric conversion system 1400 includes a general-purpose control / arithmetic unit 1409 that controls various operations and the entire digital still camera, and a timing generator 1408 that outputs various timing signals to the photoelectric conversion device 100 and the signal processor 1407. In this example, a timing signal or the like can be input from the outside, and the photoelectric conversion system 1400 only needs to include at least the photoelectric conversion device 100 and the signal processor 1407 that processes an output signal output from the photoelectric conversion device 100.

[0119] The photoelectric conversion device 100 outputs an image capture signal to the signal processor 1407. The signal processor 1407 performs predetermined signal processing on the image capture signal output from the photoelectric conversion device 100, and outputs image data. The signal processor 1407 generates an image using the image capture signal.

[0120] As described above, according to this embodiment, a photoelectric conversion system to which the above-described photoelectric conversion apparatus 100 (for example, an image capturing apparatus) is applied can be implemented.

[0121] Next, a photoelectric conversion system and a transport apparatus incorporating the photoelectric conversion apparatus 100 of this embodiment will be described with reference to Figure 15A and Figure 15B Figure 15A and 15B are block diagrams showing a configuration of a photoelectric conversion system 1500 incorporating the photoelectric conversion apparatus 100 according to this embodiment and a configuration of a transport apparatus 1501 incorporating the photoelectric conversion system 1500.

[0122] Figure 15A An example of the photoelectric conversion system 1500 with respect to a vehicle-mounted camera is shown. The photoelectric conversion system 1500 includes an image processor 1512 that performs signal processing such as image processing on a plurality of image data acquired by the photoelectric conversion apparatus 100 of this embodiment and a disparity acquirer 1514 that calculates a disparity (a phase difference between disparity images) from the plurality of image data that has undergone the signal processing by the image processor 1512. The photoelectric conversion system 1500 further includes a distance acquirer 1516 that calculates a distance to a target based on the calculated disparity and a collision determiner 1518 that determines whether there is a possibility of collision based on the calculated distance. Here, the disparity acquirer 1514 and the distance acquirer 1516 are examples of a distance information acquirer that acquires distance information to a target. That is, the distance information is information about a disparity, an amount of defocus, a distance to a target, and the like. The collision determiner 1518 can determine a possibility of collision using one of a plurality of pieces of distance information. The distance information acquirer can be implemented by a specially designed hardware or can be implemented by a software module. The distance information acquirer can be implemented by an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), or can be implemented by a combination of these.

[0123] ​The photoelectric conversion system 1500 is connected to a vehicle information acquisition device 1520 of a transportation device 1501 (e.g., a vehicle) including a driving device, and can acquire vehicle information such as a vehicle speed, a yaw rate, and a steering angle. The photoelectric conversion system 1500 is also connected to a control ECU 1530, which is a control device configured to output a control signal for generating a braking force to the vehicle based on a determination result of the collision determiner 1518. Furthermore, the photoelectric conversion system 1500 is connected to an alarm device 1540, which gives an alarm to the driver based on the determination result of the collision determiner 1518. For example, if the collision possibility is high as the determination result of the collision determiner 1518, the control ECU 1530 performs vehicle control of braking, releasing an accelerator pedal, or suppressing an engine output, thereby avoiding a collision and reducing damage. The alarm device 1540 gives an alarm, displays alarm information on a screen of a car navigation system or the like, or applies a vibration to a seat belt or a steering wheel, thereby alarming to the user.

[0124] In this embodiment, the photoelectric conversion system 1500 captures a periphery of the transportation device 1501, for example, a front side or a rear side. Figure 15B The photoelectric conversion system 1500 in a case where the front side of the transportation device 1501 is captured (an image capture range 1550) is shown. The vehicle information acquisition device 1520 transmits an instruction to the photoelectric conversion system 1500 or the photoelectric conversion device 100. With this configuration, the accuracy of distance measurement can be further improved.

[0125] The following example is described: a driving device 1560 such as a brake, an accelerator, and an engine of the transportation device 1501 is controlled based on information obtained by the photoelectric conversion device 100, so as not to collide with another vehicle. However, the present application is not limited to this, and the system can also be applied to control of performing automated driving following another vehicle or performing automated driving without deviating from a lane. Furthermore, an example in which the photoelectric conversion system 1500 equipped with the photoelectric conversion device 100 is incorporated into the transportation device 1501 is described. However, the photoelectric conversion device 100 can be incorporated into the vehicle information acquisition device 1520, the control ECU 1530, or the alarm device 1540. The photoelectric conversion system 1500 equipped with the photoelectric conversion device 100 can be applied not only to a vehicle such as a car but also to a transportation device including a driving device such as a ship, an airplane, a railway vehicle, or an industrial robot. In addition, the photoelectric conversion system can be applied not only to a moving body but also to a device such as an intelligent transportation system (ITS) in which object recognition is widely used.

[0126] While the application has been described with reference to example embodiments, it will be understood by those skilled in the art that various changes can be made and equivalents can be substituted for elements thereof without departing from the scope of the application. In addition, many modifications can be made to adapt to a particular situation and the appended claims should be construed as including both the foregoing example embodiments and all such modifications and equivalents.

Claims

1. An apparatus for photoelectric conversion, the apparatus comprising a pixel array in which a plurality of pixels are arranged in a matrix, and a signal processor configured to process a signal read out from the pixel array, wherein the plurality of pixels being configured to be operable in a plurality of driving modes using different signal readout methods, and the signal processor comprising: a selection circuit configured to select, from a region of the pixel array that has been designated for generating a correction value, a first group of pixels and a second group of pixels among the plurality of pixels based on a driving mode set for each pixel among the plurality of driving modes; a correction value generation circuit configured to generate the correction value in accordance with a first representative value based on a signal read out from the first group of pixels and a second representative value based on a signal read out from the second group of pixels; and a correction circuit configured to correct a signal read out from the pixel array based on the correction value, wherein the apparatus for photoelectric conversion is configured such that the signal is read out from the pixel array within consecutive frames, wherein the correction value generation circuit comprises a storage device configured to store a plurality of correction values respectively generated in a plurality of frames, wherein the correction circuit is configured to apply, to a signal read out from the pixel array in a first frame, a correction value generated in a second frame preceding the first frame among the plurality of correction values, and wherein, in the first frame and the second frame, the plurality of pixels are configured to be operable in a same driving mode.

2. The apparatus according to claim 1, wherein each of the plurality of pixels comprises a plurality of photoelectric converters, and the plurality of driving modes includes a first mode in which a signal corresponding to a first charge obtained by adding charges generated by the plurality of photoelectric converters is read out, and a second mode in which the signal corresponding to the first charge and a second signal corresponding to charges generated by some of the plurality of photoelectric converters are read out.

3. The apparatus according to claim 1, wherein each of the plurality of pixels comprises a plurality of photoelectric converters, the plurality of driving modes includes a first mode in which a signal corresponding to a first charge obtained by adding charges generated by the plurality of photoelectric converters is read out, and a second mode in which the signal corresponding to the first charge and a second signal corresponding to charges generated by some of the plurality of photoelectric converters are read out, the first group of pixels is formed by pixels among the plurality of pixels that are driven in the first mode, and the second group of pixels is formed by pixels among the plurality of pixels that are driven in the second mode.

4. The apparatus of claim 3, wherein, The correction value generation circuit generates the correction value based on a difference between the first representative value and the second representative value.

5. The apparatus of claim 3, wherein, The correction value generation circuit generates the correction value based on a ratio between the first representative value and the second representative value.

6. The apparatus according to claim 1, wherein the pixel array includes an optically black region that is shielded from light, and the first group of pixels and the second group of pixels are formed by pixels among the plurality of pixels that are arranged in the optically black region.

7. The apparatus according to claim 1, wherein the pixels driven in each of the plurality of driving modes are arranged in a periodic pattern in the pixel array, and the selection circuit selects the first group of pixels further based on the periodic pattern.

8. The apparatus according to claim 1, wherein the selection circuit selects a third group of pixels from the designated area of the pixel array among the plurality of pixels based on the driving mode set for each pixel among the plurality of driving modes, and the correction circuit applies the correction value to a signal read out from the third group of pixels.

9. The apparatus of claim 8, wherein, the correction circuit does not apply the correction value to a signal read out from a group of pixels other than the third group of pixels.

10. The apparatus according to claim 1, wherein the first group of pixels is arranged on a plurality of areas of the pixel array, the correction value generation circuit generates a plurality of correction values respectively corresponding to the plurality of areas, the photoelectric conversion apparatus further includes an interpolation circuit configured to generate an interpolated correction value corresponding to a position of each of the plurality of pixels in the pixel array based on positions of the plurality of areas in the pixel array and the plurality of correction values, and the correction circuit performs correction by applying the interpolated correction value to a signal read out from the pixel array.

11. A photoelectric conversion system, comprising: the photoelectric conversion apparatus according to any one of claims 1 to 10; and a signal processor configured to process a signal output from the photoelectric conversion apparatus.

12. A transport apparatus including a driving apparatus, comprising: a control apparatus including the photoelectric conversion apparatus according to any one of claims 1 to 10, and configured to control the driving apparatus based on information obtained by the photoelectric conversion apparatus.

13. A signal processing apparatus for processing a signal read out from a pixel array in which a plurality of pixels operable in a plurality of driving modes using different signal readout methods are arranged in a matrix, the signal processing apparatus comprising: a selection circuit configured to select a first group of pixels and a second group of pixels among the plurality of pixels from an area of the pixel array of the plurality of pixels that has been designated for generating a correction value based on a driving mode set for each pixel among the plurality of driving modes; a correction value generation circuit configured to generate the correction value in accordance with a first representative value based on a signal read out from the first group of pixels and a second representative value based on a signal read out from the second group of pixels; and a correction circuit configured to correct a signal read out from the pixel array based on the correction value, wherein the correction value generation circuit includes a storage device configured to store a plurality of correction values respectively generated in a plurality of frames read out from the pixel array within consecutive frames, wherein the correction circuit is configured to apply a correction value generated in a second frame preceding a first frame among the plurality of correction values to a signal read out from the pixel array in the first frame, and In the first frame and the second frame, the plurality of pixels are configured to be capable of operating in the same driving mode.

Images