Photoelectric conversion device, image processing method, imaging system, mobile object, and device

Abstract

To realize highly accurate correction processing.SOLUTION: A photoelectric conversion device according to one embodiment comprises: a calculation unit which calculates a correction value based on pixel values read from a first region and a second region; and a correction unit which corrects offset of the pixel value according to incident light based on the correction value. The calculation unit calculates an initial value of each of a plurality of first correction components based on the first pixel value read from the first region, updates each of the plurality of first correction components based on the second pixel value read from the second region and a prescribed second correction component, and calculates a correction value by using the updated first correction component and the second correction component.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a photoelectric conversion device, an image processing method, an imaging system, a mobile object, and an apparatus. [Background technology]

[0002] In recent years, development has been made on photoelectric conversion devices to be mounted on digital still cameras, video cameras, mobile phones, etc. For example, a CMOS (Complementary Metal Oxide Semiconductor) sensor is widely known as a photoelectric conversion device.

[0003] In a CMOS sensor, a certain row of pixels is selected from among a plurality of pixels arranged in a matrix in the row and column directions, and pixel signals from all pixels in the selected row are read out simultaneously. It is known that an offset is contained in the readout image signal of a CMOS sensor. Patent Documents 1 and 2 describe circuits that correct the offset of pixel signals from effective pixels using pixel signals from an optical black (hereinafter referred to as "OB") region. However, with the recent trend toward higher image quality in CMOS sensors, the number of rows and vertical signal lines has increased, making it difficult to perform high-precision correction processing with the configurations described in Patent Documents 1 and 2. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-005329 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-106186 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide a photoelectric conversion device, an image processing method, a photoelectric conversion system, a mobile object, and an apparatus that are capable of performing highly accurate correction processing. [Means for solving the problem]

[0006] A photoelectric conversion device disclosed in this specification includes a pixel section including a plurality of pixels arranged in a matrix, a drive section that drives the pixels in a first region by a first method and drives the pixels in a second region by a second method, a calculation section that calculates a correction value based on pixel values ​​read from the first region and the second region, and a correction section that corrects an offset of the pixel value according to incident light based on the correction value, wherein the calculation section calculates an initial value of each of a plurality of first correction components based on the first pixel value read from the first region, updates each of the plurality of first correction components based on the second pixel value read from the second region and a predetermined second correction component, and calculates the correction value using the updated first correction component and the second correction component.

[0007] Another image processing method disclosed in this specification includes the steps of calculating a first correction component based on first data in a first region of image data, calculating a second correction component using second data in a second region of the image data and the first correction component, updating the first correction component using the second data and the second correction component, and correcting the second data using the updated first correction component and the second correction value. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a photoelectric conversion device, an image processing method, and a photoelectric conversion system that are capable of performing highly accurate correction processing. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram of a photoelectric conversion device according to a first embodiment. [Figure 2] FIG. 2 is an equivalent circuit diagram of a pixel according to the first embodiment. [Figure 3A] FIG. 2 is a diagram showing a pixel unit according to the first embodiment. [Figure 3B] FIG. 2 is a diagram showing a pixel unit according to the first embodiment. [Figure 4A] 3A and 3B are diagrams illustrating driving of a pixel unit by a driving unit according to the first embodiment. [Figure 4B] 3A and 3B are diagrams illustrating driving of a pixel unit by a driving unit according to the first embodiment. [Figure 4C] 3A and 3B are diagrams illustrating driving of a pixel unit by a driving unit according to the first embodiment. [Figure 5] FIG. 2 is a block diagram of a signal processing unit in a calculation unit according to the first embodiment. [Figure 6] FIG. 2 is a block diagram of an OB clamp calculation unit according to the first embodiment. [Figure 7A] FIG. 2 is a block diagram of a first processing unit and a second processing unit according to the first embodiment. [Figure 7B] FIG. 2 is a block diagram of a first processing unit and a second processing unit according to the first embodiment. [Figure 7C] FIG. 2 is a block diagram of a first processing unit and a second processing unit according to the first embodiment. [Figure 7D] FIG. 2 is a block diagram of a first processing unit and a second processing unit according to the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of the configuration of a photoelectric conversion system according to a second embodiment. [Figure 9] 10 is a flowchart of a correction calculation according to the second embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of curve approximation according to the second embodiment. [Figure 11] FIG. 10 is a block diagram showing a schematic configuration of an imaging system according to a third embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of the configuration of an imaging system and a moving object according to a fourth embodiment. [Figure 13] FIG. 11 is a block diagram showing a schematic configuration of a device according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The above-described embodiments can be combined in any manner as long as no contradiction occurs. In addition, in the following embodiments, an imaging device will be mainly described as an example of a photoelectric conversion device. However, the embodiments are not limited to imaging devices, and can also be applied to other examples of photoelectric conversion devices. For example, there are distance measurement devices (devices that measure distance using focus detection or TOF (Time Of Flight)), photometry devices (devices that measure the amount of incident light, etc.), etc.

[0011] [First embodiment] (Overall composition) The circuit configuration of a photoelectric conversion device 10 according to this embodiment will be described with reference to Fig. 1. The photoelectric conversion device 10 is, for example, a CMOS image sensor, and includes a pixel unit 11, a drive unit 12, a readout unit 14, a signal processing unit 15, and a timing generator (TG) 16.

[0012] The pixel unit 11 includes a plurality of pixels 110 arranged in a two-dimensional array, each of which includes a photoelectric conversion unit that generates and accumulates signal charges corresponding to the amount of light received. In this specification, the row direction refers to the horizontal direction in the drawing, and the column direction refers to the vertical direction in the drawing. Microlenses and color filters may be arranged on the pixels 110. The color filters are, for example, primary color filters of red (R), blue (B), and green (Gr, Gb), and are arranged on each pixel 110 in a Bayer array. Some of the pixels 110 are light-shielded as OB pixels (optical black pixels). The pixel unit 11 may also include NULL pixels that are not connected to vertical signal lines VL. The pixel unit 11 may further include a ranging row in which focus detection pixels that output pixel signals for focus detection are arranged, and multiple imaging rows in which imaging pixels that output pixel signals for generating an image are arranged. A vertical signal line VL is arranged for each column of pixels 110, and multiple pixels 110 in the same column output pixel signals to a common vertical signal line VL. The pixel section 11 may include NULL pixels that are not connected to the vertical signal lines VL. Although not shown, a constant current circuit is connected to each of the vertical signal lines VL, and the constant current circuit functions as a load circuit for the pixels 110.

[0013] The drive unit 12 is composed of a shift register, a gate circuit, a buffer circuit, etc., and outputs control signals to the pixels 110 based on vertical synchronization signals, horizontal synchronization signals, clock signals, etc., to drive the pixels 110 row by row. In this embodiment, the drive unit 12 is capable of performing vertical scanning, which sequentially selects four rows of the pixel unit 11 per horizontal scanning period. The selected four rows of pixels 110 are read out by the readout unit 14 via vertical signal lines VL. In the following description, the four vertical signal lines VL that can be simultaneously read out are referred to as vertical signal lines VL1 to VL4. For each horizontal scanning period (hereinafter sometimes referred to as "1H"), pixel signals are read out from the vertical signal lines VL1 to VL4, and the OB correction value is calculated.

[0014] The readout unit 14 is capable of reading out pixel signals from the pixels 110 via the vertical signal lines VL, and includes an amplifier circuit, an ADC (Analog to Digital Converter) circuit, a column memory, a horizontal scanning circuit, etc. The readout unit 14 is capable of simultaneously reading out pixel signals from multiple rows, for example, four rows, of vertical signal lines VL, and outputting them as digital data (pixel values).

[0015] The signal processing unit 15 performs digital signal processing such as OB clamping, digital gain, digital correlated double sampling, digital offset, and linearity correction on the pixel data output from the readout unit. In this embodiment, the signal processing unit 15 can perform digital signal processing such as CDS (Correlated Double Sampling) processing and OB clamping on pixel signals for four rows during one horizontal scanning period. The signal processing unit 15 also includes an LVDS (Low Voltage Differential Signaling) serial output circuit, and outputs the processed digital signals to the outside of the photoelectric conversion device at high speed and with low power consumption.

[0016] (pixel section) Fig. 2 is a diagram showing an example of the configuration of a pixel 110 according to this embodiment. As shown in Fig. 1, a plurality of pixels 110 are arranged in a matrix across a plurality of rows and a plurality of columns. The pixel 110 may include, for example, a photoelectric conversion unit 111, a transfer transistor 112, a floating diffusion (FD) 113, an amplification transistor 114, a selection transistor 115, and a reset transistor 116.

[0017] The photoelectric conversion unit 111 is composed of, for example, a photodiode. The anode of the photoelectric conversion unit 111 is connected to a ground node (GND) and the cathode is connected to the source of the transfer transistor 112. The drain of the transfer transistor 112 is connected to the source of the reset transistor 116 and the gate of the amplification transistor 114. The connection node of the drain of the transfer transistor 112, the source of the reset transistor 116, and the gate of the amplification transistor 114 is a so-called floating diffusion (FD) 113. The FD 113 includes a capacitance component and functions as a charge storage unit and a charge-voltage conversion unit. The drain of the reset transistor 116 and the drain of the amplification transistor 114 are connected to a power supply node (voltage VDD). The source of the amplification transistor 114 is connected to the drain of the selection transistor 115. The source of the selection transistor 115 is connected to a vertical signal line VL. The names of the source and drain of a transistor may differ depending on the conductivity type, application, etc. of the transistor, and the source and drain may be referred to inversely.

[0018] The row selection line for each row includes a signal line connected to the gate of the transfer transistor 112, a signal line connected to the gate of the reset transistor 116, and a signal line connected to the gate of the selection transistor 115. A control signal PTX is supplied from the driver 12 to the signal line connected to the gate of the transfer transistor 112. A control signal PRES is supplied from the driver 12 to the signal line connected to the gate of the reset transistor 116. A control signal PSEL is supplied from the driver 12 to the signal line connected to the gate of the selection transistor 115. When each transistor is an N-type transistor, the transistor is turned on by a high-level control signal and turned off by a low-level control signal.

[0019] When light is incident on the pixel unit 11, the photoelectric conversion unit 111 of each pixel 110 converts the incident light into charges corresponding to the amount of light (photoelectric conversion) and accumulates the generated charges. When the transfer transistor 112 is turned on, it transfers the charges of the photoelectric conversion unit 111 to the FD 113. The FD 113 holds the charges transferred from the photoelectric conversion unit 111. The charges transferred from the photoelectric conversion unit 111 are converted into a voltage by the capacitance component of the FD 113.

[0020] A voltage VDD is supplied to the drain of the amplification transistor 114, and a bias current is supplied to the source from a current source 118 via a selection transistor 115. The amplification transistor 114 forms an amplifier (source follower circuit) with its gate as an input node. This causes the amplification transistor 114 to output a signal based on the voltage of the FD 113 to the vertical signal line VL via the selection transistor 115. The reset transistor 116, when turned on, resets the FD 113 to a voltage corresponding to the voltage VDD.

[0021] The transfer transistor 112, reset transistor 116, and selection transistor 115 of the pixel 110 are controlled row by row by control signals PTX, PRES, and PSEL supplied from the drive unit 12. Normal readout of the pixel 110 will now be described. The pixel 110 is reset except during the accumulation period (the period from shutter scanning to readout scanning, which will be described later). During shutter scanning, when the reset by the reset transistor 116 is released, the photoelectric conversion unit 111 begins accumulating charge. Next, after a predetermined accumulation time, readout scanning is performed. During readout scanning, the row is selected, and a pixel signal (N signal) at the reset level of the FD 113 is read out. Thereafter, the charge in the photoelectric conversion unit 111 is transferred to the FD 113, and a pixel signal (S signal) based on the charge is read out. Thereafter, the photoelectric conversion unit 111 and FD 113 are reset. The signal processing unit 15 calculates the difference between the S signal and the N signal (CDS processing) to obtain a pixel signal based on the charge accumulated in the photoelectric conversion unit 111.

[0022] 3A and 3B are diagrams illustrating a pixel unit according to this embodiment. The pixel unit 11 illustrated in FIG. 3A includes an aperture region 11a and OB regions 11b and 11c, while the pixel unit 11 illustrated in FIG. 3B includes an aperture region 11a and OB regions 11b, 11c, and 11d. The aperture region 11a includes unshielded pixels 110 and can output pixel signals corresponding to incident light. The OB regions 11b, 11c, and 11d include shielded pixels 110 or NULL pixels not connected to the vertical signal line VL and are used for offset correction, etc. In FIGS. 3A and 3B, the OB region 11b is located in the row above the aperture region 11a, and the OB region 11c is located in the column to the left of the aperture region 11a. In FIG. 3B, the OB region 11d is located in the row below the aperture region 11a. The positions of the OB regions are not limited to those illustrated in FIGS. 3A and 3B, and may be located in the column to the right of the aperture region 11a.

[0023] As will be described later, the photoelectric conversion device of this embodiment can perform cyclic readout, in which multiple rows of pixels 110 are successively read out multiple times, and normal readout other than cyclic readout. Cyclic readout may be performed, for example, in the upper row of the OB region 11b, which is divided into two, as shown in FIG. 3A, or in the OB region 11d provided below the opening region 11a, as shown in FIG. 3B. In either case, normal readout is typically performed after cyclic readout.

[0024] (Drive unit) The driving unit 12 in this embodiment can realize various driving operations in the pixel unit 11 for shutter driving, OB value calculation, and the like.

[0025] 4A, 4B, and 4C are diagrams showing the driving of the pixel unit 11 by the driver 12 according to this embodiment. In the diagrams, the horizontal axis represents time, and the vertical axis represents the row direction (vertical direction) of the pixel device. Hatched rectangles represent shutter scanning, and white rectangles represent readout scanning. The numbers in the rectangles in the readout scanning correspond to the vertical signal lines VL1 to VL4.

[0026] FIG. 4A is a diagram showing driving of the pixel unit 11 by the driver unit 12 according to this embodiment, illustrating shutter driving. At time t101, the driver unit 12 performs shutter scanning by setting the control signals PTX and PRES for the first to fourth rows to high level, and resets the charges in the photoelectric conversion units 111 and FDs 113 of the pixels 110 to voltage VDD. Next, the driver unit 12 sets the control signals PTX and PRES for the first to fourth rows from high level to low level, and the photoelectric conversion units 111 begin accumulating charges in response to incident light. At time t102, the driver unit 12 sets the control signals PTX and PRES for the fifth to eighth rows to high level, and resets the charges in the photoelectric conversion units 111 and FDs 113 of the pixels 110 to voltage VDD. Next, the driver unit 12 sets the control signals PTX and PRES for the fifth to eighth rows to high level, and resets the charges in the photoelectric conversion units 111 and FDs 113 of the pixels 110 to voltage VDD. Next, the driver unit 12 sets the control signals PTX and PRES from high level to low level, and the photoelectric conversion units 111 begin accumulating charges in response to incident light. Similarly, after time t103, the drive unit 12 performs shutter scanning every four rows.

[0027] From time t111 to time t112, a predetermined accumulation time (exposure time) after the shutter scan, the driver 12 performs readout scanning of the first to fourth rows. That is, the driver 12 sets the control signals PSEL for the first to fourth rows to high level and turns on the selection transistors 115. As a result, N signals at reset are output from the pixels 110 to the vertical signal lines VL1 to VL4. Next, the driver 12 sets the control signals PTX for the first to fourth rows to high level and turns on the transfer transistors 112. As a result, the charges accumulated in the photoelectric conversion units 111 are transferred to the FDs 113. The voltage of the FDs 113 decreases in accordance with the transferred charges, and S signals are output from the sources of the amplification transistors 114 via the selection transistors 115 to the vertical signal lines VL1 to VL4, respectively. That is, the pixel signals of the pixels 110 in the first to fourth rows are simultaneously read out via the vertical signal lines VL1 to VL4.

[0028] Between times t112 and t113, the driver 12 sets the control signal PSEL for the fifth to eighth rows to high level, and then sets the control signal PTX to high level. As a result, pixel signals are output from the pixels 110 in the fifth to eighth rows to the vertical signal lines VL1 to VL4 in sequence. In this manner, readout scanning is performed every four rows in the pixel unit 11. The numbers in the readout scanning rectangles correspond to the vertical signal lines VL1 to VL4. That is, the pixels 110 in the first, fifth, ninth, ..., (4×N-3)th rows are read out via the vertical signal line VL1, and the pixels 110 in the second, sixth, tenth, ..., (4×N-2)th rows are read out via the vertical signal line VL2. In addition, the pixels 110 in the 3rd, 7th, 11th, ..., (4×N-1)th rows are read out via vertical signal line VL3, and the pixels 110 in the 4th, 8th, 12th, ..., (4×N)th rows are read out via vertical signal line VL4.

[0029] The driver 12 may perform various driving operations in addition to normal readout, for example, readout of NULL pixels. The driver 12 may also perform NN readout, which reads out S signals with the transfer transistors turned off after reading out N signals. The driver 12 may also perform cyclic readout, which repeatedly reads out a predetermined region.

[0030] 4B is a diagram showing driving of the pixel unit 11 by the driver 12 according to this embodiment, illustrating cyclic readout driving. As shown in FIGS. 3A and 3B, cyclic readout can be performed in the OB region 11b or the OB region 11d.

[0031] From time t201 to t202, the drive unit 12 sequentially turns on the selection transistors 115 and transfer transistors 112 in the first to fourth rows, causing the pixels 110 to output pixel signals of N and S signals to the vertical signal lines VL1 to VL4. Similarly, from time t202 to t203, the drive unit 12 causes the pixels 110 in the fifth to eighth rows to output pixel signals to the vertical signal lines VL1 to VL4, and from time t203 to t204, the drive unit 12 causes the pixels 110 in the ninth to twelfth rows to output pixel signals to the vertical signal lines VL1 to VL4. Between times t204 and t205, the drive unit 12 again causes the pixels 110 in the first to fourth rows to output pixel signals to the vertical signal lines VL1 to VL4, and between times t205 and t206, the drive unit 12 causes the pixels 110 in the fifth to eighth rows to output pixel signals to the vertical signal lines VL1 to VL4. Similarly, readout scanning is repeated for every four rows of the pixels 110 in the first to twelfth rows.

[0032] In the above-mentioned cyclic readout, shutter scanning is not necessarily required; in this case, the period of the readout scanning corresponds to the accumulation time. Note that shutter scanning may be performed between readout scans, but the period of the readout scanning is the upper limit of the accumulation time. Although there is a restriction on the accumulation time, cyclic readout makes it possible to acquire a large amount of data from a small area in a short time.

[0033] FIG. 4C is a diagram showing driving of the pixel section 11 by the driver section 12 according to this embodiment, and shows driving in which normal readout is performed after cyclic readout.

[0034] In the following description of this embodiment, it is assumed that normal readout is performed after cyclic readout of a portion of the OB region 11b. Normal readout can be performed on the remaining region of the OB region 11b that has not been subjected to cyclic readout, the OB region 11c, the opening region 11a, etc. As shown in FIG. 3A, the region above the OB region 11b may be assigned to cyclic readout, and the region below the OB region 11b may be assigned to normal readout. As shown in FIG. 3B, the OB region 11d located below the opening region 11a may be assigned to cyclic readout, and the OB region 11b located above the opening region 11a may be assigned to normal readout. When cyclic readout is performed in the opening region 11a, it is preferable to perform NN readout. In either case, normal readout can be performed after cyclic readout.

[0035] In the following description, it is assumed that the cyclic readout is performed on, for example, 12 rows of the OB area 11b. Reading for three horizontal scanning periods is repeated eight times to obtain cyclic readout data for 24 horizontal scanning periods. The cyclic readout data is sometimes referred to as advance data or reference data, and the normally readout data is sometimes referred to as actual data.

[0036] During the cyclic readout period T1, cyclic readout is performed for every four rows of the 12 rows of the OB region 11b. That is, from time t301 to t302, the drive unit 12 performs readout scanning of the first to fourth rows, and causes the pixels 110 to output N and S pixel signals to the vertical signal lines VL1 to VL4. Similarly, the drive unit 12 performs readout scanning of the fifth to eighth rows (times t302 to t303), readout scanning of the ninth to twelfth rows (times t303 to t304), and readout scanning of the first to fourth rows (times t304 to t305). Furthermore, from time t305 to t306, the drive unit 12 performs readout scanning of the fifth to eighth rows and shutter scanning of the thirteenth to sixteenth rows of the aperture region 11a. Thereafter, shutter scanning is similarly performed following the cyclic readout. The pixel signal during cyclic readout is used as the initial value of the first correction component, which will be described later.

[0037] During a normal readout period T2 after the cyclic readout period T1, normal readout is performed on every four rows in the remaining area of ​​the OB region 11b that was not subjected to cyclic readout, the OB region 11c, and the aperture region 11a. Between times t310 and t311, the driver 12 causes the pixels 110 in the 13th to 16th rows to output N and S pixel signals to the vertical signal lines VL1 to VL4. Similarly, between times t311 and t312, the driver 12 causes the pixels 110 in the 17th to 20th rows to output pixel signals to the vertical signal lines VL1 to VL4. The pixel signals during period T20 (times t310 to t312) are used as initial values ​​for the filter section of the signal processor 15, as described below. During period T21 after time t312, the signal processor 15 uses the pixel signals during normal readout to calculate a second correction component that depends on the vertical position. Furthermore, the signal processing unit 15 continues to sequentially update the first correction component using the pixel signal and the second correction component, as will be described in detail later.

[0038] (Signal processing unit) FIG. 5 is a block diagram of the signal processing unit 15 according to this embodiment. The signal processing unit 15 includes calculation units 15R, 15B, 15Gr, and 15Gb, one for each color of the color pixels 110 arranged in a Bayer array. That is, the signal processing unit 15 includes a calculation unit 15R that processes signals from the R pixels 110, a calculation unit 15B that processes signals from the B pixels 110, a calculation unit 15Gr that processes signals from the Gr pixels 110, and a calculation unit 15Gb that processes signals from the Gb pixels 110. The calculation unit 15R processes signals from the R pixels 110 read out via vertical signal lines VL1 and VL3, while the calculation unit 15B processes signals from the B pixels 110 read out via vertical signal lines VL2 and VL4. The calculation unit 15Gr processes signals from the Gr pixels 110 read out via vertical signal lines VL1 and VL3, while the calculation unit 15Gb processes signals from the Gb pixels 110 read out via vertical signal lines VL2 and VL4.

[0039] The calculation unit 15R includes CDS calculation units 151R and 152R, a memory unit 153R, an OB clamp calculation unit 150R, and a data output unit 154R. The CDS calculation units 151R and 152R calculate the difference between the S signal and the N signal read from the column memory in the readout unit 14 via the vertical signal line VL1, and output the result to the memory unit 153R. That is, the CDS calculation unit 151R calculates the difference between the S signal R_VL1S and the N signal R_VL1N read from the R pixel 110 via the vertical signal line VL1. Similarly, the CDS calculation unit 152R calculates the difference between the S signal R_VL3S and the N signal R_VL3N read from the R pixel 110 via the vertical signal line VL3.

[0040] The memory unit 153R can store data for two rows (one horizontal scanning period) of R pixels 110 read out via the vertical signal lines VL1 and VL3. The memory unit 153R alternately outputs data for the vertical signal lines VL1 and VL3 to the OB clamp calculation unit 150R every half horizontal scanning period. At this time, the memory unit 153R adds an identifier (vertical signal line identifier) ​​indicating the vertical signal line VL from which each data was read to the data. The OB clamp calculation unit 150R calculates the OB value primarily using data from the OB region 11b and subtracts the OB value from the data from the aperture region 11a to achieve the OB clamp function (OB value correction function). The OB value includes dark current, circuit-dependent components, and the like, and therefore contains a different value for each vertical signal line VL, i.e., a vertical line difference. For this reason, the OB clamp calculation unit 150R separately calculates the OB value for the vertical signal line VL1 and the OB value for the vertical signal line VL3. The OB clamp calculation unit 150R outputs the OB clamped data to the data output unit 154R. The data output unit 154R converts the input data into a predetermined format and outputs it to the outside of the device.

[0041] The other calculation units 15B, 15Gr, and 15Gb are configured similarly to the calculation unit 15R. That is, the calculation unit 15B can perform CDS processing and OB clamping on data of B pixels 110 read out from vertical signal lines VL2 and VL4. The calculation unit Gr can perform CDS processing and OB clamping on data of Gr pixels 110 read out from vertical signal lines VL1 and VL3, and the calculation unit 15Gb can perform CDS processing and OB clamping on data of Gb pixels 110 read out from vertical signal lines VL2 and VL4. In this way, the calculation units 15R, 15B, 15Gr, and 15Gb can calculate OB correction values ​​for each color of the color pixels 110 and perform OB clamping.

[0042] (OB clamp calculation section) FIG. 6 is a block diagram of the OB clamp calculation unit 150R in the calculation unit 15R. The other OB clamp calculation units 150B, 150Gr, and 150Gb are configured similarly, so only the OB clamp calculation unit 150R will be described below. As shown in FIG. 6, the OB clamp calculation unit 150R includes a defect correction unit 1501, an averaging unit 1502, a first processing unit 1503, a second processing unit 1504, a subtraction unit 1505, and a control unit 1506. When OB value calculation is performed on data input to the OB clamp calculation unit 150R, the data is input to the defect correction unit 1501. When OB clamping is performed on data input to the OB clamp calculation unit 150R, the data is input to the subtraction unit 1505.

[0043] The defect correction unit 1501 performs defect correction processing using a predetermined value or the OB value calculated by the second processing unit 1504 as a reference value. Specifically, the defect correction unit 1501 obtains the normal range of the OB value by performing a predetermined calculation on the reference value, and determines data outside the normal range as defect data. Data determined to be defect data is replaced with the reference value or deleted. This makes it possible to prevent data outside the normal range from affecting the OB value calculation.

[0044] The averaging unit 1502 calculates an average value by dividing the integrated value of data over ½ horizontal scanning period, i.e., one horizontal scanning period for each vertical signal line VL, by the number of data. The first processing unit 1503 calculates a first correction component for each vertical signal line VL using the average value calculated by the averaging unit 1502 and the second correction component calculated by the second processing unit 1504. The second processing unit 1504 calculates a second correction component using the average value calculated by the averaging unit 1502 and the first correction component calculated by the first processing unit 1503. The second processing unit 1504 also calculates an OB value from the first correction component and the second correction component, and outputs the OB value to the subtraction unit 1505 and the defect correction unit 1501.

[0045] The subtraction unit 1505 subtracts the OB value from the input data and performs OB clamping. The control unit 1506 controls the operation of the entire OB clamp calculation unit 150R. For example, the control unit 1506 determines whether the input data is data in an area that is the target for OB value calculation. Furthermore, the control unit 1506 controls switching of the operation modes of the first processing unit 1503 and the second processing unit 1504.

[0046] Figures 7A, 7B, 7C, and 7D are block diagrams of first processing unit 1503 and second processing unit 1504 according to this embodiment. Figure 7A shows the overall configuration of first processing unit 1503 and second processing unit 1504, and in Figures 7B, 7C, and 7D, blocks indicated by solid lines represent blocks in an operating state, and blocks indicated by dashed lines represent blocks in an inoperating state.

[0047] The first processing unit 1503 includes a subtraction unit 1530, a filter unit 1531, a filter unit 1532, an average value calculation unit 1533, and multiplexers 1534 and 1535. The multiplexers 1534 and 1535 include multiple switch circuits, and switch the respective switch circuits depending on the operating state of the signal processing unit 15. The first processing unit 1503 performs predetermined signal processing on the average values ​​R1 and R3 output from the averaging unit 1502 in FIG. 6 to calculate a first correction component Va that depends on the vertical signal line VL. The second processing unit 1504 includes a subtraction unit 1540, a filter unit 1541, and an adder 1542, and uses the first correction component Va to update a second correction component Vb that corresponds to the vertical position. The adder 1542 outputs the sum of the first correction component Va and the second correction component Vb as the OB value.

[0048] In the above configuration, the respective units may share a hardware configuration as long as they perform equivalent functions. For example, since the filter units 1531 and 1532 operate alternately, the filter units 1531 and 1532 may be configured by a common circuit.

[0049] FIG. 7B is a block diagram of the first processing unit 1503 and the second processing unit 1504 according to this embodiment, illustrating their operation during the cyclic readout period T1 (times t301 to t310) of FIG. 4C. The filter unit 1531 calculates an average value Va1 by dividing the integrated value of the average value data R1 from the averaging unit 1502 of FIG. 6 by the number of horizontal scanning periods. That is, the filter unit 1531 calculates the average value Va1 of the data R1 of the vertical signal line VL1 during the same period. Similarly, the filter unit 1532 calculates the average value Va3 of the data R3 of the vertical signal line VL3 during the same period. In this way, the filter units 1531 and 1532 calculate the average values ​​Va1 and Va3 for each vertical signal line VL based on the cyclic readout data R1 and R3. Furthermore, the average value calculation unit 1533 calculates an average value Va0 of the average values ​​Va1 and Va3. The average value Va0 represents the average value of the data of all vertical signal lines VL during the cyclic readout period. The calculated average values ​​Va1, Va3, and Va0 are classified according to the vertical signal line identifier as initial values ​​of first correction components, which will be described later, and are held in the filter units 1531, 1532, and average value calculation unit 1533. At the end of the cyclic readout (time t310), the first correction components Va1, Va3, and Va0 held in the filter units 1531 and 1532 and average value calculation unit 1533 are respectively expressed by the following equations.

[0050]

number

number

number

[0051] In the above formulas 1 to 3, "R 1,k "," "R 3,k " indicates data R1, R3 during cyclic readout of vertical signal lines VL1, VL3 in the k-th horizontal scanning period. "N1" indicates the number of horizontal scanning periods for cyclic readout, and "k" indicates the k-th horizontal scanning period among the 1st to N1th horizontal scanning periods.

[0052] Generally, the vertical line difference does not depend on the accumulation time, so the difference between the first correction component Va1 and the first correction component Va3 corresponds to the vertical line difference. By averaging the data R1 and R3 obtained during cyclic readout every horizontal scanning period over N1 horizontal scanning periods, the random noise contained in the data R1 and R3 is reduced. The first correction components Va1, Va3, and Va0 used as initial values ​​may be representative values ​​of the respective target regions (vertical signal lines VL), such as medians.

[0053] FIG. 7C is a block diagram of the first processing unit 1503 and the second processing unit 1504 according to this embodiment, illustrating their operating states during the normal readout period T20 (times t310 to t312) in FIG. 4C. As described above, the period T20 includes data S1 and S3 obtained during normal readout during the first few horizontal scanning periods of the normal readout period T2. The data S1 and S3 during the period T20 are used to calculate the initial value of the filter unit 1541. The data S1 and S3 averaged by the averaging unit 1502 are input to the second processing unit 1504. The first correction component Va calculated by the average calculation unit 1533 is output to the second processing unit 1504, and the subtraction unit 1540 calculates the difference between the data S1 and S3 and the first correction component Va. The filter unit 1541 calculates an average value Vb0 by dividing the integrated value of the difference by the number of horizontal scanning periods N2. The average value Vb0 is used as the initial value of the second correction component Vb, which will be described later. At the end of the first period T20 of normal readout (time t312), the second correction component Vb0 held in the filter unit 1541 is expressed by Equation 4.

number

[0054] In the above equation 4, "N2" represents the number of horizontal scanning periods in the period T20, and "S 1,k "," "S 3,k" represents the data S1 and S3 during normal reading of the vertical signal lines VL1 and VL3 in the k-th horizontal scanning period of the normal reading area. By subtracting the first correction component Va0 calculated by Equation 3 from the sum of the data S1 and S3, the level difference (DC difference) between the data R1 and R3 during cyclic reading and the data S1 and S3 during normal reading is reduced. As a result, the continuity of the smoothing process described later can be ensured, and rapid value changes can be suppressed. Note that the level difference between the data R1, R3 and the data S1, S3 is mainly due to the difference in the accumulation time.

[0055] FIG. 7D is a block diagram of the first processing unit 1503 and the second processing unit 1504 according to the present embodiment, showing the operating state during the normal reading period T21 (after time t312) in FIG. 4C. During the period T21, normal reading after the N2-th horizontal scanning period is performed. The subtraction unit 1530 sequentially updates the first correction components Va1 and Va3 based on the difference between the data S1 and S3 output from the averaging unit 1502 and the second correction component Vb. The calculated difference is classified according to the vertical signal line identifier and input to the filter unit 1531 or the filter unit 1532. The filter units 1531 and 1532 perform a smoothing process (low-pass filter process) using the input difference and the operation results held by the filter units 1531 and 1532 up to now. The smoothing process can be executed, for example, by the following first-order IIR (Infinite Impulse Response) process. Va 1,N ={Va 1,(N-1) ×A1+(S 1,N -Vb (N-1) )×(1 - A1)} (Equation 5) Va 3,N ={Va 3,(N-1) ×A1+(S 3,N -Vb (N-1) )×(1 - A1)} (Equation 6)

[0056] In the above Equations 5 and 6, "A1" represents the attenuation coefficient of the first-order IIR process and is a value in the range of 0 < A1 < 1. Also, "Va 1,N ", "Va 3,N" represents the first correction component Va updated in the N-th horizontal scanning period of each of the vertical signal lines VL1 and VL3, and "Vb (N-1) " represents the second correction component Vb in the (N-1)th horizontal scanning period. That is, the first correction component Va for each vertical signal line VL is updated using the second correction component Vb that depends on the vertical position and the data S.

[0057] The subtraction unit 1530 is not essential, and the input data "S 1,N "," "S 3,N " may be input to the filter units 1531 and 1532. However, if the image contains steep vertical shading, it is preferable to provide the subtraction unit 1530. The reason for this is explained in detail below. The calculation results of Equations 5 and 6 are updated only once per horizontal scanning period. Therefore, if the image contains steep vertical shading, a relatively large attenuation coefficient A1 is required to smooth the vertical shading. In other words, to track steep vertical shading, the cutoff frequency of the low-pass filter must be set high. On the other hand, to avoid the effects of noise, a small attenuation coefficient A1 is desirable during high-sensitivity shooting, for example. Therefore, the attenuation coefficient A1 can be determined by a trade-off between conflicting factors. If the subtraction unit 1530 is provided, the second correction component Vb is subtracted from the data S1 and S3, thereby reducing the effects of steep vertical shading. The calculation results of filter unit 1541 are updated twice per horizontal scanning period, so it can more easily follow steep vertical shading than filter units 1531 and 1532. In this way, subtraction unit 1530 is particularly effective when vertical shading is large.

[0058] The subtraction unit 1540 calculates the difference between the input data S1, S3 and the first correction components Va1, Va3 respectively. Since the first correction components Va1, Va3 include the vertical line difference, the vertical line difference is reduced or not included in the difference obtained by subtracting the first correction components Va1, Va3. The difference obtained by the subtraction unit 1540 is input to the filter unit 1541. The data S1 of the vertical signal line VL1 and the data S3 of the vertical signal line VL3 are alternately input to the filter unit 1541 every half horizontal scanning period. The filter unit 1541 performs a smoothing process using the input difference and the calculation result held in the filter unit 1541. The smoothing process can be implemented by, for example, the following first-order IIR process. Vb N ={Vb N-(1 / 2) ×A2+(S i,N -Va i,N )×(1 - A2)} (Equation 7)

[0059] In Equation 7 above, "i" indicates the identification number of the vertical signal line. "A2" indicates the attenuation coefficient of the first-order IIR process and is a value in the range of 0 < A2 < 1. "Vb N " represents the second correction component in the Nth horizontal scanning period. The second correction component Vb N may vary depending on the vertical position but is a common value for the vertical signal lines VL1, VL3. In Equation 7, for the data S1 of the vertical signal line VL1, the first correction component Va1, which is the calculation result of the filter unit 1531, is referred to, and for the data S3 of the vertical signal line VL3, the first correction component Va3, which is the calculation result of the filter unit 1532, is referred to. The second correction component Vb is common to the vertical signal lines VL1, VL3 and does not depend on the vertical signal line VL. Therefore, the second correction component Vb obtained from the data S1 of the vertical signal line VL1 can be referred to for the second correction component Vb of the vertical signal line VL3, and the second correction component Vb obtained from the data S3 of the vertical signal line VL3 can be referred to for the second correction component Vb of the vertical signal line VL1. "Vb N-(1 / 2)" indicates that the next second correction component Vb is obtained from the second correction component Vb that is half the horizontal scanning period before. Even if vertical shading is present, the filter unit 1532 can output the second correction component Vb with reduced noise while following the vertical shading.

[0060] The adder 1542 adds the updated first correction component Va and second correction component Vb and outputs the added value as the OB value (offset value). The subtractor 1505 of the OB clamp calculator 150 subtracts the OB value from the pixel data in the opening region 11a, thereby realizing OB value correction that reduces vertical line difference, vertical shading, and the like.

[0061] As described above, the photoelectric conversion device in this embodiment decomposes the OB value into a first correction component and a second correction component, and sequentially updates the first correction component using the second correction component. First, multiple first correction components Va, including vertical line differences, are calculated from the data R during cyclic readout. The first correction components Va for each of the multiple vertical signal lines VL are sequentially updated using the second correction component common to the multiple first correction components and the data S during normal readout. Therefore, this embodiment makes it possible to achieve high-precision OB correction without providing many OB regions.

[0062] In particular, the photoelectric conversion device in this embodiment subtracts the first correction component Va from the data S during normal readout to obtain the second correction component Vb that does not depend on the vertical signal line VL. Therefore, when calculating the second correction component Vb, it is not necessary to obtain a correction value separately for each vertical signal line VL, and high-precision OB correction is possible with a small amount of data and OB region.

[0063] Furthermore, the initial value of the first correction component Va can be calculated from the data R of a smaller area by cyclic readout. By setting the number of repetitions of cyclic readout according to the amount of noise, the vertical line difference can be calculated with high accuracy from a small area.

[0064] The effects of this embodiment will now be described in comparison with other configurations. Generally, the OB value includes a horizontal position-dependent component, a vertical position-dependent component, and a position-independent component. The OB clamp in this embodiment targets correction of the vertical position-dependent component and the position-independent component, and the horizontal position-independent component may be corrected by a different configuration. The vertical position-dependent component and the position-independent component each include a component that depends on the vertical signal line VL and other components such as dark current. Here, it is assumed that, among the vertical position-dependent components, the component that depends on the vertical signal line is small.

[0065] Another method for OB correction is to calculate the OB value by calculating the data for each row and smoothing the vertical data for each vertical signal line independently. This method requires a large OB area depending on the number of vertical signal lines to reduce random noise. In addition, the sampling interval becomes wider, making it difficult to follow vertical shading.

[0066] On the other hand, in this embodiment, as described above, the OB value is decomposed into a first correction component Va and a second correction component Vb, and the first correction component Va is sequentially updated using the second correction component Vb. Therefore, when calculating the second correction component Vb, it is not necessary to calculate correction values ​​separately for each vertical signal line VL, and it is also not necessary to provide many OB regions corresponding to the number of vertical signal lines VL. Furthermore, in the data S during normal readout, the first correction component Va is sequentially updated, so the first correction component Va can track vertical shading. Therefore, according to this embodiment, it is possible to calculate the OB value with high accuracy without requiring many OB regions.

[0067] Although the above description assumes that the number of vertical signal lines VL is four, the number of vertical signal lines VL may be any number equal to or greater than two. Although the first correction component Va is classified (calculated) for each vertical signal line VL, it may also be classified for each group of other circuit elements. For example, if a floating diffusion is shared by multiple pixels, differences in the OB value may occur depending on the order in which the pixels transfer charge to the floating diffusion. In this case, the first correction component Va may be classified according to the order in which the pixels use the floating diffusion FD. Furthermore, if the pixel is a distance measurement pixel having multiple photoelectric conversion units, the first correction component Va may also be classified according to the pixel drive method. For example, the first correction component Va may be classified according to the readout method for each of the images A, B, and A+B. Furthermore, if the image A is read out intermittently, the first correction component Va may also be classified according to whether the row is from which the image A is read out. Multiple classifications may also be combined. The greater the number of classifications, the more accurately the OB value can be calculated.

[0068] Furthermore, although the filter units 1531, 1532, and 1541 have been described as being first-order IIR, they may be other IIRs such as second-order IIRs, or FIRs (Finite Impulse Responses).

[0069] Furthermore, cyclic readout may further include a combination of NN readout and NULL readout. Alternatively, if vertical scanning is thinned out during normal readout and not all rows are read out, the rows that are not read out may be used to calculate the OB value. Note that NN readout is preferable when the pixel 110 used to calculate the OB value is included in the aperture region 11a. In this way, various readout methods can be used to calculate OB values ​​such as vertical line difference.

[0070] The number of cycles (number of horizontal scanning periods N1) of the cyclic readout can be set arbitrarily. The greater the number of cycles, the more noise contained in the initial value of the first correction component Va is reduced. On the other hand, from the viewpoints of power consumption and frame rate, a smaller number of cycles is preferable. The number of cycles can be determined by a trade-off among multiple factors. For example, the number of cycles may be dynamically changed depending on the vertical scanning mode, ISO sensitivity, temperature, etc.

[0071] Although the vertical signal line identifier is used to classify the first correction component Va, the vertical signal line identifier changes every half of the horizontal scanning period. Therefore, the vertical signal line identifier is not essential, and the vertical signal lines may be identified from the timing of the horizontal scanning.

[0072] [Second embodiment] Next, a photoelectric conversion system according to this embodiment will be described. In the first embodiment, the OB clamp calculation is performed within the photoelectric conversion device, but in this embodiment, the OB clamp calculation can be performed outside the photoelectric conversion device.

[0073] FIG. 8 shows an example of the configuration of a photoelectric conversion system according to this embodiment. The photoelectric conversion system includes a photoelectric conversion device 10 and a computing device 80. The computing device 80 receives data from the photoelectric conversion device 10 via a communication line 20 and executes a computer program to perform a predetermined correction calculation. The communication line 20 may be wired or wireless, and may also be public wireless communication or short-range wireless communication. The computing device 80 is capable of performing the functions of the OB clamp calculation unit 150 according to the first embodiment and receives data input to the OB clamp calculation unit 150. The computing device 80 stores one frame of data, including cyclically read data and normally read data, in its memory and can repeatedly refer to the data stored in the memory during calculation processing. The computing device 80 may be a standalone personal computer, an edge terminal, a cloud server, or the like.

[0074] FIG. 9 is a flowchart showing the operation of the arithmetic device 80 in this embodiment. In step S101, the arithmetic device 80 calculates a first correction component Va for each vertical signal line from a first region of one frame of data stored in memory. Here, the first region may be a data region corresponding to the OB region 11b or the like. The arithmetic device 80 classifies the data R during cyclic readout for each vertical signal line VL, and calculates the average value of each of the classified data R. The method of calculating the average value is the same as Equation 1 and Equation 2 in the first embodiment. The average value calculated here is used as the initial value of the first correction component Va.

[0075] In step S102, the calculation device 80 calculates the average value S for each row in the second region, which is a correction value calculation region for one frame of data. Note that, since the processes of step S101 and step S102 are independent of each other, the process of step S102 may be executed before the process of step S101.

[0076] In step S103, the calculation device 80 calculates a difference by subtracting the first correction component Va of the vertical signal line VL from the average value S calculated in step S102.

[0077] In step S104, the calculation device 80 performs curve approximation in the vertical direction for the difference values ​​calculated in step S103. FIG. 10 is a diagram showing an example of curve approximation according to this embodiment. In FIG. 10, the horizontal axis represents the difference, and the vertical axis represents the vertical position (Y coordinate of the image). Each point corresponds to the difference for each row, and the solid line represents a curve obtained by functionally approximating the difference. In this way, a correction value corresponding to the shading component in the vertical direction can be obtained by curve approximation. The curve may be approximated by a polynomial for the Y coordinate, for example, or by any function such as a spline curve. The curve obtained in this way becomes the second correction component Vb, which has a value that varies depending on the vertical position N.

[0078] In step S105, the calculation device 80 calculates a difference by subtracting the second correction component Vb in step S104 from the average value S of each row calculated in step S102.

[0079] In step S106, the calculation device 80 performs curve approximation on the difference in step S105 in the same manner as in step S104. A curve is obtained for each vertical signal line VL. The calculation device 80 updates the obtained curve as the first correction component Va.

[0080] In step S107, the calculation device 80 determines whether the update has converged. That is, in the update processing in steps S104 and S106, the calculation device 80 calculates the difference between the first correction component Va and the second correction component Vb before the update and the first correction component Va and the second correction component Vb after the update, and determines whether the difference is equal to or less than a predetermined value. If the update has not converged, that is, if the difference exceeds the predetermined value (NO in step S107), the calculation device 80 repeats the processing of steps S103 to S107. If the update has converged, that is, if the difference is equal to or less than the predetermined value (YES in step S107), the calculation device 80 executes the processing of step S108. The calculation device 80 repeats the processing of steps S103 to S107 and continues to update the first correction component Va and the second correction component Vb until the difference converges. This reduces the error in the first correction component Va and the second correction component Vb.

[0081] In step S108, the calculation device 80 calculates an OB value by adding the second correction component Vb in step S104 and the first correction component Va in step S106. The calculation device 80 subtracts the OB value from the pixel data in the opening region 11a, thereby enabling highly accurate OB correction.

[0082] In this way, the correction calculation is performed in the calculation device 80. In this embodiment, since data is repeatedly referenced, a relatively large memory is required, but highly accurate correction is possible.

[0083] In this embodiment, a vertical curve is obtained to accommodate vertical shading, but to accommodate horizontal shading as well, a curved surface approximation may be performed in the three-dimensional space of the vertical direction, horizontal direction, and pixel value.

[0084] [Third embodiment] An imaging system according to this embodiment will be described. FIG. 11 is a block diagram showing a schematic configuration of the imaging system according to this embodiment. The imaging system 200 shown in FIG. 11 includes an imaging device 201, a lens 202 that forms an optical image of a subject on the imaging device 201, an aperture 204 that adjusts the amount of light passing through the lens 202, and a barrier 206 that protects the lens 202. The lens 202 and the aperture 204 are an optical system that focuses light on the imaging device 201. The imaging device 201 includes a first or No. 2 Embodiment in In the photoelectric conversion device 10 described above, the optical image formed by the lens 202 is converted into image data.

[0085] The imaging system 200 also includes a signal processing unit 208 that processes an output signal output from the imaging device 201. The signal processing unit 208 generates image data from a digital signal output by the imaging device 201. The signal processing unit 208 also performs various corrections and compressions as necessary and outputs the image data. The imaging device 201 may include an AD conversion unit that generates a digital signal to be processed by the signal processing unit 208. The AD conversion unit may be formed in a semiconductor layer (semiconductor substrate) on which a photoelectric conversion unit of the imaging device 201 is formed, or may be formed on a semiconductor substrate different from the semiconductor layer on which the photoelectric conversion unit of the imaging device 201 is formed. The signal processing unit 208 may also be formed on the same semiconductor substrate as the imaging device 201.

[0086] The imaging system 200 further includes a memory unit 210 for temporarily storing image data, and an external interface unit (external I / F unit) 212 for communicating with an external computer or the like. The imaging system 200 also includes a recording medium 214 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 216 for recording or reading out data from the recording medium 214. The recording medium 214 may be built into the imaging system 200 or may be removable.

[0087] The imaging system 200 further includes an overall control / calculation unit 218 that performs various calculations and controls the entire digital still camera, and a timing generation unit 220 that outputs various timing signals to the imaging device 201 and the signal processing unit 208. Here, timing signals and the like may be input from outside, and the imaging system 200 only needs to include at least the imaging device 201 and the signal processing unit 208 that processes the output signal output from the imaging device 201.

[0088] The imaging device 201 outputs an imaging signal to the signal processing unit 208. The signal processing unit 208 performs predetermined signal processing on the imaging signal output from the imaging device 201 and outputs image data. The signal processing unit 208 generates an image using the imaging signal.

[0089] As described above, according to this embodiment, it is possible to realize an imaging system to which the photoelectric conversion device 10 according to the first to fifth embodiments is applied.

[0090] [Fourth embodiment] An imaging system and a moving body according to this embodiment will be described below. Fig. 12 is a diagram showing the configuration of an imaging system and a moving body according to this embodiment.

[0091] FIG. 12(a) shows an example of an imaging system related to an in-vehicle camera. The imaging system 300 includes an imaging device 310. The imaging device 310 is the photoelectric conversion device 10 described in any one of the first to sixth embodiments. The imaging system 300 includes an image processing unit 312 that performs image processing on multiple pieces of image data acquired by the imaging device 310, and a parallax acquisition unit 314 that calculates parallax (phase difference between parallax images) from the multiple pieces of image data acquired by the imaging system 300. The imaging system 300 also includes a distance acquisition unit 316 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 318 that determines whether or not there is a possibility of a collision based on the calculated distance. Here, the parallax acquisition unit 314 and the distance acquisition unit 316 are examples of distance information acquisition means that acquire information about the distance to the object. That is, the distance information is information about the parallax, the defocus amount, the distance to the object, etc. The collision determination unit 318 may determine the possibility of a collision using any of this distance information. The distance information acquisition means may be realized by dedicated hardware, a software module, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a combination thereof.

[0092] The imaging system 300 is connected to a vehicle information acquisition device 320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The imaging system 300 is also connected to a control ECU 330, which is a control means that outputs a control signal to generate a braking force on the vehicle based on the determination result of a collision determination unit 318. The imaging system 300 is also connected to an alarm device 340 that issues an alarm to the driver based on the determination result of the collision determination unit 318. For example, if the determination result of the collision determination unit 318 indicates a high possibility of a collision, the control ECU 330 performs vehicle control to avoid the collision and mitigate damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 340 warns the user by sounding an alarm, displaying alarm information on a screen of a car navigation system, etc., or vibrating the seat belt or steering wheel.

[0093] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the imaging system 300. Fig. 12(b) shows the imaging system when imaging the area in front of the vehicle (imaging range 350). The vehicle information acquisition device 320 sends instructions to the imaging system 300 or the imaging device 310. This configuration can further improve the accuracy of distance measurement.

[0094] Although the above describes an example of control to prevent collision with other vehicles, the system can also be applied to control of automatic driving by following other vehicles, control of automatic driving to prevent deviation from a lane, etc. Furthermore, the imaging system is not limited to vehicles such as the subject vehicle, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the imaging system can be applied not only to moving bodies but also to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).

[0095] [Fifth embodiment] The device according to this embodiment will now be described with reference to Fig. 13, which is a block diagram showing a schematic configuration of the device according to this embodiment.

[0096] FIG. 13 is a schematic diagram showing equipment EQP including a photoelectric conversion device APR. The photoelectric conversion device APR has the functions of the photoelectric conversion device 10 of the first embodiment. All or part of the photoelectric conversion device APR is a semiconductor device IC. The photoelectric conversion device APR of this example can be used, for example, as an image sensor, an AF (Auto Focus) sensor, a photometry sensor, or a distance measurement sensor. The semiconductor device IC has a pixel area PX in which pixel circuits PXC including photoelectric conversion units are arranged in a matrix. The semiconductor device IC can have a peripheral area PR around the pixel area PX. Circuits other than pixel circuits can be arranged in the peripheral area PR.

[0097] The photoelectric conversion device APR may have a structure (chip stacking structure) in which a first semiconductor chip provided with a plurality of photoelectric conversion units and a second semiconductor chip provided with peripheral circuits are stacked. The peripheral circuits in the second semiconductor chip may be column circuits corresponding to the pixel columns of the first semiconductor chip. The peripheral circuits in the second semiconductor chip may also be matrix circuits corresponding to the pixels or pixel blocks of the first semiconductor chip. The first and second semiconductor chips may be connected by through-silicon vias (TSVs), inter-chip wiring formed by direct bonding of a conductor such as copper, connection by microbumps between chips, connection by wire bonding, or the like.

[0098] The photoelectric conversion device APR may include, in addition to the semiconductor device IC, a package PKG that houses the semiconductor device IC. The package PKG may include a base to which the semiconductor device IC is fixed, a cover such as glass that faces the semiconductor device IC, and connecting members such as bonding wires or bumps that connect terminals provided on the base to terminals provided on the semiconductor device IC.

[0099] The equipment EQP may further include at least one of an optical device OPT, a control device CTRL, a processing device PRCS, a display device DSPL, a memory device MMRY, and a mechanical device MCHN. The optical device OPT corresponds to the photoelectric conversion device APR as a photoelectric conversion device, and is, for example, a lens, a shutter, or a mirror. The control device CTRL controls the photoelectric conversion device APR and is, for example, a semiconductor device such as an ASIC. The processing device PRCS processes signals output from the photoelectric conversion device APR and constitutes an AFE (analog front end) or a DFE (digital front end). The processing device PRCS is a semiconductor device such as a CPU (central processing unit) or an ASIC (application-specific integrated circuit). The display device DSPL is an EL display device or a liquid crystal display device that displays information (images) obtained by the photoelectric conversion device APR. The memory device MMRY is a magnetic device or a semiconductor device that stores information (images) obtained by the photoelectric conversion device APR. The memory device MMRY is a volatile memory such as an SRAM or a DRAM, or a non-volatile memory such as a flash memory or a hard disk drive. The mechanical device MCHN has a moving part or a propulsion part such as a motor or an engine. The device EQP displays the signal output from the photoelectric conversion device APR on a display device DSPL and transmits the signal to the outside using a communication device (not shown) provided in the device EQP. For this purpose, the device EQP preferably further includes a memory device MMRY and a processing device PRCS in addition to the memory circuit unit and arithmetic circuit unit provided in the photoelectric conversion device APR.

[0100] The device EQP shown in FIG. 13 can be an electronic device such as an information terminal with a photographing function (e.g., a smartphone or a wearable device) or a camera (e.g., an interchangeable lens camera, a compact camera, a video camera, or a surveillance camera). The mechanical device MCHN in the camera can drive components of the optical device OPT for zooming, focusing, and shutter operation. The device EQP can also be transportation equipment (mobile object) such as a vehicle, a ship, or an aircraft. The device EQP can also be medical equipment such as an endoscope or a CT scanner. The device EQP can also be medical equipment such as an endoscope or a CT scanner.

[0101] The mechanical device MCHN in the transportation equipment can be used as a moving device. The device EQP as a transportation equipment is suitable for transporting the photoelectric conversion device APR and for assisting and / or automating driving (piloting) using a photographing function. The processing device PRCS for assisting and / or automating driving (piloting) can perform processing to operate the mechanical device MCHN as a moving device based on information obtained by the photoelectric conversion device APR.

[0102] The photoelectric conversion device APR according to this embodiment can provide high value to its designer, manufacturer, seller, purchaser, and / or user. Therefore, if the photoelectric conversion device APR is installed in a device EQP, the value of the device EQP can also be increased. Therefore, when manufacturing and selling the device EQP, deciding to install the photoelectric conversion device APR according to this embodiment in the device EQP is advantageous in increasing the value of the device EQP.

[0103] [Other embodiments] The present invention is not limited to the above-described embodiments and can be modified in various ways. For example, an example in which part of the configuration of one embodiment is added to another embodiment, or an example in which part of the configuration of another embodiment is replaced with another embodiment, is also an embodiment of the present invention.

[0104] It should be noted that the above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0105] 10 Photoelectric conversion device 11 Pixel section 11a Opening area 11b~11d OB area 110 pixels 12 Drive unit 14 Readout section 15 Signal processing section 151, 152 CDS Department 153 memory 150 OB clamp part 1503 First Processing Section 1504 Second Processing Section

Claims

1. a pixel section including a plurality of pixels arranged in a matrix; a driver that drives the pixels in a first region by a first method and drives the pixels in a second region by a second method; a calculation unit that calculates a correction value based on pixel values ​​read from the first region and the second region; a correction unit that corrects an offset of a pixel value according to incident light based on the correction value, The calculation unit calculating initial values ​​of a plurality of first correction components based on the first pixel values ​​read from the first region; updating each of the plurality of first correction components based on a second pixel value read from the second region and a predetermined second correction component; a correction value calculating unit that calculates the correction value using the updated first correction component and the updated second correction component;

2. 2. The photoelectric conversion device according to claim 1, wherein the second correction component is calculated from the second pixel value and one of the plurality of first correction components.

3. 3. The photoelectric conversion device according to claim 1, wherein the second correction component is calculated from a difference between the second pixel value and one of the plurality of first correction components.

4. 4. The photoelectric conversion device according to claim 1, wherein the plurality of first correction components are representative values ​​of the plurality of first pixel values.

5. 5. The photoelectric conversion device according to claim 1, wherein the plurality of first correction components are average values ​​of the plurality of first pixel values.

6. a plurality of signal lines provided corresponding to columns of the pixel unit; the first correction component is calculated for each of the signal lines; 6. The photoelectric conversion device according to claim 1, wherein the second correction component is common to the plurality of signal lines and has a value corresponding to a row of the pixel unit.

7. the pixel includes a plurality of color pixels; 7. The photoelectric conversion device according to claim 6, wherein the first correction component is further calculated for each color of the color pixels.

8. the pixel includes a first photoelectric conversion unit and a second photoelectric conversion unit, 8. The photoelectric conversion device according to claim 6, wherein the first correction component is further calculated for each method of reading out charges from the first photoelectric conversion unit and the second photoelectric conversion unit.

9. 2. The photoelectric conversion device according to claim 1, wherein the first method and the second method differ in accumulation time.

10. 10. The photoelectric conversion device according to claim 1, wherein the first method is a cyclic readout method in which a predetermined area is repeatedly read out.

11. 11. The photoelectric conversion device according to claim 10, wherein the number of times of cyclic readout is changed depending on at least one of ISO sensitivity, temperature, and a driving method used by the driving unit.

12. When the signal line identifier is represented by "i", the second pixel value is represented by "S", the first correction component is represented by "Va", and the second correction component is represented by "Vb", The first correction component (Va ｉ，Ｎ ) is the difference (S ｉ，Ｎ -Vb (Ｎ－１) 7. The photoelectric conversion device according to claim 6, wherein the calculation is performed for each horizontal scanning period based on the following:

13. When the coefficient A1 of the IIR (Infinite Impulse Response) processing is 0<A1<1, the first correction component (Va ｉ，Ｎ )teeth, Wow ｉ，Ｎ ={ｶa ｉ，(Ｎ－１) ×A1+(S ｉ，Ｎ -Vb (Ｎ－１) )×(1-A1)} 13. The photoelectric conversion device according to claim 12, wherein the calculation is performed according to the following formula:

14. The second correction component (Vb Ｎ ) is the difference (S ｉ，Ｎ -Va ｉ，Ｎ 14. The photoelectric conversion device according to claim 12, wherein the calculation is based on the following:

15. When the coefficient A2 of the IIR (Infinite Impulse Response) processing is 0<A2<1, the second correction component (Vb Ｎ )teeth, 6b Ｎ ={Vb N-(1 / 2)×A2+(S ｉ，Ｎ -6 ｉ，Ｎ )×(1-A2)} 15. The photoelectric conversion device according to claim 12, wherein the photoelectric conversion efficiency is calculated according to the following formula:

16. When the number of horizontal scanning periods in the first region is "N1", the first pixel value is "R", and the kth horizontal scanning period among the first to N1th horizontal scanning periods is "k", the first correction component (Va ｉ ) the initial value of [Equation 1] 16. The photoelectric conversion device according to claim 12, wherein the photoelectric conversion efficiency is calculated according to the following formula:

17. The photoelectric conversion device according to any one of claims 1 to 16, a signal processing unit that processes a signal output from the photoelectric conversion device; An imaging system comprising:

18. A mobile object, The photoelectric conversion device according to any one of claims 1 to 16, a distance information acquisition means for acquiring distance information to an object from a parallax image based on a signal from the photoelectric conversion device; a control means for controlling the moving object based on the distance information; A moving object characterized by having:

19. The photoelectric conversion device according to any one of claims 1 to 16, an optical device corresponding to the photoelectric conversion device; a control device that controls the photoelectric conversion device; a processing device that processes a signal output from the photoelectric conversion device; a mechanical device controlled based on the information obtained by the photoelectric conversion device; a display device that displays information obtained by the photoelectric conversion device; and a storage device that stores information obtained by the photoelectric conversion device; An apparatus characterized by comprising:

Images