Solid-state image sensor, imaging device, and imaging method

Abstract

To reduce noise while suppressing power consumption. [Solution] The solid-state image sensor comprises a plurality of pixels capable of accumulating charge corresponding to incident light, a drive control unit capable of controlling the driving of the plurality of pixels by classifying them into drive pixels and stop pixels, a readout selection unit that selects the readout of a signal corresponding to the charge accumulated in the drive pixels and a signal corresponding to the charge accumulated in the stop pixels, and a correction processing unit that corrects the signal value of the drive pixels using the signal value of the stop pixels.

Description

Technical Field

[0001] The present invention relates to a solid-state imaging device, an imaging apparatus, and an imaging method.

Background Art

[0002] Solid-state imaging devices are used in various fields. As one of the fields where solid-state imaging devices are used, a distance image imaging device capable of measuring the distance to a subject is known. For example, a time-of-flight (ToF) distance image imaging device (ranging device) that measures the distance to a subject based on the flight time of light in space (measurement space) by utilizing the fact that the speed of light is known has been realized (see, for example, Patent Document 1). Such a distance image imaging device generally includes a light source that generates light to irradiate a subject and a solid-state imaging device that receives the light reflected by the subject and accumulates charges.

[0003] It is known that dark current occurs in a solid-state imaging device even in a state where the device does not receive light. The dark current increases in proportion to temperature and time, and fixed pattern noise is generated in the signal (image data) output from the solid-state imaging device due to the dark current. Particularly when the signal amount is small, the ratio of noise in the signal amount becomes large, so the image quality is likely to deteriorate. In the case of a distance image imaging device using a solid-state imaging device, since the distance is measured from the signal amount, the ranging accuracy decreases. Therefore, a technique for correcting noise in a solid-state imaging device has been disclosed (see, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technology described in Patent Document 2, during the charge accumulation period before reading out the charge from multiple pixels arranged in a matrix, the pixel accumulating charge is separated from the subsequent column circuit. By reading out the column noise generated in the column circuit and correcting it, correction is only possible by summing the column noise for each column. Furthermore, since additional driving is required to read out the noise during the accumulation period, there is a concern that power consumption will increase.

[0006] The present invention has been made in view of the above-mentioned problems, and one of its objectives is to provide a solid-state image sensor, an imaging device, and an imaging method that can reduce noise while suppressing power consumption. [Means for solving the problem]

[0007] A solid-state image sensor according to one aspect of the present invention comprises: a plurality of pixels capable of accumulating charge corresponding to incident light; a drive control unit capable of controlling the driving of the plurality of pixels by classifying them into drive pixels and stop pixels; a readout selection unit that selects to read out a signal corresponding to the charge accumulated in the drive pixels and a signal corresponding to the charge accumulated in the stop pixels; and a correction processing unit that corrects the signal value of the drive pixels using the signal value of the stop pixels.

[0008] Furthermore, an imaging device according to one aspect of the present invention comprises a solid-state image sensor having a plurality of pixels capable of accumulating charge corresponding to incident light, a drive control unit capable of classifying the plurality of pixels into drive pixels and stop pixels and controlling their drive, a readout selection unit that selects to read out a signal corresponding to the charge accumulated in the drive pixels and a signal corresponding to the charge accumulated in the stop pixels, and a correction processing unit that corrects the signal value of the drive pixels using the signal value of the stop pixels, and an image processing unit that performs processing based on the signal value of the drive pixels corrected by the solid-state image sensor.

[0009] Furthermore, an imaging device according to one aspect of the present invention comprises a solid-state image sensor having a plurality of pixels capable of accumulating charge corresponding to incident light, a drive control unit capable of controlling the driving of the plurality of pixels by classifying them into drive pixels and stop pixels, a readout selection unit that selects to read out a signal corresponding to the charge accumulated in the drive pixels and a signal corresponding to the charge accumulated in the stop pixels, and a correction processing unit that corrects the signal value of the drive pixels using the signal value of the stop pixels.

[0010] Furthermore, an imaging method performed by a solid-state image sensor having a plurality of pixels capable of accumulating charge corresponding to incident light, according to one aspect of the present invention, includes the steps of: a drive control unit classifying the plurality of pixels into drive pixels and stop pixels and driving them; a readout selection unit selecting the readout of a signal corresponding to the charge accumulated in the drive pixels and a signal corresponding to the charge accumulated in the stop pixels; and a correction processing unit correcting the signal value of the drive pixels using the signal value of the stop pixels. [Effects of the Invention]

[0011] According to the present invention, it is possible to reduce noise while suppressing power consumption. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing a schematic configuration of a distance image acquisition device according to the first embodiment. [Figure 2] This is a block diagram showing a schematic configuration of a distance image sensor according to the first embodiment. [Figure 3] This is a circuit diagram showing an example of the pixel configuration of a distance image sensor according to the first embodiment. [Figure 4] This figure shows an example of controlling the driving pixel and the stopping pixel according to the first embodiment. [Figure 5] This figure shows an example of a correction process performed by the correction processing unit according to the first embodiment. [Figure 6] This flowchart shows an example of the correction process according to the first embodiment. [Figure 7]It is a diagram showing an example of correction processing executed by a correction processing unit according to a second embodiment. [Figure 8] It is a diagram showing an example of correction processing executed by a correction processing unit according to a third embodiment. [Figure 9] It is a diagram showing an example of correction processing executed by a correction processing unit according to a fourth embodiment.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, a distance image capturing device according to an embodiment of the present invention will be described with reference to the drawings.

[0014] <First Embodiment> First, the first embodiment will be described. FIG. 1 is a block diagram showing a schematic configuration of a distance image capturing device according to the present embodiment. The distance image capturing device 1 is a distance image capturing device that measures the distance to an object (distance measurement) using the TOF method, and includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. FIG. 1 also shows a subject OB, which is an object for which the distance is measured in the distance image capturing device 1.

[0015] The light source unit 2 irradiates the space of the measurement target with an optical pulse PO in accordance with control from the distance image processing unit 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0016] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band with a wavelength of 850 nm to 940 nm) that becomes the optical pulse PO for irradiating the space of the measurement target. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits pulsed laser light in response to control from the distance image processing unit 4.

[0017] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 over the area of the surface that irradiates the space of the measurement target. The pulsed laser light diffused by the diffusion plate 22 is emitted as the optical pulse PO and irradiates the space of the measurement target.

[0018] When the subject OB exists in the space of the measurement target where the distance is measured in the distance image capturing device 1, the light receiving unit 3 receives the reflected light RL of the optical pulse PO irradiated from the light source unit 2 and reflected by the subject OB, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32.

[0019] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 side and causes it to be received (incident) by the pixels provided in the light receiving area of the distance image sensor 32.

[0020] The distance image sensor 32 is an imaging element (solid-state imaging element) used in the distance image capturing device 1. The distance image sensor 32 includes a plurality of pixels in a two-dimensional light receiving area. In each pixel of the distance image sensor 32, one photoelectric conversion element, a plurality of charge storage units corresponding to this one photoelectric conversion element, and a component that distributes charges to each charge storage unit are provided. That is, the pixel is an imaging element having a distribution configuration that distributes and accumulates charges in a plurality of charge storage units.

[0021] The distance image sensor 32 distributes the charges generated by the photoelectric conversion element to each charge storage unit according to the control from the timing control unit 41. Further, the distance image sensor 32 outputs a voltage signal corresponding to the amount of charge distributed to the charge storage unit as a pixel signal. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and pixel signals corresponding to one frame of each pixel are output.

[0022] Here, the range in the depth direction (range of distance) in the space of the object to be measured by the distance image acquisition device 1 is determined mainly by the light intensity of the light pulse PO emitted from the light source unit 2 and the light receiving sensitivity of the light receiving unit 3. The range in the plane direction in which distance can be measured is determined by the irradiation angle (spread of light) of the light pulse PO emitted from the light source unit 2 and the light receiving angle (angle at which light can be received) of the light receiving unit 3.

[0023] In this embodiment, there is a method of measuring the distance of the space to be measured (measurable range) in one frame, and a method of dividing the space to be measured (measurable range) into multiple distance measuring ranges in the depth direction and measuring the distance for each of the multiple distance measuring ranges in multiple subframes (frames obtained by dividing one frame into multiple parts), and it is possible to switch between the two methods.

[0024] The distance image processing unit 4 controls the distance image acquisition device 1 and calculates the distance to the subject OB. The distance image processing unit 4 comprises a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0025] The timing control unit 41 controls the timing of outputting various control signals required for measurement, in accordance with the control of the measurement control unit 43. These various control signals include, for example, a signal to control the irradiation of the light pulse PO, a signal to distribute and store the reflected light RL in multiple charge storage units, and a signal to control the number of storage cycles per frame. The number of storage cycles is the number of times the process of distributing and storing charge in the charge storage unit CS (see Figure 3) is repeated. The product of this number of storage cycles and the time (storage time) for storing charge in each charge storage unit per charge distribution cycle is the storage time.

[0026] The distance calculation unit 42 outputs distance information calculated based on the pixel signals output from the distance image sensor 32, determining the distance to the subject OB. The distance calculation unit 42 calculates the delay time from the irradiation of the light pulse PO to the reception of the reflected light RL based on the amount of charge accumulated in the multiple charge storage units. The distance calculation unit 42 calculates the distance to the subject OB according to the calculated delay time.

[0027] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of times to accumulate and the accumulation time for one frame, and controls the timing control unit 41 so that imaging is performed according to the set settings.

[0028] In this configuration, the distance image acquisition device 1 receives reflected light RL from the light pulse PO in the near-infrared wavelength band that the light source unit 2 irradiates onto the subject OB, and the light receiving unit 3 receives the reflected light RL from the subject OB. The distance image processing unit 4 then outputs distance information that measures the distance to the subject OB.

[0029] In Figure 1, the distance image processing unit 4 is shown as being located inside the distance image imaging device 1. However, the distance image processing unit 4 may be a component located outside the distance image imaging device 1.

[0030] Here, the configuration of the distance image sensor 32 used as an image sensor in the distance image acquisition device 1 will be explained using Figure 2. Figure 2 is a block diagram showing the schematic configuration of the image sensor (distance image sensor 32) used in the distance image acquisition device 1 according to this embodiment.

[0031] As shown in Figure 2, the distance image sensor 32 includes, for example, a light-receiving area 320 on which a plurality of pixels 321 are arranged, a drive control unit 322 that can control the driving of a specific pixel, a readout selection unit 323 that performs control independently of the drive control unit 322, a readout unit 324 that reads signal values ​​from the pixels 321, and a correction processing unit 325 that corrects the signal values ​​of the driven pixels using the signal values ​​of the stopped pixels.

[0032] The light-receiving region 320 is a region in which multiple pixels 321 are arranged, and Figure 2 shows an example in which they are arranged in a two-dimensional matrix of 8 rows and 8 columns. Each pixel 321 accumulates a charge corresponding to the amount of light it receives.

[0033] Here, the configuration of the pixels 321 arranged within the light-receiving area 320 of the distance image sensor 32 will be described using Figure 3. Figure 3 is a circuit diagram showing an example of the configuration of pixels 321 arranged within the light-receiving area 320 of the distance image sensor 32 according to this embodiment. Figure 3 shows an example of the configuration of one pixel 321 among a plurality of pixels 321 arranged within the light-receiving area 320. The pixel 321 is an example of a configuration that includes four pixel signal output units RU.

[0034] Pixel 321 comprises one photoelectric conversion element PD, a drain gate transistor GD, and four pixel signal output units RU that output a voltage signal from the corresponding output terminal O. Each pixel signal output unit RU comprises a read gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal output unit RU, a charge storage unit CS is formed by the floating diffusion FD and the charge storage capacitor C.

[0035] In Figure 3, the four pixel signal output units RU are distinguished by adding the numbers "1", "2", "3", or "4" after the code "RU" of each pixel signal output unit RU. Similarly, each component of the four pixel signal output units RU is represented by indicating the number representing the respective pixel signal output unit RU after its code, thus distinguishing the pixel signal output unit RU that each component corresponds to.

[0036] In the pixel 321 shown in Figure 3, the pixel signal output unit RU1, which outputs a voltage signal from the output terminal O1, comprises a read gate transistor G1, a floating diffusion transistor FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a selection gate transistor SL1. In the pixel signal output unit RU1, the floating diffusion transistor FD1 and the charge storage capacitor C1 constitute a charge storage unit CS1. Pixel signal output units RU2 to RU4 have a similar configuration.

[0037] The photoelectric conversion element PD is an embedded photodiode that converts incident light into electricity to generate an electric charge and stores the generated charge. The structure of the photoelectric conversion element PD can be arbitrary. For example, the photoelectric conversion element PD may be a PN photodiode with a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode with a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Furthermore, the photoelectric conversion element PD is not limited to a photodiode, but may be, for example, a photogate type photoelectric conversion element.

[0038] In pixel 321, the photoelectric conversion element PD converts the incident light into electricity and generates charge, which is then distributed to each of the four charge storage units CS. The system then outputs a voltage signal corresponding to the amount of charge distributed to each unit.

[0039] The configuration of the pixels 321 arranged in the distance image sensor 32 is not limited to the configuration with four pixel signal output units RU as shown in Figure 3, but any pixel with multiple pixel signal output units RU is acceptable. In other words, the number of pixel signal output units RU (charge storage units CS) in the pixels 321 arranged in the distance image sensor 32 may be two, three, or five or more.

[0040] Furthermore, in the pixel 321 configuration shown in Figure 3, an example is shown in which the charge storage unit CS is composed of a floating diffusion FD and a charge storage capacitance C. However, the charge storage unit CS only needs to be composed of a floating diffusion FD, and the pixel 321 may be configured without a charge storage capacitance C.

[0041] Furthermore, while Figure 3 shows an example of a configuration in which pixel 321 includes a drain gate transistor GD, a configuration without a drain gate transistor GD is also acceptable if there is no need to discard the charge accumulated (remaining) in the photoelectric conversion element PD.

[0042] Returning to Figure 2, the drive control unit 322 controls the driving of the multiple pixels 321 arranged in the light-receiving area 320. For example, the drive control unit 322 distributes the charge converted by the photoelectric conversion element PD to each of the charge storage units CS of the pixels 321 and stores it (see Figure 3).

[0043] Furthermore, when controlling the driving of multiple pixels 321, the drive control unit 322 can also stop the driving of any pixel 321. Among the multiple pixels 321, the pixels that are driven by the control of the drive control unit 322 are called "driven pixels," and the pixels whose driving is stopped are called "stopped pixels." In other words, the drive control unit 322 can classify the multiple pixels 321 into driven pixels and stopped pixels and control their driving.

[0044] The drive control unit 322 controls the charge accumulation time in a stationary pixel to 0 (complete stop) as an example, but it is not limited to 0. The charge accumulation time in a stationary pixel may be controlled to be shorter than the charge accumulation time in a driving pixel. In other words, a stationary pixel is a pixel with a shorter charge accumulation time compared to a driving pixel.

[0045] Figure 4 shows an example of control between driven pixels and stopped pixels according to this embodiment. In the example shown in Figure 4(A), the drive control unit 322 controls the driving by classifying a plurality of pixels 321 into driven pixels and stopped pixels on a column basis. Here, an example is shown in which pixels 321 are alternately classified into driven pixels and stopped pixels on a column basis.

[0046] In the example shown in Figure 4(B), the drive control unit 322 controls the driving by classifying multiple pixels 321 into drive pixels and stop pixels on a row-by-row basis. Here, an example is shown in which pixels 321 are alternately classified into drive pixels and stop pixels on a row-by-row basis.

[0047] In this way, power consumption can be suppressed by providing a stop pixel in the control of driving the pixels 321 by the drive control unit 322. For example, the drive control unit 322 can reduce the power consumption during driving to about half by controlling each column or row alternately to drive pixels and stop pixels.

[0048] The drive control unit 322 may also be controlled to provide stop pixels every multiple columns or every multiple rows. For example, if one column is controlled as a stop pixel every N columns, power consumption can be reduced to about 1 / N of that when all pixels are controlled as drive pixels (where N is an integer of 2 or more). Similarly, if one row is controlled as a stop pixel every M rows, power consumption can be reduced to about 1 / M of that when all pixels are controlled as drive pixels (where M is an integer of 2 or more).

[0049] Furthermore, this power consumption reduction effect suppresses temperature rise, which in turn makes it possible to suppress dark current.

[0050] Returning to Figure 2, the readout selection unit 323 controls the selection of pixels from which to read pixel signals from among the multiple pixels 321 arranged in the light-receiving area 320. A pixel signal is a voltage signal corresponding to the charge (amount of charge) accumulated in the pixel 321. At this time, the readout selection unit 323 controls independently of the drive control unit 322, and controls the unit to read not only from the drive pixels but also from the stationary pixels. In other words, the readout selection unit 323 selects whether to read pixel signals from the drive pixels or from the stationary pixels.

[0051] The readout selection unit 323 selects either a drive pixel or a stop pixel from a plurality of pixels 321 as the pixel to be read out. By selecting to read from the stop pixel, the readout selection unit 323 can acquire noise data, including dark current, in real time.

[0052] The readout unit 324 reads pixel signals from the pixels 321 selected by the readout selection unit 323 from among a plurality of pixels 321. For example, the readout unit 324 reads pixel signals from the driven pixels or stopped pixels selected by the readout selection unit 323. The readout unit 324 then performs A / D conversion processing on the readout pixel signals and sequentially outputs them as digital data (hereinafter referred to as "readout data") to the correction processing unit 325.

[0053] In other words, the pixel signal output from pixel 321 is an analog voltage signal corresponding to the amount of charge stored in pixel 321, and the pixel signal output from readout unit 324 is a pixel signal (readout data) converted digitally based on the pixel signal output from pixel 321. Furthermore, the pixel signal (readout data) output from readout unit 324 is corrected by the correction processing unit 325, which will be described next, and then output from distance image sensor 32.

[0054] The correction processing unit 325 performs correction processing based on the pixel signals output from the readout unit 324. Specifically, the correction processing unit 325 corrects the signal value of the driven pixel's pixel signal using the signal value of the stopped pixel's pixel signal. For example, the correction processing unit 325 outputs a pixel signal from which noise such as dark current has been removed by subtracting the signal value of the stopped pixel's pixel signal from the signal value of the driven pixel's pixel signal.

[0055] Figure 5 shows an example of the correction process performed by the correction processing unit 325 according to this embodiment. Figure 5(A) shows an example of the correction process when the drive control unit 322 drives the pixels 321 by alternating between drive pixels and stop pixels in each column. When the readout unit 324 reads out the readout data 3240 row by row, the readout data 3240 is data in which the signal values ​​of drive pixels and the signal values ​​of stop pixels are arranged alternately. The correction processing unit 325 obtains noise-free correction data 3250 (pixel signal) for each pixel by subtracting the signal values ​​of adjacent pixels.

[0056] Figure 5(B) shows an example of the correction process when the drive control unit 322 drives pixels 321 by alternately classifying them into drive pixels and stop pixels row by row. When the readout unit 324 reads out readout data 3240 row by row, the readout data 3240 is data in which the signal values ​​of rows of drive pixels and the signal values ​​of rows of stop pixels are arranged alternately. The correction processing unit 325 obtains noise-free corrected data 3250 (pixel signals) row by subtracting the signal values ​​of each pixel in adjacent rows.

[0057] In the correction process shown in Figure 5, the correction can be performed within the read data 3240 of one frame, making it possible to perform the correction without the need to save intermediate data to frame memory.

[0058] Next, we will explain the operation of the correction process performed by the distance image sensor 32. Figure 6 is a flowchart showing an example of the correction process according to this embodiment.

[0059] (Step S101) The drive control unit 322 classifies the multiple pixels 321 into drive pixels and stop pixels and controls their drive. Then, the process proceeds to step S103.

[0060] (Step S103) The readout unit 324 reads out the pixel signal (voltage signal corresponding to the amount of charge) of the drive pixel and the pixel signal (voltage signal corresponding to the amount of charge) of the stop pixel, respectively, selected by the readout selection unit 323. Then, the process proceeds to step S105.

[0061] (Step S105) The correction processing unit 325 performs a correction calculation to correct the signal value of the pixel signal of the driven pixel using the signal value of the pixel signal of the stopped pixel output from the readout unit 324. For example, the correction processing unit 325 obtains a pixel signal from which noise such as dark current has been removed by subtracting the signal value of the pixel signal of the stopped pixel from the signal value of the pixel signal of the driven pixel. Then, the process proceeds to step S107.

[0062] (Step S107) The correction processing unit 325 outputs the corrected pixel signal obtained in step S105. For example, the correction processing unit 325 outputs the corrected pixel signal to the distance image processing unit 4.

[0063] As described above, the distance image sensor 32 (an example of a solid-state image sensor) according to this embodiment comprises a plurality of pixels 321 capable of accumulating charge corresponding to incident light, a drive control unit 322 that controls the driving of the plurality of pixels 321, a readout selection unit 323, and a correction processing unit 325. The drive control unit 322 can control the driving of the plurality of pixels 321 by classifying them into drive pixels and stationary pixels. The readout selection unit 323 selects the readout of a signal (pixel signal) corresponding to the charge accumulated in the drive pixels and a signal (pixel signal) corresponding to the charge accumulated in the stationary pixels. The correction processing unit 325 corrects the signal value of the drive pixels using the signal value of the stationary pixels.

[0064] As a result, the distance image sensor 32 classifies multiple pixels 321 into driven pixels and stopped pixels and drives them, and corrects the signal value of the driven pixels using the signal value of the stopped pixels, thereby reducing noise while suppressing power consumption.

[0065] For example, the drive control unit 322 controls the charge accumulation time in pixels classified as stationary pixels to be shorter than the charge accumulation time in pixels classified as driving pixels.

[0066] This allows the distance image sensor 32 to acquire noise data such as dark current from the stationary pixels.

[0067] Furthermore, the distance image sensor 32 includes a readout unit 324 that reads out signals corresponding to the charges accumulated in multiple pixels 321. The readout unit 324 reads out signals (pixel signals) corresponding to the charges accumulated in the drive pixels or stop pixels selected by the readout selection unit 323.

[0068] As a result, the distance image sensor 32 controls the driving of pixels by classifying them into driven pixels and stopped pixels, and it reads out pixel signals not only from the driven pixels but also from the stopped pixels, so it can acquire noise data such as dark current in real time.

[0069] Furthermore, multiple pixels 321 are arranged in a matrix. For example, the drive control unit 322 classifies the multiple pixels 321 into drive pixels and stop pixels on a column basis and performs drive control.

[0070] As a result, the distance image sensor 32 can reduce power consumption by classifying pixels into drive pixels and stop pixels on a column-by-column basis and controlling their operation.

[0071] Furthermore, the multiple pixels 321 are arranged in a matrix. For example, the drive control unit 322 classifies the multiple pixels 321 into drive pixels and stop pixels on a row-by-row basis and controls the drive.

[0072] As a result, the distance image sensor 32 can reduce power consumption by classifying pixels into driven and stopped pixels on a row-by-row basis and controlling their operation. Furthermore, when the distance image sensor 32 classifies pixels into driven and stopped pixels on a column-by-column basis and controls their operation, it is not possible to acquire noise data for the same column and can only correct using values ​​from adjacent columns. However, when classifying pixels into driven and stopped pixels on a row-by-row basis and controlling their operation, noise data for the same column can be acquired, making it possible to correct column noise.

[0073] Furthermore, the distance image imaging device 1 (an example of an imaging device) according to this embodiment includes a distance image sensor 32 (an example of a solid-state image sensor) comprising the above-mentioned pixels 321, a drive control unit 322, a readout selection unit 323, and a correction processing unit 325, and a distance image processing unit 4 (an example of an image processing unit) that performs processing based on the signal values ​​of the drive pixels corrected by the distance image sensor 32.

[0074] As a result, the distance image acquisition device 1 classifies and drives multiple pixels 321 into drive pixels and stop pixels, and corrects the signal value of the drive pixels using the signal value of the stop pixels, thereby reducing noise while suppressing power consumption.

[0075] Furthermore, according to this embodiment, the imaging method in a distance image sensor 32 (an example of a solid-state image sensor) equipped with a plurality of pixels 321 capable of accumulating charge corresponding to incident light includes the steps of: a drive control unit 322 classifying the plurality of pixels 321 into drive pixels and stop pixels and driving them; a readout selection unit 323 selecting the readout of a signal (pixel signal) corresponding to the charge accumulated in the drive pixels and a signal (pixel signal) corresponding to the charge accumulated in the stop pixels; and a correction processing unit 325 correcting the signal value of the drive pixels using the signal value of the stop pixels.

[0076] As a result, the imaging method in the distance image sensor 32 classifies multiple pixels 321 into driven pixels and stopped pixels and drives them, and corrects the signal value of the driven pixels using the signal value of the stopped pixels, thereby reducing noise while suppressing power consumption.

[0077] <Second Embodiment> Next, a second embodiment will be described. The correction process according to this embodiment differs in part from the correction process according to the first embodiment. The basic configuration of the distance image acquisition device 1 according to this embodiment is the same as that of the first embodiment, so its description is omitted.

[0078] Figure 7 shows an example of the correction process performed by the correction processing unit 325 according to this embodiment.

[0079] Figure 7(A) shows an example of the correction process when the drive control unit 322 drives pixels 321 by alternating between drive pixels and stop pixels in each column. When the readout unit 324 reads out readout data 3240 row by row, the readout data 3240 is data in which the signal values ​​of drive pixels and the signal values ​​of stop pixels are arranged alternately, similar to Figure 5(A). Here, the drive control by the drive control unit 322 and the readout control by the readout unit 324 are repeated for each frame. The correction processing unit 325 calculates a moving average of the signal values ​​of the stop pixels in the readout data 3240 read by the readout unit 324 and the signal values ​​of the stop pixels in the previous frame, and obtains moving average data 3251 for each pixel. The correction processing unit 325 subtracts the signal values ​​of adjacent pixels from the moving average data 3251 of the signal values ​​of the drive pixels and the signal values ​​of the stop pixels in the readout data 3240, and obtains corrected correction data 3250 (pixel signal) for each pixel.

[0080] Figure 7(B) shows an example of the correction process when the drive control unit 322 drives pixels 321 by alternately classifying them into drive pixels and stop pixels row by row. When the readout unit 324 reads out readout data 3240 row by row, the readout data 3240 is data in which the signal values ​​of rows of drive pixels and the signal values ​​of rows of stop pixels are arranged alternately, similar to Figure 5(B). The readout unit 324 calculates the moving average of the signal values ​​of the rows of stop pixels in the readout data 3240 read out by the readout unit 324 and the signal values ​​of the rows of stop pixels in the previous frame, and obtains the moving average data 3251 for each row. The correction processing unit 325 obtains noise-removed correction data 3250 (pixel signal) for each row by subtracting the signal value of each drive pixel and the signal value of each stop pixel in adjacent rows.

[0081] Thus, in the distance image sensor 32 (an example of a solid-state image sensor) according to this embodiment, the drive control unit 322 and the readout selection unit 323 repeatedly perform their respective controls frame by frame. The correction processing unit 325 then corrects the signal value of the drive pixel using the signal values ​​of multiple frames of the stopped pixel.

[0082] As a result, the distance image sensor 32 calculates a moving average of the stopped pixels with respect to the previous frame, thereby averaging out the random components contained in the noise data and enabling more stable correction.

[0083] The frames used when calculating the moving average may be two frames, including the current frame and the previous frame, or three or more frames, including the current frame and multiple previous frames.

[0084] <Third Embodiment> Next, a third embodiment will be described. The correction process according to this embodiment differs in part from the correction processes according to the first and second embodiments. The basic configuration of the distance image acquisition device 1 according to this embodiment is the same as that of the first embodiment, so its description is omitted.

[0085] Figure 8 shows an example of the correction process performed by the correction processing unit 325 according to this embodiment. In this embodiment, the pixels classified as drive pixels and the pixels classified as stop pixels among the plurality of pixels 321 are changed during the process.

[0086] Figure 8(A) shows an example of correction processing when the drive control unit 322 drives pixels 321 by alternating between drive pixels and stop pixels in each column. In this example, the drive control unit 322 alternately switches the target column for drive pixels and stop pixels every frame. When the readout unit 324 reads out readout data 3240 row by row, the readout data 3240 consists of data in which the signal values ​​of drive pixels and the signal values ​​of stop pixels are arranged alternately, but the signal value of each pixel switches between the signal value of a drive pixel and the signal value of a stop pixel every frame. The previous frame storage unit 3252 stores the readout data 3240 from frames prior to the current frame, and pixels classified as drive pixels in the current frame are classified as stop pixels in the previous frame. The correction processing unit 325 subtracts the signal value of the drive pixel and the signal value of the stop pixel for the same pixel where the drive pixel and stop pixel roles have switched between the current frame and the previous frame to obtain corrected correction data 3250 (pixel signal) for each pixel. After the correction process, the read data 3240 of the current frame is stored in the previous frame storage unit 3252.

[0087] Figure 8(B) shows an example of correction processing when the drive control unit 322 drives pixels 321 by alternating between drive pixels and stop pixels row by row. In this example, the drive control unit 322 alternately switches the target row of drive pixels and stop pixels every frame. When the readout unit 324 reads out the readout data 3240 row by row, the readout data 3240 consists of data in which the signal values ​​of drive pixel rows and stop pixel rows are arranged alternately, but the signal value of each pixel in each row switches between the signal value of a drive pixel and the signal value of a stop pixel every frame. The previous frame storage unit 3252 stores the readout data 3240 from frames prior to the current frame, and pixels in rows classified as drive pixels in the current frame are classified as stop pixels in the previous frame. The correction processing unit 325 obtains corrected correction data 3250 (pixel signal) for each row by subtracting the signal value of the driving pixel from the signal value of the stopping pixel for each pixel in the same row where the driving pixel and stopping pixel switch between the current frame and the previous frame. After the correction processing, the read data 3240 of the current frame is stored in the previous frame storage unit 3252.

[0088] In this embodiment, the distance image sensor 32 (an example of a solid-state image sensor) repeatedly performs its respective control functions for each frame. The drive control unit 322 then changes which pixels among the multiple pixels 321 are classified as drive pixels and which are classified as stop pixels during the repeated control for each frame.

[0089] As a result, the distance image sensor 32 subtracts the signal value when the same pixel is a driven pixel from the signal value when it is a stationary pixel, thus correcting for noise that occurs specifically for each pixel 321.

[0090] <Fourth Embodiment> Next, a fourth embodiment will be described. The correction process according to this embodiment differs in part from the correction processes according to the first to third embodiments. The basic configuration of the distance image acquisition device 1 according to this embodiment is the same as that of the first embodiment, so its description is omitted.

[0091] In the above embodiment, an example was described in which multiple pixels 321 are classified into drive pixels and stop pixels on a column-by-column or row-by-row basis to control the drive. However, the classification of drive pixels and stop pixels is not limited to row-by-row or column-by-column basis. For example, a configuration may be used in which a group of N × M pixels (where N and M are integers of 2 or more) is used as the unit, and only one pixel in each group of pixels is classified as a stop pixel, while the other pixels are classified as drive pixels to control the drive.

[0092] Figure 9 shows an example of the correction processing performed by the correction processing unit 325 according to this embodiment. In the example shown in Figure 9(A), the drive control unit 322 controls the driving of multiple pixels 321, using a 2x2 pixel group (a group of pixels in 2 rows and 2 columns) as the unit, and for each pixel group, it sets only the bottom right pixel as a stop pixel and controls the driving of the other three pixels as drive pixels.

[0093] As shown in Figure 9(B), in the two lines of read data 3240 read by the read unit 324, the data consists of a 2x2 pixel group where only the bottom right pixel has the signal value of a stop pixel and the other three pixels have the signal values ​​of drive pixels. The correction processing unit 325 obtains corrected data 3250 (pixel signal) by subtracting the signal value of the bottom right stop pixel from the signal value of each of the three drive pixels for each pixel group.

[0094] As described above, in the distance image sensor 32 (an example of a solid-state image sensor) according to this embodiment, the multiple pixels 321 are arranged in a matrix. The drive control unit 322 then controls the driving of the multiple pixels 321 by classifying them into N×M pixel groups, with each group consisting of drive pixels and stationary pixels.

[0095] As a result, the distance image sensor 32 does not decimate data on a column-by-column or row-by-row basis, as in the first to third embodiments, thus suppressing a decrease in the resolution of the correction data 3250 compared to the first to third embodiments. In addition, the correction processing unit 325 may fill in the signal values ​​of stop pixels included in the correction data 3250 by interpolation. Interpolation can be performed, for example, by taking the average value of the eight pixels adjacent to the stop pixel. By performing interpolation, correction data 3250 that does not include the signal values ​​of stop pixels is obtained.

[0096] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configurations are not limited to these embodiments and include designs and the like that do not depart from the spirit of this invention.

[0097] In the above embodiment, an example configuration was shown in which the distance image sensor 32 is equipped with a correction processing unit 325, but the correction processing unit 325 may be provided in the distance image processing unit 4. For example, the distance image imaging device 1 (an example of an imaging device) may be equipped with a distance image sensor 32 (an example of a solid-state image sensor) comprising the above-mentioned pixels 321, a drive control unit 322, a readout selection unit 323, and a readout unit 324, and may also be equipped with a correction processing unit 325 separately from the distance image sensor 32.

[0098] Furthermore, the control performed by the timing control unit 41 of the distance image acquisition device 1 may be performed by the drive control unit 322 of the distance image sensor 32. For example, if the drive control unit 322 performs the control performed by the timing control unit 41, the distance image acquisition device 1 may be configured without the timing control unit 41.

[0099] Furthermore, in the above embodiment, an example was described in which driving is controlled by classifying pixels into driven pixels and stopped pixels on a column-by-column, row-by-row, or pixel group-by-pixel basis. However, two or more of these methods may be combined.

[0100] Furthermore, in the above embodiment, the image sensor (solid-state image sensor) according to this embodiment was described using a distance image sensor 32 used for distance measurement as an example, but it is not limited to this, and a general image sensor for capturing images may also be used. That is, when the image sensor captures an image, the noise acquisition and noise correction processing according to this embodiment may be applied. Also, although an example in which the pixel 321 has multiple (for example, four) pixel signal output units RU was described using Figure 3, a configuration with one pixel signal output unit RU is also possible.

[0101] The distance image acquisition device 1 and distance image processing unit 4 in the above-described embodiment may be implemented in whole or in part by a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. Moreover, "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in such cases. Furthermore, the above-mentioned program may be for implementing a part of the above-mentioned function, or it may be a program that can implement the above-mentioned function in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA. [Explanation of symbols]

[0102] 1... Distance imaging device 2...Light source section 21...Light source device 22... Diffuser 3...Light receiving section 31... Lens 32... Distance image sensor 320…Light receiving area 321... pixels 322... Drive control unit 323...Read selection section 324...Reading section 325...Correction Processing Unit 4… Distance image processing unit 41... Timing control unit 42...Distance calculation section 43...Measurement Control Unit CS…Charge storage section PO... Light pulse RL…Reflected light OB…Subject

Claims

1. Multiple pixels capable of accumulating charge corresponding to incident light, A drive control unit capable of controlling the driving of multiple pixels by classifying them into drive pixels and stop pixels, A readout selection unit that selects to read out a signal corresponding to the charge accumulated in the drive pixel and a signal corresponding to the charge accumulated in the stop pixel, A correction processing unit that corrects the signal value of the drive pixel using the signal value of the stop pixel, A solid-state image sensor equipped with the following features.

2. The drive control unit, The charge accumulation time in pixels classified as stationary pixels is controlled to be shorter than the charge accumulation time in pixels classified as driving pixels. The solid-state image sensor according to claim 1.

3. A readout unit that reads out signals corresponding to the charges accumulated in multiple pixels. Equipped with, The aforementioned reading unit, The readout selection unit reads out a signal corresponding to the charge accumulated in the drive pixel or the stop pixel selected by the readout selection unit. A solid-state image sensor according to claim 1 or claim 2.

4. The multiple pixels are arranged in a matrix, The drive control unit, Multiple pixels are classified into drive pixels and stop pixels in a column-by-column order, and the driving is controlled accordingly. The solid-state image sensor according to claim 1.

5. The multiple pixels are arranged in a matrix, The drive control unit, Multiple pixels are classified into drive pixels and stop pixels on a row-by-row basis, and the driving is controlled accordingly. The solid-state image sensor according to claim 1.

6. The multiple pixels are arranged in a matrix, The drive control unit, Multiple pixels are classified into N×M pixel groups, and the pixels within each pixel group are classified into driving pixels and stationary pixels to control their operation. The solid-state image sensor according to claim 1.

7. The drive control unit and the readout selection unit each repeatedly perform their respective controls for each frame. The correction processing unit, The signal value of the drive pixel is corrected using the signal values ​​of multiple frames of the stop pixel. The solid-state image sensor according to claim 1.

8. The drive control unit and the readout selection unit each repeatedly perform their respective controls for each frame. The drive control unit, During the repeated control process for each frame, the pixels classified as drive pixels and the pixels classified as stop pixels are changed among the multiple pixels. The solid-state image sensor according to claim 1.

9. A solid-state image sensor comprising: a plurality of pixels capable of accumulating charge corresponding to incident light; a drive control unit capable of controlling the driving of the plurality of pixels by classifying them into drive pixels and stop pixels; a readout selection unit that selects the readout of a signal corresponding to the charge accumulated in the drive pixels and a signal corresponding to the charge accumulated in the stop pixels; and a correction processing unit that corrects the signal value of the drive pixels using the signal value of the stop pixels. An image processing unit that performs processing based on the signal value of the drive pixel corrected by the solid-state image sensor, An imaging device equipped with the following features.

10. A solid-state image sensor comprising: a plurality of pixels capable of accumulating charge corresponding to incident light; a drive control unit capable of classifying the plurality of pixels into drive pixels and stop pixels and controlling their drive; and a readout selection unit that selects the readout of a signal corresponding to the charge accumulated in the drive pixels and a signal corresponding to the charge accumulated in the stop pixels. A correction processing unit that corrects the signal value of the drive pixel using the signal value of the stop pixel, An imaging device equipped with the following features.

11. An imaging method performed by a solid-state image sensor having multiple pixels capable of accumulating charge corresponding to incident light, The drive control unit performs the steps of classifying the plurality of pixels into drive pixels and stop pixels and driving them, The readout selection unit selects to read out a signal corresponding to the charge accumulated in the drive pixel and a signal corresponding to the charge accumulated in the stop pixel, The correction processing unit performs the steps of correcting the signal value of the drive pixel using the signal value of the stop pixel, An imaging method that includes this.

Images