Imaging apparatus

Abstract

To provide an imaging apparatus capable of reducing flicker stripes while having an electronic shutter.SOLUTION: The imaging apparatus has a plurality of pixels that photoelectrically convert an image of a subject and a control unit for controlling the exposure of each of the plurality of pixels. The exposure start times and exposure end times of the plurality of pixels differ in at least a portion of the row direction and the column direction. The exposure time of the plurality of pixels is equal.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an imaging device.

Background Art

[0002] In recent years, light sources that generate flicker at higher frequencies than conventional ones, such as digital signage and LEDs, have been used. However, since such light sources have a high frequency and a steep rise in brightness, a large number of clear flicker stripes (flicker stripes) may be recorded in the captured image.

[0003] In addition, since an electronic shutter that utilizes the driving of an imaging element has a higher shutter efficiency than a mechanical shutter, flicker stripes are more clearly recorded in an image acquired using an electronic shutter than in an image acquired using a mechanical shutter, resulting in an image with a sense of discomfort.

[0004] Patent Document 1 discloses a method of reducing flicker stripes by obtaining the flicker frequency from an image and setting the shutter speed to an integer multiple of the flicker frequency.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, in the method of Patent Document 1, the shutter speed cannot be freely selected, and it is difficult to detect flicker when the flicker generation range is narrow.

[0007] An object of the present invention is to provide an imaging device that can reduce flicker stripes while having an electronic shutter.

Means for Solving the Problems

[0008] An imaging device according to one aspect of the present invention includes a plurality of pixels that photoelectrically convert a subject image, and a control unit that controls the exposure of each of the plurality of pixels. The exposure start times and exposure end times of the plurality of pixels are at least partially different in the row direction and the column direction, and the exposure times of the plurality of pixels are equal.

Effects of the Invention

[0009] An object of the present invention is to provide an imaging device that can reduce flicker stripes while having an electronic shutter.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same members are denoted by the same reference numerals, and redundant descriptions are omitted. [First Embodiment] FIG. 12 is an explanatory diagram of a subject image that flickers by a conventional focal plane (slit rolling) type electronic shutter, and shows how the brightening and dimming of a light source appear in an image due to the drive of the conventional electronic shutter. In FIG. 12(a), the plurality of quadrilaterals are each pixel 1201. In FIG. 12(b), the plurality of quadrilaterals are each pixel 1202 on the image corresponding to each of the plurality of pixels 1201. In FIG. 12(a), the position of the front curtain of the electronic shutter is shown as a shaded row V1. When the light source is turned on at the exposure start timing of row V1, as shown in FIG. 12(b), a step appears at the position corresponding to row V1 of the image.

[0012] FIG. 1 is an explanatory diagram of an example of a subject image that flickers due to the electronic shutter of the present embodiment, and shows how the flickering of the light source appears in the image due to the driving of the electronic shutter of the present embodiment. In FIG. 1(a), each of the plurality of quadrilaterals is a pixel 101. In FIG. 1(b), each of the plurality of quadrilaterals is a pixel 102 on the image corresponding to each of the plurality of pixels 101. In FIG. 1(a), the position of the front curtain of the electronic shutter is shown as a shaded pixel. The shaded pixels are driven to shift one row at a time in units of 4 columns, not in the same row on the imaging device. That is, the exposure start times and exposure end times of the plurality of pixels are at least partially different in the row direction and the column direction. Also, the exposure times of the plurality of pixels are equal. When the light source of the subject is turned on at the exposure start timing of the shaded pixel group, a step of the shape shown in FIG. 1(b) appears in the image. In this case, the projection output in the direction perpendicular to the running direction of the shutter becomes sloped. When viewed in a state where the sawtooth shape is reduced to an extent that it is not noticeable, although it is an image obtained using an electronic shutter, the flicker stripes due to the flickering of the light source of the subject are alleviated.

[0013] In FIG. 1, in order to clearly explain the difference in the exposure time of each pixel of the present invention, a sawtooth drive in which each row is shifted one by one for each column is shown, but the present invention is not limited to this. For example, as shown in FIG. 2, the number of rows to be shifted may be set to every 8 rows, and the shift amount (deviation amount) may be set randomly (so as not to be regular). In this case, even at a lower reduction rate, the shape due to the difference in exposure time is less noticeable, and the flicker stripes due to the flickering of the light source of the subject are alleviated.

[0014] As described above, according to the configuration of the present embodiment, it is possible to make the boundary of the flicker stripes inconspicuous by forming them into the shapes shown in FIGS. 1 and 2.

[0015] FIG. 3 is a block diagram of the imaging device 301 of this embodiment. The imaging element 302 photoelectrically converts an optical image (subject image) of a subject. The photographing lens 303 forms a subject image on the imaging surface of the imaging element 302. The lens drive circuit 304 performs focus control and the like using the photographing lens 303. The signal processing circuit 305 performs various arithmetic processes such as gain processing, offset correction, and white balance correction on the image signal. The imaging element 302 and the signal processing circuit 305 are connected by a plurality of signal lines. The signal output from a horizontal signal line (not shown) is input to the signal processing circuit 305. The control circuit (control unit) 306 controls the entire imaging device 301 while performing various processes such as drive control of the imaging element 302, autofocus calculation, and shutter control. The signal processing circuit 305 may be incorporated in the control circuit 306. The storage circuit 307 is a storage memory that holds image data and offset correction values, and is connected to the signal processing circuit 305. The recording circuit 308 is a removable semiconductor memory or the like, and records image data. The operation / display circuit 309 accepts user operations and displays various information.

[0016] Hereinafter, with reference to FIG. 4, the configuration of the pixel and the peripheral circuit of the imaging element 302 of this embodiment will be described. FIG. 4 is a block diagram of the pixel and the peripheral circuit of the imaging element 302 of this embodiment. In the pixel array 401, pixels 101 are arranged in a matrix. In this embodiment, it is described that there is one vertical signal line 403 in each column, but the number of vertical signal lines 403 is not limited to this, and any number according to the required performance may be used.

[0017] A selection pulse PSEL, a reset pulse PRES, and a transfer pulse PTX are input to the pixel 101. Three signal lines for transmitting different types of signals input to the pixel 101 are collectively represented as a signal line (bus) 404. A row selection switch (not shown) controls the potential selection of a specific pixel group via the signal line 404 from the vertical scanning circuit 406, and scans the pixels in the row direction (vertical scanning direction) via the vertical scanning circuit 406. The signal line 404 is connected to the pixels 101 corresponding to the attached numbers 0 to Vmax, and a common signal can be sent across rows. Thereby, pixels with the same number can be read out simultaneously.

[0018] Note that in FIG. 4, the case where the maximum value of the shift amount in the row direction of the pixel group to be read out simultaneously is four rows is shown, but the maximum value of the shift amount in the row direction can be arbitrarily set. Also, in FIG. 4, the pixel 101 and the signal line 404 are shown in a connected state, but the pixel 101 may be configured to be selectively connected to a plurality of different signal lines 404 by a switch. The signal of the pixel 101 is connected to the vertical signal line 403 by a transfer switch (not shown) and output to the column circuit 405 for each row.

[0019] A timing generator (hereinafter, TG) 407 generates a pulse signal for controlling transistors in the vertical scanning circuit 406 and the pixel 101, and a comparison reference signal. A D / A converter (hereinafter, DAC) 408 generates a reference signal (a slope signal or a ramp signal) whose level changes with time. The reference signal is input as one signal of the comparator 409 through the control of the TG 407.

[0020] The column circuit 405 is provided corresponding to each of the plurality of vertical signal lines 403. The column circuit 405 includes a comparator 409, a counter 410, and a column memory 411. A reference signal generated by the DAC 408 is input to one input of the comparator 409, and the vertical signal line 403 is connected to the other input. The comparator 409 compares the potential V of the vertical signal line 403 with a reference signal that changes with time, and detects the time until the magnitude relationship is reversed. The counter 410 measures the time until the magnitude relationship is reversed based on a clock, and uses the measured time as a digital signal. The column memory 411 holds the digital signal measured by the counter 410.

[0021] The horizontal scanning circuit 412 is controlled by the TG 507, scans the column circuit 405 in the column direction (the direction opposite to the vertical scanning direction), and outputs the digital signal held in the column memory 411 through the horizontal signal line 413 commonly connected for each column.

[0022] FIG. 5 is a circuit diagram of the pixel 101 of the image sensor 302. The photodiode (hereinafter, PD) 501 is connected to a common floating diffusion (hereinafter, FD) 503 via a transfer switch 502. The transfer switch 502 is controlled by the transfer pulse PTX output from the vertical scanning circuit 406. The FD 503 temporarily accumulates the charge transferred from the PD 501 and converts the charge into a voltage. The reset switch 504 is controlled by the reset pulse PRES and supplies the reference potential VDD to the FD 503. The pixel amplifier 505 is a source follower circuit composed of a MOS transistor and a constant current source. The selection switch 506 is controlled by the selection pulse PSEL, and outputs the potential fluctuation of the pixel amplifier 505 from the vertical signal line 403 to the column circuit 405.

[0023] FIG. 6 is a timing chart showing the read mode of the pixel 101 of the image sensor 302. FIG. 6(a) shows the operation of a pixel group (one of 0 to Vmax) read simultaneously.

[0024] At time t601, when the transfer pulse PTX becomes Hi with the reset pulse PRES in the Hi state, the pixel 101 is reset. Electron accumulation starts at PD501 from the time when the transfer pulse PTX becomes Lo. During accumulation, the reset pulse PRES is set to Hi and the transfer pulse PTX is set to Lo.

[0025] At time t602, a certain row is connected to the vertical signal line 403 by the selection pulse PSEL.

[0026] At time t603, when the reset pulse PRES changes from Hi to Lo, the potential V (reset signal N) of FD503 after reset release is input to the comparator 409 via the vertical signal line 403.

[0027] At time t604, the DAC 408 starts outputting a reference signal VL that changes in a ramp shape. Simultaneously with the output of the ramp signal by the DAC 408, the counter 410 starts counting.

[0028] At time t605, when the magnitude relationship between the input signal and the reference signal is reversed, the comparator 409 is inverted, and the value of the counter 410 at that time is held in the column memory 411.

[0029] At time t606, after the reference signal reaches a predetermined upper limit value, a signal is output by the horizontal scanning circuit 412, and the reading (N read) of the reset signal N of the pixel 101 is completed.

[0030] At time t607, the charge of the pixel 101 is transferred to FD503 by the transfer pulse PTX, and the potential V of the vertical signal line 403 becomes a potential corresponding to the pixel signal. Also, the comparator 409 is reset.

[0031] At time t608, the DAC 408 starts outputting a reference signal VL that changes in a ramp shape. Simultaneously with the output of the ramp signal by the DAC 408, the counter 410 starts counting.

[0032] At time t609, when the magnitude relationship between the input signal and the reference signal reverses, comparator 409 reverses, and the value of counter 410 at that time is held in column memory 411.

[0033] At time t610, after the reference signal reaches a predetermined upper limit value, a signal is output by horizontal scanning circuit 412, and the reading (S read) of the pixel signal of pixel 101 is completed. Thereafter, predetermined signal processing such as subtracting the N signal from the S signal is performed.

[0034] The above is the operation timing of one horizontal scan. Next, the relationship between horizontal scans will be described with reference to FIG. 6(b). FIG. 6(b) shows the relationship between pixel unit groups represented by selection pulse PSEL.

[0035] First, the start of one frame is indicated by the VD pulse. Thereafter, selection pulse PSEL is sequentially sent by vertical scanning circuit 406 to the pixels corresponding to pixel unit groups (0 to Vmax). Selection pulse PSEL becomes Hi after the selection pulse PSEL corresponding to the previous pixel unit group becomes Lo. The reading of one frame is completed when the selection pulse PSEL corresponding to the final pixel unit group Vmax becomes Lo.

[0036] When the reading is performed as described above, the signals of pixels 101 located in different rows are included in the signal horizontally transferred at one time. Therefore, when performing image generation processing based on the output of image sensor 302, it will be output as an image shifted in the row direction, so rearrangement to return the shift is necessary. Such processing may be performed by signal processing circuit 305 or inside image sensor 302. When rearrangement is performed to return the shift in the row direction, the rows at the upper and lower ends of the image do not match, so the image may be cropped and output up to the area where all row signals exist.

[0037] In addition, in this embodiment, although the method of reading the signals of the pixels 101 arranged two-dimensionally by vertical scanning and horizontal scanning has been described as an example, the present invention is not limited to this. Reading may be performed using a two-dimensional decoder that can read the pixels 101 at any position among the pixels 101 arranged two-dimensionally at any timing.

[0038] As described above, according to the configuration of this embodiment, it is possible to reduce (mitigate) flicker stripes while having an electronic shutter, and the quality of the image can be improved. [Second Embodiment] The configuration of the imaging device of this embodiment is the same as that of the imaging device 301 of the first embodiment. In this embodiment, only the configuration different from that of the first embodiment will be described, and the description of the same configuration will be omitted.

[0039] FIG. 7 is a block diagram of the pixels and peripheral circuits of the image sensor 302 of this embodiment. In the pixel array 701, pixels 702 are arranged in a matrix. In this embodiment, it will be described assuming that there is one vertical signal line 703 in each column, but the number of vertical signal lines 703 is not limited to this, and any number may be used according to the required performance.

[0040] A selection pulse PSEL, a reset pulse PRES, and a transfer pulse PTX are input to the pixel 702. In this embodiment, one signal line 704a for the selection pulse PSEL and one signal line 704b for the reset pulse PRES are provided per row of the image sensor 302. Also, in this embodiment, a plurality of signal lines 704c for the transfer pulse PTX are provided per row of the image sensor 302. In FIG. 7, the number of signal lines 704c for the transfer pulse PTX is eight, but any number may be selected according to the amount of shift in the exposure timing within a row. Also, in FIG. 7, the signal line 705 for the transfer pulse PTX is shown in a connected state, but the pixel 702 may be configured to be selectable and connectable to each signal line by a switch. The signal of the pixel 702 is connected to the vertical signal line 703 by a transfer switch (not shown) and output to the column circuit 405 row by row.

[0041] The operations after the column circuit are the same as those in the first embodiment, so the description thereof will be omitted.

[0042] FIG. 8 is a circuit diagram of the pixel 702 of the image sensor 302. Since the operations and functions of each part are the same as those in the first embodiment, the description thereof will be omitted. In the first embodiment, the signal lines for the row selection pulse PSEL, the reset pulse PRES, and the transfer pulse PTX are described by a bus, but in this embodiment, those signal lines are described individually.

[0043] FIG. 9 is a timing chart showing the readout mode of the pixel 702 of the image sensor 302 in this embodiment.

[0044] At time t901, when the transfer pulse PTX becomes Hi with the reset pulse PRES in the Hi state, the pixel 702 is reset. The accumulation of electrons starts at PD501 from the time when the transfer pulse PTX becomes Lo. The times when the transfer pulse PTX becomes Lo are shifted by a time Δt among the signal lines of the eight transfer pulses PTX. During the accumulation, the reset pulse PRES is set to Hi and the transfer pulse PTX is set to Lo.

[0045] At time t902, a certain row is connected to the vertical signal line 703 by the selection pulse PSEL.

[0046] At time t903, when the reset pulse PRES changes from Hi to Lo, the potential V (reset signal N) of FD503 after the reset release is input to the comparator 409 via the vertical signal line 703.

[0047] At time t904, the DAC 408 starts outputting a reference signal VL that changes in a ramp shape. Simultaneously with the output of the ramp signal by the DAC 408, the counter 410 starts counting.

[0048] At time t905, when the magnitude relationship between the input signal and the reference signal is reversed, the comparator 409 is inverted, and the value of the counter 410 at that time is held in the column memory 411.

[0049] After the reference signal reaches a predetermined upper limit value at time t906, a signal is output by the horizontal scanning circuit 412, and the reading (N-reading) of the reset signal N of the pixel 702 is completed.

[0050] At time t907, the charge of the pixel 702 is transferred to the FD503 by the transfer pulse PTX, and the potential V of the vertical signal line 703 becomes a potential corresponding to the pixel signal. The time when the transfer pulse PTX becomes Lo is shifted by a time Δt between the signal lines of the eight transfer pulses PTX. The amount of time shift Δt of the transfer pulses on the signal lines of the eight transfer pulses PTX is the same at the start and end of accumulation in PTX1 to 8, and the accumulation time of each pixel is equal. Also, the comparator 409 is reset.

[0051] At time t908, the DAC 408 starts outputting a reference signal VL that changes in a ramp shape. Simultaneously with the output of the ramp signal by the DAC 408, the counter 410 starts counting.

[0052] At time t909, the comparator 409 is inverted because the magnitude relationship between the input signal and the reference signal is reversed, and the value of the counter 410 at that time is held in the column memory 411.

[0053] After the reference signal reaches a predetermined upper limit value at time t910, a signal is output by the horizontal scanning circuit 412, and the reading (S-reading) of the pixel signal of the pixel 702 is completed. Thereafter, predetermined signal processing such as subtracting the N signal from the S signal is performed.

[0054] Hereinafter, the flicker stripe image obtained by the configuration of this embodiment will be described. FIGS. 10 and 11 are explanatory diagrams of a subject image that brightens and dims by the electronic shutter of this embodiment.

[0055] As shown in Fig. 10(a), the position of the front curtain of the electronic shutter when there is one vertical signal line 703 is shown as the shaded row V1. When the light source is turned on at the timing of the exposure start of row V1 with the timing of the transfer pulse PTX for each column shifted, as shown in Fig. 10(b), a region where the step is alleviated appears at the position corresponding to row V1 of the image.

[0056] As shown in Fig. 11(a), when there are a plurality of vertical signal lines 703, the selection pulse PSEL becomes Hi simultaneously for a plurality of pixels in the same column. Therefore, for example, in the imaging element 302 where there are 8 vertical signal lines 703, when the front curtain is at the positions of rows V1 to V8 and the light source is turned on at the timing when the timing of the transfer pulse PTX for each column is shifted, as shown in Fig. 11(b), a region where the step is alleviated appears.

[0057] In addition, in this embodiment, although the method of reading the signals of the pixels 702 arranged two-dimensionally by vertical scanning and horizontal scanning has been described as an example, the present invention is not limited to this. Reading may be performed using a two-dimensional decoder capable of sending a transfer pulse PTX to the pixel 702 at an arbitrary position among the pixels 702 arranged two-dimensionally at an arbitrary timing.

[0058] As described above, according to the configuration of this embodiment, it is possible to make the exposure times between columns different with a shorter time difference than in the first embodiment. Also, it is possible to combine this embodiment and the first embodiment.

[0059] The disclosure of this embodiment includes the following configurations. (Configuration 1) A plurality of pixels that photoelectrically convert a subject image, A control unit that controls the exposure of each of the plurality of pixels, and has, The exposure start time and exposure end time of the plurality of pixels are at least partially different in the row direction and the column direction, An imaging device characterized in that the exposure times of the plurality of pixels are equal. (Configuration 2) The imaging device according to Configuration 1, wherein the exposure start time and the exposure end time of pixels arranged along the direction facing the vertical scanning direction are different. (Configuration 3) The imaging device according to Configuration 1, wherein the exposure start time and the exposure end time of a region corresponding to one pixel of the output image are different in the column direction. (Configuration 4) The imaging device according to any one of Configurations 1 to 3, wherein a plurality of signal lines for transmitting the same type of signal are connected to each of the plurality of pixels. (Configuration 5) The imaging device according to any one of Configurations 1 to 4, wherein signal lines for transmitting different types of signals are connected to each of the plurality of pixels. (Configuration 6) The imaging device according to any one of Configurations 1 to 5, wherein the exposure times of a plurality of pixels having the same exposure start time and exposure end time are shifted in the row direction or the column direction between columns. (Configuration 7) The imaging device according to any one of Configurations 1 to 6, wherein the amount of shift between the exposure start time and the exposure end time in the row direction and the column direction is not regular. (Configuration 8) The imaging device according to any one of Configurations 1 to 7, wherein it is possible to switch between a reading method in which the exposure start time and the exposure end time are different in the row direction or the column direction and a reading method in which the exposure start time and the exposure end time are the same in the row direction or the column direction.

[0060] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist thereof.

Explanation of Reference Numerals

[0061] 101, 702 pixels 301 Imaging device 306 Control circuit (control unit)

Claims

1. A plurality of pixels that photoelectrically convert a subject image, and a control unit that controls the exposure of each of the plurality of pixels, wherein at least a part of the exposure start time and the exposure end time of the plurality of pixels differ in the row direction and the column direction, and the exposure times of the plurality of pixels are equal. An imaging device characterized by this.

2. The imaging device according to claim 1, wherein the exposure start time and the exposure end time of the pixels arranged along the direction facing the vertical scanning direction are different.

3. The imaging device according to claim 1, wherein the exposure start time and the exposure end time of the region corresponding to one pixel of the output image differ in the column direction.

4. The imaging device according to any one of claims 1 to 3, wherein a plurality of signal lines that transmit the same type of signal are connected to each of the plurality of pixels.

5. The imaging device according to any one of claims 1 to 3, wherein signal lines that transmit different types of signals are connected to each of the plurality of pixels.

6. The imaging device according to any one of claims 1 to 3, wherein the exposure times of the plurality of pixels having the same exposure start time and exposure end time are shifted in the row direction or the column direction between columns.

7. The imaging device according to any one of claims 1 to 3, wherein the amount of shift between the exposure start time and the exposure end time in the row direction and the column direction is not regular.

8. The imaging device according to any one of claims 1 to 3, characterized in that it is possible to switch between a reading method in which the exposure start time and the exposure end time differ in the row direction or the column direction and a reading method in which the exposure start time and the exposure end time are the same in the row direction or the column direction.

Images