Image capture device, image capture device control method and program

Abstract

To provide an imaging apparatus that detects a high frequency flicker by the exposure and reading at less times.SOLUTION: For achieving the above purpose, a camera 100 in the present invention, includes: an image pickup device A106 having a pixel regions PA in which a plurality of unit pixels 300 having a light reception part and a charge holding part are arranged in a matrix shape; and an entire control part / operation part 101 detecting the presence / absence of a flicker. The entire control part / operation part 101 performs control so that the light reception part in a predetermined line in a different group in a plurality of groups obtained by dividing the pixel region PA in each predetermined line accumulates an optical charge in a storage period at the same timing for a unit time shorter than a flicker period, and a total sum of the storage periods that the light reception part in a plurality of different lines in the same group accumulates the optical charge by accumulating the optical charge in the storage period at a different timing when the light reception part in the plurality of different lines in the same group in the plurality of groups for unit time each other becomes a predetermined time longer than the flicker period.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging device such as a digital camera, and more particularly to an imaging device, a control method, and a program for detecting changes in the amount of external light (generally called flicker) that occur during shooting. [Background technology]

[0002] It has been known that artificial light sources such as fluorescent lamps cause so-called flicker, which is a change in the amount of light with a certain period. Patent Document 1 discloses a technique for repeating accumulation and readout at high speed, acquiring the surface average of each image, and detecting the presence or absence of flicker and its frequency based on the acquired surface average value. Patent Document 2 discloses a technique for capturing a still image without being affected by flicker, based on the detected flicker frequency. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-010317 A [Patent Document 2] JP 2019-212989 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the prior art disclosed in the above-mentioned patent documents is aimed at detecting the flicker frequency (100 Hz to 120 Hz) of conventionally known fluorescent lamps, and therefore cannot detect the high-frequency flicker (approximately 800 Hz to 1000 Hz) that occurs under recent LED light sources.

[0005] In addition, when detecting high-frequency flicker that occurs under a high-frequency light source such as an LED by extending the conventional technology disclosed in the above-mentioned patent document, it is necessary to repeat exposure at a shorter cycle than before. In this case, there is a problem that the load on the transmission band increases in order to read out the image at high speed. On the other hand, if the transmission band is limited, the amount of information to be transferred is reduced, resulting in a problem that the detection accuracy decreases. In addition, there is a problem that program control becomes difficult because exposure and readout are repeated at a short cycle.

[0006] The present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide an imaging device that detects high-frequency flicker with fewer exposures and readouts. [Means for solving the problem]

[0007] In order to achieve the above object, an imaging device according to the present invention includes an imaging element having a pixel region in which a plurality of unit pixels are arranged in a matrix, the unit pixels including a light receiving section that generates and accumulates a photocharge in response to an amount of received light and a charge holding section that holds the photocharge generated and accumulated in the light receiving section, and a control section that detects the presence or absence of flicker in a captured image based on image data output from the imaging element, and the control section controls the pixel region. multiple For each row Multiple Groups Divide into And the above Different groups in multiple groups handle Row light receiving part To During the same accumulation period, the photoelectric charge is accumulated for a unit time shorter than the flicker period. Let, The light receiving units in different rows of the same group in the plurality of groups To The photocharges are accumulated for each unit time during accumulation periods with different timings. Let, The light receiving units in the different rows of the same group are controlled so that a total of the accumulation periods during which the light charges are accumulated is a predetermined time longer than the flicker period. Effect of the Invention

[0008] According to the present invention, high frequency flicker can be detected with a small number of exposures and readouts. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of an imaging device. [Diagram 2] FIG. 2 is a diagram showing the layout of each block in FIG. 1 in an actual imaging device. [Diagram 3] FIG. 2 is a schematic diagram illustrating a configuration of a solid-state imaging element included in the imaging device. [Figure 4] 1 is a circuit configuration diagram of a unit pixel (one pixel) of an image sensor according to a first embodiment. [Diagram 5] FIG. 2 is an explanatory diagram of a part of a circuit configuration of a vertical scanning circuit. [Figure 6] 4 is a timing chart of a method for driving an image sensor according to the first embodiment. [Figure 7] 4 is a timing chart of a method for driving an image sensor according to the first embodiment. [Figure 8] 4 is a timing chart of a method for driving an image sensor according to the first embodiment. [Figure 9] 5 is a diagram showing plots of changes in light amount of flicker, accumulation timing, and surface average value according to the first embodiment. FIG. [Figure 10] FIG. 11 is a circuit configuration diagram of a unit pixel (one pixel) of an image sensor according to a second embodiment. [Figure 11] 10 is a timing chart of a driving method of an image sensor according to a second embodiment. [Figure 12] 10 is a timing chart of a driving method of an image sensor according to a second embodiment. [Figure 13] 10 is a timing chart of a driving method of an image sensor according to a second embodiment. [Figure 14] 10 is a timing chart of a driving method of an image sensor according to a second embodiment. [Figure 15] 13A to 13C are diagrams illustrating a change in the amount of flicker light, accumulation timing, and readout timing according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention.

[0011] [First embodiment] Fig. 1 is a block diagram showing a schematic configuration of an imaging device. Fig. 2 is a layout diagram of each block in an actual imaging device shown in Fig. 1. In this embodiment, a digital single-lens reflex camera will be described as an example of the imaging device.

[0012] 1 and 2, a digital single-lens reflex camera 100 (imaging device) can be fitted with an interchangeable lens 200. An overall control / calculation unit 101 (control unit) controls the digital single-lens reflex camera 100 and the interchangeable lens 200 in an integrated manner.

[0013] The imaging lens 202 forms an optical image of a subject on the imaging element A 106. The lens driving device 203 drives the imaging lens so as to bring the subject into focus. The aperture mechanism 204 controls the amount of reflected light from the subject image that passes through the lens. The aperture driving device 205 drives the aperture mechanism 204.

[0014] The interchangeable lens 200 for photographing can be detached from the digital single-lens reflex camera 100. Furthermore, communication for the purpose of exchanging information is carried out between the interchangeable lens 200 and the digital single-lens reflex camera 100. This communication is carried out by the overall control / calculation unit 101 of the digital single-lens reflex camera 100 and the lens control unit 201 of the interchangeable lens 200, which manage transmission / reception between them.

[0015] When the viewfinder is in use, the mirror 102 guides an optical image that has passed through the imaging lens 202 to the viewfinder and the AE imaging element B (AE sensor) 116. When shooting, the mirror 102 flips up to guide the optical image to the imaging element A 106. The mirror 102 is generally called a quick return (QR) mirror. The mirror drive device 103 drives the mirror 102.

[0016] The shutter 104 is a shutter mechanism having shutter curtains equivalent to front and rear curtains of a focal plane type, and controls the exposure time and blocks light of an optical image that has passed through the imaging lens 202. The shutter driver 105 drives the shutter 104.

[0017] The image sensor A106 is a solid-state image sensor for capturing an optical image of a subject formed by the imaging lens 202 as an image signal. The image sensor A106 of this embodiment is, for example, a two-dimensional XY address scanning type image sensor such as a CMOS image sensor. The image sensor A106 is configured to be capable of controlling the charge accumulation time for each line by addressing and accumulating charges simultaneously on multiple lines.

[0018] The imaging signal processor A107 performs various processes on the image data output from the imaging element A106. The various processes include amplification of the image data, various correction processes such as defect correction for the image data, and compression of the image data.

[0019] The timing generating unit A108 outputs various timing signals to the image sensor A106 and the image signal processing unit A107. The overall control / calculation unit 101 detects the presence or absence of flicker in the captured image based on the image data output from the image sensor A106 and processed by the image signal processing unit A107.

[0020] The memory unit 109 temporarily stores image data processed by the imaging signal processing unit A107, and also permanently stores various adjustment values ​​and programs for executing various controls by the overall control / calculation unit 101. The recording medium control interface (I / F) unit 110 performs recording processing of image data, etc. onto a recording medium 111, or performs reading processing of image data, etc. from the recording medium 111.

[0021] The recording medium 111 is a removable recording medium for the digital single-lens reflex camera 100, and is made up of a semiconductor memory or the like for recording various data such as image data. The display drive unit 112 drives a display device 113 that displays captured still images, moving images, etc. The external interface (I / F) 114 exchanges information such as image signals and control signals with an external device such as a computer 115.

[0022] The imaging element B116 acquires an AE signal (light source detection signal). The imaging element B116 is an RGB imager type sensor, and in this embodiment, a CCD or CMOS image sensor is used. As shown in FIG. 2, image light that has passed through the imaging lens 202 of the interchangeable lens 200 is reflected by the mirror 102, dispersed by the dispersing section 125, and input to the pentaprism 119. The pentaprism 119 guides the light beam bent by the mirror 102 to the finder 123 and the lens 122, respectively. The light focused by the lens 122 is reflected by the mirror 124 and enters the imaging element B116 (see part A in FIG. 2).

[0023] Returning to Fig. 1, the imaging signal processing unit B117 performs various processes on the image signal output from the imaging element B116. The various processes include amplification of the image signal, A / D conversion for converting from analog to digital, various correction processes such as defect correction for image data after A / D conversion, compression processing for compressing image data, etc. The timing generation unit B118 outputs various timing signals to the imaging element B116 and the imaging signal processing unit B117.

[0024] The phase difference distance measuring unit 121 (AF sensor) is a distance measuring unit using a phase difference method, and obtains a pair of images with a change in phase from a light beam transmitted through the semi-transmissive mirror 102. The phase difference distance measuring unit 121 calculates the defocus amount of the subject from the amount of shift between the pair of images, and moves the imaging lens 202. The distance measuring drive device 120 drives the phase difference distance measuring unit 121.

[0025] 3 is a schematic diagram of a solid-state imaging element A106 included in the imaging device. The pixel area PA is the pixel area of ​​the imaging element A106. In the pixel area PA, a plurality of unit pixels 300 are arranged two-dimensionally (in a matrix) across a plurality of rows and a plurality of columns, such as p11 to pkn. Each unit pixel 300 includes a light receiving section that generates and accumulates photocharges in accordance with the amount of received light, and a charge holding section that holds the photocharges generated and accumulated in the light receiving section.

[0026] Here, the circuit configuration of the unit pixel 300 (one pixel) will be described with reference to Fig. 4. Fig. 4 is a circuit configuration diagram of the unit pixel 300 (one pixel) of the image sensor according to the first embodiment. The PD (photodiode) 401 of the pixel photoelectrically converts an incident optical signal (optical image) and accumulates an electric charge according to the amount of exposure. When the control signal tx1 becomes high level, the transfer switch 403 transfers the electric charge accumulated in the PD 401 to the MEM section 404. The MEM section 404 is a charge holding section for temporarily holding the electric charge accumulated in the PD 401.

[0027] When the control signal tx2 becomes high level, the transfer switch 405 transfers the charge held in the MEM unit 404 to an FD unit (floating diffusion unit) 406. The FD unit 406 is connected to the gate of an amplifier 407, and converts the amount of charge transferred from the PD 401 into a voltage amount.

[0028] The FD reset switch 408 is a switch for resetting the FD unit 406. The FD reset switch 408 resets the FD unit 406 when the control signal res becomes high level. Furthermore, when the control signal ofd becomes high level, the reset switch 402 turns on. This resets the charge of the PD 401. When the control signal sel becomes high level, the pixel selection switch 409 outputs the pixel signal converted into a voltage by the FD and amplified by the amplifier 407 to the output vout of the unit pixel 300 (pixel).

[0029] Returning to FIG. 3, the vertical scanning circuit 301 supplies each of the unit pixels 300 with control signals such as ofd, res, tx1, tx2, and sel that control each transistor of the unit pixel 300. These control signals are common to each row of the pixel area PA. The control signals of the nth scanning line (hereinafter, the nth line) scanned and selected by the vertical scanning circuit 301 are denoted as φOFDn, φRESn, φTX1n, φTX2n, and φSELn, respectively. To avoid confusion, the symbol "_" is used as necessary in the following description. For example, the control signal φTX1 for the 2nth row may be denoted as "φTX1_2n".

[0030] An output vout of each pixel is connected to a column common readout circuit 305 via a vertical output line 302 for each column. The vertical output line 302 is provided for each column, and outputs vout of unit pixels 300 for one column are connected to the vertical output line 302. A current source 304 is connected to the vertical output line 302, and the current source 304 and an amplifier 407 of the unit pixel 300 connected to the vertical output line 302 form a source follower circuit.

[0031] The output of the column common readout circuit 305 is connected to a horizontal transfer switch 306. The horizontal transfer switch 306 is a switch that selects pixel data (pixel signals) read out by the column common readout circuit 305, and is driven by a horizontal scanning circuit 307. The pixel data selected by the horizontal scanning circuit 307 is amplified by an output amplifier 308 and output from the image sensor A106.

[0032] 5(A) and (B) are explanatory diagrams of a part of the circuit configuration of the vertical scanning circuit 301. FIG. 5(A) is a circuit configuration diagram of (a part of) the vertical scanning circuit 301, and FIG. 5(B) is a diagram 504 showing an example of a counter output and an encoder output in this embodiment. The counter 501 generates a count value cnt from an input clock. The generated count value cnt is input to the encoder 502.

[0033] The encoder 502 generates an ON / OFF switching control signal enc[n] for each scan line based on the count value cnt. The generated control signal enc[n] is input to the control unit controller n503 for each scan line. The ON-controlled scan line outputs the above-mentioned control signals such as φOFDn, φRESn, φTX1n, φTX2n, and φSELn. The OFF-controlled scan line does not output the above control signals.

[0034] Moreover, a diagram 504 in FIG. 5B shows an example of the encoder output when the phase is shifted for each count and the control signal is turned ON for every 12 lines.

[0035] When the counter output is 0, the encoder output enc[12n] (n=0, 1, 2...) is ON, and the rest are OFF. This causes the controller 12n (n=0, 1, 2...) to be controlled to be ON. Then, the control signals φOFD12n, φRES12n, φTX1_12n, φTX2_12n, and φSEL12n (n=0, 1, 2...) are output from the vertical scanning circuit 301.

[0036] At counter output 1, the encoder output enc[12n+1] (n=0, 1, 2...) is ON, and the rest are OFF. This causes the controller 12n+1 (n=0, 1, 2...) to be controlled to be ON. Then, control signals φOFD12n+1, φRES12n+1, φTX1_12n+1, φTX2_12n+1, and φSEL12n+1 (n=0, 1, 2...) are output from the vertical scanning circuit 301.

[0037] As described above, in FIG. 5B, when the counter output is m, the controller 12n+m (n=0, 1, 2, . . . ) is controlled to be ON. Then, the control signals φTX1_12n+m, φOFD12n+m, φRES12n+m, φTX2_12n+m, and φSEL12n+m (n=0, 1, 2, . . . ) are output. FIG. 5A shows the circuit configuration of the vertical scanning circuit as described above. Note that the number of phases to be skipped and the number of scan lines to be skipped that are controlled to be ON at the same timing are arbitrary, and the example given in FIG. 5B is merely an example.

[0038] Next, the operation from the start of charge accumulation of the image sensor A106 to the readout of image data according to the driving method of this embodiment will be described with reference to Figs. 6 to 8. Figs. 6 to 8 are timing charts of the driving method of the image sensor according to the first embodiment, in which Fig. 6 shows the control signals φOFD and φRES, Fig. 7 shows the control signals φTX1 and φTX2, and Fig. 8 shows the control signal φSEL. The driving method of this embodiment executes an accumulation operation of the global shutter method for each pixel for the same period for each predetermined row. In this embodiment, the predetermined row is n rows. Note that the number used for n is a natural number.

[0039] Between time t501 and t502, pulses of the control signals φRES and φTX2 are applied to all lines, turning on the transfer switch 405 and the reset switch 408. This removes unnecessary charges accumulated in the MEM section 404 and the FD section 406 of all lines, and a reset operation is performed.

[0040] At time t502, the application of the pulses of the control signals φRES and φTX2 is released, and the transfer switch 405 and the reset switch 408 are turned off.

[0041] Between time t503 and t504, pulses of the control signals φOFDn and φOFD2n are applied, turning the reset switch 402 on.

[0042] At time t504, the application of the pulses of the control signals φOFDn and φOFD2n is released, the reset switch 402 is turned off, and the accumulation operation of the charges generated in the PDs 401 of the nth line and the 2nth line is started.

[0043] Between time t505 and t506, pulses of the control signals φTX1_n and φTX1_2n are applied, the transfer switch 403 is turned on, and a MEM transfer operation for transferring the charge accumulated in the PD 401 to the MEM unit 404 is performed.

[0044] Between time t506 and t507, pulses of the control signals φOFDn+1 and φOFD2n+1 are applied, turning the reset switch 402 on.

[0045] At time t507, the application of the pulses of the control signals φOFDn+1 and φOFD2n+1 is released, the reset switch 402 is turned off, and the accumulation operation of the charges generated in the PDs 401 of the (n+1)th line and the 2n+1th line is started.

[0046] Between time t508 and t509, pulses of the control signals φTX1_n+1 and φTX1_2n+1 are applied, the transfer switch 403 is turned on, and a MEM transfer operation for transferring the charge accumulated in the PD 401 to the MEM unit 404 is performed.

[0047] Between time t509 and t510, pulses of the control signals φOFDn+2 and φOFD2n+2 are applied, turning the reset switch 402 on.

[0048] At time t510, the application of the pulses of the control signals φOFDn+2 and φOFD2n+2 is released, the reset switch 402 is turned off, and the accumulation operation of the charges generated in the PDs 401 of the (n+2)th line and the 2n+2th line is started.

[0049] Between times t511 and t512, pulses are applied to the control signals φTX1_n+2 and φTX1_2n+2, the transfer switch 403 is turned on, and a MEM transfer operation for transferring the charge accumulated in the PD 401 to the MEM unit 404 is performed.

[0050] By repeating the above operation from line n to line 2n-1, a drive can be realized in which accumulation is performed for the same period for each predetermined line, and accumulation is performed for a different period for each line.

[0051] Next, the charge transferred by the above driving to the MEM unit 404 is read out. First, the charge transferred to the MEM unit 404 in the n-th line is read out.

[0052] Between time t516 and t517, a pulse of the control signal φTX2_n is applied, the transfer switch 405 is turned on, and an FD transfer operation is performed to transfer the charge held in the MEM unit 404 to the FD 406.

[0053] Following the end of the FD transfer operation for the nth line, a pulse of the control signal φSELn is applied between times t517 and t518 to turn on the selection switch 409. This causes the charge held in the FD 406 to be converted into a voltage, which is output as pixel data (pixel signal) to the readout circuit and sequentially output by the horizontal scanning circuit 307. Then, the readout of pixel data from the pixels in the nth line is completed.

[0054] By repeating the above operation from line n to line 3n-1, sequential reading of pixel data is completed.

[0055] Fig. 9 is a diagram showing plots of the change in light quantity of flicker, accumulation timing, and surface average value according to the first embodiment. As shown in Fig. 6 to Fig. 8, in this embodiment, electric charges are accumulated in an accumulation period with the same timing for multiple lines of every n lines using the global shutter method, and within the n lines, electric charges are accumulated in an accumulation period with different timing for each line. Fig. 9 explains a case where electric charges are accumulated in an accumulation period with the same timing for multiple lines of every 12 lines. The accumulation periods are from t0 to t11, and the accumulation times are the same, t seconds (unit time).

[0056] Here, the surface average refers to the average value of the output (photometric value) of the image signal for multiple lines accumulated during a specified accumulation period. For example, it refers to the average value of the output of the image signal for multiple lines accumulated during a unit time of t seconds. Note that the unit time is shorter than the flicker period to be detected.

[0057] On the other hand, the total time (predetermined time) of the accumulation periods t0 to t11 is longer than the flicker cycle to be detected. The predetermined time is an integer multiple of the unit time, and is preferably at least twice the flicker cycle to be detected. If the predetermined time is shorter than twice the flicker cycle to be detected, it is preferable to accumulate photocharges over multiple frames.

[0058] Line 0, line 12, line 24, and so on through line 12n are corresponding lines in different groups, and charge accumulation is performed on these lines during accumulation period t0. The average value of the exposure results for multiple lines obtained by accumulating charge during this accumulation period t0 is obtained as the surface average value AE(0).

[0059] Next, lines 12n+1 such as line 1, line 13, line 25, etc. are corresponding lines in different groups, and charge accumulation is performed on these lines in accumulation period t1. The average value of the exposure results for multiple lines obtained by accumulating charges in this accumulation period t1 is obtained as the surface average value AE(1). In this way, when accumulating photocharges in an accumulation period t1 different from the accumulation period t0, photocharges of rows (lines) of adjacent unit pixels 300 are accumulated in adjacent accumulation periods such as accumulation period t0 and accumulation period t1.

[0060] The above charge accumulation, readout, and average value processing are repeated to obtain AE(0) through AE(11). From these AE(0) through AE(11), an evaluation value used to determine the period (frequency) of change in light intensity of flicker is calculated. In this embodiment, the evaluation value used to determine the period (frequency) of change in light intensity of flicker is defined by the following formula (1).

number

[0061] In formula (1), SAD is an abbreviation for Sum of Absolute Difference, and is an index of similarity used in fields such as pattern matching. The number m in formula (1) indicates how many outputs ahead the similarity is calculated for the nth output AE(n) out of 12 charge accumulations. Therefore, SAD(m) is a formula for calculating the similarity with the output after m x t seconds have elapsed. As can be seen from formula (1) above, the higher the similarity, the smaller the value of SAD(m).

[0062] For example, under a 1000 Hz flicker light source, the light intensity change period of the flicker is approximately 1 ms, and if the flicker detection period t is 0.167 ms, then the relationship between the light intensity change period and the detection period is 1÷0.167≒6. Therefore, regardless of the timing of charge accumulation, the same output is obtained in six periods, and the relationship is AE(n)≒AE(n+6). Due to this property, when SAD(6) is calculated under a 1000 Hz flicker light source, SAD(6)≒0.

[0063] Furthermore, to detect the presence of 1000 Hz flicker, SAD(3) is additionally calculated. SAD(3) is a value calculated as the similarity with the output after 0.167 x 3 = 0.5 ms has elapsed. Under a light source with 1000 Hz flicker, the photometric values ​​at a timing shifted by 0.5 ms have an inverse phase relationship, so SAD(3) will be a very large value compared to SAD(6). In other words, if SAD(3) is large and SAD(6) is small, it is considered that flicker corresponding to a light intensity change period of 1000 Hz may occur (may exist).

[0064] The results of the determination of the occurrence or absence of flicker for each of the light amount change periods described above are integrated to perform final flicker detection. The flicker period detected by the above method is used to capture a still image at the peak timing of the flicker.

[0065] As described above, the overall control / calculation unit 101 accumulates photocharges in accumulation periods with different timings for the unit pixels 300 in different rows of the pixel area PA, for a unit time that is shorter than the high-frequency flicker cycle to be detected. At this time, the overall control / calculation unit 101 controls the total accumulation period to be a predetermined time that is longer than the high-frequency flicker cycle.

[0066] The photoelectric charges accumulated during the accumulation periods with different timings are read out and the values ​​obtained are used to detect the presence or absence of flicker based on the similarity value described above. This makes it possible to detect high-frequency flicker with fewer exposures and readouts. In addition, by plotting the surface average, the period of high-frequency flicker can also be detected.

[0067] In this embodiment, the pixel area PA is divided into a plurality of groups each consisting of a predetermined number of rows of the unit pixels 300. Then, in different groups among the plurality of groups, photocharges are accumulated in corresponding unit pixels 300 of a predetermined row during an accumulation period having the same timing.

[0068] Furthermore, the unit pixels 300 in different rows in each group among the multiple groups accumulate photocharges during unit time in accumulation periods with different timings. This allows the parts in the pixel area PA that accumulate photocharges during accumulation periods with the same timing to be distributed, and the average light amount of the entire screen can be obtained. This improves the accuracy of flicker detection.

[0069] In this embodiment, the pixel area PA is divided into a plurality of groups by dividing the pixel area PA into groups of 12 lines of unit pixels 300, but this is not limiting. The number of groups into which the pixel area PA is divided and the number of predetermined rows per group can be set appropriately. In this case, it is preferable that the number of predetermined rows per group is the same.

[0070] Here, the overall control / calculation unit 101 sets a flicker cycle to be detected in advance in order to detect the presence or absence of flicker. The flicker cycle set here may be one that the overall control / calculation unit 101 changes and sets for each predetermined frame, or it may be one that the user sets as a flicker cycle that he or she wishes to detect. Note that the method of setting the flicker cycle is not limited to this.

[0071] [Second embodiment] In the first embodiment, a method of acquiring a surface average value used for determining the period of change in light quantity in a single frame is adopted. In the present embodiment, a method of acquiring a surface average value used for determining the period of change in light quantity in multiple consecutive frames is adopted.

[0072] 10 is a circuit diagram of a unit pixel 300 (one pixel) of an image sensor according to the second embodiment. The PD 801 of the pixel photoelectrically converts an incident optical signal (optical image) and accumulates an electric charge according to the amount of exposure. When the control signal tx1 or tx3 becomes a high level, the transfer gates 803a and 803b transfer the electric charge accumulated in the PD 801 to the MEM unit 804a or 804b. In this embodiment, the configuration has a plurality of MEM units. The MEM unit 804a or 804b is a charge holding unit for temporarily holding the electric charge accumulated in the PD 801.

[0073] When the control signal tx2 or tx4 becomes high level, the transfer switches 805a and 805b transfer the charge held in the MEM unit 804a or 804b to the FD unit 806. The FD unit 806 is connected to the gate of the amplifier 807, and converts the amount of charge transferred from the PD 801 into a voltage amount.

[0074] The FD reset switch 808 is a switch for resetting the FD section 806. The FD reset switch 808 resets the FD section 806 when the control signal res goes to high level. Also, when the control signal ofd goes to high level, the reset switch 802 turns on. This resets the charge of the PD 801. When the control signal sel goes to high level, the pixel selection switch 809 outputs the pixel signal converted to a voltage by the FD and amplified by the amplifier 807 to the output vout of the unit pixel 300 (pixel).

[0075] 3, in this embodiment, the vertical scanning circuit 301 supplies each of the unit pixels 300 with control signals such as ofd, res, tx1, tx2, tx3, tx4, and sel that control each transistor of the unit pixel 300. These control signals are common to each row of the pixel area PA. The control signals of the nth scanning line (hereinafter, the nth line) scanned and selected by the vertical scanning circuit 301 are denoted as φOFDn, φRESn, φTX1n, φTX2n, φTX3n, φTX4n, and φSELn, respectively.

[0076] An output vout of each pixel is connected to a column common readout circuit 305 via a vertical output line 302 for each column. The vertical output line 302 is provided for each column, and outputs vout of the unit pixels 300 for one column are connected to the vertical output line 302. A current source 304 is connected to the vertical output line 302, and the current source 304 and the amplifier 807 of the unit pixel 300 connected to the vertical output line 302 form a source follower circuit.

[0077] The output of the column common readout circuit 305 is connected to a horizontal transfer switch 306. The horizontal transfer switch 306 is a switch that selects pixel data (pixel signals) read out by the column common readout circuit 305, and is driven by a horizontal scanning circuit 307. The pixel data selected by the horizontal scanning circuit 307 is amplified by an output amplifier 308 and output from the image sensor A106.

[0078] Next, the operation from the start of charge accumulation of the image sensor A106 according to the driving method of this embodiment to the readout of image data will be described with reference to Fig. 11 to Fig. 14. Fig. 11 to Fig. 14 are timing charts of the driving method of the image sensor according to the first embodiment. Fig. 11 shows the control signals φOFD and φRES, Fig. 12 shows the control signals φTX1 and φTX2, Fig. 13 shows the control signals φTX3 and φTX4, and Fig. 14 shows the control signal φSEL.

[0079] Between time t901 and t902, pulses of control signals φRES, φTX2, and φTX4 are applied to all lines, turning on the transfer switch 805a, the transfer switch 805b, and the reset switch 808. This removes unnecessary charges accumulated in the MEM unit 804a, the MEM unit 804b, and the FD unit 806 of all lines, and a reset operation is performed.

[0080] At time t902, application of the pulses of the control signals φRES, φTX2, and φTX4 is released, and the transfer switch 805a, the transfer switch 805b, and the reset switch 808 are turned off.

[0081] Between time t903 and t904, pulses of the control signals φOFDn and φOFD2n are applied, turning the reset switch 802 on.

[0082] At time t904, the application of the pulses of the control signals φOFDn and φOFD2n is released, the reset switch 802 is turned off, and the accumulation operation of the charges generated in the PDs 801 of the nth line and the 2nth line is started.

[0083] Between time t905 and t906, pulses of the control signals φTX1n and φTX1_2n are applied, the transfer switch 803a is turned on, and a MEM transfer operation is performed to transfer the charge accumulated in the PD 801 to the MEM unit 804a.

[0084] Between time t906 and t907, pulses of the control signals φOFDn+1 and φOFD2n+1 are applied, turning the reset switch 802 on.

[0085] At time t907, the application of the pulses of the control signals φOFDn+1 and φOFD2n+1 is released, the reset switch 802 is turned off, and the accumulation operation of the charges generated in the PDs 801 of the (n+1)th line and the 2n+1th line is started.

[0086] Between time t908 and t909, pulses of the control signals φTX1_n+1 and φTX1_2n+1 are applied, the transfer switch 803a is turned on, and a MEM transfer operation is performed to transfer the charge accumulated in the PD 801 to the MEM unit 804a.

[0087] Between time t909 and t910, pulses of the control signals φOFDn+2 and φOFD2n+2 are applied, turning the reset switch 802 on.

[0088] At time t910, the application of the pulses of the control signals φOFDn+2 and φOFD2n+2 is released, the reset switch 802 is turned off, and the accumulation operation of the charges generated in the PDs 801 of the (n+2)th line and the 2n+2th line is started.

[0089] Between times t911 and t912, pulses are applied to the control signals φTX1_n+2 and φTX1_2n+2 to turn on the transfer switch 803a, and a MEM transfer operation is performed to transfer the charge accumulated in the PD 801 to the MEM unit 804a.

[0090] By repeating the above operation from line n to line 2n-1, a drive can be realized in which accumulation is performed for the same period for each predetermined line, and accumulation is performed for a different period for each line.

[0091] Next, the operation of reading out the electric charge transferred to the MEM unit 804a by the above driving and the operation of storing electric charge in the PD 801 and transferring the electric charge to the MEM unit 804b are performed in parallel.

[0092] At time t916, pulses of the control signals φOFDn and φOFD2n are applied to turn on the reset switch 802. At the same time, a pulse of the control signal φTX2_n is applied to turn on the transfer switch 805a, and an FD transfer operation is performed to transfer the charge held in the MEM unit 804a to the FD 806.

[0093] At time t917, the application of the pulses of the control signals φOFDn and φOFD2n is released, the reset switch 802 is turned off, and the accumulation operation of the charges generated in the PDs 801 of the nth line and the 2nth line is started. At the same time, the application of the pulse of the control signal φTX2_n is released, the pulse of the control signal φSELn is applied, and the selection switch 809 is turned on.

[0094] As a result, the charge held in the FD 806 is converted into a voltage, output as pixel data (pixel signal) to the readout circuit, and sequentially output by the horizontal scanning circuit 307. Then, the readout of pixel data from the pixels in the nth line is completed.

[0095] Between time t918 and t919, pulses of the control signals φTX3_n and φTX3_2n are applied, the transfer switch 803b is turned on, and a MEM transfer operation is performed to transfer the charge accumulated in the PD 801 to the MEM unit 804b.

[0096] At time t919, pulses of the control signals φOFDn+1 and φOFD2n+1 are applied to turn on the reset switch 802. At the same time, a pulse is applied to φTX2_n+1 to turn on the transfer switch 805a, and an FD transfer operation is performed to transfer the charge held in the MEM unit 804a to the FD 806.

[0097] At time t920, the application of the pulses of the control signals φOFDn+1 and φOFD2n+1 is released, the reset switch 802 is turned off, and the accumulation operation of the charges generated in the PDs 801 of the (n+1)th and (2n+1)th lines is started. At the same time, the application of the pulse of the control signal φTX2_n+1 is released, and the pulse of the control signal φSELn+1 is applied, turning on the selection switch 809.

[0098] As a result, the charge held in the FD 806 is converted into a voltage, output as pixel data (pixel signal) to the readout circuit, and sequentially output by the horizontal scanning circuit 307. Then, the readout of pixel data from the pixels in the nth line is completed.

[0099] Between time t921 and t922, pulses of the control signals φTX3_n+1 and φTX3_2n+1 are applied, the transfer switch 803b is turned on, and a MEM transfer operation is performed to transfer the charge accumulated in the PD 801 to the MEM unit 804b.

[0100] By repeating the above read operation from line n to line 3n-1, sequential readout of pixel data is completed. In addition, by repeating the above accumulation operation from line n to line 2n-1, a drive can be realized in which accumulation is performed for the same period for each predetermined line, and also for different periods for each line. By performing these read and accumulation operations in parallel, it is possible to obtain the surface average value used to determine the period of change in the light amount over multiple consecutive frames.

[0101] In this embodiment, reading and storage are performed at the same time as an example, but there is no limitation to these times, and various modifications and changes are possible within the scope of the gist of the present invention.

[0102] The unique effect of the second embodiment is that even when the flicker period is long, multiple charge storage units are arranged in parallel. This makes it possible, for example, when there are two charge storage units, to store photocharges in one charge storage unit while reading out photocharges from the other charge storage unit. This makes it possible to obtain a surface average value used to determine the period of change in the amount of light over multiple consecutive frames, making it possible to detect flicker.

[0103] In this embodiment, the multiple charge storage units are arranged in parallel in the unit pixel 300, but the present invention is not limited to this, and the multiple charge storage units may be arranged in series. Even in this case, when photocharges are accumulated over multiple frames, the photocharges can be accumulated continuously in time over the multiple frames.

[0104] [Third embodiment] In the second embodiment, when obtaining a surface average value used for the light amount change period over multiple frames, a method was adopted in which photocharge accumulation and readout were performed on the same row over multiple consecutive frames. In this embodiment, a surface average value is obtained over multiple consecutive frames, but a method is adopted in which accumulation and readout are performed simultaneously on exclusive lines.

[0105] Fig. 15 is a diagram showing the change in light quantity of flicker, the accumulation timing, and the readout timing according to the third embodiment. In this embodiment, electric charges are accumulated in an accumulation period with the same timing for multiple lines of n lines in a global shutter system, and within the n lines, electric charges are accumulated in an accumulation period with different timing for each line. Fig. 15 explains a case where electric charges are accumulated at the same time for every 12 lines. The accumulation period is from t0 to t11, and the accumulation time for each is the same, t seconds.

[0106] In this embodiment, every t seconds is divided into one frame, a vertical synchronization signal VD is issued, and the accumulation line is read out in the frame following accumulation. Also, charge accumulation for the next frame is performed during the accumulation period of the same timing. That is, in the same frame, accumulation and readout of photocharges are performed simultaneously in different lines.

[0107] In the accumulation period t0, accumulation is performed on the line 12n (n=0, 1, 2...). In the accumulation period t1, readout is performed on the line 12n (n=0, 1, 2...), and then accumulation is performed on the line 12n+1 (n=0, 1, 2...).

[0108] FIG. 15 shows the relationship in which readout of line 12n+m-1 (n=0, 1, 2 . . . ) is performed and then accumulation of line 12n+m (n=0, 1, 2 . . . ) is performed during accumulation period tm.

[0109] The unique effect of the third embodiment is that photocharges are stored and read out simultaneously on different lines in multiple consecutive frames. This makes it possible to obtain information on the entire screen compared to conventional methods. This increases the amount of information used to determine the light intensity change period, making it possible to improve the flicker detection accuracy.

[0110] [Other embodiments] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. In addition, parts of the above-mentioned embodiments may be combined as appropriate.

[0111] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0112] 100…Digital SLR camera 101... Overall control unit / calculation unit 106...Image sensor A 300...Unit pixel 404…MEMS section 804a, 804b…MEM section PA…Pixel area

Claims

1. an image sensor having a pixel region in which a plurality of unit pixels are arranged in a matrix, the unit pixels each having a light receiving section that generates and accumulates a photocharge in accordance with an amount of received light and a charge holding section that holds the photocharge generated and accumulated in the light receiving section; a control unit that detects the presence or absence of flicker in a captured image based on image data output from the imaging element, The control unit is an imaging device comprising: dividing the pixel region into a plurality of groups each consisting of a plurality of rows; causing light receiving sections of corresponding rows in different groups among the plurality of groups to accumulate the photoelectric charge during an accumulation period of the same timing for a unit time shorter than a flicker period; causing light receiving sections of different rows of the same group among the plurality of groups to accumulate the photoelectric charge for the unit time during accumulation periods of different timings from each other; and controlling so that a total accumulation period during which the light receiving sections of the different rows of the same group accumulate the photoelectric charge is a predetermined time longer than the flicker period.

2. The control unit detects a period of the flicker based on an average value of values ​​obtained by reading out the photoelectric charges accumulated during the accumulation periods of the same timing and an average value of values ​​obtained by reading out the photoelectric charges accumulated during the accumulation periods of the different timing.

2. The imaging device according to claim 1 .

3. The plurality of groups are each formed of consecutive rows of the light receiving units, and are formed such that the number of rows of the light receiving units is the same.

3. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

4. When accumulating the photoelectric charges in the accumulation periods with different timings, the photoelectric charges of adjacent rows of the photoreceptor are accumulated at adjacent times.

4. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

5. The predetermined time is an integer multiple of the unit time.

5. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

6. The predetermined time is equal to or longer than twice the flicker period.

6. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

7. The control unit accumulates the photoelectric charge over a plurality of frames when the predetermined time is shorter than twice the flicker period.

6. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

8. The unit pixel includes a plurality of the charge holding portions.

8. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

9. The charge holding sections are arranged in parallel in the light receiving section.

9. The imaging device according to claim 8.

10. The control unit controls the light receiving unit to accumulate the photocharges over a plurality of frames, and to accumulate the photocharges in different rows and to read out the photocharges in the same frame.

10. The imaging device according to claim 8, wherein the first and second lenses are arranged in a first direction.

11. A control method for an imaging device that controls an imaging element having a pixel region in which a plurality of unit pixels are arranged in a matrix, the unit pixels including a light receiving section that generates and accumulates a photocharge in response to an amount of received light and a charge storage section that stores the photocharge generated and accumulated in the light receiving section, the method comprising: dividing the pixel region into a plurality of groups for each plurality of rows, causing light receiving units of corresponding rows in different groups among the plurality of groups to accumulate the photoelectric charges for a unit time shorter than a flicker period during accumulation periods of the same timing, causing light receiving units of different rows of the same group among the plurality of groups to accumulate the photoelectric charges for the unit time during accumulation periods of different timings from each other, and controlling the imaging element so that a total accumulation period during which the light receiving units of the different rows of the same group accumulate the photoelectric charges is a predetermined time longer than the flicker period; detecting the presence or absence of flicker based on image data output from the imaging element; 13. A method for controlling an imaging apparatus comprising:

12. The method for controlling an imaging apparatus according to claim 11 is characterized in that the method is executed by a computer. program.

Images