Imaging apparatus, method for controlling imaging apparatus, and program

Abstract

To provide an imaging apparatus, a method for controlling the imaging apparatus, and a program that can reduce a difference in exposure during imaging under a flicker light source.SOLUTION: An imaging apparatus 1 can perform imaging in a state where light is emitted from a flicker light source 10. The imaging apparatus 1 comprises imaging means (image pick-up device 100), detection means (flicker detection unit 109) that detects a light emission cycle of the flicker light source 10, and control means (CPU 104). The imaging means includes photoelectric conversion means (photoelectric conversion unit 301) that performs photoelectric conversion to generate electric charges, and storage means 300 that stores the electric charges from the photoelectric conversion means sequentially at least twice, that is, first storage and second storage, per frame, and maintains the storage state. According to the light emission cycle detected by the detection means, the control means controls the time from the start of the first storage until the start of the second storage to be 1 / N of the light emission cycle.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Shooting may be performed under a light source such as a fluorescent lamp that repeatedly blinks at a cycle of several hundred to several thousand Hz (hereinafter referred to as a "flicker light source"). In this case, the captured image may have flicker (flicker phenomenon) in which stripes or light and dark appear in the image due to the influence of the flicker light source. Some imaging devices are equipped with a global shutter function (hereinafter referred to as "GS") that allows simultaneous exposure over the entire imaging surface. In imaging devices equipped with GS, the exposure of the imaging surface may shift for each frame due to the influence of the flicker light source. As devices for reducing the influence of the flicker light source, devices described in Patent Documents 1 and 2 are known. Patent Document 1 discloses an imaging device that changes the charge accumulation period in the imaging device according to the light emission cycle of the flicker light source to reduce the exposure difference for each frame. Patent Document 2 discloses an imaging device that reduces the influence of the flicker by ensuring an accumulation time that is sufficiently longer than the flicker cycle and asynchronously setting the accumulation start timing with the blinking cycle (flicker cycle) of the flicker light source. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-78038 [Patent Document 2] JP 2019-126014 A Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in the imaging device described in Patent Document 1, the exposure time (accumulation time) is changed for each frame, so the degree of image blur (subject blur) changes for each frame, that is, the amount of subject blur may vary.

[0005] The present invention has been made in view of the above problems, and an object of the present invention is to provide an imaging device, a control method for an imaging device, and a program that can reduce exposure differences when capturing images under a flickering light source. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the imaging device of the present invention is an imaging device capable of capturing images when light is emitted from a light source that periodically blinks, and comprises an imaging means for capturing the images, a detection means for detecting the light emission period of the light source, and a control means for controlling the operation of the imaging means and the detection means, wherein the imaging means comprises a photoelectric conversion means for performing photoelectric conversion to generate electric charges, and an accumulation means for accumulating the electric charges from the photoelectric conversion means at least twice per frame, a first accumulation and a second accumulation, and maintaining the accumulation state, and wherein the control means controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission period (where N is an even number) in accordance with the light emission period detected by the detection means. Effect of the Invention

[0007] According to the present invention, it is possible to reduce the exposure difference when capturing an image under a flickering light source. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an overall configuration of an imaging device according to a first embodiment. [Diagram 2] 1 is a block diagram illustrating an overall configuration of an imaging element. [Diagram 3] FIG. 2 is a circuit diagram illustrating a schematic circuit configuration of a pixel. [Figure 4]4 is a timing chart showing the operation timing of a flicker light source and an image sensor. [Diagram 5] 10 is a timing chart (modification) showing the operation timing of a flicker light source and an imaging element. [Figure 6] 4 is a flowchart showing an imaging operation in the imaging device. [Figure 7] 10 is a timing chart showing the operation timing of a flicker light source and an image sensor according to a second embodiment. [Figure 8] 4 is a flowchart showing an imaging operation in the imaging device. [Figure 9] 13 is a timing chart showing the operation timing of a flicker light source and an image sensor according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can exert a similar function. In addition, any configuration may be added. In addition, any two or more configurations (features) of each embodiment can be combined.

[0010] First Embodiment Hereinafter, the first embodiment will be described with reference to Figs. 1 to 6. Fig. 1 is a block diagram showing an outline of the overall configuration of an imaging device according to the first embodiment. The imaging device 1 shown in Fig. 1 has an imaging element (imaging means) 100, a signal processing circuit 101, a lens 102, a lens control unit 103, a CPU (control means) 104, a memory 105, a recording unit 106, a display unit 107, an operation unit 108, and a flicker detection unit (detection means) 109. This imaging device 1 can capture an image of a subject Q such as a human being in a state where light is emitted from a light source (hereinafter referred to as "flicker light source 10") that periodically blinks. The flicker light source 10 is not particularly limited, and may be, for example, a fluorescent lamp that repeatedly blinks at a cycle of several hundred to several thousand Hz.

[0011] The image sensor 100 has, for example, a global shutter function (hereinafter referred to as "GS") capable of simultaneous exposure over the entire image sensing surface, and can capture an image (image sensing process). The image sensor 100 has a pixel group 200 (described later) and converts an optical image incident via a lens 102 into an electric signal and transfers the electric signal to a signal processing circuit 101. The signal processing circuit 101 processes the electric signal from the image sensor 100 to generate a captured image and an image for focus detection. The lens 102 is driven by a lens control unit 103, and forms an optical image incident via the lens 102 on the image sensor 100. Examples of control for forming an optical image include zoom control, focus control, and aperture control. The CPU 104 is communicably connected to the image sensor 100, the signal processing circuit 101, the lens control unit 103, the memory 105, the recording unit 106, the display unit 107, the operation unit 108, and the flicker detection unit 109, and controls their operations (control process). Various control programs, such as programs for causing the CPU 104 (computer) to execute each part and each means (method of controlling the imaging device) of the imaging device 1, are stored in advance in a storage unit (not shown). The memory 105 is used to temporarily record image data. The recording unit 106 is used to read and write image data to a semiconductor memory or the like. The display unit 107 displays various information such as exposure conditions of the imaging device 1, image data, and the like. The operation unit 108 is composed of, for example, buttons and dials. For example, the imaging conditions of the imaging device 1 can be set by operating the operation unit 108. The flicker detection unit 109 detects the emission cycle of the flicker light source 10 (detection process). The flicker detection unit 109 may be capable of detecting the emission frequency of the flicker light source 10.

[0012] FIG. 2 is a block diagram showing a schematic overall configuration of an image sensor. As shown in FIG. 2, the image sensor 100 includes a pixel group 200, a vertical scanning circuit 202, a column signal line 203, a column circuit 204, a reference signal generating circuit 205, a horizontal scanning circuit 206, a register group 207, and a timing generator 208. The pixel group 200 includes a plurality of pixels 201 arranged in a matrix. The pixels 201 include transistors (not shown). The vertical scanning circuit 202 supplies driving signals to the transistors of the pixels 201 to control the transistors. These driving signals are common to each row of the pixels 201. At least one column signal line 203 is connected to each column of the pixels 201, and transfers the signals of each pixel 201 to the column circuit 204. The column circuit 204 performs processes such as amplifying the signals of the pixels 201 transferred from the column signal line 203 and analog-to-digital conversion (hereinafter referred to as "AD conversion"). The reference signal generating circuit 205 supplies a reference voltage to a circuit composed of a plurality of column circuits 204. Then, an AD conversion unit (not shown) of the column circuit 204 compares the signal of the pixel 201 with the reference voltage. The horizontal scanning circuit 206 transfers the signal of the pixel 201 processed by the column circuit 204 to the signal processing circuit 101. The register group 207 stores the settings of each operation mode of the imaging device 1 operated by, for example, the operation unit 108. The timing generator 208 operates the vertical scanning circuit 202 and the horizontal scanning circuit 206 based on the settings stored in the register group 207.

[0013] FIG. 3 is a circuit diagram that shows a schematic circuit configuration of a pixel. As shown in FIG. 3, the pixel 201 has a photoelectric conversion unit (photoelectric conversion means) 301, a switch 302, a pixel memory 303, a switch 304, a switch 305, a switch 306, and a switch 307. The photoelectric conversion unit 301 has a photodiode (not shown). Light that has passed through the lens 102 is incident on the photodiode. As a result, the photodiode can perform photoelectric conversion and generate (produce) an electric charge. Note that the number of photoelectric conversion units 301 included in the pixel 201 is one in the configuration shown in FIG. 3, but is not limited thereto, and may be, for example, more than one. When the pixel 201 has more than one photoelectric conversion unit 301, focus detection is possible by comparing the signals of the photoelectric conversion units 301. In addition, a signal obtained by adding the signals of the photoelectric conversion units 301 can be used as an image capture signal. The switch 302 is disposed between the photoelectric conversion unit 301 and the pixel memory 303, and sets a drive signal supplied to a gate of the switch 302 to a high level. This makes it possible to transfer the charge generated in the photoelectric conversion unit 301 to the pixel memory 303.

[0014] The pixel memory 303 is a memory having a capacity sufficient to temporarily store the charge generated in the photoelectric conversion unit 301. When charges are transferred from the photoelectric conversion unit 301 multiple times before the switch 304 is turned on, the pixel memory 303 can add up all of the charges (intermittent accumulation operation). In this way, the pixel memory 303 functions as an accumulation means 300 that accumulates the charge (signal) from the photoelectric conversion unit 301. The pixel memory 303 is also communicatively connected to the signal processing circuit 101. The signal processing circuit 101 processes a signal corresponding to the charge stored in the pixel memory 303 and can temporarily store the charge. The signal processing circuit 101 functions as the accumulation means 300 together with the pixel memory 303.

[0015] The switch 304 is disposed between the pixel memory 303 and a source follower (not shown), and sets a drive signal supplied to the gate of the switch 304 to a high level. This allows the charge held in the pixel memory 303 to be transferred to the source follower. The switch 305 sets a drive signal supplied to the gate of the switch 305 to a high level. This allows the charge in the photoelectric conversion unit 301 to be reset and adjusted to a reference potential. The switches 306 and 304 simultaneously set drive signals supplied to the gates of the switches to a high level. This allows the charge in the pixel memory 303 to be reset and adjusted to a reference potential. At this time, the photoelectric conversion unit 301 can also be reset at the same time by further turning on the switch 302 at the same time. When the photoelectric conversion unit 301 is reset, the switch 305 can be omitted. The switch 307 is a selector switch that selects the pixel 201 connected to the column signal line 203. By setting the drive signal supplied to the gate of the switch 307 to a high level, the corresponding pixel 201 is connected to the column signal line 203 , and a signal can be transferred to the column circuit 204 .

[0016] As described above, the imaging device 1 is capable of capturing images under the flickering light source 10. In such imaging, the influence of the flickering light source 10 may cause flicker (flicker phenomenon) in which stripes or light and dark appear in the captured image. If the imaging element 100 has a GS function, the influence of the flickering light source 10 may cause a shift in exposure of the imaging surface for each frame. Therefore, the imaging device 1 is configured to be able to reduce such a phenomenon. The configuration and operation will be described below.

[0017] FIG. 4 is a timing chart showing the timing of the operation of the flicker light source and the image sensor. The timing chart shown in FIG. 4 is a timing chart in the case where intermittent accumulation in the accumulation means 300 is performed four times in one vertical synchronization period. In FIG. 4, the switch 302 is described as "SW_302", the switch 305 is described as "SW_305", the switch 306 is described as "SW_306", and the pixel memory 303 is described as "MEM_303". In addition, the flicker detection unit 109 can detect the light emission period of the flicker light source 10 (hereinafter referred to as "flicker period T"). As shown in FIG. 4, at time t1, the gate drive signals of the switches 305 and 306 are set to a High level. As a result, the pixel memory 303 is reset by the reference voltage Vcc. Similarly, the photoelectric conversion unit 301 is also reset by the reference voltage Vcc.

[0018] At time t2, when a predetermined reset period has elapsed since time t1, the gate drive signal of switch 306 is set to low level, thereby completing the resetting of pixel memory 303.

[0019] Next, at time t3, the gate drive signal of switch 305 is set to low level. As a result, photoelectric conversion is performed in photoelectric conversion unit 301 and charge begins to be generated. Also, at time t3, switch 302 is set to high level. As a result, charge generated in photoelectric conversion unit 301 begins to be transferred to pixel memory 303, that is, exposure of pixel 201 begins. Then, from time t3, charge is gradually accumulated in pixel memory 303.

[0020] At time t4, the gate drive signal of switch 305 is set to High level, and the gate drive signal of switch 302 is set to Low level. This interrupts the exposure of photoelectric conversion unit 301, and also interrupts the charge transfer to pixel memory 303, i.e., exposure of pixel 201 ends. Furthermore, between time t3 and time t4 (period Acc1), the amount of light from flicker light source 10 is large, i.e., the period includes maximum (peak) P1 of flicker cycle T. This increases the amount of charge excited in photoelectric conversion unit 301, and as a result, the amount of charge stored in pixel memory 303 also increases.

[0021] At time t5, similar to the control of the drive signal performed at time t3, exposure of pixel 201 is started. At this time, CPU 104 controls the interval from time t3 to time t5 to 1 / 2 of the flicker period T according to the flicker period T of flicker light source 10 detected by flicker detection unit 109.

[0022] At time t6, exposure of pixel 201 is terminated in the same manner as the control of the drive signal performed at time t4. Furthermore, the light intensity of flicker light source 10 is low between time t5 and time t6 (period Acc2), that is, the period includes minimum P2 of flicker cycle T. This reduces the amount of charge excited in photoelectric conversion unit 301, and as a result, the amount of charge stored in pixel memory 303 also reduces. In the graph of MEM_303, the slope (gradient) during period Acc2 is smaller than the slope during period Acc1.

[0023] At time t7, exposure of the pixel 201 starts in the same manner as the control of the drive signal performed at time t3.

[0024] At time t8, exposure of pixel 201 is terminated in the same manner as the control of the drive signal performed at time t4. Also, between time t7 and time t8 (period Acc3), like period Acc1, the amount of light from flicker light source 10 is large, that is, the period includes maximum P3 of flicker cycle T. This increases the amount of charge excited in photoelectric conversion unit 301, and as a result, the amount of charge stored in pixel memory 303 also increases. Note that maximum P3 is the same size as maximum P1.

[0025] At time t9, similar to the drive signal control performed at time t3, exposure of the pixel 201 is started. At this time, the CPU 104 controls the interval from time t7 to time t9 to be 1 / 2 of the flicker period T.

[0026] At time t10, exposure of pixel 201 is terminated in the same manner as the control of the drive signal performed at time t4. In addition, the period from time t9 to time t10 (period Acc4) includes the minimum P4 of the flicker cycle T, in which the amount of light from the flicker light source 10 is small, as in the period Acc2. This reduces the amount of charge excited in the photoelectric conversion unit 301, and as a result, the amount of charge stored in the pixel memory 303 is also reduced. In the graph of MEM_303, the slope (gradient) in period Acc4 is smaller than that in period Acc3. In addition, the slope in period Acc1 is the same as the slope in period Acc3, and the slope in period Acc2 is the same as the slope in period Acc4. Note that the minimum P4 is the same in magnitude as the minimum P2.

[0027] At time t11, the drive signal is controlled in the same manner as at time t1.

[0028] As described above, in the imaging device 1, electric charge from the photoelectric conversion unit 301 is accumulated four times in sequence per frame, and this accumulation state is maintained; that is, the number of times electric charge is accumulated per frame is controlled to four times (an even number). Note that, although the number of times electric charge is accumulated is four in this embodiment, it is not limited to four as long as it is an even number, and can be two or six times, for example, depending on the vertical synchronization period and the absence of a flicker cycle T. Then, the CPU 104 sets the first accumulation (period Acc1) of the four accumulations as the first accumulation, sets the second accumulation (period Acc2) as the second accumulation, and controls the time from the start of the first accumulation (time t3) to the start of the second accumulation (time t5) to 1 / 2 of the flicker cycle T. Furthermore, CPU 104 sets the third accumulation (period Acc3) as the first accumulation, sets the fourth accumulation (period Acc4) as the second accumulation, and controls the time from the start of the first accumulation (time t7) to the start of the second accumulation (time t9) to 1 / 2 of the flicker cycle T. Note that the time from the start of the first accumulation to the start of the second accumulation is controlled to 1 / 2 of the flicker cycle T in this embodiment, but is not limited to this as long as it is 1 / N (where N is an even number), and may be, for example, 1 / 4 or 1 / 8. Note that when subject Q is moving, the larger N is, the shorter the intermittent accumulation interval can be, and as a result, it is possible to prevent or suppress an unnatural feeling in the image due to image blur of subject Q. Also, CPU 104 matches the length from time t3 to time t7 to the flicker cycle T, and matches the length from time t7 to time t11 to the flicker cycle T. The length from time t3 to time t7 and the length from time t7 to time t11 may each be a multiple of the flicker cycle T. In this way, the CPU 104 can adjust the total accumulation period of the charge per frame in accordance with the flicker cycle T, that is, within the flicker cycle T.

[0029] By determining the times t3, t5, t7, and t9, exposure occurs when the light amount of the flicker light source 10 is large during the period Acc1, and exposure occurs when the light amount of the flicker light source 10 is small during the period Acc2. Similarly, exposure occurs when the light amount of the flicker light source 10 is large during the period Acc3, and exposure occurs when the light amount of the flicker light source 10 is small during the period Acc4. Therefore, the total amount of charge accumulated during the periods Acc1 and Acc2 and the total amount of charge accumulated during the periods Acc3 and Acc4 are constant values ​​regardless of the light amount of the flicker light source 10. This makes it possible to reduce the exposure difference when capturing an image under the flicker light source 10. It is also possible to reduce flicker that causes stripes or light and dark to appear in a captured image (flicker reduction effect). Note that when the flicker light source 10 repeats cyclic emission, if the flicker light source 10 is bright during a certain period, it is considered that the flicker light source 10 is dark during a period shifted by 1 / 2 period from that period.

[0030] FIG. 5 is a timing chart (modification) showing the timing of operation of the flicker light source and the imaging element. In FIG. 4, the accumulation periods of two intermittent accumulations (from time t3 to time t7, and from time t7 to time t11) are matched to the flicker cycle T, but in FIG. 5, the accumulation periods of four intermittent accumulations (from time t3 to time t11) are matched to the flicker cycle T. As shown in FIG. 5, the CPU 104 sets the first accumulation (period Acc1) of the four accumulations as the first accumulation and the third accumulation (period Acc3) as the second accumulation, and controls the time from the start of the first accumulation (time t3) to the start of the second accumulation (time t7) to 1 / 2 of the light emission cycle. In addition, the CPU 104 sets the second accumulation (period Acc2) as the first accumulation and the fourth accumulation (period Acc4) as the second accumulation, and controls the time from the start of the first accumulation (time t5) to the start of the second accumulation (time 9) to 1 / 2 of the light emission cycle. By such control, the total amount of charge accumulated in periods Acc1 and Acc3 and the total amount of charge accumulated in periods Acc2 and Acc4 are constant regardless of the light amount of flicker light source 10. This makes it possible to reduce the exposure difference when capturing an image under flicker light source 10, as described above.

[0031] FIG. 6 is a flowchart showing an imaging operation of the imaging device. As shown in FIG. 6, in step S101, the CPU 104 judges whether or not the flickerless shooting mode is selected. This judgment is made based on, for example, whether or not the flickerless shooting mode is set in the operation unit 108. Then, if it is judged that the flickerless shooting mode is selected as a result of the judgment in step S101, the process proceeds to step S103. In steps S103 to S105, control of the timing charts shown in FIG. 4 and FIG. 5 suitable for imaging under the flickering light source 10 is performed. On the other hand, if it is judged that the flickerless shooting mode is not selected as a result of the judgment in step S101, the process proceeds to step S102. In step S102, the control of the timing charts shown in FIG. 4 and FIG. 5 is not necessary.

[0032] In step S102, the CPU 104 determines the number of intermittent accumulations (the number of times electric charges are accumulated), the intermittent accumulation interval (the accumulation interval of electric charges), and the like. In the imaging device 1, the operation unit 108 functions as a change unit capable of changing various conditions (user settings and driving modes) such as the number of intermittent accumulations and the intermittent accumulation interval according to the user's desire. The CPU 104 can determine the number of intermittent accumulations and the intermittent accumulation interval, and the like, based on the operation result of the operation unit 108. For example, it is assumed that the number of intermittent accumulations is set to 8 times as an initial setting (default). If the number of intermittent accumulations is increased from 8 times (for example, 16 times, etc.), blurring of the subject Q can be suppressed when the subject Q is moving. Conversely, if the number of intermittent accumulations is decreased from 8 times (for example, 4 times, etc.), noise during imaging can be suppressed. After step S102 is executed, the process proceeds to step S106.

[0033] In step S103, CPU 104 controls flicker detection unit 109 to detect the presence or absence of flicker cycle T in flicker detection unit 109. If it is determined in step S103 that there is a flicker cycle T, the process proceeds to step S104. On the other hand, if it is determined in step S103 that there is no flicker cycle T, the process proceeds to step S102.

[0034] In step S104, the CPU 104 controls the flicker detection unit 109 to detect the flicker cycle T.

[0035] In step S105, the CPU 104 determines the optimal number of intermittent accumulations and the intermittent accumulation interval based on the flicker cycle T detected in step S104, that is, performs control of the timing charts shown in FIG. 4 and FIG. 5 suitable for imaging under the flicker light source 10. The control in step S105 is not limited to the control of the timing charts shown in FIG. 4 and FIG. 5. In addition, in step S105, the number of intermittent accumulations and the intermittent accumulation interval may be changed by operating the operation unit 108. For example, when the user sets the noise priority mode (desiring to reduce the number of intermittent accumulations), the number of intermittent accumulations may be reduced from four to two, that is, the pair of periods Acc1 and Acc2, or the pair of periods Acc3 and Acc4 may be omitted. Then, the accumulation time of each period of the pair that is not omitted is doubled. This provides a flicker reduction effect. In addition, when the user sets the moving object priority selection mode (desiring to increase the number of intermittent accumulations), the number of intermittent accumulations calculated based on the flicker cycle T can be increased. After step S105 is executed, the process proceeds to step S106.

[0036] In step S106, the CPU 104 transmits the intermittent accumulation count and the intermittent accumulation interval determined in step S102 or step S105 to the register group 207 and sets them.

[0037] In step S107, the CPU 104 controls the lens control unit 103 based on the setting of the register group 207 in step S106, thereby performing exposure control.

[0038] In step S108, the CPU 104 reads out the pixel signal of the pixel 201 to the signal processing circuit 101 based on the setting of the register group 207 in step S106.

[0039] In step S109, the CPU 104 performs correction and development processing on the pixel signals read out in step S in the signal processing circuit 101, and ends the imaging operation of the imaging device 1.

[0040] By performing the above-described control, the exposure difference during imaging under the flickering light source 10 can be reduced.

[0041] <Second embodiment> Hereinafter, the second embodiment will be described with reference to FIG. 7 and FIG. 8, but the differences from the above-mentioned embodiment will be mainly described, and the description of the similar matters will be omitted. FIG. 7 is a timing chart showing the operation timing of the flicker light source and the image pickup element according to the second embodiment. The timing chart shown in FIG. 7 is a timing chart when the flicker cycle T is longer than one vertical synchronization period. The control (combination of digital signals) of the timing chart shown in FIG. 7 is effective when the flicker cycle T is longer than one vertical synchronization period and the charge is saturated due to intermittent accumulation. As shown in FIG. 7, when the first vertical synchronization period ends, the charge during the period Acc1 and the period Acc2 is accumulated in the pixel memory 303. Then, when the second vertical synchronization period starts, the charge accumulated during the period Acc1 and the period Acc2 is read out by the column signal line 203 and transferred to the signal processing circuit 101. When the second vertical synchronization period ends, the charge during the period Acc3 and the period Acc4 is accumulated in the pixel memory 303. Then, when the third vertical synchronization period starts, the charge accumulated between the periods Acc3 and Acc4 is read out by the column signal line 203 and transferred to the signal processing circuit 101. In addition, in the process up to this point, the CPU 104 controls the interval from time t3 to time t7 to 1 / 2 the flicker cycle T, as in the first embodiment, and controls the interval from time t5 to time t9 to 1 / 2 the flicker cycle T. As a result, the total amount of charge accumulated during the periods Acc1 and Acc3 and the total amount of charge accumulated during the periods Acc2 and Acc4 become constant values. Therefore, by adding (or averaging) the signal transferred during the first vertical synchronization period and the signal transferred during the second vertical synchronization period in the signal processing circuit, a flicker reduction effect can be obtained as in the first embodiment.

[0042] Fig. 8 is a flowchart showing an imaging operation in the imaging device. The flowchart shown in Fig. 8 is the same as the flowchart shown in Fig. 6, except that steps S201 to S203 are provided between steps S105 and S106. As shown in Fig. 8, in step S201, the CPU 104 judges whether or not the flicker cycle T detected in step S104 is shorter than the vertical synchronization period. If it is judged in step S201 that the flicker cycle T is shorter than the vertical synchronization period, the process proceeds to step S202. On the other hand, if it is judged in step S201 that the flicker cycle T is not shorter than the vertical synchronization period, the process proceeds to step S203.

[0043] In step S202, the CPU 104 accumulates (adds) charges in the pixel memory 303.

[0044] In step S203, the CPU 104 accumulates (adds) charges in the pixel memory 303 and the signal processing circuit 101.

[0045] As described above, in this embodiment, depending on the length of the flicker period T detected by flicker detection unit 109, the charge accumulation destination is switched between pixel memory 303 or pixel memory 303 and signal processing circuit 101. The above-described control enables operation according to the timing chart shown in FIG.

[0046] In the present embodiment, it is necessary to secure an intermittent accumulation period of at least the flicker cycle T. In this case, if pixel signals are read out every vertical synchronization period, the intermittent accumulation period falls short of the flicker cycle T. Furthermore, when pixel signals are read out, the charge in the pixel memory 303 is reset, making it difficult to accumulate across vertical synchronization periods. The imaging device 1 can reduce flicker by making the accumulation time constant, and instead of accumulating across vertical synchronization periods, it reads pixel signals and adds charges in the signal processing circuit 101 for each vertical synchronization period. Furthermore, when the subject brightness is high or the accumulation time per pixel signal readout is long, the charge in the pixel memory 303 tends to be easily saturated. Therefore, the prevention of saturation of the charge is avoided, and pixel signals are read out and charges are added in the signal processing circuit 101. This makes it possible to obtain pixel signals necessary for image formation.

[0047] <Third embodiment> Hereinafter, the third embodiment will be described with reference to FIG. 9, but the differences from the above-mentioned embodiment will be mainly described, and the description of the similar matters will be omitted. FIG. 9 is a timing chart showing the operation timing of the flicker light source and the image pickup element according to the third embodiment. The timing chart shown in FIG. 9 is a timing chart in the case where there are multiple flicker light sources 10 and the light emission periods of each flicker light source 10 are different from each other. In this embodiment, the flicker light source 10 includes a first flicker light source and a second flicker light source, and the flicker period T1 of the first flicker light source is longer than the flicker period T2 of the second flicker light source, that is, T1>T2. In addition, the flicker detection unit 109 can detect the flicker period T1 and the flicker period T2. In this embodiment, the number of flicker light sources 10 is two, but is not limited to this, and may be, for example, three or more.

[0048] The operations from time t1 to time t10 in the timing chart shown in Fig. 9 are the same as those in the timing chart shown in Fig. 4. As shown in Fig. 9, at time t11, exposure of the pixel 201 starts in the same manner as the control of the drive signal performed at time t3.

[0049] At time t12, the exposure of pixel 201 is ended in the same manner as the control of the drive signal performed at time t4.

[0050] At time t13, exposure of pixel 201 starts in the same manner as the control of the drive signal performed at time t3.

[0051] At time t14, similar to the control of the drive signal performed at time t4, the exposure of pixel 201 is ended.

[0052] At time t15, the drive signal is controlled in the same manner as at time t1.

[0053] The CPU 104 controls the time from the start of the first accumulation to the start of the second accumulation to 1 / 2 of the flicker cycle T2 according to the shorter (shortest) flicker cycle T2 of the flicker cycle T1 and the flicker cycle T2. Specifically, the CPU 104 controls the period from time t3 to time t5, the period from time t7 to time t9, and the period from time t11 to time t13 to 1 / 2 of the flicker cycle T2. This can reduce the influence of the second flicker light source. The CPU 104 also adjusts the total accumulation period of the charge according to the longer (longest) flicker cycle T1. Specifically, the CPU 104 matches the accumulation period of the intermittent accumulation (the period from time t3 to time t15) to the flicker cycle T1 (or a multiple of the flicker cycle T1). This allows the charge to be accumulated within the flicker cycle T1 whether the light amount of the first flicker light source is dark or bright, and therefore the total amount of charge can be kept constant. As described above, in this embodiment, in addition to reducing the influence of the first flicker light source in the third embodiment, the influence of the second flicker light source can also be reduced.

[0054] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. The present invention can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0055] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) An imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, Imaging means for performing the imaging; A detection means for detecting a light emission period of the light source; A control means for controlling the operation of the imaging means and the detection means, The imaging means is A photoelectric conversion means for generating electric charges by photoelectric conversion; a storage means for storing the electric charge from the photoelectric conversion means at least twice per frame, i.e., a first storage and a second storage, and maintaining the stored state; The imaging device is characterized in that the control means controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission period (where N is an even number) in accordance with the light emission period detected by the detection means. (Configuration 2) The storage means stores the electric charge four times, The imaging device described in configuration 1, characterized in that the control means controls the first accumulation of the four accumulations as the first accumulation, the second accumulation as the second accumulation, and controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle, and controls the third accumulation as the first accumulation, the fourth accumulation as the second accumulation, and controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle. (Configuration 3) The storage means stores the electric charge four times, The imaging device described in configuration 1, characterized in that the control means sets the first accumulation of the four accumulations as the first accumulation and the third accumulation as the second accumulation, and controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle, and sets the second accumulation as the first accumulation and the fourth accumulation as the second accumulation, and controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle. (Configuration 4) When there are a plurality of light sources and the light emission periods of the light sources are different from each other, The detection means detects the light emission period of each of the light sources, The imaging device described in any one of configurations 1 to 3, characterized in that the control means controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the shortest light emission period among the light emission periods of the light sources detected by the detection means, in accordance with the shortest light emission period. (Configuration 5) The imaging device according to any one of configurations 1 to 4, wherein the control means controls the number of times the electric charge is accumulated per frame in the accumulation means to be an even number. (Configuration 6) The imaging device according to configuration 5, further comprising a change means capable of changing the number of times the electric charge is accumulated. (Configuration 7) The imaging device according to any one of configurations 1 to 6, wherein the control means adjusts a total accumulation period of the electric charges per frame in the accumulation means in accordance with the light emission cycle. (Configuration 8) The imaging device according to configuration 7, wherein the control means keeps the total accumulation period of the electric charges within the light emission cycle. (Configuration 9) The imaging device according to configuration 7, wherein the control means sets the total accumulation period of the electric charges to a multiple of the light emission period. (Configuration 10) When there are a plurality of light sources and the light emission periods of the light sources are different from each other, The detection means detects the light emission period of each of the light sources, The imaging device according to any one of configurations 7 to 9, wherein the control means adjusts the total accumulation period of the electric charges in accordance with the longest light emission period among the light emission periods of the light sources detected by the detection means. (Configuration 11) An imaging device described in any one of configurations 1 to 10, characterized in that the storage means includes a pixel memory that temporarily stores the electric charge, and a signal processing circuit that is communicatively connected to the pixel memory, processes a signal corresponding to the electric charge stored in the pixel memory, and is capable of temporarily storing the electric charge. (Configuration 12) The imaging device described in Configuration 11, characterized in that the storage means switches the storage destination of the electric charge between the pixel memory or the pixel memory and the signal processing circuit depending on the length of the light emission period detected by the detection means. (Configuration 13) An imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, Imaging means for performing the imaging; A detection means for detecting a light emission period of the light source; A control means for controlling the operation of the imaging means and the detection means, The imaging means is A photoelectric conversion means for generating electric charges by photoelectric conversion; a storage means for storing the electric charge from the photoelectric conversion means at least twice per frame, i.e., a first storage and a second storage, and maintaining the stored state; The imaging device, wherein the control means controls the number of times the electric charge is accumulated per frame in the accumulation means to be an even number. (Configuration 14) An imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, Imaging means for performing the imaging; A detection means for detecting a light emission period of the light source; A control means for controlling the operation of the imaging means and the detection means, The imaging means is A photoelectric conversion means for generating electric charges by photoelectric conversion; a storage means for storing the electric charge from the photoelectric conversion means at least twice per frame, i.e., a first storage and a second storage, and maintaining the stored state; The imaging device, wherein the control means adjusts a total accumulation period of the electric charges per frame in the accumulation means in accordance with the light emission cycle. (Method 1) A method for controlling an imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, comprising the steps of: an imaging step of performing the imaging; a detection step of detecting a light emission period of the light source; a control step of controlling the imaging step and the detection step, The imaging step includes: a photoelectric conversion step of generating electric charges by photoelectric conversion; an accumulation step of accumulating the electric charge from the photoelectric conversion step at least twice per frame, i.e., a first accumulation and a second accumulation, and maintaining the accumulation state; A control method for an imaging device, characterized in that in the control process, the time from the start of the first accumulation to the start of the second accumulation is controlled to 1 / N of the light emission period (where N is an even number) in accordance with the light emission period detected in the detection process. (Program 1) A program for causing a computer to execute each means of the imaging device according to any one of configurations 1 to 14. [Explanation of symbols]

[0056] 1. Imaging device 10 Flickering Light Source 100 Imaging element (imaging means) 101 Signal Processing Circuit 104 CPU (control means) 108 Operation section 109 Flicker detection unit (detection means) 300 Means of storage 301 Photoelectric conversion unit (photoelectric conversion means) 303 pixel memory

Claims

1. An imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, An imaging means for performing the imaging; A detection means for detecting a light emission period of the light source; A control means for controlling the operation of the imaging means and the detection means, The imaging means is A photoelectric conversion means for generating electric charges by photoelectric conversion; a storage means for storing the electric charge from the photoelectric conversion means at least twice per frame, i.e., a first storage and a second storage, and maintaining the stored state; an imaging device characterized in that the control means controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission period (where N is an even number) in accordance with the light emission period detected by the detection means.

2. The storage means stores the charge four times; The imaging device according to claim 1, characterized in that the control means controls a first accumulation of the four accumulations as the first accumulation, a second accumulation as the second accumulation, and controls a time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle, and controls a third accumulation as the first accumulation, a fourth accumulation as the second accumulation, and controls a time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle.

3. The storage means stores the charge four times; The imaging device according to claim 1, characterized in that the control means controls the first accumulation of the four accumulations as the first accumulation and the third accumulation as the second accumulation, and controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle, and controls the second accumulation of the second accumulation as the first accumulation and the fourth accumulation as the second accumulation, and controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission cycle.

4. When there are a plurality of light sources and the light emission periods of the light sources are different from each other, The detection means detects the light emission period of each of the light sources, 2. The imaging device according to claim 1, wherein the control means controls the time from the start of the first accumulation to the start of the second accumulation to 1 / N of the shortest light emission period among the light emission periods of the light sources detected by the detection means, in accordance with the shortest light emission period.

5. 2. The image pickup apparatus according to claim 1, wherein the control means controls the number of times the electric charges are accumulated per frame in the accumulation means to be an even number.

6. 6. The imaging apparatus according to claim 5, further comprising a change unit capable of changing the number of times the electric charge is accumulated.

7. 2. The image pickup apparatus according to claim 1, wherein the control means adjusts a total accumulation period of the electric charges per frame in the accumulation means in accordance with the light emission cycle.

8. 8. The image pickup apparatus according to claim 7, wherein the control means controls the total accumulation period of the electric charges to fall within the light emission period.

9. 8. The image pickup apparatus according to claim 7, wherein the control means sets the total accumulation period of the charges to a multiple of the light emission period.

10. When there are a plurality of light sources and the light emission periods of the light sources are different from each other, The detection means detects the light emission period of each of the light sources, 8. The imaging device according to claim 7, wherein the control means adjusts the total accumulation period of the electric charges in accordance with the longest light emission period among the light emission periods of the light sources detected by the detection means.

11. 2. The imaging device according to claim 1, wherein the storage means comprises: a pixel memory that temporarily stores the electric charge; and a signal processing circuit that is communicatively connected to the pixel memory, processes a signal corresponding to the electric charge stored in the pixel memory, and is capable of temporarily storing the electric charge.

12. 12. The imaging device according to claim 11, wherein the storage means switches the storage destination of the electric charge between the pixel memory or the pixel memory and the signal processing circuit depending on the length of the light emission period detected by the detection means.

13. An imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, An imaging means for performing the imaging; A detection means for detecting a light emission period of the light source; A control means for controlling the operation of the imaging means and the detection means, The imaging means is A photoelectric conversion means for generating electric charges by photoelectric conversion; a storage means for storing the electric charge from the photoelectric conversion means at least twice per frame, i.e., a first storage and a second storage, and maintaining the stored state; The imaging device according to claim 1, wherein the control means controls the number of times the electric charge is accumulated per frame in the accumulation means to be an even number.

14. An imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, An imaging means for performing the imaging; A detection means for detecting a light emission period of the light source; A control means for controlling the operation of the imaging means and the detection means, The imaging means is A photoelectric conversion means for generating electric charges by photoelectric conversion; a storage means for storing the electric charge from the photoelectric conversion means at least twice per frame, i.e., a first storage and a second storage, and maintaining the stored state; The imaging device according to claim 1, wherein the control means adjusts a total accumulation period of the electric charges per frame in the accumulation means in accordance with the light emission cycle.

15. A method for controlling an imaging device capable of capturing an image in a state where light is emitted from a light source that periodically blinks, comprising the steps of: an imaging step of performing the imaging; a detection step of detecting a light emission period of the light source; a control step of controlling the imaging step and the detection step, The imaging step includes: a photoelectric conversion step of generating electric charges by photoelectric conversion; an accumulation step of accumulating the electric charge from the photoelectric conversion step at least twice per frame, i.e., a first accumulation and a second accumulation, and maintaining the accumulation state; a control step of controlling a time from the start of the first accumulation to the start of the second accumulation to 1 / N of the light emission period (where N is an even number) in accordance with the light emission period detected in the detection step,

16. 15. A program for causing a computer to execute each of the means of the imaging apparatus according to claim 1, 13 or 14.

Images