Imaging apparatus, control method of the same, and program

Abstract

To provide an imaging apparatus, a control method of the same, and a program, in which an appearance of a LV image in an iris driving is improved.SOLUTION: An imaging apparatus having a an iris control part 103 of changing and controlling an iris amount of a photographing lens 101, comprises: an imaging element 104 that includes a plurality of pixels arranged in a matrix shape; a brightness detection part 107 that is one for obtaining brightness information in a moving image read from the imaging element 104, and containing one or more acquisition times in a period when a change control of the iris amount is performed, and acquires the brightness information at a plurality of acquisition times; and an exposure adjustment part 108 that adjusts an exposure of the moving image read from the imaging element 104 on the basis of a change amount of the brightness information acquired in each of the plurality of acquisition times.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an imaging device having an aperture control unit that changes and controls the aperture value of a photographing lens, and a control method and program for the same. [Background technology]

[0002] In imaging devices such as digital cameras, a method of adjusting the amount of light taken in by adjusting the lens aperture is known as a method for performing appropriate exposure control. Here, when the aperture value is changed significantly due to a change in shooting conditions, the driving control time of the aperture is longer than usual. In such a scene, a steep exposure change occurs before the desired aperture value is set. Furthermore, the aperture driving unit is driven at a timing different from the accumulation time setting and gain setting of the electronic shutter for the imaging element, which causes "flickering" in live view images, etc.

[0003] Taking such circumstances into consideration, the technology disclosed in Patent Document 1 calculates a predicted aperture value indicating an aperture value a predetermined time after the detection time of the latest aperture value of the variable aperture detected by an aperture value detection unit that detects the aperture value of the variable aperture, and then includes an exposure control unit that controls exposure of the image sensor based on the predicted aperture value obtained from a lens unit having an optical system including the variable aperture. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-3830 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the conventional technology disclosed in the above-mentioned Patent Document 1, it is necessary to provide a mechanism for predicting the movement of the aperture value on the lens control side. For example, when an interchangeable lens is attached to which the aperture value is not known in advance, it cannot be handled, and there is a problem that "flicker" occurs. In addition, when making predictions using a mechanism for predicting the change in the aperture value on the lens control side, tuning according to the characteristics of each type of lens may be required.

[0006] In the end, when an LV image is displayed while the aperture is changing, the change in aperture causes "flicker" in the LV image. If the LV image is frozen (frame stopped) while the aperture is changing in order to hide this "flicker," the continuity of the LV image display is then broken.

[0007] An object of the present invention is to provide an imaging device that improves the appearance of moving images (LV images) while the aperture value is being changed, and a control method and program for the imaging device. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the present invention provides an imaging device having an aperture control unit that changes and controls the aperture amount of a shooting lens, and is characterized in that it comprises an imaging element having a plurality of pixels arranged in a matrix, a luminance detection unit that acquires luminance information in a moving image read from the imaging element and acquires luminance information at a plurality of acquisition times including one or more acquisition times during a period in which the aperture amount is changed and controlled, and an exposure adjustment unit that adjusts the exposure of the moving image read from the imaging element based on the amount of change in the luminance information acquired at each of the plurality of acquisition times. Effect of the Invention

[0009] According to the present invention, it is possible to obtain an effect of realizing an imaging device that improves the appearance of a moving image (LV image) during a change in aperture amount, and a control method and program therefor. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating a configuration of an imaging device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an explanatory diagram of a pixel portion according to the embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram illustrating the configuration of an image sensor according to an embodiment of the present invention. [Figure 4] 4 is a flowchart showing a process flow in the first embodiment. [Diagram 5] 4 is a timing chart showing "processing when the live view accumulation time is short" in the first embodiment. [Figure 6] 4 is a timing chart showing "processing when the accumulation time of live view is long" in the first embodiment. [Figure 7] 10 is a flowchart showing a process flow in a second embodiment. [Figure 8] 10 is a timing chart showing a process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited to the configurations described in the embodiments. In this specification, "live view (image)" will also be referred to as "LV (image)" hereinafter.

[0012] <Figure 1: Example of configuration of imaging device 100> 1 is a diagram showing an example of the configuration of an imaging device 100 according to an embodiment of the present invention. The imaging device 100 has a photographing lens 101, an image sensor 104, an image processing unit 110, a brightness detection unit 111, a display unit 112, an operation unit 113, and a control unit 114.

[0013] <Photographic Lenses 101> The photographing lens 101 is composed of a photographing lens unit included in the imaging device 100, or an interchangeable lens unit that can be attached to the main body of the imaging device 100. The photographing lens 101 is composed of an aperture 102 that narrows down the light from a subject, and an aperture control unit 103 that drives and controls the aperture amount of the aperture 102. The aperture amount of the aperture 102 is driven and controlled by the aperture control unit 103 based on a control command from the control unit 114. Note that specific control of the aperture 102 is realized, for example, by a shutter and a motor that drives and controls the shutter. The aperture control unit 103 controls the aperture amount (also called "aperture value") of the aperture 102 to adjust the amount of light to an appropriate amount and guide it to the imaging element 104, so that an image of the subject is generated on the imaging element 104.

[0014] <Image sensor 104> The imaging element 104 is an image sensor or the like manufactured using a CMOS (complementary metal oxide semiconductor) or the like. The imaging element 104 has a pixel unit 105, an A / D conversion unit 106, a brightness detection unit 107, an exposure adjustment unit 108, and an output unit 109. Although only one pixel unit 105 is shown in FIG. 1, the imaging element 104 has a large number of pixel units 105 equal to the number of pixels included in the imaging element 104, and each pixel unit 105 has a photoelectric conversion function of receiving incident light and converting it into an electrical signal. The A / D conversion unit 106 performs analog-to-digital conversion on the electrical signal output by the pixel unit 105, and outputs the converted signal to the brightness detection unit 107 and the output unit 109.

[0015] <Brightness detection unit 107> The luminance detection unit 107 detects pixel values ​​in a specific region of the image by integrating a predetermined pixel range of the input digital signal, and passes the integration result for each frame to the exposure adjustment unit 108. In terms of this embodiment, the luminance detection unit 107 acquires luminance information in a moving image (such as an LV image, the same applies below) read from the pixel unit 105. Furthermore, the luminance detection unit 107 acquires luminance information at "multiple acquisition times" including one or more acquisition times during a period in which the aperture amount change control is being performed.

[0016] <Exposure adjustment section 108> The exposure adjustment unit 108 detects a change in luminance information between frames by calculating an inter-frame difference of the integration results for each frame passed thereto, and calculates an exposure change amount required for exposure correction. In terms of this embodiment, the luminance detection unit 107 adjusts the exposure of the moving image read from the pixel unit 105 based on the change amount of luminance information acquired at each of the above-mentioned "plurality of acquisition times". The output unit 109 outputs the digital signal supplied from the A / D conversion unit 106 to the outside of the image sensor 104. The output unit 109 is realized by, for example, an I / O cell, an I / O pin, etc. The digital signal transmitted from the output unit 109 is supplied to the image processing unit 110.

[0017] 1, the image sensor 104 includes a luminance detection unit 107 and an exposure adjustment unit 108 to reduce the size of the image sensor 100. However, the luminance detection unit 107 and the exposure adjustment unit 108 do not necessarily need to be provided inside the image sensor 104. For example, a luminance detection unit 111 may be provided outside the image sensor 104, and the control unit 114 may use the integration result of the luminance detection unit 111 to calculate the change in luminance between frames.

[0018] <Image processing unit 110> The image processing unit 110 performs various correction processes on the image data acquired by the imaging element 104. The image processing unit 110 executes various correction processes such as shading correction, gain correction, and defect correction. Various correction process data obtained by the correction processes of the image processing unit 110 is output to the brightness detection unit 111. In addition, the image processing unit 110 performs processes such as white balance adjustment and pixel complementation on the image data after various correction processes, generates display image data that is converted into YUV data and supplied to the display unit 112, and executes data compression processing in JPEG format or the like.

[0019] <Brightness detection unit 111> The luminance detection unit 111 detects pixel values ​​in a specific region of the image by integrating a predetermined pixel range of the input signal in the same manner as the luminance detection unit 107. Basically, the luminance detection unit 111 has the same function as the luminance detection unit 107, but the luminance detection unit 111 obtains luminance information for an image that has been subjected to various image corrections by the image processing unit 110.

[0020] <Control unit 114> The control unit 114 executes AE (automatic exposure) processing using the luminance information obtained by the luminance detection unit 111. In addition, in the case of a configuration in which the exposure adjustment unit 108 is not provided in the image sensor 104, the control unit 114 obtains a change in luminance between frames using the integration result performed by the luminance detection unit 111 in the same manner as the exposure adjustment unit 108. The control unit 114 can be realized by a processor such as a CPU or a DSP, a ROM as a recording medium for recording a program, and a RAM or the like for expanding the program. The processor such as the CPU expands the program recorded in the ROM into the RAM and executes the expanded program to realize various processes described later. In other words, the control unit 114 (CPU) controls the overall operation of the image capture device 100.

[0021] For example, the control unit 114 controls each functional block of the imaging device 100, and controls transmission and reception of required data between each functional block. In addition, the processes performed by the aperture control unit 103, the image processing unit 110, the brightness detection unit 111, the display unit 112, and the control unit 114 are also realized by a processor such as a CPU executing a control program recorded in a ROM.

[0022] <Display section 112> The display unit 112 is a display device formed of, for example, a TFT (thin film transistor) type liquid crystal display. The display unit 112 displays various setting screens in the imaging device 100, displays captured still images and moving images, and plays back images. The display unit 112 may have multiple display devices, such as an EVF (electronic viewfinder) and a rear monitor provided on the photographer's side (rear). The display unit 112 may be capable of simultaneously outputting to multiple display devices, or may be configured to selectively display by switching between them.

[0023] <Operation unit 113> The operation unit 113 is made up of various operation members, operation devices, etc., and provides a signal corresponding to the operation of the operation unit 113 by the user to the control unit 114. For example, the operation members as the operation unit 113 are a release button, a mode setting button, etc. provided on the imaging device 100. Also, the operation device as the operation unit 113 is a touch sensor or the like provided on the display screen of the display unit 117. The operation unit 113 also includes various shooting switches, etc. with which the user issues an instruction to the imaging device 100 to shoot still images or videos, and also includes operators, etc. for setting imaging conditions, etc.

[0024] <Figure 2: An explanatory diagram of the pixel section 105> Next, the configuration of the pixel units 105 included in the imaging element 104 will be described with reference to FIG. 2. One pixel unit 105 corresponds to one pixel of the imaging element 104. FIG. 2 is a configuration diagram of one pixel unit 105. A photodiode (PD) 201 functions as a "photoelectric conversion element" that photoelectrically converts an incident optical signal and accumulates an electric charge according to the amount of exposure. The transfer gate 202 is turned "on (conducting state)" by setting a control signal "tx" to a high level. As a result, the electric charge accumulated in the PD 201 is transferred to an FD unit (floating diffusion unit) 203. The FD unit 203 is connected to the gate of an amplification MOS transistor 204.

[0025] The amplification MOS transistor 204 outputs a voltage signal according to the amount of charge transferred from the PD 201 to the FD section 203. A reset switch (res) connected to a gate 205 of the amplification MOS transistor 208 is a switch for resetting the charges of the FD section 203 and the PD 201. By setting the control signal "res" to a high level, the amplification MOS transistor 208 is turned "on (conductive state)" and the FD section 203 is reset. A power supply voltage Vdd is applied to the amplification MOS transistors 204 and 208.

[0026] Moreover, when resetting the charge of the PD 201, the control signal "tx" and the control signal "res" are simultaneously set to high level. This sets both the transfer gate 202 and the reset switch (res) to the "on" state, and the PD 201 is reset via the FD section 203. Moreover, the pixel selection switch 206 is set to "on (conducting state)" by setting its control signal "sel" to "high level", and the amplification MOS transistor 204 and the "output terminal vout" 207 of the pixel section 105 are electrically connected.

[0027] As a result, the "pixel signal" converted into a voltage by the amplification MOS transistor 204 is output to the "output terminal vout" 207 of the pixel unit 105, and the "pixel signal" is output. At this time, the timing at which the control signal "sel" is turned "on" is changed. This timing change allows the order in which the "pixel signals" are read out in the row direction (horizontal direction) to be devised, thereby changing the thinning rate in the row direction of the pixels to be read out, and outputting multiple image signals.

[0028] <Figure 3: Configuration diagram of the image sensor 104> FIG. 3 shows an example of the circuit configuration of the image sensor 1004. A plurality of pixel units 105 are arranged in a matrix (rows and columns). In FIG. 3, the reference numeral 207 indicated by a black circle "·" corresponds to one pixel unit 105. Note that, to make the drawing easier to understand, the reference numeral 207 is only attached to the black circle "·" in the upper left. Also, to make the explanation easier to understand, FIG. 3 shows "3 rows × 12 columns" of pixel units 105, but in reality, hundreds of thousands to tens of millions of pixel units 105 are usually arranged.

[0029] Also, reference numeral 200 denotes a color filter provided for each pixel. To avoid complicating the drawing, only the color filter corresponding to the upper left pixel (pixel section 105) in Fig. 3 is given the reference numeral "200". "R" indicates that a red color filter, "G" indicates that a green color filter, and "B" indicates that a blue color filter are disposed on the pixel. Note that, although the pixel section 105 corresponding to "B" is not shown in Fig. 3, "B" is generally also disposed on many pixels.

[0030] A vertical scanning circuit 303 extending vertically (pixel column direction) at the left edge of the drawing supplies the control signals "res", "tx", and "sel" described in FIG. 2 to each pixel (each pixel unit 105). Meanwhile, an A / D conversion unit 106 is connected to a horizontal scanning circuit 304 extending horizontally (pixel row direction) at the top of the drawing. Each of the vertical output lines 300a to 300f is connected to an input side of the A / D conversion unit 106. The A / D conversion unit 106 performs analog-to-digital conversion on the pixel signals output from the pixel unit 105.

[0031] In addition, the lower side of each of the vertical output lines 300a to 300f is connected to a current source 302. By connecting the vertical output lines 300a to 300f to the current source 302, the amplifying MOS transistors 204 (see FIG. 2) of the pixel unit 105 form a source follower circuit. The horizontal scanning circuit 304 performs "horizontal scanning", which is scanning in the horizontal direction (row direction). The horizontal scanning circuit 304 repeatedly reads out signals in the row direction at a predetermined cycle, and the read analog signals are converted into digital signals by the A / D conversion unit 106.

[0032] Moreover, "Vout207" of pixel section 105 in a certain row direction (see FIG. 2) is connected to a pair of different vertical output lines 300a and 300b in the column direction. This configuration is similar to the connection of "Vout207" of pixel section 105 in other row directions. Thus, pixel signals of pixel section 105 are output to brightness detection section 107 and output section 109 as "first image signal" and "second image signal" (see FIG. 1). This will be explained in more detail below.

[0033] The vertical output lines 300a and 300b read out the "first image" and the "second image", respectively. At this time, the pixel area read out by the vertical output line 300b is different from the pixel area read out by the vertical output line 300a. Similarly, the vertical output lines 300c and 300d read out the "first image" and the "second image", respectively. At this time, the pixel area read out by the vertical output line 300d is different from the pixel area read out by the vertical output line 300c. Furthermore, the vertical output lines 300e and 300f read out the "first image" and the "second image", respectively. At this time, the pixel area read out by the vertical output line 300f is different from the pixel area read out by the vertical output line 300e. In this way, the accumulation of the "second image" and the readout of the image signal are performed independently of the accumulation and readout of the "first image".

[0034] <First readout mode: "Live view display mode"; Second readout mode: "Aperture drive amount detection mode"> In this embodiment, the image sensor 104 has a "first readout mode" in which a moving image is read out frame by frame, and a "second readout mode" in which a pixel signal different from that in the first readout mode is read out during the operation of the first readout mode. In this embodiment, as shown in Figs. 5 and 6, the "first readout mode" corresponds to a "live view display mode", and the "second readout mode" corresponds to an "aperture drive amount detection mode". In the first embodiment, the brightness detection unit 107 acquires brightness information using image data read out in the "second readout mode". This image data is a moving image during the driving of the aperture control unit 103. Note that in the second embodiment described later, only the "live view display mode (first readout mode)" is used.

[0035] <FIG. 4: Flowchart showing the flow of processing in the first embodiment: FIG. 5: Timing chart when the live view accumulation time is short> Next, with reference to the flowchart of FIG. 4, the flow of the exposure control process according to the change in the aperture amount during live view in the present embodiment will be described. First, in S401, the control unit 114 executes an LV image display process (LV display process) of acquiring the image data captured by the imaging device 104 in the "live view display mode" which is the "first read mode". That is, in S401, the control unit 114 starts the execution of the "LV display process". The "LV display process" is a process of sequentially displaying the captured image on the display screen of the display unit 112, and the "LV display process" is continuously executed repeatedly until the control unit 114 determines in the next S402 whether the next frame is an aperture operation frame.

[0036] Next, in S402, the control unit 114 determines whether it is necessary to operate the aperture 102 in the frame of the next LV image according to the user operation or the exposure change. If the control unit 114 determines that it is not necessary to operate the aperture 102 (No), it returns to S402 and enters the wait state. On the other hand, if the control unit 114 determines that it is necessary to operate the aperture 102 (Yes), it proceeds to S403.

[0037] <Processing when the accumulation time of the LV image is short> <frame0> In the following, the "processing when the accumulation time of the LV image is short" in this embodiment will be described with reference to the timing chart in FIG. 5. In FIG. 5, the horizontal axis indicates time, and the further to the right in the drawing, the more time has passed. In "Frame 0" in FIG. 5, if the control unit 114 determines in S402 that the aperture operation is necessary in the next frame, the following processing is executed. That is, in S403, the control unit 114 starts driving the "aperture drive amount detection mode", which is the "second readout mode", at the interval of the "vertical synchronization signal VD" of the LV image, that is, at a predetermined timing within one frame of the LV image. This can be seen by referring to FIG. 5. That is, the start time and end time of the "aperture drive amount detection mode" are within "Frame 0" of the "vertical synchronization signal VD".

[0038] In this case, as shown in Fig. 5, the start time of the "live view display mode" is a short time after the start time of "Frame 0," and its end time is a short time after the start time of "Frame 1." This is the same for all frames as shown in Fig. 5. In other words, the start time of the "live view display mode" is a short time after the start time of the "current Frame," and its end time is a short time after the start time of the "next Frame."

[0039] Furthermore, control unit 114 executes control at a timing such that the processing in S406 to S408 described below fits within one frame of the LV image. Next, in S404, image sensor 104 reads out moving images in "aperture drive amount detection mode". Next, in S405, luminance detection unit 107 or luminance detection unit 111 acquires "luminance information P0" of the image read out at "time t0" in FIG. 5.

[0040] The "aperture change amount prediction calculation" in S406 starts from the time "t0" shown in FIG. 5, and the control unit 114 or the exposure adjustment unit 108 holds the "brightness information P0" that is the reference before the aperture operation starts, acquired in S405, in a memory or the like not shown (not shown or described below). Next, in S407, the control unit 114 determines whether or not the "exposure adjustment process" in S408 is necessary. If the control unit 114 determines that it is not necessary (No), S408 is skipped and the process proceeds to S409. On the other hand, if the control unit 114 determines that it is necessary (Yes), the process proceeds to S408. The control unit 114 determines that "Frame0" is a frame before the aperture operation starts, and skips S408. The exposure adjustment process in S408 will be described later when the process is necessary.

[0041] Next, in S409, the control unit 114 updates the "vertical synchronization signal VD" in the "LV display mode". Then, in S410, the control unit 114 communicates with the aperture control unit 103 to check whether or not aperture driving by the aperture control unit 103 has ended. If the control unit 114 checks that aperture driving has ended (Yes), it ends the processing shown in FIG. 4. On the other hand, if the control unit 114 checks that aperture driving has not ended (No), it proceeds to S403 and continues processing. Specifically, it transitions from "Frame 0" to "Frame 1".

[0042] <frame1> Next, in "Frame1" shown in Fig. 5, the control unit 114 executes "aperture drive amount detection mode start processing" in S403 and "moving image readout processing in aperture drive amount detection mode" in S404, similar to "Frame0". In the "aperture drive amount detection mode" in S403, the accumulation time and readout gain settings are the same as those in "Frame0" even after "Frame1". Next, in S405, the luminance detection unit 107 or the luminance detection unit 111 acquires "luminance information P1" of the moving image read out at "time t1" in Fig. 5.

[0043] 5, the process in S406 starts, and the control unit 114 or the exposure adjustment unit 108 stores the "brightness information P1" acquired in S405 in a memory. Here, the control unit 114 or the exposure adjustment unit 108 calculates the amount of change in the aperture using the previously stored "brightness information P0" and "brightness information P1". Based on the relational equation of APEX (Addictive System of Photographic Exposure), the relational equation "Bv=Av+Tv-Sv" holds, where "brightness Bv", "shutter speed Tv", "aperture value Av" and "imaging sensitivity Sv" are used.

[0044] In the "aperture drive amount detection mode", since the "shutter speed Tv" and "imaging sensitivity Sv" are constant, the degree of change in "brightness Bv" corresponds to the degree of change in "aperture value Av". Therefore, the change in aperture drive amount between "time t1" and "time t0" can be predicted as "log2(P1 / P0)". Note that in FIG. 5, "brightness information P1" is lower than "brightness information P0" by "ΔP1=ΔPtotal1".

[0045] Next, in S407, the control unit 114 determines that this is the frame immediately after the aperture starts moving (Yes in S407), and proceeds to S408. In S408, the control unit 114 or the exposure adjustment unit 108 executes exposure adjustment processing to adjust the exposure by the amount of change "Δp1=ΔPtotal1" between "brightness information P0" and "brightness information P1". In other words, the control unit 114 or the exposure adjustment unit 108 performs settings on the image sensor 104 such that the readout gain setting of the LV image of "Frame1" and the accumulation time setting of the LV image of "Frame2" are adjusted by "log2(P1 / P0)".

[0046] Next, in S409, the control unit 114 updates the "vertical synchronization signal VD" for the LV display mode. Then, the control unit 114 determines in S410 that the diaphragm driving has not ended (No), and returns the process to S403, so that the process proceeds to "Frame 2" shown in FIG. 5.

[0047] <frame2> 5, similarly to "Frame 0" and "Frame 1", the control unit 114 executes "aperture drive amount detection mode start processing" in S403 and "moving image read processing in aperture drive amount detection mode" in S404. Next, in S405, the luminance detection unit 107 or the luminance detection unit 111 acquires "luminance information P2" of the moving image read at "time t2" in FIG.

[0048] The process of S406 starts from "time t2" shown in Fig. 5, and the control unit 114 or the exposure adjustment unit 108 holds in a memory the "brightness information P2" acquired in S405. The control unit 114 or the exposure adjustment unit 108 calculates the difference in the amount of aperture change from the previous "time t1" using "brightness information P1" and "brightness information P2" in consideration of the amount of aperture change from before the aperture drive using "brightness information P0" and "brightness information P2". As shown in Fig. 5, the brightness information has decreased by "ΔPtotal2" since "time t0", and the brightness information has decreased by "ΔP2" since "time t1".

[0049] As with the previous frame (Frame1), in S407, the control unit 114 determines that this is a frame after the diaphragm movement has started (Yes), and proceeds to S408. In S408, the control unit 114 or the exposure adjustment unit 108 executes the following process. That is, in order to adjust the exposure by the amount of change in luminance "ΔP2", the control unit 114 executes exposure adjustment processing such as adjusting the readout gain setting of the LV image of "Frame2" for the image sensor 104 by "log2(P2 / P1)".

[0050] Additionally, to adjust the exposure for the amount of change in luminance "ΔPtotal2" from "time t0", the accumulation time setting for the LV image of "Frame3" is set to be adjusted by "log2(P2 / P0)". Similarly, for subsequent frames, if a certain frame is "FrameN", an adjustment is made to add "log2(PN / PN-1)" to the readout gain setting of the LV image of "FrameN" to adjust the exposure change for the difference in luminance from the previous frame "ΔPN". Additionally, to adjust the exposure change for the total change in luminance "ΔPtotalN", an adjustment is made to add "log2(pN / p0)" to the accumulation time setting of the LV image of "FrameN+1".

[0051] In the above explanation, we have described the exposure adjustment process that adjusts the exposure for the change in aperture amount using "shutter speed Tv" and "imaging sensitivity Sv". However, it is also possible to perform the exposure adjustment process using only the read gain setting by adjusting the gain setting of the LV image of "FrameN" by adding "log2(PN / P0)". In the following, "read gain setting" will also be referred to simply as "gain setting".

[0052] <Another Frame> Next, in S409, the control unit 114 updates the "vertical synchronization signal VD" in the "LV display mode". Then, in S410, the control unit 114 repeats the same processing as in "Frame 2" in the subsequent "Frames" until it determines that the aperture driving has ended (until S410 becomes Yes). If the control unit 114 determines in S410 that the aperture driving has ended (Yes), the processing in this embodiment ends.

[0053] <Figure 6: Timing chart for processing when the LV image accumulation time is long> Next, the processing in this embodiment "when the accumulation time of the LV image is long" will be described with reference to the timing chart in Fig. 6. Specifically, "when the accumulation time of the LV image is long" refers to a case where a period in which multiple frames can be photographed can be secured during the accumulation time of the LV image in the "aperture drive amount detection mode". In this case, the "aperture drive amount detection mode" starts earlier than the start time of the "LV display mode" compared to Fig. 5. The following mainly describes the differences from the case in Fig. 5. Note that the horizontal axis in Fig. 6 also indicates time, with more time having passed as one moves to the right side of the drawing.

[0054] First, in S403, the control unit 114 starts the "aperture drive amount detection mode" which is the "second readout mode" in accordance with the start timing of accumulation of the LV image. At this time, the control unit 114 controls so that a plurality of frames are captured in the "aperture drive amount detection mode" while the LV image is being accumulated. At this time, the control unit 114 secures the processing time of S406 to S408 and ends the capture in the "aperture drive amount detection mode". As a result, timing control is performed so that the process in S408 is completed by the end timing of the "vertical synchronization signal VD" of the LV image. FIG. 6 shows a case where six frames are captured in the "aperture drive amount detection mode" while each frame (Frame 0 to Frame 4) is being read out in the "LV display mode". Note that this number of captured frames is merely an example.

[0055] <frame0> By capturing a plurality of frames in the "aperture drive amount detection mode", the brightness detection unit 107 or the brightness detection unit 111 can obtain a plurality of "brightness information" per frame of the LV image. In other words, in the case of "Frame0" shown in FIG. 6, "brightness information" can be obtained at a plurality of (six) times from "time t01" to "time t06". When performing the process in S406 from "time t06" shown in FIG. 6, the control unit 114 or the exposure adjustment unit 108 stores "Pave0", which is the average value of the plurality of "brightness information" acquired from time "t01" to time "t06", in the memory. Note that it is not necessarily the average value that is stored, and it may be the median value of the plurality of pieces of brightness information, for example.

[0056] In this way, when multiple frames are captured in the "aperture drive amount detection mode", the average or median value of multiple pieces of luminance information acquired for each of the multiple frames by the luminance detection unit 107 can be used as the luminance information corresponding to the multiple frames.

[0057] <frame1> Similarly, when executing the process of S406 from "time t16" in "Frame1" shown in FIG. 6, the control unit 114 or the exposure adjustment unit 108 holds "Pave1", which is the average value of a plurality of luminance information acquired from "time t11" to "time t16", in the memory. Here, the plurality is "6" as an example. Then, the control unit 114 or the exposure adjustment unit 108 calculates the change amount of the aperture using "Pave0" and "Pave1", which are the average values of the luminance information. Specifically, the luminance information between the two has decreased by "ΔPave1 = ΔPtotal1". Therefore, it can be predicted that the aperture drive amount changed while the LV image was being accumulated in "Frame1" shown in FIG. 6 has changed by "log2(Pave1 / Pave0)" compared to before the aperture drive.

[0058] Therefore, in S408, the control unit 114 or the exposure adjustment unit 108 executes the following process to perform an exposure adjustment process for adjusting the exposure by the change amount "ΔPave1" of the luminance information of "Pave0" and "Pave1". That is, for the imaging device 104, the gain setting of the LV image of "Frame1" and the accumulation time setting of the LV image of "Frame2" are adjusted by "log2(Pave1 / Pave0)" step by performing a process of adjusting by that amount.

[0059] <After Frame2> Similarly, the process for "Frame2" and subsequent frames shown in FIG. 6 will be described. In S406, the control unit 114 or the exposure adjustment unit 108 holds "PaveN", which is the average value of a plurality of luminance information acquired per "1 frame" of the LV image, in the memory. In S408, the control unit 114 or the exposure adjustment unit 108 designates a certain frame as "FrameN". Then, the control unit 114 or the exposure adjustment unit 108 adjusts the luminance change by "ΔPaveN", which is the difference in luminance from the previous frame "FrameN - 1". For this purpose, the control unit 114 or the exposure adjustment unit 108 performs an adjustment of adding "log2(PaveN / PaveN - 1)" to the gain setting of the LV image of "FrameN".

[0060] Furthermore, the control unit 114 or the exposure adjustment unit 108 adjusts the exposure change by the total luminance change "ΔPtotalN", which is the change from "Pave0" in "Frame0". To achieve this, the control unit 114 or the exposure adjustment unit 108 adjusts the accumulation time setting of the LV image in "FrameN+1" by adding "log2(PaveN / Pave0)".

[0061] By controlling in the above manner, even if the accumulation time of the LV image is long, it is possible to sample a large amount of change in the aperture that moves during the accumulation time of the LV image. According to this embodiment, by obtaining a change in luminance information during the aperture drive and adjusting the gain setting and accumulation time setting according to the obtained amount of change, it is possible to suppress "flickering" of the LV image display until the aperture movement ends, and to achieve high tracking performance of the LV image display.

[0062] <First timing control, second timing control> In the first embodiment, the control unit 114 (first timing control unit) controls the accumulation start timing of the moving images read out in the "aperture drive amount detection mode" so that the exposure setting by the exposure adjustment unit 108 is completed by the time readout starts in the "live view display mode." Furthermore, when the accumulation time of the LV image is long, the control unit 114 may perform the following timing control.

[0063] That is, the control unit (second timing control unit) can also execute the following process when the moving images read out in the "aperture drive amount detection mode" have a faster frame rate than the moving images read out in the "live view display mode". The control unit 114 starts accumulation of the moving images read out in the "aperture drive amount detection mode" during the accumulation time of the moving images read out in the "live view display mode". Then, the control unit 114 controls the number of frames to be shot during the "aperture drive amount detection mode" period depending on the length of the accumulation time of the moving images read out in the "live view display mode".

[0064] Note that the control unit 114 (switching unit) can also execute control to switch between the first timing control unit and the second timing control unit depending on the length of accumulation time of the moving image read out in the "live view display mode".

[0065] <Summary of the First Embodiment> As described above, the imaging device 100 has an aperture control unit 103 that controls and changes the aperture amount of the photographing lens 101. The brightness detection unit 107 acquires brightness information in the moving image read from the imaging element 104. Furthermore, the brightness detection unit 107 acquires brightness information at a plurality of acquisition times including one or more acquisition times during the period in which the aperture amount is being changed and controlled. Then, the exposure adjustment unit 108 adjusts the exposure of the moving image read from the imaging element 104 based on the amount of change in the brightness information acquired by the brightness detection unit 107 at each of the plurality of acquisition times. This makes it possible to prevent "flicker" from occurring in the LV image and improve the appearance of the moving image (LV image) during the change in the aperture amount.

[0066] Specifically, the control unit 114 (first prediction unit) predicts the amount of aperture change based on the amount of change in luminance information acquired by the luminance detection unit 107 in multiple frames of the moving image read out in the "live view display mode". Then, the exposure adjustment unit 108 reflects the exposure corresponding to the predicted amount of aperture change in the readout gain setting in the "live view display mode" or in the setting of the accumulation time of the moving image read out in the first readout mode.

[0067] <Variation 1> The control unit 114 (parallel readout unit) performs readout in the "live view display mode" and readout in the "aperture drive amount detection mode" in parallel when the aperture control unit 103 starts driving. This can speed up the processing.

[0068] <Variation 2> The control unit 114 (third prediction unit) predicts the amount of change in the drive amount of the aperture control unit 103 based on the difference between the two pieces of luminance information. The first piece is the luminance information acquired by the luminance detection unit 107 at a time within a frame period during which moving images read out in the "live view display mode" were accumulated. The second piece is the luminance information acquired by the luminance detection unit 107 at a time within a frame period during which the moving images were accumulated before the aperture was driven.

[0069] Then, the exposure adjustment unit 108 performs exposure correction by reflecting the exposure corresponding to the predicted change in the drive amount of the aperture control unit 103 in the readout gain setting for the "live view display mode" or the setting of the accumulation time of the moving image read out in the "live view display mode". This exposure correction mode also makes it possible to prevent "flicker" from occurring in the LV image and improve the appearance of the moving image (LV image) during the change in the aperture amount.

[0070] <Second embodiment> Next, a second embodiment of the present invention will be described. In the first embodiment, the vertical output line 300b reads out a pixel area different from the pixel area read out by the vertical output line 300a, so that the "second image" is independent of the readout of the "first image". The readout of the "second image" corresponds to the accumulation and readout of the image signal in the "aperture drive amount detection mode". The readout of the "first image" corresponds to the accumulation and readout of the image signal in the "live view display mode".

[0071] In the second embodiment, a process is described that is performed using only the LV image read out in the "live view display mode." Below, differences from the first embodiment are mainly described, and detailed descriptions of matters similar to those in the first embodiment are omitted by using the same symbols and signs already used.

[0072] <FIG. 7: Flowchart showing the process flow of the second embodiment; FIG. 8: Timing chart showing the process of the second embodiment> The processing in the second embodiment will be described below with reference to the flowchart in Fig. 7 and the timing chart in Fig. 8. In Fig. 8, "LV0" to "LV5" correspond to "Frame0" to "Frame5", and "LV0" to "LV5" are read out in "live view display mode". In Fig. 8, the horizontal axis also indicates time, with more time passing as one moves to the right in the drawing.

[0073] First, in S701, the control unit 114 reads out image data captured by the image sensor 104 in the "live view display mode" and executes "LV display processing" (start of LV mode). The "LV display processing" is processing for sequentially displaying LV images on the display screen of the display unit 112. In S702, the "LV display processing" is repeatedly executed until the control unit 114 determines that the next frame is a frame for performing an aperture operation.

[0074] More specifically, if the control unit 114 determines that the next frame is not a frame in which the diaphragm operation is to be performed (No in S702), the process returns to S701 and enters a wait state. On the other hand, if the control unit 114 determines that the next frame is a frame in which the diaphragm operation is to be performed (Yes in S702), the process proceeds to S703. That is, in S702, the control unit 114 determines whether or not the frame of the next LV image is a frame in which the diaphragm 102 needs to be operated in response to a user operation or a change in exposure, and proceeds to a step according to the determination result.

[0075] <frame0> In "Frame 0" shown in Fig. 8, if the control unit 114 determines in S702 that an aperture operation is necessary in the next frame (Yes), the process proceeds to S703. Next, in S703, the brightness detection unit 107 or the brightness detection unit 111 starts acquiring brightness information of the LV image read out in the "live view display mode". As shown in "LV0" in Fig. 8, in "Frame 0", reading of the LV image has not yet been completed, and brightness information cannot be acquired. For this reason, the control unit 114 determines "No" in S704 and skips S705 to S707.

[0076] Next, in S708, the control unit 114 updates the "vertical synchronization signal VD" in the "LV display mode." Next, in S709, the control unit 114 communicates with the aperture control unit 103 to check whether the aperture driving has ended. If the control unit 114 determines as a result of the check that the aperture driving has not ended (No), the process proceeds to S703. In other words, the control unit 114 returns the process to S703 and moves to the timing of "Frame 1" shown in FIG. 8. On the other hand, if the control unit 114 determines as a result of the check that the aperture driving has ended (Yes), the process ends.

[0077] <frame1> In "Frame1" shown in Fig. 8, the control unit 114 executes the process in S703. As a result, the brightness detection unit 107 or the brightness detection unit 111 acquires "LV0", that is, "brightness information P0" before the aperture is driven. Since the brightness information has started to be acquired, the control unit 114 proceeds to the process of S705 as "Yes" in S704. In S705, the control unit 114 or the exposure adjustment unit 108 stores in a memory the "brightness information P0" that is the reference before the aperture starts operating.

[0078] In "Frame 1", the control unit 114 determines that luminance information after the aperture drive has not yet been acquired, and so determines "No" in S706 and skips S707. Next, in S708, the control unit 114 updates the "vertical synchronization signal VD" in the "LV display mode." Since the aperture drive has not yet ended (No in S709), the control unit 114 advances the process to S703.

[0079] <frame2> The process of S708 transitions to the timing of "Frame2" shown in Fig. 8. In "Frame2" shown in Fig. 8, the control unit 114 executes the process of S703. That is, the luminance detection unit 107 or the luminance detection unit 111 acquires "luminance information P1" of "LV1". In addition, since the luminance information was acquired in S703, the control unit 114 determines "Yes" in S704 and proceeds to S705.

[0080] Next, in S705, the control unit 114 or the exposure adjustment unit 108 stores the "brightness information P1" in the memory. Then, the control unit 114 or the exposure adjustment unit 108 predicts the aperture drive amount at the "exposure center timing" of "LV2" shown in FIG. 8 using the "brightness information P0" and the "brightness information P1". The "brightness information P0" and the "brightness information P1" are assumed to be the brightness information at "t0" and "t1", which are the "exposure center timing" of "LV0" and "LV1". The "exposure center timing" is the timing of the center of gravity of the exposure amount. As shown in FIG. 8, the intersection of the two diagonals of the parallelograms of "LV0" and "LV1" is the "center of gravity", which is the "center of gravity of the exposure amount". The times corresponding to the centers of gravity of the respective exposure amounts are "t0" and "t1". As a result, the "exposure center timing" of "LV0" and "LV1" is "t0" and "t1".

[0081] Here, the slope [a1] and intercept [b1] of the straight line “L1” connecting “P0” and “P1” shown in FIG. 8 can be expressed as follows: "Slope: a1 = (P1 - P0) / (t1 - t0 )": "Intercept: b1 = -a t1 + P1 (1) Therefore, the predicted luminance value "Pest2" at the "exposure center timing t2" of "LV2" can be expressed as the following equation (2). Pest2 = a1 t2 + b1 (2) Therefore, as explained above, in S705, the control unit 114 or the exposure adjustment unit 108 calculates the predicted brightness "Pest2."

[0082] Since the "predicted brightness" could be calculated by acquiring the brightness information after the aperture is driven, the control unit 114 determines "Yes" in S706 and proceeds to S707. In S707, the control unit 114 or the exposure adjustment unit 108 executes an exposure adjustment process to adjust the exposure by "ΔPest2" in Fig. 8. To this end, the control unit 114 adjusts and sets the gain setting of "LV2" in "Frame2" and the accumulation time setting of "LV3" in "Frame3" for the image sensor 104 by "log2(Pest2 / P0)".

[0083] Next, in S708, the control unit 114 updates the "vertical synchronization signal VD" in the "LV display mode." Then, in S709, since the aperture driving has not ended (No in S709), the control unit 114 advances the process to S703. As a result, the process moves to the timing of "Frame 3" in FIG. 8.

[0084] <frame3> In "Frame3" in FIG. 8, the control unit 114 executes the process in S703. That is, the brightness detection unit 107 or the brightness detection unit 111 acquires "brightness information P2" of "LV2". The "brightness information P2" is a value obtained by subtracting "log2(Pest2 / P0)", which is the previous exposure adjustment amount, from the brightness information actually acquired from the image. Since the brightness information was acquired in S703, the control unit 114 sets "Yes" in S704 and proceeds to S705. Next, in S705, the control unit 114 or the exposure adjustment unit 108 holds the "brightness information P2" in memory.

[0085] Then, the control unit 114 or the exposure adjustment unit 108 predicts the aperture drive amount at the "exposure center timing" of "LV3" shown in FIG. 8 using the "brightness information P1" and "brightness information P2." As shown in FIG. 8, the "exposure center timing" of "LV3" is time "t3." Let "brightness information P1" and "brightness information P2" be the luminance information at the exposure center timings "t1" and "t2" of "LV1" and "LV2." As a result, the slope "a2" and intercept "b2" of the straight line "L2" connecting P1 and P2 shown in FIG. 8 can be expressed as the following equation (3). "Slope: a2=(P2-P1) / (t2-t1)": "Intercept: b2=-a t2+P2"...(3) Therefore, equation (3) can be generalized to the following equation (4). "Tilt: a N =(P N -P N-1 ) / (t N -t N-1 )”:”Intercept:b N =-a t N +P N (4)

[0086] Further, the predicted luminance value "Pest3" at the exposure center timing "t3" of "LV3" can be expressed as in the following equation (5). Pest3 = a2 t3 + b2 (5) If we generalize this equation (5), we get equation (6). PestN+1=an t N+1 +b N (6)

[0087] In other words, the process in S705 is to predict the luminance information of the next frame by calculating the derivative (difference) between the luminance information of a certain frame and the luminance information of the frame immediately before. When the "predicted luminance" is calculated, the control unit 114 sets "Yes" in S706 and proceeds to S707. In S707, the control unit 114 or the exposure adjustment unit 108 performs exposure adjustment processing to adjust the exposure by the amount of change in luminance "ΔPest3".

[0088] That is, the control unit 114 sets the image sensor 104 to adjust the gain setting of the LV image in "Frame3" by "log2(Pest3 / Pest2)". The control unit 114 also sets the accumulation time setting of the LV image in "Frame4" to be adjusted by "log2(Pest3 / P0)" in order to adjust the exposure by the amount of change "ΔPtotal3" from "luminance information P0" to "luminance information P3".

[0089] <Frame4、Frame5> Next, in S708, the control unit 114 updates the "vertical synchronization signal VD" in the "LV display mode." Next, in S709, the control unit 114 repeatedly executes the same processing as in "Frame 3" in the subsequent "Frames 4 and 5" until it is determined that the aperture driving has ended (until S709 becomes "No"). Then, in S709, if the control unit 114 determines that the aperture driving has ended (Yes), the processing in this embodiment ends.

[0090] <When predicted brightness information exceeds the aperture target value> Finally, a process will be described for a case where predicted luminance information, such as "Pest4" shown in Fig. 8, is predicted to exceed the planned number of steps for actually driving the aperture. For example, assume that the user sets the aperture for still image shooting using the operation unit 113 and issues a shooting instruction, and it is necessary to change the aperture value from the current aperture value to the aperture value for still image shooting by a total of "X steps." Here, the notation "step" refers to the fact that the luminance change in this specification is a logarithm with the base "2," so if the "exposure" is halved, it is "minus 1 step," and if the exposure is "doubled," it is "plus 1 step."

[0091] During aperture driving, it is assumed that an adjustment of "log2(Pest4 / P0)" is required in S707 for the control unit 114 to adjust the exposure change by the amount of change in brightness "ΔPtotal4" based on the "brightness information Pest4" predicted in S705. Here, "P0" is the "brightness information" at time t0. If the result of this calculation exceeds the adjustment of "X steps", the control unit 114 does not use the predicted result "Pest4". In this case, the control unit 114 uses "P4=P0×2" instead of "Pest4". x In other words, in S707, the exposure adjustment unit 108 executes the process of S707 using the value "P0×2 x " Adjust the exposure.

[0092] That is, the control unit 114 or the exposure adjustment unit 108 sets the image sensor 104 to adjust the gain setting of the LV image in "Frame4" by "log2(P4 / Pest3)" in order to perform exposure adjustment for the amount of change in luminance "ΔPest4". Also, in order to perform exposure adjustment for the amount of change in luminance "ΔPtotal4", the control unit 114 or the exposure adjustment unit 108 sets the accumulation time setting of the LV image in "Frame5" to be adjusted by "log2(P4 / P0)".

[0093] In this way, when the predicted value of the aperture change at the exposure gravity center timing exceeds the target drive amount of the aperture control unit 103, the control unit 114 limits and controls the predicted value so as not to exceed the target drive amount. Therefore, even if the predicted luminance information exceeds the aperture target value, it is possible to maintain the quality of the LV image by executing appropriate and reasonable control.

[0094] <Summary of the second embodiment> According to the second embodiment, the luminance detection unit 107 acquires luminance information for the moving image read in the "live view display mode". More specifically, the luminance detection unit 107 further acquires luminance information at frame-by-frame intervals for the moving image read in the "live view display mode". The control unit 114 (second prediction unit) makes the next prediction based on the difference between the luminance information acquired within a period of a certain frame in the "live view display mode" and the luminance information acquired within a period of one frame before the certain frame. In other words, the control unit 114 (second prediction unit) predicts the amount of aperture change at the timing of the exposure center of gravity of the moving image accumulation in the next frame.

[0095] The exposure adjustment unit 108 performs exposure correction by reflecting the exposure corresponding to the predicted aperture change amount in the readout gain setting for the "live view display mode" or the setting of the accumulation time of the moving image read out in the "live view display mode". The second embodiment also makes it possible to prevent "flicker" from occurring in the LV image and improve the appearance of the moving image (LV image) during the change in the aperture amount.

[0096] <Additional Note> The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) An imaging device having an aperture control unit that changes and controls the aperture amount of a photographing lens, An imaging element having a plurality of pixels arranged in a matrix; A luminance detection unit that acquires luminance information in a moving image read from the imaging element, the luminance detection unit acquiring luminance information at a plurality of acquisition times including one or more acquisition times during a period in which an aperture value change control is being performed; an exposure adjustment unit that adjusts the exposure of a moving image read out from the imaging element based on an amount of change in luminance information acquired by the luminance detection unit at each of the multiple acquisition times. (Configuration 2) The imaging element includes: The imaging device described in configuration 1 is characterized in that it has a first readout mode in which a moving image is read out frame by frame, and a second readout mode in which a pixel signal different from that read out in the first readout mode during operation of the first readout mode, and the acquisition of luminance information by the luminance detection unit is performed on image data read out in the second readout mode. (Configuration 3) 3. The imaging device according to configuration 2, wherein the second readout mode is a mode for reading out a moving image while the aperture control unit is being driven. (Configuration 4) The imaging device described in configuration 2 or 3, further comprising a first prediction unit that predicts an amount of aperture change based on an amount of change in luminance information acquired by the luminance detection unit in multiple frames of a moving image read out in the first readout mode, and the exposure adjustment unit further performs exposure correction by reflecting the exposure for the amount of aperture change predicted by the first prediction unit in a readout gain setting for the first readout mode or in a setting of an accumulation time for the moving image read out in the first readout mode. (Configuration 5) The imaging device described in configuration 3 or 4, further comprising a first timing control unit that controls a timing for starting accumulation of a moving image to be read out in the second readout mode so that exposure setting by the exposure adjustment unit is completed until readout starts in the first readout mode. (Configuration 6) the video read in the second readout mode has a frame rate higher than that of the video read in the first readout mode; The imaging device described in configuration 5, further comprising a second timing control unit that starts accumulation of a moving image to be read out in the second readout mode during an accumulation time of the moving image to be read out in the first readout mode, and controls the number of frames to be captured during the second readout mode period in accordance with a length of accumulation time of the moving image to be read out in the first readout mode. (Configuration 7) The imaging device described in configuration 6, further comprising a switching unit that switches between the first timing control unit and the second timing control unit depending on the length of accumulation time of the moving image to be read out in the first readout mode. (Configuration 8) The imaging device described in configuration 6, wherein the exposure adjustment unit is further characterized in that, when capturing multiple frames during the second readout mode, the average value or median of multiple luminance information acquired by the luminance detection unit for each of the multiple frames is set as the luminance information acquired within a frame period in the first readout mode corresponding to the captured multiple frames. (Configuration 9) 9. The imaging device according to any one of configurations 2 to 8, wherein the luminance detection section further obtains luminance information for the moving image read out in the first readout mode. (Configuration 10) The imaging device described in configuration 9, characterized in that the brightness detection unit further acquires brightness information at frame-by-frame intervals of the moving image read out in the first readout mode, and further includes a second prediction unit that predicts an aperture change amount at the exposure center timing of the moving image accumulation in the next frame based on a difference between the brightness information acquired within a period of a certain frame in the first readout mode and the brightness information acquired within a period of one frame before the certain frame, and the exposure adjustment unit further performs exposure correction by reflecting the exposure for the aperture change amount predicted by the second prediction unit in the readout gain setting of the first readout mode or in the setting of the accumulation time of the moving image read out in the first readout mode. (Configuration 11) The imaging device described in configuration 10, further comprising a control unit that, when a predicted value of the aperture change at the exposure center timing exceeds a target drive amount of the aperture control unit, limits and controls the predicted value so that it does not exceed the target drive amount. (Configuration 12) 12. The imaging device according to any one of configurations 1 to 11, wherein the imaging element includes the brightness detection section and the exposure adjustment section therein. (Configuration 13) The imaging device described in any one of configurations 2 to 8, further comprising a parallel readout unit that performs readout in the first readout mode and readout in the second readout mode in parallel when the driving of the aperture control unit starts. (Configuration 14) The present invention further includes a third prediction unit that predicts a change amount of the drive amount of the aperture control unit based on a difference between luminance information acquired by the luminance detection unit at a time within a frame period in which the moving image read out in the first readout mode is accumulated and luminance information acquired by the luminance detection unit at a time within a frame period in which the moving image read out in the first readout mode is accumulated, and the exposure adjustment unit further performs exposure correction by reflecting the exposure corresponding to the change amount predicted by the third prediction unit in a readout gain setting in the first readout mode or a setting of an accumulation time for the moving image read out in the first readout mode. (method) A method for controlling an image capturing apparatus having an image capturing element having a plurality of pixels arranged in a matrix and an aperture control unit that changes and controls an aperture amount of a photographing lens, comprising: A luminance detection process for acquiring luminance information in a moving image read from the imaging element, the luminance information being acquired at a plurality of acquisition times including one or more acquisition times during a period in which control of changing the aperture amount is being performed; and an exposure adjustment process for adjusting the exposure of a moving image read out from the imaging element based on the amount of change in luminance information acquired by the luminance detection process at each of the multiple acquisition times. (program) A program for causing a computer to execute a control method for an imaging device having an imaging element having a plurality of pixels arranged in a matrix and an aperture control unit that changes and controls an aperture amount of a photographing lens, The control method includes: A luminance detection process for acquiring luminance information in a moving image read from the imaging element, the luminance information being acquired at a plurality of acquisition times including one or more acquisition times during a period in which control of changing the aperture amount is being performed; and an exposure adjustment process for adjusting the exposure of a moving image read out from the imaging element based on the amount of change in luminance information acquired by the luminance detection process at each of the multiple acquisition times.

[0097] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and various modifications and changes are possible within the scope of the gist of the present invention. For example, the present invention can be realized by supplying a program that realizes one or more functions of the above-mentioned embodiment to a system or device via a network or a recording medium, and having a computer processor of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0098] 100 Imaging device 101 Photographic Lens 102 Aperture 103 Aperture control section 104 Image sensor 105 Pixel section 106 A / D conversion section 107 Luminance detection section 111 Luminance detection section 108 Exposure adjustment section 109 Output section 110 Image processing section 111 Luminance detection section 112 Display section 113 Operation section 114 Control section 206 Pixel selection switch 300 vertical output lines 303 Vertical Scanning Circuit

Claims

1. An imaging device having an aperture control unit that changes and controls the aperture amount of a photographing lens, an imaging element having a plurality of pixels arranged in a matrix; a luminance detection unit for acquiring luminance information in a moving image read from the imaging element, the luminance information being acquired at a plurality of acquisition times including one or more acquisition times during a period in which control of changing the aperture amount is being performed; an exposure adjustment unit that adjusts the exposure of a moving image read out from the imaging element based on an amount of change in luminance information acquired by the luminance detection unit at each of the multiple acquisition times.

2. The imaging element includes: The imaging device according to claim 1, characterized in that it has a first readout mode in which a moving image is read out on a frame-by-frame basis, and a second readout mode in which a pixel signal different from that read out in the first readout mode during operation of the first readout mode, and the brightness detection unit acquires brightness information from image data read out in the second readout mode.

3. 3. The image pickup apparatus according to claim 2, wherein the second readout mode is a mode for reading out a moving image while the aperture control unit is being driven.

4. a first prediction unit that predicts an aperture change amount based on an amount of change in luminance information acquired by the luminance detection unit in a plurality of frames of the moving image read in the first read mode, The exposure adjustment unit further includes: The imaging device according to claim 2 or 3, characterized in that exposure correction is performed by reflecting the exposure corresponding to the aperture change amount predicted by the first prediction unit in a read gain setting for the first readout mode or in a setting of an accumulation time for the moving image to be read out in the first readout mode.

5. 4. The imaging device according to claim 3, further comprising a first timing control unit that controls the accumulation start timing of the moving image to be read out in the second readout mode so that exposure setting by the exposure adjustment unit is completed until readout begins in the first readout mode.

6. the video read in the second readout mode has a frame rate higher than that of the video read in the first readout mode; 6. The imaging device according to claim 5, further comprising a second timing control unit that starts accumulation of the moving image to be read out in the second readout mode during an accumulation time of the moving image to be read out in the first readout mode, and controls the number of frames to be captured during the second readout mode period in accordance with the length of the accumulation time of the moving image to be read out in the first readout mode.

7. 7. The imaging device according to claim 6, further comprising a switching unit that switches between the first timing control unit and the second timing control unit depending on the length of accumulation time of the moving image to be read out in the first readout mode.

8. The exposure adjustment unit further includes: The imaging device described in claim 6, characterized in that when multiple frames are captured during the second readout mode, the average value or median of multiple luminance information acquired by the luminance detection unit for each of the multiple frames is used as the luminance information acquired within a frame period in the first readout mode corresponding to the captured multiple frames.

9. The luminance detection unit further 4. The imaging apparatus according to claim 2, wherein luminance information for the moving image read out in the first readout mode is obtained.

10. The luminance detection unit further acquires luminance information at intervals of frames of the moving image read out in the first readout mode, a second prediction unit that predicts an aperture change amount at an exposure center timing of a moving image accumulation in a next frame based on a difference between luminance information acquired within a period of a certain frame in the first readout mode and luminance information acquired within a period of one frame before the certain frame, The exposure adjustment unit further includes: The imaging device according to claim 9, characterized in that exposure correction is performed by reflecting the exposure corresponding to the aperture change amount predicted by the second prediction unit in a read gain setting for the first readout mode or in a setting of an accumulation time for the moving image to be read out in the first readout mode.

11. 11. The imaging device according to claim 10, further comprising a control unit that, when a predicted value of the aperture change at the exposure center timing exceeds a target drive amount of the aperture control unit, limits and controls the predicted value so that it does not exceed the target drive amount.

12. 4. The imaging apparatus according to claim 1, wherein the imaging element includes the brightness detection section and the exposure adjustment section therein.

13. 4. The imaging apparatus according to claim 2, further comprising a parallel readout unit that performs readout in the first readout mode and readout in the second readout mode in parallel when the driving of the aperture control unit is started.

14. a third prediction unit that predicts a change amount of a drive amount of the aperture control unit based on a difference between luminance information acquired by the luminance detection unit at a time within a frame period during which a moving image read out in the first readout mode is accumulated and luminance information acquired by the luminance detection unit at a time within a frame period during which the moving image is accumulated before the aperture is driven, The exposure adjustment unit further includes: An imaging device as described in any one of claims 2, 3, 6, 7 and 8, characterized in that exposure correction is performed by reflecting the exposure change amount predicted by the third prediction unit in the read gain setting of the first readout mode or in the setting of the accumulation time of the moving image read out in the first readout mode.

15. A method for controlling an imaging device having an imaging element having a plurality of pixels arranged in a matrix and an aperture control unit that changes and controls an aperture amount of a photographing lens, comprising: a luminance detection step for acquiring luminance information in a moving image read from the imaging element, the luminance information being acquired at a plurality of acquisition times including one or more acquisition times during a period in which control of changing the aperture amount is being performed; an exposure adjustment step of adjusting exposure of a moving image read out from the imaging element based on an amount of change in luminance information acquired by the luminance detection step at each of the multiple acquisition times.

16. A program for causing a computer to execute a control method for an imaging device having an imaging element having a plurality of pixels arranged in a matrix and an aperture control unit that changes and controls an aperture amount of a photographing lens, The control method includes: a luminance detection step for acquiring luminance information in a moving image read from the imaging element, the luminance information being acquired at a plurality of acquisition times including one or more acquisition times during a period in which control of changing the aperture amount is being performed; and an exposure adjustment step of adjusting exposure of a moving image read out from the imaging element based on an amount of change in luminance information acquired by the luminance detection step at each of the plurality of acquisition times.

Images