Imaging device, control method, and control program

Abstract

Provided are an imaging device, a control method, and a control program capable of reducing PLS noise while suppressing the effects thereof on other functions.　The imaging device comprises: an electronic ND filter (13) capable of controlling the transmittance of light incident through an imaging lens (11); an imaging element (14) having a plurality of pixels; and a system control unit (16). The imaging element (14) can perform simultaneous readout control for simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge holding unit (52). The system control unit (16) performs transmittance decrease control (dimming control = ON) for decreasing the transmittance of the electronic ND filter (13) after the simultaneous readout control, and performs transmittance increase control (dimming control = OFF) for increasing the transmittance of the electronic ND filter (13) during a readout (43) period in which the charges read out to the charge holding unit (52) are read out as an image signal.

Description

Imaging device, control method, and control program

[0001] The present invention relates to an imaging device, a control method, and a control program.

[0002] Patent Document 1 describes a camera system that includes an aperture mechanism as a means for changing the amount of incident light, and that controls the aperture mechanism to an aperture value for normal photography during an exposure period in which the exposure outputs of all photodiodes (PDs) are simultaneously transferred to a charge storage section (FD) by the global shutter (GS) operation of the MOS-type image sensor, and sets a high aperture value during a sequential signal readout period after exposure to suppress incident light on the MOS-type image sensor (the generation of false signals due to leaked light).

[0003] Patent Document 2 describes an imaging device that includes a liquid crystal panel whose transmittance for light from a subject is variable depending on the applied voltage value, and that functions as an ND (Neutral Density) filter that adjusts the amount of light incident on the imaging element, and that is driven in a normal drive mode or a high-speed drive mode, and in the high-speed drive mode, applies to the liquid crystal panel a voltage value higher than the voltage corresponding to the saturation value of the density (transmittance) (overdrive drive).

[0004] Patent document 3 describes an imaging device that has a CMOS image sensor that can operate using a global shutter system, is capable of recording both moving images and still images as shooting modes, and has a liquid crystal ND filter in front of the imaging element (on the lens side) that can attenuate the amount of light by a predetermined amount using a drive signal from a drive circuit, and is capable of adjusting the amount of light in conjunction with an aperture (in the lens).

[0005] Patent Document 4 describes a solid-state imaging device in which unit pixels are used in a global shutter type solid-state imaging element, and which is switchable between two operation modes: a first mode which is a normal imaging mode, and a second mode which detects only a smear detection signal based on noise charge of a parasitic light component generated in a memory section (MEM) which holds charge in a photodiode; the arrangement and ratio of pixels for the first mode and pixels for the second mode can be changed; and all pixels can be used in the first mode if a high light intensity region is not detected.

[0006] Japanese Patent Publication No. 2008-028516 Japanese Patent Publication No. 2021-002772 Japanese Patent Publication No. 2017-130856 Japanese Patent Publication No. 2021-125716

[0007] One embodiment of the technique of the present disclosure provides an imaging device, a control method, and a control program that can reduce PLS noise while suppressing the impact on other functions.

[0008] (1) An imaging device comprising a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of a first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge holding section, and the processor performs a first transmittance control of lowering the transmittance of the light control device after the first control, and performs a second transmittance control of increasing the transmittance of the light control device during a readout period in which the charges read out to the charge holding section are read out as an image signal.

[0009] (2) The imaging device according to (1), in which the processor performs the first transmittance control immediately after the first control.

[0010] (3) The imaging device according to (1) or (2), in which the processor performs the second transmittance control before exposure of the plurality of pixels starts during the readout period.

[0011] (4) The imaging device according to any one of (1) to (3), wherein the processor determines the timing of performing the second transmittance control based on a shutter speed for imaging.

[0012] (5) The imaging device according to any one of (1) to (4), wherein the processor determines a set value of the transmittance of the light control device in the first transmittance control based on a shutter speed for imaging.

[0013] (6) The imaging device according to any one of (1) to (5), wherein the light control device is insertable into and removable from an optical path of the incident light.

[0014] (7) The imaging device according to any one of (1) to (6), wherein the processor determines whether or not to execute the first transmittance control and the second transmittance control based on the image signal read out from the charge holding unit.

[0015] (8) The imaging device according to (7), wherein the processor executes the first transmittance control and the second transmittance control when the image signal includes a pixel whose brightness is equal to or greater than a threshold value.

[0016] (9) The imaging device according to any one of (1) to (8), wherein the processor starts the first transmittance control based on a readout signal that instructs the charge storage unit to read out.

[0017] (10) The imaging device according to any one of (1) to (9), wherein the processor, when not capturing an image, controls the light control device to move away from the optical path of the incident light or to set the transmittance of the light control device to a predetermined value.

[0018] (11) The imaging device according to any one of (1) to (10), wherein the processor, in the second transmittance control, increases the transmittance of the light control device to a transmittance value determined based on a user setting.

[0019] (12) The imaging device according to any one of (1) to (11), wherein the processor determines a timing for performing the second transmittance control based on the image signal read out from the charge holding unit.

[0020] (13) The imaging device according to (12), wherein the processor determines the timing of performing the second transmittance control based on a position of a pixel on the image represented by the image signal whose brightness is equal to or greater than a threshold value.

[0021] (14) The imaging device according to any one of (1) to (13), wherein the processor determines a timing for performing the second transmittance control based on a set value of the transmittance of the light control device in the first transmittance control.

[0022] (15) The imaging device according to any one of (1) to (14), wherein the imaging element is capable of resetting only the plurality of pixels among the plurality of pixels and the charge holding section.

[0023] (16) An imaging device including a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge holding unit, and the processor performs first transmittance control of lowering the transmittance of the light control device after the first control, and performs second transmittance control of raising the transmittance of the light control device before exposure of the plurality of pixels begins.

[0024] (17) A control method for an imaging device including a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of a first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge holding section, and the processor performs a first transmittance control of lowering the transmittance of the light control device after the first control, and performs a second transmittance control of increasing the transmittance of the light control device during a readout period in which the charges read out to the charge holding section are read out as an image signal.

[0025] (18) A control method for an imaging device including a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of a first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge holding section, and the processor performs a first transmittance control of lowering the transmittance of the light control device after the first control, and performs a second transmittance control of raising the transmittance of the light control device before exposure of the plurality of pixels begins.

[0026] (19) A control program for an imaging device including a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of a first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge holding section, and the control program causes the processor to execute the following processes: after the first control, a first transmittance control of lowering the transmittance of the light control device; and during a readout period in which the charges read out to the charge holding section are read out as an image signal, a second transmittance control of increasing the transmittance of the light control device.

[0027] (20) A control program for an imaging device including a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of a first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge holding unit, and the control program causes the processor to execute a process of: after the first control, performing a first transmittance control of lowering the transmittance of the light control device; and before exposure of the plurality of pixels begins, performing a second transmittance control of increasing the transmittance of the light control device.

[0028] According to the present invention, it is possible to provide an imaging device, a control method, and a control program that can reduce PLS noise while suppressing the influence on other functions.

[0029] 1 is a diagram showing the configuration of an imaging device 10 of the present embodiment; FIG. 2 is a cross-sectional view showing the configuration of an electronic ND filter 13; FIG. 3 is a diagram showing a first example of a control sequence of a system control unit 16 when capturing a still image; FIG. 4 is a diagram showing a second example of a control sequence of a system control unit 16 when capturing a still image; FIG. 5 is a diagram showing an example of a control sequence of a system control unit 16 when capturing a moving image; FIG. 6 is a diagram showing the configuration of pixels of an imaging element 14; and FIG. 7 is a diagram showing an example of a mechanism for replacing the electronic ND filter 13.

[0030] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0031] <Imaging Device> Fig. 1 is a diagram showing the configuration of an imaging device 10 according to this embodiment. As shown in Fig. 1, the imaging device 10 includes an imaging lens 11, an aperture 12, an electronic ND filter 13, an imaging element 14, a digital signal processing unit 15, and a system control unit 16. The imaging device 10 also includes an EVF (Electronic Viewfinder) 17, a display LCD (Liquid Crystal Display) 18, a card storage unit 19, and an operation unit 20. The imaging device 10 is, for example, a digital still camera or a digital video camera capable of capturing moving images.

[0032] The imaging lens 11 is a lens that forms an image of incident light on the imaging element 14. The imaging lens 11 includes a focus lens that is movable in the optical axis direction. The focus lens is a lens for adjusting the focus of the imaging optical system that includes the imaging lens 11 and the aperture 12, and is composed of a single lens or multiple lenses. The imaging lens 11 moves the focus lens in the optical axis direction based on an imaging control signal transmitted from the system control unit 16, thereby changing the position of the principal point of the focus lens along the optical axis and changing the focal position on the subject side.

[0033] The diaphragm 12 is capable of mechanically adjusting the amount of light incident thereon, and changes the aperture size (F-number) based on a drive control signal sent from the system control unit 16.

[0034] The electronic ND (Neutral Density) filter 13 is an example of the "light control device" of the present invention. The electronic ND filter 13 can electrically control the transmittance of light incident through the imaging lens 11. The higher the filter density of the electronic ND filter 13, the lower the transmittance. The electronic ND filter 13 is a guest-host type electronic variable ND filter constructed using, for example, a guest-host (GH) type liquid crystal containing a dye (dichroic dye). Because the dichroic dye has different light absorption levels depending on the axial direction, when a voltage is applied to the liquid crystal, the liquid crystal molecules change their alignment in response to the voltage, following the movement of the liquid crystal and creating a transmissive state and an absorbing (light-blocking) state. The electronic ND filter 13 is a filter that adjusts only the amount of light without affecting color. The electronic ND filter 13 creates a transmissive state and a light-blocking state based on an imaging control signal transmitted from the system control unit 16. The electronic ND filter 13 is configured to be insertable into and removable from the optical path of light that has passed through the imaging lens 11 and is incident thereon.

[0035] The image sensor 14 captures an image of a subject through an imaging optical system including an imaging lens 11, an aperture 12, and an electronic ND filter 13. The image sensor 14 captures an image of the subject based on an imaging control signal transmitted from a system control unit 16. The image sensor 14 has a light-receiving surface on which a plurality of pixels are arranged two-dimensionally, and converts a subject image formed on the light-receiving surface by the imaging optical system into pixel signals using the plurality of pixels and outputs the pixel signals. The image sensor 14 captures an image of the target subject at a predetermined frame rate. The "predetermined frame rate" refers to, for example, several tens to several hundreds of frames per second. The image sensor 14 is configured, for example, by a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The image sensor 14 has, for each of the multiple pixels, a photodiode, for example, that generates and accumulates charge according to the amount of light received, and a charge storage unit from which the charge accumulated in the photodiode is read. The image sensor 14 is capable of simultaneous readout control, which simultaneously exposes multiple pixels and reads the charge accumulated in the photodiodes of the multiple pixels into the charge storage unit. "Simultaneous exposure of multiple pixels" means that all pixels are exposed with a very short time difference, meaning that there is a time during which all pixels are exposed. However, due to the characteristics of the electronic circuit and control, there may be a minute time difference on the nanosecond or picosecond level. "Reading out the charge accumulated in multiple pixels into the charge storage unit" means that the charge accumulated in multiple pixels is simultaneously read out to the charge storage unit for all pixels. The image sensor 14 is a global shutter type image sensor that simultaneously exposes all pixels. Simultaneous readout control is the "first control" of this invention.

[0037] The digital signal processing unit 15 processes the image signal output from the imaging element 14 to generate captured image data suitable for display on the display LCD 18 and suitable for storage in the card storage unit 19. The digital signal processing unit 15 generates captured image data by performing signal processing such as demosaic processing and gamma correction processing on the captured image data output from the imaging element 14. The digital signal processing unit 15 performs each signal processing based on an imaging control signal transmitted from the system control unit 16.

[0038] The system control unit 16 controls the entire imaging device 10. The hardware structure of the system control unit 16 is made up of various processors that execute programs and perform various processes. These various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes programs and performs various processes; a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture; and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically to perform specific processing. More specifically, the structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor devices. The system control unit 16 is an example of a "processor" in the present invention.

[0039] The system control unit 16 may be configured with one of various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0040] The system control unit 16 drives the imaging lens 11, the aperture 12, the electronic ND filter 13, and the imaging element 14, and outputs the subject image captured through the imaging optical system including the imaging lens 11, the aperture 12, and the electronic ND filter 13 as an image signal from the imaging element 14 to the digital signal processing unit 15.

[0041] Specifically, after simultaneous readout control in which charges of multiple pixels accumulated by simultaneous exposure are read out to the charge storage unit, the system control unit 16 performs transmittance decrease control to decrease the transmittance of the electronic ND filter 13, and then performs transmittance increase control to increase the transmittance of the electronic ND filter 13 during the readout period in which the charges read out to the charge storage unit by simultaneous readout control are read out as an image signal. The transmittance decrease control corresponds to the "first transmittance control" in this invention. The transmittance increase control corresponds to the "second transmittance control" in this invention.

[0042] "Lowering the transmittance" means lowering the transmittance compared to immediately before the transmittance decrease control is performed. "Increasing the transmittance" means raising the transmittance compared to immediately before the transmittance increase control is performed, for example, returning the transmittance to the transmittance before the transmittance decrease control is performed. "A readout period in which the charges read out to the charge storage section are read out as an image signal" refers to a period in which each pixel is read out sequentially, and is a period in which PLS (Parasitic Light Sensitivity) noise may occur. PLS noise is noise that occurs when light leaks into the charge storage section and degrades the image.

[0043] The system control unit 16 performs the transmittance lowering control, for example, immediately after simultaneous readout control. "Immediately after simultaneous readout control" refers to immediately after the readout (global transfer) of charges accumulated in multiple pixels simultaneously read out to the charge holding unit for all pixels is completed. Also, "immediately after simultaneous readout control" refers to the timing at which the charges read out to the charge holding unit by simultaneous readout control start being read out from the charge holding unit as an image signal.

[0044] During a readout period in which the charges read out to the charge storage portion are sequentially read out as image signals, the system control unit 16 performs transmittance increase control before the start of exposure of multiple pixels. The system control unit 16 performs transmittance increase control before the start of the next exposure during the readout period after simultaneous exposure. The next exposure refers to the exposure for live view immediately after capture or the exposure for auto control in the case of a still image, and the exposure for capturing the next frame in the case of a video. "Before the start of the next exposure" means, for example, to be performed in time for the next simultaneous exposure, taking into account the transmittance increase time constant of the electronic ND filter 13 in the transmittance increase control. Performing transmittance increase control means turning off the dimming control, i.e., increasing the transmittance of the electronic ND filter 13.

[0045] The system control unit 16 determines the timing of executing transmittance increase control, which increases the transmittance of the electronic ND filter 13, based on the shutter speed for imaging. The system control unit 16 determines the timing of executing transmittance increase control, i.e., the timing of turning off dimming control, based on the shutter speed. The system control unit 16 determines the timing of executing transmittance increase control so as to be in time for the next exposure. For example, in still image imaging, if a slow shutter speed is set for the main exposure, the shutter speed for the live view exposure or the auto control exposure may also be controlled to be slow. In this case, the transmittance increase control needs to be executed early, taking into account the time constant of the transmittance increase control, in order to be in time for the live view exposure or the auto control exposure. Furthermore, in video imaging, if a slow shutter speed is set for the main exposure, the transmittance increase control needs to be executed early, taking into account the time constant of the transmittance increase control, in order to be in time for the exposure of the next frame. Therefore, the system control unit 16 executes the transmittance increase control earlier, for example, the slower the shutter speed.

[0046] The system control unit 16 determines the transmittance setting value of the electronic ND filter 13 in the transmittance reduction control, which reduces the transmittance of the electronic ND filter 13, based on the shutter speed for imaging. The "transmittance setting value of the electronic ND filter 13" refers to a target value of the transmittance, such as the voltage applied to the electronic ND filter 13. For example, if a slow shutter speed is set for the main exposure, the start timing of the exposure of the next frame is earlier as described above. Therefore, even if the transmittance reduction control is performed, the transmittance may not be fully reduced to the set value by the start timing of the transmittance increase control due to the time constant of the transmittance reduction control. For this reason, the system control unit 16 sets a higher transmittance setting value (target value) to be reduced by the transmittance reduction control as the shutter speed becomes slower, thereby reducing the amount of transmittance reduction. This reduces the time required to complete the transmittance reduction control, even when a slow shutter speed is set for the main exposure.

[0047] The system control unit 16 determines whether to perform transmittance decrease control and transmittance increase control based on the image signal read out from the charge storage unit. The system control unit 16 performs transmittance decrease control and transmittance increase control when the image signal read out from the charge storage unit includes pixels with brightness equal to or greater than a threshold value. For example, when the image signal does not contain bright spots (pixels with brightness equal to or greater than a threshold value), the system control unit 16 does not perform transmittance decrease control because PLS noise is unlikely to occur. As a result, the system control unit 16 also does not perform transmittance increase control. This improves flexibility in the start timing of the next exposure.

[0048] The system control unit 16 initiates transmittance reduction control based on a read signal that instructs readout to the charge holding unit in simultaneous readout control in the image sensor 14. The "readout signal" is a signal that specifies the timing of global transfer. The system control unit 16 acquires the readout signal from the image sensor 14 and performs transmittance reduction control to reduce the transmittance of the electronic ND filter 13 in synchronization with the readout signal. When it is time to read out charges to the charge holding unit, the system control unit 16 immediately controls the filter density of the electronic ND filter 13 to maximize (all black). By synchronizing the readout timing of reading out charges to the charge holding unit and the transmittance reduction control timing to reduce the transmittance of the electronic ND filter 13 and linking the two processes, processing time is shortened.

[0049] The system control unit 16 controls the electronic ND filter 13 to retract from the optical path of light incident through the imaging lens or to set the transmittance of the electronic ND filter 13 to a predetermined value when not capturing images. "When not capturing images" refers, for example, to when the power is off or during playback. The "predetermined value" is, for example, the highest transmittance (e.g., approximately 100%). Note that the system control unit 16 may both retract the electronic ND filter 13 and control the transmittance of the electronic ND filter 13. Retracting the electronic ND filter 13 suppresses fading of the electronic ND filter 13 due to exposure to ultraviolet light. Furthermore, maintaining a high transmittance of the electronic ND filter 13 suppresses fading.

[0050] In transmittance increase control for increasing the transmittance of the electronic ND filter 13, the system control unit 16 increases the transmittance of the electronic ND filter 13 to a transmittance value determined based on a user setting. The "user setting" is, for example, the transmittance originally set by the user before the transmittance decrease control. In the transmittance decrease control, the transmittance is set to, for example, the lowest value. This allows imaging to be performed at the ND density desired by the user.

[0051] The system control unit 16 determines the timing to perform transmittance increase control based on the image signal read out from the charge storage unit. The system control unit 16 determines the timing to perform transmittance increase control based on the position of pixels whose brightness on the image represented by the image signal is equal to or greater than a threshold. The system control unit 16 identifies high-brightness areas based on the image signal, and starts transmittance increase control when readout of the high-brightness areas (areas where PLS occurs) is completed. The system control unit 16 may also start the next exposure accordingly. This improves the degree of freedom in determining the start timing of the next exposure.

[0052] The system control unit 16 determines the timing to perform the transmittance increase control based on the transmittance setting value of the electronic ND filter 13 in the transmittance decrease control. The system control unit 16 advances the timing to perform the transmittance increase control because the lower the transmittance setting value in the transmittance decrease control (the higher the density), the longer it takes to return the transmittance to its original state. The transmittance setting value in the transmittance decrease control is, for example, the minimum transmittance value. However, the transmittance setting value in the transmittance decrease control may change depending on the shutter speed and user settings.

[0053] The EVF 17 is an electronic viewfinder that enables viewing of images converted by the digital signal processing unit 15. The EVF 17 allows viewing of images displayed on the display LCD 18 through the viewfinder. The display LCD 18 is configured, for example, with an organic EL (electroluminescence) panel or a liquid crystal panel. The card storage unit 19 is where an SD memory card is inserted. The operation unit 20 receives instruction signals from the user. The operation unit 20 includes a touch panel integrated with the display LCD 18, various buttons, etc.

[0054] <Electronic ND Filter> Fig. 2 is a cross-sectional view showing the configuration of the electronic ND filter 13. As shown in Fig. 2, the electronic ND filter 13 includes transparent substrates 31a and 31b, transparent electrodes 32a and 32b, alignment films 33a and 33b, and sealing members 34a and 34b. The direction indicated by arrow A in Fig. 2 is the direction in which light incident on the electronic ND filter 13 travels.

[0055] A transparent substrate 31 a, a transparent electrode 32 a, and an alignment film 33 a are stacked in this order on the light incident side. A transparent substrate 31 b, a transparent electrode 32 b, and an alignment film 33 b are stacked in this order on the light exit side. A liquid crystal layer 35 containing liquid crystal molecules and a dye (dichroic dye) is provided between the alignment films 33 a and 33 b.

[0056] The transparent substrates 31a and 31b support the transparent electrodes 32a and 32b and the alignment films 33a and 33b, and also seal the liquid crystal layer 35. The transparent substrates 31a and 31b are made of, for example, glass substrates.

[0057] The transparent electrodes 32a and 32b are electrodes for applying a control voltage to the liquid crystal layer 35. The transparent electrodes 32a and 32b are made of, for example, indium tin oxide.

[0058] The alignment films 33a and 33b are films that align the liquid crystal molecules in a specific direction in the liquid crystal layer 35. The alignment films 33a and 33b are made of a polymer material such as polyimide.

[0059] The sealing members 34a and 34b are members that seal the liquid crystal molecules and dye molecules from leaking from the edges of the liquid crystal layer 35. The sealing members 34a and 34b are made of adhesive such as epoxy or acrylic.

[0060] In this example, a single liquid crystal cell having such a configuration is shown, but the present invention is not limited to this. For example, a plurality of liquid crystal cells may be stacked with different light distribution directions of the alignment films.

[0061] <Control Sequence During Imaging> FIG. 3 is a diagram showing a first example of a control sequence of the system control unit 16 during still image capturing.

[0062] In FIG. 3 , "dimming control" refers to controlling the transmittance (filter density) of the electronic ND filter 13. "Dimming control: ON" refers to controlling the transmittance of the electronic ND filter 13 to the lowest transmittance. The lowest transmittance means setting the filter density to the highest density. "Dimming control: OFF" refers to controlling the transmittance of the electronic ND filter 13 to the highest transmittance. The highest transmittance means controlling the filter density to the lowest density. "Main exposure" refers to the exposure used when capturing a still image. "Live view exposure" refers to the exposure used for live view immediately after capturing a still image. "Readout" refers to sequentially reading out the charges read into the charge storage unit during the main exposure from the charge storage unit as an image signal. "GR" refers to erasing the charges accumulated in the photodiodes of each pixel. "GS" refers to simultaneous readout (global transfer) in which the charges of multiple pixels accumulated during the main exposure are simultaneously read out to the charge storage unit for all pixels.

[0063] In the live view 47 state in which an image of a subject is being captured while being displayed on the display LCD 18, it is assumed that the user presses, for example, the shutter button at time t1 to capture a still image.

[0064] When the shutter button is pressed, the system control unit 16 causes the image sensor 14 to perform GR to erase the charge accumulated in the photodiode of each pixel. Once GR is complete, the system control unit 16 causes simultaneous exposure of multiple pixels to be performed by main exposure 42. At time t2, the system control unit 16 causes simultaneous readout control (global transfer: GS) to read out the charge accumulated in the photodiodes of multiple pixels by main exposure 42 to the charge storage unit for all pixels simultaneously.

[0065] Immediately after GS is completed, the system control unit 16 turns on the dimming control signal 41 to perform transmittance reduction control to reduce the transmittance of the electronic ND filter 13 to, for example, the minimum transmittance. Also, upon completion of GS, the system control unit 16 starts readout 43, which sequentially reads out the charges read out to the charge holding unit in main exposure 42 as image signals for each pixel.

[0066] When capturing a still image is completed and the system returns to live view 47, the system control unit 16 executes live view exposure 44 for the first frame image after capture during readout 43, which sequentially reads out the charge from main exposure 42. In this first example, the system control unit 16 executes live view exposure 44 at time t3 during readout 43. The timing at which live view exposure 44 is executed is determined based on, for example, the shutter speed for capturing the image. Before starting live view exposure 44, the system control unit 16 executes GR on the image sensor 14 to erase the charge accumulated in the photodiode of each pixel during main exposure 42.

[0067] Before the start of the live view exposure 44, the system control unit 16 turns off the control signal 41 for light reduction control, and performs transmittance increase control to maximize the transmittance of the electronic ND filter 13. The timing for turning off the control signal 41 is determined so as to be in time for the start time t3 of the live view exposure 44, which is determined based on the shutter speed of the image capture. In this case, the system control unit 16 may determine the timing for turning off the control signal 41 taking into consideration the time constant of the transmittance increase control.

[0068] When the live view exposure 44 is completed, the system control unit 16 executes simultaneous readout control (global transfer: GS) at time t4 to read out the charges accumulated in the plurality of pixels to the charge holding unit.

[0069] 3, when GS at time t4 is completed, the system control unit 16 starts readout, in which the charges read out to the charge holding unit during live view exposure 44 from time t3 to time t4 are sequentially read out as image signals for each pixel during live view 47 after time t4. Furthermore, the system control unit 16 executes GR, which erases the charges accumulated in the photodiodes of each pixel during live view exposure 44, and then executes live view exposure for the next frame.

[0070] 4 is a diagram showing a second example of the control sequence of the system control unit 16 when capturing a still image. This second example differs from the first example in that the shutter speed for the main exposure is set to a shutter speed faster than the shutter speed for the main exposure in the first example shown in FIG.

[0071] For this reason, in the second example, the period from time t1 to time t2 of the main exposure 42 is shorter than the period from time t1 to time t2 of the main exposure 42 in the first example shown in FIG. Therefore, the period from time t3 to time t4 of the live view exposure 44 determined based on the main exposure 42 is also shorter than the period from time t3 to time t4 of the live view exposure 44 in the first example shown in FIG. As a result, the period from time t2 to time t3 during which the control signal 41 for dimming control is turned ON to perform transmittance decrease control to lower the transmittance of the electronic ND filter 13 is longer than the period from time t2 to time t3 during which the transmittance decrease control is performed in the first example shown in FIG. That is, during the period (time t2 to time t4) of readout 43 in which the charges of the main exposure 42 are sequentially read out, the period during which the image is dark is longer in the second example than in the first example shown in FIG.

[0072] FIG. 5 is a diagram showing an example of a control sequence of the system control unit 16 when capturing a moving image.

[0073] In the case of capturing a moving image, for example, in frame a, the system control unit 16 executes simultaneous exposure of a plurality of pixels by the exposure 45a. At time t11, the system control unit 16 executes simultaneous readout control (global transfer: GS) to simultaneously read out the charges accumulated in the photodiodes of the plurality of pixels by the exposure 45a to the charge storage units of all the pixels.

[0074] Immediately after GS is completed, the system control unit 16 turns on the dimming control signal 41 to perform transmittance reduction control, thereby reducing the transmittance of the electronic ND filter 13 to, for example, the minimum transmittance. Furthermore, upon completion of GS, the system control unit 16 executes readout 46a, which sequentially reads out the charges read into the charge storage unit during exposure 45a of frame a as image signals for each pixel, from time t11 to time t13 of the next frame b. However, readout 46a does not necessarily have to end at time t13, and may end earlier than time t13.

[0075] The system control unit 16 starts exposure 45b for frame b at time t12 during readout 46a, in which the charges from exposure 45a for frame a are sequentially read out as image signals. The start timing of exposure 45b for frame b is determined based on, for example, the shutter speed for video capture. Before starting exposure 45b for frame b, the system control unit 16 causes the image sensor 14 to execute GR to erase the charges accumulated in the photodiodes of each pixel during exposure 45a for frame a.

[0076] Before the start of exposure 45b for frame b, the system control unit 16 turns off the control signal 41 for dimming control, thereby performing transmittance increase control to maximize the transmittance of the electronic ND filter 13. The timing for turning off the control signal 41 is determined so as to be in time for time t12, when exposure 45b for frame b starts. In this case, the system control unit 16 determines the timing for turning off the control signal 41, taking into consideration the time constant of the transmittance increase control.

[0077] Similar control is performed for the c frame following the b frame. That is, the system control unit 16 turns on the dimming control signal 41 at the start of readout 46b (time t13) in which the charges of the b frame exposure 45b are sequentially read out as an image signal, and turns off the dimming control signal 41 at the start of the c frame exposure 45c during readout 46b (time t14). Thereafter, the system control unit 16 repeats similar control during video capture.

[0078] For example, when capturing video at 60 fps, if the sequential readout period for exposure of each frame is set to 1 / 60 sec and the electronic shutter is released so that the exposure period of each frame is 1 / 125 sec, the period during which the dimming control is ON will be 1 / 125 sec, resulting in a sequence such as that shown in Figure 5.

[0079] When the exposure time changes in accordance with a change in the shutter speed, the ON and OFF periods of the dimming control also change accordingly.

[0080] As described above, the imaging device 10 of this embodiment is capable of simultaneous readout control in which a plurality of pixels are exposed simultaneously and the charges accumulated in the plurality of pixels by this simultaneous exposure are simultaneously read out to the charge holding unit for all pixels, and after the simultaneous readout control, transmittance decrease control is performed to decrease the transmittance of the electronic ND filter 13, and during the readout period in which the charges read out to the charge holding unit by the simultaneous readout control are sequentially read out as image signals for each pixel, transmittance increase control is performed to increase the transmittance of the electronic ND filter 13. With this configuration, the transmittance of the electronic ND filter 13 can be decreased during the period from the transmittance decrease control to the transmittance increase control during the readout period in which the charges in the charge holding unit are sequentially read out as image signals, thereby suppressing the occurrence of PLS ​​noise during that period.

[0081] Furthermore, according to the imaging device 10, immediately after the completion of simultaneous readout control for reading out the charges accumulated in a plurality of pixels to the charge holding unit, transmittance decrease control is performed to decrease the transmittance of the electronic ND filter 13. This makes it possible to suppress the occurrence of PLS ​​noise from the beginning of the readout period in which the charges in the charge holding unit are sequentially read out as an image signal.

[0082] Furthermore, according to the imaging device 10, during a readout period in which the charges read out to the charge storage unit are sequentially read out as image signals, transmittance increase control is performed to increase the transmittance of the electronic ND filter 13 that has undergone transmittance decrease control before the next exposure of multiple pixels is initiated. This configuration allows exposure for live view immediately after capture of a still image, or exposure for capturing the next frame in the case of a video, to be performed without being interrupted by the transmittance decrease control of the electronic ND filter 13. This prevents the screen from darkening when returning to live view from capture of a still image, or the screen of the next frame from capture of a video. Furthermore, it is possible to suppress the impact on auto controls such as autofocus and autoexposure.

[0083] <Pixels of the image sensor 14> Fig. 6 is a diagram showing the configuration of a pixel of the image sensor 14. As shown in Fig. 6, a pixel 50 of the image sensor 14 includes a photoelectric conversion unit 51, a charge holding unit 52, a floating diffusion 53, an initialization transistor 54, transfer transistors 55 and 56, a reset transistor 57, an output transistor 58, and a selection transistor 59.

[0084] The photoelectric conversion unit 51 receives light that has passed through an imaging optical system including the imaging lens 11, the aperture 12, and the electronic ND filter 13, and generates and accumulates electric charges according to the amount of received light. The photoelectric conversion unit 51 is composed of a photodiode (PD) or the like.

[0085] The charge holding unit 52 holds the charge transferred from the photoelectric conversion unit 51 by the transfer transistor 55 .

[0086] The floating diffusion 53 converts the charge into a signal, and the charge held in the charge holding section 52 is transferred by a transfer transistor 56 .

[0087] A constant voltage (for example, ground voltage) is supplied to one end (anode side) of the photoelectric conversion unit 51 , one end of the charge holding unit 52 , and one end of the floating diffusion 53 .

[0088] The initialization transistor 54 is a transistor that applies a predetermined voltage (VDD) to the photoelectric conversion unit 51 in order to erase the charge accumulated in the photoelectric conversion unit 51. The initialization transistor 54 is connected between the power supply line (VDD) and the other end (cathode side) of the photoelectric conversion unit 51.

[0089] The transfer transistor 55 is a transistor for transferring the charges generated in the photoelectric conversion unit 51 to the charge holding unit 52. The transfer transistor 55 is connected between the other end (cathode side) of the photoelectric conversion unit 51 and the other end of the charge holding unit 52.

[0090] The transfer transistor 56 is a transistor for transferring the charge in the charge holding portion 52 to the floating diffusion 53. The transfer transistor 56 is connected between the other end of the charge holding portion 52 and the other end of the floating diffusion 53.

[0091] The reset transistor 57 is a transistor that applies a predetermined voltage (VRST) to the floating diffusion 53 in order to reset the potential of the floating diffusion 53. The reset transistor 57 is connected between the power supply line (VRST) and the other end of the floating diffusion 53.

[0092] The output transistor 58 is a transistor for converting the potential of the floating diffusion 53 into a pixel signal and outputting the pixel signal. The gate electrode of the output transistor 58 is connected to the other end of the floating diffusion 53.

[0093] The selection transistor 59 is a transistor for selectively reading out the pixel signal output from the output transistor 58 to a signal line 60. The selection transistor 59 is connected between the output transistor 58 and the signal line 60.

[0094] The image sensor 14 is capable of resetting (erasing) only the charge accumulated in the photoelectric conversion unit 51 of each pixel 50, among the charge accumulated in the photoelectric conversion unit 51 and the charge held in the charge holding unit 52. Before starting simultaneous exposure of a plurality of pixels, the image sensor 14 erases only the charge accumulated in the photoelectric conversion unit 51 using the initialization transistor 54. This allows simultaneous exposure of a plurality of pixels with the same exposure time.

[0095] <Mechanism of the Electronic ND Filter 13> FIG. 7 is a diagram showing an example of a switching mechanism for the electronic ND filter 13. The electronic ND filter 13 is insertable into and removable from the optical path of light incident through the imaging lens 11. As shown in FIG. 7 , the imaging device 10 includes a switching mechanism that can switch, for example, the electronic ND filter unit 13a provided in the dimming unit 71a and the clear glass 13b provided in the dimming unit 71b. The switching between the dimming units 71a and 71b is driven by a DC motor 72 of an ND filter drive unit (not shown) connected to the system control unit 16. The dimming units 71a and 71b are driven in the directions indicated by arrows B1 and B2, for example. The dimming units 71a and 71b are switched by being inserted into and removed from the optical path of the incident light as driven by the DC motor 72. By retracting the electronic ND filter 13 from the optical path, it is possible to prevent light from being attenuated when passing through the electronic ND filter 13. It is also possible to suppress fading of the electronic ND filter 13 caused by exposure to ultraviolet light.

[0096] The control method described in the above-described embodiment can be realized by executing a prepared control program on a computer. The control program is recorded on a computer-readable storage medium and executed by being read from the storage medium. The control program may be provided in a form stored on a non-transitory storage medium such as a flash memory, or may be provided via a network such as the Internet. The computer that executes the control program may be included in the imaging device, or may be included in an electronic device such as a smartphone, tablet, or personal computer that can communicate with the imaging device, or may be included in a server device that can communicate with these imaging devices and electronic devices.

[0097] In this embodiment, each process is executed by a computer. The computer may execute these processes by a processor, a program, or a combination thereof. The computer may be a general-purpose computer, a computer for specific applications, a system such as a workstation, or other hardware element capable of executing a program.

[0098] The processor may be configured with one or more pieces of hardware, and the type of hardware is not limited. For example, the processor may be configured with hardware such as a programmable logic device such as a central processing unit (CPU), a micro processing unit (MPU), a field programmable gate array (FPGA), a dedicated circuit for executing specific processing such as an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or a neural processing unit (NPU). The processor also has various units or means for executing various processes in this embodiment. The type of hardware may also be a combination of different types of hardware. When multiple pieces of hardware are configured to execute one or more processes of a certain processor, the multiple pieces of hardware may be located in devices physically separated from each other, or may be located in the same device. Furthermore, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware is configured by an electric circuit (circuitry) or the like that combines circuit elements such as semiconductor elements.

[0099] Furthermore, the present embodiment may be implemented by hardware, software, firmware, microcode, or a combination thereof. Software, firmware, and microcode may be configured by a program. A program may also be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform the respective function. The program may be program code or multiple code segments stored in one or more non-transitory computer-readable media (e.g., storage media or other storages). The program may be stored in multiple non-transitory computer-readable media that reside in physically separate devices. Program code or a code segment may represent a procedure, a function, a subprogram, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. Program code or a code segment may be connected to another code segment or a hardware circuit by sending or receiving information, data, arguments, parameters, or memory contents.

[0100] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0101] This application is based on a Japanese patent application (Patent Application No. 2024-149012) filed on August 30, 2024, the contents of which are incorporated herein by reference.

[0102] 10 Imaging device 11 Imaging lens 13 Electronic ND filter 13a Electronic ND filter section 13b Clear glass 14 Imaging element 15 Digital signal processing section 16 System control section 17 EVF 18 Display LCD 19 Card storage section 20 Operation section 31a, 31b Transparent substrate 32a, 32b Transparent electrode 33a, 33b Alignment film 34a, 34b Sealing member 35 Liquid crystal layer 41 Control signal 42 Main exposure 43, 46a, 46b Readout 44 Live view exposure 45a, 45b, 45c Exposure 47 Live view 50 Pixel 51 Photoelectric conversion section 52 Charge storage section 53 Floating diffusion 54 Initialization transistor 55, 56 Transfer transistor 57 Reset transistor 58 Output transistor 59 Selection transistor 60 Signal line 71a, 71b Dimming unit 72 DC motor t1 to t4, t11 to t14 Time

Claims

1. An imaging device comprising a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of a first control of simultaneously exposing the plurality of pixels and reading out electric charges accumulated in the plurality of pixels to a charge storage unit, and the processor, after the first control, performs a first transmittance control of lowering the transmittance of the light control device, and performs a second transmittance control of raising the transmittance of the light control device during a readout period in which the electric charges read out to the charge storage unit are read out as an image signal.

2. An imaging device according to claim 1, wherein the processor performs the first transmittance control immediately after the first control.

3. An imaging device according to claim 1, wherein the processor performs the second transmittance control before exposure of the plurality of pixels starts during the readout period.

4. An imaging device according to claim 1, wherein the processor determines the timing for performing the second transmittance control based on a shutter speed for imaging.

5. An imaging device according to claim 1, wherein the processor determines the transmittance setting value of the light control device in the first transmittance control based on the shutter speed of the imaging device.

6. An imaging device according to claim 1, wherein the light control device is insertable into and removable from the optical path of the incident light.

7. An imaging device according to claim 1, wherein the processor determines whether or not to execute the first transmittance control and the second transmittance control based on the image signal read out from the charge holding section.

8. An imaging device according to claim 7, wherein the processor executes the first transmittance control and the second transmittance control when the image signal includes pixels whose brightness is equal to or greater than a threshold value.

9. An imaging device according to claim 1, wherein the processor starts the first transmittance control based on a readout signal that instructs the charge storage section to read out.

10. An imaging device according to claim 1, wherein the processor controls the light control device to move away from the optical path of the incident light or to set the transmittance of the light control device to a predetermined value when not capturing an image.

11. An imaging device according to claim 1, wherein the processor, in the second transmittance control, increases the transmittance of the light control device to a transmittance value determined based on a user setting.

12. An imaging device according to claim 1, wherein the processor determines the timing for performing the second transmittance control based on the image signal read out from the charge holding section.

13. An imaging device according to claim 12, wherein the processor determines the timing to perform the second transmittance control based on the position of a pixel on the image represented by the image signal whose brightness is equal to or greater than a threshold value.

14. An imaging device according to claim 1, wherein the processor determines the timing for performing the second transmittance control based on the transmittance setting value of the light control device in the first transmittance control.

15. An imaging device according to claim 1, wherein the imaging element is capable of resetting only the plurality of pixels among the plurality of pixels and the charge holding section.

16. An imaging device comprising a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge storage unit, and the processor performs first transmittance control of lowering the transmittance of the light control device after the first control, and performs second transmittance control of raising the transmittance of the light control device before exposure of the plurality of pixels begins.

17. A control method for an imaging device comprising a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge storage section, and the processor performs first transmittance control of lowering the transmittance of the light control device after the first control, and performs second transmittance control of increasing the transmittance of the light control device during a readout period in which the charges read out to the charge storage section are read out as an image signal.

18. A control method for an imaging device comprising a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of first control of simultaneously exposing the plurality of pixels and reading out charges accumulated in the plurality of pixels to a charge storage unit, and the processor performs first transmittance control of lowering the transmittance of the light control device after the first control, and performs second transmittance control of raising the transmittance of the light control device before exposure of the plurality of pixels begins.

19. A control program for an imaging device comprising a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of first control of simultaneously exposing the plurality of pixels and reading out the charges accumulated in the plurality of pixels to a charge holding section, and the control program causes the processor to execute the following processes: after the first control, first transmittance control of lowering the transmittance of the light control device, and during a readout period in which the charges read out to the charge holding section are read out as an image signal, second transmittance control of increasing the transmittance of the light control device.

20. A control program for an imaging device comprising a light control device capable of controlling the transmittance of light incident through an imaging lens, an imaging element having a plurality of pixels, and a processor, wherein the imaging element is capable of first control of simultaneously exposing the plurality of pixels and reading out electric charges accumulated in the plurality of pixels to an electric charge storage unit, and the control program causes the processor to execute processing to perform first transmittance control of lowering the transmittance of the light control device after the first control, and second transmittance control of raising the transmittance of the light control device before exposure of the plurality of pixels begins.

Images