Imaging device, control method, control program, and imaging system

Abstract

Provided are an imaging device, a control method, a control program, and an imaging system that are capable of suppressing the effects of PLS noise on other functions while reducing the PLS noise.　The present invention comprises: an electronic ND filter (13) capable of controlling, for each area, the transmittance of light that enters through an imaging lens (11); an imaging element (14) capable of performing simultaneous readout control for reading out accumulated charges to a charge-holding unit by simultaneously exposing a plurality of pixels; and a system control unit (16). The system control unit (16) performs a process for detecting a high-luminance subject by analyzing image data obtained through imaging performed before the simultaneous readout control, determines the high-luminance area of a dimming device corresponding to a region where the high-luminance subject is present when the high-luminance subject is detected, and performs transmittance-lowering control for lowering the transmittance of the high-luminance area in the electronic ND filter (13) after the simultaneous readout control.

Description

Imaging Device, Control Method, Control Program, and Imaging System

[0001] The present invention relates to an imaging device, a control method, a control program, and an imaging system.

[0002] Patent Document 1 discloses a camera system that includes an aperture mechanism as an incident light amount changing means. The aperture mechanism is controlled to the aperture value for normal shooting during the exposure period in which the exposure outputs of all photodiodes (PDs) are simultaneously transferred to the charge accumulation section (FD) in the global shutter (GS) operation of the MOS type imaging device, and the aperture value is set high during the sequential signal readout period after exposure to suppress incident light (generation of false signals due to leakage light) to the MOS type imaging device.

[0003] Patent Document 2 discloses an imaging device that includes a vertical operation circuit, a horizontal operation circuit, a readout amplifier, a pixel region, a liquid crystal light shielding plate, a liquid crystal driving section, and a control section. The liquid crystal light shielding plates are each set to either a light shielding state or a transmission state by the liquid crystal driving section, and the liquid crystal driving section is controlled by the control section. When a region to be read out in the pixel region is set as a selected pixel region and a region not to be read out is set as a non-selected pixel region, when reading out the selected pixel region, by shielding the non-selected pixel region with the liquid crystal light shielding plate, it is possible to avoid the occurrence of smear and blooming even when extremely strong light hits the non-selected pixel region.

[0004] Patent Document 3 discloses an imaging device that includes an optical system, a lens driving section, an image sensor, and an electronic ND filter. The electronic ND filter has liquid crystal elements encapsulated between a plurality of glass substrates and can independently adjust the transmittance of incident light in pixel units. To suppress overexposure or underexposure, the transmittance of the electronic ND filter corresponding to a bright region is relatively lowered, and the transmittance of the electronic ND filter corresponding to a dark region is relatively increased.

[0005] Patent Document 4 describes a surveillance camera comprising a camera unit, a processor, memory, storage, and a communication device, wherein the camera unit includes an optical system, an image sensor, and a neutral density filter, the neutral density filter includes a liquid crystal layer, and the processor controls the attenuation of light incident on the image sensor via the optical system in units of divided regions, and when it is determined that there is a high-brightness area in the image captured by the image sensor, the attenuation of the neutral density filter in the corresponding region is set relatively higher to suppress the overflow of the dynamic range of the image sensor.

[0006] Patent Document 5 describes an imaging device comprising an imaging optical system, an image sensor, a control unit, a memory, a display unit, and an operation unit, wherein the image sensor is made of a CMOS image sensor or the like and has a global shutter function and a rolling shutter function, the control unit comprises a detection unit, an imaging control unit, and an image data generation unit, the detection unit detects areas of the image sensor's pixel area where the amount of light received from the subject is large as high-brightness areas, and generates information regarding the position of the high-brightness areas, information regarding the direction of movement of the high-brightness areas, information regarding the speed of movement of the high-brightness areas, etc., and detects high-brightness areas using pixel signals obtained by preliminary imaging to acquire image data for live view images before imaging, and the imaging control unit, in imaging using a global shutter method, controls the readout process to read out the pixel signals of high-brightness areas before the pixel signals of other low-brightness areas in order to counter PLS, thereby shortening the time until readout is performed in the high-brightness areas.

[0007] Japanese Patent Publication No. 2008-028516, Japanese Patent Publication No. 2005-311846, Japanese Patent Publication No. 2023-154269, Japanese Patent Publication No. 2022-175162, Japanese Patent Publication No. 2022-127306

[0008] One embodiment of the technology described herein provides an imaging device, control method, control program, and imaging system that can reduce PLS noise while suppressing the impact on other functions.

[0009] (1) An imaging device comprising: a light control device capable of controlling the transmittance of light incident through an imaging lens for each area; an image sensor capable of a first control that reads the charge accumulated by simultaneously exposing multiple pixels to a charge holding unit; and a processor, wherein the processor performs a high-brightness subject detection process by analyzing image data obtained by imaging performed before the first control, determines a first area of ​​the light control device corresponding to the region where the high-brightness subject is located when the high-brightness subject is detected, and performs a first transmittance control that reduces the transmittance of the first area in the light control device after the first control.

[0010] (2) The imaging device described in (1), wherein the detection process includes the detection of the position and size of the high-luminance subject.

[0011] (3) An imaging device according to (1) or (2), wherein the detection process includes detection of the movement state of the high-luminance subject.

[0012] (4) The imaging apparatus described in (3), wherein the processor determines the first area based on the charge holding time from the completion of the first control until the charge is read out from the charge holding unit and the movement state.

[0013] (5) An imaging apparatus according to (4), wherein the processor determines the range of movement of the high-luminance subject during the period from the completion of the first control until the charge is read out from the charge holding unit, based on the charge holding time and the movement state, and defines the area encompassing the range of movement as the first area.

[0014] (6) An imaging device according to (4), wherein the processor estimates the movement trajectory of the high-luminance subject during the period from the completion of the first control until the charge is read out from the charge holding unit, based on the charge holding time and the movement state, and moves the first area in the first transmittance control based on the movement trajectory.

[0015] (7) An imaging device as described in (4), wherein the charge retention time varies depending on the region of the image sensor.

[0016] (8) An imaging device according to any one of (4) to (7), wherein the processor does not perform the first transmittance control according to the movement state.

[0017] (9) A control method for an imaging device comprising: a light control device capable of controlling the transmittance of light incident through an imaging lens for each area; an image sensor capable of a first control that reads the charge accumulated by simultaneously exposing multiple pixels to a charge holding unit; and a processor, wherein the processor performs a high-brightness subject detection process by analyzing image data obtained by imaging performed before the first control; when a high-brightness subject is detected, it determines a first area of ​​the light control device corresponding to the region where the high-brightness subject exists; and after the first control, it performs a first transmittance control that reduces the transmittance of the first area in the light control device.

[0018] (10) A control program for an imaging device comprising: a light control device capable of controlling the transmittance of light incident through an imaging lens for each area; an image sensor capable of a first control that reads the charge accumulated by simultaneously exposing multiple pixels to a charge holding unit; and a processor, the control program causing the processor to perform a process of detecting a high-brightness subject by analyzing image data obtained by imaging performed before the first control; determining a first area of ​​the light control device corresponding to the region where the high-brightness subject is located if the high-brightness subject is detected; and performing a first transmittance control that reduces the transmittance of the first area in the light control device after the first control.

[0019] (11) An imaging system comprising: an imaging lens; a light control device capable of controlling the transmittance of light incident through the imaging lens for each area; an image sensor capable of a first control that reads the charge accumulated by simultaneously exposing multiple pixels to a charge holding unit; and a processor, wherein the processor performs a high-brightness subject detection process by analyzing image data obtained by imaging performed before the first control; when the high-brightness subject is detected, it determines a first area of ​​the light control device corresponding to the region where the high-brightness subject is located; and after the first control, it performs a first transmittance control that reduces the transmittance of the first area in the light control device.

[0020] According to the present invention, it is possible to provide an imaging device, control method, control program, and imaging system that can reduce PLS noise while suppressing the impact on other functions.

[0021] This figure shows the configuration of the imaging system 1 including the imaging device 10 of this embodiment. This is a cross-sectional view showing the configuration of the electronic ND filter 13. This figure shows an example of the control sequence of the system control unit 16 during still image capture. This is a flowchart showing the control processing of the system control unit 16 during imaging. This figure shows an example of subject movement during the charge readout period. This figure shows an example of PLS ​​noise generated in the captured image due to subject movement. This figure shows the identification of a high-luminance subject in the first example of PLS ​​noise countermeasures. This figure shows the transmission rate reduction control in the first example of PLS ​​noise countermeasures. This figure shows the identification of a high-luminance subject in the second example of PLS ​​noise countermeasures. This figure shows the transmission rate reduction control in the second example of PLS ​​noise countermeasures. This figure shows the identification of a high-luminance subject in the third example of PLS ​​noise countermeasures. This figure shows the transmission rate reduction control in the third example of PLS ​​noise countermeasures. This is a flowchart showing a modified example of the control processing of the system control unit 16 during imaging. This figure shows an example of the replacement mechanism for the electronic ND filter 13.

[0022] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0023] <Imaging System> Figure 1 is a diagram showing the configuration of the imaging system 1, including the imaging device 10 of this embodiment. As shown in Figure 1, the imaging system 1 includes an imaging lens 11, an aperture 12, an electronic ND filter 13, an image sensor 14, a digital signal processing unit 15, and a system control unit 16. The imaging system 1 also includes an EVF (Electronic View Finder) 17, a display LCD (Liquid Crystal Display) 18, a card storage unit 19, and an operation unit 20. The imaging system 1 is, for example, a digital still camera or a digital video camera capable of capturing video.

[0024] The imaging lens 11 is a lens that forms an image of incident light on the image sensor 14. The imaging lens 11 includes a focus lens that is movable in the optical axis direction. The focus lens is a lens for adjusting the focus of the imaging optical system, which includes the imaging lens 11 and the aperture 12, and is composed of a single lens or multiple lenses. The imaging lens 11 changes the focal position on the subject side by moving the focus lens in the optical axis direction based on the imaging control signal transmitted from the system control unit 16, thereby changing the position of the principal point of the focus lens along the optical axis direction.

[0025] The aperture 12 is mechanically adjustable in terms of the amount of incident light. The aperture 12 changes its aperture amount (F-number) based on a drive control signal transmitted from the system control unit 16.

[0026] The electronic ND (Neutral Density) filter 13 is an example of the "light control device" of the present invention. The electronic ND filter 13 allows control of the transmittance of light incident through the imaging lens 11 for each area of ​​the electronic ND filter 13. The transmittance decreases as the filter density of the electronic ND filter 13 increases. The electronic ND filter 13 is a guest-host type electronic variable ND filter, for example, constructed using a guest-host (GH) type liquid crystal containing a dye (dichroic dye). Since the dichroic dye has different light absorption rates depending on the axial direction, when a voltage is applied to the liquid crystal, the liquid crystal molecules change their arrangement in response to the voltage, and follow the movement of the liquid crystal to create a transmittance state and an absorption (light-blocking) state.

[0027] The electronic ND filter 13 is a filter that adjusts only the amount of light without affecting color. The electronic ND filter 13 is positioned between the imaging lens 11 and the image sensor 14, close to the image sensor 14. The electronic ND filter 13 creates a transmission state and a light-blocking state based on the imaging control signal transmitted from the system control unit 16. The electronic ND filter 13 can create a transmission state and a light-blocking state for any area. Furthermore, the electronic ND filter 13 is configured to be insertable and removable from the optical path of light incident through the imaging lens 11. A liquid crystal shutter may be used as an example of a "light control device".

[0028] The image sensor 14 images the subject through an imaging optical system including an imaging lens 11, an aperture 12, and an electronic ND filter 13. The image sensor 14 images the subject based on an imaging control signal transmitted from the system control unit 16. The image sensor 14 has a light-receiving surface in which multiple pixels are arranged in two dimensions, and the imaging optical system converts the subject image formed on the light-receiving surface into a pixel signal using the multiple pixels and outputs it. The image sensor 14 images the target subject at a predetermined frame rate. The "predetermined frame rate" refers to, for example, tens of frames per second to hundreds of frames per second. The image sensor 14 is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The image sensor 14 has, for example, a photodiode in each of its multiple pixels that generates and stores an electric charge corresponding to the amount of light received, and a charge holding unit that reads out the charge stored in the photodiode. The image sensor 14 is capable of simultaneous exposure of multiple pixels and simultaneous readout control that reads out the charge stored in the photodiodes of the multiple pixels during the simultaneous exposure to the charge holding unit. "Simultaneous exposure of multiple pixels" means that all pixels are exposed with a very short time difference, and that there is a time when all pixels are exposed. However, due to the characteristics of the electronic circuit and control, minute time differences at the nanosecond or picosecond level may exist. "Reading out the charge stored in multiple pixels to the charge holding unit" means that the charge stored in multiple pixels is read out to the charge holding unit at the same time as all pixels. The image sensor 14 is a global shutter type image sensor that exposes all pixels simultaneously. Simultaneous readout control is the "first control" of the present invention.

[0030] The digital signal processing unit 15 processes the image signal output from the image sensor 14 to generate image data suitable for display on the display LCD 18 and image data suitable for storage in the card storage unit 19. The digital signal processing unit 15 generates image data by applying signal processing such as demosaicing and gamma correction to the image data output from the image sensor 14. The digital signal processing unit 15 executes each signal processing based on the image control signal transmitted from the system control unit 16.

[0031] The system control unit 16 provides overall control for the entire imaging system 1. The hardware structure of the system control unit 16 consists of various processors that execute programs and perform processing. These various processors include general-purpose processors such as CPUs (Central Processing Units) that execute programs and perform various processing tasks, programmable logic devices (PLDs) such as FPGAs (Field Programmable Gate Arrays) whose circuit configurations can be changed after manufacturing, and dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits) which have circuit configurations specifically designed to perform specific processing tasks. More specifically, the structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements. The system control unit 16 is an example of a "processor" in the present invention.

[0032] The system control unit 16 may be composed of one of various processors, or it may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0033] The system control unit 16 drives the imaging lens 11, aperture 12, electronic ND filter 13, and image sensor 14, and outputs the subject image captured through the imaging optical system including the imaging lens 11, aperture 12, and electronic ND filter 13 as an image signal from the image sensor 14 to the digital signal processing unit 15.

[0034] Specifically, the system control unit 16 performs high-brightness subject detection processing by analyzing image data obtained by imaging performed before simultaneous readout (global transfer) control. "Image data obtained by imaging performed before simultaneous readout control" refers, for example, to the through image displayed on the display LCD 18. "High-brightness subject" refers to, for example, a subject that has a brightness value above a predetermined threshold (in the state of proper exposure). "High-brightness subject" may also refer to a subject with an extremely high brightness value compared to other areas, such as when a light source is reflected as a subject.

[0035] The detection process for high-luminance objects includes detecting the position and size of the high-luminance object. Furthermore, the detection process for high-luminance objects includes detecting the movement state of the high-luminance object. "Movement state" refers to at least one of the direction and speed of movement of the high-luminance object.

[0036] Furthermore, when the system control unit 16 detects a high-brightness subject by analyzing the image data, it determines the high-brightness area of ​​the electronic ND filter 13 corresponding to the region where the high-brightness subject exists. After simultaneous readout control, the system control unit 16 performs transmittance reduction control to lower the transmittance of the high-brightness area in the electronic ND filter 13.

[0037] "After simultaneous readout control" refers to the period immediately following the completion of simultaneous readout control, that is, the start of the readout when the charge read out to the charge holding unit during simultaneous readout control is read out as an image signal. The readout period during which the charge read out to the charge holding unit is read out as an image signal is the period during which each pixel is read out sequentially, and therefore is a period during which PLS (Parasitic Light Sensitivity) noise may occur. PLS noise is noise that degrades the image due to light leaking into the charge holding unit. "Reducing transmittance" means making the transmittance lower than that of areas other than the high-brightness area. The high-brightness area is the "first area" in this invention. Transmittance reduction control is the "first transmittance control" in this invention.

[0038] Furthermore, the system control unit 16 determines the high-brightness area of ​​the electronic ND filter 13 based on the charge retention time from the completion of simultaneous readout control until the charge is sequentially read out as an image signal from the charge retention unit, and the movement state of the high-brightness subject. The charge retention time differs depending on the region of the image sensor 14. A "region" refers to a row (line) to be read out when the region of the image sensor 14 is divided into a predetermined matrix. Since the readout of charge from the charge retention unit is performed sequentially row by row, there is a difference in charge retention time depending on the region.

[0039] Furthermore, the system control unit 16 determines the range of movement of the high-brightness subject during the period from the completion of simultaneous readout control until the charge is read out as an image signal from the charge holding unit, based on the charge holding time and the movement state. The system control unit 16 defines the area encompassing the movement range as the high-brightness area of ​​the electronic ND filter 13.

[0040] Furthermore, the system control unit 16 estimates the movement trajectory of the high-brightness subject during the period from the completion of simultaneous readout control until the charge is read out as an image signal from the charge holding unit, based on the charge holding time and the movement state, and moves the high-brightness area by transmittance reduction control based on the estimated movement trajectory. "Moving the high-brightness area" means moving the high-brightness area so as to follow the movement of the high-brightness subject.

[0041] Furthermore, the system control unit 16 will not perform transmittance reduction control depending on the movement state of the high-luminance subject. For example, the system control unit 16 will not perform transmittance reduction control if the movement speed of the high-luminance subject is below a threshold.

[0042] EVF17 is an electronic viewfinder that can view the image converted by the digital signal processing unit 15. EVF17 can view the image projected on the display LCD18 through the viewfinder. The display LCD18 is composed of, for example, an organic EL (electroluminescence) panel or a liquid crystal panel. The card storage unit 19 is a place where an SD memory card is set. The operation unit 20 receives an instruction signal from the user. The operation unit 20 includes a touch panel integrated with the display LCD18 and various buttons.

[0043] <Electronic ND Filter> FIG. 2 is a cross-sectional view showing the configuration of the electronic ND filter 13. As shown in FIG. 2, the electronic ND filter 13 includes transparent substrates 31a and 31b, transparent electrodes 32a and 32b, alignment films 33a and 33b, and seal members 34a and 34b. The direction indicated by arrow A in FIG. 2 is the direction in which the light incident on the electronic ND filter 13 travels.

[0044] On the light incident side, the transparent substrate 31a, the transparent electrode 32a, and the alignment film 33a are laminated in this order. Also, on the light exit side, the transparent substrate 31b, the transparent electrode 32b, and the alignment film 33b are laminated in this order. And a liquid crystal layer 35 containing liquid crystal molecules and a dye (dichroic dye) is provided between the alignment film 33a and the alignment film 33b.

[0045] The transparent substrates 31a and 31b are substrates that support the transparent electrodes 32a and 32b and the alignment films 33a and 33b and seal the liquid crystal layer 35. The transparent substrates 31a and 31b are composed of, for example, glass substrates.

[0046] The transparent electrodes 32a and 32b are electrodes for applying a control voltage to the liquid crystal layer 35. The transparent electrodes 32a and 32b are composed of, for example, indium tin oxide.

[0047] The alignment films 33a and 33b are films that align each liquid crystal molecule of the liquid crystal layer 35 in a specific direction. The alignment films 33a and 33b are composed of, for example, a polymer material such as polyimide.

[0048] The seal members 34a and 34b are members that seal the liquid crystal molecules and dye molecules in the liquid crystal layer 35 from leaking out at the ends. The seal members 34a and 34b are made of an adhesive such as epoxy or acrylic, for example.

[0049] In this example, a single liquid crystal cell with such a configuration is shown, but it is not limited to this. For example, a plurality of liquid crystal cells may be stacked by changing the light distribution direction of the alignment film.

[0050] <Imaging control sequence> Fig. 3 is a diagram showing an example of the control sequence of the system control unit 16 during still image imaging.

[0051] In Fig. 3, "dimming control" means controlling the transmittance (filter density) of the electronic ND filter 13. "Dimming control: ON" means controlling the transmittance of the electronic ND filter 13 to the lowest transmittance. The lowest transmittance means controlling the filter density to the maximum density and controlling the entire image to black. "Dimming control: OFF" means controlling the transmittance of the electronic ND filter 13 to the highest transmittance. The highest transmittance means controlling the filter density to the minimum density and controlling the entire image to white. "Main exposure" means the exposure when taking a still image. "Reading" means the reading of sequentially reading out the charges read into the charge holding unit as image signals from the charge holding unit during the main exposure. "GR" means erasing the charges accumulated in the photodiodes of each pixel. "GS" means the simultaneous reading (global transfer) of the charges of a plurality of pixels accumulated during the main exposure to the charge holding unit simultaneously for all pixels.

[0052] In the state of the live view 47 where the image of the subject is being displayed on the display LCD 18 and imaging is being performed, the system control unit 16 analyzes the data of the through image and performs a detection process to determine whether there is a high-brightness subject. When the system control unit 16 detects a high-brightness subject, it determines the high-brightness area of the electronic ND filter 13 corresponding to the area where the high-brightness subject exists.

[0053] In the state of the live view 47, assume that the user presses, for example, the release button at time t1 to take a still image.

[0054] When the release button is pressed, the system control unit 16 causes the image sensor 14 to perform GR to erase the charge accumulated in the photodiode of each pixel. Once GR is complete, the system control unit 16 causes the main exposure 42 to perform simultaneous exposure of multiple pixels. At time t2, the system control unit 16 causes the image sensor 14 to perform simultaneous readout control (global transfer: GS) to read the charge accumulated in the photodiodes of multiple pixels by the main exposure 42 to the charge holding unit at the same time for all pixels.

[0055] When the GS is complete, the system control unit 16 starts the readout process 43, which sequentially reads out the charge read out to the charge holding unit in the main exposure 42 as an image signal for each pixel. Also, immediately after the GS is complete, the system control unit 16 turns on the light-reducing control signal 41 to perform transmittance reduction control, which reduces the transmittance of the area of ​​the electronic ND filter 13 that corresponds to the area where the high-brightness subject detected in the above detection process exists. The system control unit 16 performs transmittance reduction control to set the transmittance in the high-brightness area of ​​the electronic ND filter 13 to, for example, the lowest transmittance, making the image black only in the area where the high-brightness subject exists. The system control unit 16 controls the transmittance in areas other than the high-brightness area of ​​the electronic ND filter 13 to, for example, the highest transmittance, making the image white in areas where no high-brightness subject exists.

[0056] When the readout process 43 is completed (time t3), the system control unit 16 turns off the dimming control signal 41. Also, when the still image capture is finished, the system control unit 16 returns to live view 47 (time t3).

[0057] Although not shown in Figure 3, the system control unit 16 may, upon returning to live view 47, perform the live view exposure of the first frame image after imaging during the readout 43 period in which the charge of the main exposure 42 is sequentially read out, that is, before time t3. When performing live view exposure, the system control unit 16 causes the image sensor 14 to perform GR before starting the exposure to erase the charge accumulated in the photodiode of each pixel during the main exposure 42.

[0058] <Control Processing During Image Capture> Figure 4 is a flowchart showing the control processing of the system control unit 16 during image capture. This control processing is started, for example, when the mode setting button on the operation unit 20 is set to the still image capture mode.

[0059] The system control unit 16 starts a live view display process to display a live view image of the subject on the display LCD 18 (step S11). The system control unit 16 also starts a detection process to analyze the data of the through image to determine whether or not a high-brightness subject exists (step S11). The high-brightness subject detection process is performed repeatedly. Examples of high-brightness subjects include lights from vehicles and aircraft.

[0060] Next, the system control unit 16 determines whether or not still image capture has been performed (step S12). The system control unit 16 determines whether or not image capture has been performed, for example, by whether or not the release button has been pressed.

[0061] If it is determined in step S12 that imaging has not been performed (step S12: No), the system control unit 16 repeats the process of step S12. If it is determined in step S12 that imaging has been performed (step S12: Yes), the system control unit 16 determines whether or not a high-luminance subject was detected in the detection process of step S11 (step S13).

[0062] If no high-brightness subject is detected in step S13 (step S13: No), the system control unit 16 erases the charge accumulated in the photodiode of each pixel and then performs the main exposure to capture a still image by simultaneous exposure (step S14). The system control unit 16 also performs simultaneous readout control (global transfer: GS) to read the charge accumulated in the photodiodes of multiple pixels by the main exposure to the charge holding unit at the same time for all pixels.

[0063] Next, the system control unit 16 starts a readout process to sequentially read out the charge read out by the charge holding unit during this exposure as an image signal (step S15).

[0064] The system control unit 16 completes the readout process by sequentially reading the charge read out from the charge holding unit as an image signal for each pixel (step S16). The system control unit 16 does not perform transmittance reduction control if no high-brightness subject is detected.

[0065] Once the reading process is complete, the system control unit 16 records the image data of the still image captured in step S12 into the card storage unit 19 (step S17).

[0066] On the other hand, if a high-brightness subject is detected in step S13 (step S13: Yes), the system control unit 16 performs the main exposure after erasing the charge, similar to step S14 (step S18). The system control unit 16 also performs simultaneous readout control to read the charge to the charge holding unit at the same time for all pixels.

[0067] Next, the system control unit 16 starts a readout process to sequentially read out the charges read out by the charge holding unit during this exposure as image signals, and also performs transmittance reduction control to partially lower the transmittance of the high-brightness areas corresponding to the detected high-brightness subjects (step S19).

[0068] The system control unit 16 completes the readout process by sequentially reading the charge read out from the charge holding unit as an image signal for each pixel, and also stops the transmittance reduction control that partially lowers the transmittance, restoring the transmittance to its original value (step S20).

[0069] Once the reading process is complete and the transmittance is restored, the system control unit 16 records the image data of the still image capture performed in step S12 into the card storage unit 19 (step S17).

[0070] <Generation of PLS ​​Noise> Next, examples of PLS ​​noise generation during still image capture will be explained with reference to Figures 5 and 6. Figure 5 is a diagram showing an example of subject movement during the charge readout period. Figure 6 is a diagram showing an example of PLS ​​noise generated in the captured image due to subject movement.

[0071] As shown in Figure 5, let's assume the subject to be imaged is a vehicle 51. Let's assume the vehicle 51 is traveling at a constant speed, for example. Let's also assume that the vehicle 51 is traveling with its headlights 52 on.

[0072] As described above, when the user presses the release button in live view mode, the main exposure for still image capture is performed, and then a readout process begins in which the charge read out to the charge holding unit during the main exposure is sequentially read out as an image signal for each pixel.

[0073] When the release button is pressed while the vehicle 51 is moving, during the readout process, the vehicle 51 moves from driving position A to driving position B within the still image capture range 50 shown in Figure 5.

[0074] In this case, since the vehicle 51 has its headlights 52 illuminated, the trajectory of the headlights 52, which moves along with the movement of the vehicle 51 during the readout period, is superimposed as a bright spot on the still image of the vehicle 51. As a result, the trajectory of the headlights 52 appears as PLS noise 53 in the still image of the vehicle 51, as shown in Figure 6.

[0075] Next, examples of countermeasures against PLS noise during still image capture will be explained with reference to Figures 7 and 12.

[0076] <First Example of PLS ​​Noise Countermeasures> Figure 7 shows the identification of a high-luminance subject in the first example of PLS ​​noise countermeasures. As shown in Figure 7, the system control unit 16 divides the imaging range 50 to be imaged into predetermined areas. In this example, the imaging range 50 is divided into a 2x3 area.

[0077] The system control unit 16 performs image analysis of the imaging range 50 and performs detection processing to detect, for example, the position, size, direction of movement, and speed of a high-brightness subject having a brightness value above a predetermined threshold.

[0078] In this example, the headlights 52 of vehicle 51 are detected as high-luminosity objects. The system control unit 16 performs detection processing for the position, size, direction of movement, and speed of movement of the headlights 52 as vehicle 51 moves. It is assumed that vehicle 51 moved from driving position A to driving position B within the imaging range 50 during the readout process.

[0079] When the system control unit 16 detects a high-brightness subject, it identifies the region within the imaging range 50 that includes the detected high-brightness subject. For example, the system control unit 16 identifies the region that includes the high-brightness subject at the time the release button is pressed.

[0080] In this example, the area in the second row and second column where the headlight 52 of vehicle 51 is located (the area enclosed by the dashed line) is identified as the area containing a high-luminance subject.

[0081] The system control unit 16 determines the area containing the identified high-luminance subject as the high-luminance area of ​​the electronic ND filter 13.

[0082] In this example, the area corresponding to the 2nd row and 2nd column region (the area enclosed by the dashed line) where the headlight 52 of the vehicle 51 is located is determined to be the high-luminance area in the electronic ND filter 13.

[0083] Figure 8 shows the transmittance reduction control in the first example of PLS ​​noise countermeasures. The system control unit 16 performs transmittance reduction control to lower the transmittance in the high-luminance area of ​​the electronic ND filter 13 determined in Figure 7, that is, the area in the 2nd row and 2nd column (shaded area) where the headlight 52 is located in the imaging range 50 shown in Figure 8.

[0084] <Second example of PLS ​​noise countermeasures> Figure 9 shows the identification of high-luminance subjects in the second example of PLS ​​noise countermeasures. As shown in Figure 9, the system control unit 16 divides the imaging range 50 into a 2x3 region, similar to the first example (Figure 7).

[0085] The system control unit 16 performs image analysis of the imaging range 50 and detects, for example, the headlights 52 of the vehicle 51 as high-luminance subjects with a luminance value above a predetermined threshold. The system control unit 16 performs detection processing for the position, size, direction of movement, and speed of movement of the headlights 52 as the vehicle 51 moves. It is assumed that the vehicle 51 moved from driving position A to driving position B within the imaging range 50 during the readout process.

[0086] When the system control unit 16 detects a high-brightness subject, it identifies a region within the imaging range 50 that includes the expected movement range of the detected high-brightness subject. The system control unit 16 identifies the region including the expected movement range of the high-brightness subject based on the charge holding time from the completion of simultaneous readout control (global transfer: GS) of the main exposure in still image acquisition until the charge is read out as an image signal from the charge holding unit.

[0087] In this example, the region including the area in the 2nd row, 1st column and the area in the 2nd row, 2nd column where the headlight 52 of vehicle 51 is located (the region enclosed by the dashed line) is identified as the region containing the expected movement range of the high-luminance subject. This expected movement range is, for example, the region containing the PLS noise 53 shown in Figure 6.

[0088] The system control unit 16 determines that the area encompassing the identified predicted movement range is the high-brightness area of ​​the electronic ND filter 13.

[0089] In this example, the areas corresponding to the 2nd row, 1st column and the 2nd row, 2nd column (areas enclosed by dashed lines) where the headlights 52 of the vehicle 51 are located are determined to be the high-luminance areas in the electronic ND filter 13.

[0090] Figure 10 shows the transmittance reduction control in the second example of PLS ​​noise countermeasures. The system control unit 16 performs transmittance reduction control to lower the transmittance of the high-luminance area of ​​the electronic ND filter 13 determined in Figure 9, that is, the area in the 2nd row, 1st column and the 2nd row, 2nd column (shaded area) where the headlights 52 are located in the imaging range 50 shown in Figure 10.

[0091] <Third example of PLS ​​noise countermeasures> Figure 11 shows the identification of high-luminance subjects in the third example of PLS ​​noise countermeasures. As shown in Figure 11, the system control unit 16 divides the imaging range 50 into a 2x3 region, similar to the first example (Figure 7).

[0092] The system control unit 16 performs image analysis of the imaging range 50 and detects, for example, the headlights 52 of the vehicle 51 as high-luminance subjects with a luminance value above a predetermined threshold. The system control unit 16 performs detection processing for the position, size, direction of movement, and speed of movement of the headlights 52 as the vehicle 51 moves. It is assumed that the vehicle 51 moved from driving position A to driving position B within the imaging range 50 during the readout process.

[0093] When the system control unit 16 detects a high-brightness subject, it estimates the movement trajectory of the detected high-brightness subject within the imaging range 50. The system control unit 16 estimates the movement trajectory of the high-brightness subject based on the charge holding time from the completion of simultaneous readout control (global transfer: GS) of the main exposure in still image acquisition until the charge is read out as an image signal from the charge holding unit. The system control unit 16 identifies the region that includes the estimated movement trajectory of the high-brightness subject.

[0094] In this example, for instance, the trajectory indicated by the dashed arrow 54 is estimated to be the movement trajectory of the high-luminance subject. Furthermore, the region including the area in the 2nd row, 1st column and the area in the 2nd row, 2nd column (the region enclosed by the dashed line) is identified as the region containing the movement trajectory of the high-luminance subject.

[0095] The system control unit 16 determines that the area containing the identified movement trajectory is the high-brightness area of ​​the electronic ND filter 13.

[0096] In this example, the areas corresponding to the 2nd row, 1st column region and the 2nd row, 2nd column region (the area enclosed by the dashed line) where the movement trajectory of the headlight 52 exists are determined to be the high-luminance areas in the electronic ND filter 13.

[0097] Figure 12 shows the transmittance reduction control in the third example of PLS ​​noise countermeasures. The system control unit 16 performs transmittance reduction control to lower the transmittance for each region of the matrix that divides the imaging range 50 into high-brightness areas of the electronic ND filter 13 determined in Figure 11.

[0098] In this example, the determined high-luminance area is divided into two regions: the 2nd row, 1st column region and the 2nd row, 2nd column region, and the transmittance reduction control is performed accordingly.

[0099] The system control unit 16 moves the high-brightness area for which transmittance reduction control is performed in accordance with the estimated movement trajectory of the high-brightness subject.

[0100] For example, based on the estimated movement trajectory of the headlight 52, the system control unit 16 first performs transmittance reduction control to lower the transmittance of the area in the second row and second column (shaded area) of the imaging range 50, as shown in the upper diagram of Figure 12. Next, following the movement of the headlight 52, the system control unit 16 performs transmittance reduction control to lower the transmittance of the area in the second row and first column (shaded area) of the imaging range 50, as shown in the lower diagram of Figure 12.

[0101] As described above, the imaging system 1 of this embodiment performs headlight detection processing by analyzing the through image data obtained by imaging, which is performed before the simultaneous readout control that reads the charges of multiple pixels accumulated by simultaneous exposure into the charge holding unit. If a headlight 52 is detected, the system determines the high-brightness area of ​​the electronic ND filter 13 corresponding to the region where the headlight 52 is located, and after the simultaneous readout control, performs transmittance reduction control to lower the transmittance of the determined high-brightness area. With this configuration, the transmittance of the high-brightness area in the electronic ND filter 13 corresponding to the region where a high-brightness subject exists during the readout period, in which the charges of the charge holding unit are sequentially read out as image signals, can be reduced, thereby suppressing PLS noise that occurs during the readout period in the GS method. In addition, since the transmittance of only the high-brightness area in the electronic ND filter 13 is reduced, the brightness of areas other than the high-brightness area can be maintained. As a result, the through image does not become darker, making framing easier. Also, exposure errors can be reduced. Furthermore, live view exposure, which is performed after imaging, can be performed earlier (during the readout period 43).

[0102] Furthermore, the imaging system 1 determines the range of movement of the headlight 52 during the charge holding period based on the charge holding time from the completion of simultaneous readout control until the charge is read out as an image signal from the charge holding unit, and the movement state of the headlight 52. The area encompassing the determined range of movement is designated as the high-brightness area of ​​the electronic ND filter 13. With this configuration, it is possible to reduce the transmittance of the high-brightness area of ​​the electronic ND filter 13 corresponding to the range of movement of the high-brightness subject during the readout period in which the charge from the charge holding unit is sequentially read out as an image signal. This further suppresses PLS noise that occurs during the readout period in the GS method. In addition, since the transmittance of only the high-brightness area can be reduced, the darkening of the through image can be suppressed, making framing easier and reducing exposure errors. Furthermore, exposure for live view after imaging can be performed earlier.

[0103] Furthermore, the imaging system 1 estimates the movement trajectory of the headlight 52 during the charge holding period based on the charge holding time from the completion of simultaneous readout control until the charge is read out as an image signal from the charge holding unit, and the movement state of the headlight 52. It then moves the high-brightness area to which transmittance reduction control is performed in accordance with the estimated movement trajectory. With this configuration, the transmittance of only the high-brightness area tracked by the electronic ND filter 13 can be reduced in accordance with the movement of the high-brightness subject during the readout period in which the charge from the charge holding unit is sequentially read out as an image signal. As a result, PLS noise generated during the readout period in the GS method can be further suppressed. In addition, since the transmittance of only the tracked high-brightness area can be reduced, the darkening of the through image can be further suppressed, making framing easier and reducing exposure errors. Furthermore, exposure for live view after imaging can be performed earlier.

[0104] <Modified Control Processing During Imaging> Figure 13 is a flowchart showing a modified control processing of the system control unit 16 during imaging. As shown in Figure 13, the processing from step S11 to step S17 is the same as the processing from step S11 to step S17 described in Figure 4.

[0105] In this modified example, if a high-luminance subject is detected in step S13 (step S13: Yes), the system control unit 16 determines whether the subject's movement speed is greater than or equal to a predetermined value (step S31). The subject's movement speed refers to the movement speed of the subject within the captured image.

[0106] If, in step S31, the movement speed is not above a predetermined value (step S31: No), the system control unit 16 proceeds to step S14 and performs the main exposure to capture a still image. In other words, even if a high-brightness subject is detected, the system control unit 16 does not perform transmittance reduction control if the movement of the high-brightness subject is slow.

[0107] In step S31, if the movement speed is greater than or equal to a predetermined value (step S31: Yes), the system control unit 16 proceeds to step S18 and performs the main exposure to capture a still image. That is, the system control unit 16 performs transmittance reduction control when a high-brightness subject is detected and the movement speed of the high-brightness subject is greater than or equal to a predetermined value.

[0108] When the detected high-brightness subject is moving slowly or is not moving, the superposition of bright spots of the high-brightness subject during the readout process, where the image signal is sequentially read out, has little effect on the generation of PLS ​​noise. Therefore, when the high-brightness subject is moving slowly or is not moving, transmittance reduction control is not performed. This allows the brightness of the through image to be maintained, making framing easier and improving exposure performance.

[0109] The control method described in the above-mentioned embodiment can be implemented by executing a pre-prepared control program on a computer. This control program is recorded on a computer-readable storage medium and executed when read from the storage medium. This control program may also be provided in the form of a non-transient storage medium such as flash memory, or it may be provided via a network such as the Internet. The computer that executes this control program may be included in the imaging system, included in an electronic device such as a smartphone, tablet terminal, or personal computer that can communicate with the imaging system, or included in a server device that can communicate with these imaging systems and electronic devices.

[0110] <Mechanism of Electronic ND Filter 13> Figure 14 shows an example of the replacement mechanism for the electronic ND filter 13. The electronic ND filter 13 can be inserted into and removed from the optical path of light incident through the imaging lens 11. As shown in Figure 14, the imaging system 1 is equipped with a replacement mechanism that allows for the replacement of, for example, the electronic ND filter 13a provided in the light-adjusting unit 61a and the clear glass 13b provided in the light-adjusting unit 61b. The replacement of the light-adjusting unit 61a and the light-adjusting unit 61b by mechanical movement is driven by a DC motor 62 of an ND filter drive unit (not shown) connected to the system control unit 16. The light-adjusting unit 61a and the light-adjusting unit 61b are driven in the directions indicated by arrows B1 and B2, for example. The light-adjusting unit 61a and the light-adjusting unit 61b are replaced by being inserted into and removed from the optical path of the incident light by the drive of the DC motor 62. By moving the electronic ND filter 13 out of the optical path, attenuation of light as it passes through the electronic ND filter 13 can be avoided. In addition, discoloration of the electronic ND filter 13 caused by exposure to ultraviolet light can be suppressed.

[0111] <Other Embodiments> For example, the system control unit 16 may start transmittance reduction control based on a readout signal that instructs the image sensor 14 to read out to the charge holding unit during simultaneous readout control. The "readout signal" is a signal that defines the timing of global transfer. The system control unit 16 acquires the readout signal from the image sensor 14 and performs transmittance reduction control to lower the transmittance of the electronic ND filter 13 in synchronization with it. When it is time to read the charge to the charge holding unit, the system control unit 16 immediately performs control to maximize the filter density of the electronic ND filter 13 (to all black). By synchronizing the readout timing for reading the charge to the charge holding unit and the timing of the transmittance reduction control to lower the transmittance of the electronic ND filter 13 and linking the two processes, the processing can be simplified.

[0112] Furthermore, when performing transmittance reduction control in a high-brightness area, if the high-brightness subject is located in the first row and first column of the imaging range 50 shown in Figure 7, that is, if the high-brightness subject is located near the readout start position where it is sequentially read out as an image signal, the transmittance reduction control of the high-brightness area may be performed during the readout period of the area containing the high-brightness subject, and the transmittance reduction control of the high-brightness area may be stopped immediately after the readout of that area is completed. In that case, for example, in the sequence shown in Figure 3, the light reduction control signal 41 may be turned OFF in synchronization with the stop of the transmittance reduction control.

[0113] Furthermore, if the location of a high-brightness subject is, for example, in the second row and third column region of the imaging range 50 shown in Figure 7, that is, if the high-brightness subject is in the lower region of the imaging range 50, the readout process, in which the charge is sequentially read out as an image signal from the charge holding unit, may be started from the lower region of the imaging range 50. Similarly, in this case as well, the transmittance reduction control of the high-brightness area may be stopped immediately after the readout of the region containing the high-brightness subject is completed.

[0114] This invention can also be applied to programs and program products.

[0115] In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0116] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of programmable logic devices such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field Programmable Gate Array), dedicated circuits for executing specific processes such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, these multiple hardware components may reside in physically separate devices or in the same device. Furthermore, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware is composed of an electrical circuit (circuitry) or the like, which is a combination of circuit elements such as semiconductor elements.

[0117] Furthermore, the program may be firmware or software such as microcode. Alternatively, the program may be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on physically separate devices. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0118] Although various embodiments have been described above, it goes without saying that the present invention is not limited to these examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of the present invention. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the invention.

[0119] This application is based on a Japanese patent application (JP 2024-230232) filed on December 26, 2024, the contents of which are incorporated by reference within this application.

[0120] 1 Imaging system 10 Imaging device 11 Imaging lens 13, 13a Electronic ND filter 13b Clear glass 14 Image sensor 15 Digital signal processing unit 16 System control unit 17 EVF 18 Display LCD 19 Card storage unit 20 Operation unit 31a, 31b Transparent substrate 32a, 32b Transparent electrode 33a, 33b Alignment film 34a, 34b Sealing member 35 Liquid crystal layer 41 Control signal 42 Main exposure 43 Readout 47 Live view 50 Imaging range 51 Vehicle 52 Headlight 53 PLS noise 61a, 61b Dimming unit 62 DC motor t1, t2, t3 Time

Claims

1. An imaging device comprising: a light control device capable of controlling the transmittance of light incident through an imaging lens for each area; an image sensor capable of a first control that reads the accumulated charge into a charge holding unit by simultaneously exposing multiple pixels; and a processor, wherein the processor performs a high-brightness subject detection process by analyzing image data obtained by imaging performed before the first control; when a high-brightness subject is detected, it determines a first area of ​​the light control device corresponding to the region where the high-brightness subject exists; and after the first control, it performs a first transmittance control that reduces the transmittance of the first area in the light control device.

2. An imaging device according to claim 1, wherein the detection process includes the detection of the position and size of the high-luminance subject.

3. An imaging device according to claim 1, wherein the detection process includes detection of the movement state of the high-luminance subject.

4. An imaging apparatus according to claim 3, wherein the processor determines the first area based on the charge holding time from the completion of the first control until the charge is read out from the charge holding unit, and the movement state.

5. An imaging apparatus according to claim 4, wherein the processor determines the range of movement of the high-luminance subject during the period from the completion of the first control until the charge is read out from the charge holding unit, based on the charge holding time and the movement state, and defines the area encompassing the range of movement as the first area.

6. An imaging apparatus according to claim 4, wherein the processor estimates the movement trajectory of the high-luminance subject during the period from the completion of the first control until the charge is read out from the charge holding unit, based on the charge holding time and the movement state, and moves the first area in the first transmittance control based on the movement trajectory.

7. An imaging device according to claim 4, wherein the charge retention time varies depending on the region of the image sensor.

8. An imaging apparatus according to any one of claims 4 to 7, wherein the processor does not perform the first transmittance control according to the movement state.

9. A control method for an imaging device comprising: a light-adjusting device capable of controlling the transmittance of light incident through an imaging lens for each area; an image sensor capable of a first control that reads the accumulated charge into a charge-holding unit by simultaneously exposing multiple pixels; and a processor, wherein the processor performs a high-brightness subject detection process by analyzing image data obtained by imaging performed before the first control; when a high-brightness subject is detected, determines a first area of ​​the light-adjusting device corresponding to the region where the high-brightness subject exists; and after the first control, performs a first transmittance control that reduces the transmittance of the first area in the light-adjusting device.

10. A control program for an imaging device comprising: a light-adjusting device capable of controlling the transmittance of light incident through an imaging lens for each area; an image sensor capable of a first control that reads the accumulated charge into a charge-holding unit by simultaneously exposing multiple pixels; and a processor, wherein the control program causes the processor to perform a process of detecting a high-brightness subject by analyzing image data obtained by imaging performed before the first control; determining a first area of ​​the light-adjusting device corresponding to the region where the high-brightness subject is located if the high-brightness subject is detected; and performing a first transmittance control that reduces the transmittance of the first area in the light-adjusting device after the first control.

11. An imaging system comprising: an imaging lens; a light control device capable of controlling the transmittance of light incident through the imaging lens for each area; an image sensor capable of a first control that reads the charge accumulated by simultaneously exposing multiple pixels to a charge holding unit; and a processor, wherein the processor performs a high-brightness subject detection process by analyzing image data obtained by imaging performed before the first control; when a high-brightness subject is detected, it determines a first area of ​​the light control device corresponding to the region where the high-brightness subject exists; and after the first control, it performs a first transmittance control that reduces the transmittance of the first area in the light control device.

Images