Electronic equipment and its control method

Abstract

To create a high-quality image with a wider dynamic range, while reducing the number of times of photographing.SOLUTION: An electronic apparatus has: imaging means that applies at least one of a plurality of different gains to an image signal obtained by photoelectrically converting an optical image of a subject and outputs a resultant image signal; detection means that detects the luminance value of the image signal output from the imaging means; determination means that, based on the luminance value, determines whether to perform a plurality of times of photographing while changing a photography condition including the plurality of different gains for every photographing; decision means that decides the photography condition according to a result of determination made by the determination means; and control means that controls the imaging means to perform photographing based on the photography condition decided by the decision means.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an electronic device and a control method therefor, and particularly relates to a technology for generating high dynamic range images. [Background Art]

[0002] Conventionally, technologies for expanding the dynamic range related to gradation have been proposed using imaging elements such as CCD image sensors and CMOS image sensors that are used in common digital cameras.

[0003] In the imaging apparatus of Patent Document 1, a technology for expanding the dynamic range is disclosed, in which a plurality of different amplification gains are applied to signals of the same pixel, the signals are read out simultaneously and combined through signal processing. Further, Patent Document 2 discloses a technology for expanding the dynamic range, in which a plurality of images are captured by switching the number of capturing operations according to the detected state of a subject, capturing mode and the like, and the plurality of images are combined through signal processing. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Laid-Open No. 2015-128253 [Patent Document 2] Japanese Patent Laid-Open No. 2014-171146 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] When capturing is performed with an increased difference between amplification gains to further expand the dynamic range by the method disclosed in Patent Document 1, black crushing in pixels whose pixel signal is equal to or lower than a predetermined output level and blown-out highlights in pixels exceeding the predetermined output level become less likely to occur. On the other hand, gradation of pixels at intermediate output levels is lost.

[0006] Furthermore, if the number of shots is increased to further expand the dynamic range using the method disclosed in Patent Document 2, an unnatural composite image may be generated if the subject is moving.

[0007] This invention was made in view of the above-mentioned problems, and aims to generate high-quality images with a wider dynamic range while suppressing the number of shots taken. [Means for solving the problem]

[0008] To achieve the above objective, the electronic device of the present invention includes: an imaging means that outputs an image signal obtained by photoelectric conversion of an optical image of a subject by applying at least one of a plurality of different gains; a detection means that detects the brightness value of the image signal output from the imaging means; and a determination based on the brightness value whether to perform multiple shots, changing the shooting conditions including the plurality of different gains for each shot. death, judgment severance Depending on the result Decision The aforementioned shooting conditions in The imaging means but Take a photo Control it so that Control means and 、 to have The control means determines to perform multiple captures if the brightness difference between the bright and dark parts of the image signal is greater than or equal to a predetermined first threshold, and controls the imaging means to produce more types of image signals than when the brightness difference is greater than or equal to a second threshold greater than the first threshold. do. [Effects of the Invention]

[0009] According to the present invention, it is possible to generate high-quality images with a wider dynamic range while suppressing the number of shots taken. [Brief explanation of the drawing]

[0010] [Figure 1] A block diagram showing the configuration of an image processing apparatus in an embodiment of the present invention. [Figure 2] A block diagram showing an example of the circuit configuration of an image sensor according to an embodiment of the present invention. [Figure 3] A circuit diagram showing the configuration of each pixel according to an embodiment of the present invention. [Figure 4] A circuit diagram showing the configuration of an image sensor according to an embodiment of the present invention. [Figure 5] A flowchart illustrating imaging processing according to the first embodiment. [Figure 6] A flowchart illustrating imaging condition determination processing when the luminance difference is large according to the first embodiment. [Figure 7] A flowchart illustrating imaging condition determination processing when the luminance difference is small according to the first embodiment. [Figure 8] A diagram illustrating an example of imaging conditions according to the first embodiment. [Figure 9] A flowchart illustrating imaging processing according to the second embodiment. [Figure 10] A diagram illustrating an example of an HDR mode setting screen according to the third embodiment. [Figure 11] A diagram illustrating an example of screen display during imaging in HDR mode according to the third embodiment. [Figure 12] A flowchart illustrating imaging processing according to the third embodiment. [Figure 13] A flowchart illustrating imaging processing according to the fourth embodiment. [Figure 14] A timing chart illustrating imaging processing according to the fourth embodiment. [Figure 15] A timing chart illustrating change of recording frame rate by additional imaging according to the fourth embodiment. [Figure 16] A diagram illustrating a motion vector detection method according to the fourth embodiment. [Figure 17] A diagram illustrating an example of display according to the fourth embodiment. Description of Embodiments

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and a plurality of features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0012] <First Embodiment> ●Composition Figure 1 is a block diagram showing the configuration of the imaging device 100 in this embodiment. In Figure 1, the photographic lens 10 is an interchangeable lens unit that can be attached to the main body of the imaging device 100, or a lens unit incorporated into the main body, and consists of multiple lens groups such as a focus lens and a zoom lens, and an aperture mechanism. The mechanical shutter 12 has an aperture function and is used to control the amount of light incident on the image sensor 14.

[0013] The image sensor 14 is a CMOS image sensor having multiple pixels. It performs photoelectric conversion at each pixel to generate an image signal corresponding to the charge generated according to the amount of incident light, based on the optical image of the subject formed by the imaging lens 10. This signal is then given a gain and output. By outputting multiple image signals with different gains, it is possible to obtain multiple image signals with different signal levels in a single shot. The A / D converter 16 converts the analog signal output of the image sensor 14 into a digital signal. The A / D converter 16 may be built into the image sensor 14.

[0014] The timing generation unit 18 supplies clock signals and control signals to the image sensor 14 and the A / D converter 16, and is controlled by the memory control unit 22 and the system control unit 50. In addition to the mechanical shutter 12, the timing generation unit 18 can control the reset timing of the image sensor 14, allowing it to function as an electronic shutter with controlled storage time, which can be used for video recording and other applications.

[0015] The image processing unit 20 performs various signal processing, such as predetermined pixel interpolation, color conversion, gamma processing, and exposure compensation, on the image data from the A / D converter 16 or the image data from the memory control unit 22, and outputs the processed image data. Furthermore, the image processing unit 20 performs image cropping and scaling processing to realize an electronic zoom function.

[0016] The image processing unit 20 further generates an HDR (High Dynamic Range) image using an arbitrary synthesis method. For example, one method is to use a high-gain image signal for the image portion below a predetermined signal level (normal image), and a low-gain image signal for the image portion where the signal level exceeds the predetermined signal level and is bright and overexposed. It is preferable that the normal image used for the signal in the dark areas of the synthesized image has suppressed random noise in the dark areas.

[0017] The memory control unit 22 controls the A / D converter 16, the timing generation unit 18, the image processing unit 20, the memory 30, and the compression / decompression unit 32. Image data output from the A / D converter 16 is written to the memory 30 via the image processing unit 20 and the memory control unit 22, or directly via the memory control unit 22.

[0018] Memory 30 is a storage unit that stores the instructions given by the user to the imaging device 100, and is, for example, an electrically erasable and recordable non-volatile memory. That is, it is used to store captured still images and moving images, and has sufficient storage capacity to store a predetermined number of still images or moving images of a predetermined duration.

[0019] This enables high-speed and large-volume image writing to the memory 30, even in cases of continuous shooting or panoramic shooting where multiple still images are captured in sequence. Furthermore, the memory 30 can also be used as a workspace for the system control unit 50.

[0020] The image display unit 28 is for displaying captured images, information at the time of shooting, and a user interface for operation by the operation unit 70, and is composed of, for example, a TFT-LCD. The image data for display written to the memory 30 is displayed by the image display unit 28. By sequentially displaying the captured image data using the image display unit 28, an electronic viewfinder (EVF) function can be realized. In addition, the image display unit 28 can be arbitrarily turned ON / OFF by instruction from the system control unit 50, and when the display is turned OFF, the power consumption of the imaging device 100 can be significantly reduced.

[0021] The system control unit 50 controls the entire imaging device 100 and includes a brightness difference detection unit 107a, a motion detection unit 107b, and the like. The brightness difference detection unit 107a uses image data output from the A / D converter 16 to calculate the brightness difference between bright and dark areas. For example, it divides the entire screen into multiple areas, measures the brightness (luminance) for each area, and calculates the brightness difference between bright areas and dark areas. The motion detection unit 107b performs a process to detect the amount of motion of the subject from the image data of the current frame output from the A / D converter 16 and the image data of the previous frame.

[0022] The exposure control unit 40 calculates the exposure conditions to be used for image capture based on the calculation results from the image processing unit 20. Furthermore, as will be described later, during HDR shooting, the number of shots to be taken is determined based on the brightness difference of the subject calculated by the brightness difference detection unit 107a. In addition, the exposure conditions for shooting are determined using the calculation results from the image processing unit 20 and the amount of motion detected by the motion detection unit 107b, and control signals are transmitted to the timing generation unit 18 and the shutter 12 equipped with an aperture function.

[0023] The focus adjustment unit 42 controls the focusing of the photographic lens 10, and the zoom control unit 44 controls the zooming of the photographic lens 10. The flash 48 has an AF assist light projection function and a flash exposure control function.

[0024] Furthermore, the image processing unit 20 performs predetermined calculations using the image data output from the A / D converter 16. Based on the obtained calculation results, the system control unit 50 controls the exposure control unit 40 to perform automatic exposure (AE) processing. It also controls the flash 48 to perform flash pre-flash (EF) processing. Furthermore, it controls the focus adjustment unit 42 to perform TTL (through-the-lens) autofocus (AF) processing. In addition, it performs automatic white balance (AWB) processing based on the obtained calculation results.

[0025] The operation units 60, 62, 64, 66, 70, and 72 are used by the user to input instructions to the imaging device 100 and consist of one or more combinations of switches such as release buttons, mode switching dials, zoom operation levers, touch panels, eye-tracking pointing, and voice recognition devices. Hereinafter, a detailed explanation of these operation means will be given. User input via the operation units 60, 62, 64, 66, 70, and 72 is notified to the system control unit 50.

[0026] The Mode Dial 60 allows you to switch between various function modes, including power off, automatic shooting mode, shooting mode, panorama shooting mode, video shooting mode, playback mode, and PC connection mode.

[0027] The shutter switch SW1(62) turns ON during the operation of the shutter button (not shown) and instructs the start of operations such as AF processing, AE processing, and AWB processing.

[0028] The shutter switch SW2(64) turns ON when the shutter button (not shown) is fully pressed, instructing the start of a series of processes consisting of readout, development, and recording.

[0029] In the readout process, in the case of flash photography, after EF processing, the image sensor 14 is exposed for the exposure time determined by AE processing, and light is emitted during this exposure period. At the end of the exposure period, the exposure control unit 40 blocks the light, thereby ending the exposure to the image sensor 14. Then, the signal read from the image sensor 14 is written to the memory 30 via the A / D converter 16 and the memory control unit 22.

[0030] Next, development processing is performed using calculations in the image processing unit 20 and the memory control unit 22. Furthermore, recording processing is performed in which image data is read from the memory 30, compressed in the compression / decompression unit 32, and written to the recording medium 200.

[0031] The non-volatile memory 31 is composed of FlashROM or the like. The program code executed by the system control unit 50 is written to the non-volatile memory 31 and executed by sequentially reading it. In addition, the non-volatile memory 31 has areas for storing system information and user setting information, enabling the retrieval and restoration of various information and settings at the next startup.

[0032] The compression / decompression unit 32 compresses and decompresses image data using known compression methods such as adaptive discrete cosine transform (ADCT). The compression / decompression unit 32 also reads images stored in memory 30, performs compression or decompression processing, and writes the processed data back to memory 30.

[0033] The display switching switch 66 can switch the display of the image display unit 28. This function allows for power saving by cutting off the current supply to the image display unit 28 when taking pictures using the optical viewfinder 77.

[0034] The control unit 70 consists of various buttons, a touch panel, a rotary dial, etc., and includes, for example, a menu button, a set button, a macro button, a multi-screen playback page change button, a flash setting button, and a single shot / continuous shot / self-timer switch button. It also includes a menu movement + (plus) button, a menu movement - (minus) button, a playback image movement + (plus) button, a playback image movement - (minus) button, a shooting quality selection button, an exposure compensation button, a date / time setting button, etc.

[0035] The zoom switch 72 is used by the user to instruct the camera to change the magnification of the captured image. The zoom switch 72 consists of a tele-switch that changes the imaging angle of view to the telephoto side and a wide-angle switch that changes it to the wide-angle side. By using this zoom switch 72, the zoom control unit 44 is instructed to change the imaging angle of view of the shooting lens 10, triggering optical zoom operation. It also triggers electronic zooming changes of the imaging angle of view by the image processing unit 20, such as image cropping and pixel interpolation processing.

[0036] The thermistor 74 measures the temperature inside the imaging device 100. Since defective pixels in the image sensor 14 are affected by temperature, the defect correction process needs to be changed depending on the temperature during imaging. The thermistor 74 is placed near the image sensor 14 inside the imaging device 100 to measure the temperature of the image sensor 14 itself.

[0037] The power supply unit 86 consists of primary batteries such as alkaline batteries, secondary batteries such as NiCd batteries, NiMH batteries, or Li-ion batteries, an AC adapter, or an external battery. Interface 90 is an interface with recording media such as memory cards and hard disks, and connector 92 connects to recording media such as memory cards and hard disks.

[0038] The optical viewfinder 77 allows for shooting using only the optical viewfinder, without using the electronic viewfinder function provided by the image display unit 28. The communication unit 75 has various communication functions such as USB, IEEE1394, LAN, and wireless communication. 76 is a connector that connects the imaging device 100 to other devices via the communication unit 75, or an antenna in the case of wireless communication.

[0039] The recording medium 200 is a recording medium such as a memory card or a hard disk. This recording medium 200 includes a recording unit 202 made of semiconductor memory or a magnetic disk, an interface 204 with the imaging device 100, and a connector 206 for connecting to the imaging device 100.

[0040] Figure 2 shows an example of the circuit configuration of the image sensor 14. In the pixel section 250, multiple pixels 240 are arranged in a matrix such as pixels R1_1 to Bm_n (where m and n are arbitrary integers). In each pixel 240, R represents a red color filter, G represents a green color filter, and B represents a blue color filter. R(G,B)i_j indicates that the pixel 240 is the i-th row and j-th column pixel. Among the pixels 240 in the same column, pixels 240 in odd-numbered rows are connected to the vertical output line 252, and pixels 240 in even-numbered rows are connected to the vertical output line 256.

[0041] When the column circuit switch 255 is off, image signals from pixels in odd-numbered rows are input only to column circuit 253 via the vertical output line 252. When the column circuit switch 255 is on, the signals are input to both column circuit 253 and column circuit 257, respectively. Similarly, when the column circuit switch 255 is off, image signals from pixels in even-numbered rows are input only to column circuit 257 via the vertical output line 256. When the column circuit switch 255 is on, the signals are input to both column circuit 253 and column circuit 257, respectively. When the column circuit switch 255 is off, image signals from pixels in even-numbered and odd-numbered rows can be read out simultaneously. When it is on, image signals from pixels in even-numbered and odd-numbered rows are read out sequentially.

[0042] The drive signals RES, TX, SEL, and FDINC are supplied from the vertical scanning circuit 251 to the pixel unit 250, respectively. These drive signals are denoted as RES_m, TX_m, SEL_m, and FDINC_m, respectively, by adding a natural number variable m (=1, 2, ..., n) that represents the number corresponding to the row.

[0043] Here, we will explain the configuration of each individual pixel of the 240 pixels using Figure 3. The photodiode (PD) 301 converts incident light into photoelectric energy and stores charge corresponding to the amount of incident light. The transfer switch 302 transfers the charge stored in PD 301 to the floating diffusion (FD) unit 303 when the level of the signal tx is high. The FD unit 303 is connected to the gate of the transistor 304 that constitutes the source follower and converts the amount of charge transferred from PD 301 into a voltage value.

[0044] The reset switch 305 is a switch element for resetting the FD unit 303, and resets the FD unit 303 when the level of signal res is high. Also, when the levels of signal tx and signal res are both set to high, both the transfer switch 302 and the FD reset switch 305 are turned on, and the PD301 is reset via the FD unit 303.

[0045] The FDinc switch 307 connects the additional capacitor 308 to the FD unit 303 when the level of the signal fdinc is high. When the level of the signal fdinc is low, the FDinc switch 307 is off, and the additional capacitor 308 and the FD unit 303 are disconnected. This changes the capacitance generated in the FD unit 303. In other words, the FDinc switch 307 and the additional capacitor 308 function as variable capacitance means that make the capacitance value of the input node variable.

[0046] The FDinc switch 307 can be switched on or off to change the conversion ratio of the output voltage of transistor 304 to the charge transferred to FD section 303, i.e., the gain (hereinafter referred to as FD gain). When the additional capacitor 308 is connected to FD section 303, the capacitance value increases, and the gain becomes relatively smaller compared to the unconnected state. Conversely, when the additional capacitor 308 and FD section 303 are unconnected, the capacitance value decreases, and the gain becomes relatively larger compared to the connected state.

[0047] Thus, the image sensor 14 of this embodiment can vary the amount of charge held in the FD unit 303 according to the signal fdinc, and switch the output gain of the pixel 240.

[0048] The pixel selection switch 306 outputs the pixel signal converted to a voltage in the FD303 (vout) when the level of the signal sel is at a high level. The output of each pixel 240 is input to the column circuits 253 and 257 via the vertical output lines 252 and 256 for each column, as described above. In this embodiment, one vertical output line 252 or 256 is connected to one pixel 240, but multiple vertical output lines may be connected to one pixel 240. Furthermore, by turning on the column circuit switch 255, it becomes possible to connect one vertical output line to two column circuits 253 and 257, making it possible to amplify the output of one pixel 240 with multiple types of analog gain using the two column circuits 253 and 257.

[0049] Next, the configuration of the column circuit 253 will be described with reference to Figure 4. The output (vout) of one row of pixels 240 is input to the column circuit 253 via a vertical output line 252 connected to its output terminal. The vertical output lines 252 provided for each column are grounded via a current source 254. A source follower circuit is formed by the current source 254 and the transistor 304 of the pixel 240 connected to the vertical output line 252.

[0050] In the column circuit 253, the clamp capacitor 401 has capacitance C1, and the feedback capacitor 402 has capacitance C2. The output (vout) of the pixel unit 250 is input to the inverting input terminal of the operational amplifier 403 via the clamp capacitor 401. A reference voltage Vref is supplied to the non-inverting input terminal of the operational amplifier 403. A switch 404 is connected in parallel with the feedback capacitor 402, which is connected to the inverting input terminal and output terminal of the operational amplifier 403. The switch 404 is a switching element for shorting both ends of the feedback capacitor 402 and is controlled by the signal cfs. Since the gain setting can be changed by controlling the on / off state of the switch 404, the input pixel signal can be amplified and output with multiple types of analog gains.

[0051] The S-signal transfer switch 405 is a switch element for transferring the pixel signal (hereinafter referred to as "S-signal") read from the pixel unit 250 to the S-signal holding capacitor 407. By setting the level of the signal ts to a high level, the S-signal amplified by the operational amplifier 403 is held in the S-signal holding capacitor 407 via the S-signal transfer switch 405.

[0052] The N signal transfer switch 406 is a switch element for transferring the noise signal (hereinafter referred to as "N signal") read from the pixel unit 250 to the N signal holding capacitor 408. By setting the level of signal tn to a high level, the N signal amplified by the operational amplifier 403 is held in the N signal holding capacitor 408 via the N signal transfer switch 406.

[0053] The S signal and N signal held in the S signal holding capacitor 407 and N signal holding capacitor 408 are output from the output terminals vs and vn, respectively. The difference between the S signal and the N signal output from the output terminals vs and vn is then converted into a digital signal by an analog-to-digital conversion circuit (not shown). In the column circuit 253, drive signals such as cfs, ts, and tn are supplied from the timing generator (TG) 260 in Figure 2, according to the control of the timing generation unit 18 in Figure 1.

[0054] Furthermore, the column circuit 257 has the same configuration as the column circuit 253 and processes the output of one row of pixels 240 connected to the vertical output line 256.

[0055] Returning to Figure 2, the memory 261 holds the output data, which has been converted into a digital signal, according to the write signal memwr output from the TG260. The data held in the memory 261 is sequentially transferred to the data output unit 263 by scanning with the horizontal scanning circuit 262. The data output unit 263 outputs the data to the outside of the image sensor 14 using a transmission method such as LVDS (Low Voltage Differential Signaling).

[0056] ● HDR shooting processing Next, the processing in the system control unit 50 of the imaging device 100 according to an embodiment of the present invention during HDR shooting will be described with reference to Figure 5.

[0057] First, referring to the flowchart in Figure 5, we will explain the process of determining the shooting conditions using the photometric results and taking HDR images. This process starts when HDR shooting is instructed by the release button (not shown) included in the operation unit 70. In other words, by applying multiple gains to the image signal obtained from a single accumulation and reading it out, two images with different signal levels are acquired simultaneously. By combining these two images, a high dynamic range image called HDR can be generated.

[0058] In S201, a pre-shoot is performed with a predetermined exposure before the main shoot, and the image processing unit 20 performs photometry based on the image signal output from the image sensor 14. The image signal obtained at this time may be read from all pixels of the image sensor 14, or it may be read by decimation or addition, and is output with a predetermined gain. In addition, as will be described later, it is read repeatedly at predetermined intervals in order to detect the movement of the subject. In S202, the system control unit 50 receives the photometry result performed in S201 from the image processing unit 20, and the brightness difference detection unit 107a calculates the brightness difference between the bright area and the dark area of ​​the subject.

[0059] In S203, the exposure control unit 40 determines whether the brightness difference of the subject calculated in S202 is greater than or equal to a predetermined first threshold Thd1. If the brightness difference is greater than or equal to the first threshold Thd1, it is determined that the number of shots will be 2, including the additional shot, and the process proceeds to S204. If the brightness difference is less than the first threshold Thd1, it is determined that no additional shot will be taken, and the process proceeds to S208.

[0060] In S204, the shooting conditions for two sets of shots, including an additional shot, are determined. Details of how the specific shooting conditions are determined in S204 will be described later, referring to Figure 3.

[0061] In S205 and S206, the first and second shots are taken according to the two shooting conditions determined in S204. Specifically, the system control unit 50 controls the shooting lens 10 and image sensor 14 via the focus adjustment unit 42, zoom control unit 44, and timing generation unit 18 to acquire an image signal.

[0062] In S207, the image processing unit 20 combines the multiple image signals acquired in S205 and S206 to generate an HDR image, and the HDR shooting process ends.

[0063] On the other hand, in S208, no additional shooting is performed, so the shooting conditions for one shot are determined. For details on how the specific shooting conditions are determined in S208, please refer to Figure 8 and see the following explanation.

[0064] In S209, shooting is performed in the same way as in S205, according to the shooting conditions determined in S208. In S210, the image processing unit 20 combines the multiple image signals acquired in S209 to generate an HDR image, and the HDR shooting process is terminated.

[0065] ● How to determine shooting conditions Next, we will explain how to determine the shooting conditions by referring to the flowcharts in Figures 6 and 7. First, the method for determining the shooting conditions in S204 in Figure 5 will be explained using Figure 6.

[0066] In S401, it is determined whether the luminance difference of the subject calculated in S202 in Figure 5 is greater than or equal to a predetermined second threshold Thd2. The second threshold Thd2 is set to a value greater than the first threshold Thd1 used in S203. If it is greater than or equal to the second threshold Thd2 (large luminance difference), the process proceeds to S402. If it is less than the second threshold Thd2 and greater than or equal to the first threshold Thd1 (medium luminance difference), the process proceeds to S409.

[0067] In S402, the motion detection unit 107b calculates the motion vector component of the subject between the image used for photometry in S201 and the previously obtained image, which was obtained through pre-shooting. If the motion vector component is greater than or equal to a predetermined threshold Thv, it is determined that the subject is moving, and the process proceeds to S403. On the other hand, if the motion vector component is less than the threshold Thv, it is determined that the subject is not moving, and the process proceeds to S406.

[0068] In S403, the brightness of the main subject is determined based on the photometric results acquired by the image processing unit 20. If the photometric value of the main subject is greater than or equal to a predetermined threshold Thp, the main subject is judged to be bright, and the process proceeds to S404 to determine the shooting conditions as A1, which will be described later. On the other hand, if the photometric value of the main subject is less than the threshold Thp, the main subject is judged to be dark, and the process proceeds to S405 to determine the shooting conditions as A2, which will be described later.

[0069] In S406, the same judgment as in S403 is made. If the metering value of the main subject is above the threshold Thp, the main subject is judged to be bright, and the process proceeds to S407 to determine the shooting conditions as B1, which will be described later. On the other hand, if the metering value of the main subject is below the threshold Thp, the main subject is judged to be dark, and the process proceeds to S408 to determine the shooting conditions as B2, which will be described later.

[0070] In S409, the same judgment as in S402 is made. If the motion vector component of the subject is greater than or equal to a predetermined threshold Thv, it is determined that the subject is moving and the process proceeds to S410. If it is less than the threshold Thv, it is determined that the subject is not moving and the process proceeds to S413.

[0071] In steps S410 and S413, the same judgments as in S403 are made. If the photometric value of the main subject in S410 is equal to or greater than the threshold Thp, the system determines that the main subject is bright and proceeds to S411 to determine the shooting conditions as C1, which will be described later. If the value is less than the threshold Thp, the system determines that the main subject is dark and proceeds to S412 to determine the shooting conditions as C2, which will be described later.

[0072] Furthermore, if the photometric value of the main subject in S413 is equal to or greater than the threshold Thp, the system determines that the main subject is bright and proceeds to S414 to determine the shooting conditions as D1, which will be described later. If the value is less than the threshold Thp, the system determines that the main subject is dark and proceeds to S415 to determine the shooting conditions as D2, which will be described later.

[0073] Here, we will explain specific examples of shooting conditions A1 to D2 by referring to the table in Figure 8. In Figure 8, item 601 shows the shooting conditions A1 to D2 described in Figure 6. Item 604 shows the subject's state as determined in Figure 3: brightness difference, presence or absence of subject movement, and brightness of the main subject. Item 602 shows the specific details of the shooting conditions used for the first shot, and item 603 shows the specific details of the shooting conditions used for the second shot, where the shutter speed Tv and two types of gain 1 and gain 2 set on the image sensor 14 are shown.

[0074] As shown in Figure 8, if the brightness difference in S401 in Figure 6 is greater than or equal to the second threshold Thd2 (large brightness difference), the number of images output from the image sensor 14 is set to 4. If the brightness difference in S401 in Figure 6 is less than the second threshold Thd2 and greater than or equal to the first threshold Thd1 (medium brightness difference), the number of images output from the image sensor 14 is set to 3. In other words, since a maximum of 2 images can be acquired in one shooting, if the number of images output from the image sensor 14 is 3 or more, two shooting sessions are required.

[0075] Furthermore, when the subject is moving, the shutter speed is increased to reduce motion blur, and the gain is set according to the brightness of the main subject: low gain for bright subjects and high gain for dark subjects. Since high gain can increase noise and degrade image quality, it is desirable to shoot with the lowest possible gain. Therefore, when the subject is not moving, a lower gain is set than when the subject is moving, even at the same brightness, and the shutter speed is adjusted to accommodate subjects with different brightness levels.

[0076] When the brightness difference is moderate, three images are acquired through two exposures. Therefore, under exposure conditions C1 and C2, one image amplified with gain 1 is output during the first exposure, and under exposure conditions D1 and D2, one image amplified with gain 1 is output during the second exposure. In this case, the image sensor 14 outputs one image signal with one type of gain applied. If the image sensor 14 has two amplification units, only one of them needs to be used. Also, the number of images output by the image sensor 14 can be set to one during either the first or second exposure.

[0077] In step S207 of Figure 5, four or three image signals obtained under different shooting conditions are combined to generate an HDR image.

[0078] Next, using Figure 4, we will explain how to determine the imaging conditions in S208 when the brightness difference is judged to be less than the first threshold Thd1 (small brightness difference) in S203 of Figure 5. In S501, the brightness of the main subject is determined based on the photometric results acquired by the image processing unit 20. If the photometric value of the main subject is greater than or equal to a predetermined threshold Thp, the subject is judged to be bright, and the process proceeds to S502 to determine the shooting conditions as E1 shown in Figure 8. On the other hand, if the photometric value of the main subject is less than the threshold Thp, the subject is judged to be dark, and the process proceeds to S503 to determine the shooting conditions as E2 shown in Figure 5.

[0079] In S208, it is determined in S203 that the brightness difference is less than the first threshold Thd1, i.e., the brightness difference is small. Therefore, only one image is captured, and two images are output from the image sensor 14. Depending on the brightness of the main subject, the shooting conditions E1 and E2 used in a single capture are determined as shown in Figure 8.

[0080] As described above, according to the first embodiment, by determining the shooting conditions according to the state of the subject based on the photometric results, it is possible to generate a high-quality HDR image with a wider dynamic range while suppressing the number of shots and the number of images output from the image sensor 14.

[0081] In the example described above, the brightness difference was divided into three stages: large, medium, and small, and the number of images output from the image sensor 14 was reduced from four to two. However, the present invention is not limited to this, and it is also possible to use two stages without comparing with the second threshold Thd2, or conversely, to further divide the brightness value into four or more stages. In that case as well, the number of times required should be captured according to the number of images needed.

[0082] <Second Embodiment> Next, a second embodiment of the present invention will be described. Note that the imaging device in the second embodiment is the same as that described in the first embodiment with reference to Figure 1, so its description will be omitted. In the first embodiment described above, the case in which the shooting conditions for HDR shooting are determined based on the photometering results of the image obtained in pre-shooting was explained. In contrast, the second embodiment describes a case in which the shooting conditions are determined based on the image obtained in the first HDR shooting.

[0083] The following describes the process of determining the shooting conditions based on the image information obtained in the first HDR shoot and then taking an HDR image, referring to the flowchart in Figure 9. This process, like the process shown in Figure 2, starts when the release button (not shown) included in the operation unit 70 instructs the camera to take an HDR image.

[0084] In S301, similar to S201 described above, pre-shooting is performed before the main shooting, and the image signal output from the image sensor 14 is acquired by the image processing unit 20, and photometering is performed.

[0085] In S302, the system control unit 50 receives the photometering results from the image processing unit 20 performed in S301 and determines the conditions for the first shot. In this embodiment, the conditions for the first shot are determined based on the bright areas of the photometering results, and the aperture control value, shutter speed, gain 1, and gain 2 (low gain and high gain) are determined accordingly.

[0086] In S303, according to the first shooting conditions determined in S302, the system control unit 50 controls the shooting lens 10 and image sensor 14 via the focus adjustment unit 42, zoom control unit 44, and timing generation unit 18 to perform the first shooting and acquire an image signal.

[0087] In S304, the system control unit 50 receives the first image signal captured in S303 from the image processing unit 20, and the brightness difference detection unit 107a calculates the brightness difference between the bright and dark areas of the subject. The image sensor 14 outputs image signals amplified with gain 1 and gain 2, but the calculation is performed using either one or both. Alternatively, the calculation may be performed using image signals amplified with gain 1 and gain 2.

[0088] In S305, the exposure control unit 40 determines whether the brightness difference of the subject calculated in S304 is greater than or equal to a predetermined third threshold Thd3. If the brightness difference is greater than or equal to the third threshold Thd3, it is decided to perform additional shooting and proceeds to S306. If the brightness difference is less than the third brightness difference, it is decided not to perform additional shooting and proceeds to S309.

[0089] In S306, the conditions for a second additional shot are determined based on the shooting conditions of the image taken in S303 and the brightness difference calculated in S304. In this embodiment, since the conditions for the first shot performed in S303 are determined based on the bright areas of the subject, the conditions for the second shot are set to compensate for the lack of gradation in the dark areas that was insufficient in the first shot.

[0090] In S307, a second image is taken. In this shooting process, the system control unit 50 controls the imaging lens 10 and image sensor 14 via the focus adjustment unit 42, zoom control unit 44, and timing generation unit 18 according to the shooting conditions determined in S306, and acquires an image signal. Then, in S308, the image processing unit 20 combines the multiple image signals acquired in the two shooting processes in S303 and S307 to generate an HDR image, and the process ends.

[0091] For the second shot, the shooting conditions may be set to two different gains for the image sensor 14, or the same gain may be set, as needed. If the same gain is set, and the image sensor 14 has two amplification units, either one may be used.

[0092] Meanwhile, in S309, the image processing unit 20 combines the multiple image signals acquired in the shooting process of S303 to generate an HDR image, and then the process ends.

[0093] As described above, according to this second embodiment, by determining whether or not to take a second shot depending on the state of the subject in the image obtained in the first HDR shot in S302, it is possible to generate a high-quality HDR image with a wider dynamic range while suppressing the number of shots.

[0094] In the embodiments described above, two additional shots are given as an example, but the present invention is not limited to two shots, and three or more (multiple) shots may be taken.

[0095] <Third Embodiment> Next, a third embodiment of the present invention will be described.

[0096] In the third embodiment, the user interface in this embodiment will be described with reference to Figures 10 and 11. Note that, as in the second embodiment, the imaging device 100 described in Figure 1 can be used, so its description will be omitted here.

[0097] Figure 10 shows an example of a user interface for setting whether or not to allow additional shooting in HDR mode by operating the control unit 70 on the imaging device 100. In this embodiment, on the HDR menu screen 701, the user can select and set either the button 702 to allow additional shooting or the button 703 to disallow additional shooting. The set information is stored in the non-volatile memory 31.

[0098] Figure 11 shows the display method when additional shooting is performed during HDR shooting. In this embodiment, when it is determined that additional shooting is necessary, a message display 802 is displayed on the display screen 801 during HDR shooting, as instructed by the system control unit 50.

[0099] Next, the HDR shooting process in the third embodiment will be described using Figure 12. Figure 12 shows the imaging process shown in Figure 5, which was described in the first embodiment, with the addition of processing related to whether or not additional shooting is possible. Hereafter, the same step numbers will be used for processes similar to those shown in Figure 5, and explanations will be omitted as appropriate.

[0100] In S203, the exposure control unit 40 determines whether the brightness difference of the subject calculated in S202 is greater than or equal to the first threshold Thd1. If the brightness difference is greater than or equal to the first threshold Thd1, the process proceeds to S901; if the brightness difference is less than the first threshold Thd1, the process proceeds to S208.

[0101] In S901, the settings related to additional shooting stored in the non-volatile memory 31 are retrieved, and the process proceeds to S902. In S902, depending on the settings related to additional shooting retrieved in S901, the process proceeds to S903 if additional shooting is permitted, and to S208 if additional shooting is not permitted.

[0102] In S903, the system control unit 50 displays the message 802 shown in Figure 11 on the image display unit 28, and the process proceeds to S204. In this embodiment, the case in which a message is displayed on the image display unit 28 has been described, but the present invention is not limited to this, and notification may also be given using, for example, sound or light-emitting parts such as LEDs.

[0103] In the example shown in Figure 12, the case where the shooting conditions for two shots are determined using the photometric results was described, similar to the first embodiment. However, the present invention is not limited to this, and may also be applied when deciding whether or not to perform additional shooting based on the conditions of the image obtained in the first shot, as described in the second embodiment. In that case, if the result in S305 in Figure 9 is YES, the processes S901 to S903 shown in Figure 12 should be performed before processing S306.

[0104] As described above, according to the third embodiment, in addition to the same effects as the first and second embodiments, the ability to allow or deny additional shooting by the user allows for shooting that better reflects the user's intentions.

[0105] In the embodiments described above, an example was explained in which two images with different brightness levels are obtained by applying two types of gain to the image signal obtained in a single capture, and an HDR image is generated by combining these two images. However, the present invention is not limited to two types, and three or more types of gain may be applied.

[0106] <Fourth Embodiment> Next, a fourth embodiment of the present invention will be described.

[0107] Figure 13 is a flowchart of the imaging process in the fourth embodiment. In S1401, the image signal obtained from a single accumulation is read out with multiple gains applied, thereby acquiring two images with different signal levels simultaneously. By combining these two images, a high dynamic range image can be generated through HDR shooting. In S202, the brightness difference and motion vector of the subject in the image acquired in S201 are detected.

[0108] In S1403, it is determined whether the brightness difference detected in S1402 exceeds the threshold bTh, and whether the dynamic range of the subject is insufficient under the condition that two images were taken simultaneously. If the brightness difference exceeds the threshold bTh, it is determined that additional shooting is necessary, and the process proceeds to S1404, where it is determined whether the motion vector of the subject acquired in S1402 is smaller than the threshold vTh.

[0109] If the motion vector of the subject is smaller than the threshold vTh, the composite blur from additional shooting will not be noticeable, so the system decides to take additional shots and proceeds to S1405. On the other hand, if the brightness difference in S1403 is less than or equal to the threshold bTh, the system decides that the dynamic range of the subject is sufficient, and if the motion vector in S1404 is greater than or equal to the threshold vTh, the system decides not to take additional shots because the composite blur from additional shooting will be noticeable, and returns to S1401.

[0110] In S1405, it is determined whether there is time to insert additional shooting. If the recording frame rate does not decrease even with additional shooting, the brightness difference is determined again in S1406. If the brightness difference is greater than the threshold bTh2, it is a scene with a very large difference in brightness, so the process proceeds to S1407 to perform additional shooting with multiple gains. If the brightness difference is less than or equal to the threshold bTh2, the process proceeds to S1408 to perform additional shooting with a single gain. These additional shooting procedures will be described in detail later using Figures 14 and 15.

[0111] On the other hand, if it is determined in S1405 that there is not enough time to take additional shots, the recording frame rate will decrease if additional shots are taken, so the recording frame rate is changed in S1409 and additional shots are taken in S1410. This process of changing the recording frame rate will be described in detail later with reference to Figure 17. By performing the above actions in sequence, HDR video recording is achieved.

[0112] Figure 14 is a timing chart showing additional imaging with multiple gains in this embodiment. Figure 14(a) shows the timing chart when it is decided not to take additional images in S1403 or S1404. Each frame is stored once, the obtained pixel signals are read out with two types of gain (H / L), and the two H / L images are combined to generate an HDR image.

[0113] Figure 14(b) shows a timing chart for performing additional imaging in S1407 when it is determined in S1405 that there is time to perform additional imaging. The determination of whether there is time to perform additional imaging (second imaging operation) is made by whether it is possible to add readout frames between readout frames (first imaging operation). In Figure 15(b), H1 / L1 are the original readout frames, and H2 / L2 are the readout frames to be additionally captured. By combining the four images H1 / L1 / H2 / L2 obtained in this way, a high dynamic range image can be generated.

[0114] In Figure 14(b), four types of images are acquired by changing the storage time between normal shooting and additional shooting. However, depending on the shooting conditions, additional shooting may be performed by keeping the storage time the same and only changing the readout gain. As a specific example, H1=ISO400, L1=ISO100 H2=ISO1600, L2=ISO6400 The image is taken using this combination. You can also try changing both the storage time and the gain during shooting. Similarly, in the next frame, the four images H3 / L3 / H4 / L4 are combined.

[0115] Figure 14(c) shows the timing chart for when additional imaging is performed in S1408 if it is determined in S1406 to perform additional imaging with single gain. In Figure 14(c), H1 / L1 are the original readout frames, and M2 is the readout frame from the additional imaging. By combining the three images H1 / L1 / M2 obtained in this way, a high dynamic range image can be generated.

[0116] As shown in Figure 14(b), combining four images can generate an image with a very wide dynamic range, but the number of images read out doubles, resulting in a significant increase in power consumption. In contrast, additional shooting with a single gain allows for securing the necessary dynamic range while minimizing the increase in power consumption when the brightness difference is slightly insufficient with normal shooting.

[0117] Figure 15 is a timing chart showing the change in the recording frame rate due to additional shooting in this embodiment, and shows the processing when there is no time to take additional shots in S1405. As shown in Figure 15(a), if there is not enough time to insert additional shots between frames, the frame rate of the composite images is changed by changing the recording frame rate, thereby enabling additional shots. The recording frame rate can be changed by modifying the frame rate information of the frame in question, or by outputting the same image twice, so that the frame rate appears to change.

[0118] Figure 15(b) is a timing chart showing the case where additional shots are taken by reducing the recording frame rate. By reducing the recording frame rate so that it is possible to acquire four images H1 / L1 / H2 / L2, the frame rate of the combined image is halved.

[0119] Figure 15(c) is a timing chart showing the case where additional images are taken using a single gain by reducing the recording frame rate. By reducing the recording frame rate so that it is possible to acquire three images H1 / L1 / M2, the frame rate of the combined image is halved. As explained in Figure 14(c), this is a method to secure the necessary dynamic range while suppressing the increase in power consumption when the dynamic range is slightly insufficient.

[0120] Figure 16 is a diagram illustrating the motion vector detection method of this embodiment. Motion vectors can be estimated by detecting the difference between images. When detecting motion vectors between H1 / L1 frames and H2 / L2 frames, in this embodiment, detection is performed between H1 / H2 frames that are close to the optimal image. By converting each of the H1 / H2 frames to the required resolution and calculating the sum of the absolute values ​​of the differences between the images, motion vectors S can be detected as shown in Figure 16. Furthermore, if the calculated motion vector S exceeds the threshold bTh, the motion can be considered large.

[0121] If there is an exposure difference between frames in which motion vectors S are detected, it is necessary to normalize the exposure difference between images. A normalization coefficient Gain is calculated from the exposures Tv1 / Av1 / Sv1 of image H1 and Tv2 / Av2 / Sv2 of image H2. Then, by multiplying the H2 image by the calculated normalization coefficient Gain, it can be treated as an image with the same exposure as image H1, and motion vectors can be detected between images.

[0122] Figure 17 is a diagram showing an example of the display in this embodiment. The "A: Current number of composite images" display shows the number of images being used for the composite, as explained in Figures 14(b) and 14(c). Through this display, users can see how many images the camera has captured and is currently compositing. The "B: Current Recording Rate" display represents the recording frame rate, as explained in Figures 15(b) and 15(c). Through this display, users can verify whether the recording frame rate is as intended, depending on the dynamic range required by the camera. If these displays appear in a way that is not what the user intended, the user can change the settings to ensure that they can take photos as intended.

[0123] As described above, according to the fourth embodiment, even when images are being taken continuously, an image with an expanded dynamic range can be obtained.

[0124] Although the above-described embodiment described the application to an imaging device, the present invention is not limited to this. The present invention can be applied to various electronic devices as long as they are capable of dynamic range expansion synthesis by setting different gains, for example, smartphones, camcorders, game consoles with cameras, etc.

[0125] Furthermore, the embodiments described above can be used in combination, either partially or entirely.

[0126] <Other Embodiments> Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0127] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0128] 10: Imaging lens, 14: Image sensor, 16: A / D converter, 18: Timing generation unit, 20: Image processing unit, 22: Memory control unit, 28: Image display unit, 30: Memory, 40: Exposure control unit, 50: System control unit, 60: Mode dial switch, 70: Operation unit, 75: Communication unit, 76: Connector (or antenna), 90: Interface, 92: Connector, 100: Imaging device, 107a: Brightness difference detection unit, 107b: Motion detection unit, 200: Recording medium, 202: Recording unit, 204: Interface, 206: Connector

Claims

1. An imaging means that outputs an image signal obtained by photoelectric conversion of the optical image of a subject by applying at least one of several different gains, A detection means for detecting the brightness value of the image signal output from the imaging means, The system includes a control means that determines, based on the brightness value, whether to perform multiple shots while changing the shooting conditions, including the multiple different gains, for each shot, and controls the imaging means to perform the shot under the shooting conditions determined according to the determination result, The control means determines to perform multiple shots when the luminance difference between the bright and dark parts of the image signal is greater than or equal to a predetermined first threshold, and controls the system so that the number of types of image signals output from the imaging means is greater than when the luminance difference is greater than or equal to a second threshold which is greater than the first threshold.

2. An imaging means that outputs an image signal obtained by photoelectric conversion of an optical image of a subject by applying at least one of a plurality of different gains, A detection means for detecting the brightness value of the image signal output from the imaging means, The system includes a control means that determines, based on the brightness value, whether to perform multiple shots while changing the shooting conditions, including the multiple different gains, for each shot, and controls the imaging means to perform the shot under the shooting conditions determined according to the determination result, The control means is characterized by controlling the multiple different gains to be smaller when the brightness value of the main subject is above a predetermined threshold than when it is below the predetermined threshold.

3. An imaging means that outputs an image signal obtained by photoelectric conversion of an optical image of a subject by applying at least one of a plurality of different gains, A detection means for detecting the brightness value of the image signal output from the imaging means, Motion detection means for detecting the movement of a subject, The system includes a control means that determines, based on the brightness value, whether to perform multiple shots while changing the shooting conditions, including the multiple different gains, for each shot, and controls the imaging means to perform the shot under the shooting conditions determined according to the determination result, The control means is characterized by determining the shooting conditions for each shooting based on the movement of the subject and the brightness value of the main subject.

4. The electronic device according to claim 3, characterized in that the control means controls the shutter speed included in the shooting conditions to be faster when the movement of the subject is greater than or equal to a predetermined movement than when the movement is less than the predetermined movement.

5. The electronic device according to claim 4, characterized in that the control means controls the shutter speed to be slower than when the brightness value of the main subject is less than the predetermined threshold, and to reduce the plurality of different gains when the movement of the subject is smaller than the predetermined movement, and when the brightness value of the main subject is greater than or equal to a predetermined threshold, compared to when the brightness value of the main subject is less than the predetermined threshold, and when the movement of the subject is greater than or equal to the predetermined movement.

6. An imaging means that outputs an image signal obtained by photoelectric conversion of an optical image of a subject by applying at least one of a plurality of different gains, A detection means for detecting the brightness value of the image signal output from the imaging means, The system includes a control means that determines, based on the brightness value, whether to perform multiple shots while changing the shooting conditions, including the multiple different gains, for each shot, and controls the imaging means to perform the shot under the shooting conditions determined according to the determination result, The control means further determines a first shooting condition based on the brightness value of the bright areas in the image signal obtained by pre-shooting with the imaging means, determines to perform multiple shots if the brightness difference between the bright areas and the dark areas in the image signal obtained by shooting with the first shooting condition is greater than or equal to a predetermined first threshold, and determines a second shooting condition based on the brightness value of the dark areas in the image signal obtained by shooting with the first shooting condition when it is determined to perform multiple shots.

7. It further has operating means, The electronic device according to any one of claims 1 to 3, 6, characterized in that, when the operating means gives an instruction not to take multiple shots, the control means determines not to take multiple shots regardless of the brightness value.

8. The electronic device according to claim 7, characterized in that, when the operating means gives an instruction not to take multiple shots, the notification means notifies that multiple shots will not be taken.

9. The electronic device according to any one of claims 1 to 3, 6, further comprising a synthesis means for synthesizing image signals of a plurality of images output from the imaging means based on the shooting conditions determined by the control means.

10. It further has a means of display, The electronic device according to any one of claims 1 to 3, 6, characterized in that the display means displays the number of images obtained under the shooting conditions determined by the control means.

11. An imaging means that outputs an image signal obtained by photoelectric conversion of an optical image of a subject by applying at least one of a plurality of different gains, A detection means for detecting the brightness value of the image signal output from the imaging means, The system includes a control means that determines, based on the brightness value, whether to perform multiple shots while changing the shooting conditions, including the multiple different gains, for each shot, and controls the imaging means to perform the shot under the shooting conditions determined according to the determination result, The control means controls the imaging means to repeatedly perform a first shooting operation in which the image signal is output with at least one of a plurality of different gains applied, and when it is determined that multiple shooting operations should be performed, it controls the imaging means to perform a second shooting operation using different shooting conditions from the first shooting operation between consecutive first shooting operations. The electronic device is characterized in that the multiple shooting operations consist of one first shooting operation and one second shooting operation.

12. The electronic device according to claim 11, characterized in that the control means determines whether there is time to perform the second shooting operation between consecutive first shooting operations, and if it is determined that there is no time to perform the second shooting operation between consecutive first shooting operations, controls the imaging means to change the interval at which it performs the first shooting operation so that it can perform the second shooting operation.

13. The system further includes motion detection means for detecting the movement of a subject in an image obtained by the aforementioned continuous first shooting operation, The electronic device according to claim 11 or 12, further characterized in that the control means determines to perform the second shooting operation when the movement is smaller than a predetermined threshold.

14. The electronic device according to claim 11 or 12, further comprising a combining means for combining the image signals output from the imaging means by the first shooting operation and the second shooting operation performed following the first shooting operation, in order to expand the dynamic range.

15. The electronic device according to claim 14, further comprising recording means for recording the image signal of the image obtained by the first shooting operation when the second shooting operation is not performed, and for recording the image signal of the image synthesized by the synthesis means when the second shooting operation is performed.

16. It further has a means of display, The electronic device according to claim 15, characterized in that the display means displays the frame rate at which the image signal is recorded by the recording means.

Images