Imaging device and electronic apparatus

Abstract

Provided is an imaging device with which individual adjustment in a wafer process or on the user side can be reduced and correction accuracy improvement based on real-time correction value calculation can be achieved. The imaging device comprises a pixel array unit, a compositing gain calculation unit, and a compositing unit. The pixel array unit is formed by arranging, in a matrix pattern, a plurality of pixel units that each output a first pixel signal having a first sensitivity and a second pixel signal having a second sensitivity lower than the first sensitivity. The compositing gain calculation unit calculates a compositing gain by which the second pixel signal is to be multiplied. The compositing unit composites the first pixel signal and the second pixel signal multiplied by the compositing gain. At least some of the plurality of pixel units form captured images for one plane of effective frames. In accordance with a preset condition, the compositing gain calculation unit makes a gradual change from a first adding-up range, in which pieces of data for compositing gain calculation are added up, to a second adding-up range differing from the first adding-up range.

Description

Imaging devices and electronic devices

[0001] The present disclosure relates to an imaging device and an electronic device including the imaging device.

[0002] In order for an imaging device to obtain good image quality in an environment with a large difference between light and dark (brightness difference), it is required to have a wide dynamic range, and various types of dynamic range expansion techniques have been proposed. Among them, a sub-pixel structure that has large and small pixels with different sensitivities and is equipped with a capacitor for storing electric charges generated in the small pixels is known as a structure for expanding the dynamic range of pixels (for example, Patent Document 1).

[0003] For example, the time-division method is a technology that achieves a high dynamic range (DR) by combining four types of pixel signals with different sensitivities, such as SP1H / L and SP2H / L, outputted in a time-division manner using light-receiving elements with different sensitivities, while the space-division method is a technology that achieves a high DR by combining pixel signals outputted from light-receiving elements with different sensitivities.

[0004] Japanese Patent Application Laid-Open No. 2005-269339

[0005] The technology described in Patent Document 1 requires that the sensitivity ratio between each pixel signal be measured in advance. Furthermore, in previous technology, when combining multiple pixel signals with different sensitivities, the sensitivity ratio between the pixel signals to be combined is applied as a correction value to one of the pixel signals. This correction value is calculated by integrating the signal levels of the pixel signals to be combined. However, if there is no signal within the integration range used to calculate the correction value immediately after startup, an unstable correction value may continue to be generated.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide an imaging device and electronic device that can improve correction accuracy by calculating correction values ​​in real time and can stably calculate correction values ​​even immediately after startup.

[0007] One aspect of the present disclosure is an imaging device comprising: a pixel array section in which a plurality of pixel units are arranged in a matrix, each pixel unit outputting a first pixel signal having a first sensitivity and a second pixel signal having a second sensitivity lower than the first sensitivity; a composite gain calculation section that calculates a composite gain to be multiplied by the second pixel signal; and a combination section that combines the second pixel signal multiplied by the composite gain and the first pixel signal, wherein at least a portion of the plurality of pixel units constitute an captured image of one effective frame, and the composite gain calculation section gradually changes, in accordance with preset conditions, from a first integration range in which data for calculating the composite gain is integrated to a second integration range different from the first integration range.

[0008] Another aspect of the present disclosure is an electronic device including an imaging device comprising: a pixel array section including a plurality of pixel units arranged in a matrix, each pixel unit outputting a first pixel signal having a first sensitivity and a second pixel signal having a second sensitivity lower than the first sensitivity; a composite gain calculation section that calculates a composite gain to be multiplied by the second pixel signal; and a combination section that combines the second pixel signal multiplied by the composite gain and the first pixel signal, wherein at least a portion of the plurality of pixel units constitute a captured image of one effective frame, and the composite gain calculation section gradually changes, in accordance with preset conditions, a first integration range in which integration of data for calculating the composite gain is performed to a second integration range different from the first integration range.

[0009] 5 is a block diagram showing an example of a schematic configuration of an imaging device according to a first embodiment of the present disclosure. FIG. 6 is a circuit diagram showing an example configuration of a pixel arranged in the pixel array unit of FIG. 1. FIG. 7 is a timing chart showing operation at the start of exposure of a pixel. FIG. 8 is a timing chart showing operation at the time of pixel readout. FIG. 9 is a block diagram showing an example configuration of a signal processing unit of FIG. 1. FIG. 10 is a circuit diagram of the HDR synthesis unit of FIG. 5. FIG. 6 is a characteristic diagram showing the relationship between a low-sensitivity pixel signal, a high-sensitivity pixel signal, and a corrected low-sensitivity pixel signal with respect to the amount of incident light. FIG. 7 is a block diagram of a correction gain generation circuit according to a first embodiment of the present disclosure. FIG. 8 is a diagram showing timing of variable setting of an integrated effective pixel range by the correction gain generation circuit according to the first embodiment of the present disclosure. FIG. 9 is a flowchart showing a control procedure and control content for variable setting of an integrated effective pixel range by the correction gain generation circuit according to the first embodiment of the present disclosure. FIG. 9 is a block diagram of a correction gain generation circuit provided in an imaging device according to a second embodiment of the present disclosure. FIG. 10 is a diagram showing timing of variable setting of an integrated effective pixel level range by the correction gain generation circuit according to the second embodiment of the present disclosure. FIG. 11 is a flowchart showing a control procedure and control content for variable setting of an integrated effective pixel level range by the correction gain generation circuit according to the second embodiment of the present disclosure. 19 is a block diagram of a correction gain generation circuit provided in an imaging device according to a third embodiment of the present disclosure. FIG. 20 is a plan view showing how color filters are arranged in unit pixels in the third embodiment of the present disclosure. FIG. 21 is a diagram showing operation timings by the correction gain generation circuit according to the third embodiment of the present disclosure. FIG. 22 is a flowchart showing a control procedure and control contents for integrating a composite gain by the correction gain generation circuit according to a fourth embodiment of the present disclosure. FIG. 23 is a block diagram showing an example configuration of an imaging system as an electronic device to which the present technology is applied. FIG. 24 is a block diagram showing a schematic example configuration of a vehicle control system which is an example of a mobile body control system to which the technology according to the present disclosure can be applied. FIG. 25 is a diagram showing an example installation position of the imaging unit shown in FIG.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings referred to in the following description, identical or similar parts will be designated by identical or similar reference numerals, and duplicate explanations will be omitted. Note that the effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be present.

[0011] <First Embodiment> (Overall Configuration of Imaging Device) Fig. 1 is a block diagram showing an example of a schematic configuration of an imaging device according to a first embodiment of the present disclosure. The imaging device 1 is a semiconductor device configured as, for example, a CMOS image sensor, which uses photoelectric conversion elements such as photodiodes constituting each pixel to convert an amount of charge corresponding to the intensity of light imaged on the pixel into an electrical signal and outputs this as image data. The imaging device 1 can be configured as an integrated system-on-chip (SoC) such as a CMOS LSI, for example, but may also be configured with several components shown below as separate LSIs, for example.

[0012] As shown in the figure, the imaging device 1 is configured to include components such as a pixel array section 11, a vertical driving section 12, a column processing section 13, a horizontal driving section 14, a system control section 15, a signal processing section 16, and a data storage section 17.

[0013] The pixel array section 11 includes a group of photoelectric conversion elements such as photodiodes that constitute unit pixels 110 (an example of a pixel unit) that are arrayed in the horizontal direction (row direction) and the vertical direction (column direction). The pixel array section 11 converts the amount of charge corresponding to the intensity of incident light that is imaged on each unit pixel 110 into an electrical signal and outputs it as a pixel signal. Note that the unit pixels 110 may have a subpixel structure including large pixels and small pixels. Furthermore, a captured image of one effective frame is formed by pixel signals output from at least some of the multiple unit pixels 110.

[0014] The vertical drive unit 12 is configured to include a shift register, an address decoder, etc. The vertical drive unit 12 supplies drive signals, etc. to each unit pixel 110 via a plurality of pixel drive lines 18, thereby driving each unit pixel 110 of the pixel array unit 11, for example, simultaneously or row by row.

[0015] The column processing unit 13 reads out pixel signals from each pixel for each pixel column of the pixel array unit 11 via vertical signal lines (VSL) 19, and performs noise removal processing, correlated double sampling (CDS) processing, A / D (Analog-to-Digital) conversion processing, etc. The pixel signals processed by the column processing unit 13 are output to the signal processing unit 16.

[0016] The horizontal driving unit 14 is configured to include a shift register, an address decoder, etc. The horizontal driving unit 14 sequentially selects the unit pixels 110 corresponding to the pixel columns of the column processing unit 13. By the selective scanning by this horizontal driving unit 14, pixel signals processed for each unit pixel 110 in the column processing unit 13 are output sequentially to the signal processing unit 16.

[0017] The system control unit 15 includes a timing generator that generates various timing signals, etc. The system control unit 15 controls the driving of the vertical driving unit 12, the column processing unit 13, and the horizontal driving unit 14 based on timing signals generated by, for example, a timing generator (not shown).

[0018] The signal processing unit 16 performs signal processing such as arithmetic processing on the pixel signals supplied from the column processing unit 13, while temporarily storing data in the data storage unit 17 as necessary, and outputs an image signal of one effective frame based on each pixel signal.

[0019] It should be noted that the imaging device 1 to which the present technology is applied is not limited to the configuration described above. For example, the imaging device 1 may be configured such that the data storage unit 17 is disposed after the column processing unit 13, and pixel signals output from the column processing unit 13 are supplied to the signal processing unit 16 via the data storage unit 17. Alternatively, the imaging device 1 may be configured such that the column processing unit 13, the data storage unit 17, and the signal processing unit 16 are cascade-connected, and process the pixel signals in parallel.

[0020] (Circuit Configuration of Unit Pixel) Fig. 2 is a circuit diagram showing an example configuration of a unit pixel 110 arranged in the pixel array section 11 of Fig. 1. The unit pixel 110 is configured to include a first photoelectric conversion section 111, a first transfer gate section 112, a second photoelectric conversion section 113, a second transfer gate section 114, a third transfer gate section 115, a charge accumulation section 116, a reset gate section 117, an FD (floating diffusion) section 118, an amplification transistor 119, and a selection transistor 120.

[0021] Furthermore, a plurality of drive lines are wired to the unit pixels 110 as pixel drive lines 18, for example, for each pixel row. Various drive signals TGL, TGS, FCG, RST, and SEL are supplied from the vertical drive unit 12 via the plurality of drive lines. Because each transistor in the unit pixel 110 is an NMOS transistor, these drive signals are pulse signals that are active when at a high level (for example, power supply voltage VDD) and inactive when at a low level (for example, negative potential).

[0022] The first photoelectric conversion unit 111 is, for example, a PN junction photodiode. The first photoelectric conversion unit 111 generates and accumulates electric charges according to the amount of light received. The first transfer gate unit 112 is connected between the first photoelectric conversion unit 111 and the FD unit 118. A drive signal TGL is applied to the gate electrode of the first transfer gate unit 112. When the drive signal TGL becomes active, the first transfer gate unit 112 becomes conductive, and the electric charges accumulated in the first photoelectric conversion unit 111 are transferred to the FD unit 118 via the first transfer gate unit 112.

[0023] The second photoelectric conversion unit 113 is formed of, for example, a PN junction photodiode, similar to the first photoelectric conversion unit 111. The second photoelectric conversion unit 113 generates and accumulates electric charges according to the amount of light received. Comparing the first photoelectric conversion unit 111 and the second photoelectric conversion unit 113, the first photoelectric conversion unit 111 has a larger light-receiving surface area and higher sensitivity, while the second photoelectric conversion unit 113 has a smaller light-receiving surface area and lower sensitivity.

[0024] The second transfer gate portion 114 is connected between the charge accumulation portion 116 and the FD portion 118. A drive signal FCG is applied to the gate electrode of the second transfer gate portion 114. When the drive signal FCG becomes active, the second transfer gate portion 114 becomes conductive, and the potentials of the charge accumulation portion 116 and the FD portion 118 are coupled.

[0025] The third transfer gate unit 115 is connected between the second photoelectric conversion unit 113 and the charge accumulation unit 116. A drive signal TGS is applied to the gate electrode of the third transfer gate unit 115. When the drive signal TGS becomes active, the third transfer gate unit 115 becomes conductive, and the charges accumulated in the second photoelectric conversion unit 113 are transferred via the third transfer gate unit 115 to the charge accumulation unit 116 or to a region where the potentials of the charge accumulation unit 116 and the FD unit 118 are coupled.

[0026] Furthermore, the potential is slightly deeper below the gate electrode of the third transfer gate unit 115, and an overflow path is formed that transfers charge that exceeds the saturated charge amount of the second photoelectric conversion unit 113 and overflows from the second photoelectric conversion unit 113 to the charge accumulation unit 116. Note that, hereinafter, the overflow path formed below the gate electrode of the third transfer gate unit 115 will be simply referred to as the overflow path of the third transfer gate unit 115.

[0027] The charge accumulation unit 116 is, for example, a capacitor, and is connected between the second transfer gate unit 114 and the third transfer gate unit 115. The counter electrode of the charge accumulation unit 116 is connected between a variable power supply VCB that supplies a voltage VCB different from the power supply voltage VDD. The charge accumulation unit 116 accumulates the charges transferred from the second photoelectric conversion unit 113.

[0028] The reset gate unit 117 is connected between the variable power supply VCB and the FD unit 118. A drive signal RST is applied to the gate electrode of the reset gate unit 117. When the drive signal RST becomes active, the reset gate unit 117 becomes conductive, and the potential of the FD unit 118 is reset to the level of the power supply voltage VCB.

[0029] The FD section 118 converts the charges into a voltage signal and outputs the voltage signal. The amplifier transistor 119 has a gate electrode connected to the FD section 118 and a drain electrode connected to a power supply VDD, and serves as an input section of a readout circuit (a so-called source follower circuit) that reads out the charges held in the FD section 118. That is, the amplifier transistor 119 has a source electrode connected to a vertical signal line 19 via a selection transistor 120, and thereby forms a source follower circuit together with a constant current source 121 connected to one end of the vertical signal line 19.

[0030] The selection transistor 120 is connected between the source electrode of the amplification transistor 119 and the vertical signal line 19. A drive signal SEL is applied to the gate electrode of the selection transistor 120. When the drive signal SEL is in an active state, the selection transistor 120 is turned on, and the unit pixel 110 is turned to a selected state. As a result, the pixel signal output from the amplification transistor 119 is output to the vertical signal line 19 via the selection transistor 120.

[0031] The power supply voltage VCB of the variable power supply VCB is set to, for example, a high-level voltage VH or a low-level voltage VL. For example, the voltage VH is set to a level similar to the power supply voltage VDD, and the voltage VL is set to the ground (GND) level.

[0032] In the following, when each drive signal is in an active state, it is referred to as the drive signal being turned on, and when each drive signal is in an inactive state, it is referred to as the drive signal being turned off. In the following, when each gate unit or each transistor is in a conductive state, it is referred to as the gate unit or each transistor being turned on, and when each gate unit or each transistor is in a non-conductive state, it is referred to as the gate unit or each transistor being turned off.

[0033] (Operation of Unit Pixel 110) Next, the operation of the unit pixel 110 will be described with reference to the timing charts of FIGS.

[0034] (Operation of Unit Pixel 110 at the Start of Exposure) First, the operation of the unit pixel 110 at the start of exposure will be described with reference to the timing chart in Fig. 3. This process is performed in a predetermined scanning order, for example, for each pixel row or for each set of multiple pixel rows in the pixel array section 11. Fig. 3 also shows a timing chart of the horizontal synchronization signal XHS, the drive signal SEL, the power supply voltage VCB, and the drive signals RST, TGS, FCG, and TGL.

[0035] First, at time t1, the horizontal synchronization signal XHS is input, and exposure processing of the unit pixel 110 begins. Next, at time t2, the power supply voltage VCB is changed from voltage VL to voltage VH. After that, at time t3, the drive signal RST is turned on, and the reset gate unit 117 is turned on. This resets the potential of the FD unit 118 to the level of the power supply voltage VCB.

[0036] Next, at time t4, the drive signals TGL, FCG, and TGS are turned on, turning on the first transfer gate unit 112, the second transfer gate unit 114, and the third transfer gate unit 115. This couples the potentials of the charge storage unit 116 and the FD unit 118. Furthermore, the charge stored in the first photoelectric conversion unit 111 is transferred to the coupled region via the first transfer gate unit 112, and the charge stored in the second photoelectric conversion unit 113 is transferred to the coupled region via the third transfer gate unit 115. Then, the coupled region is reset.

[0037] Next, at time t5, the drive signals TGL and TGS are turned off, turning off the first transfer gate unit 112 and the third transfer gate unit 115. This starts the accumulation of charges in the first photoelectric conversion unit 111 and the second photoelectric conversion unit 113, and the exposure period begins.

[0038] Next, at time t6, the drive signal RST is turned off, turning off the reset gate unit 117. Next, at time t7, the drive signal FCG is turned off, turning off the second transfer gate unit 114. As a result, the charge accumulation unit 116 starts accumulating the charge that overflows from the second photoelectric conversion unit 113 and is transferred via the overflow path of the third transfer gate unit 115.

[0039] Next, at time t8, the power supply voltage VCB is changed from voltage VH to voltage VL, and then at time t9, the horizontal synchronization signal XHS is input.

[0040] (Operation when reading out unit pixel 110) Next, the operation when reading out pixel signals from the unit pixel 110 will be described with reference to the timing chart of Fig. 4. This process is performed, for example, for each pixel row or for each set of pixel rows in the pixel array section 11 in a predetermined scanning order a predetermined time after the process of Fig. 3 is performed. Note that Fig. 4 shows a timing chart of the horizontal synchronization signal XHS, the drive signal SEL, the power supply voltage VCB, and the drive signals RST, TGS, FCG, and TGL.

[0041] First, at time t21, the horizontal synchronization signal XHS is input, starting the readout period of the unit pixel 110. Next, at time t22, the drive signal SEL is turned on, turning on the selection transistor 120. This causes the unit pixel 110 to enter a selected state. Also, the power supply voltage VCB is changed from voltage VL to voltage VH.

[0042] Next, at time t23, the drive signal RST is turned on, turning on the reset gate unit 117. This resets the potential of the FD unit 118 to the level of the power supply voltage VDD. Next, at time t24, the drive signal RST is turned off, turning off the reset gate unit 117.

[0043] Next, at time t25, the drive signals FCG and TGS are turned on, turning on the second transfer gate unit 114 and the third transfer gate unit 115. This couples the potentials of the charge accumulation unit 116 and the FD unit 118, and the charge accumulated in the second photoelectric conversion unit 113 is transferred to the coupled region. As a result, the charge accumulated in the second photoelectric conversion unit 113 and the charge accumulation unit 116 during the exposure period is accumulated in the coupled region.

[0044] At time t25, readout of pixel signals starts, and the exposure period ends. Next, at time t26, the drive signal TGS is turned off, turning off the third transfer gate unit 115. This stops the transfer of charges from the second photoelectric conversion unit 113.

[0045] Next, at time ta between time t26 and time t27, a pixel signal SP1L based on the potential of the region where the potentials of the charge accumulation unit 116 and the FD unit 118 are coupled is output to the vertical signal line 19 via the amplification transistor 119 and the selection transistor 120. The pixel signal SP1L is a signal based on the charges generated by the second photoelectric conversion unit 113 during the exposure period and accumulated in the second photoelectric conversion unit 113 and the charge accumulation unit 116. Furthermore, the pixel signal SP1L is a signal based on the potential of the region where the potentials of the charge accumulation unit 116 and the FD unit 118 are coupled, in a state where the charges accumulated in the second photoelectric conversion unit 113 and the charge accumulation unit 116 during the exposure period are accumulated in the region where the potentials of the charge accumulation unit 116 and the FD unit 118 are coupled. Therefore, the capacitance for converting the charges into a charge voltage when the pixel signal SP1L is read out is the combined capacitance of the charge accumulation unit 116 and the FD unit 118. Hereinafter, the pixel signal SP1L will also be referred to as a low-sensitivity data signal SP1L.

[0046] Next, at time t27, the drive signal RST is turned on, turning on the reset gate unit 117. This resets the region where the potentials of the charge storage unit 116 and the FD unit 118 are coupled.

[0047] Next, at time t28, the selection signal SEL is turned off, turning off the selection transistor 120. This unselects the unit pixel 110. Next, at time t29, the drive signal RST is turned off, turning off the reset gate portion 117.

[0048] Next, at time t30, the selection signal SEL is turned on, turning on the selection transistor 120. This causes the unit pixel 110 to be in a selected state. Next, at time tb between time t30 and time t31, a pixel signal SP2L based on the potential of a region where the potentials of the charge accumulation unit 116 and the FD unit 118 are coupled is output to the vertical signal line 19 via the amplification transistor 119 and the selection transistor 120. This pixel signal SP2L is a signal based on the potential of the coupled region when the region where the potentials of the charge accumulation unit 116 and the FD unit 118 are coupled is in a reset state. Note that hereinafter, the pixel signal SP2L will also be referred to as a low-sensitivity reset signal SP2L.

[0049] Next, at time t31, the drive signal FCG is turned off, and the second transfer gate unit 114 is turned off. Next, at time tc between time t31 and time t32, a pixel signal SP2H based on the potential of the FD unit 118 is output to the vertical signal line 19 via the amplification transistor 119 and the selection transistor 120. The pixel signal SP2H is a signal based on the potential of the FD unit 118 in a reset state. Note that hereinafter, the pixel signal SP2H is also referred to as a high-sensitivity reset signal SP2H.

[0050] Next, at time t32, the drive signal TGL is turned on, turning on the first transfer gate unit 112. As a result, the charges generated and accumulated in the first photoelectric conversion unit 111 during the exposure period are transferred to the FD unit 118 via the first transfer gate unit 112.

[0051] Next, at time t33, the drive signal TGL is turned off, turning off the first transfer gate unit 112. This stops the transfer of charges from the first photoelectric conversion unit 111 to the FD unit 118.

[0052] Next, at time td between time t33 and time t34, a pixel signal SP1H based on the potential of the FD section 118 is output to the vertical signal line 19 via the amplification transistor 119 and the selection transistor 120. The pixel signal SP1H is a signal based on the charge generated and accumulated in the first photoelectric conversion section 111 during the exposure period. The pixel signal SP1H is also a signal based on the potential of the FD section 118 in a state in which the charge accumulated in the first photoelectric conversion section 111 during the exposure period is accumulated in the FD section 118. Therefore, the capacitance for charge-to-voltage conversion during readout of the pixel signal SP1H is the capacitance of the FD section 118, which is smaller than that during readout of the low-sensitivity data signal SP1L at time ta. Hereinafter, the pixel signal SP1H will also be referred to as the high-sensitivity data signal SP1H.

[0053] Then, at time t34, the selection signal SEL is turned off, turning off the selection transistor 120. This causes the unit pixel 110 to enter a non-selected state. Also, the power supply voltage VCB is changed from voltage VH to voltage VL. Next, at time t35, the horizontal synchronization signal XHS is input, and the readout period of the pixel signal of the unit pixel 110 ends.

[0054] In the unit pixel 110, the power supply voltage VCB is set to the voltage VL during the period from the start of exposure to the start of readout when charges are accumulated in the charge accumulation unit 116. This reduces the electric field applied to the charge accumulation unit 116 during the period when charges are accumulated in the charge accumulation unit 116, thereby suppressing dark current generated in the charge accumulation unit 116.

[0055] (Configuration of Signal Processing Unit) Fig. 5 is a block diagram showing an example configuration of the signal processing unit 16 of Fig. 1. The signal processing unit 16 includes an HDR synthesis unit 21 and a correction gain generation circuit 22. The HDR synthesis unit 21 generates an HDR signal by synthesizing pixel signals SP1H / L and SP2H / L. The correction gain generation circuit 22 calculates a synthesis gain for correcting the difference in sensitivity between the different pixel signals SP1H / L and SP2H / L when performing HDR synthesis.

[0056] (Circuit Configuration of HDR Combining Unit) Fig. 6 is a circuit configuration diagram of HDR combining unit 21. As shown in Fig. 6(a), HDR combining unit 21 is configured to include a first blending unit 211, a first multiplication unit 212, a second blending unit 213, a second multiplication unit 214, a third blending unit 215, and a third multiplication unit 216.

[0057] The first blending unit 211 receives the pixel signal SP1H and the pixel signal SP1L multiplied by the synthesis gain G1 in the first multiplier 212. The first blending unit 211 then selectively outputs the pixel signal SP1H or the pixel signal SP1L by changing the blending ratio α from 0 to 1.

[0058] The second blending unit 213 receives the pixel signal SP2H and the pixel signal SP2L multiplied by the synthesis gain G2 in the second multiplier 214. The second blending unit 213 then selectively outputs the pixel signal SP2H or the pixel signal SP2L by changing the blending ratio α from 0 to 1.

[0059] The third blending unit 215 receives as input the pixel signal SP1 output from the first blending unit 211 and the pixel signal SP2 output from the second blending unit 213 and multiplied by a synthesis gain G3 in the third multiplier 216. The third blending unit 215 then selectively outputs the pixel signal SP2 or the pixel signal SP1 by changing the blending ratio α from 0 to 1. That is, as shown in FIG. 6B , when two inputs are A and B, the first blending unit 211, the second blending unit 213, and the third blending unit 215 output A(1−α)+Bα.

[0060] (Processing Example of Pixel Signal Arithmetic Processing) First, the first blending unit 211 outputs the product of the low-sensitivity pixel signal SP1L and the composite gain G1 as the correction value of the low-sensitivity pixel signal SP1L. Here, if the correction value of the low-sensitivity pixel signal SP1L (hereinafter referred to as the corrected low-sensitivity pixel signal) is SP1L', the composite gain G1 and the corrected low-sensitivity pixel signal SP1L' can be calculated based on the following equation: SP1L'=G1×SP1L

[0061] 7 shows the relationship between the amount of incident light and the low-sensitivity pixel signal SP1L, the high-sensitivity pixel signal SP1H, and the corrected low-sensitivity pixel signal SP1L'. The first blending unit 211 uses predetermined thresholds Vt1 and Vt2. The thresholds Vt1 and Vt2 are set in advance in a region of the photoresponse characteristics where the high-sensitivity pixel signal SP1H is not yet saturated and the photoresponse characteristics are linear. The second blending unit 213 uses thresholds Vt11 and Vt21, which are different from the thresholds Vt1 and Vt2. The third blending unit 215 uses thresholds Vt12 and Vt22.

[0062] If the high-sensitivity pixel signal SP1H does not exceed the threshold value Vt1, the first blending unit 211 outputs the high-sensitivity pixel signal SP1H as the pixel signal SP1 of the pixel to be processed. That is, if SP1H<Vt1, the pixel signal SP1=the high-sensitivity pixel signal SP1H.

[0063] On the other hand, when the high-sensitivity pixel signal SP1H exceeds the threshold value Vt1 and is less than the threshold value Vt2, the first blending unit 211 outputs a blended output of the high-sensitivity pixel signal SP1H and the corrected low-sensitivity pixel signal SP1L' as the pixel signal SP1 of the processing target pixel. That is, when the high-sensitivity pixel signal SP1H is equal to or greater than the threshold value Vt1 and equal to or less than the threshold value Vt2, the pixel signal SP1 is a blended output of the high-sensitivity pixel signal SP1H and the corrected low-sensitivity pixel signal SP1L'.

[0064] <Comparative Example of Embodiment> In the current method, a composite gain measured in advance is read from a register. In this case, it is necessary to measure the composite gain for each device in advance to correct for sensitivity differences and store the value in a register. Furthermore, if the actual device environment (color temperature, etc.) differs from the pre-measurement, a signal step occurs at the composite boundary due to a correction error, resulting in defects such as color unevenness.

[0065] 8 is a block diagram of a correction gain generation circuit 22 according to a first embodiment of the present disclosure. The correction gain generation circuit 22 includes a first selector 221, a second selector 222, a first accumulator 223, a second accumulator 224, a composite gain processor 225, an IIR filter 226, a composite gain register 227, a moving subject determination unit 228, a color determination unit 229, a pixel area determination unit 230, a signal level determination unit 231, an AND circuit 232, a counter 233, a filter strength calculation unit 234, an update determination counter 235, various setting registers 236, and an accumulation condition threshold generation unit 237.

[0066] The first selector 221 receives the high-sensitivity pixel signal SP1H and outputs the high-sensitivity pixel signal SP1H when a logical value of "1" is input to the control terminal. The second selector 222 receives the low-sensitivity pixel signal SP1L and outputs the low-sensitivity pixel signal SP1L when a logical value of "1" is input to the control terminal.

[0067] The first integrator 223 integrates the signal level of the high-sensitivity pixel signal SP1H output from the first selector 221 for each pixel 110 to obtain a first integrated value, and outputs the first integrated value to the composite gain processor 225. The second integrator 224 integrates the signal level of the low-sensitivity pixel signal SP1L output from the second selector 222 for each pixel 110 to obtain a second integrated value, and outputs the second integrated value to the composite gain processor 225. The first integrator 223 and the second integrator 224 are reset, for example, when a vertical reset signal is input from the system controller 15. The first integrator 223 and the second integrator 224 output the integrated number of pixels to the counter 233. The first integrator 223 and the second integrator 224 are provided for each color filter of the pixel 110.

[0068] The composite gain processing unit 225 calculates the sensitivity ratio between the high-sensitivity pixel signal SP1H and the low-sensitivity pixel signal SP1L from the input first integrated value and second integrated value as a composite gain G1, and outputs it to the IIR filter 226.

[0069] The IIR filter 226 smooths the past synthetic gain G1 stored in the synthetic gain register 227 and the new synthetic gain G1 calculated by the synthetic gain processing unit 225 in the time direction, and outputs the smoothed synthetic gain G1 to the synthetic gain register 227. The smoothing strength of the IIR filter 226 is variably controlled by the filter strength calculation unit 234. Here, when the smoothing strength is strong, the IIR filter 226 reduces the contribution rate of new data to make the convergence change gentler. On the other hand, when the smoothing strength is weak, the IIR filter 226 increases the contribution rate of new data to speed up the convergence change rate.

[0070] The synthesis gain register 227 stores and holds the input synthesis gain G1, and outputs it to the first multiplication unit 212 of the HDR synthesis unit 21. Furthermore, the synthesis gain register 227 reads the initial value of the synthesis gain G1 from the various setting registers 236 when the power is turned on.

[0071] The moving subject determination unit 228 detects the movement of the subject based on the input moving subject determination signal, and if it determines that the subject is a moving subject, it outputs a logical value of "0" to the AND circuit 232. Furthermore, if the moving subject determination unit 228 determines that the high-sensitivity pixel signal SP1H and the low-sensitivity pixel signal SP1L have different exposure timings and are not a moving subject, it outputs a logical value of "1" to the AND circuit 232. Furthermore, the moving subject determination unit 228 outputs a logical value of "1" when invalidating the determination. Whether the determination is valid or invalid depends on the settings of the various setting registers 236.

[0072] The color determination unit 229 determines the color of each of the first and second integrated values ​​based on the ratio between the color signal near the integrated value of the high-sensitivity pixel signal SP1H and the color signal near the integrated value of the low-sensitivity pixel signal SP1L, and separately provided white balance (WB) gain information, and outputs a logical value of "1" to the AND circuit 232 if it determines that the color is within the white range. The color determination unit 229 also outputs a logical value of "1" if it invalidates the determination. Whether the determination is valid or invalid depends on the settings of the various setting registers 236.

[0073] The pixel area determination unit 230 determines whether or not the area is within an effective pixel range for integration based on the input horizontal and vertical addresses and a control signal output from an integration condition threshold generation unit 237 (described later), and if it determines that the area is within an effective pixel range for integration, it outputs a logical value of "1" to the AND circuit 232. The pixel area determination unit 230 also outputs a logical value of "1" if it determines that the determination is invalid. Whether the determination is valid or invalid depends on the settings of various setting registers 236.

[0074] The signal level determination unit 231 receives the high-sensitivity pixel signal SP1H and determines whether the signal level of the high-sensitivity pixel signal SP1H is within the synthesis boundary, that is, whether the signal level of the high-sensitivity pixel signal SP1H is equal to or greater than the threshold value Vt1 and equal to or less than the threshold value Vt2. If the signal level determination unit 231 determines that the signal level is within the synthesis boundary, it outputs a logical value of "1" to the AND circuit 232. Furthermore, the signal level determination unit 231 outputs a logical value of "1" when invalidating the determination. Whether the determination is valid or invalid depends on the settings of the various setting registers 236. Note that the respective determination reference values ​​of the moving subject determination unit 228, color determination unit 229, pixel area determination unit 230, and signal level determination unit 231 are stored in the various setting registers 236.

[0075] If the moving subject determination unit 228, color determination unit 229, pixel area determination unit 230, and signal level determination unit 231 all have a logical value of "1", the AND circuit 232 outputs a logical value of "1" to the first selector 221, the second selector 222, and the counter 233.

[0076] The counter 233 counts the cumulative number of effective pixels within 1V (one frame period) and outputs the output value to the filter strength calculation unit 234 .

[0077] The filter strength calculation unit 234 receives temperature information, WB gain information, and the output value of the counter 233. The filter strength calculation unit 234 weakens the smoothing strength of the IIR filter 226 when the color temperature changes based on the temperature information and WB gain information, and operates to quickly generate gain that matches the changed light source conditions.

[0078] In addition, the filter strength calculation unit 234 weakens the smoothing strength of the IIR filter 226 when the ambient temperature changes based on the temperature information and WB gain information, so that gain generation that quickly matches the changed temperature conditions is performed.

[0079] Furthermore, based on the output value of the counter 233, when the number of data accumulated for calculating the composite gain is equal to or less than a certain number, the filter strength calculation unit 234 stops the smoothing process of the IIR filter 226 and controls the data so that it is not updated.

[0080] The update determination counter 235 counts the number of updates of the smoothing process from the startup initial value of the IIR filter 226. If the number of updates of the smoothing process from the startup initial value of the IIR filter 226 by the update determination counter 235 is equal to or less than a certain number, the filter strength calculation unit 234 weakens the smoothing strength of the IIR filter 226, causing the IIR filter 226 to operate so that gain generation is performed quickly in accordance with the actual environment.

[0081] Furthermore, the filter strength calculation unit 234 calculates the difference between the new composite gain generated from the new image and the current composite gain, and if the difference is a deviation of at least a set value A and exceeds n (n is an integer) frames, starts a smoothing operation by the IIR filter 226 and controls the IIR filter 226 so that the smoothing operation by the IIR filter 226 continues until the difference is equal to or less than a set value B that is smaller than the set value A. Thereafter, the difference determination is returned to the set value A. Note that if the deviation of at least the set value A is n frames or less, the filter strength calculation unit 234 does not start the smoothing operation by the IIR filter 226.

[0082] The accumulation condition threshold generation unit 237 according to the first embodiment of the present disclosure controls the image area determination unit 230 in accordance with the number of valid frames since startup, and variably sets the accumulation pixel range.

[0083] (Setting Operation of Effective Pixel Range in First Embodiment) Next, a description will be given of the variable setting control operation of the effective pixel range by the correction gain generation circuit 22 configured as described above. Fig. 9 is a diagram showing the timing of variable setting of the effective pixel range by the correction gain generation circuit 22. Fig. 10 is a flowchart showing the variable setting control procedure and control contents of the effective pixel range by the correction gain generation circuit 22.

[0084] If the effective integrating pixel range is constant and set narrow, there will be no pixels with signal levels used to calculate the composite gain in the effective integrating pixel range immediately after startup, and the composite gain will remain unstable.On the other hand, if the effective integrating pixel range is set wide, the composite gain will continue to be generated from an integrated value that includes the effects of image height-dependent shading.

[0085] Therefore, in the first embodiment of the present disclosure, the effective integrated pixel range is gradually narrowed depending on the number of effective frames of integrated data since startup. Assume that the imaging device 1 is powered on. Then, the correction gain generation circuit 22 sets the effective integrated pixel range within the input imaging screen IG to a maximum value A11 (step ST10a and FIG. 9A). The correction gain generation circuit 22 then counts the number of effective frames of integrated data counted by the counter 233 (step ST10b) and determines whether the count value is equal to or greater than a predetermined value, e.g., four effective frames (step ST10c). If the count value is determined to be less than the predetermined value (step ST10c: No), the correction gain generation circuit 22 continues the processing of step ST10b.

[0086] On the other hand, if it is determined that the count value is equal to or greater than the predetermined value (step ST10c: Yes), the correction gain generation circuit 22 narrows the integrated effective pixel range in the input image IG by aa (step ST10d), sets it to A12 (FIG. 9B), and determines whether the integrated effective pixel range is the minimum value A13 (step ST10e). In the state of FIG. 9B, the integrated effective pixel range is not the minimum value A13 (step ST10e: No), so the correction gain generation circuit 22 proceeds to the processing of step ST10b.

[0087] Thereafter, when the count value has passed a predetermined value or more from the state of Fig. 9(b), the correction gain generation circuit 22 narrows the integrated effective pixel range in the input imaging screen IG by aa and sets it to the minimum value A13 (Fig. 9(c)). Since the integrated effective pixel range has reached the minimum value A13 in step ST10e (step ST10e: Yes), the correction gain generation circuit 22 ends the variable setting control process for the integrated effective pixel range. Note that the integrated effective pixel range may be set to a value smaller than the minimum value A13.

[0088] In the first embodiment of the present disclosure, an example of calculating the composite gain G1 to be multiplied by the pixel signal SP1L has been described. However, for example, calculation of the composite gain G2 to be multiplied by the pixel signal SP2L in the second multiplication unit 214 and calculation of the composite gain G3 to be multiplied by the pixel signal SP2 in the third multiplication unit 216 can also be performed in parallel by the correction gain generation circuit 22.

[0089] <Effects of First Embodiment> As described above, according to the first embodiment, immediately after startup of the imaging device 1, there may be cases where pixels with signal levels used for calculating the composite gain are not present in the effective accumulating pixel range. Therefore, by setting the effective accumulating pixel range to a wide value of A11 at startup, it is possible to prevent the composite gain from remaining unstable, and by setting the effective accumulating pixel range to a narrow value at the time of the next effective frame, it is possible to prevent the composite gain from continuing to be generated from an integrated value that includes the influence of shading that is dependent on image height. As a result, it is possible to calculate the composite gain stably even immediately after startup.

[0090] Second Embodiment In a second embodiment of the present disclosure, the range of effective integrated pixel levels is gradually narrowed according to the number of effective frames of integrated data from startup. Fig. 11 is a block diagram of a correction gain generation circuit 22A provided in an imaging device 1A according to a second embodiment of the present disclosure. Note that in Fig. 11, the same components as those in Fig. 8 above are designated by the same reference numerals, and detailed description thereof will be omitted. In the second embodiment of the present disclosure, an example will be described in which a high-sensitivity pixel signal SP1L and a low-sensitivity pixel signal SP2H are input.

[0091] The signal level determination unit 231A inputs the high-sensitivity pixel signal SP1L and determines whether it is within the effective integrated pixel level range based on a control signal output from the integration condition threshold generation unit 237A described later.If it determines that it is within the effective integrated pixel level range, it outputs a logical value of "1" to the AND circuit 232.

[0092] The accumulation condition threshold generation unit 237A according to the second embodiment of the present disclosure controls the signal level determination unit 231A in accordance with the number of valid frames since startup, and variably sets the range of accumulated pixel levels.

[0093] (Setting Operation of Effective Integrated Pixel Level Range in Second Embodiment) Next, a description will be given of the variable setting control operation of the effective integrated pixel level range by the correction gain generation circuit 22A configured as described above. FIG. 12 is a diagram showing the timing of variable setting of the effective integrated pixel level range by the correction gain generation circuit 22A. FIG. 13 is a flowchart showing the variable setting control procedure and control content of the effective integrated pixel level range by the correction gain generation circuit 22A. In FIG. 12 (1-1), the horizontal axis represents time and the vertical axis represents signal level. In FIG. 12 (1-2), the horizontal axis represents signal level and the vertical axis represents blend ratio. Furthermore, in FIG. 12, the high-sensitivity pixel signal SP1L is represented by a solid line, and the low-sensitivity pixel signal SP2H is represented by a dashed line.

[0094] When the range of effective integrated pixel levels is constant, if it is set narrow, pixels with signal levels used for calculating the composite gain will not exist in the range of effective integrated pixel levels immediately after startup, and the composite gain will remain unstable.On the other hand, if the range of effective integrated pixel levels is set wide, pixels with poor signal-to-noise ratios (S / N ratios) will be included in the integration.

[0095] Therefore, in a second embodiment of the present disclosure, the integrated effective pixel level range is gradually narrowed depending on the number of effective frames of integrated data since startup. Assume that the imaging device 1A is powered on. The correction gain generation circuit 22A then sets the integrated effective pixel level range to a maximum value B11 equal to or greater than a threshold value Vt3, which is smaller than the threshold value Vt1 of the in-composite boundary level, and equal to or smaller than a threshold value Vt4, which is larger than the threshold value Vt2 of the in-composite boundary level (step ST13a and FIGS. 12 (1-1) and (1-2)). The correction gain generation circuit 22A then counts the number of effective frames of integrated data counted by the counter 233 (step ST13b) and determines whether the count value is equal to or greater than a predetermined value, e.g., four effective frames (step ST13c). If the correction gain generation circuit 22A determines that the count value is less than the predetermined value (step ST13c: No), it continues processing in step ST13b.

[0096] On the other hand, if it is determined that the count value is equal to or greater than the predetermined value (step ST13c: Yes), the correction gain generation circuit 22A narrows the integrated effective pixel level range in the input image IG by bb (step ST13d), sets the maximum value B11 to a value B12 equal to or greater than a threshold value Vt5 that is greater than the threshold value Vt3 and equal to or less than a threshold value Vt6 that is less than the threshold value Vt4 (FIGS. 12(2-1)(2-2)), and determines whether the integrated effective pixel level range is equal to or less than the minimum value B13, i.e., the level within the synthesis boundary (step ST13e). In the state of FIG. 12(2-1)(2-2), the integrated effective pixel level is not equal to the minimum value B13 (step ST13e: No), so the correction gain generation circuit 22A proceeds to the processing of step ST13b.

[0097] 12(2-1)(2-2)。 After that, when the count value has passed a predetermined value or more from the state of Figure 12 (2-1) (2-2), the correction gain generation circuit 22A narrows the integrated effective pixel level range in the input image IG by bb and sets it to the minimum value B13 (Figure 12(3-1) (3-2)). In step ST13e, since the minimum value B13 has been reached (step ST13e: Yes), the correction gain generation circuit 22A ends the variable setting control process for the integrated effective pixel level range. The integrated effective pixel level range may be set to be smaller than the minimum value B13.

[0098] In the second embodiment of the present disclosure, an example of calculating the composite gain G1 to be multiplied by the pixel signal SP2H has been described. However, for example, calculation of the composite gain G2 to be multiplied by the pixel signal SP1L and calculation of the composite gain G3 to be multiplied by the pixel signal SP2L can also be performed in parallel by the correction gain generation circuit 22A.

[0099] <Effects of the Second Embodiment> As described above, according to the second embodiment, since there may be cases where pixels with signal levels used for calculating the composite gain are not present in the integration area immediately after startup of the image capture device 1A, by setting the range of effective integrated pixel levels to a wide maximum value B11 at startup, it is possible to prevent the composite gain from remaining unstable for a long time, and by setting the range of effective integrated pixel levels to a narrower range at the time of the next effective frame, it is possible to prevent pixels with poor S / N ratios from being included in the integration. As a result, the composite gain can be calculated stably even immediately after startup.

[0100] Third Embodiment In a third embodiment of the present disclosure, when the composite gain of one or more color filters is calculated immediately after startup, the calculated composite gain is used to apply the result to color filters for which the composite gain has not yet been calculated. Fig. 14 is a block diagram of a correction gain generation circuit 22B provided in an image pickup device 1B according to a third embodiment of the present disclosure. Note that in Fig. 14, the same components as those in Fig. 8 above are designated by the same reference numerals, and detailed description thereof will be omitted. The image pickup device 1B according to the third embodiment of the present disclosure is applied to a sensor in which a different color filter 300 is arranged for each unit pixel 110, as shown in Fig. 15.

[0101] The color filter 300 is disposed on the light incident surface (direction indicated by arrow Z in FIG. 15 ) of the unit pixel 110. The color filter 300 is an optical filter that selectively transmits light of a predetermined wavelength among light collected by an on-chip lens (not shown). In this example, color filters 300 that selectively transmit wavelengths of R light (red light), Gr light (green light), Gb light (green light), and B light (blue light) are used, but this is not limiting. Of the four unit pixels 110, an R filter 310 (shown by diagonal lines in FIG. 15 ) is disposed in the upper left unit pixel 110 in FIG. 15 , and a Gb filter 340 (shown by dots in FIG. 15 ) is disposed in the lower left unit pixel 110 in FIG. 15 . Furthermore, a Gr filter 330 (shown by dots in FIG. 15 ) is disposed in the upper right unit pixel 110 in FIG. 15 , and a B filter 320 is disposed in the lower right unit pixel 110 in FIG. 15 .

[0102] The accumulation condition threshold generation unit 237B according to the third embodiment of the present disclosure controls the composite gain processing unit 225B so that, immediately after startup, when a composite gain corresponding to one or more color filters 300 is calculated from among the color filters 300, the calculated composite gain is used to calculate a composite gain corresponding to an uncalculated color filter 300.

[0103] A storage unit 240 is connected to the composite gain processing unit 225B according to the third embodiment of the present disclosure. The composite gain processing unit 225B also includes a storage control unit 2251 and an update control unit 2252. When a composite gain is calculated by the composite gain processing unit 225B, the storage control unit 2251 stores the composite gain in the storage unit 240. The update control unit 2252 updates the composite gain value of the corresponding color filter 300 stored in the storage unit 240 each time a composite gain is calculated by the composite gain processing unit 225B, except immediately after startup. Then, immediately after startup, under the control of the integration condition threshold generation unit 237B, the composite gain processing unit 225B calculates a composite gain corresponding to a color filter 300 for which no calculation has been performed, using the composite gain stored in the storage unit 240.

[0104] (Operation in Third Embodiment) Next, the operation of the correction gain generation circuit 22B configured as above will be described. Fig. 16 is a diagram showing the operation timing of the correction gain generation circuit 22B.

[0105] In the imaging device 1B, the sensitivity differs depending on the type of color filter 300. Therefore, even if the unit pixels 110 of one color filter 300 are included in the integration area, it is possible that the unit pixels 110 of another color filter 300 are not included in the integration area. In this case, coloring occurs in the input image IG due to a mismatch in the synthesis gain for each unit pixel 110. For example, when the input image IG is an entirely green scene, the levels of the R (red) unit pixels 110 and the B (blue) unit pixels 110 are lower than those of the G (green) unit pixels 110, and only the G (green) unit pixels 110 may be included in the integration area.

[0106] Therefore, in the correction gain generation circuit 22B according to the third embodiment of the present disclosure, when a composite gain is calculated in the initial STEP (immediately after startup) (FIG. 16 (1-1)), the correction gain generation circuit 22B stores the values ​​of the composite gains corresponding to the relevant Gr filter 330 and Gb filter 340 in the memory unit 240, and calculates the additive average value of the composite gains corresponding to the Gr filter 330 and Gb filter 340, and applies this to the unit pixel 110 in which the other R filter 310 and B filter 320 are arranged.

[0107] After the initial step (FIG. 16 (1-2)), each time the composite gain processing unit 225B calculates a composite gain, the correction gain generation circuit 22B updates the composite gain value of the corresponding color filter 300 stored in the storage unit 240. Here, only the composite gain values ​​corresponding to the Gr filter 330 and the Gb filter 340 are updated.

[0108] In the third embodiment of the present disclosure, an example has been described in which the composite gain G1 to be multiplied by the pixel signal SP1L is calculated. However, for example, the second multiplier 214 can calculate the composite gain G2 to be multiplied by the pixel signal SP2L, and the third multiplier 216 can calculate the composite gain G3 to be multiplied by the pixel signal SP2, and these operations can also be performed in parallel by the correction gain generation circuit 22B.

[0109] <Effects of Third Embodiment> As described above, according to the third embodiment, immediately after startup, when composite gains are calculated for one or more color filters 300, an arithmetic average is calculated from the calculated composite gains, and the arithmetic average is set as the composite gain corresponding to the uncalculated color filter 300. This makes it possible to reduce the occurrence of coloring in the captured image IG due to mismatches in composite gain for each unit pixel 110.

[0110] Furthermore, according to the third embodiment, each time the composite gain processing unit 225B calculates a composite gain, the composite gain value of the corresponding color filter 300 stored in the memory unit 240 is updated. Therefore, even immediately after startup, the latest composite gain at that time can always be used to calculate a composite gain corresponding to an uncalculated color filter 300.

[0111] <Fourth embodiment> A fourth embodiment of the present disclosure includes a mode 1 in which the integrated effective pixel range is variably set, a mode 2 in which the integrated effective pixel level range is variably set, and a mode 3 in which the calculated composite gain is applied to an uncalculated color filter 300, and these modes 1 to 3 are selectively operated in accordance with a selection instruction from the user.

[0112] 17 is a flowchart showing the control procedure and control contents of the composite gain integration process performed by the correction gain generation circuit 22C according to the fourth embodiment of the present disclosure. The correction gain generation circuit 22C determines whether a mode has been set by the user by referring to the contents stored in the various setting registers 236 (step ST17c). For example, if the mode is set according to the time of day, and mode 1 is set for the morning and evening rush hours when there is a lot of pedestrian and vehicular movement, the correction gain generation circuit 22C sets the integrated effective pixel range to a maximum value A11 and gradually narrows the integrated effective pixel range to a minimum value A13 according to the number of effective frames of the integrated data (step ST17b).

[0113] Furthermore, in step ST17a, when mode 2 is set during the daytime when there is little movement of people and vehicles, the correction gain generation circuit 22C sets the integrated effective pixel level range to a maximum value B11 that is equal to or greater than a threshold value Vt3 that is smaller than the threshold value Vt1 of the level within the composite boundary and equal to or less than a threshold value Vt4 that is larger than the threshold value Vt2 of the level within the composite boundary, and gradually narrows the integrated effective pixel level range to the minimum value B13 according to the number of effective frames of the integrated data (step ST17c).

[0114] Furthermore, in step ST17d above, when mode 3 is set during the nighttime hours, the correction gain generation circuit 22C stores the calculated composite gain values ​​corresponding to the Gr filter 330 and the Gb filter 340 in the memory unit 240, and calculates the additive average value of the composite gains corresponding to the Gr filter 330 and the Gb filter 340, and applies this to unit pixels 110 in which other R filters 310 and B filters 320 are arranged.

[0115] In the above processing procedure, an example has been described in which mode 1, mode 2, or mode 3 is selected according to the time of day, but mode 1, mode 2, or mode 3 may also be selected according to the usage environment. Furthermore, a combination of mode 1 and mode 3, or a combination of mode 2 and mode 3 may also be selected according to the time of day.

[0116] <Effects of the Fourth Embodiment> As described above, according to the fourth embodiment, by setting a mode for each time period, for example, the integrated effective pixel range is gradually narrowed during the morning and evening rush hours when there is a lot of movement of people and cars, while the integrated effective pixel level range is narrowed during the daytime when there is little movement of people and cars. During the nighttime, if composite gains corresponding to one or more color filters 300 can be calculated, the calculated composite gains are used to calculate composite gains corresponding to uncalculated color filters 300. Thus, optimal integration ranges can be set for each time period. Furthermore, by setting any combination of modes, for example, not only the integrated pixel range but also the integrated effective pixel level range can be narrowed simultaneously. Furthermore, even if the integrated pixel range is narrowed, composite gains corresponding to one or more color filters 300 can still be used to calculate composite gains corresponding to other uncalculated color filters 300.

[0117] Fifth Embodiment In a fifth embodiment of the present disclosure, the integration range for integrating data for calculating a composite gain is variably set to an optimal integration range based on the number of valid frames since startup, the time elapsed since startup, and changes in the environment of the subject since startup.

[0118] For example, immediately after startup, there may be cases where there is no signal in the integration range where data for calculating the composite gain is integrated, so the integration range is set wide, and if the environment of the subject differs depending on the location of use or changes over time, the integration range is variably set to an optimal range for each location of use or at each time. This prevents the device from continuing to generate an unstable composite gain even when there is no signal in the integration range, making it possible to calculate the composite gain stably even immediately after startup.

[0119] In the first to fifth embodiments, an example has been described in which the integration range for integrating the data for calculating the composite gain is gradually narrowed, but it is also possible to widen the integration range.

[0120] <Other Embodiments> As described above, the present technology has been described using the first to fifth embodiments. However, the descriptions and drawings that form part of this disclosure should not be understood to limit the present technology. Upon understanding the gist of the technical content disclosed in the above embodiment, it will be apparent to those skilled in the art that various alternative embodiments, examples, and operational techniques may be included in the present technology. Furthermore, the configurations disclosed in the first to fifth embodiments may be appropriately combined within a range that does not cause contradictions. For example, configurations disclosed in multiple different embodiments may be combined, or configurations disclosed in multiple different modified examples of the same embodiment may be combined.

[0121] <Application Example to Electronic Devices> The above-described imaging device can be applied to various electronic devices, such as imaging devices such as digital still cameras and digital video cameras, mobile phones with imaging functions, or other devices with imaging functions. Fig. 18 is a block diagram showing a configuration example of an imaging system as an electronic device to which the present technology is applied.

[0122] The imaging system 2201 shown in Figure 18 is configured with an optical system 2202, a shutter device 2203, a solid-state imaging element 2204 as an imaging device, a control circuit 2205, a signal processing circuit 2206, a monitor 2207, and a memory 2208, and is capable of capturing still images and moving images.

[0123] The optical system 2202 is configured with one or more lenses, and guides light (incident light) from a subject to the solid-state image sensor 2204, forming an image on the light receiving surface of the solid-state image sensor 2204. The shutter device 2203 is disposed between the optical system 2202 and the solid-state image sensor 2204, and controls the light irradiation period and light blocking period of the solid-state image sensor 2204 under the control of the control circuit 2205.

[0124] The solid-state imaging element 2204 is configured by a package including the above-mentioned solid-state imaging element. The solid-state imaging element 2204 accumulates signal charges for a certain period of time in response to light that is imaged on the light-receiving surface via the optical system 2202 and the shutter device 2203. The signal charges accumulated in the solid-state imaging element 2204 are transferred in accordance with a drive signal (timing signal) supplied from the control circuit 2205.

[0125] The control circuit 2205 outputs a drive signal that controls the transfer operation of the solid-state image sensor 2204 and the shutter operation of the shutter device 2203 , thereby driving the solid-state image sensor 2204 and the shutter device 2203 .

[0126] The signal processing circuit 2206 performs various signal processing on the signal charges output from the solid-state imaging element 2204. The image (image data) obtained by performing the signal processing by the signal processing circuit 2206 is supplied to a monitor 2207 for display, or supplied to a memory 2208 for storage (recording). In the imaging system 2201 configured in this manner, the imaging devices 1 and 1A can also be applied in place of the solid-state imaging element 2204 described above.

[0127] <Application to a Mobile Body> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0128] Fig. 19 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 19, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 also includes a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (interface) 12053.

[0129] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0130] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0131] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0132] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0133] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0134] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.

[0135] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0136] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0137] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of Fig. 19, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0138] 20 is a diagram showing an example of the installation position of the image capturing unit 12031. In FIG. 20, a vehicle 12100 has image capturing units 12101, 12102, 12103, 12104, and 12105 as the image capturing unit 12031.

[0139] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The forward images acquired by the imaging units 12101 and 12105 are mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0140] 20 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0141] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0142] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which runs autonomously without relying on driver operation.

[0143] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0144] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0145] The present disclosure may also be configured as follows: (1) An imaging device comprising: a pixel array unit configured to arrange a plurality of pixel units in a matrix, each outputting a first pixel signal having a first sensitivity and a second pixel signal having a second sensitivity lower than the first sensitivity; a composite gain calculation unit that calculates a composite gain to be multiplied by the second pixel signal; and a combination unit that combines the second pixel signal multiplied by the composite gain with the first pixel signal, wherein at least a portion of the plurality of pixel units constitutes a captured image of one effective frame, and the composite gain calculation unit gradually changes, in accordance with a preset condition, a first integration range for integrating data for calculating the composite gain to a second integration range different from the first integration range. (2) The imaging device according to (1), wherein the composite gain calculation unit calculates a sensitivity ratio between the first pixel signal and the second pixel signal from the first integrated value and the second integrated value, as the composite gain. (3) The imaging device according to (1), wherein the composite gain calculation unit uses at least one of an elapsed time from startup, an environmental change of the subject from startup, and a number of valid frames arranged in chronological order from startup to determine the condition. (4) The imaging device according to (1), wherein the second integration range is narrower than the first integration range. (5) The imaging device described in (4) above, wherein the first integration range is a first integration effective pixel range consisting of a plurality of pixel units that executes integration of data for calculating the composite gain within a captured image of one effective frame, the second integration range is a second integration effective pixel range that is made up of a smaller number of pixel units than the first integration effective pixel range, and the composite gain calculation unit gradually narrows the first integration effective pixel range to the second integration effective pixel range in accordance with the number of effective frames that are arranged in chronological order from startup.(6) The imaging device described in (4) above, wherein the first integration range is a first integrated effective pixel level range in which integration of data for calculating the composite gain is performed when it is determined that the first pixel signal is equal to or greater than a first signal level and equal to or less than a second signal level, and the second integration range is a second integrated effective pixel level range in which integration of data for calculating the composite gain is performed when it is determined that the first pixel signal is equal to or greater than a third signal level that is greater than the first signal level and equal to or less than a fourth signal level that is less than the second signal level, and the composite gain calculation unit gradually narrows the first integrated effective pixel level range to the second integrated effective pixel level range in accordance with the number of effective frames arranged in chronological order from startup. (7) The imaging device according to (1), wherein each of the plurality of pixel units is provided with a color filter that transmits light of a wavelength of one color from a plurality of mutually different colors, and wherein the composite gain calculation unit, immediately after startup, when a composite gain corresponding to one or more color filters from the plurality of color filters is calculated, calculates a composite gain corresponding to an uncalculated color filter using the calculated composite gain. (8) The imaging device according to (7), wherein the composite gain calculation unit includes: a processing unit that calculates a composite gain based on the first pixel signal and the second pixel signal output from each of the plurality of pixel units in which the color filters are arranged; and a memory control unit that, when a composite gain is calculated by the processing unit, stores the composite gain in a memory unit, and the processing unit calculates a composite gain corresponding to an uncalculated color filter using the composite gain stored in the memory. (9) The imaging device according to (8), wherein the composite gain calculation unit further includes an update control unit that updates the value of the composite gain of the corresponding color filter stored in the memory each time a composite gain is calculated by the processing unit.(10) The composite gain calculation unit has: a first mode in which a first integrating effective pixel range consisting of a plurality of pixel units for integrating data for calculating the composite gain is gradually narrowed to a second integrating effective pixel range consisting of a smaller number of pixel units than the first integrating effective pixel range, in accordance with the number of effective frames arranged in chronological order from startup; and a second mode in which a first integrating effective pixel level range consisting of a plurality of pixel units in which the first pixel signal is equal to or greater than a first signal level and equal to or less than a second signal level is gradually narrowed to a second integrating effective pixel level range consisting of a plurality of pixel units in which the first pixel signal is equal to or greater than a third signal level that is greater than the first signal level and equal to or less than a fourth signal level that is less than the second signal level, in accordance with the number of effective frames arranged in chronological order from startup. The imaging device described in (1) above further comprises: a third mode in which, when a color filter that transmits light wavelengths of one color from a plurality of different colors is arranged in each of the plurality of pixel units, if a composite gain corresponding to one or more color filters from the plurality of color filters is calculated immediately after startup, the calculated composite gain is used to calculate a composite gain corresponding to an uncalculated color filter; and a mode selection control unit that selectively executes the first to third modes, or any combination of the first to third modes, in accordance with a mode designation operation. (11) An electronic device comprising: a pixel array section in which a plurality of pixel units are arranged in a matrix, each outputting a first pixel signal having a first sensitivity and a second pixel signal having a second sensitivity lower than the first sensitivity; a composite gain calculation section that calculates a composite gain to be multiplied by the second pixel signal; and a combination section that combines the second pixel signal multiplied by the composite gain and the first pixel signal, wherein at least a portion of the plurality of pixel units constitutes a captured image of one effective frame, and the composite gain calculation section gradually changes, in accordance with a preset condition, a first integration range in which integration of data for calculating the composite gain is performed to a second integration range different from the first integration range.

[0146] 1, 1A, 1B, 1C Imaging device 11 Pixel array section 11A Sensor chip 12 Vertical drive section 13 Column processing section 14 Horizontal drive section 15 System control section 16 Signal processing section 16A Signal processing section 17 Data storage section 18 Pixel drive line 19 Vertical signal line (VSL) 21 HDR synthesis section 22, 22A, 22B, 22C Correction gain generation circuit 110 Pixel 111 First photoelectric conversion section 112 First transfer gate section 113 Second photoelectric conversion section 114 Second transfer gate section 115 Third transfer gate section 116 Charge accumulation section 117 Reset gate section 118 FD (floating diffusion) section 119 Amplifying transistor 120 Selection transistor 121 Constant current source 211 First blend section 212 First multiplication unit 213 Second blending unit 214 Second multiplication unit 215 Third blending unit 216 Third multiplication unit 221 First selector 222 Second selector 223 First accumulating unit 224 Second accumulating unit 225, 225B Synthesis gain processing unit 226 IIR filter 227 Synthesis gain register 228 Moving subject determination unit 229 Color determination unit 230 Pixel area determination unit 231 Signal level determination unit 232 AND circuit 233 Counter 234 Filter strength calculation unit 235 Update determination counter 236 Various setting registers 237, 237A,237B Integration condition threshold generation unit 240 Memory unit 300 Color filter 2201 Imaging system 2202 Optical system 2203 Shutter device 2204 Solid-state imaging element 2205 Control circuit 2206 Signal processing circuit 2207 Monitor 2208 Memory 12000 Vehicle control system 12001 Communication network 12010 Drive system control unit 12020 Body system control unit 12030 Outside vehicle information detection unit 12031 Imaging unit 12040 Inside vehicle information detection unit 12041 Driver state detection unit 12050 Integrated control unit 12051 Microcomputer 12052 Audio / image output unit 12061 Audio speaker 12062 Display unit 12063 Instrument panel 12100 Vehicle 12101 to 12105 Imaging unit 12111 to 12114 Imaging range A11 Maximum value of integrated effective pixel range A12 Integrated effective pixel range A13 Minimum value of integrated effective pixel range B11 Maximum value of integrated effective pixel level range B12 Integrated effective pixel level range B13 Minimum value of integrated effective pixel level range

Claims

1. An imaging device comprising: a pixel array section in which a plurality of pixel units are arranged in a matrix, each outputting a first pixel signal having a first sensitivity and a second pixel signal having a second sensitivity lower than the first sensitivity; a composite gain calculation section that calculates a composite gain to be multiplied by the second pixel signal; and a synthesis section that synthesizes the second pixel signal multiplied by the composite gain and the first pixel signal, wherein at least a portion of the plurality of pixel units constitute a captured image of one effective frame, and the composite gain calculation section gradually changes, in accordance with preset conditions, from a first integration range in which data for calculating the composite gain is integrated to a second integration range different from the first integration range.

2. The imaging device of claim 1, wherein the composite gain calculation unit calculates a first integrated value by integrating the signal level of the first pixel signal, and calculates a second integrated value by integrating the signal level of the second pixel signal, and calculates the sensitivity ratio between the first pixel signal and the second pixel signal as the composite gain from the first integrated value and the second integrated value.

3. The imaging device according to claim 1, wherein the composite gain calculation unit uses at least one of the following to determine the conditions: the time elapsed since startup, changes in the subject's environment since startup, and the number of valid frames arranged in chronological order since startup.

4. The imaging device according to claim 1, wherein the second integration range is narrower than the first integration range.

5. The imaging device of claim 4, wherein the first integration range is a first integration effective pixel range consisting of a plurality of pixel units within a captured image of one effective frame for performing integration of data for calculating the composite gain, the second integration range is a second integration effective pixel range consisting of a smaller number of pixel units than the first integration effective pixel range, and the composite gain calculation unit gradually narrows the first integration effective pixel range to the second integration effective pixel range in accordance with the number of effective frames arranged in chronological order from startup.

6. The imaging device of claim 4, wherein the first integration range is a first integrated effective pixel level range in which integration of data for calculating the composite gain is performed when it is determined that the first pixel signal is equal to or greater than a first signal level and equal to or less than a second signal level, and the second integration range is a second integrated effective pixel level range in which integration of data for calculating the composite gain is performed when it is determined that the first pixel signal is equal to or greater than a third signal level that is greater than the first signal level and equal to or less than a fourth signal level that is less than the second signal level, and the composite gain calculation unit gradually narrows the first integrated effective pixel level range to the second integrated effective pixel level range in accordance with the number of effective frames arranged in chronological order from startup.

7. The imaging device of claim 1, wherein each of the plurality of pixel units is provided with a color filter that transmits a wavelength of light of one color from among a plurality of mutually different colors, and the composite gain calculation unit, immediately after startup, when a composite gain corresponding to one or more color filters from among the plurality of color filters is calculated, calculates a composite gain corresponding to a color filter that has not yet been calculated using the calculated composite gain.

8. The imaging device described in claim 7, wherein the composite gain calculation unit comprises: a processing unit that calculates a composite gain based on the first pixel signal and the second pixel signal output from each of a plurality of pixel units in which the color filters are arranged; and a memory control unit that stores the composite gain in a memory unit when the composite gain is calculated by the processing unit; and the processing unit calculates a composite gain corresponding to an uncalculated color filter using the composite gain stored in the memory unit.

9. The imaging device according to claim 8, wherein the composite gain calculation unit further includes an update control unit that updates the composite gain value of the corresponding color filter stored in the memory unit each time the composite gain is calculated by the processing unit.

10. The composite gain calculation unit has: a first mode in which a first integrated effective pixel range consisting of a plurality of pixel units for integrating data for calculating the composite gain is gradually narrowed to a second integrated effective pixel range consisting of a smaller number of pixel units than the first integrated effective pixel range, in accordance with the number of effective frames arranged in chronological order from startup; and a second mode in which a first integrated effective pixel level range consisting of a plurality of pixel units in which the first pixel signal is equal to or greater than a first signal level and equal to or less than a second signal level is gradually narrowed to a second integrated effective pixel level range consisting of a plurality of pixel units in which the first pixel signal is equal to or greater than a third signal level that is greater than the first signal level and equal to or less than a fourth signal level that is less than the second signal level, in accordance with the number of effective frames arranged in chronological order from startup.

2. The imaging device according to claim 1, further comprising: a third mode in which, when a color filter that transmits a wavelength of light of one color from a plurality of mutually different colors is arranged in each of the plurality of pixel units, if a composite gain corresponding to one or more color filters from the plurality of color filters is calculated immediately after startup, the calculated composite gain is used to calculate a composite gain corresponding to an uncalculated color filter; and a mode selection control unit that selectively executes the first to third modes, or any combination of the first to third modes, in accordance with a mode designation operation.

11. An electronic device comprising: a pixel array section in which a plurality of pixel units are arranged in a matrix, each outputting a first pixel signal having a first sensitivity and a second pixel signal having a second sensitivity lower than the first sensitivity; a composite gain calculation section that calculates a composite gain to be multiplied by the second pixel signal; and a synthesis section that synthesizes the second pixel signal multiplied by the composite gain and the first pixel signal, wherein at least a portion of the plurality of pixel units constitute a captured image of one effective frame, and the composite gain calculation section gradually changes, in accordance with preset conditions, a first integration range in which data for calculating the composite gain is integrated to a second integration range different from the first integration range.

Images