Analog / digital converter, signal processing device, imaging device, and electronic apparatus

Abstract

To enable an implementation of an AD conversion while applying a gain by using a ΔΣAD converter.SOLUTION: An image processing device includes: analog / digital conversion means of being configured to perform an analog / digital conversion on an image signal output from a pixel using ΔΣ modulation; determination means of being configured to determine a gain applied to the image signal; and control means of being configured to control a feedback amount used by the analog / digital conversion means in the analog / digital conversion on the image signal based on the gain. When the gain is a first gain, the control means makes the feedback amount smaller than when the gain is a second gain smaller than the first gain.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an analog-to-digital converter, and a signal processing device, an imaging device, and an electronic device that use the same, and in particular to a technology for analog-to-digital conversion using ΔΣ modulation. [Background technology]

[0002] In recent years, CMOS image sensors have been achieving higher pixel counts and faster readout speeds. These efforts have been supported by arranging and operating multiple analog-to-digital converters (ADCs) in parallel. However, the conventional slope-type ADCs have issues with the increased circuit size and power consumption associated with the parallel arrangement of multiple ADCs.

[0003] For this reason, CMOS image sensors have been proposed that use ΔΣ ADCs, which have a smaller circuit scale, require a lower voltage for the ADC, and are capable of high-speed AD conversion compared to slope-type ADCs. Patent Document 1 discloses the configuration of a CMOS image sensor that uses ΔΣ modulation in the AD conversion circuit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 3904111 Summary of the Invention [Problem to be solved by the invention]

[0005] However, whereas a slope-type ADC can apply gain during AD conversion depending on the slope of the reference voltage used during conversion, a ΔΣ ADC configured as in Patent Document 1 cannot apply gain during conversion. Therefore, in a ΔΣ ADC, a digital gain is applied in the signal processing unit after AD conversion, but the greater the digital gain, the lower the bit precision of the final output becomes.

[0006] Furthermore, since the digital gain amplifies noise along with the signal, the larger the digital gain, the more noticeable the noise becomes, especially in low-brightness areas.

[0007] The present invention has been made in consideration of the above problems, and has as its object to enable AD conversion to be performed while applying gain using a ΔΣ AD converter. [Means for solving the problem]

[0008] In order to achieve the above object, the analog-to-digital converter of the present invention comprises an analog-to-digital conversion means that performs analog-to-digital conversion using ΔΣ modulation on an image signal output from a pixel, a determination means that determines a gain to be applied to the image signal, and a control means that controls the amount of feedback used by the analog-to-digital conversion means in the analog-to-digital conversion of the image signal based on the gain, wherein the control means reduces the amount of feedback when the gain is a first gain compared to when the gain is a second gain that is smaller than the first gain. [Effects of the Invention]

[0009] According to the present invention, it is possible to perform AD conversion while applying gain using a ΔΣ AD converter. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of an imaging apparatus according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram showing a schematic configuration of an image sensor according to an embodiment. [Figure 3] FIG. 2 is a block diagram showing the configuration of a readout circuit according to the embodiment. [Figure 4] FIG. 2 is a block diagram showing the configuration of a ΔΣ ADC according to an embodiment. [Figure 5A] FIG. 3 is a diagram showing an example of an input voltage and an output voltage of a comparator according to the first embodiment. [Figure 5B] FIG. 4 is a diagram showing another example of the input voltage and the output voltage of the comparator according to the first embodiment. [Figure 6] 5 is a flowchart showing an AD conversion operation when the digital gain is replaced with the feedback amount in the first embodiment. [Figure 7] 5 is a timing chart showing the timing of brightness determination, switching of the feedback amount, and AD conversion in the first embodiment. [Figure 8] 10A and 10B are diagrams illustrating input / output characteristics of non-Log and Log moving images in the second embodiment. [Figure 9] FIG. 10 is a diagram showing outputs for input signals of each ISO in the second embodiment. [Figure 10] 10 is a flowchart showing an AD conversion operation according to the luminance of an input signal in the second embodiment. [Figure 11] 10A and 10B are diagrams showing examples of a gamma curve in SDR mode and gamma curves in the PQ and HLG formats in HDR mode according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] First Embodiment 1 is a block diagram showing the configuration of an imaging device using an imaging element according to a first embodiment of the present invention. The imaging device to which the present invention can be applied may be any electronic device equipped with a camera function. Electronic devices include, for example, video cameras, computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, robots, drones, drive recorders, etc. These are merely examples, and the present invention can also be applied to other electronic devices.

[0013] 1, lens unit 101 is configured to include a plurality of lenses, such as a zoom lens and a focus lens, and an aperture, and zoom control, focus control, aperture control, etc., are performed by lens drive device 102. An optical image of a subject incident through lens unit 101 is formed on image sensor 105. Note that lens unit 101 may be configured as an integral part of the image capture device, or may be configured to be detachable. The mechanical shutter 103 is controlled by a shutter driving device 104. The image sensor 105 photoelectrically converts the optical image of the subject formed by the lens unit 101, and outputs an image signal.

[0014] The signal processing circuit 106 performs digital gain processing to apply digital gain to the image signal output from the image sensor 105, various corrections, data compression, etc., and outputs image data. The memory unit 107 is used to temporarily store image data. The system control unit 108 performs various calculations and controls the entire imaging device. The I / F unit 109 is an interface for recording to or reading from a recording medium 110, which is a semiconductor memory such as a flash memory for holding image data, etc., and is configured to be removable. The display unit 111 displays various information and captured images.

[0015] Next, the operation of the imaging device having the above configuration during photography will be briefly described. When a main power supply (not shown) is turned on, the system control unit 108 is powered on, and further, the power of imaging system circuits such as the signal processing circuit 106 is also powered on. Then, when a command to shoot is issued by pressing a release button (not shown), the shooting operation starts. When the shooting operation is completed, the image signal output from the image sensor 105 is subjected to image processing, including various corrections and digital gain processing, in the signal processing circuit 106, and the obtained image data is written to the memory unit 107 at the command of the system control unit 108. The image data held in the memory unit 107 is recorded on a removable recording medium 110, such as a semiconductor memory, via the I / F unit 109 under the control of the system control unit 108.

[0016] Furthermore, the RAW image data may be sent to a computer or the like via an external I / F unit (not shown), and image processing may be performed in the computer or the like.

[0017] FIG. 2 is a block diagram showing a schematic configuration of the image sensor 105 in this embodiment. The image sensor 105 includes a pixel section 200, a vertical scanning section 202, a timing generator (TG) 203, a circuit control section 204, a CLK generating section 205, vertical transfer lines 206, a signal readout section 207, and an output section 209.

[0018] The pixel unit 200 has a plurality of pixels 201 arranged in a matrix. For ease of explanation, the pixels 201 are shown as a 4×4 pixel array in the pixel unit 200, but in practice, a large number of pixels, for example, several million or more, are arranged. The signal readout unit 207 has a plurality of readout circuits 208.

[0019] The vertical scanning unit 202 selects pixels 201 in the pixel unit 200 on a row-by-row basis and supplies a plurality of drive signals to each pixel 201 in the selected row. As a result, pixel signals of the pixels 201 in the selected row are transferred to a signal readout unit 207 via vertical transfer lines 206. The transferred pixel signals are converted into digital signals in each readout circuit 208 and output via an output unit 209.

[0020] The readout circuit 208 converts the input pixel signals into digital signal values ​​by AD conversion using a predetermined number of bits. The AD conversion performed here uses a so-called ΔΣ AD conversion method. The output unit 209 converts the digital signal value of each pixel into a predetermined signal format and outputs it to the outside of the image sensor 105 via a transmission path.

[0021] The TG 203 sends a timing signal to the vertical scanning unit 202, and the vertical scanning unit 202 generates and drives drive signals for driving the pixels 201 based on the timing signal. The circuit control unit 204 controls the CLK generation unit 205 and the signal readout unit 207 based on the timing signal from the TG 203. Furthermore, the CLK generation unit 205 generates a clock signal to be supplied to the signal readout unit 207 based on the timing signal supplied from the TG 203 and control by the circuit control unit 204.

[0022] FIG. 3 is a block diagram showing the configuration of each read circuit 208 shown in FIG. The readout circuit 208 includes an analog-to-digital converter (ΔΣ ADC) 300 that uses ΔΣ modulation and a digital filter 301. The ΔΣ ADC 300 converts pixel signals input via the vertical transfer lines 206 into digital signals using ΔΣ modulation. The digital filter 301 removes out-of-band quantization noise that has been shifted to higher frequencies by ΔΣ modulation. It also performs decimation and moving average on the high-rate output of the ΔΣ ADC 300 to reduce the output rate or convert it into a multi-bit signal value. The digital signal processed by the digital filter 301 is output to the output unit 209.

[0023] Furthermore, in a luminance detection process described later, the digital signal processed by the digital filter 301 is fed back to the circuit control unit 204. Based on this digital signal, the circuit control unit 204 controls the DA conversion magnification for controlling the amount of feedback in the ΔΣ ADC 300, or controls the digital filter 301 so as not to perform AD conversion. This control by the circuit control unit 204 will be described in detail later.

[0024] FIG. 4 is a block diagram showing the configuration of the ΔΣ ADC 300 shown in FIG. As shown in FIG. 4, the ΔΣ ADC 300 includes a subtraction circuit 400, an integration circuit 401, a comparator 402, and a digital-to-analog converter (DAC) 403.

[0025] The subtraction circuit 400 outputs the difference between the pixel signal input via the vertical transfer line 206 and the output signal of the DAC 403 to the integration circuit 401. Note that a sample-and-hold circuit may be provided between the vertical transfer line 206 and the subtraction circuit 400. In this case, the pixel signal output from the sample-and-hold circuit is input to the subtraction circuit 400.

[0026] The integrating circuit 401 includes an integrator that integrates the input signal from the subtracting circuit 400. As the integrating circuit 401, for example, a general integrating circuit such as a gm-C integrating circuit using a transconductor or an RC integrating circuit using an operational amplifier can be used.

[0027] The comparator 402 synchronizes with the clock signal, compares the reference voltage Vref with the voltage signal output from the integrating circuit 401, and outputs the comparison result as a 1-bit digital signal. For example, if the voltage signal output from the integrating circuit 401 is lower than the reference voltage Vref, it outputs 0, and if the voltage signal is equal to or higher than the reference voltage Vref, it outputs 1. This digital signal is supplied to the digital filter 301 and the DAC 403.

[0028] The DAC 403 converts the digital signal output from the comparator 402 into a predetermined analog signal amount and outputs it to the subtraction circuit 400. For example, when the digital signal output is 1, the DAC 403 outputs an analog signal (feedback amount) at a level according to a DA conversion magnification determined by the circuit control unit 204 as described below. Note that the DAC 403 may be configured using various known circuits.

[0029] Next, the DA conversion ratio and the feedback amount in this embodiment will be described with reference to FIGS. 5A and 5B.

[0030] 5A and 5B show an example of the input voltage Vcomp to the comparator 402 and the output voltage COMPout of the comparator 402. Note that in Fig. 5A and 5B, an example is shown in which the input / output voltage range is 1V. 5A shows an example of a case where the DA conversion magnification at which the DAC 403 converts the digital signal output from the comparator 402 into an analog signal (feedback amount) is changed for pixel signals Vin of the same level input to the ΔΣ ADC 300 via the vertical transfer line 206. Such a waveform pattern is repeated for a predetermined time.

[0031] 5A(a) shows the input voltage Vcomp and reference voltage Vref to the comparator 402, and FIG. 5A(c) shows the output voltage COMPout, which is a digital signal ΔΣ-modulated by the comparator 402 with respect to the input voltage Vcomp shown in FIG. 5A(a). In contrast, FIG. 5A(b) shows the input voltage Vcomp and reference voltage Vref to the comparator 402 when the DA conversion magnification is set to 1 / 2 of that in FIG. 5A(a) and the amount of feedback input to the subtraction circuit 400 is halved when a pixel signal Vin of the same level as in FIG. 5A(a) is input. FIG. 5A(d) shows the output voltage COMPout of the comparator 402 with respect to the input voltage Vcomp shown in FIG. 5A(b).

[0032] 5A, by halving the DA conversion magnification and halving the amount of feedback, the number of times that the output signal COMPout from the comparator 402 becomes Hi doubles. In this way, by setting the DA conversion magnification of the DAC 403 to 1 / N and the amount of feedback to 1 / N, the number of times that the output signal COMPout from the comparator 402 becomes Hi, i.e., the number of times that Hi is output by AD conversion of the ΔΣ ADC 300, becomes N times.

[0033] The voltage waveform shown in FIG. 5B shows an example in which the DA conversion magnification is changed when the level of the input pixel signal Vin is higher than the level of the pixel signal Vin shown in FIG. 5A. Fig. 5B(a) shows the input voltage Vcomp to the comparator 402 and the reference voltage Vref when the DA conversion magnification is the same as Fig. 5A(a). Fig. 5B(c) shows the output signal COMPout of the comparator 402 with respect to the input voltage Vcomp shown in Fig. 5B(a). Compared to Figs. 5A(a) and (c), Figs. 5B(a) and (c) show that the output voltage COMPout becomes Hi more frequently due to AD conversion by the ΔΣ ADC 300 depending on the level of the pixel signal Vin.

[0034] 5B(b) shows the input voltage Vcomp to the comparator 402 and the reference voltage Vref when the DA conversion magnification is set to 1 / 2 of that in FIG. 5B(a) and the feedback amount input to the subtraction circuit 400 is halved when the pixel signal Vin at the same level as that in FIG. 5B(a) is input. FIG. 5B(d) shows the output voltage COMPout of the comparator 402 with respect to the input voltage Vcomp shown in FIG. 5B(b). In this case, by setting the DA conversion magnification to 1 / 2 of that in FIG. 5B(a), after a certain time, the output of the integrator circuit 401 always becomes greater than 0, the input voltage Vcomp to the comparator 402 diverges, and the operation of the ΔΣ ADC 300 becomes unstable.

[0035] In consideration of the above phenomenon, in this embodiment, the magnitude of the pixel signal is determined, and the DA conversion magnification used in the DAC 403 is determined according to the determination result. The magnitude of this pixel signal may be determined by performing AD conversion in advance using the ΔΣADC 300 at a resolution required for the determination, or by providing a separate determination circuit. An example of a separate determination circuit is one in which the voltage of the input pixel signal is compared with a changeable comparison voltage, and 0 or 1 is output depending on the comparison result. The voltage indicating the comparison result is changed depending on the digital gain, as described below. A processing example using the ΔΣADC 300 will be described later.

[0036] Table 1 shows the relationship between ISO sensitivity, DA conversion ratio, digital gain, and bit precision.

[0037] [Table 1]

[0038] Table 1(a) shows the case where the DA conversion multiplier is set to 1 regardless of ISO sensitivity, and the bit precision of AD conversion, including digital filter processing, is set to 14 bits at ISO 100. In this case, to increase the ISO sensitivity to ISO 200, ISO 400, etc., it is necessary to apply digital gain Dgain to the digital signal after AD conversion. As a result, the bit precision of the final output is 1 / Dgain, which becomes 10 bits at ISO 1600. Furthermore, images obtained by applying digital gain Dgain are subject to adverse effects such as tone jumps and quantization noise being multiplied by the gain.

[0039] To eliminate these effects and prevent a decrease in the bit precision of the final output regardless of the ISO sensitivity, in this embodiment, the feedback amount is changed by changing the DA conversion magnification according to the ISO sensitivity, as shown in Table 1(b). For example, by setting the DA conversion magnification to 1 / 16 at ISO 1600, the comp output ratio, which indicates the output ratio of the comparator 402 to the feedback amount at ISO 100 (digital gain is 1), becomes 16 times as a result of AD conversion. In this way, an output equivalent to that when the digital gain Dgain is 16 can be obtained without applying the digital gain Dgain. Furthermore, the bit precision of the final output becomes 14 bits, and quantization noise is no longer amplified by the digital gain Dgain.

[0040] In the above example, the digital gain determined by the ISO speed was described. However, the digital gain used for various corrections, such as peripheral illumination correction, may be replaced with a feedback amount. That is, the DA conversion magnification is set to the reciprocal of the digital gain (1 / Dgain), and the feedback amount when the digital gain is 1 is set to the reciprocal of the digital gain. For example, if a lens with significant peripheral illumination falloff is attached and digital gain is applied to the periphery of the image according to the image height through peripheral illumination correction, tone jumps and noise in the periphery of the image can be reduced even after correction by replacing the digital gain with the feedback amount as shown in Table 1(b) according to the image height instead of the ISO speed.

[0041] Alternatively, a portion of the digital gain may be replaced with the feedback amount, and the AD conversion coefficient of the feedback amount and the digital gain may be used together. That is, the DA conversion magnification is set to the reciprocal of the digital gain divided by an integer, which is smaller than the feedback amount when the digital gain is 1. Then, the gain portion not replaced with the feedback amount is amplified by the digital gain.

[0042] For high-brightness input signals that would saturate if digital gain were applied, it is more efficient to control the digital filter 301 to output a signal indicating saturation without performing AD conversion. For example, when applying a digital gain of 2x at ISO 200, signals that do not saturate even when gain is applied are signals that are equal to or less than half the input range of the ΔΣ ADC 300. In other words, if the input signal is larger than half the input range of the ΔΣ ADC 300, setting the DA conversion magnification to 1 / 2 and the feedback amount to 1 / 2 will cause the operation of the ΔΣ ADC 300 to become unstable.

[0043] Therefore, when replacing 2x digital gain with a DA conversion multiplier of 1 / 2, the brightness of the input signal is determined using a threshold of 1 / 2 the input range, and if the input signal is below the threshold, it is replaced with a DA conversion multiplier of 1 / 2 and AD conversion is performed.Furthermore, if the input signal is greater than the threshold, performing AD conversion at a DA conversion multiplier of 1 / 2 will not only result in saturation but will also make the operation of the ΔΣADC300 unstable, so the ΔΣADC300 is controlled so that AD conversion itself is not performed.

[0044] As described above, the brightness determination threshold for determining whether to perform AD conversion is determined according to the content of various corrections such as ISO sensitivity and peripheral illumination correction, and the magnitude of the digital gain when the DA conversion magnification is 1, as shown in Table 1(a). More specifically, 1 / Dgain of the input range is determined as the threshold.

[0045] Next, the AD conversion operation in this embodiment when the digital gain is replaced with the feedback amount (DA conversion magnification) will be described with reference to the flowchart in Fig. 6. This operation is repeated for each readout circuit 208 every time a new signal is input.

[0046] First, in S601, the circuit control unit 204 sets a threshold value to be used for brightness determination according to the digital gain to be replaced with the feedback amount in the digital filter 301. For example, the threshold value for brightness determination is determined as ½ of the input range of the ΔΣADC 300 when the normal digital gain shown in Table 1(a) is 2x, or ¼ of the input range when the digital gain is 4.

[0047] In S602, the digital filter 301 determines whether the input pixel signal is equal to or less than a threshold value, and sends a Judge signal indicating the result of the determination to the circuit control unit 204. If the pixel signal is equal to or less than the threshold value based on the Judge signal, the circuit control unit 204 transmits a DA conversion magnification corresponding to the replacement digital gain to the DAC 403 in S603, and the DAC 403 changes the feedback amount of the DA conversion magnification. Then, in S605, the ΔΣ ADC 300 performs AD conversion.

[0048] On the other hand, if the input pixel signal is greater than the threshold, replacing the digital gain with the feedback amount and performing AD conversion will cause the operation of the ΔΣ ADC 300 to become unstable, so the AD conversion is stopped in S604. Then, in S606, the circuit control unit 204 controls the digital filter 301 to output a saturation signal. When the process of S605 or S606 is completed, the process for one pixel signal is completed.

[0049] 7 shows a timing chart of the processing from S601 to S605 when the input pixel signal is equal to or lower than the threshold. In Fig. 7, period P1 corresponds to the processing in S601 and S602, period P2 corresponds to the processing in S603, and period P3 corresponds to the processing in S605.

[0050] 7, the circuit control unit 204 sets a threshold value in the digital filter 301 based on the digital gain to be replaced with the feedback amount. Then, the digital filter 301 sets the DA conversion magnification of the DAC 403 to 1, does not change the feedback amount, performs AD conversion on the input voltage Vcomp to the comparator 402 with low bit accuracy in a short period of time, compares the resulting digital signal with the set threshold value, and outputs a Judge signal indicating the comparison result.

[0051] Next, in a period P2, the circuit control section 204 controls the DA conversion magnification of the DAC 403 based on the Judge signal input from the digital filter 301 in the period P1.

[0052] Here, because the digital signal of the input pixel signal is equal to or less than the threshold, AD conversion is performed during period P3 with the amount of feedback from the DAC 403 adjusted by the DA conversion magnification. For example, when ISO200 is selected, the circuit control unit 204 sets the DA conversion magnification in the DAC 403 to 1 / 2, and controls the amount of feedback to 1 / 2. As a result, the amount of feedback from the DAC 403 when the output voltage COMPout becomes Hi decreases, so the waveform of the input voltage Vcomp to the comparator 402 during period P3 differs from that during period P1, and the cycle in which the output voltage COMPout becomes Hi is 1 / 2 of that during period P1.

[0053] As described above, according to the first embodiment, by replacing the digital gain with the feedback amount, it is possible to obtain an output equivalent to the digital gain, without reducing the bit precision of the final output, and to obtain a good image with reduced tone jumps and increased quantization noise.

[0054] In addition, by controlling the system to output a saturated signal without performing AD conversion when the input signal is greater than the threshold, the ΔΣ ADC can be converted to a digital signal without compromising its stability.

[0055] <Second embodiment> Next, a second embodiment of the present invention will be described. In the second embodiment, a method for reducing noise in a low-luminance input signal by changing the amount of feedback will be described. Note that the imaging device in the second embodiment can be the same as that described in the first embodiment with reference to Figures 1 to 4, and therefore a description thereof will be omitted.

[0056] Log video, which has different input / output characteristics from normal video, is known as a shooting mode for obtaining a wide dynamic range. For example, Log video has input / output characteristics such as those shown in Figure 8. The horizontal axis in Figure 8 shows the number of stops when 18% gray is set to 0, and the vertical axis shows the signal level in a 10-bit recording file. The dashed line shows the characteristics of non-Log BT.709, and the solid line shows the characteristics of Log video. As Figure 8 shows, the slope of the dark areas is steeper in Log video than in BT.709, and by allocating more signal level to the dark areas, the dark gradation is enriched. However, this results in a state where a high gain is applied, which makes noise in the dark areas more noticeable.

[0057] In contrast, in the second embodiment, for low luminance signals, the amount of feedback is changed to amplify the output without amplifying the quantization noise, and the quantization noise is reduced by dividing this amplified amount by the gain.

[0058] Table 2 shows the relationship between ISO sensitivity, DA conversion magnification, return gain, digital gain, and bit precision in the second embodiment. Note that in the second embodiment, the ISO sensitivity and digital gain are also assumed to correspond to each other.

[0059] [Table 2]

[0060] Table 2(a) shows the relationship between ISO sensitivity, thresholds, and digital gain in the second embodiment. Threshold Th1 is a threshold for determining whether the operation of the ΣADC300 becomes unstable, and threshold Th2 is a threshold for determining whether changing the feedback amount will have a noise improvement effect. As described in the first embodiment, the thresholds are determined according to the input range of the ΔΣADC300, and therefore Table 2(a) shows a multiplication factor for the input range. Therefore, thresholds Th1 and Th2 are each a value obtained by multiplying the input range by the multiplication factor shown in Table 2(a). The method for comparing the input signal with the threshold is the same as the method described in the first embodiment, and therefore will not be described here.

[0061] Table 2(b) shows the case where the pixel signal Vin is at a low luminance level equal to or lower than the threshold value Th2. Here, the DA conversion magnification is further halved compared to the first embodiment, amplifying the output on the low-luminance side without amplifying the quantization noise. The quantization noise is further reduced by dividing the amplified amount by the digital gain. For example, in the case of ISO 100, the threshold value Th2 is half the input range of the ΔΣ ADC 300. Therefore, by dividing the DA conversion magnification by half, inputs at low luminance levels equal to or lower than the threshold value Th2 are converted to AD signals with double the output without using digital gain, and then multiplied by half the divided gain, thereby reducing the quantization noise by half. The divided gain is equal to the threshold value Th2 divided by the threshold value Th1. Thus, as described above, noise reduction is achieved when the luminance is equal to or lower than the threshold value Th2.

[0062] Table 2(c) shows the case where the pixel signal Vin is greater than the threshold Th2 and equal to or less than the threshold Th1, and the DA conversion magnification is changed according to the brightness to perform AD conversion and expand the dynamic range on the high-brightness side. For example, at ISO 200, the DA conversion magnification is set to 1 / 2, so that AD conversion with doubled output is performed without using digital gain. Furthermore, no rebate is performed using the rebate gain. This process is the same as the process in the first embodiment. By changing the amount of feedback for pixel signals with medium brightness instead of digital gain, it is possible to obtain a good image with reduced tone jumps and increased quantization noise without reducing the bit precision of the final output.

[0063] When the input pixel signal Vin has a high luminance exceeding the threshold value Th1, the AD conversion is stopped, as in the first embodiment. Then, by controlling the digital filter 301 to output a saturation signal, the conversion to a digital signal can be performed without impairing the stability of the operation of the ΔΣ ADC 300.

[0064] Figure 9 shows the output for an input signal at each ISO. The solid line portion for each ISO in Figure 9 corresponds to low luminance below threshold Th2, and shows the output for the input signal after AD conversion using the DA conversion multiplier shown in Table 2(b) and dividing by the division gain. The dashed line portion corresponds to medium luminance above threshold Th2 and below threshold Th1, and shows the output for the input signal after AD conversion using the DA conversion multiplier shown in Table 2(c) and dividing by the division gain. The dashed line portion corresponds to high luminance above threshold Th1, and shows the output for the input signal as a saturated signal.

[0065] 9, for example, in the case of ISO200, the threshold value Th1 at which the digital gain can be replaced with the feedback amount is 0.5, which is half the threshold value Th1 of 1 for ISO100. Also, the input range at which the output of the AD conversion for ISO200 becomes saturated is half that of ISO100, so if the dark areas are less than half the output of the AD conversion, the threshold value Th2 in order to obtain a noise improvement effect is 0.25, which is half the threshold value Th1 of 0.5 for ISO100. In FIG. 9, the input corresponding to 8192 LSB, which is the median value of the 14-bit output at each ISO sensitivity, is the threshold value Th2.

[0066] In this way, a threshold value is set to divide the input signal into low brightness, medium brightness, and high brightness, and AD conversion is performed by changing the DA conversion magnification and return gain according to the brightness.

[0067] Next, the flow of AD conversion operation according to the luminance of an input signal in the second embodiment will be described with reference to the flowchart in Fig. 10. Note that this operation is repeatedly performed for each readout circuit 208 every time a new signal is input.

[0068] First, in S1001, the circuit control unit 204 sets thresholds Th1 and Th2 to be used for brightness judgment in the digital filter 301 according to the digital gain. The thresholds Th1 and Th2 can be determined based on Table 2(a). Next, in S1002, the digital filter 301 compares the input pixel signal with the thresholds Th1 and Th2 and sends a Judge signal indicating the comparison result to the circuit control unit 204. Note that when this brightness judgment is performed using the ΔΣ ADC 300, with the settings shown in Table 2(b), it is necessary to judge the smallest threshold, 1 / 32, and therefore a higher resolution is required.

[0069] If the Judge signal indicates that the pixel signal is equal to or less than the threshold value Th2, the circuit control unit 204 transmits the DA conversion magnification to the DAC 403 in S1003, and the DAC 403 changes the feedback amount of the DA conversion magnification and performs AD conversion. Then, in S1004, the digital signal obtained by the AD conversion is divided by the division gain. The DA conversion magnification and division gain at this time are determined according to the ISO, as shown in Table 2(b).

[0070] If the Judge signal indicates that the pixel signal exceeds the threshold value Th2 and is equal to or less than the threshold value Th1, the circuit control unit 204 transmits the DA conversion magnification to the DAC 403 in S1005, and the DAC 403 changes the feedback amount of the DA conversion magnification and performs AD conversion. The DA conversion magnification at this time is determined according to ISO, as shown in Table 2(c). Furthermore, no refund processing is performed.

[0071] Furthermore, if the Judge signal indicates that the pixel signal exceeds the threshold value Th1, the AD conversion operation by the ΔΣ ADC 300 becomes unstable. Therefore, the AD conversion is stopped in S1006, and in S1007, the circuit control unit 204 controls the digital filter 301 to output a saturation signal. When the processing of S1004, S1005, or S1006 is completed, the processing for one pixel signal is completed.

[0072] In the above example, the DA conversion multiplier and the return gain are determined according to the ISO sensitivity, but as in the first embodiment, they may be determined according to the digital gain used for various corrections such as peripheral illumination correction.

[0073] FIG. 11 shows gamma curves for the Perceptual Quantization (PQ) method and the Hybrid Log Gamma (HLG) method in Standard Dynamic Range (SDR) mode and High Dynamic Range (HDR) mode, as examples of shooting modes other than Log video mode. The solid line shows an example of the gamma curve for SDR, the dotted line shows an example of the gamma curve for HLG, and the dashed line shows an example of the gamma curve for PQ. Unlike FIG. 8, the horizontal axis represents luminance. As with Log video, HLG allocates a higher signal level to dark areas compared to SDR, resulting in richer dark gradations, but more noticeable noise in dark areas. This tendency is even more pronounced with the PQ method. Therefore, for low luminance, the feedback amount is changed and the multiplied output is divided by the digital gain to compress quantization noise. For high luminance, noise in dark areas can be improved by performing AD conversion without changing the feedback amount. Therefore, this mode can also be used as a high-quality still image mode for SDR still images. In addition to the above, the control described in this embodiment may also be applied to live view mode.

[0074] As described above, according to the second embodiment, the amount of feedback and the rebate gain are controlled in accordance with the luminance of the input signal, thereby making it possible to further improve noise in dark areas.

[0075] In the above-described embodiment, a first-order ΔΣ modulator forming a first-order loop filter was used for explanation. However, the present invention can also be applied to a circuit configuration using a second-order or higher-order ΔΣ modulator to stabilize the feedback loop. In the case of a second-order or higher-order ΔΣ modulator, the feedback amount may be changed for each individual feedback loop. Furthermore, the subtraction circuit 400 may be an addition circuit.

[0076] Furthermore, in an incremental ΔΣAD, although the digital signal waveform differs from those shown in FIGS. 5A and 5B, the same effect can be obtained by changing the amount of feedback.

[0077] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.

[0078] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0079] <Summary> The disclosure of this embodiment includes the following configuration.

[0080] (Item 1) an analog-to-digital conversion means for performing analog-to-digital conversion using ΔΣ modulation on an image signal output from a pixel; a determination means for determining a gain to be applied to the image signal; a control means for controlling a feedback amount used in the analog-to-digital conversion of the image signal by the analog-to-digital conversion means based on the gain; The analog-to-digital converter is characterized in that the control means reduces the amount of feedback when the gain is a first gain compared to when the gain is a second gain that is smaller than the first gain. (Item 2) 2. The analog-to-digital converter according to item 1, wherein the control means multiplies the amount of feedback when the gain applied to the image signal is 1 by the reciprocal of the gain determined by the determination means. (Item 3) 2. The analog-to-digital converter according to item 1, wherein the control means controls the amount of feedback between a feedback amount when the gain applied to the image signal is 1 and a feedback amount obtained by multiplying the gain by the reciprocal determined by the determination means. (Item 4) The analog-to-digital converter according to any one of items 1 to 3, characterized in that the determining means determines the gain based on at least one of ISO sensitivity and a correction method including correction of peripheral light falloff. (Item 5) the control means further controls the output of the analog-to-digital conversion means; 5. The analog-to-digital converter according to any one of items 1 to 4, wherein when the image signal is greater than a predetermined first threshold, the control means controls the analog-to-digital conversion means to output a predetermined value without performing the analog-to-digital conversion. (Item 6) 6. The analog-to-digital converter according to item 5, wherein the first threshold value is a value obtained by dividing an input range in which the analog-to-digital conversion means can perform the analog-to-digital conversion by the gain. (Item 7) 7. The analog-to-digital converter according to any one of items 1 to 6, wherein when the image signal is equal to or less than a predetermined second threshold, the control means reduces the amount of feedback compared to when the image signal is greater than the second threshold. (Item 8) 8. The analog-to-digital converter according to item 7, wherein the second threshold is an integer fraction of a value obtained by dividing an input range in which the analog-to-digital conversion means can perform the analog-to-digital conversion by the gain. (Item 9) 9. The analog-to-digital converter according to item 7 or 8, wherein when a predetermined shooting mode is set, the control means controls the amount of feedback based on a result of comparing the image signal with the second threshold value. (Item 10) 10. The analog-to-digital converter according to item 9, wherein the predetermined shooting modes include at least one of a high-quality still image mode, a still image mode, a Log video mode, a video mode, a live view mode, and an HLG (Hybrid Log Gamma) method and a PQ (Perceptual Quantization) method in an HDR (High Dynamic Range) mode. (Item 11) The analog-to-digital conversion means Digital-to-analog conversion means; a subtraction means for subtracting the signal output from the analog-to-digital conversion means from the image signal; integration means for integrating the signal output from the subtraction means; a comparison means for comparing the signal output from the integration means with a predetermined threshold value and outputting a comparison result; the digital-to-analog conversion means outputs a signal indicating a feedback amount to the subtraction means in accordance with the comparison result; The control means controls the amount of feedback in the digital-to-analog conversion means. 11. An analog-to-digital converter according to any one of items 1 to 10. (Item 12) a plurality of the analog-to-digital converters according to item 7 or 8; a signal processing means for applying a gain to the digital signals converted by the plurality of analog-to-digital converters; The signal processing means applies a gain determined by the determination means and a gain according to the amount of feedback when the image signal is equal to or less than a predetermined second threshold. (Item 13) A plurality of the analog-to-digital converters according to any one of items 1 to 11; A plurality of the pixels; An imaging device comprising: (Item 14) The imaging element according to Item 13, processing means for processing the digital signal converted from analog to digital by the analog to digital converter; An electronic device comprising:

[0081] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0082] 105: imaging element, 106: signal processing circuit, 203: timing generator, 204: circuit control unit, 205: CLK generation circuit unit, 207: readout circuit unit, 208: readout circuit, 300: ΔΣ ADC, 301: digital filter, 400: subtraction circuit, 401: integration circuit, 402: comparator, 403: DAC

Claims

1. an analog-to-digital conversion means for performing analog-to-digital conversion using ΔΣ modulation on an image signal output from a pixel; a determination means for determining a gain to be applied to the image signal; a control means for controlling a feedback amount used in the analog-to-digital conversion of the image signal by the analog-to-digital conversion means based on the gain; 10. An analog-to-digital converter, wherein the control means reduces the amount of feedback when the gain is a first gain compared to when the gain is a second gain that is smaller than the first gain.

2. 2. The analog-to-digital converter according to claim 1, wherein the control means multiplies the amount of feedback when the gain applied to the image signal is 1 by the reciprocal of the gain determined by the determination means.

3. 2. The analog-to-digital converter according to claim 1, wherein the control means controls the amount of feedback between a feedback amount when a gain applied to the image signal is 1 and a feedback amount obtained by multiplying the gain determined by the determination means by the reciprocal.

4. 2. The analog-to-digital converter according to claim 1, wherein the determining means determines the gain based on at least one of ISO sensitivity and a correction method including correction of peripheral light falloff.

5. The control means further controls the output of the analog-to-digital conversion means, 2. The analog-to-digital converter according to claim 1, wherein, when the image signal is greater than a predetermined first threshold, the control means controls the analog-to-digital conversion means to output a predetermined value without performing the analog-to-digital conversion.

6. 6. The analog-to-digital converter according to claim 5, wherein the first threshold value is a value obtained by dividing an input range in which the analog-to-digital conversion means can perform the analog-to-digital conversion by the gain.

7. 2. The analog-to-digital converter according to claim 1, wherein when the image signal is equal to or less than a predetermined second threshold, the control means reduces the amount of feedback compared to when the image signal is greater than the second threshold.

8. 8. The analog-to-digital converter according to claim 7, wherein the second threshold is an integer fraction of a value obtained by dividing an input range in which the analog-to-digital conversion means can perform the analog-to-digital conversion by the gain.

9. 8. The analog-to-digital converter according to claim 7, wherein when a predetermined photographing mode is set, the control means controls the amount of feedback based on a result of comparison between the image signal and the second threshold value.

10. 10. The analog-to-digital converter according to claim 9, wherein the predetermined shooting modes include at least one of a high-quality still image mode, a still image mode, a Log video mode, a video mode, a live view mode, and an HLG (Hybrid Log Gamma) method and a PQ (Perceptual Quantization) method in an HDR (High Dynamic Range) mode.

11. The analog-to-digital conversion means Digital-to-analog conversion means; a subtraction means for subtracting the signal output from the digital-to-analog conversion means from the image signal; integration means for integrating the signal output from the subtraction means; a comparison means for comparing the signal output from the integration means with a predetermined threshold value and outputting a comparison result; the digital-to-analog conversion means outputs a signal indicating a feedback amount to the subtraction means in accordance with the comparison result; The control means controls the amount of feedback in the digital-to-analog conversion means.

2. An analog-to-digital converter according to claim 1.

12. a plurality of the analog-to-digital converters according to claim 7 or 8; a signal processing means for applying a gain to the digital signals converted from analog to digital by the plurality of analog to digital converters; The signal processing device is characterized in that, when the image signal is equal to or less than a predetermined second threshold, the signal processing means applies a gain determined by the determination means and a gain corresponding to the feedback amount.

13. A plurality of the analog-to-digital converters according to any one of claims 1 to 11; A plurality of the pixels; An imaging device comprising:

14. The imaging device according to claim 13; processing means for processing the digital signal converted from analog to digital by the analog to digital converter; An electronic device comprising:

Images