Analog-to-digital converter, method for controlling analog-to-digital converter, signal processing apparatus, image sensor, electronic apparatus, program product, and storage medium

Abstract

The invention relates to an analog-to-digital converter, a control method thereof, a signal processing apparatus, an image sensor, an electronic apparatus, a program product, and a storage medium. The analog-to-digital converter includes: an analog-to-digital conversion unit for performing analog-to-digital conversion using [delta] [sigma] modulation on an image signal output from a pixel; a determination unit configured to determine a gain to be applied to the image signal; and a control unit for controlling, based on the gain, a feedback amount used in the analog-to-digital conversion of the image signal by the analog-to-digital conversion unit. The control unit controls the feedback amount to be smaller in a case where the gain is a first gain than in a case where the gain is a second gain smaller than the first gain.

Description

TECHNICAL FIELD

[0001] The present application relates to an analog-digital converter and a control method thereof, and a signal processing apparatus, an image sensor and an electronic device using the analog-digital converter, a program product and a storage medium, and more particularly to an analog-digital conversion technology using delta sigma modulation. BACKGROUND

[0002] In recent years, the number of pixels and the speed of reading out signals from pixels have been increasing in CMOS image sensors. In order to cope with the increasing number of pixels and the increasing speed of reading signals, a large number of analog-digital converters (ADCs) are arranged in parallel and operated. However, in a conventional ramp ADC, the parallel arrangement of a large number of ADCs increases the circuit size and power consumption, which becomes a new problem.

[0003] For this reason, a CMOS image sensor using a delta sigma ADC has been proposed, which has a smaller circuit size, requires a lower voltage, and is capable of high-speed AD conversion compared to a ramp ADC. Japanese Patent 3904111 discloses a structure of a CMOS image sensor using delta sigma modulation in an AD conversion circuit.

[0004] However, a ramp ADC can apply a gain during AD conversion by using different slopes of a reference voltage during conversion, while a delta sigma ADC configured as in Japanese Patent 3904111 cannot apply a gain during AD conversion. Therefore, in a delta sigma ADC, a digital gain is applied in a signal processing unit after AD conversion, but the larger the digital gain, the lower the bit accuracy of the final output becomes.

[0005] Furthermore, since the digital gain amplifies noise along with the signal, the larger the digital gain, the more noticeable the noise becomes, particularly in a low luminance region. SUMMARY

[0006] The present application is made in view of the above circumstances, and enables a delta sigma AD converter to perform AD conversion while applying a gain.

[0007] According to the present application, there is provided an analog-digital converter including one or more processors and / or circuitry functioning as: an analog-digital conversion unit configured to perform analog-digital conversion of an image signal output from a pixel using delta sigma modulation; a determination unit configured to determine a gain to be applied to the image signal; and a control unit configured to control a feedback amount used in the analog-digital conversion of the image signal by the analog-digital conversion unit based on the gain, wherein the control unit controls the feedback amount to be smaller in a case where the gain is a first gain than in a case where the gain is a second gain smaller than the first gain.

[0008] Further, according to the present application, there is provided a signal processing apparatus including a plurality of analog-digital converters each including one or more processors and / or circuitry functioning as an analog-digital conversion unit for performing analog-digital conversion of an image signal output from a pixel using delta sigma modulation, a determination unit for determining a gain to be applied to the image signal, and a control unit for controlling a feedback amount used in the analog-digital conversion of the image signal by the analog-digital conversion unit based on the gain, wherein the control unit controls the feedback amount to be smaller in a case where the gain is a first gain than in a case where the gain is a second gain smaller than the first gain, and controls the feedback amount to be smaller in a case where the image signal is equal to or smaller than a second threshold than in a case where the image signal is larger than the second threshold, and a signal processing unit for applying a gain to a digital signal obtained by the analog-digital conversion by the plurality of analog-digital converters, wherein the signal processing unit applies a gain corresponding to the gain determined by the determination unit and the feedback amount in a case where the image signal is equal to or smaller than a predetermined second threshold.

[0009] Further, according to the present application, there is provided an image sensor including a plurality of analog-digital converters each including one or more processors and / or circuitry functioning as an analog-digital conversion unit for performing analog-digital conversion of an image signal output from a pixel using delta sigma modulation, a determination unit for determining a gain to be applied to the image signal, and a control unit for controlling a feedback amount used in the analog-digital conversion of the image signal by the analog-digital conversion unit based on the gain, wherein the control unit controls the feedback amount to be smaller in a case where the gain is a first gain than in a case where the gain is a second gain smaller than the first gain.

[0010] According to the present application, there is provided an electronic apparatus including the above-described image sensor, and a processing unit for processing a digital signal obtained by the analog-digital conversion by the analog-digital converters.

[0011] Further, according to the present application, there is provided a control method for controlling an analog-digital converter for performing analog-digital conversion of an image signal output from a pixel using delta sigma modulation, the control method including: judging a gain to be applied to the image signal; and controlling a feedback amount used in the analog-digital conversion of the image signal to be smaller in a case where the gain is a first gain than in a case where the gain is a second gain smaller than the first gain.

[0012] Further, according to the present application, there is provided a computer program product including a program for causing a computer to execute the above-described control method.

[0013] Further, according to the present application, there is provided a computer-readable storage medium storing a program for causing a computer to execute the above-described control method.

[0014] Further features of the present application will become apparent from the following description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application.

[0016] Figure 1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment.

[0017] Figure 2 is a block diagram illustrating a schematic configuration of an image sensor according to an embodiment.

[0018] Figure 3 is a block diagram illustrating a configuration of a readout circuit according to an embodiment.

[0019] Figure 4 is a block diagram illustrating a configuration of a delta sigma ADC according to an embodiment.

[0020] Figure 5A is a graph illustrating an example of an input voltage and an output voltage of a comparator according to the first embodiment.

[0021] Figure 5B is a graph illustrating another example of an input voltage and an output voltage of a comparator according to the first embodiment.

[0022] Figure 6 is a flowchart illustrating an AD conversion operation in a case where a digital gain is replaced with a feedback amount according to the first embodiment.

[0023] Figure 7 is a timing chart illustrating luminance judgment, switching of a feedback amount, and timing of AD conversion according to the first embodiment.

[0024] Figure 8 is an input / output characteristic diagram for a non-LOG video and a LOG video according to the second embodiment.

[0025] Figure 9 is a diagram illustrating output for each ISO input signal according to the second embodiment.

[0026] Figure 10 is a flowchart illustrating an AD conversion operation corresponding to luminance of an input signal according to the second embodiment.

[0027] Figure 11 is a diagram illustrating an example of a gamma curve in an SDR mode and gamma curves employing a PQ method and an HLG method in an HDR mode according to the second embodiment. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. In the embodiments, a plurality of features are described, but the invention is not limited to an invention that requires all such features, and a plurality of such features can be appropriately combined. Furthermore, in the drawings, the same reference numerals are assigned to the same or similar structures, and redundant descriptions thereof are omitted.

[0029] <First Embodiment>

[0030] Figure 1 is a block diagram illustrating a structure of an image pickup apparatus using an image sensor according to the first embodiment of the present application. Note that the image pickup apparatus to which the present application can be applied can be an electronic apparatus equipped with a camera function. Examples of the electronic apparatus include a video camera, a computer device (personal computer, tablet computer, media player, PDA, etc.), a mobile phone, a smartphone, a game machine, a robot, a drone, a drive recorder, etc. These are merely examples, and the present application can also be applied to other electronic apparatuses.

[0031] In Figure 1 , the lens unit 101 is constituted by a plurality of lenses including a zoom lens and a focus lens, and an aperture, and zoom control, focus control, aperture control, etc. are performed by the lens drive device 102. An optical image of a subject that has been incident via the lens unit 101 is formed on the image sensor 105. The lens unit 101 can be constituted integrally with the image pickup apparatus, or can be configured to be detachable from the image pickup apparatus.

[0032] The mechanical shutter 103 is controlled by the shutter drive device 104. The image sensor 105 photoelectrically converts an optical image of a subject formed by the lens unit 101, and outputs an image signal.

[0033] The signal processing circuit 106 performs digital gain processing for applying a digital gain to an image signal output from the image sensor 105, various corrections, data compression, and the like, and outputs image data. The memory unit 107 is used to temporarily store the image data. The system control unit 108 performs various calculations and controls the entire imaging device. The I / F unit 109 is an interface for recording data to and / or reading data from the recording medium 110, which is a semiconductor memory such as a flash memory for holding image data, and is configured to be removable. The display unit 111 displays various information and captured images.

[0034] Next, a brief explanation will be given of the operation of the imaging device having the above-described structure during capturing.

[0035] When the main power switch (not shown) is turned on, the system control unit 108 is turned on, and then the imaging system circuit such as the signal processing circuit 106 is also turned on. Then, when capturing is instructed by pressing a release button (not shown), the capturing operation is started. When the capturing operation is completed, the image signal output from the image sensor 105 is subjected to image processing including various corrections and digital gain processing in the signal processing circuit 106, and in response to an instruction from the system control unit 108, the obtained image data is written to the memory unit 107. The image data held in the memory unit 107 is recorded in the removable recording medium 110 such as a semiconductor memory under the control of the system control unit 108 via the I / F unit 109.

[0036] Further, RAW image data can be transmitted to a computer or the like via an external I / F unit (not shown), and image processing can be performed in the computer or the like.

[0037] Figure 2 is a block diagram illustrating a schematic structure of the image sensor 105 in the present embodiment.

[0038] The image sensor 105 includes a pixel section 200, a vertical scanning unit 202, a timing generator (TG) 203, a circuit control unit 204, a CLK generation unit 205, a vertical transfer line 206, a signal readout unit 207, and an output unit 209.

[0039] In the pixel section 200, a plurality of pixels 201 are arranged in a matrix. For ease of explanation, the pixels 201 are shown as forming a 4 x 4 pixel array in the pixel section 200, but actually, a large number of pixels, for example, several million or more, are arranged. The signal readout unit 207 includes a plurality of readout circuits 208.

[0040] The vertical scanning unit 202 selects the pixels 201 in the pixel section 200 in units of rows and supplies a plurality of drive signals to each pixel 201 in the selected row. As a result, the pixel signal of the pixel 201 in the selected row is transferred to the signal readout unit 207 via the vertical transfer line 206. The transferred pixel signal is converted into a digital signal in each readout circuit 208 and is output via the output unit 209.

[0041] The readout circuit 208 converts the input pixel signal into a digital signal value of a predetermined bit number by performing AD conversion. The AD conversion performed here uses a so-called delta sigma AD conversion method.

[0042] The output unit 209 converts the digital signal value of each pixel into a predetermined signal format and outputs it from the image sensor 105 via a transmission path.

[0043] The TG 203 sends a timing signal to the vertical scanning unit 202, and the vertical scanning unit 202 generates a drive signal for driving the pixels 201 based on the timing signal to drive the pixels 201. The circuit control unit 204 controls the CLK generation unit 205 and the signal readout unit 207 based on the timing signal from the TG 203. The CLK generation unit 205 generates a clock signal to be supplied to the signal readout unit 207 based on the timing signal supplied from the TG 203 and with the control of the circuit control unit 204.

[0044] Figure 3 is a block diagram illustrating Figure 2 the structure of each readout circuit 208 shown.

[0045] The readout circuit 208 includes an analog-digital converter (ΔΣ ADC) 300 using delta sigma modulation and a digital filter 301. The ΔΣ ADC 300 converts the pixel signal input via the vertical transfer line 206 into a digital signal using delta sigma modulation. The digital filter 301 removes the out-of-band quantization noise shifted to a higher frequency due to the delta sigma modulation. The digital filter 301 also performs interval rejection and moving average on the high-rate output of the ΔΣ ADC 300 to reduce the output rate, and converts it into a multi-bit signal value. The digital signal processed with the digital filter 301 is output to the output unit 209.

[0046] Further, in the luminance detection processing described below, the digital signal processed with the digital filter 301 is fed back to the circuit control unit 204. Based on this digital signal, the circuit control unit 204 controls the DA conversion rate for controlling the feedback amount in the ΔΣ ADC 300, or controls the digital filter 301 to not perform AD conversion. This control with the circuit control unit 204 will be described in detail later.

[0047] Figure 4 is an example of the structure of the ΔΣ ADC 300. Figure 3

[0048] As Figure 4 shown, the ΔΣ ADC 300 is composed of a subtraction circuit 400, an integration circuit 401, a comparator 402, and a digital-to-analog converter (DAC) 403.

[0049] The subtraction circuit 400 outputs a difference between a pixel signal input via the vertical transfer line 206 and an output signal of the DAC 403 to the integration circuit 401. Note that a sample-and-hold circuit can be provided between the vertical transfer line 206 and the subtraction circuit 400, and in this case, a pixel signal output from the sample-and-hold circuit is input to the subtraction circuit 400.

[0050] The integration circuit 401 includes an integrator that integrates an input signal from the subtraction circuit 400. As the integration circuit 401, for example, a gm-C integration circuit using a transconductance amplifier or an RC integration circuit using an operational amplifier, which are general integration circuits, or the like can be used.

[0051] The comparator 402 compares a reference voltage Vref and a voltage signal output from the integration circuit 401 in synchronization with a clock signal, and outputs a comparison result as a 1-bit digital signal. For example, if the voltage signal output from the integration circuit 401 is lower than the reference voltage Vref, the comparator 402 outputs 0, and if the voltage signal is equal to or higher than the reference voltage Vref, the comparator 402 outputs 1. This digital signal is supplied to the digital filter 301 and the DAC 403.

[0052] The DAC 403 converts the digital signal output from the comparator 402 into an analog signal of a predetermined amount, and outputs it to the subtraction circuit 400. For example, when the digital signal output is 1, the DAC 403 outputs an analog signal (feedback amount) at a level corresponding to a DA conversion ratio determined by the circuit control unit 204 as described later. Note that as the structure of the DAC 403, various known circuit devices can be used.

[0053] Next, the DA conversion ratio and the feedback amount in the present embodiment will be described with reference to Figure 5A and Figure 5B

[0054] Figure 5A and Figure 5B Examples of an input voltage Vcomp to the comparator 402 and an output voltage COMPout from the comparator 402 are illustrated. Note that in the present embodiment, the DA conversion ratio is determined by the circuit control unit 204 based on the pixel signal input via the vertical transfer line 206. Figure 5A and Figure 5B ​​In the present embodiment, as an example, the input / output voltage range is 1 V.

[0055] Figure 5A The illustrated voltage waveform shows an example in a case where the DAC 403 converts the digital signal output from the comparator 402 into an analog signal (feedback amount) using different DA conversion ratios for a pixel signal of the same level Vin input to the ΔΣ ADC 300 via the vertical transfer line 206. Such a waveform pattern repeats for a predetermined period of time.

[0056] The graph 501 shows the input voltage Vcomp to the comparator 402 and the reference voltage Vref, and the graph 503 shows the output voltage COMPout, which is a digital signal obtained by ΔΣ modulating the input voltage Vcomp shown in the graph 501 using the comparator 402. In contrast, the graph 502 shows the input voltage Vcomp to the comparator 402 and the reference voltage Vref in a case where a pixel signal Vin of the same level as that of the graph 501 is input and the feedback amount input to the subtraction circuit 400 is halved by setting the DA conversion ratio to 1 / 2 of the DA conversion ratio of the graph 501. The graph 504 shows the output voltage COMPout of the comparator 402 with respect to the input voltage Vcomp shown in the graph 502.

[0057] As will be understood from the graphs 501 and 502, the output signal COMPout from the comparator 402 becomes Hi when the input voltage Vcomp is higher than the reference voltage Vref. Figure 5A As will be understood from the graphs 501 and 502, the output signal COMPout from the comparator 402 becomes Hi when the input voltage Vcomp is higher than the reference voltage Vref.

[0058] Figure 5B The illustrated voltage waveform exemplifies an example of using different DA conversion ratios in a case where the level of the input pixel signal Vin is higher than the reference voltage Vref. Figure 5A The illustrated voltage waveform exemplifies an example of using different DA conversion ratios in a case where the level of the input pixel signal Vin is higher than the reference voltage Vref.

[0059] The graph 511 shows the input voltage Vcomp to the comparator 402 and the reference voltage Vref in a case where the DA conversion ratio is the same as that of the graph 501. Further, the graph 512 shows the output signal COMPout of the comparator 402 with respect to the input voltage Vcomp shown in the graph 511. In the graphs 511 and 513, compared with the graphs 501 and 503, the number of times the output voltage COMPout becomes Hi due to the AD conversion of the ΔΣ ADC 300 increases depending on the level of the pixel signal Vin.

[0060] Further, the graph 512 shows the input voltage Vcomp to the comparator 402 and the reference voltage Vref in a case where the DA conversion ratio is set to 1 / 2 of the DA conversion ratio of the graph 511 to halve the feedback amount input to the subtraction circuit 400 in a case where the pixel signal Vin is input at the same level as the level of the graph 511. The graph 514 shows the output voltage COMPout of the comparator 402 with respect to the input voltage Vcomp shown in the graph 512. In this case, by setting the DA conversion ratio to 1 / 2 of the DA conversion ratio of the graph 511, after a certain time, the output of the integration circuit 401 always becomes larger than 0, the input voltage Vcomp to the comparator 402 diverges, and the operation of the ΔΣ ADC 300 becomes unstable.

[0061] In view of the above phenomenon, in the present embodiment, the size of the pixel signal is judged, and the DA conversion ratio to be used in the DAC 403 is determined depending on the judgment result.

[0062] Note that this judgment of the size of the pixel signal can be performed by performing AD conversion in advance at a resolution required for the judgment using the ΔΣ ADC 300, or can be performed by providing a separate judgment circuit. As an example of a case where a separate judgment circuit is provided, for example, it can be configured to compare the voltage of the input pixel signal with a comparison voltage that can be changed, and output 0 or 1 depending on the comparison result. Note that, as described below, the voltage representing the comparison result changes depending on the digital gain. An example of the processing when the ΔΣ ADC 300 is used will be described later.

[0063] Table 1 shows the relationship between the ISO sensitivity, the DA conversion ratio, the digital gain, and the bit accuracy.

[0064] Table 1

[0065] ISO Bit accuracy of ADC DA conversion ratio Output ratio of Comp Dgain Bit accuracy of output 100 14 1 1 1 14 200 14 1 1 2 13 400 14 1 1 4 12 800 14 1 1 8 11 1600 14 1 1 16 10

[0066] (a)

[0067] ISO Bit accuracy of ADC DA conversion ratio Output ratio of Comp Dgain Bit accuracy of output 100 14 1 1 1 14 200 14 1 / 2 2 1 14 400 14 1 / 4 4 1 14 800 14 1 / 8 8 1 14 1600 14 1 / 16 16 1 14

[0068] (b)

[0069] (a) of Table 1 shows the case where the DA conversion ratio is set to 1 regardless of the ISO sensitivity, and the bit accuracy of the AD conversion including the processing with the digital filter is set to 14 bits at ISO 100. In this case, in order to increase the ISO sensitivity to ISO 200, ISO 400, and the like, it is necessary to multiply the digital signal after the AD conversion by the digital gain Dgain. Therefore, the bit accuracy of the final output becomes 1 / Dgain, and in the case of ISO 1600, the bit accuracy of the final output becomes 10 bits. Further, the image obtained by applying the digital gain Dgain is adversely affected by tone jump and quantization noise multiplied by the gain.

[0070] In order to eliminate these effects, and to prevent the bit accuracy of the final output from decreasing regardless of the ISO sensitivity, in the present embodiment, as shown in (b) of Table 1, the feedback amount is changed by changing the DA conversion ratio according to the ISO sensitivity. For example, by setting the DA conversion ratio to 1 / 16 at ISO 1600, as a result of the AD conversion, the output ratio of Comp representing the output ratio of the comparator 402 is higher (16 times) than the output ratio of Comp in the case where the feedback amount is set for ISO 100 (the digital gain is 1). In this way, it is possible to obtain an output equivalent to the output when the digital gain Dgain is 16 without applying the digital gain Dgain. Further, the bit accuracy of the final output becomes 14 bits, and the quantization noise is not amplified by the digital gain Dgain.

[0071] In the above example, the digital gain determined by the ISO sensitivity has been explained, but the digital gain used for various corrections such as vignetting can be replaced with the feedback amount. That is, the DA conversion ratio is set to the inverse (1 / Dgain) of the digital gain so that the feedback amount when the digital gain is 1 becomes the inverse of the digital gain. For example, in the case where a lens with large vignetting is attached, and it is necessary to apply a digital gain at the peripheral portion of the image according to the image height in the peripheral illuminance correction, by replacing the digital gain corresponding to the ISO sensitivity with the feedback amount according to the image height as shown in (b) of Table 1, it is possible to reduce the tone jump and the noise in the peripheral portion of the image even after the correction.

[0072] Further, a part of the digital gain can be replaced with the feedback amount, and the AD conversion coefficient of the feedback amount and the digital gain can be used in combination. That is, the DA conversion ratio is set to the inverse of the digital gain divided by an integer, and the DA conversion ratio is made smaller than the feedback amount when the digital gain is 1. Then, the digital gain is applied to compensate for the gain not replaced with the feedback amount.

[0073] For a high luminance input signal that becomes saturated when a digital gain is applied, it is more efficient to control the digital filter 301 to output a signal indicating saturation without performing AD conversion. For example, when a digital gain x 2 is applied at ISO 200, a signal that does not become saturated even when the gain is applied is a signal equal to or smaller than 1 / 2 of the input range of the ΔΣADC 300. In other words, if the input signal is larger than 1 / 2 of the input range of the ΔΣADC 300, setting the DA conversion ratio to 1 / 2 and reducing the feedback amount to 1 / 2 will cause the operation of the ΔΣADC 300 to become unstable.

[0074] Therefore, when replacing the digital gain of x 2 with the DA conversion ratio of 1 / 2, the threshold is set to 1 / 2 of the input range to judge the luminance of the input signal, and if the input signal is equal to or smaller than the threshold, the digital gain is replaced with the DA conversion ratio of 1 / 2 and AD conversion is performed. On the other hand, if the input signal is larger than the threshold, performing AD conversion with the DA conversion ratio of 1 / 2 will not only cause saturation, but also cause the operation of the ΔΣADC 300 to become unstable, so the ΔΣADC 300 is controlled not to perform AD conversion itself.

[0075] As described above, according to the content of the ISO sensitivity and various corrections such as the peripheral illuminance correction, the threshold used for the luminance judgment for determining whether to perform AD conversion is determined according to the size of the digital gain when the DA conversion ratio is 1, as shown in (a) of Table 1. More specifically, 1 / Dgain of the input range is determined as the threshold.

[0076] Next, the AD conversion operation in the present embodiment in the case where the digital gain is replaced with the feedback amount (DA conversion ratio) will be described with reference to the flowchart in Figure 6 The operation is repeated in each readout circuit 208 every time a new signal is input.

[0077] First, in step S601, the circuit control unit 204 sets the threshold used for the luminance judgment corresponding to the digital gain to be replaced with the feedback amount in the digital filter 301. For example, the threshold used for the luminance judgment is determined so that when the normal digital gain shown in (a) of Table 1 is x 2, 1 / 2 of the input range of the ΔΣADC 300 is determined, and when the digital gain is x 4, 1 / 4 of the input range is determined.

[0078] In step S602, the digital filter 301 judges whether or not the input pixel signal is equal to or less than the threshold value, and transmits a Judge signal indicating the result of the judgment to the circuit control unit 204. If the Judge signal indicates that the pixel signal is equal to or less than the threshold value, in step S603, the circuit control unit 204 transmits a DA conversion ratio corresponding to the digital gain to be replaced to the DAC 403, and the DAC 403 changes the feedback amount corresponding to the DA conversion ratio. Then, in step S605, the ΔΣ ADC 300 performs AD conversion.

[0079] On the other hand, in a case where the input pixel signal is greater than the threshold value, if the digital gain is replaced with the feedback amount and AD conversion is performed, the operation of the ΔΣ ADC 300 becomes unstable, so in step S604, the AD conversion is stopped. Then, in step S606, the circuit control unit 204 controls the digital filter 301 to output a saturation signal.

[0080] At the time when the processing of step S605 or S606 is completed, the processing for one pixel signal is completed.

[0081] Figure 7 is a timing chart showing the processing from step S601 to step S605 in a case where the input pixel signal is equal to or lower than the threshold value. In Figure 7 , the time period P1 corresponds to the processing in steps S601 and S602, the time period P2 corresponds to the processing in step S603, and the time period P3 corresponds to the processing in step S605.

[0082] In Figure 7 , in the time period P1, the circuit control unit 204 sets the threshold value in the digital filter 301 on the basis of the digital gain to be replaced with the feedback amount. The digital filter 301 does not change the feedback amount by setting the DA conversion ratio of the DAC 403 to 1, compares a digital signal obtained by AD converting the input voltage Vcomp to the comparator 402 at low-bit precision for a short time with the set threshold value, and outputs a Judge signal indicating the comparison result.

[0083] Next, in the time period P2, the circuit control unit 204 controls the DA conversion ratio of the DAC 403 on the basis of the Judge signal input from the digital filter 301 in the time period P1.

[0084] Here, since the digital signal of the input pixel signal is equal to or below the threshold, AD conversion is performed during time period P3 while adjusting the feedback amount of DAC 403 by means of the DA conversion factor. For example, if ISO 200 is selected, the circuit control unit 204 controls DAC 403 to set the DA conversion factor in DAC 403 to 1 / 2, thus halving the feedback amount. As a result, the feedback amount from DAC 403 decreases when the output voltage COMPout changes to Hi, causing the waveform of the input voltage Vcomp to comparator 402 during time period P3 to be different from the waveform of the input voltage Vcomp to comparator 402 during time period P1, and the period during which the output voltage COMPout changes to Hi is half the period during which the output voltage COMPout changes to Hi during time period P1.

[0085] As described above, according to the first embodiment, by replacing the digital gain with the feedback amount, an output equivalent to that amplified by the digital gain can be obtained without reducing the bit precision of the final output, and a good image with suppressed tone jumps and quantization noise can be obtained.

[0086] Furthermore, when the input signal exceeds a threshold, by controlling the output of a saturated signal without performing AD conversion, the input signal can be converted into a digital signal without compromising the stability of the ΔΣADC.

[0087] <Second Embodiment>

[0088] Next, a second embodiment of the present invention will be described.

[0089] In the second embodiment, a method for reducing noise in an input signal with low brightness by changing the feedback amount will be described. Note that the camera device in the second embodiment can be referenced in the first embodiment. Figures 1 to 4 The camera equipment described is the same, so its description will be omitted.

[0090] To achieve a wide dynamic range, LOG video, with its input / output characteristics that differ from typical features, is known as one of the shooting modes. LOG video, for example, has characteristics such as... Figure 8 The input / output characteristics are shown. Figure 8 The horizontal axis represents the number of stops when 18% gray is set to 0, and the vertical axis represents the signal level in a 10-bit recording file. Dashed lines represent the characteristics of BT.709 in non-LOG format, and solid lines represent the characteristics of LOG video. For example... Figure 8As shown, in the LOG video, the slope in the dark area is steeper, and more signal levels are allocated to the dark area, compared to in BT.709, so the dark gray scale is rich. On the other hand, since it is in a state where high gain is applied, noise in the dark area is conspicuous.

[0091] In contrast, in the second embodiment, for a low luminance signal, the signal is amplified without amplifying noise by changing the feedback amount, and the amplified amount is divided by the gain to reduce the noise.

[0092] Table 2 shows the relationship between the ISO sensitivity, the DA conversion ratio, the division gain, the digital gain, and the bit accuracy in the second embodiment. Note that in the second embodiment, it is assumed that the ISO sensitivity and the digital gain correspond to each other.

[0093] Table 2

[0094] ISO Threshold Th1 (ratio) Threshold Th2 (ratio) Dgain (ISO) 100 ×1 ×1 / 2 1 200 ×1 / 2 ×1 / 4 2 400 ×1 / 4 ×1 / 8 4 800 ×1 / 8 ×1 / 16 8 1600 ×1 / 16 ×1 / 32 16

[0095] (a)

[0096] (Vin≤Th2)

[0097]

[0098] (b)

[0099] (Th2<Vin≤Th1)

[0100]

[0101] (c)

[0102] (a) of Table 2 shows the relationship between the ISO sensitivity, the threshold, and the digital gain in the second embodiment. The threshold Th1 is a threshold for judging whether the operation of the ΔΣ ADC 300 becomes unstable, and the threshold Th2 is a threshold for judging whether to reduce noise by changing the feedback amount. As described in the first embodiment, the thresholds are determined in accordance with the input range of the ΔΣ ADC 300, so (a) of Table 2 shows the ratio with respect to the input range. Therefore, the threshold Th1 and the threshold Th2 are each a value obtained by multiplying the input range by the ratio shown in (a) of Table 2. The method of comparing the input signal with the threshold is the same as that described in the first embodiment, so the description thereof will be omitted.

[0103] (b) of Table 2 shows a case where the pixel signal Vin is a low luminance signal equal to or lower than the threshold value Th2. Here, the output on the low luminance side is amplified without adding quantization noise, compared with the first embodiment, by further dividing the DA conversion ratio by 2. Then, the amplification is divided by the digital gain to further compress the quantization noise. For example, in the case of ISO 100, the threshold value Th2 is 1 / 2 of the input range of the ΔΣ ADC 300, so by setting the DA conversion ratio to 1 / 2, the input of low luminance equal to or lower than the threshold value Th2 is doubled at the time of AD conversion without using the digital gain, and by multiplying the AD conversion result by the division gain of 1 / 2, the quantization noise can be halved. The division gain is equal to the threshold value Th2 divided by the threshold value Th1. In this way, in the case where the luminance is equal to or lower than the threshold value Th2, the noise can be reduced as described above.

[0104] (c) of Table 2 shows a case where the pixel signal Vin represents a medium luminance greater than the threshold value Th2 and equal to or less than the threshold value Th1, and the DA conversion ratio is changed according to the luminance to perform AD conversion and expand the dynamic range on the high luminance side. For example, at ISO 200, by setting the DA conversion ratio to 1 / 2, AD conversion that doubles the input signal is performed without using the digital gain. In addition, multiplication using the division gain is not performed. This processing is the same as that in the first embodiment. As a result, by changing the feedback amount for the pixel signal of medium luminance instead of the digital gain, a good image in which the tone jump and the quantization noise are suppressed can be obtained without reducing the bit precision of the final output.

[0105] In the case where the input pixel signal Vin is a high luminance signal exceeding the threshold value Th1, AD conversion is stopped as in the first embodiment. Then, by controlling the digital filter 301 to output a saturated signal, the signal can be converted into a digital signal without impairing the stability of the operation of the ΔΣ ADC 300.

[0106] Figure 9 The output for the input signal at each ISO is shown. Figure 9 The solid line portion of each ISO shown corresponds to low luminance below the threshold value Th2, and shows the output for the input signal after AD conversion using the DA conversion ratio shown in (b) of Table 2 is performed and multiplied by the division gain. The dashed line portion corresponds to medium luminance above the threshold value Th2 and below the threshold value Th1, and shows the output for the input signal after AD conversion using the DA conversion ratio shown in (c) of Table 2 is performed and multiplied by the division gain. Further, the dot-dash line portion corresponds to high luminance above the threshold value Th1, and the output for the input signal is a saturated signal.

[0107] For example, in the case of ISO 100, the threshold value Th2 is 1 / 2 of the input range of the ΔΣ ADC 300, so by setting the DA conversion ratio to 1 / 2, the input of low luminance equal to or lower than the threshold value Th2 is doubled at the time of AD conversion without using the digital gain, and by multiplying the AD conversion result by the division gain of 1 / 2, the quantization noise can be halved. The division gain is equal to the threshold value Th2 divided by the threshold value Th1. In this way, in the case where the luminance is equal to or lower than the threshold value Th2, the noise can be reduced as described above. Figure 9In the illustrated example, the threshold value Thl at which the feedback amount replaces the digital gain for ISO 200 is 0.5, which is half of the threshold value Thl for ISO 100 (i.e., 1). Further, the input range at which the output of the AD conversion becomes saturated at ISO 200 is half of the input range at which the output of the AD conversion becomes saturated at ISO 100, and thus if less than half of the output range of the AD conversion is considered to be dark, the threshold value Th2 for reducing noise is 0.25, which is half of the threshold value Thl for ISO 100 (i.e., 0.5).

[0108] Further, in Figure 9 the input corresponding to 8192 LSBs, which is the median of the 14-bit output at each ISO sensitivity, is set as the threshold value Th2.

[0109] In this way, the threshold values are set to divide the input signal into low luminance, medium luminance, and high luminance, and the AD conversion is performed by changing the DA conversion multiplication and the division gain according to the luminance.

[0110] Next, the flow of the AD conversion operation corresponding to the luminance of the input signal in the second embodiment will be described with reference to the flowchart in Figure 10 Note that this operation is repeated in each readout circuit 208 each time a new signal is input.

[0111] First, in step S1001, the circuit control unit 204 sets the threshold values Thl and Th2 for luminance judgment in the digital filter 301 according to the digital gain. The threshold values Thl and Th2 can be determined based on (a) of Table 2. Next, in step S1002, the digital filter 301 compares the input pixel signal with the threshold values Thl and Th2 and transmits a Judge signal indicating the comparison result to the circuit control unit 204. Note that in the case where the luminance judgment is performed using the ΔΣ ADC 300, if set as shown in (b) of Table 2, it is necessary to judge the minimum threshold value (i.e., 1 / 32), and thus a resolution greater than or equal to the minimum threshold value is required.

[0112] If the Judge signal indicates that the pixel signal is equal to or less than the threshold value Th2, the circuit control unit 204 transmits the DA conversion multiplication to the DAC 403 in step S1003, and the DAC 403 changes the feedback amount to the feedback amount corresponding to the DA conversion multiplication and performs the AD conversion. Then, in step S1004, the digital signal obtained by the AD conversion is multiplied by the division gain. As shown in (b) of Table 2, the DA conversion multiplication and the division gain at this time are determined according to the ISO.

[0113] Further, in a case where the Judge signal indicates that the pixel signal exceeds the threshold value Th2 and is equal to or smaller than the threshold value Th1, the circuit control unit 204 transmits the DA conversion ratio to the DAC 403 in step S1005, and the DAC 403 changes the feedback amount to the feedback amount corresponding to the DA conversion ratio and performs AD conversion. The DA conversion ratio at this time is determined in accordance with ISO as shown in (c) of Table 2. Further, multiplication using the division gain is not performed.

[0114] Further, if the Judge signal indicates that the pixel signal exceeds the threshold value Th1, the AD conversion using the ΔΣ ADC 300 becomes unstable. Therefore, in step S1006, the AD conversion is stopped, and in step S1007, the circuit control unit 204 controls the digital filter 301 to output a saturated signal.

[0115] The processing for one pixel signal ends when the processing of steps S1004, S1005, or S1006 is completed.

[0116] In the above example, the DA conversion ratio and the division gain are determined in accordance with the ISO sensitivity. However, as in the first embodiment, the DA conversion ratio and the division gain can be determined in accordance with the digital gain used for various corrections such as the peripheral illuminance correction.

[0117] Figure 11 Gamma curves of a perceptual quantization (PQ) method and a hybrid log-gamma (HLG) method in a standard dynamic range (SDR) mode and a high dynamic range (HDR) mode, which are other examples of the shooting modes other than the LOG video mode, are shown. The solid line shows an example of the gamma curve of the SDR, the dotted line shows an example of the HLG, and the dashed line shows an example of the PQ. Further, unlike Figure 8 , the horizontal axis shows luminance. As with the LOG video, by assigning more signal levels to the dark part in the HLG than in the SDR, the dark gray becomes richer, while the noise in the dark part becomes more noticeable. This tendency is even more pronounced in the PQ method. Therefore, in a case of low luminance, the feedback amount is changed and the multiplied output is divided by the digital gain again to compress the quantization noise, and in a case of high luminance, the AD conversion is performed without changing the feedback amount, whereby the noise in the dark part can be reduced. Therefore, even in the SDR still image, the second embodiment can be applied as a high-quality still image mode. In addition to the above, the control in the present embodiment can be applied in the live view mode.

[0118] As described above, according to the second embodiment, the feedback amount and the division gain are controlled in accordance with the luminance of the input signal, whereby it becomes possible to further reduce the dark noise.

[0119] In the above-described embodiments, a first-order delta sigma modulator constituting a first-order loop filter is explained, but the present application can also be applied to a circuit structure in which a second-order or higher-order delta sigma modulator is used to stabilize a feedback loop. In the case of using a second-order or higher-order delta sigma modulator, the feedback amount can be changed for each feedback loop. Further, the subtraction circuit 400 can be an addition circuit.

[0120] Further, in the delta sigma ADC of the incremental type, although the digital signal waveform is different from Figure 5A and Figure 5B the digital signal waveform shown in FIG. 6, the effect of changing the feedback amount can be obtained in the same manner.

[0121] <Other Embodiments>

[0122] The present application can be applied to a system composed of a plurality of devices, or an apparatus composed of a single device.

[0123] Other Embodiments

[0124] Embodiments of the present application can also be realized by a method in which a computer (central processing unit (CPU), micro processing unit (MPU)) of a system or device reads out and executes software (a computer program) stored in a storage medium to function in accordance with said embodiments, by providing the software (including a computer program to implement the function of the above-described device) to the system or device through a network or the like, or by reading out and executing the computer program stored in the storage medium to the computer (CPU or the like) of the system or device.

[0125] While the present application has been described with reference to exemplary embodiments, it is to be understood that the application is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be construed as encompassing all such modifications and changes as fall within the true spirit and scope of the present application.

Claims

1. An analog-to-digital converter, comprising one or more processors and / or circuit devices, said one or more processors and / or circuit devices serving as: An analog-to-digital converter unit is used to perform analog-to-digital conversion on the image signal output from the pixel using ΔΣ modulation; A determination unit is used to determine the gain to be applied to the image signal; as well as A control unit is configured to control the amount of feedback used in the analog-to-digital conversion of the image signal using the analog-to-digital conversion unit, based on the gain. Wherein, when the gain is a first gain, the control unit controls the feedback amount to be smaller compared to when the gain is a second gain that is smaller than the first gain.

2. The analog-to-digital converter according to claim 1, wherein, The control unit will multiply the feedback amount used when the gain applied to the image signal is 1 by the reciprocal of the gain determined by the judgment unit.

3. The analog-to-digital converter according to claim 1, wherein, The control unit controls the feedback quantity between a first feedback quantity used when the gain to be applied to the image signal is 1 and a second feedback quantity obtained by multiplying the first feedback quantity by the reciprocal of the gain determined by the judgment unit.

4. The analog-to-digital converter according to claim 1, wherein, The determination unit determines the gain based on at least one of ISO sensitivity and a correction method including correction for peripheral light attenuation.

5. The analog-to-digital converter according to claim 1, wherein, The control unit also controls the output of the analog-to-digital conversion unit, and If the image signal is greater than a predetermined first threshold, the control unit controls the analog-to-digital conversion unit to output a predetermined value without performing the analog-to-digital conversion.

6. The analog-to-digital converter according to claim 5, wherein, The first threshold is a value obtained by dividing the input range of the analog-to-digital conversion unit capable of performing the analog-to-digital conversion by the gain.

7. The analog-to-digital converter according to claim 1, wherein, When the image signal is equal to or less than the second threshold, the control unit controls the feedback amount to be smaller compared to when the image signal is greater than the second threshold.

8. The analog-to-digital converter according to claim 7, wherein, The second threshold is obtained by dividing the value by an integer, which is obtained by dividing the input range of the analog-to-digital conversion unit capable of performing the analog-to-digital conversion by the gain.

9. The analog-to-digital converter according to claim 7, wherein, When a predetermined shooting mode is set, the control unit controls the feedback amount based on the comparison result between the image signal and the second threshold.

10. The analog-to-digital converter according to claim 9, wherein, The predetermined shooting modes include at least one of the following: high-quality still image mode, still image mode, LOG video mode, video mode, live view mode, high dynamic range mode (HDR mode) using the hybrid log-gamma method (HLG method), and perceptual quantization method (PQ method) in HDR mode.

11. The analog-to-digital converter according to claim 1, wherein, The analog-to-digital conversion unit includes: Digital-to-analog converter unit; A subtraction unit is used to subtract the signal output from the digital-to-analog conversion unit from the image signal; An integration unit is used to integrate the signal output from the subtraction unit; and The comparison unit is used to compare the signal output from the integration unit with a predetermined threshold and output the comparison result. The digital-to-analog conversion unit outputs a signal representing the feedback quantity to the subtraction unit based on the comparison result. The control unit controls the feedback quantity in the digital-to-analog conversion unit.

12. A signal processing apparatus, comprising: Multiple analog-to-digital converters according to claim 7 or 8; as well as A signal processing unit is configured to apply a gain to the digital signal obtained by analog-to-digital conversion using the plurality of said analog-to-digital converters. When the image signal is equal to or less than a predetermined second threshold, the signal processing unit applies a gain corresponding to the gain determined by the judgment unit and the feedback amount.

13. An image sensor, comprising: Multiple analog-to-digital converters according to claim 1; as well as Multiple pixels.

14. An electronic device comprising: The image sensor according to claim 13; as well as A processing unit for processing digital signals obtained by analog-to-digital conversion using the analog-to-digital converter.

15. A control method for controlling an analog-to-digital converter, the analog-to-digital converter being used to perform analog-to-digital conversion on an image signal output from a pixel using ΔΣ modulation, the control method comprising: Determine the gain to be applied to the image signal; as well as When the gain is a first gain, the amount of feedback used in the analog-to-digital conversion of the image signal is controlled to be smaller compared to when the gain is a second gain that is smaller than the first gain.

16. A computer program product comprising a program for causing a computer to perform the control method according to claim 15.

17. A computer-readable storage medium storing a program for causing a computer to perform the control method according to claim 15.

Images