Image processing device and method

Abstract

The present invention provides an image processing device and method, the image processing device comprising: a readout module, used to obtain single-frame target data after overflow judgment of an input image signal; an image gain configuration module, connected to the readout module, used to configure the image gain of the image signal; a gain calibration module, connected to the readout module and the image gain configuration module, used to combine the target data into gain output data according to the image gain. The image processing device of the present invention effectively reduces the bandwidth of the output data and reduces the dynamic power consumption of the circuit by outputting the image signal in a single frame.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing device and method. Background Art

[0002] Complementary Metal-Oxide-Semiconductor (CMOS) refers to a technology used to manufacture large-scale integrated circuit chips or chips manufactured using this technology. CMOS Image Sensor (CIS) is irreplaceable in the field of images and is widely used in security monitoring, consumer electronics, vehicle-mounted equipment, scientific measurement and other fields. The basic development directions of CIS include: multi-pixel, low noise, high dynamic range, and high image frame rate. Low-noise and high-dynamic image quality can bring users a better visual experience.

[0003] In image processing, by combining the data of photos of different intensities under the same scene conditions, we can get high dynamic range rendering (HDR), which can provide more dynamic range and image details than ordinary images. According to different exposure times of LDR (Low-Dynamic Range), the final HDR image is synthesized using the LDR image with the best details corresponding to each exposure time. It can better reflect the visual effects in the real environment.

[0004] The voltage obtained by pixel conversion is the product of exposure time, sensitivity, conversion gain (CG), and readout circuit gain. Exposure time, sensitivity, and conversion gain can all adjust the intensity of the image signal. However, after the image is processed by a high dynamic range rendering circuit, the output data bandwidth is large and the circuit power consumption is high.

[0005] Therefore, it is necessary to provide a new image processing device and method to solve the above problems existing in the prior art. Summary of the invention

[0006] The object of the present invention is to provide an image processing device and method, which effectively reduces the bandwidth of output data and the dynamic power consumption of the circuit by outputting the image signal in a single frame.

[0007] In a first aspect, to achieve the above-mentioned object, an image processing device of the present invention includes:

[0008] The readout module is used to obtain single-frame target data after overflow judgment of the input image signal;

[0009] An image gain configuration module, connected to the readout module, and used to configure the image gain of the image signal;

[0010] The gain calibration module is connected to the readout module and the image gain configuration module, and is used to combine the target data into gain output data according to the image gain.

[0011] The beneficial effect of the image processing device described in the present invention is that: the target data is obtained after comparison and judgment of the image signal through the readout module, so as to obtain the target data of a single frame after overflow judgment of the image signal, and the image gain configuration module configures the image gain for different frame signals of the image signal, and then the target data is processed by the gain calibration module, and the image gain corresponding to the target data is combined with the target data into gain output data to output single frame image data, thereby changing the final output result from multi-frame output to single frame output, which not only effectively reduces the bandwidth of the output data, but also can effectively reduce the dynamic power consumption of the circuit.

[0012] Optionally, the readout module includes:

[0013] A source follower unit, used for converting the image signal into a pixel voltage signal;

[0014] A ramp generator unit, for generating a ramp signal;

[0015] a comparator unit, connected to the source follower unit and the ramp generator unit respectively, to compare the pixel voltage signal with the ramp signal and output a comparison signal;

[0016] a counter unit, connected to the comparator unit, configured to count according to the comparison signal and output a counting result;

[0017] an overflow judgment unit, connected to the counter unit, configured to perform overflow judgment according to the counting result or the comparison signal and obtain an overflow result, and output the target data according to the overflow result;

[0018] a data cache unit, connected to the overflow judgment unit, and configured to cache the target data;

[0019] A pause switch logic unit, the input end of the pause switch logic unit is connected to the overflow judgment unit, and the output end of the pause switch logic unit is respectively connected to the source follower unit, the comparator unit and the counter unit, so as to adjust the working state of the source follower unit, the comparator unit and the counter unit according to the overflow result. Its beneficial effect is that: the above-mentioned readout module compares the pixel voltage signal and the ramp signal through the comparator unit and outputs the comparison signal, then counts the comparison signal through the counter unit to output the counting result, and overflows the counting result or the comparison signal through the overflow judgment unit, and outputs the target data according to the overflow result, and caches the target data through the data cache unit for subsequent call; at the same time, the source follower unit, the comparator unit and the counter unit of the readout module are controlled by the pause switch logic unit, and when it is determined that there is no signal overflow at present according to the overflow result, before starting to read the next frame of image signal, the source follower unit, the comparator unit and the counter unit are soft-turned off or controlled to enter the low power consumption mode, thereby effectively reducing the power consumption of the entire readout module.

[0020] Optionally, the gain calibration module includes a frame sequence gain unit, a counting output unit and a synthesis unit, the input end of the frame sequence gain unit is respectively connected to the image gain configuration module and the readout circuit to obtain the image gain corresponding to the target data according to the frame sequence of the target data, the counting output unit is connected to the readout module to output the target data output by the readout module, and the synthesis unit is respectively connected to the output ends of the frame sequence gain unit and the counting output unit to combine the target data and the corresponding image gain into gain output data.

[0021] Optionally, the image gain configuration module adjusts the image gain of the image signal by adjusting at least one of the exposure length, conversion gain, analog gain, and pixel switching of the readout module. The beneficial effect is that the image gain of the image signal can be adjusted in a variety of ways.

[0022] In a second aspect, the present invention further provides an image processing method, the processing method comprising:

[0023] Inputting an image signal to be processed, and configuring image gain for the image signal through an image gain configuration module;

[0024] Perform overflow judgment on the image signal to be processed by a readout module to obtain target data of a single frame;

[0025] According to the image gain of the image signal, the target data is combined into gain output data through a gain calibration module.

[0026] The beneficial effect of the image processing method described in the present invention is that: by performing overflow judgment on the image signal to be processed to obtain single-frame target data, and combining the obtained single-frame target data into gain output data according to the configured image gain, the final output result is changed from multi-frame output to single-frame output, which not only effectively reduces the bandwidth of the output data, but also can effectively reduce the dynamic power consumption of the circuit.

[0027] Optionally, performing overflow judgment on the image signal to be processed by a readout module to obtain single-frame target data includes:

[0028] Converting the image signal into a pixel voltage signal according to a frame sequence by a source follower unit;

[0029] Comparing the pixel voltage signal with a preset ramp signal through a comparator unit, and outputting a comparison signal according to the comparison result;

[0030] The counter unit performs counting processing according to the comparison signal to obtain a counting result;

[0031] The overflow judgment unit performs overflow judgment on the counting result to obtain an overflow result, and acquires the target data according to the overflow result.

[0032] Optionally, outputting a comparison signal according to the comparison result includes:

[0033] Outputting a high level or a low level comparison signal respectively according to the magnitude of the pixel voltage signal and the ramp signal;

[0034] The performing counting processing according to the comparison signal to obtain a counting result includes:

[0035] Counting is performed or stopped according to the level of the comparison signal, and the counting result is output after counting.

[0036] Optionally, the counting process includes a reset phase and a signal phase, and when the counting process is in the reset phase, the counting process is a negative count; when the counting process is in the signal phase, the counting process is a positive count. The beneficial effect is that it is easy to count different phases.

[0037] Optionally, performing overflow judgment on the counting result to obtain an overflow result includes:

[0038] Determine the magnitude of the counting result and a preset overflow determination threshold;

[0039] When the counting result is greater than the preset overflow judgment threshold, it is determined that the current frame signal overflows; when the counting result is less than or equal to the preset overflow judgment threshold, it is determined that the current frame signal does not overflow;

[0040] The acquiring the target data according to the overflow result includes:

[0041] When it is determined according to the overflow result that the current pixel voltage signal has not overflowed, outputting the counting result of the current frame as the target data, outputting the current frame sequence, and stopping reading out the next pixel voltage signal;

[0042] When it is determined according to the overflow result that the current pixel voltage signal overflows, continue to read the next pixel voltage signal until the pixel voltage signal does not overflow, and output the counting result of the current frame as the target data, and output the current frame sequence;

[0043] The target data is written into the data cache. Its beneficial effect is that: by presetting the overflow judgment threshold and the counting result judgment to determine whether the image signal of the current frame overflows, the target data is subsequently obtained according to the overflow result. When the current pixel voltage signal overflows, the counting result of the current frame is output as the target data, and the frame sequence is output; when the current pixel voltage signal does not overflow, the next pixel voltage signal is read until a pixel voltage signal without overflow appears, and the counting result of the frame signal is used as the target data, and the current frame sequence is output.

[0044] Optionally, the acquiring the target data according to the overflow result further includes:

[0045] When all the pixel voltage signals have not overflowed or the current pixel voltage signal corresponds to the last frame of the image signal, the counting result of the last frame is used as the target data, and the current frame sequence is output. The beneficial effect is that when there is no pixel voltage signal overflow or the current pixel voltage signal is the last frame of the image signal, the counting result of the last frame is used as the target data, and the current frame sequence is output, ensuring that there is always output data.

[0046] Optionally, after determining that the current frame signal does not overflow, the processing method further includes:

[0047] The source follower unit, the comparator unit and the counter unit are controlled to be soft-off or enter a low power consumption mode. The beneficial effect is: reducing the overall power consumption.

[0048] Optionally, configuring the image gain of the image signal by using an image gain configuration module includes:

[0049] At least one of exposure length, conversion gain, analog gain, and pixel switching of the readout module is adjusted to adjust the image gain of the image signal.

[0050] Optionally, combining the target data into gain output data according to the image gain of the image signal includes:

[0051] Acquire a target frame sequence of the target data;

[0052] Acquire the image gain of the target data according to the target frame sequence;

[0053] The image gain of the target data is multiplied by the target data to obtain the gain output data.

[0054] Optionally, the processing method further includes outputting counting of the target data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a schematic diagram of the overall structure of an image processing device according to an embodiment of the present invention;

[0056] Figure 2 A structural block diagram of a readout module of an image processing device according to an embodiment of the present invention;

[0057] Figure 3 is a flowchart of the image processing method according to an embodiment of the present invention;

[0058] Figure 4 A timing diagram of two HDR synthesis processes of reset one, signal one, reset two, and signal two in the image processing method according to an embodiment of the present invention;

[0059] Figure 5 This is a timing diagram of the synthesis process of two HDRs of reset two, reset one, signal one, and signal two in the image processing method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0061] In view of the problems existing in the prior art, an embodiment of the present invention provides an image processing device, referring to Figure 1 ,include:

[0062] The readout module 1 is used to obtain single-frame target data after overflow judgment of the input image signal;

[0063] An image gain configuration module 2, connected to the readout module 1, and used to configure the image gain of the image signal;

[0064] The gain calibration module 3 is connected to the readout module 1 and the image gain configuration module 2, and is used to combine the target data into gain output data according to the image gain.

[0065] In the above-mentioned image processing device, after the external image signal is input to the readout module 1, the readout module 1 performs overflow judgment on the input image signal, so as to obtain single-frame target data that meets the requirements, and the image gain configuration module 2 performs gain configuration on each frame of the input image signal, and then the output single-frame target data is combined with the corresponding image gain through the gain calibration module 3 to obtain the final gain output data. When realizing image synthesis, the multi-frame image output is converted into a single-frame image output, which not only reduces the data output bandwidth, but also effectively reduces the circuit power consumption of the entire device.

[0066] In some embodiments, Figure 2 is a structural block diagram of the readout module 1 of the image processing device according to an embodiment of the present invention, referring to Figure 2 , the readout module 1 comprises:

[0067] A source follower unit 101, used for converting the image signal into a pixel voltage signal;

[0068] A ramp generator unit 102, for generating a ramp signal;

[0069] A comparator unit 103, connected to the source follower unit and the ramp generator unit, respectively, to compare the pixel voltage signal with the ramp signal and output a comparison signal;

[0070] A counter unit 104, connected to the comparator unit 103, configured to count according to the comparison signal and output a counting result;

[0071] An overflow judgment unit 105, connected to the counter unit 104, configured to perform overflow judgment according to the counting result or the comparison signal and obtain an overflow result, and output the target data according to the overflow result;

[0072] The data cache unit 106 is connected to the overflow determination unit 105 and is used for caching the target data.

[0073] Specifically, the source follower unit 101 is connected to an external input terminal, and is used to convert an image signal input through the input terminal into a pixel voltage signal output. At the same time, the ramp generator unit 102 generates a ramp signal, and the two input terminals of the comparator unit 103 are respectively connected to the source follower unit 101 and the ramp generator unit 102, so that the pixel voltage signal and the ramp signal are compared through the comparator unit 103, and a comparison signal is output according to the comparison result, and the counter unit 104 is connected to the comparator unit 103, so as to count according to the comparison signal and output the counting result, and the overflow judgment unit 105 performs overflow judgment according to the counting result or the comparison signal, and outputs the final target data according to the overflow result, wherein the target data is the pixel voltage signal of the current frame, and while outputting the target data, the current frame sequence corresponding to the current target data is output; and after completing the overflow judgment, the final output target data is cached by the data cache unit 106.

[0074] Among them, the output of the comparator unit 103 is recorded as CM output. When the pixel voltage signal of the current frame is less than the ramp signal, the CM of the comparator unit 103 outputs a comparison signal of a low level signal; and when the pixel voltage signal of the current frame is greater than the ramp signal, the CM of the comparator unit 103 outputs a comparison signal of a high level.

[0075] In some embodiments, the data cache unit 106 caches the output target data after write enabling.

[0076] Optional, continue to refer to Figure 2The readout module 1 also includes a pause switch logic unit 107, the input end of the pause switch logic unit 107 is connected to the overflow judgment unit 105, and the output end of the pause switch logic unit 107 is respectively connected to the source follower unit 101, the comparator unit 103 and the counter unit 104 to adjust the working states of the source follower unit 101, the comparator unit 103 and the counter unit 104 according to the overflow result.

[0077] Specifically, when the overflow judgment unit 105 determines that the image signal of the current frame has not overflowed, that is, after the pixel voltage signal of the current frame can be output as the target data, the subsequent reading process of the pixel voltage signal is temporarily suspended, and the source follower unit 101, the comparator unit 103 and the counter unit 104 are controlled to be softly shut down or enter a low power consumption mode by pausing the switch logic unit 107, thereby reducing the power consumption of the entire readout module 1, thereby effectively reducing the overall circuit power consumption.

[0078] In some other embodiments, reference Figure 1 The gain calibration module 3 includes a frame sequence gain unit 301, a counting output unit 302 and a synthesis unit 303. The input end of the frame sequence gain unit 301 is respectively connected to the image gain configuration module 2 and the readout circuit 1 to obtain the image gain corresponding to the target data according to the frame sequence of the target data. The counting output unit 302 is connected to the readout module 1 to output the target data output by the readout module 1. The synthesis unit 303 is respectively connected to the output ends of the frame sequence gain unit 301 and the counting output unit 302 to combine the target data and the corresponding image gain into gain output data.

[0079] Specifically, the input end of the frame sequence gain unit 301 is connected to the output end of the image gain configuration module 2 and the output end of the readout module 1 respectively, so that the image gain corresponding to the target data is obtained from the image gain configured by the image gain configuration module 2 according to the frame sequence of the output target data through the frame sequence gain unit 301, and the counting output unit 302 is connected to the output end of the readout module 1 to output the target data of the readout module 1, and then the target data and the image gain corresponding to the frame sequence of the target data are multiplied by the synthesis unit 303 to obtain the final gain output data.

[0080] In this embodiment, the input end of the frame sequence gain unit 301 is connected to the output end of the image gain configuration module 2 and the output end of the readout module 1 respectively, so that the output result is converted from a traditional multi-frame image output to a single-frame image output with a frame identification signal, which effectively reduces the data output bandwidth and also reduces the circuit dynamic consumption to a certain extent.

[0081] The output end of the read module 1 is connected to the output end of the data cache unit 106 .

[0082] In some embodiments, the image gain configuration module 2 adjusts the image gain of the image signal by adjusting at least one of the exposure length, conversion gain, analog gain, and pixel switching of the readout module 1 .

[0083] Specifically, when the image gain configuration module 2 configures the gain of the image signal in the readout module 1, the image gain of the image signal is adjusted by adjusting one or more of the exposure length, conversion gain, analog gain, and pixel switching, so as to configure different image gains for image signals with different frame sequences.

[0084] It should be noted that the gain configuration of the image gain configuration module 2 includes two methods: on-chip processing and off-chip processing. During on-chip processing, the configured image gain is multiplied with the target data output by the readout module 1 and then output to the frame sequence gain unit 301; the configured image gain and the target data output by the readout module 1 can also be output to the frame sequence gain unit 301 respectively, which will not be repeated here.

[0085] The present invention also provides an image processing method, such as Figure 3 As shown, the processing method comprises the following steps:

[0086] S301 , input an image signal to be processed, and configure image gain for the image signal through an image gain configuration module.

[0087] In some embodiments, configuring the image gain of the image signal by an image gain configuration module includes:

[0088] At least one of exposure length, conversion gain, analog gain, and pixel switching of the readout module is adjusted to adjust the image gain of the image signal.

[0089] The image gain of the image signal is adjusted by adjusting one or more parameters of the exposure length, conversion gain, analog gain, and pixel switching of the readout module to meet different image gain configuration requirements.

[0090] S302 , performing overflow judgment on the image signal to be processed by a readout module to obtain target data of a single frame.

[0091] In some embodiments, the method of performing overflow judgment on the image signal to be processed by the readout module to obtain single-frame target data comprises the following steps:

[0092] Converting the image signal into a pixel voltage signal according to a frame sequence by a source follower unit;

[0093] Comparing the pixel voltage signal with a preset ramp signal through a comparator unit, and outputting a comparison signal according to the comparison result;

[0094] The counter unit performs counting processing according to the comparison signal to obtain a counting result;

[0095] The overflow judgment unit performs overflow judgment on the counting result to obtain an overflow result, and acquires the target data according to the overflow result.

[0096] After the image signal is input into the readout module, the input image signal is first converted into a pixel voltage signal output through the source follower unit, and at the same time, the ramp generator unit inside the readout module generates a corresponding ramp signal. Under the action of the comparator unit, the pixel voltage signal of the current frame is compared with the ramp signal, and a comparison signal is output according to the comparison result. After the comparator unit outputs the comparison signal, the counter unit counts the comparison signal output by the comparator unit to obtain a counting result, and then the overflow judgment unit performs an overflow judgment on the output counting result to determine whether the pixel voltage signal of the current frame overflows, and then the final target data is obtained according to the overflow judgment result of the pixel voltage signal of the current frame.

[0097] In some embodiments, outputting a comparison signal according to the comparison result includes:

[0098] Outputting a high level or a low level comparison signal respectively according to the magnitude of the pixel voltage signal and the ramp signal;

[0099] The performing counting processing according to the comparison signal to obtain a counting result includes:

[0100] Count or stop counting according to the level of the comparison signal, and output the counting result after counting

[0101] In some specific embodiments, when the pixel voltage signal is greater than the ramp signal, the comparison signal is output as a high level signal;

[0102] When the pixel voltage signal is less than the ramp signal, the comparison signal output is a low level signal;

[0103] The performing counting processing according to the comparison signal to obtain a counting result includes:

[0104] When the comparison signal is a low level signal, counting is performed according to the clock cycle to obtain the counting result;

[0105] When the comparison signal is at a high level, counting is stopped.

[0106] Specifically, when the current pixel voltage signal is less than the ramp signal, the comparator unit outputs a low-level comparison signal, and when the current pixel voltage signal is greater than the ramp signal, the comparator unit outputs a high-level signal, and the counter unit counts according to the output comparison signal. When the comparator outputs a low-level comparison signal, the counter unit counts according to the clock cycle; and when the comparator unit outputs a high-level comparison signal, the counter unit stops counting.

[0107] However, the present solution is not limited to outputting comparison signals and counting according to the above method. The above process can also be implemented in a method opposite to the above method, which will not be described in detail here.

[0108] It should be noted that, in this solution, the counting process of the counter unit includes a reset phase and a signal phase. When the counting process is in the reset phase, the counting process is a negative count; when the counting process is in the signal phase, the counting process is a positive count.

[0109] In some embodiments, the performing overflow judgment on the counting result to obtain an overflow result comprises the following steps:

[0110] Determine the magnitude of the counting result and a preset overflow determination threshold;

[0111] When the counting result is greater than the preset overflow judgment threshold, it is determined that the current frame signal overflows; when the counting result is less than or equal to the preset overflow judgment threshold, it is determined that the current frame signal does not overflow;

[0112] The acquiring the target data according to the overflow result includes:

[0113] When it is determined according to the overflow result that the current pixel voltage signal has not overflowed, outputting the counting result of the current frame as the target data, outputting the current frame sequence, and stopping reading out the next pixel voltage signal;

[0114] When it is determined according to the overflow result that the current pixel voltage signal overflows, continue to read the next pixel voltage signal until the pixel voltage signal does not overflow, and output the counting result of the current frame as the target data, and output the current frame sequence;

[0115] The target data is written into the data cache.

[0116] Specifically, after the counter unit counts and obtains the counting result, the overflow judgment unit performs overflow judgment on the current counting result. When the counting result is greater than the preset overflow judgment threshold, it is determined that the pixel voltage signal of the current frame overflows. When the counting result is less than or equal to the preset overflow judgment threshold, it is determined that the pixel voltage signal of the current frame does not overflow. After obtaining the overflow result of the pixel voltage signal of the current frame, the final target data is obtained according to the overflow result. Specifically, when it is determined that the current pixel voltage signal overflows, the pixel voltage signal of the next frame continues to be read until the pixel voltage signal does not overflow, and the counting result of the current frame without overflow is used as the target data, and the current frame sequence is output at the same time. On the other hand, after the pixel voltage signal of the current frame does not overflow, the counting result of the current frame is used as the target data, the current frame sequence is output at the same time, and the reading of the next pixel voltage signal is stopped.

[0117] In some further embodiments, obtaining the target data according to the overflow result further includes:

[0118] When all the pixel voltage signals do not overflow or the current pixel voltage signal corresponds to the last frame of the image signal, the counting result of the last frame is used as the target data, and the current frame sequence is output.

[0119] When the overflow judgment unit obtains the final target data based on the overflow result, if the pixel voltage signals of all frames have been read and it is determined that no pixel voltage signal meets the non-overflow condition or the current pixel voltage signal corresponds to the last frame of the image signal, the counting result of the last frame is used as the target data, and the frame sequence of the last frame pixel voltage signal is output as the final target data.

[0120] In some other embodiments, when the overflow judgment unit makes an overflow judgment on the current pixel voltage signal, a pulse signal may be generated in the signal stage of the ramp generator unit. When the pulse signal is at a high level and the comparison signal output by the comparator unit is at a low level, the current pixel voltage signal is judged to be overflowed; otherwise, it is not overflowed. The above judgment method and the above-mentioned method of judging overflow by presetting the overflow judgment threshold can both be used for overflow judgment of the pixel voltage signal, and the specific selection is made according to the needs of the circuit, and this scheme does not specifically limit this.

[0121] In some possible embodiments, after determining that the current frame signal has not overflowed, the source follower unit, the comparator unit and the counter unit are controlled to soft-shut down or enter a low power consumption mode, thereby effectively reducing the circuit dynamic power consumption of the entire readout module.

[0122] S303 . Combining the target data into gain output data through a gain calibration module according to the image gain of the image signal.

[0123] Through the above method, after the image signal to be processed is input, the image gain is configured for the input image signal through the image gain configuration module, and then the image signal to be processed is input into the readout module, and the readout module performs overflow judgment on the image signal to be processed, so that the readout module outputs the target data of a single frame, and then according to the configured image gain, the output target data of the single frame is combined with the corresponding image gain to obtain gain output data, so as to reduce the data bandwidth between the CIS and the image processor in the process of completing image synthesis, and at the same time reduce the dynamic power consumption of the overall circuit.

[0124] In some embodiments, combining the target data into gain output data according to the image gain of the image signal comprises:

[0125] Acquire a target frame sequence of the target data;

[0126] Acquire the image gain of the target data according to the target frame sequence;

[0127] The image gain of the target data is multiplied by the target data to obtain the gain output data.

[0128] After acquiring the target data, the target frame sequence is acquired according to the frame sequence corresponding to the target data, and the image gain corresponding to the target data is acquired according to the target frame sequence, and the target data and the corresponding image gain are multiplied to obtain the gain output data, thereby converting the output result from the traditional multi-frame image output to a single-frame image output with a frame recognition signal, effectively reducing the data output bandwidth, and also reducing the circuit dynamic consumption to a certain extent.

[0129] Specifically, the frame images with the same pixel gain are the same frame. In the HDR process, since the reading of multiple frames is performed alternately, the same row can be read out continuously. Therefore, the readout order of the image satisfies: the signals of the pixels of the same row with different image gains K should be read out continuously; the readout order of the images of the same row is arranged in the order of image gain K from high to low.

[0130] The count result read out for the nth time at the rth row and cth column is marked as D r,c,n .

[0131] The preset overflow judgment threshold is recorded as D sat ; The image gain of the nth input is marked as A n , A n =K1 / K n ; Read out D in order r,c,n, if D r,c,n Less than D sat , then the pixel signal output frame sequence n and D r,c,n , and the subsequent data is not output; conversely, the current data D is not output r,c,n , read D r,c,n+1 Repeat this process until there is a pixel signal output or the pixel signal of the coordinate is read out for the last time, and multiply the corresponding nth frame output data by A n To obtain the gain output data output.

[0132] To further illustrate this scheme, Figure 4 This is a timing diagram of the two HDR synthesis processes of reset one, signal one, reset two, and signal two in an embodiment of the present invention, referring to Figure 4 , Figure 4 The upper middle part includes a ramp signal and two sets of pixel voltage signals V1 and V2. V1 and V2 are the pixel voltage signals of signal one minus the reset stage. Among them, V1 has no data overflow; V2 has data overflow. V1 / P and V2 / P are the pixel signals of signal two minus the reset stage. The relationship between V1 and V1 / P is that a pixel gain of P times is introduced; the same is true for the relationship between V2 and V2 / P. The dotted line is a ramp signal, and the ramp signal introduces an analog gain of G times at this time, that is, the slope of signal one stage is 1 / G of signal two stage; the gain of the second frame A2 is calibrated as GP.

[0133] Reference Figure 4 When the pixel voltage signal output is V1, the comparator unit outputs a high level without overflow in the first frame, and finally outputs 1 bit F1D1, where F1 is the data of the first frame and D1 is the data used without overflow.

[0134] When the pixel output signal is V2, the comparator output CM outputs 1 overflow in the first frame and does not overflow in the second frame, and finally outputs 1 bit F2D2, where F2 is the data of the first frame and D2 is the data used when there is no overflow.

[0135] After the data output is completed, the output data is written uniformly after the data cache unit is written and enabled, so as to realize data synthesis.

[0136] Figure 5 This is a timing diagram of the synthesis process of two HDRs of reset 2, reset 1, signal 1, and signal 2 in an embodiment of the present invention, refer to Figure 5 This synthesis is used in more special signals, such as the reading of the DCG process. Since the DCG is switched, the reset signal will be read out continuously. Reset 2 and signal 2 form a low gain (low conversion gain: LCG) signal; reset 1 and signal 1 form a low gain signal. This structure is more special and requires two sets of counter logic.

[0137] Reference Figure 5 , the upper part of the figure includes a ramp signal and two sets of pixel voltage signals, V1 and V2 are the pixel signals of signal one minus the reset one stage. Among them, V1 has no data overflow; V2 has data overflow. V1 / P, V2 / P are the pixel signals of signal two minus the reset two stage. The relationship between V1 and V1 / P is that a pixel gain of P times is introduced; similarly, the relationship between V2 and V2 / P. The dotted line is the ramp signal, and the ramp signal introduces an analog gain of G times at this time, that is, the slope of the first stage of the signal is 1 / G of the second stage of the signal.

[0138] When the pixel output signal is V1, the comparator unit outputs CM output 1 in the first frame without overflow, and finally outputs 1 bit F1D1. F1 is the data of the first frame, and D1 is the data used without overflow. At this time, the output data is output through counter A.

[0139] When the pixel output signal is V2, the comparator output CM outputs 1 overflow in the first frame, does not overflow in the second frame, and finally outputs 1 bit F2D2, where F2 is the data of the first frame and D2 is the data used when there is no overflow. At this time, the output data is output through counter B.

[0140] After the data output is completed, the output data is written uniformly after the data cache unit is written and enabled, so as to realize data synthesis.

[0141] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the selection module can be a separately established processing element, or it can be integrated in a certain chip of the above system for implementation. In addition, it can also be stored in the memory of the above system in the form of program code, and called and executed by a certain processing element of the above system. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0142] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0143] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0144] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0146] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

[0147] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. An image processing device, characterized in that: include: The readout module is used to obtain single-frame target data after overflow judgment of the input image signal; An image gain configuration module, connected to the readout module, and used to configure the image gain of the image signal; a gain calibration module, connected to the readout module and the image gain configuration module, and configured to combine the target data into gain output data according to the image gain; Wherein, the readout module comprises: A source follower unit, used for converting the image signal into a pixel voltage signal; A ramp generator unit, for generating a ramp signal; a comparator unit, connected to the source follower unit and the ramp generator unit respectively, to compare the pixel voltage signal with the ramp signal and output a comparison signal; a counter unit, connected to the comparator unit, configured to count according to the comparison signal and output a counting result; an overflow judgment unit, connected to the counter unit, configured to perform overflow judgment according to the counting result or the comparison signal and obtain an overflow result, and output the target data according to the overflow result; a data cache unit, connected to the overflow judgment unit, and configured to cache the target data; A pause switch logic unit, wherein the input end of the pause switch logic unit is connected to the overflow judgment unit, and the output end of the pause switch logic unit is respectively connected to the source follower unit, the comparator unit and the counter unit to adjust the working states of the source follower unit, the comparator unit and the counter unit according to the overflow result.

2. The image processing device according to claim 1, characterized in that The gain calibration module includes a frame sequence gain unit, a counting output unit and a synthesis unit. The input end of the frame sequence gain unit is respectively connected to the image gain configuration module and the readout circuit to combine the target data output by the readout module into the gain output data according to the frame sequence image gain. The counting output unit is connected to the readout module to output the target data output by the readout module. The synthesis unit is respectively connected to the output ends of the frame sequence gain unit and the counting output unit to combine the target data and the corresponding image gain into the gain output data.

3. The image processing device according to claim 2, characterized in that: The image gain configuration module adjusts the image gain of the image signal by adjusting at least one of the exposure length, conversion gain, analog gain, and pixel switching of the readout module.

4. An image processing method, applied to the image processing device according to any one of claims 1 to 3, characterized in that: The processing method comprises: Inputting an image signal to be processed, and configuring image gain for the image signal through an image gain configuration module; Performing overflow judgment on the image signal to be processed by a readout module to obtain target data of a single frame; According to the image gain of the image signal, the target data is combined into gain output data through a gain calibration module.

5. The image processing method according to claim 4, characterized in that: The method of performing overflow judgment on the image signal to be processed by the readout module to obtain single-frame target data includes: Converting the image signal into a pixel voltage signal according to a frame sequence by a source follower unit; Comparing the pixel voltage signal with a preset ramp signal through a comparator unit, and outputting a comparison signal according to the comparison result; The counter unit performs counting processing according to the comparison signal to obtain a counting result; The overflow judgment unit performs overflow judgment on the counting result to obtain an overflow result, and acquires the target data according to the overflow result.

6. The image processing method according to claim 5, characterized in that: The step of outputting a comparison signal according to the comparison result comprises: Outputting a high level or a low level comparison signal respectively according to the magnitude of the pixel voltage signal and the ramp signal; The performing counting processing according to the comparison signal to obtain a counting result includes: Counting is performed or stopped according to the level of the comparison signal, and the counting result is output after counting.

7. The image processing method according to claim 6, characterized in that: The counting process includes a reset phase and a signal phase. When the counting process is in the reset phase, the counting process is a negative count; when the counting process is in the signal phase, the counting process is a positive count.

8. The image processing method according to claim 5, characterized in that: The performing overflow judgment on the counting result to obtain an overflow result includes: Determine the magnitude of the counting result and a preset overflow determination threshold; When the counting result is greater than the preset overflow judgment threshold, it is determined that the current frame signal overflows; when the counting result is less than or equal to the preset overflow judgment threshold, it is determined that the current frame signal does not overflow; The acquiring the target data according to the overflow result includes: When it is determined according to the overflow result that the current pixel voltage signal has not overflowed, outputting the counting result of the current frame as the target data, outputting the current frame sequence, and stopping reading out the next pixel voltage signal; When it is determined according to the overflow result that the current pixel voltage signal overflows, continue to read the next pixel voltage signal until the pixel voltage signal does not overflow, and output the counting result of the current frame as the target data, and output the current frame sequence; The target data is written into the data cache.

9. The image processing method according to claim 8, characterized in that: The acquiring the target data according to the overflow result further includes: When all the pixel voltage signals do not overflow or the current pixel voltage signal corresponds to the last frame of the image signal, the counting result of the last frame is used as the target data, and the current frame sequence is output.

10. The image processing method according to claim 8, characterized in that: After determining that the current frame signal does not overflow, the processing method further includes: The source follower unit, the comparator unit and the counter unit are controlled to be soft-shutdown or enter a low power consumption mode.

11. The image processing method according to claim 8, characterized in that: The configuring the image gain of the image signal by the image gain configuration module includes: At least one of exposure length, conversion gain, analog gain, and pixel switching of the readout module is adjusted to adjust the image gain of the image signal.

12. The image processing method according to claim 4, characterized in that: The step of combining the target data into gain output data according to the image gain of the image signal comprises: Acquire a target frame sequence of the target data; Acquire the image gain of the target data according to the target frame sequence; The image gain of the target data is multiplied by the target data to obtain the gain output data.

13. The image processing method according to claim 8, characterized in that: The processing method further includes performing output counting on the target data.

Images