A nonlinear image acquisition quantification method

Abstract

The application discloses a nonlinear image acquisition quantization method. The steps of the quantization method are as follows: (1) for the original light intensity information collected by an imaging device, the size of a bit is i, the corresponding quantization space is 0-2 i -1, the quantization space is divided into i intervals by using binary: 0-1, 1-3, 3-7,..., 2 i‑1 -1-2 i -1, sequentially recorded as S0, S1, S2, S3,..., S i‑1 ; (2) nonlinear quantization: changing the quantization step height of at least one interval, obtaining the corresponding nonlinear quantization slope; (3) substituting the original light intensity information into the nonlinear quantization slope to map to the corresponding quantized value, the value is the quantized pixel gray value, so that the gray image after nonlinear quantization coding can be obtained. The method can effectively reduce the quantization bit number and the acquisition time under the premise of ensuring the image quality, and a smaller bit number can be used to obtain a given signal-to-noise ratio of a signal region of interest.

Description

Technical Field

[0001] This invention relates to a nonlinear image acquisition quantization method, belonging to the field of image processing. Background Technology

[0002] Quantization refers to the process of approximating a signal's continuous values ​​(or a large number of discrete values) into a finite number (or a small number) of discrete values. A continuous signal in the analog domain is sampled to become a discrete signal, and then quantized to become a digital signal; this process is typically implemented using an analog-to-digital converter (ADC).

[0003] The primary purpose of quantization is to reduce data volume, playing a crucial role in lossy data compression. However, quantization errors can lead to a loss of precision and a decrease in image quality. This trade-off between image quality and data volume can be controlled by selecting different quantization bit depths. Based on the different incremental correspondences between the quantized digital signal and the original signal, quantization methods can be classified as linear or nonlinear.

[0004] When encoding an image, the brightness difference between all adjacent encoded values ​​must be less than 1% to ensure that the human eye cannot perceive sudden changes in brightness between pixels. If linear encoding is used, changes in adjacent encoded values ​​in low-intensity areas appear more pronounced, leading to non-uniformity and unevenness in the grayscale image. Therefore, various non-linear encoding methods have emerged that conform to human visual perception and effectively save encoding space; Gamma correction is one of them. Assuming the encoded value is C and the brightness is I, the relationship between them in Gamma correction is C = I. 1 / γ The value of the γ constant varies in different systems. However, Gamma correction has two main problems: one is the color banding phenomenon caused by using the mean for edge blurring during image blurring; the other is the low dynamic range of non-linear coding, which results in poor performance of the corrected image in bright areas, because a large brightness range in the high-intensity region corresponds to only a small coding range. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the present invention aims to provide a nonlinear image acquisition quantization method.

[0006] The technical solution adopted in this invention is as follows:

[0007] A nonlinear image acquisition quantization method, the steps of which are as follows:

[0008] Step 1: For the raw light intensity information of bit size i collected by the imaging device, its corresponding quantization space is 0 to 2. i -1, using binary to divide the quantization space into i intervals: 0~1, 1~3, 3~7, ..., 2 i-1 -1~2i -1, and are denoted sequentially as S0, S1, S2, S3, ..., S i-1 ;

[0009] Step 2, perform nonlinear quantization: change the quantization step height of at least one of the above intervals to obtain the corresponding nonlinear quantization ramp; wherein, when only the quantization step height of one interval is changed, that interval is the last interval S mentioned above. i-1 When changing the quantization step height of two or more intervals, the intervals that are changed include the last interval S. i-1 and from interval S i-1 The intervals proceed sequentially forward;

[0010] Step 3: Substitute the original light intensity information into the nonlinear quantization ramp and map it to the corresponding quantized value. This value is the quantized pixel grayscale value, thereby obtaining the grayscale image after nonlinear quantization encoding.

[0011] Furthermore, in step 2, when the quantization step height of two or more intervals is changed, the last interval S i-1 The quantization step height is a multiple of the quantization step height of the sequentially preceding intervals, and the multiple increases sequentially.

[0012] Furthermore, the bit size i is 12, and the quantization space is divided into 12 intervals: S 11 S 10 , S9, S8, S7, S6, S5, S4, S3, S2, S1, S0.

[0013] Furthermore, the quantization step height of the five intervals is changed: Interval S 11 The quantization step height is 32, and the interval S 10 The quantization step height is 16, the quantization step height of interval S9 is 8, the quantization step height of interval S8 is 4, the quantization step height of interval S7 is 2, and the quantization step height of the remaining intervals is 1.

[0014] Furthermore, before performing nonlinear quantization in step 2, in order to find a suitable quantization ramp for subsequent imaging, an algorithm is first used for simulation. In order to simulate the shot noise of the imaging device, nonlinear noise is added to the image. Then, the image with noise is fed into the neural network for quantization and reconstruction, and the reconstruction effect is evaluated.

[0015] This invention provides a nonlinear image acquisition quantization method, which differs from the widely used linear quantization schemes. With continuous advancements in technology, the full-well capacity of image sensors increases, requiring more bits for quantization of acquired information. This places higher demands on transmission bandwidth and storage capacity. Furthermore, during signal readout, uniform quantization does not achieve satisfactory results due to the presence of shot noise in the device. The nonlinear quantization method provided by this invention performs fine quantization under low signal strength (low SNR) conditions, while performing only coarse quantization under high signal strength (high SNR) conditions. This achieves a relatively ideal quantization effect while significantly saving transmission bandwidth and storage capacity. Moreover, compared with other existing nonlinear coding methods, the nonlinear quantization coding method of this invention can generate different optimal quantization curves for different imaging devices, exhibiting strong versatility and universality. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the imaging device structure used in an embodiment of the present invention;

[0017] Figure 2 This is a schematic diagram of the equivalent ramp of the quantized step in this invention. The horizontal axis is the original digital encoding value of the light intensity information, the left vertical axis is the corresponding output value after quantization, and the right vertical axis is the noise value (mainly shot noise).

[0018] Figure 3 This is a conceptual diagram of the simulation steps of the quantization method of the present invention;

[0019] Figure 4 This is a simulation effect diagram of the quantization method of the present invention;

[0020] Figure 5 These are the shooting effect diagrams of Embodiment 1 of the present invention. (a) is the curve corresponding to the quantization step. The vertical axis is the original digital encoding value of the light intensity information, and the horizontal axis is the corresponding gray value after quantization. (b) and (d) are the effect diagrams obtained by shooting twice. (c) and (e) are the pixel gray distribution diagrams corresponding to Figures (b) and (d), respectively.

[0021] Figure 6 These are the shooting effect diagrams of Embodiment 2 of the present invention. (a) is the curve corresponding to the quantization step. The vertical axis is the original digital encoding value of the light intensity information, and the horizontal axis is the corresponding gray value after quantization. (b) and (d) are the effect diagrams obtained by shooting twice. (c) and (e) are the pixel gray distribution diagrams corresponding to Figures (b) and (d), respectively.

[0022] Figure 7These are the shooting effect diagrams of Embodiment 3 of the present invention. (a) is the curve corresponding to the quantization step. The vertical axis is the original digital encoding value of the light intensity information, and the horizontal axis is the corresponding gray value after quantization. (b) and (d) are the effect diagrams obtained by two shootings. (c) and (e) are the pixel gray distribution diagrams corresponding to Figures (b) and (d), respectively.

[0023] Figure 8 This is a graph showing the relationship between image quality and quantization level in an embodiment of the present invention. Detailed Implementation

[0024] This embodiment is based on an existing vertical charge transfer (VPS) imaging device (patent number CN200910024504.6), and a simplified structural diagram of the device is shown below. Figure 1 As shown, other imaging devices can also be used. First, a suitable bias voltage is applied to each end of the device to form a depletion region in the substrate. When light shines on the device, photogenerated electron-hole pairs are generated in the depletion region. Then, by appropriately changing the voltage conditions of the device, a current can be generated. Since different light intensities result in different numbers of electron-hole pairs, the current value of the device can reflect the light intensity information. The current value is then quantized into pixel grayscale values ​​using the method of this invention to obtain a grayscale image.

[0025] This embodiment provides a nonlinear image acquisition quantization method, whose equivalent quantization step slope conceptual diagram is shown below. Figure 2 As shown. In this image quantization method, the increment value of the original image grayscale signal corresponding to a unit increment of the quantized digital signal is non-linear and not fixed: smaller grayscale values ​​in the image correspond to much smaller increments in the quantization result than larger grayscale values. Ideally, the quantization step size is approximately proportional to the pixel grayscale value. The specific steps of the method in this embodiment include:

[0026] Step 1: For the raw light intensity information of bit size i collected by the imaging device, its corresponding quantization space is 0 to 2. i -1, using binary to divide the quantization space into i intervals: 0~1, 1~3, 3~7, ..., 2 i-1 -1~2 i -1, and are denoted sequentially as S0, S1, S2, S3, ..., S i-1 ;

[0027] Step 2, perform nonlinear quantization: change the quantization step height of at least one of the above intervals to obtain the corresponding nonlinear quantization slope;

[0028] Step 3: Substitute the original light intensity information into the nonlinear quantization ramp and map it to the corresponding quantized value. This value is the quantized pixel grayscale value, thus obtaining the grayscale image after nonlinear quantization encoding.

[0029] Early CIS (CMOS Image Sensor) signal readout used 8-bit quantization. With continuous advancements in technology and increased full-well capacity, CIS quantization requires more bits, placing higher demands on transmission bandwidth and storage capacity. During signal readout, uniform quantization does not achieve optimal results due to shot noise. Fine quantization is necessary for low signal strength, while coarse quantization is sufficient for high signal strength, achieving a more ideal result while significantly saving transmission bandwidth and storage capacity.

[0030] Considering that the dominant noise of the imaging device VPS is shot noise, when the signal strength is low, the SNR is low, and the quantization step needs to be fine; when the signal strength is high, the SNR is high, and the quantization step can be raised accordingly without affecting the quantization effect. By changing the ramp to a multi-ramp form, the AD conversion can be completed without the full 4096 CLKs when capturing a 12-bit image, which greatly reduces the readout time.

[0031] To verify the feasibility and advantages of nonlinear quantization over linear quantization, a suitable nonlinear quantization ramp (i.e., nonlinear quantization curve) was pre-selected for subsequent imaging experiments. First, linear and nonlinear quantization simulations of converting 8-bit images to 4-bit images were performed in Python. The nonlinear quantization curve used in the simulation was: Where g represents the original light intensity information, h represents the quantized image grayscale value, and a, b, and c are the fitting parameters to be obtained through simulation training. The quantization effects are compared. Simultaneously, to simulate the shot noise of the imaging device VPS used in the experiment, nonlinear noise was added to the ground truth (GT) image. The noisy image was then fed into the neural network, where the method of changing the quantization step according to this invention was used for nonlinear quantization and reconstruction. The reconstruction effect was then evaluated. Specific steps are as follows: Figure 3 As shown.

[0032] The simulation effect is as follows Figure 4 As shown, the first column is the original ground truth (GT) image, the second column is the image after adding nonlinear noise, and the third and fourth columns are the reconstruction results after linear and nonlinear quantization. Comparing the similarity between the two reconstruction results and GT, under linear reconstruction, the image's PSNR and SSIM are 26.012 and 0.8381, respectively; under nonlinear reconstruction, the image's PSNR and SSIM reach 27.790 and 0.8404, respectively. Based solely on these two image quality metrics, the effect of nonlinear quantization even surpasses that of the original linear quantization.

[0033] Example 1

[0034] In this embodiment, 12-bit image capture is used, corresponding to a pixel grayscale value quantization space of 0–4095. This quantization space is divided into 12 intervals in binary: 0–1, 1–3, 3–7, 7–15, …, 2047–4095, denoted as S0, S1, S2, S3, …, S… 11 In the first shot, the step height for all sections was set to 1, which is equivalent to linear quantization and serves as a control group to analyze the effects of different quantization curves; the second shot... 11 The step height is 2, and the step height in other sections is 1. The quantified step corresponding curve is as follows: Figure 5 As shown in (a). Figure 5 (b) and (d) are the effect images obtained from the two shots, and (c) and (e) show the corresponding pixel grayscale distribution maps. The SSIM and PSNR of the two images are 0.9715 and 34.47, respectively, while the quantization bit of the image obtained by non-linear quantization is only 75% of that of the linear quantization case.

[0035] Example 2

[0036] In this embodiment, 12-bit image capture is used, corresponding to a pixel grayscale value quantization space of 0–4095. This quantization space is divided into 12 intervals in binary: 0–1, 1–3, 3–7, 7–15, …, 2047–4095, denoted as S0, S1, S2, S3, …, S… 11 In the first shot, the step height for all sections was set to 1, which is equivalent to linear quantization and serves as a control group to analyze the effects of different quantization curves; the second shot... 11 The step height is 32, S 10 The step height is 16, step height S9 is 8, step height S8 is 4, step height S7 is 2, and the step height of the remaining intervals is 1. The quantified step corresponding curve is as follows: Figure 6 As shown in (a). Figure 6 (b) and (d) are the effect images obtained from the two shots, and (c) and (e) show the corresponding pixel grayscale distribution maps. The SSIM and PSNR of the two images are 0.9451 and 31.75, respectively, while the quantization bit depth of the image obtained by non-linear quantization is only 12.5% ​​of that of the linear quantization case.

[0037] Example 3

[0038] In this embodiment, 12-bit image capture is used, corresponding to a pixel grayscale value quantization space of 0–4095. This quantization space is divided into 12 intervals in binary: 0–1, 1–3, 3–7, 7–15, …, 2047–4095, denoted as S0, S1, S2, S3, …, S… 11In the first shot, the step height for all sections was set to 1, which is equivalent to linear quantization and serves as a control group to analyze the effects of different quantization curves; the second shot... 11 The step height is 2048, S 10 The step height is 1024, S9 step height is 512, S8 step height is 256, S7 step height is 128, S6 step height is 64, S5 step height is 32, S4 step height is 16, S3 step height is 8, S2 step height is 4, S1 step height is 2, and S0 step height is 1. The quantified step corresponding curve is as follows: Figure 7 As shown in (a). Figure 7 (b) and (d) are the effect images obtained from the two shots, and (c) and (e) show the corresponding pixel grayscale distribution maps. The SSIM and PSNR of the two images are 0.5409 and 22.59, respectively, while the quantization bit of the image obtained by non-linear quantization is only 0.2% of that of the linear quantization case.

[0039] Based on the above data, combining SSIM and PSNR, from Figure 8 As can be seen, within the range of quantization time reduced from 4096 clocks to 900 clocks, nonlinear imaging can still achieve relatively ideal results, effectively reducing the quantization bit depth and acquisition time while ensuring image quality. This demonstrates that the nonlinear image acquisition quantization method of this invention is superior to the traditional linear quantization method to a certain extent, effectively reducing the quantization bit depth and acquisition time while ensuring image quality, and using fewer bits to obtain a given signal-to-noise ratio for the region of interest.

Claims

1. A method of non-linear image acquisition quantification, comprising: The steps of the quantification method are: Step 1, for the original light intensity information collected by the imaging device with bit size i, the corresponding quantization space is 0~2 i -1, the quantization space is divided into i intervals by binary: 0~1, 1~3, 3~7, …, 2 i-1 -1~2 i -1, sequentially recorded as S0, S1, S2, S3, …, S i-1 ; Step 2, performing non-linear quantization: changing the quantization step height of at least one of the above intervals to obtain a corresponding non-linear quantization slope; wherein when the quantization step height of only one interval is changed, the interval is the last interval S i-1 ; when the quantization step height of more than two intervals is changed, the changed intervals include the last interval S i-1 and the intervals sequentially in the forward direction from the interval S i-1 ; Step 3: Substitute the original light intensity information into the nonlinear quantization slope mapping to obtain the corresponding quantized value, which is the quantized pixel gray value, thereby obtaining the gray image after nonlinear quantization coding.

2. The method of claim 1, wherein, In the step 2, when the quantization step heights of two or more intervals are changed, the quantization step height of the last interval S i-1 has a multiple relationship with the quantization step height of the interval in the order of the preceding interval, and the multiple is sequentially increased.

3. The method of claim 1, wherein the non-linear image acquisition quantification method is a non-linear image acquisition quantification method for a non-linear image acquisition system. The bit size i is 12, and the quantization space is divided into 12 intervals: S 11 , S 10 , S9, S8, S7, S6, S5, S4, S3, S2, S1, S0.

4. The method of claim 3, wherein the non-linear image acquisition quantification method is a non-linear image acquisition quantification method for a non-linear image acquisition system. The quantization step heights of the five intervals are changed: the quantization step height of interval S 11 is 32, the quantization step height of interval S 10 is 16, the quantization step height of interval S9 is 8, the quantization step height of interval S8 is 4, the quantization step height of interval S7 is 2, and the quantization step height of the remaining intervals is 1.

5. The method of claim 1, wherein the non-linear image acquisition quantification method is a non-linear image acquisition quantification method for a non-linear image acquisition system. Before step 2, in order to find the appropriate quantization slope for subsequent shooting, first use the algorithm for simulation, in order to simulate the imaging device's shot noise, add nonlinear noise to the image, then send the image with noise into the neural network for quantization and reconstruction, and evaluate the reconstruction effect.

Images