An oversampling imaging method based on in-pixel quantum efficiency measurement

Abstract

The application discloses a supersampling imaging method based on in-pixel quantum efficiency measurement, which comprises the following steps: a single pixel of a detector is divided into k 2 micro-pixels by an in-pixel light field response model of an imaging detector, and a function relationship between a light field corresponding to the micro-pixel and a pixel gray value is established; for a light field S mn corresponding to a pixel point (m, n) of the imaging detector, the same light field S mn is imaged by different pixels of the imaging detector, and an equation group between the gray values of the different pixels and the light field corresponding to the micro-pixel is constructed; under the premise that the pixel gray value I and the response parameters [a, b, c] of the micro-pixel are known, a high-frequency light field S mn (i) corresponding to each micro-pixel in the light field S mn is solved; and the high-frequency light fields S mn (i) corresponding to all the micro-pixels are reconstructed to obtain a high-resolution image. The method does not need to construct a training data set for model training, which simplifies the complexity of image processing on one hand, and fundamentally avoids the dependence of the quality of the reconstructed image on the training data set on the other hand.

Description

Technical Field

[0001] This invention relates to the field of optical imaging technology, and in particular to an oversampling imaging method based on intra-pixel quantum efficiency measurement. Background Technology

[0002] Resolution is a crucial indicator of image quality; higher resolution results in finer details, better quality, and richer information content. Therefore, obtaining higher-resolution images has significant application value and research prospects in various computer vision tasks, such as military security, satellite surveillance, traffic management, and industrial automation. However, in practical applications, the resolution of acquired images is limited by the performance of imaging systems, especially imaging detectors, failing to meet application requirements. Therefore, it is necessary to research image resolution enhancement methods to compensate for the shortcomings of existing imaging systems and provide high-resolution images for production and daily life. Image super-resolution technology is currently the mainstream technique for improving image resolution. It can recover information from high-resolution images from low-resolution images. This technology can be implemented through interpolation-based algorithms and neural network-based methods. The earliest image super-resolution methods were based on interpolation, such as bicubic interpolation. Since super-resolution is an ill-posed problem, the mapping process from a low-resolution image to a high-resolution image for each pixel has many solutions. Interpolation-based methods only use information from low-resolution images, making it difficult to accurately simulate the visual complexity of real images. For images with complex textures and smooth coloring, this method easily produces unrealistic effects and cannot reconstruct high-resolution images well.

[0003] In recent years, with the application of Convolutional Neural Networks (CNNs) in computer vision, many CNN-based image super-resolution methods have emerged. These methods reconstruct high-resolution images by learning the mapping relationship between low-resolution and high-resolution images. The mapping is represented by a CNN, taking the low-resolution image as input and the high-resolution image as output. This method leverages the advantages of neural networks, modeling the image super-resolution problem as a neural network structure. By optimizing the objective function and training a suitable neural network, a simple and effective model for enhancing image resolution is obtained. CNN-based image super-resolution methods require constructing large datasets of both low-resolution and high-resolution images for model training. However, datasets reflecting the resolution degradation process of real-world scenes are difficult to obtain. More importantly, CNN methods generate image-to-image mappings rather than enhancing the physical parameters of the detector itself. This severely affects the realism of the reconstructed high-frequency details and textures, as well as the quality of the final high-resolution image. Summary of the Invention

[0004] The purpose of this invention is to provide an oversampling imaging method based on intra-pixel quantum efficiency measurement. Compared with existing image super-resolution methods based on neural networks, this method does not require the construction of a training dataset for model training. On the one hand, it simplifies the complexity of image processing, and on the other hand, it fundamentally avoids the dependence of the reconstructed image quality on the training dataset. This effectively ensures the authenticity of the reconstructed high-resolution image and enhances the practicality and universality of the method.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] An oversampling imaging method based on in-pixel quantum efficiency measurement, the method comprising:

[0007] Step 1: Construct an intra-pixel optical field response model for the imaging detector, dividing each pixel of the detector into k... 2 We establish a functional relationship between the light field corresponding to each micro-pixel and the pixel gray value.

[0008] Step 2: Use different pixels of the imaging detector to image the same light field Smn, and construct a set of equations between the gray values ​​of different pixels and the light field corresponding to the micro-pixel.

[0009] Step 3: Given the pixel grayscale value I and the micro-pixel response parameters [a, b, c], solve the equations from Step 2 to obtain the light field S. mn The high-frequency light field S corresponding to each micro-pixel mn (i);

[0010] Step 4: Convert the high-frequency light field S corresponding to all micro-pixels obtained in Step 3. mn (i) Reconstruct the image to obtain a high-resolution image.

[0011] As can be seen from the technical solution provided by the present invention, the above method does not require the construction of a training dataset for model training. On the one hand, it simplifies the complexity of image processing, and on the other hand, it fundamentally avoids the dependence of the reconstructed image quality on the training dataset. It can effectively ensure the authenticity of the reconstructed high-resolution image and improve the universality of the method. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0013] Figure 1 A schematic diagram of the oversampling imaging method based on in-pixel quantum efficiency measurement provided in an embodiment of the present invention;

[0014] Figure 2 This is a schematic diagram of the intra-pixel optical field response model of the imaging detector according to an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of the scheme for solving the high-frequency light field by static light field scanning imaging according to an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0017] like Figure 1 The diagram shows a flowchart of an oversampling imaging method based on intra-pixel quantum efficiency measurement provided by an embodiment of the present invention. The method includes:

[0018] Step 1: Construct an intra-pixel optical field response model for the imaging detector, dividing each pixel of the detector into k... 2 We establish a functional relationship between the light field corresponding to each micro-pixel and the pixel gray value.

[0019] In this step, such as Figure 2 The diagram shown is a schematic of the intra-pixel light field response model of the imaging detector according to an embodiment of the present invention. A single pixel of the detector is divided into k... 2 For each micro-pixel, the functional relationship between the light field corresponding to the micro-pixel and the pixel gray value is as follows:

[0020]

[0021] Where I is the grayscale value of the pixel; S is the light field intensity; a i b i c i Let S(i) be the response parameters of the i-th micro-pixel in the imaging detector, and let S(i) be the coefficients of the zero-order, first-order, and second-order terms of the light field, respectively; S(i) is the light field corresponding to the i-th micro-pixel.

[0022] Step 2: Using different pixels of the imaging detector to image the same light field S mn To perform imaging, a set of equations is constructed between the gray values ​​of different pixels and the light field corresponding to the micro-pixels;

[0023] In this step, the system of equations relating the gray values ​​of different pixels to the light field corresponding to the micro-pixels is expressed as follows:

[0024]

[0025] Among them, S mn (i) represents the light field S mn The high-frequency light field corresponding to the i-th micro-pixel; k 2 Let [a, b, c] be the number of micro-pixels segmented; [a, b, c] be the response parameters of the micro-pixels; I be the gray value of the pixel; and L be the number of equations in the system of equations, where L ≥ k. 2 Since different pixels of the detector respond differently to the light field (i.e., the response parameters [a, b, c] of the micro-pixels are different), the gray values ​​I of different pixels in the equation system are also different.

[0026] Due to the light field S mn It contains k 2 A high-frequency light field S mn (i) To accurately solve for these high-frequency optical fields, at least K is required. 2 (i.e., L≥k) 2 ) pixels for light field S mn Imaging is performed to ensure that the system of equations contains at least k 2 Each equation has a unique solution.

[0027] In specific implementations, to enable different pixels of the detector to image the same light field, the embodiments of the present invention adopt the following solutions for different scenarios:

[0028] 1) Scanning imaging for static light fields, such as Figure 3 The diagram shown is a schematic of the scheme for solving the high-frequency light field by static light field scanning imaging according to an embodiment of the present invention. The imaging detector is fixed on a one-dimensional electric displacement stage, and the lens is separated from the imaging detector and fixed separately. It should be noted that the displacement accuracy of the electric displacement stage is much higher than the pixel size of the micro-pixel. The higher the displacement accuracy, the more accurate the reconstructed light field.

[0029] During imaging, the lens position remains unchanged, and the one-dimensional electric displacement stage is controlled to move the imaging detector a distance of an integer number of pixels (generally one pixel). Then, the imaging detector is controlled to acquire one frame of image.

[0030] Repeat the above movement-image acquisition process to acquire at least k images. 2 Frame image;

[0031] When processing images, the corresponding pixels of the same light field in different images are determined according to the relative position of each frame image, and then the corresponding set of equations is constructed, as shown in formula (2).

[0032] 2) For super-resolution imaging of moving objects, the imaging detector is connected to the lens to continuously acquire multiple frames of images of the target object. During imaging, there is no need for a displacement stage to move the detector. Instead, different pixels are imaged on the same light field based on the motion of the object itself relative to the detector. The core of this scheme is to determine the position of the pixels occupied by the moving target in the image frame.

[0033] After image acquisition is complete, select one frame as the reference image R, extract a sub-region containing the target object from the reference image R, and calculate the normalized cross-correlation matrix C(u, v) between the sub-region image and all images. The calculation formula is as follows:

[0034]

[0035] Where f is the image to be calculated; t is the sub-region containing the target object cropped from the reference image R. The average value of t; is the average value of the sub-region f(u,v) corresponding to t in the image f(x,y); x and y are the two-dimensional coordinates of the image pixels; u and v refer to the relative positions of the two frames of the image;

[0036] Then, a two-dimensional Gaussian fit is performed on C(u, v) to determine the position p of the peak value of the cross-correlation between f and t; where the peak value corresponding to the reference image R is p. R The position corresponding to any other frame is p. f p R and p f The difference is the relative offset d of the target object in the two frames, and the relative offset of the pixel coordinates of the same light field in the two frames is also d.

[0037] Among all images, select the image with a relative offset d that is an integer, and locate the corresponding pixel based on the relative offset d, thus providing a reference for any light field S. mn Construct the corresponding system of equations.

[0038] Additionally, it should be noted that in practice, it is difficult to guarantee that the relative offset d is an integer. Therefore, in this embodiment, all relative offsets d that conform to the following formula are considered integers:

[0039] |dn|≤Δ (4)

[0040] Where n is any integer; Δ is the tolerance of displacement estimation. The smaller the tolerance value, the more accurate the reconstructed light field.

[0041] Step 3: Given the pixel grayscale value I and the micro-pixel response parameters [a, b, c], solve the equations from Step 2 to obtain the light field S. mn The high-frequency light field S corresponding to each micro-pixel mn (i);

[0042] In this step, when the number of equations in the system of equations from step 2 is greater than k 2 At this point, the solution process of the system of equations becomes a process of fitting multivariate functions. Therefore, the least squares method is used to perform nonlinear fitting on the system of equations in step 2, and then the light field S is obtained. mn The high-frequency light field S corresponding to each micro-pixel mn (i).

[0043] Step 4: Convert the high-frequency light field S corresponding to all micro-pixels obtained in Step 3. mn (i) Reconstruct the image to obtain a high-resolution image.

[0044] In this step, when the high-frequency light field S corresponding to the micro-pixel mn The number of (i) is k 2 At this time, the size of the micro-pixel is the pixel size. Therefore, the high-frequency light field S mn (i) The corresponding image frequency is the light field S mn Since the resolution of a high-resolution image is k times that of the original image acquired by the imaging detector, the method proposed in this invention significantly improves the resolution of the image.

[0045] For example, if the image image captured by the imaging detector has 50*30 pixels, that is, 50*30 pixel values, and the number of micro-pixels corresponding to one pixel is 16, that is, 4*4, then the number of high-frequency light fields corresponding to all micro-pixels obtained in step 3 is 200*120; then a high-resolution image with 200*120 pixels is directly reconstructed.

[0046] The method described in this embodiment of the invention has no special requirements for the imaging band of the imaging detector and is theoretically applicable to detectors of all bands. Because this invention achieves super-resolution by solving the light field within the detector pixel, the sampling rate of the light field in the final reconstructed image is higher than the sampling rate of the detector itself. Therefore, the method of this invention is called the supersampling imaging method.

[0047] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0048] In summary, the method described in this embodiment of the invention reconstructs the real physical light field using the intra-pixel light field response model of the detector, thereby reconstructing a high-resolution image. Compared with existing CNN-based image super-resolution methods, the method described in this embodiment of the invention does not require the construction of a training dataset for model training. This simplifies the complexity of image processing and fundamentally avoids the dependence of the reconstructed image quality on the training dataset, effectively ensuring the realism of the reconstructed high-resolution image and further improving the universality of the method.

[0049] Furthermore, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program controlling the relevant hardware, and the corresponding program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0050] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. An oversampling imaging method based on in-pixel quantum efficiency measurement, characterized in that, The method includes: Step 1: Construct an intra-pixel optical field response model for the imaging detector, dividing each pixel of the detector into k... 2 We establish a functional relationship between the light field corresponding to each micro-pixel and the pixel gray value. In step 1, the functional relationship between the light field corresponding to a micro-pixel and the pixel gray value is established as follows: Where I is the grayscale value of the pixel; S is the light field intensity; a i b i c i Let S(i) be the response parameter of the i-th micro-pixel in the imaging detector pixel, and let S(i) be the coefficients of the zero-order, first-order, and second-order terms of the light field, respectively; S(i) is the light field corresponding to the i-th micro-pixel. Step 2: Using different pixels of the imaging detector to image the same light field S mn To perform imaging, a set of equations is constructed between the gray values ​​of different pixels and the light field corresponding to the micro-pixels; In step 2, the system of equations relating the gray values ​​of different pixels to the light fields corresponding to the micro-pixels is expressed as follows: Among them, S mn (i) represents the light field S mn The high-frequency light field corresponding to the i-th micro-pixel; k 2 Let [a, b, c] be the number of micro-pixels segmented; [a, b, c] be the response parameters of the micro-pixels; I be the gray value of the pixel; and L be the number of equations in the system of equations, where L ≥ k. 2 ; Step 3: Given the pixel grayscale value I and the micro-pixel response parameters [a, b, c], solve the equations from Step 2 to obtain the light field S. mn The high-frequency light field S corresponding to each micro-pixel mn (i); Step 4: Convert the high-frequency light field S corresponding to all micro-pixels obtained in Step 3. mn (i) Reconstruct the image to obtain a high-resolution image.

2. The oversampling imaging method based on in-pixel quantum efficiency measurement according to claim 1, characterized in that, In step 2, the different pixels of the imaging detector are used to visualize the same light field S. mn The imaging process is as follows: For scanning imaging of static light fields, the imaging detector is fixed on a one-dimensional electric displacement stage, and the lens is separated from the imaging detector and fixed separately. During imaging, the lens position remains unchanged, and the one-dimensional electric displacement stage is controlled to move the imaging detector a distance of integer pixels. Then, the imaging detector is controlled to acquire one frame of image. Repeat the above movement-image acquisition process to acquire at least k images. 2 Frame image; When processing images, the corresponding pixels of the same light field in different images are determined based on the relative position of each frame, and then the corresponding set of equations is constructed.

3. The oversampling imaging method based on in-pixel quantum efficiency measurement according to claim 1, characterized in that, In step 2, the different pixels of the imaging detector are used to visualize the same light field S. mn The imaging process is as follows: For super-resolution imaging of moving objects, the imaging detector is connected to the lens to continuously acquire multiple frames of images of the target object. During imaging, there is no need for a displacement stage to move the detector. Instead, different pixels are imaged on the same light field based on the motion of the object itself relative to the detector. After image acquisition is complete, select one frame as the reference image R, extract a sub-region containing the target object from the reference image R, and calculate the normalized cross-correlation matrix C(u, v) between the sub-region image and all images. The calculation formula is as follows: Where f is the image to be calculated; t is the sub-region containing the target object cropped from the reference image R. The average value of t; is the average value of the sub-region f(u,v) corresponding to t in the image f(x,y); x and y are the two-dimensional coordinates of the image pixels; u and v refer to the relative positions of the two frames of the image; Then, a two-dimensional Gaussian fit is performed on C(u, v) to determine the position p of the peak value of the cross-correlation between f and t; where the peak value corresponding to the reference image R is p. R The position corresponding to any other frame is p. f p R and p f The difference is the relative offset d of the target object in the two frames, and the relative offset of the pixel coordinates of the same light field in the two frames is also d. Among all images, select the image with a relative offset d that is an integer, and locate the corresponding pixel based on the relative offset d, thus providing a reference for any light field S. mn Construct the corresponding system of equations.

4. The oversampling imaging method based on in-pixel quantum efficiency measurement according to claim 1, characterized in that, In step 3, when the number of equations in the system of equations from step 2 is greater than k 2 At this point, the process of solving the system of equations becomes a process of fitting multivariate functions. The least squares method is used to perform nonlinear fitting on the system of equations in step 2 to obtain the light field S. mn The high-frequency light field S corresponding to each micro-pixel mn (i).

5. The oversampling imaging method based on in-pixel quantum efficiency measurement according to claim 1, characterized in that, In step 4, When the high-frequency light field S corresponding to the micro-pixel mn The number of (i) is k 2 At this time, the size of the micro-pixel is the pixel size. Therefore, the high-frequency light field S mn (i) The corresponding image frequency is the light field S mn Therefore, the highest frequency that a high-resolution image can resolve is k times that of the original image acquired by the imaging detector.

Images