A high-resolution image reconstruction method based on a single detector spectral unit

Abstract

The application discloses a high-resolution image reconstruction method based on a single-detector spectral unit, relates to the technical field of image processing, and successfully reconstructs a high-resolution gray image by mining and modeling common intermediate state detail information from original data of a spectral filter unit, and then injects spatial details into low-resolution spectral data through principal component analysis fusion; the method not only significantly reduces system cost, volume and complexity, maintains the capturing capability of snapshot technology on a dynamic scene, and solves the image degradation problem caused by uneven filter transmittance and sensor gaps, and meanwhile, the technology is highly compatible with a dispersive imaging system, can be combined with depth information for weighted fusion to further optimize global definition, and is a kind of efficient, practical and extremely application potential pure algorithm solution.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically a high-resolution image reconstruction method based on a single detector spectral unit. Background Technology

[0002] Snapshot spectral imaging technology can capture the spectral information of a scene in a single exposure, which has important application value in dynamic monitoring, biomedical imaging and other fields. However, due to the limitations of filter arrays and sensor structures, the system suffers from low spatial resolution. Traditional methods improve resolution by increasing sensor accuracy or filter density, but this increases system cost, size and complexity, and is difficult to apply to dynamic scenes.

[0003] Existing software-based super-resolution methods often rely on interpolation, sparse reconstruction, or deep learning algorithms, which are computationally complex and prone to introducing spectral distortion, making it difficult to balance spectral accuracy and spatial detail. In addition, problems such as uneven filter transmittance, pixel gaps, and dispersion shift further reduce image quality, and existing fusion methods have failed to effectively overcome these physical imaging defects. Summary of the Invention

[0004] Therefore, in order to overcome the above-mentioned shortcomings, the present invention provides a high-resolution image reconstruction method based on a single detector spectral unit.

[0005] This invention is implemented by constructing a high-resolution image reconstruction method based on a single detector spectral unit, the apparatus comprising the following steps:

[0006] Step 1: Data Extraction and Preprocessing; A method combining a dispersive lens and a filter array spectral camera is used to acquire the spectral information of the raw image within an integral time. The pixel blocks of each spectral channel are located and extracted according to the arrangement of the filter array, and the values ​​within each pixel block are averaged to generate an initial low-resolution spectral image.

[0007] Step 2: Lateral resolution reconstruction and principal component analysis fusion; additional high-resolution images are used to guide the reconstruction of spectral image resolution. The high-resolution images provide rich image details, while the low-resolution spectral images provide spectral information. The high-resolution details and spectral information are fused to obtain a high lateral resolution spectral image.

[0008] Step 3: High-resolution grayscale image reconstruction algorithm;

[0009] Step 4: Dispersive multispectral image sharpening deep fusion algorithm. This method combines dispersive imaging with spectral imaging. Sharp images of objects at different depths in a 3D scene are captured by different spectral channels. The lateral resolution of the spectral image is reduced compared to the sharp imaging system.

[0010] Preferably, the set of N channels of spectral images in step two and high-resolution images The principal component analysis fusion algorithm flow is as follows:

[0011] The low-resolution spectral image is upsampled to the size of the high-resolution image to obtain a set of upsampled spectral images. PCA transformation is performed on the upsampled spectral image set to obtain the principal component set. Using high-resolution images With the first principal component Perform pixel registration and register the high-resolution image. As the new first principal component To form a new set of principal components, and to form a new set of principal components. A set of high-resolution spectral images was obtained by performing inverse PCA transform. .

[0012] Preferably, the specific implementation process of the PCA transformation and inverse transformation in step two is as follows:

[0013] First, calculate the mean value of each spectral channel. ;

[0014] Then perform matrix centering ;

[0015] Find the covariance matrix ;

[0016] in, Given the number of pixels in the spectral image and the periodicity T of the spectral primitives, calculate the eigenvectors and eigenvalues ​​of the covariance matrix. Then, rearrange the eigenvectors in descending order of the eigenvalues ​​to obtain the transformation matrix. ; Perform PCA transformation based on the transformation matrix to obtain the principal components of the spectral image. After replacing the first principal component, perform inverse PCA transformation on the new principal component: .

[0017] Preferably, the algorithm flow for step three is as follows:

[0018] From the original image The method locates the effective spectral region window using pixel indexing and then averages the values ​​within the window to obtain a low-resolution spectral image. Complete low-resolution spectral images Upsampling involves re-interpolating and filling the image pixel space to a multiple of n based on the number of spectral channels (n). The interpolated spectral intensity... Different spectral channels They have different starting coordinates, based on their starting coordinates Extract overlapping regions from... Starting from the corresponding region, for any spectral intensity Extract the starting coordinates coordinates to the destination Obtaining multispectral light intensity in a rectangular region After truncation, the coordinates satisfy the equation The interpolated and filled spectral image is obtained by performing pixel offset compensation according to the equation. ;

[0019] Interpolated filled spectral images Integrating along the spectral dimension yields the predicted integral light intensity. Original image With low-resolution spectral images Divide to obtain intermediate state Predicting the integral light intensity With intermediate state Multiplication to obtain a high-resolution grayscale image Use bicubic interpolation algorithm to fill high-resolution grayscale images The gaps in the image; using high-resolution grayscale images Alternative spectral images The first principal component was used to obtain a high-resolution spectral image. .

[0020] Preferably, the algorithm flow for step four is as follows:

[0021] From the original image The effective spectral region window is located by pixel indexing, and the low-resolution spectral image is obtained by averaging the values ​​within the window. ;Finish Upsampling padding, using pixel offset compensation to obtain interpolated filled spectral images ;;

[0022] For interpolated filled spectral images For each pixel in different channels, the weights are redistributed based on the depth value. The predicted integral light intensity is then re-optimized as follows: To obtain the predicted integral light intensity ;

[0023] Depth of focus corresponds to depth z and spectral channels The degree of distance between them determines the use of interpolation to fill the spectral image. The dense depth map is calculated using this as input. The factors influencing the image blur parameter σ are analyzed. It is related to the absolute value of the difference between the inverse of the depth of focus and the depth of focus. Defined as:

[0024] Original image With low-resolution spectral images Divide the data, separate each channel based on its pixel index, and fill in the gaps to obtain the channel-independent intermediate states. According to weight With formula Get intermediate state ;

[0025] because Low-resolution spectral images of

[0026] Times, compared to the original image With consistent resolution, intermediate states are acquired separately for each channel. Perform weighted fusion: Predicting the integral light intensity With intermediate state Multiplication to obtain a high-resolution grayscale image Use bicubic interpolation algorithm to fill high-resolution grayscale images The gaps in the image; using high-resolution grayscale images Alternative spectral images The first principal component was used to obtain a high-resolution spectral image. .

[0027] The present invention has the following advantages: It provides an improved high-resolution image reconstruction method based on a single detector spectral unit, which, compared with similar devices, has the following improvements:

[0028] This invention discloses a high-resolution image reconstruction method based on a single detector spectral unit. By mining and modeling shared "intermediate state" detail information from the raw data of the spectral filter unit, a high-resolution grayscale image is successfully reconstructed. Then, spatial details are injected into the low-resolution spectral data through principal component analysis fusion. This method not only significantly reduces system cost, size, and complexity while maintaining the snapshot technology's ability to capture dynamic scenes, but also specifically solves the image degradation problem caused by uneven filter transmittance and sensor gaps. At the same time, this technology is highly compatible with dispersive imaging systems and can be combined with depth information for weighted fusion to further optimize global sharpness. It is an efficient, practical, and highly promising pure algorithm solution. Attached Figure Description

[0029] Figure 1 This is a flowchart of the principal component analysis fusion algorithm of the present invention;

[0030] Figure 2This is a schematic diagram of the overlapping region of the interpolated and filled spectral image according to the present invention;

[0031] Figure 3 This is a flowchart of the high-resolution spectral image reconstruction process of the present invention;

[0032] Figure 4 This is a flowchart of the high-resolution spectral image reconstruction process optimized for dispersive imaging according to the present invention. Detailed Implementation

[0033] The following is in conjunction with the appendix Figures 1-4 The principles and features of the present invention are described below. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention. The invention is described more specifically in the following paragraphs by way of example with reference to the accompanying drawings. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0034] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0035] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "setting" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The embodiments of this invention will now be described according to its overall structure.

[0036] Please see Figures 1-4 The present invention provides a high-resolution image reconstruction method based on a single detector spectral unit, comprising the following steps:

[0037] Step 1: Data Extraction and Preprocessing; A method combining a dispersive lens and a filter array spectral camera is used to acquire the spectral information of the raw image within an integral time. The pixel blocks of each spectral channel are located and extracted according to the arrangement of the filter array, and the values ​​within each pixel block are averaged to generate an initial low-resolution spectral image.

[0038] Step 2: Lateral resolution reconstruction and principal component analysis fusion; Additional high-resolution images are used to guide the spectral image resolution reconstruction. The high-resolution images provide rich image details, while the low-resolution spectral images provide spectral information. The high-resolution details and spectral information are fused to obtain a high lateral resolution spectral image. The low-resolution spectral images are then upsampled to the size of the high-resolution images to obtain an upsampled spectral image set. PCA transformation is performed on the upsampled spectral image set to obtain the principal component set. Using high-resolution images With the first principal component Perform pixel registration and register the high-resolution image. As the new first principal component To form a new set of principal components, and to form a new set of principal components. A set of high-resolution spectral images was obtained by performing inverse PCA transform. The specific implementation process of PCA transform and inverse transform involves first calculating the mean value of each spectral channel. : Then perform matrix centering. ; Calculate the covariance matrix ;in, Given the number of pixels in the spectral image and the periodicity T of the spectral primitives, calculate the eigenvectors and eigenvalues ​​of the covariance matrix. Then, rearrange the eigenvectors in descending order of the eigenvalues ​​to obtain the transformation matrix. ; Perform PCA transformation based on the transformation matrix to obtain the principal components of the spectral image. After replacing the first principal component, perform inverse PCA transformation on the new principal component: ;

[0039] Step 3: High-resolution grayscale image reconstruction algorithm; from the original image The method locates the effective spectral region window using pixel indexing and then averages the values ​​within the window to obtain a low-resolution spectral image. Complete low-resolution spectral images Upsampling involves re-interpolating and filling the image pixel space to a multiple of n based on the number of spectral channels (n). The interpolated spectral intensity... Different spectral channels They have different starting coordinates, based on their starting coordinates Extract overlapping regions from... Starting from the corresponding region, for any spectral intensity Extract the starting coordinates coordinates to the destination Obtaining multispectral light intensity in a rectangular region After truncation, the coordinates satisfy the equation The interpolated and filled spectral image is obtained by performing pixel offset compensation according to the equation. ;

[0040] Interpolated filled spectral images Integrating along the spectral dimension yields the predicted integral light intensity. Original image With low-resolution spectral images Divide to obtain intermediate state Predicting the integral light intensity With intermediate state Multiplication to obtain a high-resolution grayscale image Use bicubic interpolation algorithm to fill high-resolution grayscale images The gaps in the image; using high-resolution grayscale images Alternative spectral images The first principal component was used to obtain a high-resolution spectral image. .

[0041] Step 4: Dispersive multispectral image sharpening deep fusion algorithm; from the original image The effective spectral region window is located by pixel indexing, and the low-resolution spectral image is obtained by averaging the values ​​within the window. ;Finish Upsampling padding, using pixel offset compensation to obtain interpolated filled spectral images ;

[0042] For interpolated filled spectral images For each pixel in different channels, the weights are redistributed based on the depth value. The predicted integral light intensity is then re-optimized as follows: To obtain the predicted integral light intensity ;

[0043] Depth of focus corresponds to depth z and spectral channels The degree of distance between them determines the use of interpolation to fill the spectral image. The dense depth map is calculated using this as input. The factors influencing the image blur parameter σ are analyzed. It is related to the absolute value of the difference between the inverse of the depth of focus and the depth of focus. Defined as: Original image With low-resolution spectral images Divide the data, separate each channel based on its pixel index, and fill in the gaps to obtain the channel-independent intermediate states. According to weight With formula Get intermediate state ;

[0044] because Low-resolution spectral images of Times, compared to the original image With consistent resolution, intermediate states are acquired separately for each channel. Perform weighted fusion: Predicting the integral light intensity With intermediate state Multiplication to obtain a high-resolution grayscale image Use bicubic interpolation algorithm to fill high-resolution grayscale images The gaps in the image; using high-resolution grayscale images Alternative spectral images The first principal component was used to obtain a high-resolution spectral image. .

[0045] This invention provides an improved high-resolution image reconstruction method based on a single detector spectral unit. By mining and modeling shared "intermediate state" detail information from the raw data of the spectral filter unit, a high-resolution grayscale image is successfully reconstructed. Then, spatial details are injected into the low-resolution spectral data through principal component analysis fusion. This method not only significantly reduces system cost, size, and complexity while maintaining the snapshot technology's ability to capture dynamic scenes, but also specifically addresses the image degradation problems caused by uneven filter transmittance and sensor gaps. At the same time, this technology is highly compatible with dispersive imaging systems and can be combined with depth information for weighted fusion to further optimize global sharpness. It is an efficient, practical, and highly promising pure algorithm solution.

[0046] The above description shows and illustrates the basic principles, main features, and advantages of the present invention. Standard parts used in the present invention can be purchased from the market, and irregular parts can be customized according to the description and drawings. The specific connection methods of each part adopt conventional methods such as bolts, rivets, and welding that are mature in the prior art. The machinery, parts, and equipment adopt conventional models in the prior art, and the circuit connection adopts conventional connection methods in the prior art, which will not be described in detail here.

[0047] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A single-detector spectral unit based high-resolution image reconstruction method, characterized by: Comprising the following steps: Step one, data extraction and pretreatment; the method of dispersion lens combined with filter array spectral camera is adopted to acquire the original image spectral information within an integration time According to the arrangement and positioning of the filter array, the pixel block of each spectral channel is extracted, the value in each pixel block is averaged, and the initial low-resolution spectral image is generated; Step two, lateral resolution reconstruction, principal component analysis fusion; Additional high-resolution image guided spectral image resolution reconstruction, the high-resolution image provides rich image details, the low-resolution spectral image provides spectral information, and the fusion of high-resolution details and spectral information obtains a high lateral resolution spectral image; Step three, high-resolution gray image reconstruction algorithm; Step four, dispersion multispectral image sharpening depth fusion algorithm, using the method of dispersion imaging combined with spectral imaging, the clear image of different depth objects in the three-dimensional scene is captured by different spectral channels, and the lateral resolution of the spectral image is reduced relative to the clear imaging system; The step two N Spectral image set of one channel And high resolution image The principal component analysis fusion algorithm process is as follows: up-sampling low resolution spectral images to high resolution image size to obtain a set of up-sampled spectral images performing PCA transformation on the set of up-sampled spectral images to obtain a set of principal components using a high resolution image performing pixel registration with the first principal component using the registered high resolution image as a new first principal component composing a new set of principal components performing inverse PCA transformation on the new set of principal components to obtain a set of high resolution spectral images .

2. The method of claim 1, wherein the method is based on a single detector spectral unit. The specific implementation process of the PCA transformation and inverse transformation in the step two is as follows: First, the mean of each spectral channel is calculated ; then the matrix is centered ; the covariance matrix is calculated ; Wherein, The number of pixel points in the spectral image, the spectral element arrangement period size T, the characteristic vector and the characteristic value of the covariance matrix are calculated, and the characteristic vector is rearranged in descending order of the characteristic value to obtain a transformation matrix ; PCA transformation is performed according to the transformation matrix to obtain principal components of the spectral image ; After replacing the first principal component, the new principal component is subjected to PCA inverse transformation: .

3. The method of claim 2, wherein the method is based on a single detector spectral unit. The algorithm flow of the step three is as follows: From the original image The window position is located by pixel index in the original image, and the low-resolution spectral image is obtained by averaging in the window ; complete low-resolution spectral image up-sampling, according to spectral channel arrangement number n, re-interpolating and filling image pixel space to n times, for spectral light intensity after interpolation different spectral channels with different starting coordinates, according to the starting coordinates extracting overlapping areas, overlapping areas from starting from the corresponding area, for any spectral light intensity , intercepting the starting coordinates to the end coordinates to obtain multispectral light intensity , the post-interception coordinates satisfy the equation , and according to the equation, pixel offset compensation is performed to obtain an interpolated and filled spectral image ; Interpolation filling spectral image Integrating in spectral dimension to obtain predicted integral light intensity ; original image Dividing by low-resolution spectral image To obtain intermediate state ; predicted integral light intensity Multiply by intermediate state To obtain high-resolution gray image , using bicubic interpolation algorithm to fill the gap in high-resolution gray image ; using high-resolution gray image Instead of the first principal component of spectral image , obtain high-resolution spectral image .

4. The method of claim 3, wherein the method is based on a single detector spectral unit. The algorithm flow of the step four is as follows: From the original image Position the spectral effective region window position by pixel index, take the mean value within the window to obtain a low-resolution spectral image ; complete Up-sampling padding, obtain an interpolation filled spectral image by pixel offset compensation ; For interpolating a spectral image reassigning weights according to depth values for each pixel point in different channels of the spectral image The predicted integrated light intensity is then re-optimized as: to obtain the predicted integrated light intensity ; The degree of departure between the depth z and the focusing depth of the spectral channel pair The degree of departure between the depth z and the focusing depth of the spectral channel pair The degree of departure between the depth z and the focusing depth of the spectral channel pair The degree of departure between the depth z and the focusing depth of the spectral channel pair The degree of departure between the depth z and the focusing depth of the spectral channel pair ​ Original image With low resolution spectral image Divide, separate each channel according to pixel index and fill, get channel independent intermediate state ; According to the weight With formula Get intermediate state ; because Low-resolution spectral images of Times, compared to the original image With consistent resolution, intermediate states are acquired separately for each channel. Perform weighted fusion: Predicting the integral light intensity With intermediate state Multiplication to obtain a high-resolution grayscale image Use bicubic interpolation algorithm to fill high-resolution grayscale images The gaps in the image; using high-resolution grayscale images Alternative spectral images The first principal component was used to obtain a high-resolution spectral image. .

Images