Pushbroom hyperspectral compressive imaging method and system based on line-by-line exposure sensor

Abstract

The application discloses a push-broom hyperspectral compression imaging method and system based on a line-by-line exposure sensor, and comprises the following steps: obtaining a two-dimensional compression observation image through a line-by-line exposure sensor and time sequence control; and recovering hyperspectral data of a target scene from the two-dimensional compression observation image through a reconstruction algorithm to reconstruct complete hyperspectral data. The push-broom hyperspectral compression imaging method based on the line-by-line exposure sensor combines the line-by-line exposure sensor with push-broom motion, and compresses the spatial and spectral dimensions at the physical level. The hyperspectral acquisition with ultrahigh spatial resolution is easily realized at a low cost, and the method is especially suitable for a cost-sensitive spaceborne high-orbit remote sensing load. The compression ratio can be flexibly set by controlling the relationship between the push-broom speed and the frame period. Through the reconstruction algorithm, complete and ultrahigh-resolution hyperspectral cube data are reconstructed from the two-dimensional image acquired by compression.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral compressed imaging technology, specifically to a pushbroom hyperspectral compressed imaging method and system based on a line-by-line exposure sensor. Background Technology

[0002] Traditional pushbroom hyperspectral imaging systems employ area array global exposure sensors. During operation, the system moves the pushbroom, and with each exposure of the area array sensor, it simultaneously acquires one-dimensional spatial information of the target band corresponding to the slit and complete one-dimensional spectral information of each point on that band. Each row of pixels on the sensor corresponds to a spectral band, and each column of pixels corresponds to a point on the spatial band. The system uses continuous global shutter exposures to stitch together the multiple one-dimensional spatial bands acquired to form a complete two-dimensional hyperspectral image.

[0003] However, each pixel of a global shutter sensor requires an independent sample-and-hold circuit, resulting in significantly higher chip design complexity and manufacturing costs compared to rolling shutter sensors for the same pixel size. This limits the feasibility of realizing ultra-high resolution hyperspectral systems at the hardware level.

[0004] Traditional methods strictly adhere to the Nyquist sampling theorem, performing complete, uncompressed, and intensive acquisition of hyperspectral cubic data. This approach generates massive amounts of raw data, placing enormous pressure on onboard and ground-based data transmission, storage, and processing. However, hyperspectral data often contains significant redundancy in both spatial and spectral dimensions, making intensive acquisition not entirely necessary.

[0005] Due to the technical difficulties and cost limitations of global exposure sensors at ultra-high resolutions (such as hundreds of millions of pixels), it has become extremely difficult to further improve the spatial resolution of pushbroom hyperspectral systems. Summary of the Invention

[0006] This invention provides a pushbroom hyperspectral compression imaging method and system based on a line-by-line exposure sensor. It replaces the global exposure sensor with a line-by-line exposure sensor (such as a rolling shutter exposure) and thereby achieves high-resolution hyperspectral image compression acquisition and reconstruction, which is particularly suitable for high-resolution remote sensing.

[0007] This invention provides the following technical solution: a pushbroom hyperspectral compressed imaging method based on a row-by-row exposure sensor, comprising: S1: imaging the target scene onto the sensor plane using a pushbroom hyperspectral system, and obtaining a two-dimensional compressed observation image through a row-by-row exposure sensor and timing control, wherein the row-by-row exposure direction is consistent with the pushbroom direction; S2: recovering the hyperspectral data of the target scene from the two-dimensional compressed observation image using a reconstruction algorithm, wherein the reconstruction algorithm introduces spectral gradient auxiliary interpolation as a correction term into the pushbroom direction interpolation to reconstruct complete hyperspectral data.

[0008] This pushbroom hyperspectral compression imaging method based on a line-by-line exposure sensor combines the sensor with pushbroom motion to compress spatial and spectral dimensions at the physical acquisition level. It achieves ultra-high spatial resolution hyperspectral acquisition at low cost, making it particularly suitable for cost-sensitive spaceborne high-orbit remote sensing payloads. The compression ratio can be flexibly set by controlling the relationship between the pushbroom speed and the frame period. Through a reconstruction algorithm, complete, ultra-high resolution hyperspectral cubic data is reconstructed from the compressed two-dimensional images with high quality.

[0009] As an optional embodiment of the pushbroom hyperspectral compression imaging method based on a line-by-line exposure sensor described in this invention, wherein: when compressing the image, the pushbroom movement distance is controlled to be equal to the spatial sampling interval within the full-spectrum acquisition period. times, and The system performs compressed sampling along the spatial dimension of the push-broom direction to acquire two-dimensional compressed observation data with overlapping spatial and spectral information. In each acquisition cycle, a compressed two-dimensional image is directly generated, transforming the data volume from a traditional hyperspectral cube to a compressed plane, thus significantly reducing the burden of data storage and transmission. The number of line-by-line exposures equals the number of spectral bands required by the system, and... The value is an integer, and is adjusted by adjusting the relationship between the push-broom speed and the sensor frame period. Timing control is achieved through a matrix. Implementation, matrix It is a structured sparse matrix whose elements are determined based on the temporal sequence of row-by-row exposures and the pushbroom geometry, used to describe the spatial location information detected by sensor pixels at a specific moment. At that time, matrix The number of rows is much smaller than the number of columns, thus compressing both the spatial and spectral dimensions of the push-broom direction.

[0010] As an optional scheme of the pushbroom hyperspectral compressed imaging method based on a row-by-row exposure sensor described in this invention, the reconstruction algorithm includes: S2.1: Calculating the spectral gradient: Calculating the first-order partial derivative along the spectral dimension, and obtaining the spectral dimension difference using numerical differencing. S2.2: Block interpolation reconstruction: Determining the spatial block corresponding to each full-spectrum acquisition cycle; S2.3: Interpolation processing: Within the determined spatial block, for the first row of the spatial block: directly copying the original data captured by the sensor; for the middle rows of the spatial block: introducing spectral gradient auxiliary weights and interpolation coefficients to perform linear interpolation and gradient correction; for the last full-spectrum cycle, directly copying the value of the previous cycle.

[0011] As an optional scheme of the pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor described in this invention, the following includes: introducing spectral gradient auxiliary weights and interpolation coefficients for linear interpolation and gradient correction: ,in, For the reconstructed hyperspectral data, These are the interpolation coefficients. The values ​​of the interpolation coefficients depend on the spatial proportion of the current pixel to be calculated relative to the two known measurement points before and after it. For the target scene hyperspectral data cube, This is a two-dimensional compressed observation image. For column coordinates, For row coordinates, The number of periods of the full spectrum collected. For the first One spectral acquisition cycle, For the first Each spectral band For spectral gradient, It serves as an auxiliary weight for the spectral gradient. ,in, This represents the number of rows contained within a space block. The first line within the space range. It is the last row within the space range.

[0012] A system for a pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor includes a processor and a memory, wherein the processor is used to execute the aforementioned pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor.

[0013] The present invention has the following beneficial effects:

[0014] 1. This pushbroom hyperspectral compression imaging method based on a line-by-line exposure sensor can significantly reduce hardware costs and achieve ultra-high resolution: it directly utilizes a mature and lower-cost line-by-line exposure sensor, avoiding the manufacturing difficulties and cost barriers of a global exposure sensor with the same pixel scale. This enables the system to easily achieve ultra-high spatial resolution (such as TDI camera level) hyperspectral acquisition at an acceptable cost, which is especially suitable for cost-sensitive spaceborne high-orbit remote sensing payloads.

[0015] 2. This pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor achieves efficient hardware-level compressed acquisition, significantly reducing data volume. It cleverly combines the characteristics of line-by-line temporal exposure with pushbroom motion, compressing spatial and spectral dimensions at the physical acquisition level. By controlling the relationship between the pushbroom speed and the frame period, the compression ratio can be flexibly set. In each acquisition cycle, a compressed two-dimensional image is directly generated, transforming the data volume from a traditional hyperspectral cube into a compressed plane, thus greatly reducing the burden of data storage and transmission.

[0016] 3. This pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor utilizes computational reconstruction to acquire complete hyperspectral data, avoiding image stitching: A precise mathematical model of the acquisition process is established, and through subsequent computational imaging algorithms, complete, ultra-high-resolution hyperspectral cubic data can be reconstructed from the compressed two-dimensional image with high quality. This method breaks through the physical bottleneck of traditional imaging systems limited by detector sampling modes and sampling frequencies, achieving a leap from hardware-limited sampling to information super-resolution reconstruction, and providing a new technical approach for acquiring high-dimensional spatiotemporal-spectral information beyond physical limits under existing hardware architectures. Attached Figure Description

[0017] Figure 1 This is a flowchart of the pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor according to Embodiment 1 of the present invention.

[0018] Figure 2 This is a schematic diagram of spatiotemporal exposure for compressed acquisition in Embodiment 1 of the present invention.

[0019] Figure 3 This is a schematic diagram of matrix A in Embodiment 1 of the present invention.

[0020] Figure 4 This is the original hyperspectral image of Embodiment 1 of the present invention.

[0021] Figure 5 This is a compressed observation image acquired during the first full-spectrum period in Embodiment 1 of the present invention.

[0022] Figure 6 This is the second compressed observation image acquired during the 51st full-spectrum period in Embodiment 1 of the present invention.

[0023] Figure 7 Image 3 is a compressed observation image acquired during the 101st full-spectrum period in Embodiment 1 of the present invention.

[0024] Figure 8 Image 4 is a compressed observation image acquired during the 150th full-spectrum period in Embodiment 1 of the present invention.

[0025] Figure 9 In Embodiment 1 of the present invention Figure 4 The reconstructed hyperspectral synthesized RGB image.

[0026] Figure 10 In Embodiment 1 of the present invention Figure 4 and Figure 9 The residual plot.

[0027] Figure 11 In Embodiment 1 of the present invention Figure 4 Comparison of average spectral curves of the region of interest before and after reconstruction. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example 1

[0030] Please see Figures 1 to 11 The pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor in this embodiment, such as... Figure 1 As shown, it includes:

[0031] S1: The target scene is imaged onto the sensor plane using a pushbroom hyperspectral system. A two-dimensional compressed observation image is obtained by line-by-line exposure of the sensor and timing control. The line-by-line exposure direction is consistent with the pushbroom direction.

[0032] S2: The hyperspectral data of the target scene is recovered from the two-dimensional compressed observation image by the reconstruction algorithm. The reconstruction algorithm introduces spectral gradient auxiliary interpolation as a correction term into the push-broom direction interpolation to reconstruct the complete hyperspectral data.

[0033] This method is based on the traditional pushbroom hyperspectral system. The existing global exposure sensor is replaced with a line-by-line exposure sensor, where the line-by-line exposure direction is consistent with the pushbroom direction. As with the traditional pushbroom, the image after grating dispersion in one-dimensional space is imaged in the effective area of ​​the sensor. This image is defined as a full-spectrum image. One exposure in the effective area of ​​the sensor acquires one full-spectrum data. This period is one full-spectrum acquisition period.

[0034] In this embodiment, to verify the results of reconstructing complete hyperspectral data, full-spectral acquisition is performed on the test template while the acquisition is compressed. During full-spectral acquisition, when the distance the push-broom moves within one full-spectral acquisition cycle is exactly equal to the spatial sampling interval of the sensor (i.e., the ground resolution corresponding to one row of pixels), the sequential exposure process of each row of the sensor can completely cover each row of the target being measured. At this time, the system completes a lossless and complete acquisition.

[0035] During compressed acquisition, within one full-spectrum acquisition cycle, the distance the push-broom moves is greater than the spatial sampling interval of the sensor, and the sweep... The distance between rows of pixels, In this scenario, only a portion of the spatial rows of the target object are exposed and recorded by the corresponding spectral band pixels at the appropriate time, while the remaining spatial rows are skipped. This achieves compression of the spatial dimension only in the pushbroom direction, and the compression ratio is [value missing]. A single, complete line-by-line exposure cycle of the effective pixel area no longer outputs a complete spectral cube of spatial bands, but rather a two-dimensional image that is superimposed and compressed with information from all spectral bands and different spatial locations.

[0036] Specifically, the compressed acquisition process can be precisely modeled as a linear observation model. Let the target hyperspectral data cube be... Its spatial dimensions are The number of spectral bands is Assuming the system push-broom direction is the direction the spatial column of the spectral cube points, consider the continuous spectrum covered by one full-spectrum acquisition cycle. In the line space region, the number of exposures per line within one cycle is equal to the number of spectral bands. Then all compressed observations obtained from the acquisition of the entire hyperspectral data cube are ,in , This represents the number of full-spectrum periods acquired. A schematic diagram of this compressed acquisition spatiotemporal exposure is shown below. Figure 2 As shown. When considering only one full-spectrum acquisition cycle, each column of pixels corresponds to... The high-dimensional spectral data was acquired in a compressed manner, and compressed observations were obtained by line-by-line exposure. Rearranging hyperspectral data into a one-dimensional vector can be represented as follows: The acquisition process of each column of pixels in one full-spectrum acquisition cycle can be represented as follows: ,in, This is the system transfer matrix, whose elements are determined by the line-by-line exposure timing and pushbroom geometry. Matrix It is a structured sparse matrix, where each row corresponds to the exposure time of a specific column of pixels in a certain spectral band on the sensor, and its element values ​​reflect the spatial location detected by that pixel at that moment. When the compression ratio is greater than 1, the matrix... The number of rows is much smaller than the number of columns, meaning it is an underdetermined system of linear equations, thus enabling compressed acquisition. The matrix's characteristic is that it simultaneously compresses both the spatial and spectral dimensions along the pushbroom direction. Figure 2 When , At that time, the transmission matrix A schematic diagram.

[0037] The core idea of ​​the reconstruction algorithm is that the gradient in the spectral direction reflects the degree of change in the spectral characteristics of ground features, and this change is highly correlated with spatial structure. Introducing the spectral gradient as a correction term into the push-broom interpolation can recover high-frequency spatial details lost due to compression during reconstruction. Consider the hyperspectral cube. Its compressed observation data First, calculate the first-order partial derivatives along the spectral dimension: , The spectral gradient value represents the value at the 1st spectral gradient. The time period, the first Column spatial position, number The change at each band. In a physical sense, it reflects whether the spectral curve at that point is steep or flat. In the first The time period, the first Column spatial position, number Compressed observation data at each band. Indexed by time / row number. It is a spatial column index. This is an index for spectral bands.

[0038] In this embodiment, it is achieved through numerical difference: Then, interpolation of the non-edge full-spectrum acquisition period was performed. , for the One full-spectrum acquisition cycle corresponds to OK Location, start and end positions are: .

[0039] when (First line of block), directly copy the measurement value: .

[0040] when (Within a block) Linear interpolation with gradient correction: , For the reconstructed hyperspectral data, These are the interpolation coefficients. The values ​​of the interpolation coefficients depend on the spatial proportion of the current pixel to be calculated relative to the two known measurement points before and after it. For the target scene hyperspectral data cube, This is a two-dimensional compressed observation image. For column coordinates, For row coordinates, The number of periods of the full spectrum collected. It serves as an auxiliary weight for the spectral gradient.

[0041] in, For the last full-spectrum period There are no subsequent periods to interpolate, so just copy directly: .

[0042] The overall process of the method in this embodiment is as follows: Hyperspectral data cube of the target scene The image is projected onto the sensor plane using a hyperspectral camera, and then transmitted through line-by-line exposure of the sensor and time-series control (matrix). (Function) to obtain two-dimensional compressed observation images Then, the calculation and reconstruction algorithm is used to reconstruct the data from... Recovering the hyperspectral cube .

[0043] Specifically, simulations are performed on the proposed mathematical model and reconstruction algorithm, for a spatial dimension. ( ), spectral dimension 256 ( The hyperspectral cube was compressed to a ratio of 4. Simulation is performed. For example... Figures 4 to 8 This paper presents an RGB image synthesized from three wavelength channels of the original hyperspectral image, as well as compressed observation images acquired during the 1st, 51st, 101st, and 150th full-spectral periods. Reconstructed using the proposed algorithm, the reconstructed peak signal-to-noise ratio (PSNR) is 31.08 dB, and the structural similarity (SSIM) is 0.9911. The comparison results between the original image and the reconstructed image are shown below. Figure 10 As shown. Figure 11 The comparison of the average spectral curves of the region of interest (ROI) before and after reconstruction is shown, with a calculated spectral angle difference of only 0.18°. This result reflects the good performance of the architecture and algorithm.

[0044] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0045] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor, characterized in that, include: S1: The target scene is imaged onto the sensor plane using a pushbroom hyperspectral system. A two-dimensional compressed observation image is obtained by line-by-line exposure of the sensor and timing control. The line-by-line exposure direction is consistent with the pushbroom direction. S2: The hyperspectral data of the target scene is recovered from the two-dimensional compressed observation image by the reconstruction algorithm. The reconstruction algorithm introduces spectral gradient auxiliary interpolation as a correction term into the push-broom direction interpolation to reconstruct the complete hyperspectral data. When compressing images, the push-broom movement distance is controlled to be equal to the spatial sampling interval within the full-spectrum acquisition period. times, and The spatial dimension of the push-broom direction is compressed and sampled to obtain two-dimensional compressed observation data in which spatial and spectral information are mixed. The number of line-by-line exposures equals the number of spectral bands required by the system, and The value is an integer, and is adjusted by adjusting the relationship between the push-broom speed and the sensor frame period. ; Reconstruction algorithms include: S2.1: Calculate the spectral gradient: Calculate the first-order partial derivatives along the spectral dimension to obtain the spectral dimension difference; S2.2: Block interpolation reconstruction: Determine the spatial block corresponding to each full-spectrum acquisition cycle; S2.3: Interpolation processing: Within a defined spatial block, interpolate the non-edge full-spectrum acquisition period; During interpolation processing, For the first row of the space block: directly copy the raw data captured by the sensor; For the middle row of the spatial block: introduce spectral gradient auxiliary weights and interpolation coefficients to perform linear interpolation and gradient correction; For the last full-spectrum period, simply copy the value from the previous period; Introducing spectral gradient-assisted weights and interpolation coefficients for linear interpolation and gradient correction includes: ,in, For the reconstructed hyperspectral data, These are the interpolation coefficients. The values ​​of the interpolation coefficients depend on the spatial proportion of the current pixel to be calculated relative to the two known measurement points before and after it. This is a two-dimensional compressed observation image. For column coordinates, For row coordinates, The number of periods of the full spectrum collected. For the first One spectral acquisition cycle, For the first Each spectral band For spectral gradient, It serves as an auxiliary weight for the spectral gradient.

2. The pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor according to claim 1, characterized in that: Timing control via matrix Implementation, matrix It is a structured sparse matrix whose elements are determined based on the temporal sequence of row-by-row exposures and the pushbroom geometry, used to describe the spatial location information detected by sensor pixels at a specific moment. At that time, matrix The number of rows is much smaller than the number of columns, thus compressing both the spatial and spectral dimensions of the push-broom direction.

3. The pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor according to claim 1, characterized in that: Numerical difference is used to calculate the spectral gradient.

4. The pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor according to claim 1, characterized in that: ,in, This represents the number of rows contained within a space block. The first line within the space range. It is the last row within the space range.

5. A system for pushbroom hyperspectral compressed imaging based on a line-by-line exposure sensor, characterized in that: It includes a processor and a memory, the processor being used to execute the pushbroom hyperspectral compressed imaging method based on a line-by-line exposure sensor as described in any one of claims 1-4.

Images