A Real-time Abundance Estimation Method for Hyperspectral Images Based on Band Fusion Processing Mechanism

Abstract

The present invention discloses a real-time abundance estimation method for hyperspectral images based on a band fusion processing mechanism, including: constructing a distributed hyperspectral band fusion model according to the band sequence data transmission mode of a hyperspectral imaging system; for the hyperspectral image abundance estimation task, designing a hyperspectral real-time fusion strategy for different fusion scenarios according to the distributed hyperspectral band fusion model, where the hyperspectral real-time fusion strategy includes a single-band real-time fusion strategy and a band set and single-band real-time fusion strategy; using a hyperspectral remote sensing sensor to collect data and perform high-quality imaging on ground objects, and obtaining the spectral information and spatial information of the hyperspectral image band by band according to the fusion strategy; designing a real-time band fusion algorithm to perform wavelength fusion on the obtained spectral information and spatial information to obtain a real-time abundance estimation; this method can realize the synchronous operation of data collection, band fusion, and spectral unmixing, realize the real-time processing of abundance estimation, and improve the efficiency of spectral unmixing.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image processing, and particularly to a real-time abundance estimation method for hyperspectral images based on a band fusion processing mechanism. Background Art

[0002] Hyperspectral remote sensing technology has the ability to detect ground objects. However, due to the limited spatial resolution of remote sensors and the complex diversity of natural ground objects, mixed pixels are ubiquitous in remote sensing images, becoming a major obstacle to obtaining information from remote sensing images. In order to improve the accuracy of obtaining ground object information, the spectral decomposition problem must be solved. In a mixed pixel, the area corresponding to one pixel contains two or more characteristic ground objects, and the spectral curve exhibited by one pixel is composed of the superposition of the spectral information of multiple characteristic ground objects. Then, hyperspectral unmixing is a process of decomposing a mixed pixel into endmembers and corresponding abundances.

[0003] Since hyperspectral images contain rich spectral information and a large amount of data, unmixing the spectra often takes a lot of time. Data fusion is an important means to achieve efficient processing of hyperspectral remote sensing images. Data fusion technology refers to the information processing technology that uses a computer to automatically analyze and synthesize several observed information obtained in sequence under certain criteria to complete the required decision-making and evaluation tasks. Currently, the data fusion methods for remote sensing images are mainly divided into three categories: pixel-level fusion, feature-level fusion, and decision-level fusion, and the fusion levels increase from low to high in turn. Pixel-level fusion is a low-level fusion. Although it retains as much information as possible, it has poor anti-interference ability and cannot effectively understand and analyze the image; feature-level fusion is a medium-level fusion, with a relatively long processing time and poor real-time performance; decision-level fusion is the highest-level fusion, and the fusion result provides a basis for command, control, and decision-making. Decision-level fusion has strong fault tolerance and a short processing time, but due to the high requirements for preprocessing and feature extraction, the cost of decision-level fusion is relatively high.

[0004] Under the limited data transmission bandwidth, there is a contradiction between the short hyperspectral remote sensing imaging time, the large amount of image data, the low downlink image efficiency and the user's needs, which seriously restricts the development and application of hyperspectral remote sensing technology. Summary of the Invention

[0005] In order to process a large amount of hyperspectral remote sensing data in real time, the present application discloses a real-time abundance estimation method for hyperspectral images based on a band fusion processing mechanism, which specifically includes the following steps:

[0006] Construct a distributed hyperspectral band fusion model according to the band sequence data transmission mode of the hyperspectral imaging system;

[0007] For the hyperspectral image abundance estimation task, a hyperspectral real-time fusion strategy is designed for different fusion scenarios according to the distributed hyperspectral band fusion model. The hyperspectral real-time fusion strategy includes a single-band real-time fusion strategy and a band set and single-band real-time fusion strategy;

[0008] Use hyperspectral remote sensing sensors to collect data and perform high-quality imaging of ground objects, and obtain the spectral information and spatial information of the hyperspectral image band by band according to the fusion strategy;

[0009] Design a real-time band fusion algorithm to perform short-wave fusion on the obtained spectral information and spatial information to obtain real-time abundance estimation.

[0010] When constructing a distributed hyperspectral band fusion model:

[0011] Construct different fusion frameworks for the original data block and the data block to be fused according to the transmission state of the hyperspectral data;

[0012] Considering the fusion form of the original data block x and the data block y to be fused, the single-band real-time fusion model is constructed as:

[0013] F(Z) = F(x, y) = α y F(x) + β y G(x, y) (1)

[0014] where represents the data group composed of the original data block x and the data block y to be fused, F and G respectively represent the application function and the fusion function, and α y , β y represent the fusion coefficients related to the data block y to be fused;

[0015] Considering the fusion form of the original data group X and the data block y to be fused, the band set and single-band fusion model is constructed as:

[0016] F(Z) = F(X, y) = α y F(X) + β y G(X, y) (2)

[0017] where represents the data group composed of the original data group X and the data block y to be fused.

[0018] When designing the hyperspectral real-time fusion strategy:

[0019] In the initial stage of fusion, considering the fusion strategy between single bands, the abundance estimation operator composed of a single-band image b s and another single-band image b t is expressed as Based on the Woodbury matrix identity and the block matrix theory, the complex matrix operation is vectorized and represented as:

[0020]

[0021]

[0022] where

[0023] Fusing the bands in sequence according to the band sequence data transmission order, considering the fusion strategy between the band set and the single band, the abundance estimation operator composed of the band set B s,m and the single-band image b t is represented as Combined with the Woodbury matrix identity and the block matrix theory, the complex matrix operation is vectorized and represented as:

[0024]

[0025]

[0026] where

[0027] When designing a real-time band fusion algorithm to obtain real-time abundance estimation:

[0028] The change in the abundance estimation value after fusing the k-th band is represented as v k , successively fusing 10 consecutive bands, taking the change amplitude v of the abundance estimation value always being kept within 0.01 as the judgment criterion for abundance stability, that is, v k , v k+1 , …, v k+10 are all less than 0.01, then it is determined that the abundance tends to be stable after fusing k bands, and real-time abundance estimation of the hyperspectral image is achieved.

[0029] Due to the adoption of the above technical solution, a real-time abundance estimation method for hyperspectral images based on a band fusion processing mechanism provided by the present invention. This method is a real-time fusion processing strategy for hyperspectral abundance estimation, and the first band is used as the initial band to perform band processing and fusion in sequence according to the BSQ data acquisition method. When the first band is received, the abundance is immediately estimated using this band, that is, the reception of the second band and the abundance estimation process of the first band run in parallel. When the second band is received, the single-band fusion strategy is immediately used to fuse this band with the estimation result of the existing first band to obtain a fusion result. And so on, subsequent bands are successively received and fused with the existing estimation results using the fusion strategy of the band set and the single-band until all bands are fused to obtain the final estimation result. The impact on the detection value generated by each fused band during the fusion process can be observed in real time, and the corresponding band can be evaluated according to whether this impact is beneficial to the detection. The method provided by the present invention can realize the synchronous operation of data acquisition, band fusion, and spectral unmixing, achieve real-time processing of abundance estimation, and improve the efficiency of spectral unmixing. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0031] Figure 1 It is a schematic flowchart of the method provided by the present invention;

[0032] Figure 2a It is the Samson dataset in the present invention, Figure 2b It is the true abundance map of three endmembers, Figure 2c It is the spectral curves of three endmembers;

[0033] Figures 3a - 3c It is the change curve of the progressive estimated abundance values obtained based on the real-time abundance estimation algorithm at three randomly selected pixel points in the present invention;

[0034] Figures 4a - 4c It is the abundance result map of three endmembers obtained based on the real-time abundance estimation algorithm in the present invention;

[0035] Figure 5 It is the change curve of the time consumed by the band fusion algorithm and the non-band fusion algorithm in the present invention with the continuous fusion of bands. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] To make the technical solutions and advantages of the present invention clearer, the following describes the technical solutions in the embodiments of the present invention clearly and completely in conjunction with the accompanying drawings in the embodiments of the present invention:

[0037] As Figure 1 shown, a real-time abundance estimation method for hyperspectral images based on a band fusion processing mechanism specifically includes the following steps:

[0038] S1: According to the "band sequence" data transmission mode of the hyperspectral imaging system, construct a distributed hyperspectral band fusion model;

[0039] A data group contains multiple data blocks. Different fusion frameworks are constructed for the original data blocks and the data blocks to be fused according to the transmission status of the data. Considering the fusion form of the original data block x and the data block y to be fused, the single-band real-time fusion model is constructed as:

[0040] F(Z) = F(x, y) = α y F(x) + β y G(x, y) (1)

[0041] Wherein, represents the data group composed of the original data block x and the data block y to be fused. F and G respectively represent the application function and the fusion function, and α y , β y represent the fusion coefficients related to the data block y to be fused.

[0042] Considering the fusion form of the original data group X and the data block y to be fused, the band set and the single-band fusion model are constructed as:

[0043] F(Z) = F(X, y) = α y F(X) + β y G(X, y) (2)

[0044] Wherein, represents the data group composed of the original data group X and the data block y to be fused.

[0045] S2: For the hyperspectral image abundance estimation task, design hyperspectral real-time fusion strategies for different fusion scenarios according to the distributed hyperspectral band fusion model: single-band real-time fusion strategy and band set and single-band real-time fusion strategy;

[0046] In the initial stage of fusion, considering the fusion strategy between single bands, the abundance estimation operator composed of a single-band image b s and another single-band image b t is expressed as Combining the Woodbury matrix identity and the block matrix theory, the complex matrix operation is vectorized and expressed as:

[0047]

[0048]

[0049] Among them

[0050] Fuse the bands in sequence according to the "short wavelength sequence" data transmission order, consider the fusion strategy between the band set and the single band, and fuse the band set B s,m and the single band image b t The abundance estimation operator formed is expressed as Combined with the Woodbury matrix identity and the theory of block matrices, the complex matrix operation is vectorized and expressed as:

[0051]

[0052]

[0053] S3: Use the hyperspectral remote sensing sensor to collect data and perform high-quality imaging on the ground object, and obtain the spectral information and spatial information of the hyperspectral image band by band according to the fusion strategy;

[0054] Collect the initial band b1,

[0055] Assume that Ω represents the set containing all L bands, and Ω k represents the set composed of k fused bands, represents the set composed of the bands to be fused;

[0056] S4: Design a real-time band fusion algorithm to perform short wavelength fusion on the obtained spectral information and spatial information to achieve real-time abundance estimation;

[0057] Assume that the change in the abundance estimation value after fusing the kth band is represented by v k , fuse 10 consecutive bands in sequence, and keep the change range v of the abundance estimation value within 0.01 as the judgment standard for abundance stability, that is, if v k , v k+1 , …, v k+10 are all less than 0.01, it is determined that the abundance tends to be stable after fusing k bands, and real-time abundance estimation of the hyperspectral image is achieved.

[0058] Example:

[0059] Next, according to the above method steps, two groups of publicly available real hyperspectral image data sets are used to test and illustrate a real-time abundance estimation method for hyperspectral images provided by the present invention, as well as analyze and evaluate the application effects.

[0060] 1. Dataset and Parameter Setting

[0061] The dataset used in the experiment is the Samson dataset, as shown in the appendix. Figure 2a The image has 952×952 pixels. Each pixel is recorded on 156 channels, covering wavelengths from 401 nm to 889 nm. The spectral resolution is as high as 3.13 nm. Since the original image is too large and the computational cost is high, a 95×95 pixel area is intercepted for the experiment, as shown in Figure 3 of the appendix. The image includes three endmembers, namely "soil", "trees", and "water". The appendix Figure 3b shows the true abundance image of the 3 endmembers, and the appendix Figure 3c shows the reflectance of the three endmembers.

[0062] 2. Experimental Evaluation Metrics

[0063] To compare the efficiency of each algorithm, the time consumed by the algorithm running is counted. To accurately evaluate the application effect of each method, the normalized mean square error (NMSE) and root mean square error (RMSE) are used to objectively analyze the reconstruction performance and abundance estimation effect of various algorithms. Two evaluation criteria are used for performance analysis, summarized as follows:

[0064]

[0065]

[0066] where X is the original data, is the reconstructed data, B represents the true abundance matrix of the ground object, represents the estimated abundance matrix. NMSE and RMSE are used to quantify the reconstruction error and abundance error respectively. The smaller the error value, the more accurate the abundance estimation and spectral reconstruction.

[0067] 3. Analysis and Evaluation of Experimental Results

[0068] The abundance estimation results of a hyperspectral image real-time abundance estimation method based on a band fusion processing mechanism provided by the present invention on a hyperspectral image are shown in Figure 3.

[0069] It can be seen from the figure that after fusing 120 bands, the abundance estimation values of each endmember basically remain unchanged, indicating that the fusion of bands after band 120 has little effect on abundance estimation. Figure 4 of the appendix shows the abundance estimation maps obtained by different algorithms on the Samson data when the estimation reaches the stable state and the final state. It can be observed that the abundance map after fusing 120 bands is very similar to the abundance map after fusing all 156 bands. It is difficult to visually distinguish their differences.

[0070] Tables 1 and 2 analyze and compare in detail the computational complexity of the unmixing algorithm. It is worth noting that the fusion mechanism has a great time advantage in the application of hyperspectral image abundance estimation because it does not involve the inverse operation of matrices. Since traditional abundance estimation algorithms involve a large number of matrix inverse operations, the time complexity is O(n 3 ). When processing hyperspectral images, since the number of bands is large, it inevitably involves the inverse operation of large matrices, which seriously reduces the data processing efficiency of the algorithm. The more bands there are in the hyperspectral image, the slower the data processing speed of the algorithm. The band fusion algorithm avoids the inverse operation of large matrices and fundamentally improves the efficiency of the algorithm. Figure 5 Figure Figure 5 shows the comparison of the computational time of the unmixing algorithm without band fusion and the unmixing algorithm based on band fusion. It can be seen that the time of the unmixing algorithm without band fusion increases rapidly with the increase in the number of bands. The time of the unmixing algorithm based on band fusion remains basically unchanged with the band fusion. Since it does not involve the inverse operation of large matrices, the time consumption of the unmixing algorithm based on band fusion is independent of the number of bands and is not affected by the change in the number of bands.

[0071] Table 1 Analysis of the time complexity of the unmixing algorithm without band fusion

[0072]

[0073] Table 2 Analysis of the time complexity of the unmixing algorithm based on band fusion

[0074]

[0075] A real-time hyperspectral image abundance estimation method based on a band fusion processing mechanism disclosed by the present invention. This method faces a real-time fusion processing strategy for hyperspectral abundance estimation. According to the BSQ data acquisition method, the first band is used as the initial band, and band processing and fusion are carried out in sequence. When the first band is received, the abundance is immediately estimated using this band, that is, the reception of the second band and the abundance estimation process of the first band run in parallel. When the second band is received, the single-band fusion strategy is immediately used to fuse this band with the estimation result of the existing first band to obtain a fusion result. And so on, subsequent bands are successively received and fused with the existing estimation results using the fusion strategy of the band set and the single band until all bands are fused to obtain the final estimation result. During the fusion process, the influence of each fused band on the detection value can be observed in real time, and the corresponding band can be evaluated according to whether this influence is beneficial to detection. The experimental results prove that the method provided by the present invention can realize the synchronous operation of data acquisition, band fusion, and spectral unmixing, realize the real-time processing of abundance estimation, and improve the efficiency of spectral unmixing.

[0076] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its inventive concept, making equivalent substitutions or changes should be covered within the protection scope of the present invention.

Claims

1. A real-time abundance estimation method for hyperspectral images based on a band fusion processing mechanism, characterized in that Including: Construct a distributed hyperspectral band fusion model according to the band sequence data transmission method of the hyperspectral imaging system; For the hyperspectral image abundance estimation task, design a hyperspectral real-time fusion strategy for different fusion scenarios according to the distributed hyperspectral band fusion model, where the hyperspectral real-time fusion strategy includes a single-band real-time fusion strategy and a band set and single-band real-time fusion strategy; Use a hyperspectral remote sensing sensor to collect data and perform high-quality imaging on ground objects, and obtain the spectral information and spatial information of the hyperspectral image band by band according to the fusion strategy; Design a real-time band fusion algorithm to perform short-wave fusion on the obtained spectral information and spatial information to obtain a real-time abundance estimation; When constructing the distributed hyperspectral band fusion model: Construct different fusion frameworks for the original data block and the data block to be fused according to the transmission state of the hyperspectral data; Considering the fusion form of the original data block x and the data block y to be fused, construct a single-band real-time fusion model as: F(Z)=F(x,y)=α y F(x)+β y G(x,y) (1) Among them, represents a data group composed of the original data block x and the data block y to be fused. F and G respectively represent an application function and a fusion function, and α y , β y represent fusion coefficients related to the data block y to be fused; Considering the fusion form of the original data group X and the data block y to be fused, construct a band set and single-band fusion model as: F(Z) = F(X, y) = α y F(X) + β y G(X, y) (2) Among them, represents a data group composed of the original data group X and the data block y to be fused.

2. The real-time abundance estimation method for hyperspectral images based on the band fusion processing mechanism according to claim 1, characterized in that Including: When designing the hyperspectral real-time fusion strategy: In the initial stage of fusion, considering the fusion strategy between single bands, a single-band image b s and another single-band image b t The abundance estimator formed is expressed as Based on the Woodbury matrix identity and the block matrix theory, the complex matrix operation is vectorized and expressed as: Among them Fuse the bands in sequence according to the band sequence data transmission order, consider the fusion strategy between the band set and the single band, and fuse the band set B s,m and the single-band image b t The abundance estimation operator formed is expressed as Combined with the Woodbury matrix identity and the block matrix theory, the complex matrix operation is vectorized and expressed as: Among them 3. The real-time abundance estimation method for hyperspectral images based on a band fusion processing mechanism according to claim 1, wherein Including: When designing a real-time band fusion algorithm to obtain a real-time abundance estimation: Denote the change in the abundance estimate after fusing the k-th band as v k , successively fuse 10 consecutive bands, and keep the change amplitude v of the abundance estimate within 0.01 all the time as the judgment criterion for abundance stability, that is, v k , v k+1 , …, v k+10 are all less than 0.01, then it is determined that the abundance tends to be stable after fusing k bands, and real-time abundance estimation of hyperspectral images is achieved.

Images