A hyperspectral multisensor fusion method based on generalized spectral imaging

Abstract

The application provides a hyperspectral multisensor fusion method based on generalized spectral imaging, and belongs to the technical field of hyperspectral remote sensing. Visible-near infrared spectrum and mid-infrared spectrum data are combined by using a method combining Markov transition field and Gram angle field, so that one-dimensional spectral data is efficiently converted into two-dimensional spectral images. Then, a Vision-Transformer model is used for deep feature extraction and prediction modeling of the generated two-dimensional spectral images, so that the precision of soil nutrient spectrum prediction and the model explainability are significantly improved, and more reliable technical support is provided for the in-depth implementation of precision agriculture.

Description

Technical Field

[0001] This invention provides a hyperspectral multi-sensor fusion method based on generalized spectral imaging, belonging to the field of hyperspectral remote sensing technology. Background Technology

[0002] Soil, as a fundamental resource for agricultural production, directly impacts food security and ecosystem stability. Soil nutrients, particularly the content of key elements such as organic matter, total nitrogen, phosphorus, and potassium, are core indicators for assessing soil fertility and guiding scientific fertilization. Therefore, rapidly and accurately obtaining soil nutrient content information is crucial for achieving precision agricultural management and improving crop yield and quality.

[0003] However, current acquisition of soil nutrient information mainly relies on traditional laboratory chemical analysis methods. While these methods offer high accuracy, they suffer from inherent drawbacks such as cumbersome sample pretreatment, long testing cycles, high consumption of chemical reagents, and high costs. They are ill-suited to the rapid detection and dynamic monitoring needs of large-scale soil samples within a region and are no longer adequate for the urgent requirements of modern agricultural production for real-time, high-frequency acquisition of soil property information.

[0004] In recent years, with the rapid development of hyperspectral remote sensing technology, utilizing the unique hyperspectral reflectance characteristics of soil combined with machine learning algorithms to invert soil nutrient content has become a hot topic in soil science research. This technology, due to its significant advantages such as speed, non-destructive nature, and environmental friendliness, is widely regarded as an important alternative to traditional chemical determination. The core principle of hyperspectral technology for predicting soil nutrients lies in the fact that nutrient-related components in the soil, such as organic matter, mineral components, and moisture, have different absorption and reflectance characteristics in specific spectral bands. By analyzing this spectral information, nutrient content can be indirectly quantified. Although existing research has effectively improved the prediction accuracy of this technology by combining machine learning and deep learning algorithms, most current studies still use a single sensor (such as visible-near-infrared spectroscopy or mid-infrared spectroscopy) for modeling and analysis. Using only a single sensor has significant limitations: different spectral bands have varying sensitivities to different types of nutrients, and the spectral information captured by a single sensor is limited, making it difficult to comprehensively reflect the complex spectral response characteristics of soil nutrients. In contrast, multi-sensor fusion can integrate complementary information from multiple spectral ranges, effectively compensating for the information gaps and noise interference problems of single sensors.

[0005] In recent years, scholars have attempted to fuse visible-near-infrared and mid-infrared spectra for modeling, significantly improving the accuracy of soil nutrient prediction. However, these studies mostly employ simple data stitching, feature-level, or decision-level fusion schemes, which have significant shortcomings: stitching methods ignore the inherent correlation structure between data from different sensors, feature-level fusion easily loses detailed information from the original spectra, and decision-level fusion struggles to fully utilize the synergistic and complementary information of multi-source data in the early stages, resulting in the failure to fully realize the fusion potential. Therefore, there is an urgent need to develop a novel multi-sensor fusion scheme that can effectively overcome these limitations. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a hyperspectral multi-sensor fusion method based on generalized spectral imaging. The core concept involves efficiently converting one-dimensional spectral data into two-dimensional spectral images by combining visible-near-infrared and mid-infrared spectral data with Markov transfer fields and Gram angle fields. Then, a Vision-Transformer model is used to extract deep features and perform predictive modeling on the generated two-dimensional spectral images, thereby significantly improving the accuracy and interpretability of soil nutrient spectral prediction and providing more reliable technical support for the in-depth implementation of precision agriculture.

[0007] Specifically, the following steps are included:

[0008] A hyperspectral multi-sensor fusion method based on generalized spectral imaging includes the following steps:

[0009] S1. Selection of the study area and sample collection

[0010] The selection of samples for the study area considered factors such as topography, soil parent material, soil type, and climate characteristics. A random sampling method was used to collect soil samples from the study area at a depth of 0-20 cm, with each sampling point located in the center of the farmland and at least 50 meters from the boundary. During the sampling process, the geographic coordinates of each sampling point were accurately recorded using a Global Positioning System (GPS).

[0011] S2. Soil laboratory chemical analysis and spectroscopic processing

[0012] All soil samples underwent standardized processing in the laboratory, were air-dried in a cool, ventilated place, and impurities were removed. After grinding, they were passed through 0.2 mm and 2 mm sieves, respectively. Soil samples passing through the 2 mm sieve were used for spectral acquisition, while those passing through the 0.2 mm sieve were used for chemical analysis. The chemical analysis adopted national standard methods. Soil organic matter was determined by potassium dichromate titration method-external heating method, alkaline available nitrogen was determined by alkaline diffusion method, available phosphorus was determined by sodium bicarbonate extraction-molybdenum antimony colorimetric method, and available potassium was determined by ammonium acetate extraction-flame photometry method.

[0013] Spectral acquisition was conducted in a darkroom environment using a hyperspectral ground cover spectrometer. Air-dried soil samples, sieved through a 2mm sieve, were placed in black petri dishes, and the surface was leveled with a ruler. A 50W halogen lamp was used for illumination with a fixed zenith angle and distance. The spectrometer probe was positioned vertically above the sample. Ten spectral curves were acquired for each sample, and the average value was used as the raw spectral reflectance. Whiteboard calibration was performed periodically. VIS-NIR spectra were measured using an ASD FieldSpec 4 spectrometer. Samples were placed in black sample cups, and the contact probe was used for vertical measurement. Each sample was measured 10 times, and the average value was taken, discarding outliers. MIR spectra were measured using an Agilent 4300 FTIR spectrometer in diffuse reflectance mode. Each sample was scanned 32 times, and the absorbance was derived. Both spectra were preprocessed using Savitzky-Golay smoothing (window 15, order 2) and standard normal transformation.

[0014] S3. By combining Markov transfer field and Gram angle field, one-dimensional spectral data can be efficiently converted into two-dimensional spectral images.

[0015] The dynamic characteristics of state transitions in spectral sequences are captured using MTF, while GAF is used to preserve amplitude correlation and phase structure information, thus generating a two-dimensional image representation.

[0016] The specific steps are as follows:

[0017] First, each spectrum is linearly interpolated and resampled to a uniform length of 24 to align the two types of spectra.

[0018] Then, three 24×24 grayscale images are generated for each resampled spectrum. The Gram Angular Field Summation (GASF) and Differential Field (GADF) modes are encoded for phase and amplitude using polar coordinate transformation and trigonometric sum-difference. The MTF quantizes the spectral values ​​into eight quantile intervals and calculates the transition probabilities to construct a transition field image. Thus, VIS-NIR and MIR each yield three images: GASF, GADF, and MTF.

[0019] In the fusion stage, a cross-sensor averaging strategy is adopted, in which the GASF images of the two spectra are averaged element by element as the RGB red channel, the GADF is averaged as the green channel, and the MTF is averaged as the blue channel, and stacked to obtain a 24×24×3 RGB pseudo-color image.

[0020] After performing the above process on all 694 samples, the images and corresponding soil organic carbon labels were saved as `.npz` files;

[0021] S4. Use the Vision-Transformer model to perform deep feature extraction and prediction modeling on the generated two-dimensional spectral image;

[0022] The 24×24×3 RGB image generated by fusing GAF and MTF was used for targeted parameter design of the Vision-Transformer (ViT) model:

[0023] First, the image is processed through a Patch embedding layer, a two-dimensional convolution with a kernel of 2×2 and a stride of 2, which divides the image into 144 2×2 pixel blocks. Each block is mapped to a 128-dimensional vector, and after flattening and transposing, a token sequence with shape (batch, 144, 128) is obtained.

[0024] A learnable class token is then concatenated to increase the sequence length to 145; a learnable positional encoding (1, 145, 128) is then added, and Dropout (0.1) is performed. The sequence is then fed into four identical Transformer encoder blocks. Each encoder block uses a Pre-LayerNorm structure, containing a multi-head self-attention MHSA and two MLP layers, both with residual connections. The MHSA has an embedding dimension of 128 and 4 attention heads. After normalizing the input layer, Q, K, and V are generated through a linear layer and split into 4 heads. The scaled dot product attention "softmax(Q×K)" is then calculated. T / √32) ×V”, after being stitched together, is mapped back to 128 dimensions through a projection layer and Dropout (0.1) is applied. The MLP branch first increases the dimension to 256 (GELU activation, Dropout 0.1), and then decreases it back to 128 (Dropout 0.1). All Dropout ratios are 0.1.

[0025] After four layers of encoding, the output sequence is normalized in the final layer, and the class token (shape 128) is extracted as a global representation. The SIC prediction value is then output through a single-layer linear regression head (128→1).

[0026] Training was performed using MSELoss, the Adam optimizer, a batch size of 32, and 50 rounds, along with the ReduceLROnPlateau scheduler. After training, the model returned self-attention weights: the average attention of the class token to 144 patch tokens in the last layer was taken, reshaped into a 12×12 grid, and then bilinearly upsampled to 24×24 to generate an attention heatmap.

[0027] Compared with existing technologies, the present invention, through the above-mentioned technical solution, provides a soil nutrient inversion method based on g-generalized spectral imaging and a large-scale visual model. By introducing a "Markov combined with Gram angle field" multi-sensor fusion scheme for soil spectra, it overcomes the technical bottlenecks of traditional fusion schemes, such as low prediction accuracy and weak generalization ability caused by redundant collinearity, inability to model high-order dynamic spectral modes, and fragmented feature layer interactions in decision-level fusion. The present invention achieves the following significant positive effects:

[0028] The two-dimensional spectral imaging fusion scheme maps the one-dimensional sequences of VIS-NIR and MIR to a two-dimensional space, explicitly preserving the phase angle relationship and state transition probability of the spectra. This allows the complementary information (harmonics and fundamental frequencies) of the two types of spectra to form specific textures and spatial structures in the image. Then, the self-attention mechanism of Vision-Transformer (ViT) simultaneously captures the local texture (corresponding to local spectral absorption features) and global dependencies (corresponding to cross-band collaborative changes) in the image, thereby achieving deep collaborative mining of VIS-NIR and MIR information. This fusion method no longer relies on simple stitching or linear combination of feature vectors, but transforms the physical complementarity of the two sensors into a geometric pattern that the model can directly perceive through a two-dimensional spatial structure. Therefore, it achieves prediction accuracy and transfer stability far exceeding other fusion schemes, significantly improving the prediction accuracy of single samples.

[0029] This invention breaks away from traditional multi-sensor fusion modeling approaches such as "decision-level" and "feature-level." Its core technology lies in utilizing the Markov and Gram angle fields described in Section 2.3. It converts two different one-dimensional spectral data into two-dimensional images, better fusing their information and compensating for the shortcomings of different single sensors, thus providing a stable information source for modeling. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the Markov-binding Gram angle field structure of the present invention.

[0031] Figure 2 This is a schematic diagram of the Vision-Transformer structure of the present invention.

[0032] Figure 3 The original VIS-Nir and MIR spectral curves are shown in the examples;

[0033] Figure 4 for Figure 3 The spectral curve is transformed into a two-dimensional spectral image;

[0034] Figure 5 The example shows the feature selection results of VIS-NIR spectral data of soil samples using the CNN-Transformer model;

[0035] Figure 6 The example shows the feature selection results of MIR spectral data of soil samples using the CNN-Transformer model.

[0036] Figure 7 The key feature results extracted by the ViT model from the two-dimensional spectral image generated by the fusion of VIS-NIR and MIR in the example are shown below.

[0037] Figure 8 The image shows a scatter plot of the true and predicted values ​​of the SIC prediction models established by different fusion schemes and single sensors in the embodiments. Detailed Implementation

[0038] The specific technical solutions of the present invention will be described with reference to the embodiments.

[0039] A hyperspectral multi-sensor fusion method based on generalized spectral imaging includes the following steps:

[0040] S1. Selection of the study area and sample collection

[0041] This embodiment focuses on a river basin in the Xinjiang Uygur Autonomous Region, a typical arid oasis irrigated agricultural area primarily cultivating cash crops such as cotton and jujubes, with drip irrigation under mulch film as the main irrigation method. Sample selection considered factors such as topography, soil parent material, soil type, and climate characteristics. A random sampling method was used to collect soil samples from this region at depths of 0-20 cm, with each sampling point located in the center of the farmland and at least 50 meters from the boundary. During the sampling process, a handheld GPS device (Trimble AG25, USA) was used to accurately record the geographic coordinates of each sampling point.

[0042] S2. Soil laboratory chemical analysis and spectral processing

[0043] All soil samples underwent standardized processing in the laboratory, were air-dried in a cool, ventilated place, and impurities were removed. After grinding, the samples were sieved through 0.2mm and 2mm sieves respectively. The 2mm sieve samples were used for spectral acquisition, while the 0.2mm sieve samples were used for chemical analysis. Chemical analysis employed national standard methods: soil organic matter was determined using the potassium dichromate titration method with external heating; available nitrogen was determined using the alkaline diffusion method; available phosphorus was determined using sodium bicarbonate extraction with the molybdenum-antimony colorimetric method; and available potassium was determined using ammonium acetate extraction with a flame photometer. Spectroscopic acquisition was conducted in a darkroom using a hyperspectral ground feature spectrometer. The air-dried soil samples (2mm sieve) were placed in black petri dishes, and the surface was leveled with a ruler. A 50W halogen lamp was used for illumination at a fixed zenith angle and distance. The spectrometer probe was placed vertically above the sample. Ten spectral curves were collected from each sample, and the average value was used as the raw spectral reflectance. Whiteboard calibration was performed periodically. VIS-NIR spectra were measured using an ASD FieldSpec 4 spectrometer. Samples were placed in black sample cups, and measurements were taken vertically using a contact probe. Each sample was measured 10 times, and the mean was taken, with outliers discarded. MIR spectra were measured using an Agilent 4300 FTIR spectrometer in diffuse reflectance mode, with each sample scanned 32 times and the absorbance extracted. Both spectra were preprocessed using Savitzky-Golay smoothing (window 15, order 2) and standard normal transformation.

[0044] S3. By combining Markov transfer field and Gram angle field, one-dimensional spectral data can be efficiently converted into two-dimensional spectral images.

[0045] Markov Transition Field (MTF) is a method for encoding one-dimensional sequences into two-dimensional images, focusing on capturing the dynamic transition features of the sequence. For a sequence X = {x1, x2, …, x} of length n... n First, its numerical range is divided into Q quantile zones, and each data point x is... i Mapped to the corresponding interval number q i ∈ {1,2,…,Q}. Then, construct a Q×Q Markov transition matrix W, where the elements w ij This represents the transition probability of interval j occurring after interval i. To preserve the time order, MTF defines an n×n matrix M, whose elements M... ij It equals the probability of transitioning from the interval at time point i to the interval at time point j, i.e., M. ij = w_{q i q jIn the final generated MTF image, the grayscale value of each pixel reflects the probability of state change between two time steps. Because this method is insensitive to data distribution and computationally efficient, and can visualize local trends, periodicity, and abrupt changes in sequences, it is widely used in fields such as spectral analysis and fault diagnosis to preserve the dynamic evolution information of sequences.

[0046] Gramian Angular Field (GAF) is a sequence image coding method based on polar coordinate transformation, aiming to preserve the temporal correlation and amplitude relationship of the sequence (Dotto et al., 2017; Seema et al., 2020). For a sequence X of length n, it is first normalized to the interval [-1, 1], and then each normalized value is transformed using an inverse cosine function. Mapped to polar angle φ i = arccos( Simultaneously, the timestamp is mapped to a radius r. i = i / n, thus transforming the one-dimensional sequence into a series of points in polar coordinates. GAF further utilizes trigonometric sums and differences to construct the Gram matrix: GASF (Gram summation angle field) is defined as cos(φ) i + φ j ), while GADF (Differential Angle Field) is defined as sin(φ) i - φ j Both matrices are of size n×n, with each element representing a time stamp, and each time stamp is mapped to a radius r. i = i / n, thus transforming the one-dimensional sequence into a series of points in polar coordinates. GAF further utilizes trigonometric sums and differences to construct the Gram matrix: GASF (Gram summation angle field) is defined as cos(φ) i + φ j ), while GADF (Differential Angle Field) is defined as sin(φ) i -φ j Both matrices are n×n in size, and each element represents the cosine sum or sine difference of the angle relationship between any two time points. The diagonal elements retain the absolute value information of the original sequence (cos(2φ)). i (0) or 0), while off-diagonal elements reflect the phase correlation between different time steps. GAF encoding not only restores the amplitude distribution of the sequence but also highlights local temporal correlations, generating images with symmetry, making it suitable as input to convolutional neural networks (Dotto et al., 2017; Seema et al., 2020). Unlike MTF, which focuses on transition probabilities, GAF emphasizes the covariance structure and periodic patterns of the sequence.

[0047] This embodiment combines the advantages of MTF and GAF, utilizing MTF to capture the dynamic state transition characteristics of spectral sequences while leveraging GAF to preserve amplitude correlation and phase structure information. This generates a richer and more complementary two-dimensional image representation, providing input data with both local evolution patterns and global periodic features for subsequent large-scale visual models. Figure 1 This is a schematic diagram of the MTF+GAF structure.

[0048] The specific steps are as follows: First, each spectrum is linearly interpolated and resampled to a uniform length of 24 to align the two types of spectra. Then, three 24×24 grayscale images are generated for each resampled spectrum. The Gram Angular Field Summation (GASF) and Differential Field (GADF) modes are encoded for phase and amplitude using polar coordinate transformation and trigonometric sum-difference. The MTF quantizes the spectral values ​​into eight quantile intervals and calculates the transition probabilities to construct a transition field image. Thus, VIS-NIR and MIR each obtain three images: GASF, GADF, and MTF. In the fusion stage, a cross-sensor averaging strategy is adopted. The GASF images of the two spectra are averaged element-wise as the RGB red channel, the GADF averaged as the green channel, and the MTF averaged as the blue channel, stacked to obtain a 24×24×3 RGB pseudo-color image. After performing the above process on all 694 samples, the images and corresponding soil organic carbon labels are saved as `.npz` files for subsequent Vision-Transformer training.

[0049] S4. Use the Vision-Transformer model to perform deep feature extraction and prediction modeling on the generated two-dimensional spectral image;

[0050] The proposed inversion model is Vision-Transformer (ViT). ViT is a deep learning model that directly applies the Transformer architecture to image tasks. Its core idea is to divide the image into a series of non-overlapping image patches, with each patch linearly projected as an embedding vector. A learnable positional encoding and an additional class token are introduced to form a token sequence, which is then fed into the Transformer encoder. This sequence captures global dependencies directly through a self-attention mechanism (Dotto et al., 2017; Seema et al., 2020). For the 24×24×3 RGB image generated by fusing GAF and MTF in this embodiment, the ViT model has been specifically designed with parameters such as… Figure 2 .

[0051] First, the image passes through a Patch embedding layer, a 2D convolution with a 2×2 kernel and a stride of 2 (3 input channels, 128 output channels), dividing the image into 144 2×2 pixel blocks. Each block is mapped to a 128-dimensional vector, which is flattened and transposed to obtain a token sequence of shape (batch, 144, 128). A learnable class token (128 dimensions) is then concatenated, increasing the sequence length to 145. A learnable positional encoding (1, 145, 128) is then added, and Dropout (0.1) is performed. The sequence is then sequentially fed into four identical Transformer encoder blocks. Each encoder block uses a Pre-LayerNorm structure, containing a multi-head self-attention (MHSA) layer and two MLP layers, all with residual connections. MHSA has an embedding dimension of 128 and 4 attention heads (each head with a dimension of 32). After normalizing the input layer, Q, K, and V are generated through a linear layer and split into 4 heads. The scaled dot product attention "softmax(Q×K)" is then calculated. T / √32) ×V”, after concatenation, is mapped back to 128 dimensions through a projection layer and Dropout (0.1) is applied. The MLP branch first increases the dimension to 256 (GELU activation, Dropout 0.1), then decreases it back to 128 (Dropout 0.1). All Dropout ratios are 0.1. After 4 layers of encoding, the output sequence is normalized in the final layer, and the class token (shape 128) is extracted as a global representation. The SIC prediction value is output through a single-layer linear regression head (128→1). Training uses MSELoss, Adam optimizer (learning rate 1e-4), batch size 32, number of rounds 50, and ReduceLROnPlateau scheduler (factor 0.5, patience 5). After training, the model can return self-attention weights: take the class token pairs from the last layer in 144 patches. The average attention of the tokens (average of 4 heads) is reshaped into a 12×12 grid and then bilinearly upsampled to 24×24 to generate an attention heatmap. The redder the area, the higher the model's attention, thus achieving interpretable feature selection without additional algorithms. This design aims to provide a transparent and efficient deep learning solution for SIC prediction.

[0052] This plan uses a specific soil characteristic as an example:

[0053] Figure 3 and Figure 4The VIS-NIR and MIR spectral curves of soil samples are displayed, along with the fused and encoded two-dimensional spectral images. From the original spectral curves, it can be seen that the overall reflectance in the VIS-NIR band is low, with the single-sample curve showing a trend consistent with the mean curve, exhibiting a slight increase in the band index of approximately 200-400, which is mainly related to the absorption of SOM and iron oxides. The absorbance in the MIR band is stable around 1.0, but the single-sample absorbance is slightly higher in the initial band, indicating that some samples have unique molecular vibrational characteristics in the MIR region. In the RGB two-dimensional spectral images generated by GAF and MTF fusion, samples with different SIC contents show significant differences. The two-dimensional images of low-SIC samples have an overall cool tone (mainly blue-green) and relatively uniform texture, reflecting a smooth spectral sequence change. In contrast, the two-dimensional images of medium- and high-SIC samples show red or yellow patches and increased texture complexity, which is related to the increased state transition probability captured by MTF and the increased phase difference reflected by GASF. Of particular note is that the density of bright pixels in the central region of the image shows a positive correlation with the SIC value, indicating that the “HOT” region in the fused image corresponds to the spectral band range where the characteristic absorption peaks are located. The spatial distribution pattern of these regions can serve as an important visual indicator of SIC content.

[0054] The CNN-Transformer model was used to perform feature filtering on the VIS-NIR and MIR spectral data of soil samples, thereby exploring the core spectral response regions of the soil. The results are as follows: Figure 5 and Figure 6 As shown, in the VIS-NIR band, the model's high attention weights are primarily focused on the short-wave infrared region of 2300-2450 nm. In the MIR band, the desert SIC dominant response region selected by the model is mainly concentrated in the 1420-1450 cm⁻¹ range. -1 and 870-880 cm -1 nearby.

[0055] This embodiment uses Vision-Transformer to extract features of SiC from a two-dimensional spectral image, where the heatmap results based on attention filtering by ViT are as follows: Figure 7As shown, the ViT model exhibits a clear and distinct spatial distribution pattern of key feature regions extracted from the 2D spectral image generated by the fusion of VIS-NIR and MIR. In the heatmap, red to yellow areas indicate higher attention weights assigned by the model when predicting SIC, while blue areas indicate lower weights. The feature heatmap of the training set shows that high attention regions are mainly concentrated in the upper left and lower middle pixels of the image, and the overall distribution is not uniformly diffused, but rather presents two relatively concentrated cluster-like structures. The heatmap of the test set shows a similar overall trend; the high attention regions are also located in the upper left of the image, but the cluster shapes are slightly contracted, indicating that the ViT model learned from the training set is more focused on feature selection during the testing phase, demonstrating good generalization performance.

[0056] Figure 8 Scatter plots of actual and predicted values ​​for SIC prediction models built using different fusion schemes and single sensors are presented. Overall, the predicted and measured values ​​of all models show a positive correlation, with the scatter plots of the actual and predicted values ​​closely aligned around a 1:1 line. The R-values ​​of the six SIC prediction models are shown. 2 The ranges are [0.827-0.935], RMSE is [0.46-0.75], MAE is [0.37-0.6], and RPD is [2.41-3.93]. This result indicates that all six SIC models can capture the correlation between desert soil spectra and SIC well. The four SIC prediction models established using VIS-NIR and MIR fusion data all outperformed the SIC prediction models based on single-sensor data. Compared to single-sensor models, the SIC models established using the spectral fusion scheme have higher R... 2 The improvement in RPD was 0.72% to 13.06%, the improvement in RPD was 1.62% to 1307.88%, the decrease in RMSE was approximately 1.37% to 38.67%, and the decrease in MAE was approximately 1.72% to 38.33%. Secondly, the SIC prediction model based on the two-dimensional spectral imaging fusion scheme performed best among the four fusion schemes. Compared with the SIC models established by the other three fusion schemes, R… 2 The increase was 8.09% to 11.18%, the increase in RPD was 43.96% to 56.57%, the decrease in RMSE was 43.94% to 48.61%, and the decrease in MAE was 15.91% to 35.09%.

Claims

1. A hyperspectral multi-sensor fusion method based on generalized spectral imaging, characterized in that, The process includes the following steps: using a combination of Markov transfer field and Gram angle field to efficiently convert one-dimensional spectral data into two-dimensional spectral images from visible-near-infrared and mid-infrared spectral data; and using the Vision-Transformer model to extract depth features and perform predictive modeling on the generated two-dimensional spectral images.

2. The hyperspectral multi-sensor fusion method based on generalized spectral imaging according to claim 1, characterized in that, Includes the following steps: S1. Selection of the study area and sample collection The selection of samples in the study area took into account factors such as topography, soil parent material, soil type, and climate characteristics. A random sampling method was used to collect soil samples from the study area at a depth of 0-20 cm. Each sampling point was located in the center of the farmland and at least 50 meters away from the boundary. During the sampling process, the geographical coordinates of each sampling point were accurately recorded using the Global Positioning System. S2. Soil laboratory chemical analysis and spectroscopic processing All soil samples underwent standardized processing in the laboratory, were placed in a cool and ventilated place to air dry naturally and remove impurities, and were ground and passed through 0.2 mm and 2 mm sieves respectively. Soil samples that passed through the 2 mm sieve were used for spectral acquisition, and soil samples that passed through the 0.2 mm sieve were used for chemical analysis. Spectral acquisition was conducted in a darkroom environment using a hyperspectral ground object spectrometer. Air-dried soil samples that had passed through a 2mm sieve were placed in black petri dishes and the surface was leveled with a ruler. A 50W halogen lamp was used to fix the zenith angle and distance for illumination. The spectrometer probe was placed vertically above the sample. Ten spectral curves were collected for each sample, and the average value was taken as the original spectral reflectance. Whiteboard calibration was performed periodically. S3. By combining Markov transfer field and Gram angle field, one-dimensional spectral data can be efficiently converted into two-dimensional spectral images. The state transition dynamics of spectral sequences are captured using MTF, while GAF is used to preserve amplitude correlation and phase structure information to generate a two-dimensional image representation. S4. Use the Vision-Transformer model to perform deep feature extraction and prediction modeling on the generated two-dimensional spectral image; The 24×24×3 RGB image generated by fusing GAF and MTF was used for targeted parameter design of the Vision-Transformer (ViT) model; Training was performed using MSELoss, Adam optimizer, batch size 32, number of rounds 50, and ReduceLROnPlateau scheduler. After training, the model returns the self-attention weights: the average attention of the class token to 144 patch tokens in the last layer is taken, reshaped into a 12×12 grid, and then bilinearly upsampled to 24×24 to generate an attention heatmap.

3. The hyperspectral multi-sensor fusion method based on generalized spectral imaging according to claim 2, characterized in that, In S2, chemical analysis was performed using standard methods. Soil organic matter was determined using the potassium dichromate titration method-external heating method, available nitrogen was determined using the alkaline diffusion method, available phosphorus was determined using the sodium bicarbonate extraction-molybdenum antimony colorimetric method, and available potassium was determined using the ammonium acetate extraction-flame photometer method.

4. The hyperspectral multi-sensor fusion method based on generalized spectral imaging according to claim 2, characterized in that, In S2, VIS-NIR spectra were measured using an ASD FieldSpec 4 spectrometer. Samples were placed in black sample cups, and measurements were taken vertically using a contact probe. Each sample was measured 10 times, and the mean was taken, with outliers removed. MIR spectra were measured using an Agilent 4300 FTIR spectrometer in diffuse reflectance mode. Each sample was scanned 32 times, and the absorbance was derived. Both spectra were preprocessed using Savitzky-Golay smoothing (window 15, order 2) and standard normal variable transformation.

5. The hyperspectral multi-sensor fusion method based on generalized spectral imaging according to claim 2, characterized in that, The specific steps for S3 are as follows: First, each spectrum is linearly interpolated and resampled to a uniform length of 24 to align the two types of spectra. Then, three 24×24 grayscale images are generated for each resampled spectrum. The Gram angle field summation mode GASF and the difference mode GADF are encoded by polar coordinate transformation and trigonometric sum-difference encoding of phase and amplitude. The MTF quantizes the spectral values ​​into 8 quantile intervals and calculates the transition probability to construct the transfer field image. Thus, VIS-NIR and MIR each yield three images: GASF, GADF, and MTF. In the fusion stage, a cross-sensor averaging strategy is adopted, in which the GASF images of the two spectra are averaged element by element as the RGB red channel, the GADF is averaged as the green channel, and the MTF is averaged as the blue channel, and stacked to obtain a 24×24×3 RGB pseudo-color image. After performing the above process on all 694 samples, the images and corresponding soil organic carbon labels were saved as `.npz` files.

6. The hyperspectral multi-sensor fusion method based on generalized spectral imaging according to claim 2, characterized in that, The specific parameter design for the ViT model in S4 is as follows: First, the image is processed through a Patch embedding layer, a two-dimensional convolution with a kernel of 2×2 and a stride of 2, which divides the image into 144 2×2 pixel blocks. Each block is mapped to a 128-dimensional vector, and after flattening and transposing, a token sequence with shape (batch, 144, 128) is obtained. Then, a learnable class token is concatenated to increase the sequence length to 145; a learnable positional encoding (1, 145, 128) is added and Dropout (0.1) is performed; the sequence is then fed into four identical Transformer encoder blocks; each encoder block adopts a Pre-LayerNorm structure, containing a multi-head self-attention MHSA and two MLP layers, both with residual connections; the MHSA has an embedding dimension of 128 and 4 attention heads. After normalizing the input layer, Q, K, and V are generated through a linear layer and split into 4 heads, and the scaled dot product attention "softmax(Q×K)" is calculated. T / √32) ×V”, after being stitched together, is mapped back to 128 dimensions through a projection layer and Dropout(0.1) is applied; the MLP branch first increases the dimension to 256 (GELU activation, Dropout 0.1), and then decreases it back to 128 (Dropout 0.1); all Dropout ratios are 0.1; After four layers of encoding, the output sequence is normalized in the final layer, and the class token (shape 128) is extracted as a global representation. The SIC prediction value is then output through a single-layer linear regression head (128→1).

Images