A hyperspectral snapshot compressive imaging reconstruction and classification integrated method and system

Abstract

The application discloses a hyperspectral snapshot compressive imaging reconstruction and classification integrated method and system, which comprises a training stage and a test stage: in the training stage, a compressed measurement map is generated by using a hyperspectral image and coded aperture mask information, and a training sample composed of the compressed measurement map and a corresponding multi-label vector is constructed; the compressed measurement map is input into an end-to-end network composed of a shared feature encoder, an image reconstruction branch and a multi-scale fusion classification branch, and a joint loss function with dynamic weights is used to learn network parameters; in the test stage, the compressed measurement map collected by a snapshot compressive imaging system is input into the trained network, and a multi-label ground object classification result is directly output. Compared with a serial scheme of first reconstruction and then classification, the application reduces repeated calculation and parameter size, reduces inference delay, improves remote sensing interpretation efficiency, and is suitable for fast ground object identification scenes.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral intelligent interpretation and compressed imaging processing technology, specifically relating to an integrated method and system for reconstruction and classification of hyperspectral snapshot compressed imaging. Background Technology

[0002] Hyperspectral remote sensing imagery contains both detailed spectral and spatial texture information, making it invaluable for applications such as land cover mapping, ecological environment monitoring, disaster assessment, and refined urban governance. However, hyperspectral imagery typically features a large number of channels, large data volume, and long acquisition and transmission links. Especially in spaceborne / airborne scenarios, limitations in imaging computing power, satellite-to-ground transmission bandwidth, and storage resources often make direct and complete acquisition and download of hyperspectral data costly, hindering the ability to meet the demands of large-scale, high-frequency, real-time or near-real-time applications.

[0003] To reduce sampling volume at the imaging end and alleviate transmission and storage pressures, the industry typically employs compressed imaging techniques to acquire and encode high-dimensional hyperspectral data. Hyperspectral snapshot compressed imaging is a typical approach, which modulates and encodes the incident light field in both spatial and spectral dimensions by introducing modulation elements such as encoding masks into the imaging optical path, and then obtains compressed observation data at the sensor end with a single or few exposures. This method can acquire hyperspectral information with a low data volume in scenarios with limited hardware resources, bandwidth constraints, or high real-time requirements. It offers advantages such as reduced sampling overhead, smaller data download size, and improved imaging efficiency, and therefore has gained widespread attention in spaceborne / airborne and edge deployment scenarios.

[0004] In existing engineering implementations, a common approach for the "compressed measurement - task interpretation" link is a sequential process: first, a reconstruction network or algorithm is used to restore the compressed measurement results to hyperspectral images, and then the reconstruction results are input into a classification network to complete multi-label recognition. Although this sequential approach has a clear implementation path, it has significant limitations in practical applications: on the one hand, the reconstruction module usually focuses on pixel-level or spectral consistency indicators as the main optimization goal, while the classification module focuses more on semantic discriminative features. The two goals are inconsistent, which can easily lead to situations where "the reconstruction seems clearer but the classification is not improved, or the discriminative information required for classification is smoothed / weakened in the reconstruction." On the other hand, the sequential structure brings two-stage inference and repeated feature extraction, resulting in high computational overhead and high latency, which is not conducive to deployment on resource-constrained platforms. At the same time, if there are errors or artifacts in the reconstruction stage, these errors will be passed as input to the classification stage, forming an error cascade, which leads to a decrease in classification stability.

[0005] Furthermore, hyperspectral remote sensing multi-label tasks generally exhibit characteristics such as long-tailed category distribution, coexistence of multiple scales within the same scene, and complex ground feature boundaries. Traditional sequential or single-task approaches often struggle to balance "hyperspectral reconstruction quality" and "multi-label semantic discrimination performance" within a unified framework. Moreover, if fixed weights are used for loss balancing during multi-task training, one task may converge too quickly while another is suppressed, leading to performance fluctuations in the later stages of training or ultimately, performance limitations.

[0006] Therefore, there is an urgent need for a unified method that can simultaneously achieve reconstruction and multi-label classification of hyperspectral images under coded aperture compression measurement input, enabling reconstruction and classification to be synergistically optimized based on shared representations, and adaptively balancing the optimization intensity of different tasks during training, thereby improving overall accuracy, stability and engineering deployability. Summary of the Invention

[0007] The purpose of this invention is to provide a method, system, device, and storage medium for joint reconstruction and multi-label classification of hyperspectral remote sensing images, in order to solve the problems of insufficient universality and practicality caused by the fragmentation of reconstruction and classification targets under compressed measurement conditions, insufficient multi-scale semantic representation, and difficulty in adaptively adjusting multi-task weights in the prior art.

[0008] To achieve the above objectives, the present invention employs the following technical solution: An integrated reconstruction and classification method for hyperspectral snapshot compressed imaging includes: acquiring multi-channel remote sensing image data to be processed and corresponding coded aperture mask information; performing mask modulation and measurement modeling on the image to generate compressed measurement input; feeding the compressed measurement input into a dual-task deep network with a shared encoder, and extracting hierarchical and multi-scale representation features through the shared encoder; decoding the features in the reconstruction branch to output the reconstructed multi-channel hyperspectral image; performing multi-scale fusion and aggregation on features from different scales in the classification branch to output multi-label classification results; constructing a joint optimization objective including reconstruction loss and classification loss, and introducing a dynamic weight strategy to adaptively adjust the loss weights of the reconstruction task and the classification task during training.

[0009] Furthermore, the compressed measurement input Y satisfies the following measurement model:

[0010] Where C is the number of channels. For Hadamard element-wise multiplication, For spatial displacement operators related to channel index, This is the inverse displacement recovery operator.

[0011] Furthermore, the multi-scale aggregation of the classification branches obtains aggregated features by performing channel projection, scale alignment, and fusion on features of different scales. It satisfies:

[0012] in, For channel projection operators, For the upsampling operator, To splice operators, This is the fusion function.

[0013] Furthermore, the joint loss function satisfies:

[0014] in, For classifying losses, To rebuild the losses, The reconstructed weights are dynamically changed with training iterations or rounds t.

[0015] Furthermore, the dynamic weighting strategy can adopt the Dynamic Weighted Average (DWA) method, and its update satisfies:

[0016] Where T is the temperature coefficient and K is the scaling factor.

[0017] Furthermore, the dynamic weighting strategy can also employ an uncertainty modeling approach, and its joint loss satisfies:

[0018] in, and These are learnable parameters used to characterize the uncertainties in classification and reconstruction tasks.

[0019] Furthermore, the classification loss supports class imbalance handling and can employ either weighted binary cross-entropy loss or asymmetric focus loss; when using asymmetric focus loss, the following conditions are met:

[0020] in, For tags, To predict probabilities, For the Sigmoid function, and This is the focusing factor for positive and negative samples.

[0021] Furthermore, the reconstruction loss Includes at least one or a combination of Charbonnier loss, spectral angle loss (SAM), and total variational loss (TV), wherein:

[0022]

[0023]

[0024] Where N is the number of samples or pixels involved in the calculation; and These represent the reconstructed value and the actual value, respectively. It is the smoothing stability constant; and These represent the reconstructed spectral vector and the true spectral vector at the pixel, respectively. Indicates the inner product. Let δ denote the L2 norm, and δ be the stability constant in the denominator; Represents the reconstructed pixel value at spatial coordinates, where i, j are spatial indices. It represents the absolute value.

[0025] A system for an integrated reconstruction and classification method of hyperspectral snapshot compressed imaging includes a measurement generation module, a feature extraction module, a reconstruction module, a classification module, and a training optimization module; The output of the measurement generation module is connected to the input of the feature extraction module. The output of the feature extraction module is connected to the input of the reconstruction module and the input of the classification module, respectively. The outputs of the reconstruction module and the classification module are connected to the training and optimization module, respectively. The output of the training and optimization module is connected to the feature extraction module, the reconstruction module, and the classification module, respectively. The measurement generation module is used to acquire multi-channel remote sensing image data and coded aperture masks, and generate compressed measurement input based on mask modulation and displacement superposition. The feature extraction module is used to input compressed measurement inputs into a shared encoder and output a multi-scale feature set. The reconstruction module is used to receive multi-scale feature sets and output reconstructed images. The classification module receives multi-scale feature sets, performs multi-scale fusion and aggregation on them, and outputs multi-label classification results. The training optimization module receives reconstructed images, multi-label classification results, and ground truth values ​​of multi-channel remote sensing images and multi-labels corresponding to compressed measurement inputs. It constructs a joint loss function that includes reconstruction loss and classification loss, and adopts a dynamic weighting strategy to adaptively adjust the weights between reconstruction loss and classification loss to update the network parameters of the shared encoder, reconstruction module, and classification module.

[0026] Compared with the prior art, the present invention has the following beneficial effects: This invention unifies reconstruction and classification within a shared encoder framework under compressed measurement input conditions. It enhances the stability of remote sensing multi-label classification through multi-scale semantic fusion; employs a dynamic weighting strategy to adaptively balance the multi-task training process, reducing the burden of manual parameter tuning; and the reconstruction branch outputs interpretable multi-channel reconstruction results, facilitating quality checks and reuse in downstream tasks. Therefore, this invention can improve interpretation robustness and engineering deployability in complex terrain scenarios while reducing data transmission and storage costs. Attached Figure Description

[0027] To facilitate the explanation of the technical solutions of the present invention, the accompanying drawings used in the embodiments are briefly described below. It should be understood that the drawings are only used to illustrate the basic concept and main process of the present invention and are not drawn to scale; those skilled in the art can obtain drawings of other embodiments based on these drawings without creative effort, and all such drawings should fall within the protection scope of the present invention.

[0028] Figure 1 This is a schematic diagram of the overall process of the method for joint reconstruction and multi-label classification of hyperspectral images under the coded aperture compression measurement conditions of the present invention.

[0029] Figure 2 This is a schematic diagram of the overall structure of the end-to-end deep network of the present invention.

[0030] Figure 3 This is a schematic diagram of the multi-scale fusion and aggregation structure of the classification branches of this invention.

[0031] Figure 4 This is a system structure block diagram corresponding to the present invention.

[0032] Figure 5 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0033] The exemplary embodiments of this application are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered as illustrative descriptions only. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the scope of protection of this application.

[0034] It should be noted that the term "and / or" in this document is used only to describe the relationship between related objects, indicating that multiple parallel scenarios may exist; the " / " in this document generally indicates that the related objects before and after are in an "or" relationship. The terminology used in this document is for the purpose of describing specific embodiments only and is not intended to limit this application.

[0035] The implementation steps of this invention are as follows: (1) Constructing simulation data and compressed measurement input for training: Obtain the multi-channel hyperspectral remote sensing image data X to be processed and its corresponding multi-label annotation information, and obtain the coded aperture mask information M that matches the imaging system. Based on the modulation coding mechanism of hyperspectral snapshot compressed imaging, the hyperspectral image data X is subjected to mask modulation and channel shift superposition to generate compressed measurement input Y for network training and inference. During the training phase, the compressed measurement input Y and the original hyperspectral image data X together constitute a reconstruction supervision pair, and together with the multi-label annotation, constitute a classification supervision pair, thereby obtaining the data alignment samples required for end-to-end joint training.

[0036] (2) Construct a dual-task deep network with a shared encoder and output two types of results: Construct a dual-task deep network, which includes at least a shared encoder, a reconstruction branch and a classification branch: The shared encoder is used to extract features from the compressed measurement input Y to obtain a hierarchical, multi-scale feature set. The reconstruction branch is used to reconstruct the multi-scale feature set. Decode and reconstruct the image, and output the reconstructed multi-channel hyperspectral image. The classification branch is used to align, fuse, and aggregate features at different scales, outputting multi-label classification results. The multi-label prediction results are obtained through activation functions.

[0037] (3) Constructing a joint training objective and introducing dynamic weights to achieve adaptive optimization: Construct a total loss function for joint training so that the reconstruction task and the classification task can be optimized collaboratively under the same network framework. During the training process, a dynamic weight strategy is introduced to adaptively adjust the relative weights of the reconstruction loss and the classification loss, so that the network can automatically balance the optimization emphasis of "reconstruction consistency" and "semantic discriminability" at different training stages, thereby improving the overall performance of reconstruction quality and multi-label classification performance as well as training stability.

[0038] The present invention will now be described in further detail with reference to the accompanying drawings. It should be noted that the drawings are merely illustrative of the technical solutions of the present invention and are not intended to limit specific structures or parameters; identical or similar reference numerals in the drawings denote identical or similar components or steps. Those skilled in the art can make corresponding modifications or equivalent substitutions to the present invention without departing from its spirit and scope of protection, and all such modifications or substitutions should fall within the scope of protection of the present invention.

[0039] like Figure 1As shown, this invention provides a joint reconstruction and multi-label classification method for hyperspectral remote sensing images, which revolves around the chain of "compressed measurement input—shared representation—dual-task output—joint training and optimization". Specifically, it acquires hyperspectral image data X and an coded aperture mask M, and generates a compressed measurement input Y by superimposing mask modulation and channel shift. In one embodiment, the compressed measurement process can be characterized by the following measurement model:

[0040] in, This represents the two-dimensional image of the c-th spectral channel. This represents the mask corresponding to the channel. For element-wise multiplication, This represents the spatial displacement operator associated with the channel index. This represents the inverse displacement recovery operator, where C is the number of channels. Using this method, the measurement domain input can be obtained while reducing sampling and transmission overhead, providing a unified entry point for subsequent recovery and semantic interpretation.

[0041] like Figure 2 As shown, the compressed measurement input Y enters an end-to-end deep network, which employs a shared encoder and a dual-branch structure: the shared encoder extracts multi-scale features, the reconstruction branch outputs the reconstructed image, and the classification branch outputs multi-label prediction results. This process can be abstracted as follows:

[0042] in, For shared encoders, To rebuild the branch, For classification branches, Let z represent the feature sets at different scales / levels, z be the logits of the classification output, and p be the multi-label probability vector. This is the Sigmoid function. Compared to the traditional sequential process of "reconstruction first, classification later", this shared structure allows reconstruction and classification to share the same representation basis, which can reduce redundant calculations and reduce the risk of error cascading caused by reconstruction errors propagating downstream.

[0043] like Figure 3 As shown, the classification branch adopts a multi-scale fusion and aggregation structure, utilizing multi-scale features from the shared encoder to form a discriminative representation that better fits the semantics of remote sensing. The fusion process can be represented in one implementation as follows:

[0044] in, Indicates the channel projection / alignment operator (e.g.) (convolution or equivalent transformation) This indicates a scale alignment operator (e.g., upsampling). This indicates a splicing operation. This represents a fusion transformation (e.g., convolution / normalization / nonlinear combination). By fusing shallow details with deep semantic information, it can better adapt to the characteristics of hyperspectral remote sensing scenes, such as "multiple categories coexisting in the same image, large differences in ground object scale, and complex boundaries and textures," thereby improving the stability of multi-label prediction.

[0045] like Figure 1 , Figure 2 As shown, this invention constructs a joint optimization objective during the training phase, enabling the reconstruction and classification tasks to be trained collaboratively within the same framework. Furthermore, a dynamic weighting mechanism adaptively balances the optimization intensity of different tasks during training. In one embodiment, the joint loss can be expressed as:

[0046] in, For multi-label classification loss, To rebuild the losses, These are reconstructed weights that dynamically change with the training progress t. This is to reduce training instability or task-to-task "pulling" effects caused by fixed weights. Updates can be made using a dynamic weighting strategy. In one optional implementation, a dynamic weighted averaging method can be used, and its update form can be written as:

[0047] Where T is the temperature coefficient and K is the scaling factor. This strategy allows for a greater focus on stable convergence in the early stages of training, while automatically adjusting the emphasis based on the difference in convergence speed between the two tasks in the later stages, thereby improving overall performance and the controllability of the training process.

[0048] like Figure 4 As shown, this invention also provides a system architecture corresponding to the method described above. This system includes at least: a compressed measurement generation module, a feature extraction module (shared encoder), a reconstruction module, a classification module, and a training optimization module. The compressed measurement generation module generates compressed measurement input based on the input image and mask; the feature extraction module outputs multi-scale features; the reconstruction module outputs reconstructed images; the classification module outputs multi-label predictions; and the training optimization module constructs joint loss, updates dynamic weights, and optimizes network parameters. This system can be deployed on a server or edge device, and the training and inference modes can be selected based on computing power and link conditions.

[0049] like Figure 5As shown, the present invention also provides an electronic device, including a processor, a memory, and a communication interface. The memory stores a computer program, and when the processor executes the computer program, it implements the above-described method steps. Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method steps.

[0050] Based on this, the present invention will be further described in detail below through specific embodiments: Example 1: Training and Inference Implementation of a Dual-Task Network with Shared Encoder (for BigEarthNet-v2 Data) See Figures 1-5 This embodiment uses hyperspectral remote sensing data as an example to construct a compressed measurement input and train a dual-task network with a shared encoder to achieve joint inference for reconstruction and multi-label classification. The data side can use multi-channel image patches in the BigEarthNet-v2 style, along with accompanying metadata files for multi-label annotation and reading. To facilitate engineering reproduction, the image patch spatial size can be uniformly cropped to a fixed size (e.g., 120×120), and the number of channels can be selected according to system settings (e.g., C=10). Mask data can come from pre-generated or calibrated two-dimensional coded aperture mask files, and is expanded into a three-dimensional mask with the same number of channels before participating in measurement generation.

[0051] During training, the compressed measurement input Y is used as the network input tensor, which is then fed into the shared encoder to extract hierarchical features. The reconstruction branch outputs multi-channel reconstructed images. The system is used for quality inspection and visualization; the classification branch fuses and aggregates multi-scale features, outputs multi-label predicted logits, and obtains the probability of each class through the Sigmoid function. To reduce the cost of manual parameter tuning, this embodiment enables a dynamic weight strategy during the training phase, allowing the weights of reconstruction and classification to be automatically adjusted at different training stages. The classification side can further enable class imbalance handling (e.g., pos_weight caching, or using asymmetric focus loss) to improve recall for long-tail classes.

[0052] Table 1 shows the experimental hardware, software, and operating environment.

[0053]

[0054] Table 2 shows the training hyperparameter settings.

[0055]

[0056] Example 2: Comparative Experiment and Ablation Validation (Effectiveness of Multi-Scale Fusion and Dynamic Weights) See Figure 3 and Figure 5This embodiment compares single-task, cascaded joint, and shared encoder dual-task schemes, and conducts ablation experiments on multi-scale fusion and dynamic weight strategies. Comparison objects may include: reconstruction-only networks (optimizing only reconstruction loss), classification-only networks (directly classifying input or reconstructed output), cascaded "reconstruction-after-classification" networks, and the shared encoder dual-task network of this invention. For evaluation metrics, PSNR, SSIM, etc., can be used on the reconstruction side; multi-label metrics such as mAP, F1, and Hamming loss can be used on the classification side. To avoid random interference, the comparative experiments should use the same training / test partitioning, the same number of rounds, or the same early stopping strategy, and a fixed random seed.

[0057] Table 3 compares the overall performance of different schemes.

[0058]

[0059] Example 3: System Implementation and Deployment (Training, Testing, and Result Archiving) See Figure 4 and Figure 5 This embodiment describes the implementation at the system and device levels. The system may include a data module, a measurement generation module, a shared encoder dual-task network module, an evaluation and visualization module, and a result archiving module. During the training phase, the system records the reconstruction and classification metrics for each training round, performs tests at fixed intervals, and saves the corresponding model files and test history to facilitate tracking the optimal model when performance declines in the later stages of training. During the inference phase, the system simultaneously outputs reconstructed images and classification labels, and can visualize key samples and write them into the report directory to form deliverable evaluation materials.

[0060] In terms of engineering deployment, single-machine single-card or multi-card distributed training can be adopted; on the inference side, edge or cloud deployment can be selected according to the link conditions. For multi-scenario expansion, the mask, number of channels, category set and loss strategy can be adjusted through configuration files, reducing code modifications and improving maintenance efficiency.

[0061] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Those skilled in the art can modify or make equivalent substitutions to the specific embodiments of the present invention without departing from the spirit and scope of the invention, and all such modifications and substitutions should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for integrated reconstruction and classification using hyperspectral snapshot compressed imaging, characterized in that, include: The system acquires multi-channel remote sensing image data to be processed and corresponding coded aperture mask information, performs mask modulation and measurement modeling on the images to generate compressed measurement input; feeds the compressed measurement input into a dual-task deep network with a shared encoder, and extracts hierarchical and multi-scale representation features through the shared encoder; decodes the features in the reconstruction branch and outputs the reconstructed multi-channel hyperspectral image; performs multi-scale fusion and aggregation on features from different scales in the classification branch and outputs multi-label classification results; constructs a joint optimization objective that includes reconstruction loss and classification loss, and introduces a dynamic weight strategy to adaptively adjust the loss weights of the reconstruction task and the classification task during training.

2. The method according to claim 1, characterized in that, Generating compressed measurement input includes: expanding a two-dimensional mask into a three-dimensional mask with the same number of channels as the multi-channel remote sensing image data, and performing bitwise modulation, shift stacking, and inverse shift restoration on each channel to obtain the compressed measurement input, which satisfies the following: Where C is the number of channels. For Hadamard element-wise multiplication, For spatial displacement operators related to channel index, This is the inverse displacement recovery operator.

3. The method according to claim 1, characterized in that, The shared encoder is a multi-stage downsampling structure used to extract feature sets at least two different scales from compressed measurement input. ; The reconstruction branch is a decoding structure used to output the reconstructed image based on the feature set; the classification branch is a multi-scale aggregation structure that fuses the feature set and outputs multi-label classification results, and the input of the classification branch is not limited to using only the last layer features of the shared encoder.

4. The method according to claim 3, characterized in that, Multi-scale aggregation of classification branches includes fusing features from different scales after channel projection and scale alignment to obtain aggregated features. The classification output is generated from the aggregated features, wherein the aggregated features satisfy: in, For channel projection operators, For the upsampling operator, To splice operators, For fusion transformation operators.

5. The method according to claim 1, characterized in that, The classification loss can be either a weighted binary cross-entropy loss or an asymmetric focus loss for multi-label tasks; when using the asymmetric focus loss, the loss term for a single sample and single class satisfies: in, For tags, To predict probabilities, For the Sigmoid function, and This is the focusing factor for positive and negative samples.

6. The method according to claim 1, characterized in that, A joint loss function comprising reconstruction loss and classification loss is constructed, and a dynamic weighting strategy is used to determine the reconstruction weights. The joint loss function satisfies: in, For classifying losses, The reconstruction loss is t, where t is the number of training iterations or rounds. The reconstructed weights are dynamically changed with t and updated by the dynamic weighting strategy.

7. The method according to claim 6, characterized in that, The dynamic weighting strategy includes a weight update method based on dynamic weight averaging, satisfying: Where T is the temperature coefficient and K is the scaling factor. and These are the reconstruction loss and classification loss for round t, respectively.

8. The method according to claim 6, characterized in that, The dynamic weighting strategy includes a weight update method based on uncertainty modeling, which ensures that the joint loss function satisfies: in, These are learnable parameters used to characterize the uncertainties in classification and reconstruction tasks.

9. The method according to claim 1, characterized in that, Reconstruction losses Includes at least one or a combination of Charbonnier loss, spectral angle loss (SAM), and total variational loss (TV), wherein: Where N is the number of samples or pixels involved in the calculation; and These represent the reconstructed value and the actual value, respectively. It is the smoothing stability constant; and These represent the reconstructed spectral vector and the true spectral vector at the pixel, respectively. Indicates the inner product. Let δ denote the L2 norm, and δ be the stability constant in the denominator; Represents the reconstructed pixel value at spatial coordinates, where i, j are spatial indices. It represents the absolute value.

10. A system for implementing the integrated reconstruction and classification method of hyperspectral snapshot compressed imaging according to any one of claims 1 to 9, characterized in that, It includes a measurement generation module, a feature extraction module, a reconstruction module, a classification module, and a training optimization module; The output of the measurement generation module is connected to the input of the feature extraction module. The output of the feature extraction module is connected to the input of the reconstruction module and the input of the classification module, respectively. The outputs of the reconstruction module and the classification module are connected to the training and optimization module, respectively. The output of the training and optimization module is connected to the feature extraction module, the reconstruction module, and the classification module, respectively. The measurement generation module is used to acquire multi-channel remote sensing image data and coded aperture masks, and generate compressed measurement input based on mask modulation and displacement superposition. The feature extraction module is used to input compressed measurement inputs into a shared encoder and output a multi-scale feature set; The reconstruction module is used to receive multi-scale feature sets and output reconstructed images; The classification module receives multi-scale feature sets, performs multi-scale fusion and aggregation on them, and outputs multi-label classification results. The training optimization module receives reconstructed images, multi-label classification results, and ground truth values ​​of multi-channel remote sensing images and multi-labels corresponding to compressed measurement inputs. It constructs a joint loss function that includes reconstruction loss and classification loss, and adopts a dynamic weighting strategy to adaptively adjust the weights between reconstruction loss and classification loss to update the network parameters of the shared encoder, reconstruction module, and classification module.

Images