Real-time computational imaging method of filter-modulated hyperspectral imaging guided by key band reconstruction

Abstract

This invention discloses a real-time computational hyperspectral imaging method using filtered modulation guided by key band reconstruction. The method comprises: acquiring a hyperspectral scene; processing the hyperspectral scene through liquid crystal filtering modulation to obtain a real-time image of the hyperspectral scene; obtaining an ideal hyperspectral image; and extracting spectral attention from the ideal hyperspectral image through a spectral attention module to obtain attention weights for each band of the ideal hyperspectral image. This invention addresses the technical issues in existing computational spectral imaging, such as compressed sensing imaging, heterogeneous data fusion reconstruction imaging, and spectral filtering modulation imaging, which are unable to switch spectral response functions and thus have difficulty adapting to complex and unknown scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of spectral computational imaging, and in particular to a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction. Background Art

[0002] Compared with traditional RGB images, hyperspectral images can not only preserve the spatial information of the scene, but also preserve rich spectral information of the scene. Data with more dimensions provide better support for downstream tasks. However, it is precisely because of the increase in data dimensionality that the mainstream "push-broom" hyperspectral imaging system has too many exposures, making it difficult to strike a balance between real-time performance and imaging quality, thus limiting its performance in many downstream tasks. For example, in tracking tasks, the huge imaging delay makes it difficult to meet actual tracking needs. At the same time, it is difficult for mainstream hyperspectral imaging systems to switch to the spectral response function that best suits the target scene after implementation, and they have the disadvantage of being difficult to adapt to complex and unknown scenes.

[0003] Computational spectral imaging is a technology that addresses this problem. It refers to digital image capture and processing techniques that use digital computation instead of optical processing. Therefore, computational spectral imaging can generate high-quality spectral images based on spectral scene information obtained from a small number of exposures through image post-processing techniques. Existing computational spectral imaging methods can be mainly divided into three categories: compressed sensing imaging, heterogeneous data fusion reconstruction imaging, and spectral filter modulation imaging.

[0004] Compressed sensing imaging systems suffer from a conflict between temporal and spatial-spectral resolution. Given the limited size of the camera, a single exposure often fails to capture sufficient high-quality spectral information, requiring multiple exposures to meet the imaging requirements of any downstream task. They also lack the ability to switch spectral response functions, making them difficult to adapt to complex and unknown scenes.

[0005] Heterogeneous data fusion reconstruction uses multiple spectral data sets to reconstruct images, often overcoming the low quality of a single image. However, the redundant scene data introduces additional imaging and computational overhead. Furthermore, it lacks the ability to switch spectral response functions, making it difficult to adapt to complex and unknown scenes.

[0006] In spectral filter modulation imaging, liquid crystals can switch their spectral response functions, making them difficult to adapt to complex and unknown scenes. Currently, most liquid crystals are designed based on Lyot filters. This type of liquid crystal can achieve a narrower bandwidth by increasing the number of liquid crystal stages or inserting additional bandpass filters. However, the energy transmittance of light decreases as the bandwidth narrows, resulting in poor image quality and longer image acquisition times. Therefore, the disadvantage of liquid crystal-based devices is the inherent trade-off between spectral resolution and light throughput. Summary of the Invention

[0007] The embodiments of the present invention provide a real-time computational imaging method for filtered modulation hyperspectral imaging guided by key band reconstruction, which at least solves the technical problems in existing computational spectral imaging, such as the inability to switch spectral response functions and the difficulty in adapting to complex and unknown scenes, in compressed sensing imaging, heterogeneous data fusion reconstruction imaging, and spectral filtering modulation imaging.

[0008] According to one aspect of an embodiment of the present invention, a method for real-time computational hyperspectral imaging guided by filter modulation and key band reconstruction is provided. The method may include: acquiring a hyperspectral scene; processing the hyperspectral scene through liquid crystal filter modulation to obtain a real-time image of the hyperspectral scene; acquiring an ideal hyperspectral image; extracting spectral attention from the ideal hyperspectral image through a spectral attention module to obtain attention weights for each band of the ideal hyperspectral image; processing the attention weights for each band of the ideal hyperspectral image through a maximum index to obtain the key bands of the ideal hyperspectral image; and spectrally reconstructing the real-time image of the hyperspectral scene using the key bands of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image.

[0009] Optionally, the hyperspectral scene is processed by liquid crystal filter modulation, and the expression of the hyperspectral scene image after real-time imaging is obtained as follows:

[0010]

[0011] in, represents the spectral response function, represents the imaging network formed by the spectral response function, d represents the thickness of the liquid crystal, λ is the band of the hyperspectral scene image, i ranges from 1 to m, and Δn is the voltage V i A function of Represents the hyperspectral scene image after real-time imaging.

[0012] Optionally, the spectral attention in the ideal hyperspectral image is extracted through the spectral attention module, and the expression of the attention weight of each band of the ideal hyperspectral image is obtained as follows:

[0013] S A =avgpooling(S θ (x ～ ))

[0014] Among them, S A represents the attention weights of each band of the ideal hyperspectral image, avg pooling represents average pooling, S θ represents the spectral attention module, x ～ represents an ideal hyperspectral image.

[0015] Optionally, after extracting the spectral attention in the ideal hyperspectral image through the spectral attention module and obtaining the attention weights of each band of the ideal hyperspectral image, the method further includes: multiplying the spectral attention in the ideal hyperspectral image with each band of the ideal hyperspectral image to obtain a weighted hyperspectral image; inputting the weighted hyperspectral image into the classification network to obtain a predicted value of the category of the weighted hyperspectral image; calculating based on the predicted value of the category of the weighted hyperspectral image and the true value of the category in the weighted hyperspectral image to obtain a first classification loss; determining the regularization loss of the attention weight of each band based on the attention weights of each band of the ideal hyperspectral image; and obtaining the selection loss of each band based on the first classification loss and the regularization loss.

[0016] Optionally, the attention weights of each band of the ideal hyperspectral image are processed by the maximum index, and the expression of the key band of the ideal hyperspectral image is obtained as follows:

[0017] R BS =topk(S A )

[0018] Among them, R BS represents the key band of the ideal hyperspectral image, and topk represents the maximum index.

[0019] Optionally, the spectral reconstruction of the hyperspectral scene image after real-time imaging is performed using the key bands of the ideal hyperspectral image, and the expression of the reconstructed hyperspectral scene image is obtained as follows:

[0020]

[0021] in, represents the reconstructed hyperspectral scene image, F R represents the spectral reconstruction network.

[0022] Optionally, after spectrally reconstructing the hyperspectral scene image after real-time imaging through the key bands of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image, the method further includes: inputting the reconstructed hyperspectral scene image into a classification network to obtain a predicted value of the category in the reconstructed hyperspectral scene image; calculating based on the predicted value of the category in the reconstructed hyperspectral scene image and the true value of the category in the reconstructed hyperspectral scene image to obtain a second classification loss; determining a reconstruction loss based on the image corresponding to the key bands of the ideal hyperspectral image and the reconstructed hyperspectral scene image; and determining a joint loss based on the second classification loss and the reconstruction loss.

[0023] Beneficial effects of the present invention:

[0024] The present invention proposes a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction, which includes three modules: a task-oriented band selection module, a spectral filter-modulated imaging module and a spectral reconstruction module.

[0025] The present invention improves the imaging quality of the system and ensures real-time imaging through a small amount of filtered modulation exposure and spectral reconstruction post-processing; uses a spectral filtering modulation imaging mechanism based on liquid crystal to meet the need to switch spectral response functions when adapting to complex and unknown scenes; at the same time, before spectral reconstruction, task-oriented selection of scene bands is performed as the target band for reconstruction, removing redundant bands while accelerating the reconstruction speed and improving real-time imaging; spectral reconstruction and filtering modulation are jointly trained to obtain a small amount of spectral response functions that are most conducive to reconstruction, solving the technical problems of compressed sensing imaging in computational spectral imaging, heterogeneous data fusion reconstruction imaging, and spectral filtering modulation imaging that cannot switch spectral response functions and are difficult to adapt to complex and unknown scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 is a flow chart of a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention;

[0028] Figure 2 4 is a structural block diagram of a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0031] Example 1

[0032] According to an embodiment of the present invention, a method for real-time computational imaging of filtered-modulated hyperspectral data guided by key band reconstruction is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 1 is a flow chart of a filter modulation hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0034] Step S101: Acquire a hyperspectral scene.

[0035] In the technical solution provided in the above step S101 of the present invention, Figure 2 is a structural block diagram of a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention, such as Figure 2 As shown, a hyperspectral scene is obtained, and the hyperspectral scene is light.

[0036] Step S102 : Processing the hyperspectral scene through liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging.

[0037] In the technical solution provided in step S102 of the present invention, for a hyperspectral scene x, in order to facilitate simulation, a hyperspectral image x is used. ～ As an ideal scenario of x, for the hyperspectral image x, a band reconstruction guided liquid crystal filter modulation is performed. The liquid crystal LCTF can be adjusted according to the input voltage V i Adjusting the spectral response function The relationship between the band λ, voltage V and the thickness d of the liquid crystal is:

[0038]

[0039] Liquid crystal filter modulation can form an imaging network with a small number of spectral response functions Perform real-time imaging to obtain a small number of spectral images Where m is the small number of wavelengths modulated by the liquid crystal filter, and h and w are the length and width of the spatial dimensions.

[0040]

[0041] Step S103: Acquire an ideal hyperspectral image.

[0042] In the technical solution provided in step S103 of the present invention, Figure 2 As shown, the ideal hyperspectral image is obtained. The ideal hyperspectral image is the image corresponding to the hyperspectral scene. The ideal hyperspectral image x ～ Contains three dimensions x ～ ∈R c×h×w , where c is the spectral dimension, and h and w are the length and width of the spatial dimension, which contains N classes and Q sample pixels of each class. Each pixel of the image in the spatial dimension has a label y∈R h×w , whose value is

[0043] Step S104 : extracting the spectral attention in the ideal hyperspectral image through the spectral attention module to obtain the attention weights of each band of the ideal hyperspectral image.

[0044] In the technical solution provided by the above step S104 of the present invention, the ideal hyperspectral image is first input into the spectral attention module S θ Extract spectral attention from , where θ is the parameter of the feature extractor, and get the attention weight S of each band A ∈R c×1 , whose value is S A ∈{x∈R|0≤x≤1};

[0045] S θ =Sigmoid(Linear(Relu(Linear(x ～ ))))

[0046] S A =avgpooling(S θ (x ～ ))

[0047] In step S105 , the attention weights of the respective bands of the ideal hyperspectral image are processed by the maximum index to obtain the key bands of the ideal hyperspectral image.

[0048] In the technical solution provided in step S105 of the present invention, S A After topk, we get the first n bands R with the largest weight BS As a mission-oriented reconstruction band key objectives.

[0049] Step S106 , performing spectral reconstruction on the hyperspectral scene image after real-time imaging using the key bands of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image.

[0050] In the technical solution provided in step S106 of the present invention, the R BS As the target band, Input to the spectral reconstruction network F R Spectral reconstruction is performed to obtain multispectral images with higher spectral resolution and higher quality Where n is a small number of bands modulated by the liquid crystal filter, and n>>m:

[0051]

[0052] The above method of this embodiment is further introduced below.

[0053] As an optional embodiment, in step S102, the hyperspectral scene is processed by liquid crystal filter modulation, and the expression of the hyperspectral scene image after real-time imaging is obtained:

[0054]

[0055] in, represents the spectral response function, represents the imaging network formed by the spectral response function, d represents the thickness of the liquid crystal, λ is the band of the hyperspectral scene image, i ranges from 1 to m, and Δn is the voltage V i A function of Represents the hyperspectral scene image after real-time imaging.

[0056] In this embodiment, the hyperspectral scene is imaged and processed by liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging, wherein the number of hyperspectral scene images will be the same as the number of voltages of the liquid crystal filter modulation.

[0057] As an optional embodiment, in step S104, the spectral attention in the ideal hyperspectral image is extracted by the spectral attention module, and the expression of the attention weight of each band of the ideal hyperspectral image is obtained as follows:

[0058] S A =avg pooling(Sθ (x ～ ))

[0059] Among them, S A represents the attention weights of each band of the ideal hyperspectral image, avg pooling represents average pooling, S θ represents the spectral attention module, x ～ represents an ideal hyperspectral image.

[0060] In this embodiment, the spectral attention module performs spectral attention extraction on the ideal hyperspectral image to obtain the attention weights of each band of the ideal hyperspectral image.

[0061] As an optional embodiment, in step S104, after extracting the spectral attention in the ideal hyperspectral image through the spectral attention module and obtaining the attention weights of each band of the ideal hyperspectral image, the method further includes: multiplying the spectral attention in the ideal hyperspectral image by each band of the ideal hyperspectral image to obtain a weighted hyperspectral image; inputting the weighted hyperspectral image into the classification network to obtain a predicted value of the category of the weighted hyperspectral image; calculating based on the predicted value of the category of the weighted hyperspectral image and the true value of the category in the weighted hyperspectral image to obtain a first classification loss; determining the regularization loss of the attention weight of each band based on the attention weights of each band of the ideal hyperspectral image; and obtaining the selection loss of each band based on the first classification loss and the regularization loss.

[0062] In this embodiment, S A Multiply each band by dimension to obtain a weighted hyperspectral image Each band is used to obtain a weighted hyperspectral image Input to the classification network F C Classify and get the predicted value Using hyperspectral classification loss L c To learn the optimal band attention weight for classification. At the same time, in order to achieve the selection effect, the band selection loss L s The regularization loss L of the band weight is added r .

[0063]

[0064]

[0065] L r =∥S A ∥1

[0066] L s =L c +L r

[0067] in, Represents the classification network F C Weighted hyperspectral image to classify, is the predicted value, y j Weighted hyperspectral image The true label.

[0068] As an optional embodiment, in step S105, the attention weights of each band of the ideal hyperspectral image are processed by the maximum index, and the expression of the key band of the ideal hyperspectral image is obtained as follows:

[0069] R BS =topk(S A )

[0070] Among them, R BS represents the key band of the ideal hyperspectral image, and topk represents the maximum index.

[0071] In this embodiment, S A After topk, we get the first n bands R with the largest weight BS As a mission-oriented reconstruction band key objectives.

[0072] As an optional embodiment, in step S106, the spectral reconstruction of the hyperspectral scene image after real-time imaging is performed using the key bands of the ideal hyperspectral image, and the expression of the reconstructed hyperspectral scene image is obtained as follows:

[0073]

[0074] in, represents the reconstructed hyperspectral scene image, F R represents the spectral reconstruction network.

[0075] In this embodiment, the spectral reconstruction of the hyperspectral scene image after real-time imaging is performed using the key bands of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image.

[0076] As an optional embodiment, in step S106, after spectrally reconstructing the hyperspectral scene image after real-time imaging through the key bands of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image, the method further includes: inputting the reconstructed hyperspectral scene image into a classification network to obtain a predicted value of the category in the reconstructed hyperspectral scene image; calculating based on the predicted value of the category in the reconstructed hyperspectral scene image and the true value of the category in the reconstructed hyperspectral scene image to obtain a second classification loss; determining a reconstruction loss based on the image corresponding to the key bands of the ideal hyperspectral image and the reconstructed hyperspectral scene image; and determining a joint loss based on the second classification loss and the reconstruction loss.

[0077] In this embodiment, Input classification network F C Classify and get the predicted value of the category in the reconstructed hyperspectral scene image:

[0078]

[0079] L′ S =L′ C +L′ R

[0080] in, R BS is the x of the band ～ , L′ C is the second classification loss, L′ R is the reconstruction loss, L′ S For joint losses, The image corresponding to the key band of the ideal hyperspectral image.

[0081] Experimental part:

[0082] 1. Data preprocessing

[0083] For a hyperspectral target scene x, in order to facilitate simulation, the hyperspectral image x is used. ～ As x ideal scene. For the hyperspectral image x ～ , band reconstruction guided by liquid crystal filter modulation. Hyperspectral image x ～ Contains three dimensions x ～ ∈R c×h×w , where c is the spectral dimension, and h and w are the length and width of the spatial dimension, which contains N classes and Q sample pixels of each class. Each pixel of the image in the spatial dimension has a label y∈R h×w , whose value is

[0084] In addition, the min-max normalization data preprocessing method is used for the training input images to eliminate the influence of data dimension and make the data indicators comparable.

[0085] 2. Model structure and initialization

[0086] The present invention adopts an imaging network, a band selection network, a spectrum reconstruction network and a classification network based on liquid crystal.

[0087]

[0088] R BS =topk(avgpooling(Sigmoid(Linear(Relu(Linear(x ～ )))))))

[0089]

[0090] F C =Unet(x ～ )

[0091] Among them, SST is a spectral transformation reconstructor, which consists of multiple spectral attention modules embedded in a U-shaped network; Unet is a U-shaped classification network, which is divided into a downsampling encoder and an upsampling decoder. The former transmits information to the latter in multiple layers.

[0092] 3. Model Pre-training

[0093] Since this imaging system hopes to verify its effectiveness in practical applications, it can also be decoupled from specific applications in practical applications. Therefore, the liquid crystal imaging network, band selection network, and spectral reconstruction network are pre-trained first. Pre-training can be task-related or task-independent. If the task is relevant, a task-related loss is added during band selection, such as the first classification loss L c .

[0094] After obtaining the key reconstruction target band R BS Finally, in order to better reconstruct the target band, the liquid crystal imaging network and the spectral reconstruction network are jointly pre-trained to obtain the liquid crystal spectral response function that is most conducive to spectral reconstruction, thereby obtaining the optimal voltage applied to the liquid crystal.

[0095] 4. Training process

[0096] The model obtained from this training has task-independent generalization capabilities but lacks task-specific capabilities. Based on the above model, we fine-tune the model by adding task-specific losses. During fine-tuning, we freeze the body of the reconstruction network and fine-tune only the first and last conv layers and the liquid crystal imaging network.

[0097] In an embodiment of the present invention, a hyperspectral scene is acquired; the hyperspectral scene is processed by liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging; an ideal hyperspectral image is acquired; the spectral attention in the ideal hyperspectral image is extracted by a spectral attention module to obtain the attention weights of each band of the ideal hyperspectral image; the attention weights of each band of the ideal hyperspectral image are processed by a maximum index to obtain the key bands of the ideal hyperspectral image; the hyperspectral scene image after real-time imaging is spectrally reconstructed by the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image, thereby solving the technical problems in the existing computational spectral imaging, such as the inability to switch the spectral response function and the difficulty in adapting to complex and unknown scenes, and achieving the technical effect of improving the imaging quality and real-time imaging of the system by a task-oriented band selection module, a spectral filter modulation imaging module and a spectral reconstruction module.

[0098] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0099] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0101] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0102] In addition, the functional units in various embodiments of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0103] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction, characterized in that: include: Acquire hyperspectral scenes; The hyperspectral scene is processed by liquid crystal filter modulation to obtain the hyperspectral scene image after real-time imaging, and its expression is: in, represents the spectral response function, represents the imaging network formed by the spectral response function, Indicates the thickness of the liquid crystal, is the band of the hyperspectral scene image, The value range is , For voltage A function of Represents a hyperspectral scene image after real-time imaging; Acquire ideal hyperspectral images; The spectral attention module is used to extract the spectral attention in the ideal hyperspectral image, and the attention weights of each band of the ideal hyperspectral image are obtained, which are expressed as follows: in, represents the attention weights of each band of the ideal hyperspectral image, represents average pooling, represents the spectral attention module, represents an ideal hyperspectral image; The attention weights of each band of the ideal hyperspectral image are processed by the maximum index to obtain the key bands of the ideal hyperspectral image, which is expressed as: in, represents the key bands of an ideal hyperspectral image, Indicates the maximum index; The spectral reconstruction of the hyperspectral scene image after real-time imaging is performed through the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image.

2. The method according to claim 1, characterized in that After extracting the spectral attention in the ideal hyperspectral image by the spectral attention module and obtaining the attention weights of each band of the ideal hyperspectral image, the method further includes: The spectral attention in the ideal hyperspectral image is multiplied by each band of the ideal hyperspectral image to obtain a weighted hyperspectral image; Input the weighted hyperspectral image into the classification network to obtain the predicted value of the category of the weighted hyperspectral image; The first classification loss is calculated based on the predicted value of the category of the weighted hyperspectral image and the true value of the category in the weighted hyperspectral image; Based on the attention weights of each band of the ideal hyperspectral image, the regularization loss of the attention weights of each band is determined; Based on the first classification loss and regularization loss, the selection loss of each band is obtained.

3. The method according to claim 1, characterized in that The spectral reconstruction of the hyperspectral scene image after real-time imaging is performed through the key bands of the ideal hyperspectral image. The expression of the reconstructed hyperspectral scene image is: in, represents the reconstructed hyperspectral scene image, represents the spectral reconstruction network.

4. The method according to claim 1, wherein After performing spectral reconstruction on the hyperspectral scene image after real-time imaging using the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image, the method further includes: The reconstructed hyperspectral scene image is input into the classification network to obtain the predicted value of the category in the reconstructed hyperspectral scene image; The second classification loss is calculated based on the predicted value of the category in the reconstructed hyperspectral scene image and the true value of the category in the reconstructed hyperspectral scene image; Determine the reconstruction loss based on the image corresponding to the key band of the ideal hyperspectral image and the reconstructed hyperspectral scene image; Based on the second classification loss and the reconstruction loss, a joint loss is determined.

5. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the method according to any one of claims 1 to 4 when running.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 4.

Images