Key wave band reconstruction guided filtering modulation hyperspectral real-time calculation imaging method

Abstract

The invention discloses a filtering modulation hyperspectral real-time calculation imaging method guided by key wave band reconstruction. The method comprises the following steps: acquiring a hyperspectral scene; processing the hyperspectral scene through liquid crystal filtering modulation to obtain a hyperspectral scene image after real-time imaging; obtaining an ideal hyperspectral image; spectral attention in the ideal hyperspectral image is extracted through a spectral attention module, and the attention weight of each wave band of the ideal hyperspectral image is obtained. The technical problems that spectral response functions cannot be switched in compressed sensing imaging, heterogenous data fusion reconstruction imaging and spectral filtering modulation imaging in existing spectral imaging calculation, and complex unknown scenes are difficult to adapt are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of spectral computational imaging, and in particular to a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction. Background Art

[0002] Compared with traditional RGB images, hyperspectral images can not only preserve the spatial information of the scene, but also preserve rich spectral information of the scene. Data with more dimensions provide better support for downstream tasks. However, it is precisely because of the increase in data dimensionality that the mainstream "push-broom" hyperspectral imaging system has too many exposure times, making it difficult to balance real-time performance and imaging quality, thus limiting its performance in many downstream tasks. For example, in tracking tasks, the huge imaging delay makes it difficult to meet actual tracking needs. At the same time, it is difficult for mainstream hyperspectral imaging systems to switch the spectral response function that best suits the target scene after implementation, and they have the disadvantage of being difficult to adapt to complex unknown scenes.

[0003] Computational spectral imaging is a technology that solves this problem. Computational spectral imaging refers to digital image capture and processing technology that uses digital computing instead of optical processing. Therefore, computational spectral imaging can obtain high-quality spectral images based on spectral scene information obtained from a small amount of exposure through image post-processing technology. Existing computational spectral imaging methods are mainly divided into three categories: compressed sensing imaging, heterogeneous data fusion reconstruction imaging, and spectral filter modulation imaging.

[0004] Compressed sensing imaging systems have a conflict between temporal and spatial spectral resolution. Given the limited size of the camera, a single exposure often fails to capture enough high-quality spectral information, so multiple exposures are usually required to meet the imaging requirements of any downstream task. It cannot switch spectral response functions and is difficult to adapt to complex unknown scenes.

[0005] Heterogeneous data fusion reconstruction imaging uses multiple spectral data for imaging, which can often overcome the problem of low imaging quality of a single image. However, the redundant scene data brings additional imaging and computing overhead. At the same time, it cannot switch the spectral response function and is difficult to adapt to complex unknown scenes.

[0006] In spectral filter modulation imaging, liquid crystals are able to switch spectral response functions, which makes it difficult to adapt to complex unknown scenes. Currently, most liquid crystals are designed based on Lyot filters. This type of liquid crystal can obtain a narrower bandwidth by increasing the number of liquid crystals or inserting additional bandpass filters. However, the energy transmittance of light decreases as the bandwidth becomes narrower, resulting in poor image quality and longer image acquisition time. Therefore, the disadvantage of liquid crystal-based devices is the inherent trade-off between spectral resolution and light throughput. Summary of the invention

[0007] The embodiment of the present invention provides a filtered modulated hyperspectral real-time computational imaging method guided by key band reconstruction, which at least solves the technical problems in the existing computational spectral imaging, namely, the inability to switch the spectral response function and the difficulty in adapting to complex unknown scenes in compressed sensing imaging, heterogeneous data fusion reconstruction imaging and spectral filtering modulation imaging.

[0008] According to one aspect of an embodiment of the present invention, a filter modulation hyperspectral real-time computational imaging method guided by key band reconstruction is provided. The method may include: obtaining a hyperspectral scene; processing the hyperspectral scene by liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging; obtaining an ideal hyperspectral image; extracting the spectral attention in the ideal hyperspectral image by a spectral attention module to obtain the attention weights of each band of the ideal hyperspectral image; processing the attention weights of each band of the ideal hyperspectral image by a maximum index to obtain the key band of the ideal hyperspectral image; spectrally reconstructing the hyperspectral scene image after real-time imaging by the key band of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image.

[0009] Optionally, the hyperspectral scene is processed by liquid crystal filter modulation, and the expression of the hyperspectral scene image after real-time imaging is obtained as follows:

[0010]

[0011] in, represents the spectral response function, represents the imaging network formed by the spectral response function, d represents the thickness of the liquid crystal, λ is the band of the hyperspectral scene image, i ranges from 1 to m, and Δn is the voltage V i A function of Represents a hyperspectral scene image after real-time imaging.

[0012] Optionally, the spectral attention in the ideal hyperspectral image is extracted through the spectral attention module, and the expression of the attention weight of each band of the ideal hyperspectral image is obtained as follows:

[0013] S A =avgpooling(S θ (x ～ ))

[0014] Among them, S A represents the attention weights of each band of the ideal hyperspectral image, avg pooling represents average pooling, S θ represents the spectral attention module, x ～ Represents an ideal hyperspectral image.

[0015] Optionally, after extracting the spectral attention in the ideal hyperspectral image through the spectral attention module and obtaining the attention weights of each band of the ideal hyperspectral image, the method also includes: multiplying the spectral attention in the ideal hyperspectral image with each band of the ideal hyperspectral image to obtain a weighted hyperspectral image; inputting the weighted hyperspectral image into the classification network to obtain a predicted value of the category of the weighted hyperspectral image; calculating based on the predicted value of the category of the weighted hyperspectral image and the true value of the category in the weighted hyperspectral image to obtain a first classification loss; based on the attention weights of each band of the ideal hyperspectral image, determining the regularization loss of the attention weights of each band; and obtaining the selection loss of each band based on the first classification loss and the regularization loss.

[0016] Optionally, the attention weights of each band of the ideal hyperspectral image are processed by the maximum index, and the expression of the key band of the ideal hyperspectral image is obtained as follows:

[0017] R BS =topk(S A )

[0018] Among them, R BS It represents the key band of the ideal hyperspectral image, and topk represents the maximum index.

[0019] Optionally, the spectral reconstruction of the hyperspectral scene image after real-time imaging is performed through the key bands of the ideal hyperspectral image, and the expression of the reconstructed hyperspectral scene image is obtained as follows:

[0020]

[0021] in, represents the reconstructed hyperspectral scene image, F R represents the spectral reconstruction network.

[0022] Optionally, after spectrally reconstructing the hyperspectral scene image after real-time imaging through the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image, the method also includes: inputting the reconstructed hyperspectral scene image into a classification network to obtain a predicted value of a category in the reconstructed hyperspectral scene image; calculating based on the predicted value of the category in the reconstructed hyperspectral scene image and the true value of the category in the reconstructed hyperspectral scene image to obtain a second classification loss; determining a reconstruction loss based on an image corresponding to the key bands of the ideal hyperspectral image and the reconstructed hyperspectral scene image; and determining a joint loss based on the second classification loss and the reconstruction loss.

[0023] Beneficial effects of the present invention:

[0024] The invention proposes a filter modulation hyperspectral real-time computational imaging method guided by key band reconstruction, which comprises three modules: a task-oriented band selection module, a spectral filter modulation imaging module and a spectral reconstruction module.

[0025] The present invention improves the imaging quality of the system and ensures the real-time imaging through a small amount of filtered modulation exposure and spectral reconstruction post-processing; uses a spectral filtering modulation imaging mechanism based on liquid crystal to meet the need to switch the spectral response function when adapting to complex unknown scenes; at the same time, before spectral reconstruction, the scene band is selected in a task-oriented manner as the target band for reconstruction, and the redundant bands are removed while the reconstruction speed is accelerated to improve the real-time imaging; the spectral reconstruction and filtering modulation are jointly trained to obtain a small amount of spectral response functions that are most conducive to reconstruction, which solves the technical problems of compressed sensing imaging in computational spectral imaging, heterogeneous data fusion reconstruction imaging and spectral filtering modulation imaging that cannot switch the spectral response function and are difficult to adapt to complex unknown scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 is a flow chart of a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention;

[0028] Figure 2 It is a structural block diagram of a filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Example 1

[0032] According to an embodiment of the present invention, a filtered modulated hyperspectral real-time computational imaging method guided by key band reconstruction is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0033] Figure 1 is a flow chart of a filter modulation hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0034] Step S101, obtaining a hyperspectral scene.

[0035] In the technical solution provided in the above step S101 of the present invention, Figure 2 is a structural block diagram of a filter modulation hyperspectral real-time computational imaging method guided by key band reconstruction according to an embodiment of the present invention, such as Figure 2 As shown, a hyperspectral scene is obtained, and the hyperspectral scene is light.

[0036] Step S102, processing the hyperspectral scene through liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging.

[0037] In the technical solution provided in step S102 of the present invention, for a hyperspectral scene x, in order to facilitate simulation, a hyperspectral image x is used. ～ As an ideal scenario for x, for the hyperspectral image x, a band reconstruction-guided liquid crystal filter modulation is performed. The liquid crystal LCTF can be adjusted according to the input voltage V i Adjusting the spectral response function The relationship between the band λ, voltage V and the thickness d of the liquid crystal is:

[0038]

[0039] Liquid crystal filter modulation can form an imaging network with a small number of spectral response functions Real-time imaging to obtain a small number of spectral images Where m is the small number of bands modulated by the liquid crystal filter, and h and w are the length and width of the spatial dimensions.

[0040]

[0041] Step S103, obtaining an ideal hyperspectral image.

[0042] In the technical solution provided in step S103 of the present invention, Figure 2 As shown, an ideal hyperspectral image is obtained. The ideal hyperspectral image is the image corresponding to the hyperspectral scene. The ideal hyperspectral image x ～ Contains three dimensions x ～ ∈R c×h×w , where c is the spectral dimension, and h and w are the length and width of the spatial dimension, which contains N classes and Q sample pixels of each class. Each pixel of the image in the spatial dimension has a label y∈R h×w , whose value is

[0043] Step S104: extracting the spectral attention in the ideal hyperspectral image through the spectral attention module to obtain the attention weights of each band of the ideal hyperspectral image.

[0044] In the technical solution provided in step S104 of the present invention, the ideal hyperspectral image is first input into the spectral attention module S θ Extract spectral attention from , where θ is the parameter of the feature extractor, and get the attention weight S of each band A ∈R c×1 , whose value is S A ∈{x∈R|0≤x≤1};

[0045] S θ =Sigmoid(Linear(Relu(Linear(x ～ ))))

[0046] S A =avgpooling(S θ (x ～ ))

[0047] Step S105 , processing the attention weights of each band of the ideal hyperspectral image through the maximum index to obtain the key bands of the ideal hyperspectral image.

[0048] In the technical solution provided in step S105 of the present invention, S A After topk, we get the first n bands R with the largest weight. BS As a mission-oriented reconstruction band key objectives.

[0049] Step S106 , spectrally reconstructing the hyperspectral scene image after real-time imaging by using the key bands of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image.

[0050] In the technical solution provided in step S106 of the present invention, the R in step S105 is used BS As the target band, Input to the spectral reconstruction network F R Spectral reconstruction is performed to obtain multispectral images with higher spectral resolution and higher quality Where n is a small number of bands modulated by the liquid crystal filter, and n>>m:

[0051]

[0052] The above method of this embodiment is further introduced below.

[0053] As an optional embodiment, in step S102, the hyperspectral scene is processed by liquid crystal filter modulation, and the expression of the hyperspectral scene image after real-time imaging is obtained as follows:

[0054]

[0055] in, represents the spectral response function, represents the imaging network formed by the spectral response function, d represents the thickness of the liquid crystal, λ is the band of the hyperspectral scene image, i ranges from 1 to m, and Δn is the voltage V i A function of Represents a hyperspectral scene image after real-time imaging.

[0056] In this embodiment, the hyperspectral scene is imaged by liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging, wherein there will be several hyperspectral scene images if there are several voltages of the liquid crystal filter modulation.

[0057] As an optional implementation manner, in step S104, the spectral attention in the ideal hyperspectral image is extracted by the spectral attention module, and the expression of the attention weight of each band of the ideal hyperspectral image is obtained as follows:

[0058] S A =avg pooling(Sθ (x ～ ))

[0059] Among them, S A represents the attention weights of each band of the ideal hyperspectral image, avg pooling represents average pooling, S θ represents the spectral attention module, x ～ Represents an ideal hyperspectral image.

[0060] In this embodiment, the spectral attention module performs spectral attention extraction on the ideal hyperspectral image to obtain the attention weights of each band of the ideal hyperspectral image.

[0061] As an optional implementation mode, step S104, after extracting the spectral attention in the ideal hyperspectral image through the spectral attention module and obtaining the attention weights of each band of the ideal hyperspectral image, the method also includes: multiplying the spectral attention in the ideal hyperspectral image with each band of the ideal hyperspectral image to obtain a weighted hyperspectral image; inputting the weighted hyperspectral image into the classification network to obtain a predicted value of the category of the weighted hyperspectral image; calculating based on the predicted value of the category of the weighted hyperspectral image and the true value of the category in the weighted hyperspectral image to obtain a first classification loss; based on the attention weights of each band of the ideal hyperspectral image, determining the regularization loss of the attention weights of each band; based on the first classification loss and the regularization loss, obtaining the selection loss of each band.

[0062] In this embodiment, S A Multiply each band by dimension to get a weighted hyperspectral image Each band is used to obtain a weighted hyperspectral image Input to the classification network F C Classify and get the predicted value Using hyperspectral classification loss L c To learn the optimal band attention weight for classification. At the same time, in order to achieve the selection effect, the band selection loss L s The regularization loss L of the band weight is added r .

[0063]

[0064]

[0065] L r =∥S A ∥ 1

[0066] L s =L c +L r

[0067] in, Represents the classification network F C Weighted hyperspectral image To classify, is the predicted value, y j Weighted hyperspectral image The real label.

[0068] As an optional embodiment, in step S105, the attention weights of each band of the ideal hyperspectral image are processed by the maximum index, and the expression of the key band of the ideal hyperspectral image is obtained as follows:

[0069] R BS =topk(S A )

[0070] Among them, R BS It represents the key band of the ideal hyperspectral image, and topk represents the maximum index.

[0071] In this embodiment, S A After topk, we get the first n bands R with the largest weight. BS As a mission-oriented reconstruction band key objectives.

[0072] As an optional embodiment, in step S106, the spectral reconstruction of the hyperspectral scene image after real-time imaging is performed through the key band of the ideal hyperspectral image, and the expression of the reconstructed hyperspectral scene image is obtained as follows:

[0073]

[0074] in, represents the reconstructed hyperspectral scene image, F R represents the spectral reconstruction network.

[0075] In this embodiment, the spectral reconstruction of the hyperspectral scene image after real-time imaging is performed using the key bands of the ideal hyperspectral image to obtain a reconstructed hyperspectral scene image.

[0076] As an optional implementation mode, step S106, after spectrally reconstructing the hyperspectral scene image after real-time imaging through the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image, the method also includes: inputting the reconstructed hyperspectral scene image into the classification network to obtain the predicted value of the category in the reconstructed hyperspectral scene image; calculating based on the predicted value of the category in the reconstructed hyperspectral scene image and the true value of the category in the reconstructed hyperspectral scene image to obtain a second classification loss; determining the reconstruction loss based on the image corresponding to the key bands of the ideal hyperspectral image and the reconstructed hyperspectral scene image; and determining the joint loss based on the second classification loss and the reconstruction loss.

[0077] In this embodiment, Input classification network F C Classify in the image to get the predicted value of the category in the reconstructed hyperspectral scene image:

[0078]

[0079] L′ S =L′ C +L′ R

[0080] in, R BS is the x of the band ～ , L′ C is the second classification loss, L′ R is the reconstruction loss, L′ S For joint losses, The image corresponding to the key bands of the ideal hyperspectral image.

[0081] Experimental part:

[0082] 1. Data preprocessing

[0083] For a hyperspectral target scene x, in order to facilitate simulation, the hyperspectral image x is used. ～ As x ideal scene. For the hyperspectral image x ～ , band reconstruction guided by liquid crystal filter modulation. Hyperspectral image x ～ Contains three dimensions x ～ ∈R c×h×w , where c is the spectral dimension, and h and w are the length and width of the spatial dimension, which contains N classes and Q sample pixels of each class. Each pixel of the image in the spatial dimension has a label y∈R h×w , whose value is

[0084] In addition, the min-max normalization data preprocessing method is used for the training input images to eliminate the influence of data dimension and make the data indicators comparable.

[0085] 2. Model structure and initialization

[0086] The present invention adopts an imaging network, a band selection network, a spectrum reconstruction network and a classification network based on liquid crystal.

[0087]

[0088] R BS =topk(avgpooling(Sigmoid(Linear(Relu(Linear(x ～ )))))))

[0089]

[0090] F C =Unet(x ～ )

[0091] Among them, SST is a spectral transformation reconstructor, which consists of multiple spectral attention modules embedded in a U-shaped network; Unet is a U-shaped classification network, which is divided into a downsampling encoder and an upsampling decoder, and the former transmits information to the latter in multiple layers.

[0092] 3. Model Pre-training

[0093] Since the imaging system hopes to verify its effectiveness in practical applications, it can be decoupled from specific applications in practical applications. Therefore, the liquid crystal imaging network, band selection network and spectrum reconstruction network are pre-trained first. The pre-training can be task-related or task-irrelevant. If the task is relevant, the task-related loss is added during band selection, such as the first classification loss L c .

[0094] After obtaining the key reconstruction target band R BS Finally, in order to better reconstruct the target band, the liquid crystal imaging network and the spectral reconstruction network are jointly pre-trained to obtain the spectral response function of the liquid crystal that is most conducive to spectral reconstruction, thereby obtaining the optimal voltage applied to the liquid crystal.

[0095] 4. Training process

[0096] The model obtained by the above training has task-independent generalization ability, but lacks the ability related to specific tasks. Based on the above model, the task-related loss is added to fine-tune the model. When fine-tuning the model, the body part of the reconstruction network is frozen, and only the conv of the first and last layers of the reconstruction network and the liquid crystal imaging network are fine-tuned.

[0097] In an embodiment of the present invention, a hyperspectral scene is acquired; the hyperspectral scene is processed by liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging; an ideal hyperspectral image is acquired; the spectral attention in the ideal hyperspectral image is extracted by a spectral attention module to obtain the attention weights of each band of the ideal hyperspectral image; the attention weights of each band of the ideal hyperspectral image are processed by a maximum index to obtain the key bands of the ideal hyperspectral image; the hyperspectral scene image after real-time imaging is spectrally reconstructed by the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image, thereby solving the technical problems of compressed sensing imaging, heterogeneous data fusion reconstruction imaging and spectral filter modulation imaging in existing computational spectral imaging that the spectral response function cannot be switched and it is difficult to adapt to complex unknown scenes, and achieving the technical effect of improving the imaging quality and real-time imaging of the system through a task-oriented band selection module, a spectral filter modulation imaging module and a spectral reconstruction module.

[0098] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0099] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0103] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A filter-modulated hyperspectral real-time computational imaging method guided by key band reconstruction, characterized in that: include: Acquire hyperspectral scenes; The hyperspectral scene is processed by liquid crystal filter modulation to obtain a hyperspectral scene image after real-time imaging; Obtain ideal hyperspectral images; The spectral attention module is used to extract the spectral attention in the ideal hyperspectral image and obtain the attention weights of each band of the ideal hyperspectral image. The attention weights of each band of the ideal hyperspectral image are processed by the maximum index to obtain the key bands of the ideal hyperspectral image; The spectral reconstruction of the hyperspectral scene image after real-time imaging is performed through the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image.

2. The method according to claim 1, characterized in that The hyperspectral scene is processed by liquid crystal filter modulation, and the expression of the hyperspectral scene image after real-time imaging is obtained: in, represents the spectral response function, represents the imaging network formed by the spectral response function, d represents the thickness of the liquid crystal, λ is the band of the hyperspectral scene image, i ranges from 1 to m, and Δn is the voltage V i A function of Represents a hyperspectral scene image after real-time imaging.

3. The method according to claim 1, characterized in that: The spectral attention module is used to extract the spectral attention in the ideal hyperspectral image, and the expression of the attention weight of each band of the ideal hyperspectral image is obtained as follows: S A =avg pooling(S θ (x ～ )) Among them, S A represents the attention weights of each band of the ideal hyperspectral image, avg pooling represents average pooling, S θ represents the spectral attention module, and x~ represents the ideal hyperspectral image.

4. The method according to claim 1, characterized in that: After extracting the spectral attention in the ideal hyperspectral image through the spectral attention module to obtain the attention weights of each band of the ideal hyperspectral image, the method further includes: The spectral attention in the ideal hyperspectral image is multiplied by each band of the ideal hyperspectral image to obtain a weighted hyperspectral image; Input the weighted hyperspectral image into the classification network to obtain the predicted value of the category of the weighted hyperspectral image; The first classification loss is obtained by calculating the predicted value of the category of the weighted hyperspectral image and the true value of the category in the weighted hyperspectral image; Based on the attention weights of each band of the ideal hyperspectral image, determine the regularization loss of the attention weights of each band; Based on the first classification loss and regularization loss, the selection loss of each band is obtained.

5. The method according to claim 1, characterized in that The attention weights of each band of the ideal hyperspectral image are processed by the maximum index, and the expression of the key band of the ideal hyperspectral image is obtained as follows: R BS =ball(S A ) Among them, R BS It represents the key band of the ideal hyperspectral image, and topk represents the maximum index.

6. The method according to claim 1, characterized in that The spectral reconstruction of the hyperspectral scene image after real-time imaging is performed through the key bands of the ideal hyperspectral image, and the expression of the reconstructed hyperspectral scene image is obtained as follows: in, represents the reconstructed hyperspectral scene image, F R represents the spectral reconstruction network.

7. The method according to claim 1, characterized in that After performing spectral reconstruction on the hyperspectral scene image after real-time imaging through the key bands of the ideal hyperspectral image to obtain the reconstructed hyperspectral scene image, the method further comprises: The reconstructed hyperspectral scene image is input into the classification network to obtain the predicted value of the category in the reconstructed hyperspectral scene image; The second classification loss is obtained by calculating the predicted value of the category in the reconstructed hyperspectral scene image and the true value of the category in the reconstructed hyperspectral scene image; Determine the reconstruction loss based on the image corresponding to the key band of the ideal hyperspectral image and the reconstructed hyperspectral scene image; Based on the second classification loss and the reconstruction loss, a joint loss is determined.

8. A processor, characterized in that: The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 7 when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images