High-efficiency hyperspectral imaging method, device and readable storage medium for mobile devices

Abstract

This invention proposes an efficient hyperspectral imaging method, device, and readable storage medium for mobile devices. Existing CASSI system reconstruction algorithms rely on high-performance GPU servers, making them inoperable on mobile devices and wasting resources. This invention utilizes a U-shaped network architecture, comprising initialization, feature encoding, feature enhancement, and reconstruction modules. The initialization module uses fixed-weight convolution to rapidly generate the initial input signal; the feature encoding module downsamples early and utilizes reparameterization techniques; the feature enhancement module uses structural reparameterization to facilitate training and inference; and the reconstruction module restores resolution and maps it to image space. Through the collaborative operation of these modules, efficient and high-quality reconstruction of compressed measurements from the CASSI system is achieved directly on mobile devices, providing an effective solution for the application of hyperspectral imaging on mobile devices.

Description

Technical Field

[0001] The present invention relates to the field of image signal processing technology, and in particular to a high-efficiency hyperspectral imaging method and device for mobile devices and a readable storage medium thereof. Background Art

[0002] Hyperspectral imaging technology captures a large number of narrow spectral bands to produce a spectral data cube, where each band corresponds to information at a specific spectral wavelength. Compared to traditional RGB images, hyperspectral images provide richer information and are widely used in fields such as object detection, medical diagnosis, and food safety. Traditional hyperspectral cameras primarily use a scanning method to generate images, which can only capture information in a localized space or a specific spectral band within a single exposure. The long imaging time makes it impossible to capture information about moving objects. To address this problem, researchers have developed the Coded Aperture Snapshot Spectral Imaging (CASSI) system, which can capture hyperspectral images of multiple spectral bands with a single exposure. The CASSI system modulates signals of different wavelengths using a fixed physical mask and a prism, and uses a 2D detector to capture the modulated signals to obtain compressed spectral measurements. A reconstruction algorithm is then used to recover the original hyperspectral signal from the compressed measurements. The CASSI system not only offers fast imaging speed but also low transmission bandwidth and memory consumption. However, due to the high model complexity, current state-of-the-art reconstruction algorithms typically rely on high-performance GPU servers for reconstruction, resulting in a significant waste of computing and bandwidth resources and making them unsuitable for mobile devices. Summary of the Invention

[0003] The embodiments of the present invention provide an efficient hyperspectral imaging method and device and a readable storage medium thereof for mobile devices. These methods address the current technical problems that the most advanced reconstruction algorithm model of the existing CASSI system is highly complex, relies on high-performance GPU servers for reconstruction, resulting in a large waste of computing resources and bandwidth resources, and cannot be run on mobile devices.

[0004] The core technology of this invention is to propose an efficient CASSI system reconstruction algorithm that is friendly to mobile devices. By designing an initialization module of fixed-weight convolution, a feature encoding module and a feature enhancement module with early downsampling and efficient structural reparameterization, it achieves efficient and high-quality reconstruction of the compressed measurement values of the CASSI system on mobile devices.

[0005] In a first aspect, the present invention provides an efficient hyperspectral imaging method for mobile devices, the method comprising the following steps:

[0006] S00, establishing a model, including an initialization module, a feature encoding module, a feature enhancement module group and a reconstruction module;

[0007] Based on the given compressed measurement data, the initialization module obtains the blurred hyperspectral image Xe through fixed weight convolution;

[0008] S10, the feature encoding module maps the hyperspectral image Xe to the feature space through early downsampling to obtain a feature map with reduced spatial resolution, which contains the feature information of the hyperspectral image;

[0009] S20, the feature enhancement module group uses a multi-layer symmetrical encoder-decoder to convert shallow features of the feature map into deep features; wherein the encoder stage gradually downsamples and increases the number of channels, and the decoder stage upsamples and reduces the number of channels by pixel reorganization;

[0010] S30, the reconstruction module maps the deep features to the image space through pixel reorganization operation to obtain the reconstructed hyperspectral image X.

[0011] Furthermore, step S20 also includes fusing the encoder features with the corresponding layers of the decoder along the channel dimension through a jump connection.

[0012] Furthermore, in step S00, the initialization module is composed of multiple convolution kernels, the number of convolution kernels is consistent with the number of reconstructed bands, and the convolution weights of the initialization module remain fixed during the training process; wherein, the number of reconstructed bands is the number of spectral bands contained in the hyperspectral image recovered from the compressed measurement data.

[0013] Furthermore, in step S10, the feature encoding module implements early downsampling through a 3x3 convolution with a spatial stride of 2.

[0014] Furthermore, in step S20, each encoder-decoder layer contains multiple feature enhancement modules, each feature enhancement module contains two re-parameterized modules with the same structure, each re-parameterized module consists of a residual connection and n 3x3 convolution over-parameter modules, and the regularization layer of each convolution branch is deleted and learnable parameters are used to adjust the output of each convolution.

[0015] Furthermore, in step S20, the model adopts a linear rectification activation function.

[0016] Furthermore, in step S20, during the inference phase of the model, all convolution weight parameters and residual connections of the same layer are merged to obtain a new convolution.

[0017] In a second aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned efficient hyperspectral imaging method for mobile devices.

[0018] In a third aspect, the present invention provides an efficient hyperspectral imaging device for mobile devices, comprising a coded aperture snapshot spectral imaging system CASSI and the above-mentioned electronic device, which is used to reconstruct compressed measurements captured by the CASSI system to obtain a hyperspectral image.

[0019] In a fourth aspect, the present invention provides a readable storage medium having a computer program stored therein, the computer program including a program code for controlling a process to execute a process, the process including the above-mentioned efficient hyperspectral imaging method for mobile devices.

[0020] The main contributions and innovations of the present invention are as follows:

[0021] 1. Efficiency: This paper significantly reduces the computational complexity of the model and improves reconstruction speed by designing a fixed-weight convolution initialization module, a feature encoding module with early downsampling, and a feature enhancement module with efficient structural reparameterization. This makes the hyperspectral imaging reconstruction process more efficient, enabling high-quality image reconstruction in a shorter time.

[0022] 2. Mobile-Friendly: The proposed algorithm can run efficiently on mobile devices, enabling hyperspectral imaging with low transmission bandwidth and low power consumption. This enables the application of hyperspectral imaging technology to resource-constrained mobile devices, expanding its scope of application and meeting the portability and real-time requirements of practical applications.

[0023] 3. High-quality reconstruction: By using a structural reparameterization method and the LeakyReLU activation function, this paper improves the reconstruction quality of the model. This makes the reconstructed hyperspectral image clearer and more accurate, better preserving the original spectral information, and enhances the application of hyperspectral imaging in object detection, medical diagnosis, food safety, and other fields.

[0024] 4. Real-time performance: The algorithm of this invention has better real-time performance on mobile devices, meeting the real-time requirements of practical applications. This enables the application of hyperspectral imaging technology in scenarios requiring rapid response, such as real-time monitoring and rapid detection, improving its feasibility and practicality in practical applications.

[0025] 5. Resource Optimization: This invention reduces the waste of computing and bandwidth resources by optimizing the algorithm structure. This not only reduces hardware costs but also improves resource utilization efficiency, making hyperspectral imaging technology more economical and practical.

[0026] In summary, the present invention has significant beneficial effects in terms of high efficiency, mobile device friendliness, high-quality reconstruction, real-time performance, and resource utilization optimization, overcoming the shortcomings of the existing technology and providing a new solution for the development and application of hyperspectral imaging technology.

[0027] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0029] Figure 1 is a U-shaped network architecture diagram according to an embodiment of the present invention;

[0030] Figure 2 2. FIG. 1 is a comparison diagram of a fixed convolution kernel initialization method according to an embodiment of the present invention and a conventional initialization method;

[0031] Figure 3 is an architecture diagram of a feature encoding module, a feature enhancement module, and a reconstruction module according to an embodiment of the present invention;

[0032] Figure 4 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0034] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0035] The current most advanced CASSI system reconstruction algorithm usually relies on high-performance GPU servers for reconstruction due to the high model complexity and cannot run on mobile devices, resulting in a large waste of computing resources and bandwidth resources.

[0036] Based on this, the present invention solves the problems existing in the prior art based on a new network structure.

[0037] Example 1

[0038] This paper aims to develop an efficient hyperspectral imaging method for mobile devices. It proposes a fixed-weight convolution weight initialization module to quickly obtain the initial input signal for the reconstruction network. Then, by using an early downsampling method, an efficient feature encoding module is proposed, significantly reducing the model's computational complexity. Using a structural reparameterization method, an efficient feature enhancement module is proposed, facilitating model training and efficient inference.

[0039] Specifically, the embodiment of the present invention provides an efficient hyperspectral imaging method for mobile devices, specifically, referring to Figure 1 , the method comprising:

[0040] S00, establishing a model, including an initialization module, a feature encoding module, a feature enhancement module group and a reconstruction module;

[0041] Based on the given compressed measurement data, the initialization module obtains the blurred hyperspectral image Xe through fixed weight convolution;

[0042] In this embodiment, if Figure 1 As shown in the figure, the algorithm proposed in this paper is a U-shaped network architecture (model), which is mainly composed of an initialization module, a feature encoding module, a feature enhancement module group, and a reconstruction module. The U-shaped network architecture is named because its shape resembles the English letter "U". It mainly consists of two parts: an encoder and a decoder. The U-shaped network architecture combines the feature extraction capabilities of the encoder with the image reconstruction capabilities of the decoder. It can effectively extract the features of hyperspectral images from compressed measurement data and restore these features into high-quality images.

[0043] Among them, Figure 2As shown in (a), the previous conventional initialization method requires multiple iterations. Each iteration will crop different areas of the measurement to obtain initialization results of different spectral dimensions. After all iterations are completed, all cropped spectral data are spliced along the spectral dimension to obtain the initialized result Xe. Since multiple iterations are required, the previous initialization module has low operating efficiency, which seriously affects the overall reconstruction speed of the model. Through analysis, it is found that this multiple iteration method can be completed using the method of fixed convolution kernel weights, and the convolution operation can be efficiently run on mobile devices. Therefore, the present invention proposes an initialization module based on convolution. As shown in (a) in the figure, the previous conventional initialization method requires multiple iterations. Figure 2 As shown in (b) of the figure, the initialization module consists of multiple convolution kernels (the number of which matches the number of reconstruction bands). Each convolution kernel is assigned a different initial value based on wavelength. Unlike other modules in the algorithm, the convolution weight parameters of this module remain fixed throughout the training process. Here, "number of reconstruction bands" refers to the number of spectral bands that need to be recovered from compressed measurements during hyperspectral imaging. This directly affects the number of convolution kernels in the initialization module and the complexity of the entire reconstruction algorithm.

[0044] S10, the feature encoding module maps the hyperspectral image Xe to the feature space through early downsampling to obtain a feature map with reduced spatial resolution, which contains the feature information of the hyperspectral image;

[0045] In this embodiment, to ensure the quality of model reconstruction, the feature extraction modules used by most previous end-to-end CASSI reconstruction algorithms usually keep the spatial resolution of input and output consistent. However, operating at the original spatial resolution consumes a lot of computing resources, causing the model to suffer from severe latency bottlenecks on mobile devices, especially when processing high-resolution compressed measurements. To alleviate this problem, Figure 3 As shown in (a), the present invention adds a 3x3 convolution with a spatial step size of 2 in the embedding module to achieve early downsampling, thereby reducing the spatial resolution of the feature map output. This operation effectively reduces the model complexity and inference delay. Although it will cause a certain degree of decline in the quality of model reconstruction, the result reparameterization technique of the present invention effectively alleviates this problem. The embedding module here belongs to the feature encoding module. The feature encoding module is designed to process the blurred hyperspectral image Xe output by the initialization module and map it to the feature space, and the embedding module is the key link to realize this processing process. It is located in the process of the feature encoding module processing the blurred hyperspectral image Xe, and is the specific location for performing early downsampling operations.

[0046] S20, the feature enhancement module group uses a multi-layer symmetrical encoder-decoder to convert shallow features of the feature map into deep features; wherein the encoder stage gradually downsamples and increases the number of channels, and the decoder stage upsamples and reduces the number of channels by pixel reorganization; the encoder features are also added to the corresponding layers of the decoder along the channel dimension through skip connections;

[0047] A key feature of the U-shaped network architecture is the presence of skip connections, which directly connect features from different levels in the encoder to the corresponding levels in the decoder. This allows the decoder to utilize the rich detail information extracted by the encoder during reconstruction, preventing the loss of local details caused by downsampling.

[0048] In this embodiment, for fast model training and efficient reasoning. Figure 3 As shown in (b), the present invention proposes an efficient feature enhancement module using a structural reparameterization method. Unlike existing reparameterization module implementations such as RepVGG and Mobile One, the feature enhancement module designed in this invention comprises two reparameterization modules with the same structure. Each reparameterization module consists of a residual connection and n over-parameterized modules with 3x3 convolutions. Furthermore, the present invention removes the regularization layer from each convolution branch and uses a learnable parameter to adjust the output of each convolution, facilitating rapid model training. Furthermore, the present invention replaces the rectified linear activation function (ReLU) with a leaky rectified linear activation function (LeakyReLU), which improves model reconstruction quality. During inference, the present invention merges all convolution weight parameters and residual connections of the same layer to obtain a new 3x3 convolution. During inference, only one convolution operation is required per layer, resulting in faster model reconstruction speed.

[0049] (1) The structure of the feature enhancement module is as follows:

[0050] Reparameterization Module: The Feature Enhancement Module contains two identical reparameterization modules. Reparameterization is a technique for converting multiple parameter representations into a more efficient form. In neural networks, using reparameterization modules can simplify the model structure and improve computational efficiency.

[0051] Residual Connections: Each reparameterized module contains a residual connection. Residual connections skip some layers in the network and add the input directly to the output. This design helps address the vanishing gradient problem in deep neural networks, making the model easier to train while preserving more of the original information.

[0052] Overparameterized modules: Each reparameterized module also contains n overparameterized modules consisting of 3x3 convolutions. Overparameterized modules mean that the number of parameters in the module is relatively large, which can increase the expressive power of the model and allow the model to learn more complex features.

[0053] (2) Design improvements and their benefits are as follows:

[0054] Removing regularization layers: Regularization layers (such as batch normalization) are often used to reduce overfitting, but in some cases they increase computational overhead. Removing the regularization layer from each convolutional branch and using a learnable parameter to adjust the output of each convolution can reduce computational overhead and speed up model training.

[0055] Replace the activation function: Replace the Rectified Linear Activation Function (ReLU) with the LeakyReLU. ReLU outputs 0 when the input is less than 0, which can cause some neurons to "die" and information to be lost. LeakyReLU, on the other hand, produces a small non-zero output when the input is less than 0. This avoids the problem of neuron death and allows the model to better learn information from negative inputs, thereby improving the model's reconstruction quality.

[0056] (3) The optimization of the reasoning process is as follows:

[0057] Merging convolution weights and residual connections: During inference, all convolution weight parameters and residual connections of the same layer are merged to obtain a new 3x3 convolution. The advantage of this is that only one convolution operation is required per layer during inference, greatly reducing the amount of computation.

[0058] Improved reconstruction speed: Since only one convolution operation is required for each layer, the model's computational speed during inference is significantly improved, resulting in a faster reconstruction speed for the entire model, making it more suitable for use in scenarios with high real-time requirements, such as hyperspectral imaging applications on mobile devices.

[0059] In summary, the feature enhancement module not only improves the training efficiency and reconstruction quality of the model, but also accelerates the reasoning speed of the model through reasonable structural design, improved training strategy and optimized reasoning process. It is an important component of the present invention to improve the efficiency and quality of hyperspectral imaging.

[0060] S30, the reconstruction module maps the deep features to the image space through pixel reorganization operation to obtain the reconstructed hyperspectral image X.

[0061] In this embodiment, due to the early downsampling operation of the embedding module, the spatial resolution of the deep features of the 3-layer symmetric encoder-decoder is only half the network input size. Figure 3As shown in (c), in the reconstruction module, the present invention uses a pixel reshaping operation to quickly restore the spatial resolution of the feature map to the network input size. A 3x3 convolution is then used to map the feature space to the desired spectral image space. Similarly, the present invention uses a convolutional reparameterization method with a scaling factor during training, which facilitates model training and fast inference.

[0062] The pixel shuffle operation, also known as sub-pixel convolution, rearranges the information in a low-resolution feature map to generate a high-resolution feature map. Simply put, it reorganizes the data in the channel dimension of the feature map into the spatial dimension, thereby increasing the spatial resolution of the feature map. The pixel shuffle operation plays a role in the reconstruction module and works in conjunction with other modules in the entire U-shaped network architecture. It receives the feature map output from the encoder-decoder, restores its spatial resolution, and then uses convolution operations to complete the mapping from the feature space to the desired spectral image space, thereby reconstructing the hyperspectral image. Simultaneously, the pixel shuffle operation complements the downsampling operation of the feature encoding module, jointly achieving the task of reducing computational complexity while ensuring image reconstruction quality.

[0063] Preferably, the steps of restoring the spatial resolution of the feature map using pixel binning are as follows:

[0064] Prepare the input feature map: After downsampling in the encoder stage, we get a feature map with reduced spatial resolution and increased number of channels.

[0065] Determine the upsampling factor: This is determined based on the encoder downsampling situation. The goal is to restore the network input size. For example, if the resolution is reduced to half, the upsampling factor is 2.

[0066] Channel rearrangement: Divide the input feature map channels into r 2 groups (r is the upsampling factor), the number of channels in each group is C / r 2 (C is the original number of channels), rearrange the pixels in the group to the spatial dimension according to the rules, so that the height and width become r times the original, and the number of channels becomes C / r 2 .

[0067] Output high-resolution feature map: After channel rearrangement, the feature map with improved spatial resolution is obtained. Multiple operations or appropriate selection of upsampling factors can restore it to the network input size.

[0068] Reparameterization refers to the process of expressing the model's parameters in different forms during the training and inference phases to optimize training and inference efficiency. In convolutional neural networks, relatively complex structures are typically used during training to enhance the model's expressiveness, while these complex structures are converted to simpler forms during inference to reduce computational effort. Using convolutional reparameterization with scaling factors, complex structures can be leveraged during training to enhance the model's expressiveness, enabling the model to better learn features from the data. During inference, converting complex structures into simple convolution operations significantly reduces computational effort, improves inference speed, and makes the model more suitable for resource-constrained environments such as mobile devices. Furthermore, the introduction of scaling factors helps adjust the scale of features, further optimizing the model's training results.

[0069] Preferably, the convolution reparameterization method with scaling factor is basically implemented as follows:

[0070] Complex structure construction during the training phase: The model adopts a complex structure with multiple convolutional layers and residual connections. Each convolution branch uses learnable parameters to adjust the output, and a learnable scaling factor is introduced to adjust the feature scale.

[0071] Convolution and scaling fusion: During training, the output of each convolutional layer is multiplied by the corresponding scaling factor, and the scaling factor and the convolutional layer weights are updated together through backpropagation.

[0072] Reparameterization transformation: During the inference phase, all convolution weight parameters, residual connections, and scaling factors of the same layer are merged to make multiple convolution operations equivalent to one.

[0073] New convolution kernel generation: The scaling factor is taken into account during merging. The weights of the convolution branches are multiplied by the scaling factor to obtain new weights. The new weights of each branch and the residual connection parameters are then merged to form a new convolution kernel. Only one convolution operation is required during inference.

[0074] Example 2

[0075] This embodiment also provides an electronic device, referring to Figure 4 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0076] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0077] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0078] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0079] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the efficient hyperspectral imaging methods for mobile devices in the above embodiments.

[0080] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0081] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0082] The input / output device 408 is used to input or output information. In this embodiment, the input information may be a blurred hyperspectral image Xe, and the output information may be a reconstructed hyperspectral image X.

[0083] Example 3

[0084] Based on the same concept, the present invention also proposes an efficient hyperspectral imaging device for mobile devices, including a coded aperture snapshot spectral imaging system CASSI and an electronic device of embodiment 2, which is used to reconstruct the compressed measurements captured by the CASSI system to obtain a hyperspectral image.

[0085] Example 4

[0086] This embodiment further provides a readable storage medium, in which a computer program is stored. The computer program includes program code for controlling a process to execute a process, and the process includes the efficient hyperspectral imaging method for mobile devices according to the first embodiment.

[0087] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0088] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0089] Embodiments of the present invention can be implemented by computer software, which is executable by the data processor of the mobile device, such as in the processor entity, or is implemented by hardware, or is implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer executable components configured to perform the embodiment when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow can represent a program step, or interconnected logical circuits, boxes and functions, or a combination of program steps and logical circuits, boxes and functions. The software can be stored in physical media such as memory chips or storage blocks implemented in the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. Physical media is non-transient media.

[0090] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. An efficient hyperspectral imaging method for mobile devices, characterized in that: include: S00, establishing a model, including an initialization module, a feature encoding module, a feature enhancement module group and a reconstruction module; Based on the given compressed measurement data, the initialization module obtains the blurred hyperspectral image Xe by fixed-weight convolution. The initialization module consists of multiple convolution kernels, the number of which is consistent with the number of reconstructed bands, and the convolution weights of the initialization module remain fixed during the training process. The number of reconstructed bands is the number of spectral bands contained in the hyperspectral image recovered from the compressed measurement data. S10, the feature encoding module maps the hyperspectral image Xe to the feature space through early downsampling to obtain a feature map with reduced spatial resolution, which contains the feature information of the hyperspectral image; S20, the feature enhancement module group uses a multi-layer symmetrical encoder-decoder to convert shallow features of the feature map into deep features; wherein, the encoder stage reduces the spatial resolution and increases the number of channels by gradually downsampling, and the decoder stage performs upsampling through pixel reorganization operation to improve the spatial resolution and reduce the number of channels, and the features of the corresponding layer of the encoder are fused with the features of the corresponding layer of the decoder along the channel dimension through skip connections; S30, the reconstruction module maps the deep features to the image space through pixel reorganization operation to obtain the reconstructed hyperspectral image X.

2. The high-efficiency hyperspectral imaging method for mobile devices according to claim 1, wherein: In step S10, the feature encoding module implements early downsampling through a 3x3 convolution with a spatial stride of 2.

3. The high-efficiency hyperspectral imaging method for mobile devices according to claim 1, characterized in that: In step S20, each encoder-decoder layer contains multiple feature enhancement modules, each feature enhancement module contains two re-parameterized modules with the same structure, each re-parameterized module consists of a residual connection and n 3x3 convolution over-parameter modules, and the regularization layer of each convolution branch is deleted and the learnable parameters are used to adjust the output of each convolution.

4. The high-efficiency hyperspectral imaging method for mobile devices according to claim 3, wherein: In step S20, the model uses a linear rectification activation function.

5. The high-efficiency hyperspectral imaging method for mobile devices according to any one of claims 1 to 4, characterized in that: In step S20, during the inference phase of the model, all convolution weight parameters and residual connections of the same layer are merged to obtain a new convolution.

6. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the efficient hyperspectral imaging method for mobile devices according to any one of claims 1 to 5.

7. An efficient hyperspectral imaging device for mobile devices, characterized in that: The invention comprises a coded aperture snapshot spectral imaging system CASSI and the electronic device according to claim 6, wherein the electronic device is used to reconstruct the compressed measurements captured by the CASSI system to obtain a hyperspectral image.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes the efficient hyperspectral imaging method for mobile devices according to any one of claims 1 to 5.

Images