High-efficiency hyperspectral imaging method and device for mobile equipment and readable storage medium of high-efficiency hyperspectral imaging method and device

Abstract

The invention provides a mobile equipment-oriented efficient hyperspectral imaging method and device and a readable storage medium thereof. The existing CASSI system reconstruction algorithm depends on a high-performance GPU (Graphic Processing Unit) server, cannot run in mobile equipment and wastes resources. The method adopts a U-shaped network architecture, and comprises an initialization module, a feature coding module, a feature enhancement module and a reconstruction module. The initialization module is used for rapidly generating an initial input signal by using fixed weight convolution; the feature coding module performs early-stage down-sampling and uses a re-parameterization technique; the feature enhancement module adopts structural re-parameterization to promote training and reasoning; the reconstruction module recovers the resolution and maps the resolution to an image space. Through cooperative work of all the modules, efficient and high-quality reconstruction of a CASSI system compression measurement value on mobile equipment is realized directly, and an effective solution is provided for application of hyperspectral imaging on the mobile equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of image signal processing, and particularly to an efficient hyperspectral imaging method, apparatus and readable storage medium for mobile devices. Background Art

[0002] Hyperspectral imaging technology captures a large number of narrow spectral bands to obtain a spectral data cube, and each spectral band corresponds to information of a specific spectral wavelength. Compared with traditional RGB images, hyperspectral images can provide richer information and are widely used in object detection, medical diagnosis, food safety and other fields. Traditional hyperspectral cameras mainly generate images by scanning, and can only capture information of a local space or a specific spectral band within a single exposure time. The long imaging time makes it impossible to capture information of moving objects. To solve this problem, researchers have developed a coded aperture snapshot spectral imaging system CASSI (Coded Aperture Snapshot Spectral Imaging), which can capture hyperspectral images of multiple spectral bands with only one exposure. The CASSI system modulates signals of different wavelengths through a fixed physical mask plus a prism, and uses a 2D detector to capture the modulated signals to obtain compressed spectral measurements. Then, a reconstruction algorithm is used to recover the original hyperspectral signal from the compressed measurements. The CASSI system not only has a fast imaging speed, but also has advantages such as low transmission bandwidth and low memory consumption. However, due to the high model complexity of the current state-of-the-art reconstruction algorithms, they usually need to rely on high-performance GPU servers for reconstruction, resulting in a large waste of computing resources and bandwidth resources, and cannot run on mobile devices. Summary of the Invention

[0003] Embodiments of the present invention provide an efficient hyperspectral imaging method, apparatus and readable storage medium for mobile devices, aiming at the problems existing in the current technology that the most advanced reconstruction algorithm model of the existing CASSI system has a high complexity, relies on a high-performance GPU server for reconstruction, causes a large waste of computing resources and bandwidth resources, and cannot run on mobile devices.

[0004] The core technology of the present invention mainly proposes an efficient CASSI system reconstruction algorithm friendly to mobile devices, which realizes the efficient and high-quality reconstruction of the CASSI system compressed measurements on mobile devices by designing an initialization module of fixed-weight convolution, a feature encoding module of early downsampling and efficient structure reparameterization, and a feature enhancement module.

[0005] In a first aspect, the present invention provides an efficient hyperspectral imaging method for mobile devices, and the method includes the following steps: S00. Establish a model, including an initialization module, a feature encoding module, a feature enhancement module group and a reconstruction module; Based on the given compressed measurement data, the initialization module obtains a blurred hyperspectral image Xe through fixed-weight convolution; S10. The feature encoding module maps the hyperspectral image Xe to the feature space through early downsampling to obtain a feature map with reduced spatial resolution, and this feature map contains the feature information of the hyperspectral image; S20. The feature enhancement module group uses a multi-layer symmetric encoder-decoder to convert the shallow features of the feature map into deep features; wherein, in the encoder stage, it gradually downsamples and increases the number of channels, and in the decoder stage, it upsamples through pixel recombination and reduces the number of channels; S30. The reconstruction module maps the deep features to the image space through pixel recombination operation to obtain the reconstructed hyperspectral image X.

[0006] Furthermore, in step S20, it also includes fusing the encoder features and the corresponding decoder layers along the channel dimension through skip connections.

[0007] Furthermore, in step S00, the initialization module consists of multiple convolutional kernels, and the number of convolutional kernels is the same as the number of reconstruction bands. Moreover, the convolutional weights of the initialization module remain fixed during the training process; wherein, the number of reconstruction bands is the number of spectral bands included in the hyperspectral image recovered from the compressed measurement data.

[0008] Furthermore, in step S10, the feature encoding module realizes early downsampling through a 3x3 convolution with a spatial stride of 2.

[0009] Furthermore, in step S20, each layer of the encoder-decoder contains multiple feature enhancement modules, and each feature enhancement module contains two reparameterized modules with the same structure. Each reparameterized module consists of a residual connection and an over-parameterized module with n 3x3 convolutions, and the regularization layers of each convolutional branch are deleted and learnable parameters are used to adjust the output of each convolution.

[0010] Furthermore, in step S20, the model adopts a rectified linear activation function.

[0011] Furthermore, in step S20, in the inference stage of the model, all the convolutional weight parameters and residual connections of the same layer are merged to obtain a new convolution.

[0012] In a second aspect, the present invention provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned efficient hyperspectral imaging method for mobile devices.

[0013] In a third aspect, the present invention provides an efficient hyperspectral imaging device for mobile devices, including a coded aperture snapshot spectral imaging system (CASSI) and the above-mentioned electronic device. The electronic device is used to reconstruct the compressed measurements captured by the CASSI system to obtain a hyperspectral image.

[0014] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, the computer program including program codes for controlling a process to execute the process, the process including the efficient hyperspectral imaging method for mobile devices as described above.

[0015] The main contributions and innovations of the present invention are as follows: 1. Efficiency: By designing an initialization module for fixed-weight convolution, an early downsampling, and a feature encoding module and a feature enhancement module for efficient structural reparameterization, the present invention significantly reduces the computational complexity of the model and improves the reconstruction speed. This makes the reconstruction process of hyperspectral imaging more efficient and enables high-quality image reconstruction to be completed in a shorter time.

[0016] 2. Mobile device friendly: The algorithm of the present invention can run efficiently on mobile devices, realizing hyperspectral imaging with low transmission bandwidth and low power consumption. This enables hyperspectral imaging technology to be applied to resource-constrained mobile devices, expanding its application scope and meeting the requirements for portability and real-time performance in practical applications.

[0017] 3. High-quality reconstruction: By using the structural reparameterization method and the LeakyReLU activation function, the present invention improves the reconstruction quality of the model. This makes the reconstructed hyperspectral image clearer and more accurate, better able to retain the original spectral information, and enhancing the application effect of hyperspectral imaging in fields such as object detection, medical diagnosis, and food safety.

[0018] 4. Real-time performance: The algorithm of the present invention has better real-time performance on mobile devices and can meet the requirements for real-time performance in practical applications. This enables hyperspectral imaging technology to be applied to scenarios that require quick response, such as real-time monitoring and rapid detection, improving its feasibility and practicality in practical applications.

[0019] 5. Optimized resource utilization: By optimizing the algorithm structure, the present invention reduces the waste of computing resources and bandwidth resources. This not only reduces the hardware cost but also improves the utilization efficiency of resources, making hyperspectral imaging technology more economical and practical.

[0020] In summary, the present invention has significant beneficial effects in terms of efficiency, mobile device friendliness, high-quality reconstruction, real-time performance, and optimized resource utilization, overcomes the defects of the prior art, and provides a new solution for the development and application of hyperspectral imaging technology.

[0021] Details of one or more embodiments of the present invention are set forth in the following drawings and description to make other features, objects, and advantages of the present invention more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and form a part of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a U-shaped network architecture diagram according to an embodiment of the present invention; Figure 2 is a comparison diagram of a fixed convolution kernel initialization method and a conventional initialization method according to an embodiment of the present invention; Figure 3 is an architecture diagram of a feature encoding module, a feature enhancement module, and a reconstruction module according to an embodiment of the present invention; Figure 4 is a schematic hardware structure diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: In other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0025] The currently most advanced CASSI system reconstruction algorithm usually requires a high-performance GPU server for reconstruction due to its high model complexity and cannot run on mobile devices, resulting in a large waste of computing resources and bandwidth resources.

[0026] Based on this, the present invention is based on a new network structure to solve the problems existing in the prior art.

[0027] Embodiment 1 The present invention aims to propose an efficient hyperspectral imaging method for mobile devices, and proposes an initialization module with fixed-weight convolution weights to quickly obtain the initial input signal of the reconstruction network. Then, by using the early downsampling method, an efficient feature encoding module is proposed, which greatly reduces the computational complexity of the model. By using the structural reparameterization method, an efficient feature enhancement module is proposed, which is beneficial to model training and efficient inference.

[0028] Specifically, an embodiment of the present invention provides an efficient hyperspectral imaging method for mobile devices. Specifically, referring to Figure 1 , the method includes: S00. Establish a model, including an initialization module, a feature encoding module, a feature enhancement module group, and a reconstruction module; Based on the given compressed measurement data, the initialization module obtains a blurred hyperspectral image Xe through fixed-weight convolution. In this embodiment, as Figure 1 shown, the algorithm proposed by the present invention is a U-shaped network architecture (model), which is mainly composed of an initialization module, a feature encoding module, a feature enhancement module group, and a reconstruction module. The name of the U-shaped network architecture comes from its shape similar to the English letter "U". It is mainly composed of two parts: an encoder and a decoder. The U-shaped network architecture combines the feature extraction ability of the encoder and the image reconstruction ability of the decoder, and can effectively extract the features of the hyperspectral image from the compressed measurement data and restore these features to high-quality images.

[0029] Among them, as Figure 2 shown in (a), since the previous conventional initialization method requires multiple iterations. Each iteration will crop different regions of the measurement to obtain the initialization results of different spectral dimensions. After all iterations are completed, all the cropped spectral data are stitched along the spectral dimension to obtain the initialized result Xe. Due to the need for multiple iterations, the previous initialization module has low running efficiency, seriously affecting the overall reconstruction speed of the model. Through analysis, it is found that this multiple-iteration method can be completed by using the method of fixed convolution kernel weights, and the convolution operation can be efficiently run on mobile devices. Therefore, the present invention proposes an initialization module based on convolution. As Figure 2 shown in (b), the initialization module is composed of multiple convolution kernels (the number is the same as the number of reconstruction bands). Different initial values are assigned to each convolution kernel according to the wavelength. Different from other modules in the algorithm, the convolution weight parameters of this module are fixed throughout the training process. Here, the "number of reconstruction bands" refers to the number of spectral bands that need to be recovered from the compressed measurement in the hyperspectral imaging process, which directly affects the number of convolution kernels in the initialization module and the complexity of the entire reconstruction algorithm.

[0030] S10. The feature encoding module maps the hyperspectral image Xe to the feature space through early downsampling to obtain a feature map with reduced spatial resolution, which contains the feature information of the hyperspectral image. In this embodiment, to ensure the quality of model reconstruction, the feature extraction modules used in most previous end-to-end CASSI reconstruction algorithms usually keep the spatial resolution of the input and output consistent. However, operating at the original spatial resolution consumes a large amount of computing resources, resulting in a serious latency bottleneck for the model on mobile devices, especially when processing high-resolution compressed measurements. To alleviate this problem, as Figure 3 shown in (a) of

[0031] S20. The feature enhancement module group uses a multi-layer symmetric encoder-decoder to convert the shallow features of the feature map into deep features; among them, in the encoder stage, it gradually downsamples and increases the number of channels, and in the decoder stage, it upsamples through pixel recombination and reduces the number of channels; the encoder features are also added to the corresponding layers of the decoder along the channel dimension through skip connections. Among them, an important feature of the U-shaped network architecture is the existence of skip connections. It directly connects the features of different levels in the encoder to the corresponding levels in the decoder, enabling the decoder to utilize the rich detailed information extracted by the encoder during the reconstruction process and preventing the loss of local details caused by downsampling.

[0032] In this embodiment, for fast model training and efficient inference. As Figure 3As shown in (b), the present invention proposes an efficient feature enhancement module by using the structural reparameterization method. Different from the implementation methods of the existing reparameterization modules RepVGG and Mobile One, the feature enhancement module designed by the present invention includes two reparameterization modules with the same structure. Each reparameterization module consists of a residual connection and an over-parameterized module with n 3x3 convolutions. Moreover, the present invention deletes the regularization layers of each convolution branch and uses a learnable parameter to adjust the output of each convolution, which is beneficial to the rapid training of the model. In addition, the present invention replaces the rectified linear activation function (ReLU) with the leaky rectified linear activation function (LeakyReLU), which is beneficial to improving the model reconstruction quality. During the inference process, the present invention merges all the convolution weight parameters and residual connections of the same layer to obtain a new 3x3 convolution. Only one convolution operation is required for each layer during inference, making the model have a faster reconstruction speed.

[0033] (1) The structure of the feature enhancement module is as follows: Reparameterization module: The feature enhancement module includes two reparameterization modules with the same structure. Reparameterization is a technique that converts multiple parameter representations into a more efficient form. In a neural network, using a reparameterization module can simplify the model structure and improve the computational efficiency.

[0034] Residual connection: Each reparameterization module includes a residual connection. A residual connection means skipping some layers in the network and directly adding the input to the output. This design helps to solve the vanishing gradient problem in deep neural networks, making the model easier to train and retaining more original information at the same time.

[0035] Over-parameterized module: Each reparameterization module also includes n over-parameterized modules composed of 3x3 convolutions. An over-parameterized module means that the number of parameters in the module is relatively large, which can increase the expressive power of the model and enable the model to learn more complex features.

[0036] (2) The design improvements and their benefits are as follows: Deleting the regularization layer: Regularization layers (such as batch normalization layers) are usually used to reduce overfitting, but in some cases, they will increase the computational amount. Deleting the regularization layers of each convolution branch and using a learnable parameter to adjust the output of each convolution can reduce the computational overhead and speed up the training speed of the model.

[0037] Replace the activation function: Replace the Rectified Linear Unit (ReLU) activation function with the Leaky Rectified Linear Unit (LeakyReLU). ReLU outputs 0 when the input is less than 0, which may cause some neurons to "die" and information loss. While LeakyReLU has a small non-zero output when the input is less than 0, which can avoid the problem of neuron death, enabling the model to better learn information from negative inputs, thereby improving the reconstruction quality of the model.

[0038] (3)The optimization of the inference process is as follows: Merge convolutional weights and residual connections: During the inference process, merge all the convolutional weight parameters and residual connections of the same layer to obtain a new 3x3 convolution. The advantage of doing this is that only one convolution operation needs to be performed per layer during inference, greatly reducing the computational complexity.

[0039] Improve the reconstruction speed: Since only one convolution operation needs to be performed per layer, the computational speed of the model during inference is significantly improved, enabling the entire model to have a faster reconstruction speed and being more suitable for scenarios with high real-time requirements, such as hyperspectral imaging applications on mobile devices.

[0040] In summary, through reasonable structural design, improved training strategies, and optimized inference processes, the feature enhancement module not only improves the training efficiency and reconstruction quality of the model but also speeds up the inference speed of the model, which is an important part of improving the efficiency and quality of hyperspectral imaging in the present invention.

[0041] S30. The reconstruction module maps the deep features to the image space through pixel reorganization operations to obtain the reconstructed hyperspectral image X.

[0042] In this embodiment, due to the early downsampling operation of the embedding module, the spatial resolution of the deep features of the 3-layer symmetric encoder-decoder is only half of the network input size. Therefore, as Figure 3 shown in (c), in the reconstruction module, the present invention uses pixel reorganization operations to quickly restore the spatial resolution of the feature map to the network input size. Then, 3x3 convolutions are used to complete the mapping from the feature space to the required spectral image space. Similarly, the present invention uses a convolutional reparameterization method with a scaling factor during the training process, which is beneficial for model training and fast inference.

[0043] Among them, the Pixel Shuffle operation, also known as Sub - Pixel Convolution, its core principle is to rearrange the information in the low - resolution feature map to generate a high - resolution feature map. Simply put, it reorganizes the data in the channel dimension of the feature map into the spatial dimension, thereby enhancing the spatial resolution of the feature map. The Pixel Shuffle operation plays a role in the reconstruction module and works in coordination with other modules of the entire U - shaped network architecture. It receives the feature map output from the encoder - decoder, restores its spatial resolution, and then completes the mapping from the feature space to the required spectral image space through convolution operations, thus realizing the reconstruction of hyperspectral images. At the same time, the Pixel Shuffle operation echoes the downsampling operation of the feature encoding module and jointly completes the task of ensuring the image reconstruction quality while reducing the computational complexity.

[0044] Preferably, the steps to restore the spatial resolution of the feature map using the Pixel Shuffle operation are as follows: Prepare the input feature map: After downsampling in the encoder stage, a feature map with reduced spatial resolution and increased number of channels is obtained.

[0045] Determine the upsampling factor: Determined according to the encoder downsampling situation, with the goal of restoring to the network input size. For example, if the resolution is reduced by half, the upsampling factor is 2.

[0046] Channel rearrangement: Divide the channels of the input feature map into r 2 groups (r is the upsampling factor), and the number of channels in each group is C / r 2 (C is the original number of channels). Rearrange the pixels within the group according to the rule into the spatial dimension, making the height and width become r times the original, and the number of channels become C / r 2 .

[0047] Output the high - resolution feature map: The feature map with enhanced spatial resolution is obtained through channel rearrangement. Multiple operations or appropriate selection of the upsampling factor can restore it to the network input size.

[0048] Among them, re-parameterization refers to representing the parameters of the model in different forms during the training and inference stages to optimize the training and inference efficiency. In a convolutional neural network, a more complex structure is usually used during training to enhance the model's expressive ability, and these complex structures are converted into simpler forms during inference, thereby reducing the computational amount. Through the convolutional re-parameterization method with a scaling factor, the complex structure can be utilized during the training stage to enhance the model's expressive ability, enabling the model to better learn the features in the data. During the inference stage, the complex structure is converted into a simple convolutional operation, greatly reducing the computational amount and improving the inference speed, making the model more suitable for running in resource-constrained environments such as mobile devices. At the same time, the introduction of the scaling factor helps to adjust the scale of the features and further optimize the training effect of the model.

[0049] Preferably, the basic implementation steps of the convolutional re-parameterization method with a scaling factor are as follows: Construction of the complex structure in the training stage: The model adopts a complex structure including multiple convolutional layers and residual connections. Each convolutional branch is adjusted by learnable parameters to output, and at the same time, a learnable scaling factor is introduced to adjust the feature scale.

[0050] Convolution and scaling fusion: During training, the output of each convolutional layer is multiplied by the corresponding scaling factor, and the scaling factor and the weights of the convolutional layer are updated together through backpropagation.

[0051] Re-parameterization conversion: During the inference stage, all the convolutional weight parameters, residual connections, and scaling factors of the same layer are merged, and multiple convolutional operations are equivalent to one.

[0052] Generation of the new convolution kernel: When merging, considering the scaling factor, the weights of the convolutional branches are multiplied by the scaling factor to obtain new weights, and then the new weights of each branch and the residual connection parameters are merged to form a new convolution kernel. Only one convolution operation is required during inference.

[0053] Embodiment 2 This embodiment also provides an electronic device. Refer to Figure 4 , including a memory 404 and a processor 402. A computer program is stored in the memory 404, and the processor 402 is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0054] Specifically, the above-mentioned processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0055] Among them, the memory 404 may include a mass memory 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 404 may include removable or non-removable (or fixed) media. Where appropriate, the memory 404 may be internal or external to the data processing device. In a particular embodiment, the memory 404 is non-volatile memory. In a particular embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. Where appropriate, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0056] The memory 404 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402.

[0057] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the efficient hyperspectral imaging methods for mobile devices in the above embodiments.

[0058] Optionally, the above electronic device may further include a transmission device 406 and an input / output device 408. Among them, the transmission device 406 is connected to the above processor 402, and the input / output device 408 is connected to the above processor 402.

[0059] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above network may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0060] The input / output device 408 is used to input or output information. In this embodiment, the input information can be a blurred hyperspectral image Xe, etc., and the output information can be a reconstructed hyperspectral image X, etc.

[0061] Embodiment III Based on the same concept, the present invention also proposes an efficient hyperspectral imaging device for mobile devices, including a coded aperture snapshot spectral imaging system CASSI and the electronic device of Embodiment II. The electronic device is used to reconstruct the compressed measurements captured by the CASSI system to obtain a hyperspectral image.

[0062] Embodiment IV This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes program codes for controlling a process to execute the process, and the process includes the efficient hyperspectral imaging method for mobile devices according to Embodiment I.

[0063] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, and will not be repeated here.

[0064] Generally, various embodiments can be implemented in hardware or special-purpose circuits, software, logic, or any combination thereof. Some aspects of the present invention can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although aspects of the present invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, the blocks, devices, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, special-purpose circuits or logic, general-purpose hardware or controllers, or other computing devices, or some combination thereof.

[0065] Embodiments of the present invention can be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. A computer software or program (also referred to as a program product), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer-executable components configured to perform embodiments when the program runs. One or more computer-executable components can be at least one software code or a part thereof. Additionally, in this regard, it should be noted that any block of a logical flow can represent a program step, or interconnected logical circuits, blocks, and functions, or a combination of program steps and logical circuits, blocks, and functions. The software can be stored on physical media such as memory chips or storage blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. The physical media are non-transitory media.

[0066] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope described in this specification.

[0067] The above embodiments merely represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. An efficient hyperspectral imaging method for mobile devices, characterized in that, Including: S00. Establish a model, including an initialization module, a feature encoding module, a feature enhancement module group, and a reconstruction module; Based on the given compressed measurement data, the initialization module obtains a blurred hyperspectral image Xe through fixed-weight convolution; S10. The feature encoding module maps the hyperspectral image Xe to the feature space through early downsampling to obtain a feature map with reduced spatial resolution, and this feature map contains the feature information of the hyperspectral image; S20. The feature enhancement module group converts the shallow features of the feature map into deep features using a multi-layer symmetric encoder-decoder; among them, in the encoder stage, it gradually downsamples and increases the number of channels, and in the decoder stage, it upsamples through pixel recombination and reduces the number of channels; S30. The reconstruction module maps the deep features to the image space through pixel recombination operations to obtain the reconstructed hyperspectral image X.

2. The high-efficiency hyperspectral imaging method for mobile devices according to claim 1, characterized in that In step S20, it also includes fusing the encoder features with the corresponding decoder layers along the channel dimension through skip connections.

3. An efficient hyperspectral imaging method for mobile devices according to claim 1, characterized in that, In step S00, the initialization module consists of multiple convolutional kernels, and the number of convolutional kernels is the same as the number of reconstruction bands. Moreover, the convolutional weights of the initialization module remain fixed during training; among them, the number of reconstruction bands is the number of spectral bands included in the hyperspectral image recovered from the compressed measurement data.

4. An efficient hyperspectral imaging method for mobile devices according to claim 1, characterized in that, In step S10, the feature encoding module realizes early downsampling through 3x3 convolution with a spatial stride of 2.

5. An efficient hyperspectral imaging method for mobile devices according to claim 1, characterized in that In step S20, each layer of the encoder-decoder contains multiple feature enhancement modules, and each feature enhancement module contains two reparameterized modules with the same structure. Each reparameterized module consists of a residual connection and an over-parameterized module with n 3x3 convolutions, and the regularization layers of each convolution branch are deleted and learnable parameters are used to adjust the output of each convolution.

6. The high-efficiency hyperspectral imaging method for mobile devices according to claim 5, characterized in that In step S20, the model uses a rectified linear activation function.

7. An efficient hyperspectral imaging method for mobile devices according to any one of claims 1-6, characterized in that, In step S20, in the inference stage of the model, all the convolutional weight parameters and residual connections of the same layer are merged to obtain a new convolution.

8. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is set to run the computer program to execute the efficient hyperspectral imaging method for mobile devices according to any one of claims 1 to 7.

9. An efficient hyperspectral imaging device for mobile devices, characterized in that, Including a coded aperture snapshot spectral imaging system CASSI and the electronic device according to claim 8, and the electronic device is used to reconstruct the compressed measurements captured by the CASSI system to obtain a hyperspectral image.

10. A readable storage medium, characterized in that, A computer program is stored in the readable storage medium, and the computer program includes program codes for controlling a process to execute the process, and the process includes the efficient hyperspectral imaging method for mobile devices according to any one of claims 1 to 7.

Images