Hyperspectral image generation method, device, equipment, medium and program product

Abstract

The invention relates to the technical field of image processing, and discloses a hyperspectral image generation method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining a target collection image; determining a periodic sampling unit, and performing mask processing on the target acquisition image based on the periodic sampling unit to obtain a coded image; and inputting the coded image into a preset image generation model to generate a target hyperspectral image based on the preset image generation model. According to the method, a modulation filtering device is additionally arranged in front of an image acquisition device, non-correlation of spectrum acquisition is introduced, high-frequency information acquisition required by spectrum reconstruction is realized, and a foundation is laid for accurate recovery of spectrum information. On the basis of mask processing of the periodic sampling unit, full-coverage sampling of a spectrum frequency domain is realized, the defect of low spectrum resolution caused by single spectrum acquisition frequency domain information in an existing method is overcome, the spectrum resolution is improved by more than one order of magnitude, the cost is relatively low, and portability is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment, medium and program product for generating a hyperspectral image. Background Art

[0002] Hyperspectral imaging is widely used in deep space remote sensing, environmental monitoring, geological survey, agricultural production, food safety, and biomedicine. However, the equipment used to obtain hyperspectral images is usually more complex and expensive than ordinary cameras. For example, more optical components are often used in hyperspectral imagers than in ordinary cameras. On the one hand, most hyperspectral imagers rely on precise scanning to generate three-dimensional data cubes, which are not portable and costly. On the other hand, snapshot hyperspectral imaging systems that do not require scanning have low effective spatial resolution and spectral resolution, and are difficult to meet high-precision spectral imaging. Therefore, generating hyperspectral images based on low-cost methods has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present invention provides a method, apparatus, device, medium and program product for generating a hyperspectral image to solve the problem of high cost in generating a hyperspectral image.

[0004] In a first aspect, the present invention provides a method for generating a hyperspectral image, the method comprising:

[0005] Acquire a target acquisition image, wherein the target acquisition image is determined based on an image acquisition device, wherein a filter modulation device is provided in front of a lens of the image acquisition device, and the filter modulation device includes at least one modulation filter;

[0006] Determine a periodic sampling unit, and perform mask processing on the target acquisition image based on the periodic sampling unit to obtain a coded image, wherein the size of the periodic sampling unit is positively correlated with the number of the modulation filters;

[0007] The encoded image is input into a preset image generation model to generate a target hyperspectral image based on the preset image generation model.

[0008] In a second aspect, the present invention provides a hyperspectral image generation device, the device comprising:

[0009] An image acquisition module, used for acquiring a target acquisition image, wherein the target acquisition image is determined based on an image acquisition device, wherein a filter modulation device is provided in front of a lens of the image acquisition device, and the filter modulation device includes at least one modulation filter;

[0010] A mask processing module, used for determining a periodic sampling unit, and performing mask processing on a target acquisition image based on the periodic sampling unit to obtain a coded image, wherein the size of the periodic sampling unit is positively correlated with the number of the modulation filters;

[0011] The image generation module is used to input the encoded image into a preset image generation model to generate a target hyperspectral image based on the preset image generation model.

[0012] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the hyperspectral image generation method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the hyperspectral image generation method of the first aspect or any corresponding embodiment thereof.

[0014] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for generating a hyperspectral image according to the first aspect or any corresponding embodiment thereof.

[0015] The hyperspectral image generation method provided by the embodiment of the present invention adds a modulation filter device in front of the image acquisition device, introduces the non-correlation of spectrum acquisition and realizes the high-frequency information acquisition required for computational spectrum reconstruction, laying the foundation for the accurate recovery of spectral information. Based on the mask processing of the periodic sampling unit, full coverage sampling of the spectral frequency domain is achieved, overcoming the defect of low spectral resolution caused by the single frequency domain information of spectral acquisition in the existing method, improving the spectral resolution by more than one order of magnitude, and having low cost and portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a schematic flow chart of a method for generating a hyperspectral image according to an embodiment of the present invention;

[0018] Figure 2 is a schematic structural diagram of a modulation filter according to an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of a modulation filter device and a hyperspectral reconstruction principle according to an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of a preset image generation model according to an embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of a self-supervised training framework according to an embodiment of the present invention;

[0022] Figure 6 is a schematic diagram of the effect of the method for generating a hyperspectral image according to an embodiment of the present invention;

[0023] Figure 7 is a structural block diagram of a hyperspectral image generating device according to an embodiment of the present invention;

[0024] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] Hyperspectral imaging refers to the dense sampling of spectral features in many narrow bands. Unlike traditional RGB images, each pixel of a hyperspectral image contains a continuous spectral curve to identify the material of the corresponding object. Therefore, hyperspectral imaging is widely used in deep space remote sensing, environmental monitoring, address surveying, agricultural production, food safety, and biomedicine. However, the equipment used to obtain hyperspectral images is usually more complex and expensive than ordinary cameras. Hyperspectral imagers often use more optical elements than ordinary cameras. On the one hand, most hyperspectral imagers rely on precise scanning (for example, push-broom or broom scanners) to generate three-dimensional data cubes (two-dimensional space and one-dimensional spectral information), which are poor in portability and high in cost. On the other hand, snapshot hyperspectral imaging systems that do not require scanning have low effective spatial resolution and spectral resolution, and it is difficult to meet high-precision spectral imaging. Based on this, an embodiment of the present invention provides an image processing method that uses a low-cost method to generate hyperspectral images.

[0027] According to an embodiment of the present invention, an image processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] In this embodiment, a method for generating a hyperspectral image is provided. Figure 1 is a flow chart of a method for generating a hyperspectral image according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0029] Step S101, obtaining a target acquisition image.

[0030] The target acquisition image is determined based on an image acquisition device. A filter modulation device is provided in front of the lens of the image acquisition device. The filter modulation device includes at least one modulation filter.

[0031] In this embodiment, the image acquisition device can be a color camera (RGB camera), a monochrome camera, an array camera, etc., which are widely used in daily life and can easily capture a large number of high-quality images. A filter modulation device is provided in front of the lens of the image acquisition device of this embodiment. The filter modulation device includes a modulation filter with high frequency coding in the spectral dimension, wherein the modulation filter with high frequency coding can be made of a thin film material. The number of modulation filters in the filter modulation device is not limited and can be set according to actual conditions. The modulation filter has a specific film thickness and a specific arrangement and coating order, and also includes a plurality of film structures with different intermediate layer thicknesses.

[0032] The filter modulation device is coupled to the image acquisition device, and a fixture of a specific size can be designed according to the size of the image acquisition device to be coupled, so as to fix the filter modulation device in front of the lens of the image acquisition device. For example, the modulation filter device can be fixed to the front end of the lens of the image acquisition device through a housing or a fixture, and the modulation filter device can be directly attached to the sensor or the front end of the lens of the image acquisition device, etc., without specific limitation.

[0033] In some optional embodiments, the filtering modulation device includes at least one modulation filter, which is prepared from at least two thin film materials in a preset arrangement, and the refractive indices of the thin film materials are different; when the number of modulation filters is greater than one, the thickness of the intermediate layer in each modulation filter is different.

[0034] For example, the modulation filter can be made of two thin film materials with different refractive indices, such as Ta2O5 and SiO2. The prepared modulation filter uses 1 mm thick SiO2 as a substrate, on which Ta2O5 and SiO2 are sequentially plated. Figure 2 As shown, each modulated filter can be composed of N structures, and the thickness of the intermediate layer in each structure can be set to be different. In this embodiment, the thickness of the intermediate layer decreases in sequence to form an arithmetic progression, and the thickness of the intermediate layer = 1000 + (N-1) × Δh (nm), where Δh represents the height difference of the intermediate layer, N represents the number of structures, and the thickness of the thin film material is shown in Table 1:

[0035] Number of layers Material Thickness / nm 1 <![CDATA[Ta2O5]]> 31.04 2 <![CDATA[SiO2]]> 89.08 3 <![CDATA[Ta2O5]]> 57.41 4 <![CDATA[SiO2]]> 1000+(N-1)×Δh 5 <![CDATA[Ta2O5]]> 59.96 6 <![CDATA[SiO2]]> 89.08 7 <![CDATA[Ta2O5]]> 57.41 8 <![CDATA[SiO2]]> 84.40 9 Base 1000000000

[0036] Step S102, determining a periodic sampling unit, and performing mask processing on the target captured image based on the periodic sampling unit to obtain a coded image.

[0037] The size of the periodic sampling unit is positively correlated with the number of modulation filters.

[0038] A periodic sampling unit refers to a coding array that is repeatedly arranged in a two-dimensional xy plane with a specific size of pixel unit as a period. If a 2×2 pixel unit is used as a period, the form of the 2×2 unit can be set as: [A, B; C, D], where the specific values ​​of the letters ABCD can be 1 or 0, which are used to represent light transmission or occlusion, respectively. The size of the periodic sampling unit is positively correlated with the number of modulation filters. For example, it can be set that the size of the periodic sampling unit is a proportional multiple of the number of modulation filters.

[0039] The target acquisition image is masked using the determined periodic sampling unit, and part of the pixels in the target acquisition image are blocked based on the set form, that is, the periodic units are periodically and repeatedly arranged in the target acquisition image, and the encoded image is obtained after masking.

[0040] In practical applications, the periodically arranged adaptive coding array (i.e., periodic sampling unit) can dynamically and randomly generate the minimum repeating unit according to the number of modulation filters of the actual modulation filter device. The total number of pixels in the coding array is equal to the number of image sensors in the image acquisition device.

[0041] Step S103: input the encoded image into a preset image generation model to generate a target hyperspectral image based on the preset image generation model.

[0042] The preset image generation model is an attention mechanism algorithm model. By inputting the encoded image into the preset image generation model, the encoded image is decoded, and three-dimensional hyperspectral data is reconstructed from the two-dimensional collected data, so as to obtain the target hyperspectral image.

[0043] The hyperspectral image generation method provided in this embodiment adds a modulation filter device in front of the image acquisition device, introduces the non-correlation of spectrum acquisition and realizes the high-frequency information acquisition required for computational spectrum reconstruction, laying the foundation for accurate recovery of spectral information. Based on the mask processing of the periodic sampling unit, full coverage sampling of the spectral frequency domain is achieved, overcoming the defect of low spectral resolution caused by single frequency domain information of spectral acquisition in the existing method, improving the spectral resolution by more than one order of magnitude, and having low cost and portability.

[0044] In some optional implementations, the above step S102 includes the following steps:

[0045] Step S201, determining the periodic sampling unit of the adaptive coding array based on the number of modulation filters in the filtering and modulation device.

[0046] In this embodiment, assuming that the number of modulation filters is K, the period unit is For example: when the number of modulation filters is 4, the periodic sampling unit is 2×2; when the number of modulation filters is 9, the periodic sampling unit is 3×3. For a single RGB camera with filter modulation, the minimum unit of the corresponding periodically arranged adaptive coding array is selected as 2×2. Correspondingly, three different mask units (i.e., periodic sampling units) are randomly generated: [1,0; 0,0], [0,1; 1,0], and [0,0; 0,1]. The value 1 indicates complete light transmission, and the value 0 indicates complete occlusion. The three coded masks are generated by repeating the above three units respectively, as shown in the following figure. Figure 3 shown.

[0047] Step S202: performing mask processing on the target captured image based on the periodic sampling unit to obtain a coded image.

[0048] The above three mask units are repeatedly arranged on the entire target acquisition image to form a complete coding mask. This periodic arrangement allows different parts of the image to have different information retention forms during encoding, and the encoded image is obtained after mask processing.

[0049] In some optional implementations, the above step S103 includes the following steps:

[0050] Step S301, performing convolution processing on the encoded image to determine a first image.

[0051] The dimension of the first image is consistent with the target hyperspectral image.

[0052] The preset image generation model is as follows Figure 4 As shown, Figure 4 (a) is the overall framework of the preset image generation model, which is a hierarchical attention Transformer model.

[0053] After the encoded image is input into the model, it undergoes a 1×1 convolution to obtain an initial estimated image (i.e., the first image) whose dimension is consistent with the final output three-dimensional hyperspectral data.

[0054] Step S302: sampling the first image based on the cumulative attention module in the preset image generation model to output a target hyperspectral image.

[0055] like Figure 4 As shown in (a), the first image after convolution is sampled through a series of cumulative attention modules (CAB), including upsampling and downsampling. The process will pass through different numbers of cumulative attention modules multiple times, and finally output a three-dimensional hyperspectral image (i.e., the target hyperspectral image).

[0056] Specifically, step S302 includes the following steps:

[0057] Step S3021, down-sample the first image based on a first preset number of cumulative attention modules to obtain a second image.

[0058] Step S3022, up-sample the second image based on a second preset number of cumulative attention modules, and output the target hyperspectral image based on a third preset number of cumulative attention modules.

[0059] The first image is downsampled (4×4 convolution) through N1 cumulative attention modules, reducing the spatial size to one-fourth of the original size, and the number of channels is doubled to obtain the first image. Subsequently, through N2 cumulative attention modules, in the upsampling process, it is connected to the shallow features through skip connections, and then the reconstructed target hyperspectral image is output through another set of N1 blocks. Among them, the third preset number is consistent with the first preset number, the third preset number = the first preset number = N1, and the second preset number is N2.

[0060] Among them, the cumulative attention module includes a spatial attention sub-module, a mask attention sub-module and a spectral attention sub-module.

[0061] In the spatial attention submodule, each local spatial feature map under different spectral bands is regarded as a token, and a certain area of ​​the image or image feature map (or called "local space") is encoded as a high-dimensional vector (i.e., token). This high-dimensional vector contains important information about the image features in the area, such as color, texture, shape, etc. Specifically, each local spatial feature map may be obtained through a layer of a convolutional neural network (CNN), which represents the characteristics of a local area in the image (such as a small image block). Then, these local spatial feature maps are "flattened" or "pooled" into a high-dimensional vector, which is regarded as a token.

[0062] In this embodiment, the spatial attention submodule regards each local spatial feature map under different spectral bands as a token, and calculates self-attention along the spatial dimension. The input image will perform attention calculation along the spatial dimensions of all channels, and the output is a spatial attention map with the same number of channels. The spectral attention submodule flattens the three-dimensional data obtained by the spatial attention submodule along the spatial direction, regards each spectral feature map as a token, and calculates attention along the spectral dimension. After obtaining the mask features through convolution on the mask attention submodule, it is flattened along the spatial direction and multiplied with the value in the spectral attention as a weight term. Finally, the output of the spectral attention submodule is resized to be consistent with the input dimension. The entire cumulative attention module realizes the extraction and restoration of the three-dimensional features of the spectral data.

[0063] In this embodiment, the spatial attention submodule is used to realize spatial resolution restoration, the spectral attention submodule is used to realize spectral information restoration, the residual module improves the robustness of the network, and the mask attention submodule improves the reconstruction accuracy of the network by introducing the spectral coding characteristics of the system.

[0064] Exemplarily, in the processing of the preset image generation model, taking the first preset number N1=2 and the second preset number N2=1=the third preset number as an example, the training can adopt 300 epochs (epoch is a basic unit in the deep learning training process, which represents a process in which the model completely learns the entire training data set), and the reconstructed wavelength range is selected from 400 to 700nm, and the spectral interval is 10nm.

[0065] In some optional implementations, the above hyperspectral image generation method further includes:

[0066] Step S401, obtaining an original image captured by an original image capture device without a filtering and modulation device;

[0067] The filtering and modulation device is not set as the original image acquisition device, and the original image is acquired by the original image acquisition device. In step S101, the shooting objects of the image acquisition device and the original image acquisition device are the same, and the target acquisition image and the original image are obtained respectively.

[0068] Step S402 : determining a comparison image based on the target hyperspectral image and the coding array in the coding image.

[0069] The obtained target hyperspectral image is multiplied with the encoding array in the encoding image to obtain a comparison image.

[0070] Step S403, calculating an image difference based on the comparison image and the original image, so as to optimize the preset image generation model according to the calculation result.

[0071] Calculate the root mean square error of the image difference between the comparison image and the original image, use the calculation result as the loss function of network training, and optimize the network parameters.

[0072] In this embodiment, the preset image generation model adopts Figure 5 The network is trained by the self-supervised training framework of the physical constraints shown in the figure, and the spectral responses of the modulated filter and the image acquisition device are added to the physical constraints of the hyperspectral image and the original image. There is no need for manually labeled hyperspectral-original image pairs, and the network can be directly trained by the original images collected by the system.

[0073] The hyperspectral image generation method provided by the embodiment of the present invention can be applied to endoscopes to expand its application in hyperspectral cancer detection, industrial machine vision-enhanced food quality detection, mobile phone camera-based vein enhancement, health protection and other consumer-level care points.

[0074] For example, Figure 6 These are RGB images and hyperspectral images collected by a color (RGB) camera and an image acquisition device according to an embodiment of the present invention, respectively. The example on the left mixes real leaves with fake leaves of very similar colors. After taking pictures directly with an RGB camera, it is difficult for the naked eye to distinguish them directly. After the image acquisition device of the present invention (equipped with a modulation filter device) is used to collect images and process the images, the fake leaves are clearly distinguished from the real leaves. The results show that the method of the present invention can restore the original color of the object through accurate spectral information, that is, the standard color difference ΔE is smaller.

[0075] In this embodiment, a hyperspectral image generation device is also provided, which is used to implement the above embodiments and implementation methods, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0076] This embodiment provides a hyperspectral image generation device, such as Figure 7 As shown, including:

[0077] An image acquisition module 501 is used to acquire a target acquisition image, wherein the target acquisition image is determined based on an image acquisition device, wherein a filter modulation device is provided in front of a lens of the image acquisition device, and the filter modulation device includes at least one modulation filter;

[0078] The mask processing module 502 is used to determine a periodic sampling unit, and perform mask processing on the target acquisition image based on the periodic sampling unit to obtain a coded image, wherein the size of the periodic sampling unit is positively correlated with the number of the modulation filters;

[0079] The image generation module 503 is configured to input the encoded image into a preset image generation model to generate a target hyperspectral image based on the preset image generation model.

[0080] In some optional embodiments, the modulation filter is prepared from at least two thin film materials in a preset arrangement, and the refractive indices of the thin film materials are different; when the number of modulation filters is greater than one, the thickness of the intermediate layer in each of the modulation filters is different.

[0081] In some optional implementations, the mask processing module 502 includes:

[0082] A sampling determination unit, used to determine the periodic sampling unit of the adaptive coding array based on the number of modulation filters in the filtering and modulation device;

[0083] The mask processing unit is used to perform mask processing on the target captured image based on the periodic sampling unit to obtain a coded image.

[0084] In some optional implementations, the image generation module 503 includes:

[0085] A first processing unit is used to perform convolution processing on the encoded image to determine a first image, wherein the dimension of the first image is consistent with the target hyperspectral image;

[0086] The second processing unit is used to perform sampling processing on the first image based on the cumulative attention module in the preset image generation model, and output a target hyperspectral image.

[0087] In some optional embodiments, the cumulative attention module includes a spatial attention submodule, a mask attention submodule and a spectral attention submodule, and the second processing unit includes:

[0088] a downsampling subunit, configured to downsample the first image based on a first preset number of cumulative attention modules to obtain a second image;

[0089] An upsampling subunit is used to upsample the second image based on a second preset number of cumulative attention modules, and output a target hyperspectral image based on a third preset number of cumulative attention modules.

[0090] In some optional embodiments, the device further comprises:

[0091] An original image acquisition module, used to acquire an original image acquired by an original image acquisition device that is not provided with the filtering and modulation device;

[0092] A comparison image determination module, configured to determine a comparison image based on the target hyperspectral image and a coding array in the coding image;

[0093] A difference determination module is used to calculate the image difference based on the comparison image and the original image, so as to optimize the preset image generation model according to the calculation result.

[0094] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0095] The hyperspectral image generation device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0096] The embodiment of the present invention also provides a computer device having the above Figure 7 The hyperspectral image generating device shown.

[0097] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0098] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0099] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0100] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0102] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0103] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0104] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0105] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for generating a hyperspectral image, characterized in that: The method comprises: Acquire a target acquisition image, wherein the target acquisition image is determined based on an image acquisition device, wherein a filter modulation device is provided in front of a lens of the image acquisition device, and the filter modulation device includes at least one modulation filter; Determine a periodic sampling unit, and perform mask processing on the target acquisition image based on the periodic sampling unit to obtain a coded image, wherein the size of the periodic sampling unit is positively correlated with the number of the modulation filters; The encoded image is input into a preset image generation model to generate a target hyperspectral image based on the preset image generation model.

2. The method according to claim 1, characterized in that The modulation filter is made of at least two thin film materials in a preset arrangement, and the refractive indices of the thin film materials are different; when the number of the modulation filters is greater than one, the thickness of the intermediate layer in each of the modulation filters is different.

3. The method according to claim 2, characterized in that The determining of the periodic sampling unit and performing mask processing on the target acquired image based on the periodic sampling unit to obtain the encoded image includes: Determine the periodic sampling unit of the adaptive coding array based on the number of the modulation filters in the filtering and modulation device; The target captured image is subjected to mask processing based on the periodic sampling unit to obtain a coded image.

4. The method according to claim 1, characterized in that The step of inputting the encoded image into a preset image generation model to generate a target hyperspectral image based on the preset image generation model includes: Performing convolution processing on the encoded image to determine a first image, wherein the dimension of the first image is consistent with the target hyperspectral image; The first image is sampled based on a cumulative attention module in a preset image generation model to output a target hyperspectral image.

5. The method according to claim 4, characterized in that The cumulative attention module includes a spatial attention submodule, a mask attention submodule and a spectral attention submodule. The sampling process of the first image based on the cumulative attention module to output a target hyperspectral image includes: Downsampling the first image based on a first preset number of cumulative attention modules to obtain a second image; The second image is upsampled based on a second preset number of cumulative attention modules, and a target hyperspectral image is output based on a third preset number of cumulative attention modules.

6. The method according to claim 1, characterized in that The method further comprises: Acquire an original image captured by an original image capture device that is not provided with the filtering and modulation device; determining a comparison image based on the target hyperspectral image and the coded array in the coded image; An image difference is calculated based on the comparison image and the original image, so as to optimize the preset image generation model according to the calculation result.

7. A hyperspectral image generation device, characterized in that: The device comprises: An image acquisition module, used to acquire a target acquisition image, wherein the target acquisition image is determined based on an image acquisition device, wherein a filter modulation device is provided in front of a lens of the image acquisition device, and the filter modulation device includes at least one modulation filter; A mask processing module, used for determining a periodic sampling unit, and performing mask processing on a target acquisition image based on the periodic sampling unit to obtain a coded image, wherein the size of the periodic sampling unit is positively correlated with the number of the modulation filters; The image generation module is used to input the encoded image into a preset image generation model to generate a target hyperspectral image based on the preset image generation model.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the hyperspectral image generation method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the hyperspectral image generation method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the hyperspectral image generation method according to any one of claims 1 to 6.

Images