Snapshot multi-spectral imaging system and its manufacturing method

Abstract

The present invention discloses a snapshot multi-spectral imaging system and a manufacturing method thereof, relating to the technical field of image processing, including: providing a sensor, the sensor including a plurality of pixels arranged in an array; manufacturing a spectral filter array on one side of the sensor, the spectral filter array including a plurality of filter unit arrays, the filter unit including a plurality of filters arranged in an array, the filters corresponding to the pixels one by one, the filters being random broadband filters, each filter in the filter unit corresponding to a spectral response curve, and the filters being manufactured according to an optimized spectral response curve. The present invention has a higher spectral resolution compared with the existing snapshot multi-spectral imaging system and can generate images of more spectral bands.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a snapshot multi-spectral imaging system and a manufacturing method thereof. Background Art

[0002] Compared with traditional RGB color images, multispectral images have more spectral bands and more spectral information. They are widely used in remote sensing image processing, medical image analysis, food quality testing, authenticity detection, etc.

[0003] The existing multispectral imaging technologies mainly include spatial scanning, spectral scanning and snapshot spectral imaging. Among them, spatial scanning and spectral scanning can obtain more spectral segment information by sacrificing temporal resolution in exchange for spectral resolution, but the imaging speed is slow and the spectral information in the field of view cannot be dynamically obtained. If the scene changes during the imaging process, it will affect the generated image; the snapshot spectral camera samples the spectral information in space through a multispectral array (MSFA) and records it on the sensor, and generates a multispectral image by rearranging the pixels. However, the current snapshot spectral camera mainly has the following two problems: 1. The spatial resolution and spectral resolution restrict each other. The spectral resolution increases and the spatial resolution decreases; 2. The number of spectral segments of the generated multispectral image is consistent with the number of narrow-band filters used in MSFA (generally not more than 25), which limits the spectral resolution of the generated multispectral image.

[0004] Therefore, the defects in the prior art should be improved to improve the spectral resolution of multispectral images. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a snapshot multispectral imaging system and a manufacturing method thereof. The technical problem to be solved by the present invention is achieved by the following technical solutions:

[0006] In a first aspect, the present invention provides a method for manufacturing a snapshot multispectral imaging system, comprising:

[0007] Providing a sensor, the sensor comprising a plurality of pixels arranged in an array;

[0008] A spectral filter array is made on one side of the sensor. The spectral filter array includes a plurality of filter units arranged in an array. The filter unit includes a plurality of filters arranged in an array. The filters correspond to pixels one by one. The filters are random broadband filters. Each filter in the filter unit corresponds to a spectral response curve. The filters are made according to the optimized spectral response curve.

[0009] In a second aspect, the present application also provides a snapshot multispectral imaging system, comprising:

[0010] A sensor, the sensor including a plurality of pixels arranged in an array;

[0011] A spectral filter array, located on one side of the sensor, the spectral filter array including a plurality of filter element units arranged in an array, each filter element unit including a plurality of filters arranged in an array, the filters corresponding to the pixels one by one, the filters being random broadband filters, each filter in the filter element unit corresponding to a spectral response curve, and the filters being fabricated according to an optimized spectral response curve.

[0012] Advantages of the present invention:

[0013] A snapshot multi-spectral imaging system and a manufacturing method thereof provided by the present invention include a sensor and a spectral filter array. The sensor includes a plurality of pixels arranged in an array. The spectral filter array includes a plurality of filter element units arranged in an array. Each filter element unit includes a plurality of filters arranged in an array. The filters correspond to the pixels one by one. The filters are random broadband filters. Each filter corresponds to a spectral response curve, and filters of different colors correspond to their respective spectral response curves. The filters in the present invention are fabricated according to an optimized spectral response curve, so that the obtained multi-spectral image is not limited by the number of filters in the multi-spectral array. Compared with the existing snapshot multi-spectral imaging system, it has higher spectral resolution and can generate images of more spectral bands.

[0014] The following will further elaborate on the present invention in conjunction with the accompanying drawings and embodiments. Description of the Drawings

[0015] Figure 1 is a flowchart of a manufacturing method of a snapshot multi-spectral imaging system provided by an embodiment of the present invention;

[0016] Figure 2 is a schematic structural diagram of a snapshot multi-spectral imaging system provided by an embodiment of the present invention;

[0017] Figure 3 is a schematic structural diagram of a filter element unit provided by an embodiment of the present invention;

[0018] Figure 4 is a schematic structural diagram of a spectral response curve provided by an embodiment of the present invention;

[0019] Figure 5 is a schematic structural diagram of a filter provided by an embodiment of the present invention;

[0020] Figure 6 is a schematic structural diagram of an empty spectrum reconstruction network provided by an embodiment of the present invention. Detailed Embodiments

[0021] The present invention will be further described in detail below with reference to specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0022] In the prior art, Mosaic adopted the patented technology of mosaic pixel coating to achieve a single-lens, snapshot multi-spectral imaging solution. Moreover, this technology was first commercially implemented on an InGaAs photosensor, and the working range of the snapshot multi-spectral camera was also extended to 1.65 μm. This technology was achieved by periodically coating a Fabry-Perot micro-interference filter array on a 640*480 pixel resolution InGaAs sensor in a 3*3 or 4*4 small array mode. The wavelengths transmitted by the micro-filters corresponding to each pixel within each independent small array varied between 1.1 μm and 1.7 μm. When outputting data, by extracting and rearranging the pixels with the same wavelength within the small array, a 9-channel or 16-channel spectral image cube could be segmented. Using the above method, on the one hand, an increase in the number of spectral bands would lead to a decrease in image resolution; on the other hand, since narrow-band filters were used for filtering, the number of bands of the generated multi-spectral image was limited by the number of filters. Wide-band filters could sample the entire spectrum, and with the cooperation of a reconstruction algorithm, more bands could be reconstructed with fewer filters.

[0023] Generally speaking, in the prior art, the following defects exist: 1. The existing snapshot spectral imaging system only simply extracts and rearranges the pixels on the sensor. The spatial resolution of the image in each band in the generated multi-spectral image cube is the size of the sensor divided by the number of bands, and its spatial resolution is small; 2. The number of bands of the multi-spectral image generated by the existing snapshot spectral imaging system is limited by the filters in the MSFA, resulting in fewer spectral segments in the generated multi-spectral image (generally not exceeding 25); 3. The filters used in the MSFA adopted by the existing snapshot spectral imaging system are narrow-band spectral filters, and a large amount of spectral information is lost during the sampling process, and the effect after image reconstruction is poor.

[0024] In view of this, the present invention provides a method for manufacturing a snapshot multi-spectral imaging system. A snapshot multi-spectral imaging system is constructed based on a deep neural network algorithm, mainly for reconstructing the filters included in the spectral filter array, which can enable the spectral resolution of the generated multi-spectral image to be not limited by the number of spectral filter plates in the MSFA, and more spectral segments can be reconstructed with fewer filters, and a multi-spectral image with a higher spatial resolution is generated.

[0025] Please refer to Figures 1 to 4 shown Figure 1 is a flowchart of a method for manufacturing a snapshot multi-spectral imaging system provided by an embodiment of the present invention, Figure 2 is a schematic structural diagram of a snapshot multi-spectral imaging system provided by an embodiment of the present invention, Figure 3It is a schematic structural diagram of a filter unit provided by an embodiment of the present invention. Figure 4 It is a schematic structural diagram of a spectral response curve provided by an embodiment of the present invention. A method for manufacturing a snapshot multi-spectral imaging system provided by the present invention includes:

[0026] Providing a sensor, the sensor including a plurality of pixels arranged in an array;

[0027] Fabricating a spectral filter array on one side of the sensor, the spectral filter array including a plurality of filter units arranged in an array, the filter unit including a plurality of filters arranged in an array, the filters corresponding to the pixels one by one, the filters being random broadband filters, each filter in the filter unit corresponding to a spectral response curve, and the filters being fabricated according to an optimized spectral response curve.

[0028] Specifically, please continue to refer to Figures 1 to 4 As shown, a method for manufacturing a snapshot multi-spectral imaging system provided by the present invention includes a sensor and a spectral filter array. The sensor includes a plurality of pixels arranged in an array, the spectral filter array includes a plurality of filter units arranged in an array, the filter unit includes a plurality of filters arranged in an array, the filters correspond to the pixels one by one, the filters are random broadband filters, each filter corresponds to a spectral response curve. Optionally, the filter unit includes 9 filters, each filter corresponding to a spectral response curve. It can also be understood that filters of different colors correspond to their respective spectral response curves. The filters in this embodiment are fabricated according to an optimized spectral response curve, so that the obtained multi-spectral image is not limited by the number of filters in the multi-spectral array, has a higher spectral resolution compared with the existing snapshot multi-spectral imaging system, and can generate images of more spectral bands.

[0029] It should be noted that Figure 2 The embodiment shown only schematically shows the positional relationship between the sensor and the spectral filter array, and does not represent its actual size. Among them, the pixels in the sensor and the filters in the spectral filter array do not represent the actual size either. In addition, Figure 1 Only 9 filters included in the filter unit are schematically shown, and there may also be other numbers of filters, which are not limited in this application; Figure 3 The embodiment shown only schematically shows the arrangement of the 9 filters included in one filter unit; Figure 4 The embodiment shown only schematically shows the schematic diagram of different colors corresponding to different spectral response curves.

[0030] In an optional embodiment of the present invention, please refer to Figure 5 As shown, Figure 5 It is a schematic structural diagram of a filter provided by an embodiment of the present invention. The filter includes:

[0031] Silicon dioxide layer;

[0032] An aluminum film layer is located on one side of the silicon dioxide layer, and the aluminum film layer includes a plurality of square nanopores arranged in an array, and the square nanopores penetrate the aluminum film layer in a direction perpendicular to the filter; wherein the square nanopores are formed according to an optimized spectral response curve;

[0033] The silicon layer is located on the side of the aluminum film layer away from the silicon dioxide layer. The silicon layer fills the square nanoholes and covers the aluminum film layer.

[0034] It should be noted that Figure 5 The illustrated embodiment only schematically shows the positional relationship diagram of the silicon dioxide layer, the aluminum film layer and the silicon layer included in the filter, and does not represent the actual size thereof.

[0035] In an optional embodiment of the present invention, see Figure 6 As shown, Figure 6 Schematic diagram of a structure of a spatial spectrum reconstruction network provided by an embodiment of the present invention, and an optimized spectral response curve acquisition process:

[0036] S201 : Acquire an original spectral image of a target object, and generate an initial spectral response function corresponding to a filter included in a filter unit.

[0037] Specifically, in this embodiment, the original spectral image of the target object is collected by a spectrometer, that is, the original spectral information of the target object in 31 general bands within the range of 400nm to 700nm with a step size of 10nm is obtained, and an initial spectral response function corresponding to a group of filters is generated according to the number of filters included in the smallest filter unit in the spectral filter array. It can be understood that the initial response function is a random vector. Optionally, in this embodiment, the filter unit includes 9 filters, that is, 9 random vectors with a length of 31 are generated as 9 initial spectral response functions corresponding to the filter unit.

[0038] S202: Generate a multispectral Raw image using a spectral response network according to the original spectral image and the initial spectral response function of the target object.

[0039] S2021. Use the initial spectral response function as a 1*1 convolution kernel to convolve with the original spectral image of the target object to generate a spectrally encoded aliased sampling image corresponding to the filter.

[0040] Specifically, the 9 initial spectral response functions are used as 1*1 convolution kernels with a step size of 1, and are convolved with the original spectral image of the target object to generate 9 spectrally encoded aliased sampling images corresponding to the 9 filters.

[0041] In S2022, in the same filter unit, the pixel position of the filter itself is set to 1, and the remaining pixel positions are set to 0 to generate the first mask corresponding to the filter.

[0042] Specifically, according to the positions of the respective filters in the filter unit, a first mask is generated to retain the pixel position information corresponding to the filter in the MSFA and filter the remaining pixel position information.

[0043] In S2023, the spectral aliasing sampled image corresponding to the filter is multiplied pixel by pixel with the first mask to obtain the spectral aliasing sampled image.

[0044] In S2024, multiple spectral aliasing sampled images are added together to generate a multi-spectral Raw image.

[0045] In S203, based on the multi-spectral Raw image, an empty-spectrum reconstruction network is used to generate a reconstructed spectral image.

[0046] Specifically, the empty-spectrum reconstruction network includes a filter bank convolution module (FACB) and a residual block module; among them, the residual block module includes multiple residual blocks (ResBlock), and each residual block sequentially includes a first convolutional layer (Conv1), a first activation function (PReLU1), a second convolutional layer (Conv2), a second activation function (PReLU2), a third convolutional layer (Conv3), and a third activation function (PReLU3); optionally, in this embodiment, 6 residual blocks are set, and the number of input channels of the first convolutional layer, the second convolutional layer, and the third convolutional layer in the residual block is 32, and the first activation function, the second activation function, and the third activation function each include a learnable parameter α, and its initial value is set to 0.25.

[0047] In S2031, K convolutional kernels are initialized.

[0048] Specifically, K = 9, the size of the 9 convolutional kernels is 1*1, the stride is 1, and the pixels filled around the image are 1.

[0049] In S2032, the K convolutional kernels are subjected to a convolution operation with the multi-spectral Raw image to generate the convolved features corresponding to the K filters.

[0050] In S2033, in the same filter unit, the pixel position of the filter itself is set to 1, and the remaining pixel positions are set to 0 to generate the second mask corresponding to the filter.

[0051] In S2034, the convolved features corresponding to the filter are multiplied pixel by pixel with the second mask to obtain the processed features.

[0052] In S2035, the processed features are added together to generate the spectral image after feature extraction.

[0053] S2036. Process the spectral image after feature extraction using the residual block module to obtain the output spectral image.

[0054] S2037. Perform adaptive average pooling on the pixel positions of the filter itself and use the Sigmoid activation function for probability scaling to generate the filter feature unit.

[0055] S2038. After spatially repeating the arrangement of the filter feature units, use it as the third mask to multiply with the output spectral image pixel by pixel, and add the multiplication result to the spectral image after feature extraction to obtain the reconstructed spectral image.

[0056] S204. Use the Adam algorithm to optimize the reconstructed spectral image to obtain the parameter values of the convolutional kernel corresponding to the filter included in the filter unit.

[0057] Specifically, the expression of the Adam algorithm is:

[0058]

[0059] where \(W_1\) is the parameter in the spectral response network, \(W_2\) is the parameter in the empty spectral reconstruction network, is the original spectral image of the \(i\)-th target object, \(y\) i is the reconstructed spectral image of the \(i\)-th target object, and \(M\) is the total number of target objects.

[0060] It should be noted that when using the Adam algorithm for parameter optimization, the warm-up learning rate method is adopted. The learning rate is set to \(1e - 6\) in the first 100 rounds of training, and then the learning rate is increased to \(1e - 4\).

[0061] S205. Obtain the optimized spectral response function according to the parameter values of the convolutional kernel corresponding to the filter, and construct the optimized spectral response curve.

[0062] In summary, based on the random broadband filter, the multi-spectral filter array (MSFA), where the filters used are different from the narrowband filters in traditional snapshot spectrometers, can retain all spectral information within the spectral band, enabling the pixels on the sensor to collect the aliased information of multiple randomly sampled spectral bands. The original spectral information can be restored by using a reconstruction algorithm to decode it; the spectral response function of the broadband filter based on a neural network is generated by the neural network and optimized using an optimization algorithm, eliminating the need for designers to manually generate and screen, and can generate the optimal filter response curve, making the reconstructed spectrum more fitting to the original spectrum; the spatio-spectral reconstruction algorithm based on a deep convolutional neural network is used to reconstruct the multi-spectral Raw image sampled by the MSFA with a random broadband filter, enabling the simultaneous reconstruction of both spectral and spatial resolutions to generate a multi-spectral image with high spatial and spectral resolutions; the joint optimization method of the spectral response curve in the filter array and the spatio-spectral reconstruction algorithm can simultaneously optimize the parameters of the spectral response network and the spatio-spectral reconstruction network based on the input of the original spectrum of the target object, synchronously optimizing the two to fit, achieving a one-step end-to-end design of the imaging system without distributed optimization, reducing the response operations and the non-global optimal problem during step-by-step optimization.

[0063] In an alternative embodiment of the present invention, the filter is formed according to the optimized spectral response curve and includes:

[0064] According to the optimized spectral response curve, nanoimprint lithography is performed on the aluminum film layer included in the filter to form square nanopores arranged in an array on the aluminum film layer.

[0065] Specifically, in this embodiment, when light irradiates the filter, due to the surface plasmon resonance at the Al-SiO2 and Al-Si interfaces, different transmittances can be observed at different wavelengths; the arrangement of the nano-air defense array is adjusted by the spectral response curve to make the spectral response of the filter fit the spectral response curve optimized by the spectral response network.

[0066] In an alternative embodiment of the present invention, the filter generated by the method provided by the present invention constitutes a spectral filter array. The spectral filter array is placed on an image sensor CMOS, and an optical system is built with optical components such as lenses to perform spectral aliasing encoding acquisition of the target object. An empty-spectrum reconstruction network is placed on a computing device such as a computer. A multi-spectral Raw image is obtained from the sensor through data transmission, and it is reconstructed into a multi-spectral image by the empty-spectrum reconstruction network. Through verification tests, by sampling with a spectral filter array having 9 different random broadband filters and reconstructing through the empty-spectrum reconstruction network, a multi-spectral image with 31 spectral bands can be generated, breaking through the limitation that the filter array with 9 filters in the existing snapshot spectral imaging system can only generate a multi-spectral image with 9 spectral bands.

[0067] Based on the same inventive concept, the present application also provides a snapshot multi-spectral imaging system, including:

[0068] A sensor, the sensor includes a plurality of pixels arranged in an array;

[0069] A spectral filter array, located on one side of the sensor. The spectral filter array includes a plurality of filter unit arrays. The filter unit includes a plurality of filters arranged in an array. The filters correspond to the pixels one by one. The filters are random broadband filters. Each filter corresponds to a spectral response curve, and the filters are made according to the optimized spectral response curve.

[0070] Specifically, in this embodiment, the snapshot multi-spectral imaging system is made by the method provided in the above embodiment. The multi-spectral image is reconstructed by a reconstruction algorithm. The spatial resolution is the same as the number of sensor pixels, and its spatial resolution is not limited by the spectral resolution and will not cause loss of spatial resolution due to an increase in spectral resolution.

[0071] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variants are intended to cover non-exclusive inclusion, so that an article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the article or device comprising the element. Similar words such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The orientation or positional relationship indicated by "up", "down", "left", "right", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation on the present invention.

[0072] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0073] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A manufacturing method of a snapshot multi-spectral imaging system, characterized in that, include: Providing a sensor, the sensor comprising a plurality of pixels arranged in an array; A spectral filter array is manufactured on one side of the sensor, the spectral filter array includes a plurality of filter units arranged in an array, the filter unit includes a plurality of filters arranged in an array, the filters correspond to the pixels one by one, the filters are random broadband filters, each filter in the filter unit corresponds to a spectral response curve, and the filters are manufactured according to the optimized spectral response curve; wherein the optimized spectral response curve acquisition process includes: Acquire an original spectral image of a target object, and generate an initial spectral response function corresponding to the filter included in the filter unit; Generate a multispectral Raw image using a spectral response network according to the original spectral image of the target object and the initial spectral response function; According to the multi-spectral Raw image, a spatial spectrum reconstruction network is used to generate a reconstructed spectral image; The reconstructed spectral image is optimized using the Adam algorithm to obtain a parameter value of a convolution kernel corresponding to the filter included in the filter unit; According to the parameter value of the convolution kernel corresponding to the filter, the optimized spectral response function is obtained, and an optimized spectral response curve is constructed.

2. The manufacturing method of the snapshot multi-spectral imaging system according to claim 1, characterized in that The filter comprises: Silicon dioxide layer; an aluminum film layer, located on one side of the silicon dioxide layer, the aluminum film layer comprising a plurality of square nanopores arranged in an array, the square nanopores penetrating the aluminum film layer in a direction perpendicular to the optical filter; wherein the square nanopores are formed according to the optimized spectral response curve; The silicon layer is located on a side of the aluminum film layer away from the silicon dioxide layer. The silicon layer fills the square nanoholes and covers the aluminum film layer.

3. The manufacturing method of the snapshot multi-spectral imaging system according to claim 1, characterized in that, The filter is formed according to the optimized spectral response curve and includes: According to the optimized spectral response curve, nanoimprint lithography is performed on the aluminum film layer included in the filter to form square nanoholes arranged in an array on the aluminum film layer.

4. The manufacturing method of the snapshot multi-spectral imaging system according to claim 1, characterized in that The step of generating a multispectral Raw image using a spectral response network according to the original spectral image of the target object and the initial spectral response function comprises: Use the initial spectral response function as the convolution kernel, and convolve it with the original spectral image of the target object to generate the spectral coding aliasing sampling image corresponding to the filter; In the same filter unit, the pixel position of the filter itself is set to 1, and the other pixel positions are set to 0, and a first mask corresponding to the filter is generated; Multiplying the spectrally coded aliased sampling image corresponding to the filter by the first mask pixel by pixel to obtain a spectrally aliased sampling image; The multiple spectral aliasing sampling images are added together to generate the multi-spectral Raw image.

5. The manufacturing method of the snapshot multi-spectral imaging system according to claim 1, wherein, The spatial spectrum reconstruction network includes a filter group convolution module and a residual block module; The residual block module includes a plurality of residual blocks, each of which includes a first convolutional layer, a first activation function, a second convolutional layer, a second activation function, a third convolutional layer and a third activation function in sequence; The generating a reconstructed spectral image by using a spatial spectrum reconstruction network according to the multi-spectral Raw image comprises: Initialization convolution kernels; Convolve convolution kernels with the multi-spectral Raw image to generate convolved features corresponding to the filters; In the same filter unit, the pixel position of the filter itself is set to 1, and the other pixel positions are set to 0, and a second mask corresponding to the filter is generated; Multiply the convolved features corresponding to the filter by the second mask pixel by pixel to obtain the processed features; Add up the processed features to generate the spectral image after feature extraction; Process the spectral image after feature extraction using a residual block module to obtain the output spectral image; Perform adaptive average pooling on the pixel positions of the filter itself and use the Sigmoid activation function for probability scaling to generate the filter feature unit; After spatially repeating the arrangement of the filter feature units, use it as the third mask to multiply with the output spectral image pixel by pixel, and add the multiplication result to the spectral image after feature extraction to obtain the reconstructed spectral image.

6. The manufacturing method of the snapshot multi-spectral imaging system according to claim 1, characterized in that The expression of the Adam algorithm is: ； in, are the parameters in the spectral response network, are the parameters in the empty spectrum reconstruction network, For the The original spectral image of the target object, For the The reconstructed spectral image of the target object, is the total number of target objects.

7. A snapshot multi-spectral imaging system, characterized in that, Including: A sensor, the sensor includes a plurality of pixels arranged in an array; A spectral filter array, located on one side of the sensor, the spectral filter array includes a plurality of filter units arranged in an array, each filter unit includes a plurality of filters arranged in an array, the filters correspond to the pixels one by one, the filters are random broadband filters, each filter in the filter unit corresponds to a spectral response curve, and the filters are made according to the optimized spectral response curve; wherein, the process of obtaining the optimized spectral response curve includes: Obtain the original spectral image of the target object and generate the initial spectral response function corresponding to the filters included in the filter unit; According to the original spectral image of the target object and the initial spectral response function, use a spectral response network to generate a multi-spectral Raw image; According to the multi-spectral Raw image, use a spatial spectrum reconstruction network to generate a reconstructed spectral image; Use the Adam algorithm to optimize the reconstructed spectral image to obtain the parameter values of the convolution kernel corresponding to the filters included in the filter unit; According to the parameter values of the convolution kernel corresponding to the filters, obtain the optimized spectral response function and construct the optimized spectral response curve.

Images