A multispectral camera based on computational imaging

Abstract

The present invention relates to a multispectral camera based on computational imaging. The multispectral camera includes: a light source, a structured light generating device, a lens group, a multispectral detector, a data acquisition device, and a data processing device arranged in sequence, or the lens group, the structured light generating device, the multispectral detector, the data acquisition device, and the data processing device arranged in sequence. By using computational imaging algorithms, the present invention can achieve spatial resolution using a non-array detector without spatial resolution ability, avoiding the fabrication and use of array detectors. Therefore, multiple non-array narrowband single-wavelength detectors can be used for spectral resolution, and the cost can be greatly reduced. At the same time, by designing non-array narrowband single-wavelength detectors to achieve multi-wavelength detection for a single device, the use of additional beam splitting devices and filter devices can be avoided, solving the problems of imaging speed, volume, and the number of spectra.

Description

Technical Field

[0001] The present invention belongs to the technical field of multispectral imaging, and relates to a multispectral camera based on computational imaging. Background Art

[0002] Multispectral imaging technology is an imaging technology proposed in the 1970s. By using a spectral separation element to divide the incident full-band / wide-band optical signal into several bands, it can simultaneously obtain the spectral characteristics and spatial image information of the detected target. Therefore, this imaging technology has broad application prospects in cutting-edge scientific research exploration (such as lunar surface mapping), military applications (such as pre-war reconnaissance), biomedical applications (such as tumor detection), etc.

[0003] However, existing multispectral cameras have one or more of the following disadvantages: 1. High cost; 2. Slow imaging speed, and it is difficult to achieve spatio-temporal consistency for each spectral image; 3. Large volume; 4. Small number of spectra.

[0004] Multispectral imaging is divided into active and passive types. The active type uses a light source on the camera to illuminate the imaging object, and uses an array detector (such as a charge-coupled device (CCD), complementary metal oxide semiconductor (CMOS)) to obtain the spatial information of the object and achieve the photography function. The passive type uses natural light such as sunlight (compound light, without spectral information) to illuminate the imaging object, and also uses an array detector to obtain the spatial information of the object. In order to improve the resolution, the array detector usually has more than one million pixels, so the process is complex and the cost is high. The silicon detectors used in general mobile phones and cameras have a much lower cost due to large production scale and mature technology. However, to achieve spectral resolution, the cost of fabricating a narrow-band single-wavelength detector using an array detector based on other materials is extremely high. If a narrow-band single-wavelength detector is not used, to obtain spectral information, the compound light needs to be split or filtered into monochromatic light. The splitting or filtering device can be placed at the light source or at the array detector. Common splitting devices are prisms, gratings, etc. This device can achieve continuous splitting, so the number of spectra is relatively large, but the volume is relatively large, and it is necessary to scan another dimension (y) except for the wavelength dimension (λ) and the length dimension (x), so the imaging speed is slow; common filtering devices are rotating filters, linear gradient filters, acousto-optic tunable filters, etc. The number of available spectra of the filtering device is small, and it takes a long time to switch the filtering wavelength (i.e., λ wavelength dimension scanning), so the imaging speed is slow, and it is difficult to achieve spatio-temporal consistency for each spectral image.

[0005] Therefore, how to simultaneously achieve a multispectral camera with low cost, fast imaging speed, small volume, and a large number of spectra has become a key problem to be solved urgently. Summary of the Invention

[0006] To solve the above problems existing in the prior art, the present invention provides a multispectral camera based on computational imaging. The technical problems to be solved by the present invention are achieved through the following technical solutions:

[0007] An embodiment of the present invention provides a multispectral camera based on computational imaging. The multispectral camera includes: a light source, a structured light generating device, a lens group, a multispectral detector, a data acquisition device, and a data processing device arranged in sequence, or the lens group, the structured light generating device, the multispectral detector, the data acquisition device, and the data processing device arranged in sequence.

[0008] In an embodiment of the present invention, when the multispectral camera is active:

[0009] The light source is used to emit uniform light;

[0010] The structured light generating device is used to convert the uniform light into structured light;

[0011] The lens group is used to focus the structured light onto the object to be photographed;

[0012] The multispectral detector is used to convert the light reflected by the object to be photographed into an optoelectronic signal;

[0013] The data acquisition device is used to acquire the optoelectronic signal converted by the multispectral detector;

[0014] The data processing device is used to restore the optoelectronic signal acquired by the data acquisition device into a two-dimensional or three-dimensional image.

[0015] In an embodiment of the present invention, when the multispectral camera is passive, natural light irradiates the object to be photographed to obtain diffuse reflected light:

[0016] The lens group is used to focus the diffuse reflected light onto the structured light generating device;

[0017] The structured light generating device is used to convert the diffuse reflected light into structured light;

[0018] The multispectral detector is used to convert the structured light into an optoelectronic signal;

[0019] The data acquisition device is used to acquire the optoelectronic signal converted by the multispectral detector;

[0020] The data processing device is used to restore the optoelectronic signal acquired by the data acquisition device into a two-dimensional or three-dimensional image.

[0021] In one embodiment of the present invention, the structured light generating device includes a spatial light modulator, a digital micromirror array, or a light source array.

[0022] In one embodiment of the present invention, the structured light includes Hadamard basis light, Fourier basis light, or machine spot.

[0023] In one embodiment of the present invention, the multispectral detector includes a self-filtering multi-wavelength narrowband detector or a multi-wavelength narrowband detector array.

[0024] In one embodiment of the present invention, the self-filtering multi-wavelength narrowband detector includes n groups of detectors stacked in sequence from top to bottom. The n groups of detectors are used to detect light of n different wavelengths, and each group of detectors includes a first electrode, a photosensitive layer, and a second electrode stacked in sequence from top to bottom.

[0025] In one embodiment of the present invention, both the first electrode and the second electrode are transparent electrodes.

[0026] In one embodiment of the present invention, the data acquisition device includes a multi-channel data acquisition device.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] The present invention is a multispectral camera based on computational imaging. By using computational imaging algorithms, spatial resolution can be achieved using a non-array detector without spatial resolution ability, avoiding the fabrication and use of array detectors. Therefore, multiple non-array narrowband single-wavelength detectors can be used for spectral resolution, and the cost brought by using multiple narrowband single-wavelength array detectors in the prior art can be greatly reduced. At the same time, by designing a non-array narrowband single-wavelength detector to achieve multi-wavelength detection of a single device, additional spectroscopic devices and filtering devices can be avoided, solving the problems of imaging speed, volume, and number of spectra; or by using different non-array narrowband single-wavelength detectors in parallel, the same function can also be achieved.

[0029] Through the following detailed description with reference to the accompanying drawings, other aspects and features of the present invention become apparent. However, it should be understood that the drawings are only designed for the purpose of explanation and not as a limitation of the scope of the present invention, as it should refer to the appended claims. It should also be understood that, unless otherwise indicated, the drawings are not necessarily drawn to scale and are only intended to conceptually illustrate the structures and processes described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a schematic structural diagram of an active multispectral camera based on computational imaging provided by an embodiment of the present invention;

[0031] Figure 2It is a schematic structural diagram of a passive multispectral camera based on computational imaging provided by an embodiment of the present invention;

[0032] Figure 3 It is a schematic structural diagram of a self-filtering multispectral detector provided by an embodiment of the present invention;

[0033] Figure 4 It is a schematic diagram of the hardware and software processing process provided by an embodiment of the present invention. Detailed implementation manners

[0034] The following further describes the present invention in detail with reference to specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0035] Embodiment 1

[0036] An embodiment of the present invention provides a multispectral camera based on computational imaging. The multispectral camera includes: a light source, a structured light generating device, a lens group, a multispectral detector, a data acquisition device, and a data processing device arranged in sequence, or a lens group, a structured light generating device, a multispectral detector, a data acquisition device, and a data processing device arranged in sequence.

[0037] In a specific embodiment, please refer to Figure 1 , this embodiment provides an active multispectral camera based on computational imaging. When the multispectral camera is active:

[0038] The light source is used to emit uniform light;

[0039] The structured light generating device is used to convert the uniform light into structured light;

[0040] The lens group is used to focus the structured light onto the object to be photographed;

[0041] The multispectral detector is used to convert the light reflected by the object to be photographed into an optoelectronic signal;

[0042] The data acquisition device is used to acquire the optoelectronic signal converted by the multispectral detector;

[0043] The data processing device is used to restore the optoelectronic signal acquired by the data acquisition device into a two-dimensional or three-dimensional image.

[0044] Specifically, the active multispectral camera uses the light source on the camera to illuminate the imaging object (i.e., the object to be photographed). The uniform light emitted by the light source becomes structured light after passing through the structured light generating device, and is projected onto the imaging object after passing through the lens group. The image signal is obtained by using the multispectral detector and the data acquisition device, and the object space information is obtained by using the computational imaging algorithm in the data processing device to realize the photographing function.

[0045] In a specific embodiment, please refer toFigure 2 , this embodiment provides a passive multispectral camera based on computational imaging. When the multispectral camera is passive, natural light irradiates the object to be photographed, and diffused reflection light is obtained:

[0046] A lens group for focusing the diffused reflection light onto the structured light generating device;

[0047] A structured light generating device for converting the diffused reflection light into structured light;

[0048] A multispectral detector for converting the structured light into an optoelectronic signal;

[0049] A data acquisition device for acquiring the optoelectronic signals converted by the multispectral detector;

[0050] A data processing device for restoring the optoelectronic signals acquired by the data acquisition device into a two-dimensional or three-dimensional image.

[0051] Specifically, the passive multispectral camera uses natural light such as sunlight to illuminate the imaging object. The obtained diffused reflection light is focused by a lens group (i.e., the lens), and then becomes structured light through the structured light generating device. Then, an image signal is obtained using the multispectral detector and the data acquisition device, and the object space information is obtained using computational imaging algorithms in the data processing device to achieve the photographing function.

[0052] In this embodiment, the light source can emit uniform light, similar to the flash of a traditional camera.

[0053] In this embodiment, for the active type, the function of the structured light generating device is to convert uniform light into structured light. For the passive type, the function of the structured light generating device is to convert diffused reflection light into structured light. The structured light generating device can be a spatial light modulator, a digital micromirror array, or an electrically controllable light source array, such as an LED array.

[0054] Preferably, the structured light can be a set of Hadamard basis lights, or Fourier basis lights, or random light spots, or light spots of other bases.

[0055] In this embodiment, the lens group is a device for focusing light of a matching wavelength (i.e., the lens). In an active multispectral camera, the lens group can be the projection lens of a projector, or the lens of a telescopic system, or a single convex lens. In a passive spectral camera, the lens group can be a traditional camera lens, or a single convex lens.

[0056] In this embodiment, the function of the multispectral detector is to convert light of each wavelength into corresponding optoelectronic signals.

[0057] Preferably, the multispectral detector is a self-filtering multi-wavelength narrowband detector.

[0058] Furthermore, the multispectral detector includes n groups of detectors stacked in sequence from top to bottom. The n groups of detectors are used to detect light of n different wavelengths. Each group of detectors includes a first electrode, a photosensitive layer, and a second electrode stacked in sequence from top to bottom. Please refer to Figure 3 , the first group of detectors includes electrode 1, photosensitive layer 1, and electrode 2, the second group of detectors includes electrode 3, photosensitive layer 2, and electrode 4, and so on. The nth group of detectors includes electrode 2n - 1, photosensitive layer n, and electrode 2n. Light irradiates from above. The shortest wavelength part is absorbed by photosensitive layer 1 and converted into an optoelectronic signal, and the longer wavelength part passes through; the second shortest wavelength part is absorbed by photosensitive layer 2 and converted into an optoelectronic signal, and the longer wavelength part passes through, and so on. Electrodes 1 - 2n are all transparent electrodes. Connecting electrode 1 and electrode 2 forms detector 1, connecting electrode 3 and electrode 4 forms detector 2, and so on. Therefore, detectors 1 - n detect different wavelengths respectively. In addition, at this time, there may be an incomplete absorption phenomenon of the short wavelength part. For example, there is still a small amount of the shortest wavelength light reaching detector 2. At this time, a weighted superposition algorithm needs to be performed before calculating the imaging algorithm to ensure that detectors 1 - n detect different wavelengths respectively. Photosensitive layers 1 - n can be made of different materials, or they can be of the same series of materials with different ratios. If they are of different materials, for example, MAPbCl3, MAPbBr3, MAPbI3 can be selected. If they are of the same series of materials, for example, MAPbBr x Cl 3-x (methyl cesium lead bromide), photosensitive layers 1 - n can respectively select MAPbBr3,…MAPbBr x Cl 3-x ,…MAPbCl3, that is, the x of photosensitive layers 1 - n gradually decreases. The different wavelength light detected by the self-filtering multi-wavelength narrowband detector comes from the same optical path, and the spatio-temporal consistency of the multi-spectral image output by the camera can be ensured under the premise of being collected simultaneously.

[0059] In addition, the multispectral detector is preferably a multi-wavelength narrowband detector array, that is, n narrowband detector arrays are closely stacked together, that is, an array composed of n stacked narrowband detectors, which can ensure temporal consistency.

[0060] In this embodiment, the data acquisition device is a multi-channel data acquisition device that acquires the optoelectronic signals collected by the multispectral detector, and the number of channels matches the spectral quantity of the multispectral detector.

[0061] In this embodiment, the data processing device functions to restore the collected optoelectronic signals into two-dimensional or three-dimensional images using computational imaging algorithms. It can be a computer or other programmable circuits. The computational imaging algorithm is the corresponding algorithm for matching structured light. For example, if the structured light is Fourier basis light, the corresponding algorithm is the inverse Fourier algorithm. For other different types of structured light, corresponding algorithms can be selected accordingly, which will not be elaborated here.

[0062] The present invention is a multispectral camera based on computational imaging. Using computational imaging algorithms, spatial resolution can be achieved using non-array detectors without spatial resolution capabilities, avoiding the fabrication and use of array detectors. Therefore, multiple non-array narrowband single-wavelength detectors can be used for spectral resolution, and the cost brought by using multiple narrowband single-wavelength array detectors in the prior art can be greatly reduced. At the same time, by designing non-array narrowband single-wavelength detectors, multi-wavelength detection of a single device can be achieved, avoiding the use of additional beam splitting devices and filter devices, and solving the problems of imaging speed, volume, and the number of spectra; or different non-array narrowband single-wavelength detectors can be used side by side to achieve the same function.

[0063] Embodiment 2

[0064] This embodiment of the present invention specifically illustrates the invention using the active photographic mode and Fourier basis light as an example. The hardware and software processing procedures are as shown in the appendix Figure 4 as follows.

[0065] First, a white light source emits uniformly distributed outgoing light, which is irradiated onto a spatial light modulator through an expanding lens group to generate structured light. A spatial light modulator refers to a device that, under active control, can modulate a certain parameter of the light field through liquid crystal molecules. For example, it can modulate the amplitude of the light field, modulate the phase through the refractive index, modulate the polarization state through the rotation of the polarization plane, or achieve the conversion of incoherent-coherent light, thereby writing certain information into the light wave to achieve the purpose of light wave modulation. The spatial light modulator with the model number V-6501VIS is composed of 1920×1080 digital micromirrors with a size of 7.56μm and can achieve the modulation of light in the range of 400 - 2200nm. The multispectral detector is equivalent to a camera and can convert the light signal into an electrical signal for processing. Subsequently, the optoelectronic signals detected by the photodetector are collected by the data source table (data acquisition device) and the computer. The data source table has a relatively high acquisition accuracy and is very suitable for the feature analysis of optoelectronic signals. However, the acquisition speed of the data source table is relatively slow, only 50Hz. A multi-channel high-speed acquisition card can be purchased according to actual needs to meet the imaging speed requirements for imaging. It can be attempted to use multiple (m) groups of detectors and use the multi-channel averaging method to reduce noise. Therefore, the requirement for the multi-channel high-speed acquisition card is greater than or equal to mn.

[0066] After obtaining the required optoelectronic signals, the inverse Fourier transform method is to be used to reconstruct individual spectral images of photoacoustic images, and then an image fusion algorithm is used to synthesize high-quality multispectral images (data cubes). The software of this imaging system is written in the LabVIEW program. After the software runs, the imaging parameters are first set, including light intensity, imaging wavelength, etc., and at the same time the image matrix is initialized (set to zero). Then, the spatial light modulator is controlled to change the light intensity distribution of the structured light (i.e., change the fringes). At this time, a set of optoelectronic signals of the photodetector are collected and stored in the data register (I(m)). Then, it is judged whether the specified number of samplings is completed. If not, it returns to the step of controlling the spatial light modulator and loops in turn; if so, the instrument is turned off.

[0067] Subsequently, the data in the data register is normalized (four-step phase shift and averaging) to calculate the real and imaginary parts of the Fourier coefficients (α Re and α Im ), and then through the inverse Fourier transform in space (IFT), it is projected onto the corresponding positions of the image matrix to form a monochromatic image. At this time, the option can be selected to display the image on the screen, store it in the image register and display it, and the software exits to complete the photographing process.

[0068] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0069] In the description of this specification, the description with reference to terms such as "an embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristic data points described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristic data points described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0070] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A multispectral camera based on computational imaging, characterized in that, The multispectral camera includes: a light source, a structured light generating device, a lens group, a multispectral detector, a data acquisition device, and a data processing device arranged in sequence, or the lens group, the structured light generating device, the multispectral detector, the data acquisition device, and the data processing device arranged in sequence; Among them, the multispectral detector includes a self-filtering multi-wavelength narrowband detector or a multi-wavelength narrowband detector array; the self-filtering multi-wavelength narrowband detector includes n groups of detectors stacked layer by layer from top to bottom, and the n groups of detectors are used to detect light of n different wavelengths. Each group of detectors includes a first electrode, a photosensitive layer, and a second electrode stacked layer by layer from top to bottom; Among them, if the preparation materials of each photosensitive layer are different, the materials of the photosensitive layer include: MAPbCl3, MAPbBr3, MAPbI3; if the preparation materials of each photosensitive layer are the same, the materials of the photosensitive layer include MAPbBr x Cl 3-x , and the value of x in the photosensitive layer with a longer absorption wavelength is smaller.

2. The multispectral camera based on computational imaging according to claim 1, wherein When the multispectral camera is active: The light source is used to emit uniform light; The structured light generating device is used to convert the uniform light into structured light; The lens group is used to focus the structured light onto the object to be photographed; The multispectral detector is used to convert the light reflected by the object to be photographed into an optoelectronic signal; The data acquisition device is used to acquire the optoelectronic signal converted by the multispectral detector; The data processing device is used to restore the optoelectronic signal acquired by the data acquisition device into a two-dimensional or three-dimensional image.

3. The multispectral camera based on computational imaging according to claim 1, wherein When the multispectral camera is passive, natural light irradiates the object to be photographed to obtain diffuse reflected light: The lens group is used to focus the diffuse reflected light onto the structured light generating device; The structured light generating device is used to convert the diffuse reflected light into structured light; The multispectral detector is used to convert the structured light into an optoelectronic signal; The data acquisition device is used to acquire the optoelectronic signal converted by the multispectral detector; The data processing device is used to restore the optoelectronic signal acquired by the data acquisition device into a two-dimensional or three-dimensional image.

4. The multispectral camera based on computational imaging according to claim 1, wherein, The structured light generating device includes a spatial light modulator, a digital micromirror array, or a light source array.

5. The multispectral camera based on computational imaging according to claim 1, wherein The structured light includes Hadamard basis light, Fourier basis light, or machine spots.

6. The multispectral camera based on computational imaging according to claim 1, wherein Both the first electrode and the second electrode are transparent electrodes.

7. The multispectral camera based on computational imaging according to claim 1, wherein, The data acquisition device includes a multi-channel data acquisition device.

Images