Hyperspectral camera and hyperspectral reconstruction method
By using electrochromic devices for spectral encoding and compression-sensing reconstruction in hyperspectral cameras, the problems of large size and high cost of traditional hyperspectral cameras are solved, and miniaturized and low-cost hyperspectral data recovery are achieved.
Patent Information
- Application Number
- CN202310538075.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
Traditional hyperspectral cameras rely on mechanical filters and complex tunable filters, resulting in large size, high cost and difficult to miniaturize, which cannot meet the needs of low-cost, easy-to-use and miniaturization.
The electrochromic device is used as the spectral encoding device, and the real-time spectral regulation capability under the action of external electric field is used to realize sampling and encoding of the spectra of the object to be measured, and the hyperspectral data is restored in combination with the compressed sensing reconstruction algorithm, reducing encoding redundancy and improving encoding capabilities.
The development of micro hyperspectral cameras has been realized, reducing costs, improving the device's preparation repeatability and stability, and is suitable for portable and consumer electronics applications.
Smart Images

Figure CN118936634A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of spectral analysis, and in particular to a hyperspectral camera and a hyperspectral reconstruction method. Background Art
[0002] Hyperspectral imaging technology combines spectral detection and spatial imaging to construct a three-dimensional data cube that associates one-dimensional spectral domain data with two-dimensional spatial domain data. This technology can detect the spectral information corresponding to each pixel while obtaining the spatial position and morphological information of the target object or scene. With its unified characteristics of image and spectrum, hyperspectral imaging technology has been widely used in fields such as materials science, environmental monitoring, military reconnaissance, and life medicine.
[0003] According to the different data acquisition methods, hyperspectral cameras can be divided into three categories. (I) Scanning hyperspectral camera: using linear array (or area array) detectors and dispersive elements, collecting all spectral domain data on a single (or single row) pixel each time, and obtaining all cubic data volumes through the swing of the scanning mirror and the movement of the carrying platform. Relatively speaking, this method is the slowest to obtain data. (II) Filtering hyperspectral camera: using an area array detector, using a set of narrow-band filters or tunable wavelength devices for filtering, collecting single spectral dimension data of all pixels corresponding to the spatial domain each time, and completing the collection of all data cubes through filter switching or wavelength tuning of the device. This principle takes into account both speed and resolution and has become one of the current mainstream technical solutions. (III) Snapshot hyperspectral camera: using an area array detector, encoding the target or scene in the spatial domain through special components or detection forms, and then decoding the detector collected data after dispersion by the subsequent optical system, the entire three-dimensional data cube can be collected. It has the fastest detection speed, but the spectral resolution is not high, and the data decoding speed is slow.
[0004] With the development of informatization and intelligence, the demand for low-cost, easy-to-use, miniaturized hyperspectral cameras in scientific research and industrial production is growing rapidly. These cameras are mainly used in spectral imaging components of portable detection instruments, distributed intelligent devices, and consumer electronic products.
[0005] The filter wheel or tunable filter (such as acousto-optic tunable filter, liquid crystal birefringence tunable filter) used in traditional filtering hyperspectral cameras decomposes the continuous wide spectrum into energy emission within a specific narrow-band wavelength range, and its spectral resolution depends on the number of narrow-band filtering; and the filter wheel is large in size due to the presence of mechanical devices and is not conducive to miniaturization, while the tunable filter has a high production cost due to its complex optical structure.
[0006] In recent years, thanks to the progress in computational imaging, signal processing and artificial intelligence, a new spectral reconstruction technology based on the principle of compressed sensing has emerged. This technology can encode the incident light in the spectral domain through a set of broadband filter elements or structures (such as broadband filters, quantum dots, metasurfaces, photonic crystals, etc.), and use the compressed sensing reconstruction algorithm to invert the full spectrum information of the target from the light response received by the detector. This method makes the spectroscopic element no longer limited to the principle of narrow-band filtering, and breaks the limitation of Nyquist sampling theorem. Its sampling and encoding times are often less than the number of restored spectral channels, which can effectively reduce the number of filters or the number of tunable filters.
[0007] In the field of materials science, the phenomenon that certain materials undergo a redox reaction under the action of an external electric field and undergo a reversible change in spectral characteristics (absorption rate, transmittance, reflectivity) is called the electrochromic effect. Devices composed of electrochromic materials show reversible changes in color and transparency when working. This device that can reversibly regulate its own wide spectrum provides an opportunity for the development of wide-spectrum filtering hyperspectral cameras based on the principle of compressed sensing spectral encoding. Summary of the invention
[0008] The present invention innovatively provides a hyperspectral camera and a hyperspectral reconstruction method. The present invention proposes for the first time to use an electrochromic device as a universal spectral encoding device, and utilizes the real-time spectral regulation capability of the electrochromic device under the action of an external electric field to achieve sampling and encoding of the spectrum of the object to be measured, thereby realizing the measurement matrix effect of the compressed sensing sampling process. The broadband transmission spectrum of the electrochromic device can cover the entire detection band, and the correlation between the transmission spectra of different color change states is low, which can improve the degree of differentiated response of each band between the spectrum of the object to be measured and the spectrum of the electrochromic device, reduce the degree of coding redundancy, and thus obtain higher coding capabilities with fewer electrochromic times. The simple electrochromic device structure can avoid the size problem caused by the mechanical filter wheel, reduce the processing complexity of the tunable filter, and improve the repeatability and stability of the device preparation. Therefore, the hyperspectral camera according to the present invention can be made into a miniature and low-cost, easy to commercialize.
[0009] According to a first aspect of the present invention, a hyperspectral camera is provided, comprising: an electrochromic coding component for spectrally encoding an object to be measured by transforming into different transmission spectral states under a variable voltage; a camera for recording an image of the object after passing through the electrochromic coding component in different states; and a computing processing unit for decoding the image to restore the hyperspectral data of the object.
[0010] In some optional embodiments, the camera includes a lens and a photodetector; the lens is used to collect spatial domain information of the object and focus it on the photodetector; and the photodetector is used to receive an optical signal contained in the spatial domain information.
[0011] In some optional embodiments, the electrochromic coding component includes an electrochromic device and a power driving device; the power driving device is used to apply a variable voltage to the electrochromic device; the transmission spectrum of the electrochromic device is continuously adjustable in the visible light to near infrared range according to the change of the applied voltage.
[0012] In some optional embodiments, the photodetector includes: an area array CCD image sensor and an area array CMOS image sensor, and the operating wavelength range of the photodetector includes visible light to near infrared.
[0013] In some optional embodiments, the electrochromic device is configured to change m transmission spectrum states, the camera is configured to capture an image in each transmission spectrum state of the electrochromic device, and the computing processing unit is configured to perform the following operations:
[0014] S1: constructing an equivalent measurement matrix Φ by cross-multiplying the quantum efficiency of the photodetector by each transmission spectrum of the electrochromic device;
[0015] S2: Select the aliased spectral data of a single pixel point at the same position in the image space domain in m images to form its spectral observation value y, where the spectral dimension is m, y = Φx, x is the spectral data of the pixel point, and set the reconstructed spectral dimension to n, where m and n are both positive integers and n ≥ m;
[0016] S3: Calculate the sampling rate m / n, and execute steps S4.1-S4.2 when the sampling rate is 1, and execute steps S5.1-S5.3 when the sampling rate is less than 1;
[0017] S4.1: Compute the generalized inverse matrix Φ of the equivalent observation matrix Φ -1 ;
[0018] S4.2: The generalized inverse matrix Φ -1 Multiply y on the left to obtain the reconstructed spectral data of the pixel point;
[0019] S5.1: construct a discrete cosine transform basis sparse dictionary matrix Ψ for sparsely representing the spectral data x of the pixel point, that is, x=Ψs, where s is the sparse coefficient vector of the spectral data in the discrete cosine transform basis;
[0020] S5.2: Let ΦΨ=Θ, Θ is the sensor matrix, then y=ΦΨs=Θs, when y and Θ are known, the sparse coefficient vector s is calculated by iterative update using the sparsity adaptive matching pursuit algorithm;
[0021] S5.3: Multiply the sparse coefficient vector s by the sparse dictionary matrix Ψ to obtain the reconstructed spectral data of the pixel point;
[0022] S6: after step S4.2 or S5.3, repeat steps S2-S3 for the remaining pixel points in the image space domain;
[0023] S7: Combine the reconstructed spectral data of all pixel points according to the spatial domain position to obtain a three-dimensional hyperspectral data cube.
[0024] In some optional embodiments, the hyperspectral camera is a miniature hyperspectral camera.
[0025] According to a second aspect of the present invention, there is provided a hyperspectral reconstruction method, comprising: spectrally encoding an object to be measured by applying a variable voltage to an electrochromic device to transform it into different transmittance spectral states; using a camera to record an image of the object after passing through the electrochromic device in different states; and decoding the image to restore the hyperspectral data of the object.
[0026] In some optional embodiments, the camera includes a lens and a photodetector; the lens is used to collect spatial domain information of the object and focus it on the photodetector; and the photodetector is used to receive an optical signal contained in the spatial domain information.
[0027] In some optional embodiments, the transmission spectrum of the electrochromic device is continuously adjustable in the range from visible light to near infrared according to changes in the applied voltage.
[0028] In some optional implementations, the electrochromic device is changed into m transmission spectrum states, and an image is captured by the camera in each transmission spectrum state of the electrochromic device, wherein decoding the image comprises the following operations:
[0029] S1: constructing an equivalent measurement matrix Φ by cross-multiplying the quantum efficiency of the photodetector by each transmission spectrum of the electrochromic device;
[0030] S2: Select the aliased spectral data of a single pixel point at the same position in the image space domain in m images to form its spectral observation value y, where the spectral dimension is m, y = Φx, x is the spectral data of the pixel point, and set the reconstructed spectral dimension to n, where m and n are both positive integers and n ≥ m;
[0031] S3: Calculate the sampling rate m / n, and execute steps S4.1-S4.2 when the sampling rate is 1, and execute steps S5.1-S5.3 when the sampling rate is less than 1;
[0032] S4.1: Compute the generalized inverse matrix Φ of the equivalent observation matrix Φ -1 ;
[0033] S4.2: The generalized inverse matrix Φ -1 Multiply y on the left to obtain the reconstructed spectral data of the pixel point;
[0034] S5.1: construct a discrete cosine transform basis sparse dictionary matrix Ψ for sparsely representing the spectral data x of the pixel point, that is, x=Ψs, where s is the sparse coefficient vector of the spectral data in the discrete cosine transform basis;
[0035] S5.2: Let ΦΨ=Θ, Θ is the sensor matrix, then y=ΦΨs=Θs, when y and Θ are known, the sparse coefficient vector s is calculated by iterative update using the sparsity adaptive matching pursuit algorithm;
[0036] S5.3: Multiply the sparse coefficient vector s by the sparse dictionary matrix Ψ to obtain the reconstructed spectral data of the pixel point;
[0037] S6: after step S4.2 or S5.3, repeat steps S2-S3 for the remaining pixel points in the image space domain;
[0038] S7: Combine the reconstructed spectral data of all pixel points according to the spatial domain position to obtain a three-dimensional hyperspectral data cube.
[0039] Compared with the prior art, one or more embodiments of the above scheme have at least the following advantages or beneficial effects: The present invention proposes for the first time to use the electrochromic device as a universal spectral encoding device, and utilizes the real-time spectral regulation capability of the electrochromic device under the action of an external electric field to realize the sampling encoding of the spectrum of the object to be measured, thereby realizing the measurement matrix effect of the compressed sensing sampling process. The broadband transmission spectrum of the electrochromic device can cover the entire detection band, and the correlation between the transmission spectra of different color change states is low, which can improve the degree of differentiated response of each band between the spectrum of the object to be measured and the spectrum of the electrochromic device, reduce the degree of coding redundancy, and thus obtain a higher coding capability with fewer electrochromic times. The simple electrochromic device structure can avoid the size problem caused by the mechanical filter wheel, reduce the processing complexity of the tunable filter, and improve the repeatability and stability of the device preparation. Therefore, the hyperspectral camera according to the present invention can be made into a miniature and low-cost, easy to commercialize. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure, but do not constitute a limitation of the present disclosure. It should be noted that for the sake of clarity, the components illustrated in the accompanying drawings are not necessarily drawn to scale.
[0041] Figure 1 is a schematic diagram of a hyperspectral camera according to an embodiment of the present invention.
[0042] Figure 2 is a flow chart of a hyperspectral reconstruction method according to an embodiment of the present invention.
[0043] Figure 3 Graph showing partial transmission spectra of an exemplary electrochromic device at different voltages.
[0044] Figure 4 is a flowchart of an operation of decoding an image according to an embodiment of the present invention.
[0045] Figure 5 The result of hyperspectral reconstruction is shown as an example.
[0046] Figure 6 The full spectrum curve fitting results of 4 pixels randomly selected in the reconstructed image are shown. DETAILED DESCRIPTION
[0047] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that as long as there is no conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other, and the technical solutions formed are all within the protection scope of the present invention.
[0048] Meanwhile, in the following description, many specific details are set forth for the purpose of explanation to provide a thorough understanding of the embodiments of the present invention. However, it is apparent to those skilled in the art that the present invention may be implemented without using the specific details or the specific manner described herein.
[0049] Figure 1 is a schematic diagram of a hyperspectral camera according to an embodiment of the present invention. Figure 1 As shown, the hyperspectral camera includes: an electrochromic coding component, which is used to spectrally encode the object to be measured by converting it into different transmission spectrum states under a variable voltage; a camera, which is used to record the image of the object after passing through the electrochromic coding component in different states; and a computing processing unit, which is used to decode the image to restore the hyperspectral data of the object.
[0050] The electrochromic coding assembly may include an electrochromic device and a power drive device. The power drive device is used to apply a variable voltage to the electrochromic device. The transmission spectrum of the electrochromic device is continuously adjustable in the visible light to near infrared range according to the change of the applied voltage.
[0051] In the embodiment of the present invention, the power driving device of the electrochromic device can be a current-voltage source meter, which can programmably adjust the voltage value of the electrochromic device and simultaneously measure its current value. The voltage value can be used to adjust the electrochromic device to achieve the expected spectral state, and the current value can be used to determine the life attenuation state of the electrochromic device, as well as to calibrate and compensate for the resulting spectral reconstruction error.
[0052] The camera may include a lens and a photodetector. The lens is used to collect spatial domain information of the object and focus it on the photodetector. The photodetector is used to receive the light signal contained in the spatial domain information.
[0053] The photodetector may be a CCD image sensor or a CMOS image sensor. The working wavelength range of the photodetector may include visible light to near infrared, for example, the working wavelength range of the photodetector may be 400nm to 1100nm. The photodetector is arranged in the optical path parallel to the electrochromic device, and the target surface size is not less than the area of the entire color changing area of the electrochromic device projected onto the surface of the photodetector.
[0054] In the embodiment of the present invention, the hyperspectral camera can have a wide spectral detection range. The electrochromic device used for sampling and encoding can have excellent spectral control performance in the range from visible light to micron-level near-infrared, and the spectral modulation of the electrochromic device can be maintained within a high transmittance range, thereby improving the detection sensitivity.
[0055] In the embodiment of the present invention, the hyperspectral camera can dynamically balance the sampling time and the resolution accuracy. By extending the sampling time, that is, changing more spectral states, and based on the principle of broadband spectrum compression sampling, more coding information can be obtained to achieve high resolution.
[0056] The hyperspectral camera according to the present invention has a simple overall structure. It only needs to install or integrate an electrochromic coding component in front of the lens of an existing industrial camera or consumer-grade image sensor device, and the full spectral information of the target can be reconstructed using the back-end algorithm and matching device spectral parameters.
[0057] The hyperspectral camera according to the present invention has low cost, the electrochromic device used for sampling and coding is simple to prepare, the process is mature, and the raw material price is low. The price of the electrochromic coding component is negligible compared to the photodetector.
[0058] The hyperspectral camera according to the present invention can be made into a miniature hyperspectral camera, which is a low-cost, easy-to-manufacture, and integrable miniature hyperspectral camera and is expected to be widely used in portable industrial detection, consumer electronics, scientific research, and teaching, and other fields.
[0059] When using the hyperspectral camera according to the present invention, the electrochromic device is regulated to continuously transform into different transmission spectrum states, that is, different broadband transmission spectrum responses are generated in the full detection band to encode the spectrum of the object to be measured. For the same object to be measured, the camera is continuously exposed to take an image in each transmission spectrum state of the electrochromic device. After all the image signals acquired within the sampling time range are input into the calculation processing unit, the observation signal is decoded using the compressed sensing reconstruction algorithm in combination with the transmission spectrum data of the electrochromic device matched at the exposure time, thereby restoring the hyperspectral data of the original object to be measured.
[0060] Figure 2 FIG. 4 is a flow chart of a hyperspectral reconstruction method according to an embodiment of the present invention. Figure 2 As shown, the hyperspectral reconstruction method may include the following steps.
[0061] S201: spectrally encode the object to be measured by applying a variable voltage to the electrochromic device to transform it into different transmission spectrum states.
[0062] S202: Using a camera to record images of the object after passing through the electrochromic device in different states.
[0063] S203: Decode the image to restore the hyperspectral data of the object.
[0064] In an embodiment of the present invention, the camera may include a lens and a photodetector. The lens is used to collect spatial domain information of an object and focus it on the photodetector. The photodetector is used to receive an optical signal contained in the spatial domain information.
[0065] In the embodiment of the present invention, the transmission spectrum of the electrochromic device is continuously adjustable in the range from visible light to near infrared according to the change of the applied voltage.
[0066] In an embodiment of the present invention, the hyperspectral reconstruction method may further include: regulating the electrochromic device to continuously transform into different transmission spectrum states, that is, generating different broadband transmission spectrum responses in the full detection band to encode the spectrum of the object to be measured; for the same object to be measured, the camera continuously exposes and captures an image in each transmission spectrum state of the electrochromic device. After all the image signals acquired within the sampling time range are input into the calculation processing unit, the observation signal is decoded using the compressed sensing reconstruction algorithm in combination with the transmission spectrum data of the electrochromic device matched at the exposure time, thereby restoring the hyperspectral data of the original object to be measured.
[0067] Figure 3 is a partial transmission spectrum of an exemplary electrochromic device at different voltages. Figure 4 It can be clearly seen that the electrochromic device has a broadband random filtering effect on the external light beam.
[0068] Figure 4 is a flowchart of an operation of decoding an image according to an embodiment of the present invention. In this embodiment, the decoding operation can be performed by a computing processing unit. Assuming that the electrochromic device changes m transmission spectrum states, the camera captures an image in each transmission spectrum state of the electrochromic device. Figure 4 As shown, the decoding operation may include the following steps.
[0069] S1: construct an equivalent measurement matrix Φ by cross-multiplying the quantum efficiency of the photodetector with each transmission spectrum of the electrochromic device.
[0070] Assume that the wavelength range of the spectrum detection is λ min to λ max The transmittance of the electrochromic device corresponding to each transmission spectrum is T t (λ), t=1,2,...,m, the quantum efficiency of the photodetector is E(λ), then the specific construction method of the equivalent measurement matrix Φ is as follows:
[0071]
[0072] Where n is the reconstructed spectral dimension, that is, the continuous spectral values are (λ max -λ min ) / (n-1), where m and n are both positive integers and n≥m. The quantum efficiency of the photodetector can be provided by its manufacturer.
[0073] S2: Select the aliased spectral data of a single pixel point at the same position in the image space domain in m images to form its spectral observation value y, where the spectral dimension is m, y=Φx, and x is the spectral data of the pixel point.
[0074] After m times of spectral modulation, each pixel can obtain m observation data y t (t=1,2,...,m), then the observed data y t It can be expressed as:
[0075]
[0076] After discretization, it can be expressed as:
[0077]
[0078] S3: Calculate the sampling rate m / n. When the sampling rate is 1, execute steps S4.1-S4.2 to perform full sampling reconstruction. When the sampling rate is less than 1, execute steps S5.1-S5.3 to perform under-sampling reconstruction.
[0079] S4.1: Compute the generalized inverse matrix Φ of the equivalent observation matrix Φ -1 .
[0080] S4.2: The generalized inverse matrix Φ -1 Multiply y on the left to obtain the reconstructed spectral data of the pixel.
[0081] S5.1: Construct a discrete cosine transform basis sparse dictionary matrix Ψ for sparsely representing the spectral data x of the pixel point, that is, x=Ψs, where s is the sparse coefficient vector of the spectral data under the discrete cosine transform basis.
[0082] The sparse dictionary matrix Ψ is an n×n dimensional orthogonal matrix, and the specific construction method is as follows:
[0083]
[0084] Where Ψ(μ,ν) is the element in the μ+1th row and ν+1th column of the Ψ matrix (μ=0,1,2,...,n-1; ν=0,1,2,...,n-1).
[0085] Right now
[0086] S5.2: Let ΦΨ=Θ, Θ is the sensing matrix, then y=ΦΨs=Θs. When y and Θ are known, the sparse coefficient vector s is calculated by iterative update using the sparsity adaptive matching pursuit algorithm.
[0087] The input of the sparse adaptive matching pursuit algorithm is the m×n-dimensional sensor matrix Θ, the m×1-dimensional observation data y and the step size step, and the output is the n×1-dimensional sparse coefficient estimate s. The specific implementation process of the algorithm is as follows:
[0088] (1) Initialize the residual r0 = y, index set Candidate Set Support set Support set length L = step, iteration number t = 1, stage stage = 1;
[0089] (2) The correlation coefficient μ is calculated by taking the absolute value of the inner product between the residual and each atom of the sensor matrix, that is, μ = {μ j |μ j =|r t-1 ,θ j >|, j = 1, 2, 3, ..., n}, and find the index values corresponding to the L maximum values from μ and store them in Λ;
[0090] (3) Merge the index set and the support set to obtain the candidate set CΛ∪F;
[0091] (4) Calculate the correlation coefficient between the atoms and the residuals corresponding to the index values in C, and extract the index values corresponding to the L maximum values and store them in F new middle;
[0092] (5) Find y = Θ F s t The least squares solution of : s t =argmin s ||y-Θ F s t ||2=(Θ F T Θ F ) -1 Θ F T y;
[0093] (6) Update the residual r new =y-θ F s t =y-Θ F (Θ F T Θ F ) -1 Θ F T y;
[0094] (7) If || r new ||2≤ε, then stop the iteration and go to step (8);
[0095] If || r new ||2≥||r t-1 ||2 (the current support set size cannot meet the requirements), then enter the next stage, update stage = stage + 1, L = stage × step, and return to step (2);
[0096] If none of them are satisfied, update F = F new , r=r new , t=t+1, return to step (2);
[0097] (8) Output the current sparse coefficient estimate s, where s is in Λ t There is a non-zero value in , and the corresponding non-zero value is the s obtained in the last iteration t .
[0098] In the above process, represents a non-empty set, θ j represents the j-th column of the matrix Θ, <·,·> represents the vector inner product, Λ trepresents the index set of the tth iteration, ∪ represents the set union operation, Θ Λ Indicates by index Λ t The set of selected columns, ||r new ||2 represents the residual r new The second norm of , ε represents the given error limit.
[0099] S5.3: Multiply the sparse coefficient vector s by the sparse dictionary matrix Ψ to obtain the reconstructed spectral data of the pixel point;
[0100] S6: after step S4.2 or S5.3, repeat steps S2-S3 for the remaining pixel points in the image space domain;
[0101] S7: Combine the reconstructed spectral data of all pixel points according to the spatial domain position to obtain a three-dimensional hyperspectral data cube.
[0102] The effect of the present invention is further described below in conjunction with simulation experiments.
[0103] The simulation experiment of the present invention is verified on the Columbia University CAVE dataset, and the multispectral image of flowers is selected as input. The image set contains the full spectral reflectance data of the target scene in the range of 400-700nm (31 bands in total), each band is stored as a 16-bit grayscale image, and the image size is 512×512.
[0104] Figure 5 The results of hyperspectral reconstruction are shown as an example. Specifically, Figure 5 The results of observation and reconstruction of the flowers multispectral data set under the conditions of sampling rate of 1 (31 exposures) and sampling rate less than 1 (25 exposures) according to the present invention are shown. Since the system of the present invention reconstructs many characteristic spectral bands, only the reconstructed images of the 400nm, 550nm, and 700nm spectral bands are selected for display.
[0105] Figure 6 The full spectrum curve fitting results of 4 randomly selected pixel points in the reconstructed image are shown. It is easy to see that in the case of full sampling, the reconstructed spectrum is basically consistent with the original spectrum; in the case of undersampling, the reconstructed spectrum has only a small error with the original spectrum.
[0106] The present invention proposes for the first time to use the electrochromic device as a universal spectral encoding device, and utilizes the real-time spectral regulation capability of the electrochromic device under the action of an external electric field to realize the sampling and encoding of the spectrum of the object to be measured, thereby realizing the measurement matrix effect of the compressed sensing sampling process. The broadband transmission spectrum of the electrochromic device can cover the entire detection band, and the correlation between the transmission spectra of different color change states is low, which can improve the degree of differentiated response of each band between the spectrum of the object to be measured and the spectrum of the electrochromic device, reduce the degree of coding redundancy, and thus obtain a higher coding capability with fewer electrochromic times. The simple electrochromic device structure can avoid the size problem caused by the mechanical filter wheel, reduce the processing complexity of the tunable filter, and improve the repeatability and stability of the device preparation. Therefore, the hyperspectral camera according to the present invention can be made into a miniature and low-cost, easy to commercialize.
[0107] It should be understood that the embodiments disclosed in the present invention are not limited to the specific processing steps or materials disclosed herein, but should be extended to equivalent substitutions of these features understood by ordinary technicians in the relevant field. It should also be understood that the terms used herein are only used for the purpose of describing specific embodiments and are not meant to be limiting.
[0108] The "embodiment" mentioned in the specification means that the specific features or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrase "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.
[0109] Although the embodiments disclosed in the present invention are as above, the contents described are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the protection scope of the present invention shall still be subject to the scope defined in the attached claims.
Claims
1. A hyperspectral camera, comprising: An electrochromic coding component for spectrally coding the object to be measured by changing to different transmission spectrum states under a variable voltage; A camera, used to record images of the object after passing through the electrochromic coding component in different states; as well as A computing processing unit is used to decode the image to restore the hyperspectral data of the object.
2. The hyperspectral camera according to claim 1, wherein: The camera includes a lens and a photodetector; The lens is used to collect spatial domain information of the object and focus it onto the photodetector; The photoelectric detector is used to receive the optical signal contained in the spatial domain information.
3. The hyperspectral camera according to claim 2, wherein: The electrochromic coding assembly includes an electrochromic device and a power drive device; The power driving device is used to apply a variable voltage to the electrochromic device; The transmission spectrum of the electrochromic device is continuously adjustable in the range from visible light to near infrared according to the change of applied voltage.
4. The hyperspectral camera according to claim 2, wherein: The photoelectric detector includes: an area array CCD image sensor and an area array CMOS image sensor, and the working wavelength range of the photoelectric detector includes visible light to near infrared.
5. The hyperspectral camera according to claim 3, wherein: The electrochromic device is configured to change m transmission spectrum states, the camera is configured to capture an image in each transmission spectrum state of the electrochromic device, and the computing processing unit is configured to perform the following operations: S1: constructing an equivalent measurement matrix Φ by cross-multiplying the quantum efficiency of the photodetector by each transmission spectrum of the electrochromic device; S2: Select the aliased spectral data of a single pixel point at the same position in the image space domain in m images to form its spectral observation value y, where the spectral dimension is m, y = Φx, x is the spectral data of the pixel point, and set the reconstructed spectral dimension to n, where m and n are both positive integers and n ≥ m; S3: Calculate the sampling rate m / n, and execute steps S4.1-S4.2 when the sampling rate is 1, and execute steps S5.1-S5.3 when the sampling rate is less than 1; S4.1: Compute the generalized inverse matrix Φ of the equivalent observation matrix Φ -1 ; S4.2: The generalized inverse matrix Φ -1 Multiply y on the left to obtain the reconstructed spectral data of the pixel point; S5.1: construct a discrete cosine transform basis sparse dictionary matrix Ψ for sparsely representing the spectral data x of the pixel point, that is, x=Ψs, where s is the sparse coefficient vector of the spectral data in the discrete cosine transform basis; S5.2: Let ΦΨ=Θ, Θ is the sensor matrix, then y=ΦΨs=Θs, when y and Θ are known, the sparse coefficient vector s is calculated by iterative update using the sparsity adaptive matching pursuit algorithm; S5.3: Multiply the sparse coefficient vector s by the sparse dictionary matrix Ψ to obtain the reconstructed spectral data of the pixel point; S6: after step S4.2 or S5.3, repeat steps S2-S3 for the remaining pixel points in the image space domain; S7: Combine the reconstructed spectral data of all pixel points according to the spatial domain position to obtain a three-dimensional hyperspectral data cube.
6. The hyperspectral camera according to any one of claims 1 to 5, wherein: The hyperspectral camera is a miniature hyperspectral camera.
7. A hyperspectral reconstruction method, comprising: Spectral encoding of the object to be measured is performed by applying a variable voltage to the electrochromic device to change it into different transmission spectrum states; Using a camera to record images of the object after passing through the electrochromic device in different states; as well as The image is decoded to recover the hyperspectral data of the object.
8. The hyperspectral reconstruction method according to claim 7, wherein: The camera includes a lens and a photodetector; The lens is used to collect spatial domain information of the object and focus it onto the photodetector; The photoelectric detector is used to receive the optical signal contained in the spatial domain information.
9. The hyperspectral reconstruction method according to claim 8, wherein: The transmission spectrum of the electrochromic device is continuously adjustable in the range from visible light to near infrared according to the change of applied voltage.
10. The hyperspectral reconstruction method according to claim 8, wherein: The electrochromic device is changed into m transmission spectrum states, and an image is captured by the camera in each transmission spectrum state of the electrochromic device, wherein decoding the image includes the following operations: S1: constructing an equivalent measurement matrix Φ by cross-multiplying the quantum efficiency of the photodetector by each transmission spectrum of the electrochromic device; S2: Select the aliased spectral data of a single pixel point at the same position in the image space domain in m images to form its spectral observation value y, where the spectral dimension is m, y = Φx, x is the spectral data of the pixel point, and set the reconstructed spectral dimension to n, where m and n are both positive integers and n ≥ m; S3: Calculate the sampling rate m / n, and execute steps S4.1-S4.2 when the sampling rate is 1, and execute steps S5.1-S5.3 when the sampling rate is less than 1; S4.1: Compute the generalized inverse matrix Φ of the equivalent observation matrix Φ -1 ; S4.2: The generalized inverse matrix Φ -1 Multiply y on the left to obtain the reconstructed spectral data of the pixel point; S5.1: construct a discrete cosine transform basis sparse dictionary matrix Ψ for sparsely representing the spectral data x of the pixel point, that is, x=Ψs, where s is the sparse coefficient vector of the spectral data in the discrete cosine transform basis; S5.2: Let ΦΨ=Θ, Θ is the sensor matrix, then y=ΦΨs=Θs, when y and Θ are known, the sparse coefficient vector s is calculated by iterative update using the sparsity adaptive matching pursuit algorithm; S5.3: Multiply the sparse coefficient vector s by the sparse dictionary matrix Ψ to obtain the reconstructed spectral data of the pixel point; S6: after step S4.2 or S5.3, repeat steps S2-S3 for the remaining pixel points in the image space domain; S7: Combine the reconstructed spectral data of all pixel points according to the spatial domain position to obtain a three-dimensional hyperspectral data cube.
Citation Information
Cited By
Intelligent micro-grid multi-module electrochromic collaborative spectrum monitoring device
CN121347420A
