A multi-spectral single-pixel computational imaging method, system, device and storage medium
Through cosine coding multiplexing strategy and Fourier transform, the problem of low information acquisition efficiency in single-pixel imaging technology is solved, and efficient reconstruction of multispectral images is achieved.
Patent Information
- Application Number
- CN202210565217.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-23
- Publication Date
- 2025-07-11
- Estimated Expiration
- 2042-05-23
AI Technical Summary
The existing single-pixel imaging technology has shortcomings in information acquisition efficiency, and the problems of system complexity and increase in data volume.
The cosine encoding multiplexing strategy is adopted to construct a multispectral illumination pattern by generating a Hadamard basis pattern and a cosine encoding, and a backscattered signal is received using a single pixel detector, and the multispectral image of the target object is reconstructed through Fourier transform and fusion.
The multispectral imaging system is simplified, the imaging efficiency is improved, and the multispectral image of the target scene can be effectively restored.
Smart Images
Figure CN115018937B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of spectral single-pixel computational imaging, and particularly relates to a multi-spectral single-pixel computational imaging method, system, device and storage medium. Background Art
[0002] As a new type of computational imaging technology, single-pixel imaging technology has received extensive attention in recent years due to its imaging ability in the non-visible light band. However, so far, the information acquisition efficiency of single-pixel imaging still needs to be improved. Multiplexed structured illumination, as a new method, can effectively improve the information acquisition efficiency of single-pixel imaging.
[0003] Single-pixel imaging is an emerging computational imaging scheme. Different from traditional area array imaging that acquires the spatial and color information of the target scene in a point-to-point manner, it uses a non-spatial resolution detector to acquire the spatial information of the scene and restores the image through correlation calculation. Multi-spectral single-pixel imaging technology is an organic combination of spectral technology and imaging technology. By extracting multi-color channel information, the multi-spectral image of the target scene can be effectively restored.
[0004] The deficiencies of the prior art are that the spectral splitting at either the light source end or the detection end in the existing methods will increase the complexity of the system and the amount of data, affecting the imaging efficiency of the system. Summary of the Invention
[0005] The purpose of the present invention is to overcome the deficiencies of the prior art. To achieve the above purpose, a multi-spectral single-pixel computational imaging method, system, device and storage medium are adopted to solve the problems raised in the above background art.
[0006] The first technical solution: A multi-spectral single-pixel computational imaging method based on cosine coding multiplexing, the specific steps include:
[0007] S1. Generate a set of Hadamard basis patterns and cosine coding, and use the cosine coding multiplexing strategy to construct a multi-spectral illumination pattern;
[0008] S2. Modulate according to the constructed multi-spectral illumination pattern, and irradiate the target object with the modulated broadband light source to obtain a backscattered signal;
[0009] S3. Use a single-pixel detector to receive the backscattered signal, and restore the aliased spectral image of the target object according to the algorithm;
[0010] S4. Perform Fourier transform on the aliased spectral image to obtain multiple spectral component images, and fuse and reconstruct the multiple spectral component images to obtain the multi-spectral image of the target object.
[0011] As a further solution of the present invention: The specific steps of S1 are as follows:
[0012] First, encode the broadband light source by using the cosine coding multiplexing strategy;
[0013] Construct three N×N two-dimensional gray cosine structure coding matrices F red 、F green and F blue , and the expression is:
[0014]
[0015] where f x 、f y are the frequencies of the coding matrix, is the initial phase;
[0016] Then, according to the generated deterministic orthogonal basis Hadamard basis pattern H i , construct the multispectral illumination pattern P i :
[0017] P i =F red *H i +F green *H i +F blue *H i ;
[0018] where, F red *H i 、F green *H i and F blue *H i correspond to the illumination patterns of the red, green, and blue spectral channels respectively.
[0019] As a further solution of the present invention: The specific steps of S2 are as follows:
[0020] Load the multispectral illumination pattern into the modulation system to modulate the broadband light source, and after irradiating the target object with the modulated broadband light source, obtain the backscattered signal.
[0021] As a further solution of the present invention: The specific steps of S3 are as follows:
[0022] Use a single-pixel detector to receive and capture the backscattered signal and save it. The expression of the backscattered signal D i is:
[0023]
[0024] Among them, the parameters r, g, and b are the response coefficients of the single-pixel detector to the red, green, and blue spectral bands respectively, and M red , M green and M blue correspond to the three spectral information of the target object respectively;
[0025] The expression of the backscattering signal D i is simplified to:
[0026]
[0027] Among them, M represents the aliased spectral image of the imaging object, which contains the spectral information of three color channels and can be expressed as:
[0028] M = r = F red *M red + g * F green *M green + b * F blue *M blue ;
[0029] For the deterministic orthogonal basis Hadamard basis pattern H i , the aliased spectral image M of the N×N dimension of the target object can be restored by an iterative algorithm as:
[0030]
[0031] As a further solution of the present invention: The specific steps of the S4 for performing Fourier transform on the aliased spectral image are:
[0032] Convert the aliased spectral image M from the spatial domain to the Fourier frequency domain for spectral component recombination. According to the transformation process of the cosine coding frequency of the spectral channel:
[0033]
[0034] Among them, f x , f y are the frequencies of the coding matrix, is the initial phase, and the parameters r, g, and b are the response coefficients of the single-pixel detector to the red, green, and blue spectral bands respectively;
[0035] As a further solution of the present invention: The specific steps of the S4 for fusing and reconstructing multiple spectral component images to obtain the multispectral image of the target object are:
[0036] Filter the spectral information of each color channel through a low-pass filter, and multiple spectral component images can be obtained after two-dimensional inverse Fourier transform;
[0037] Fuse and reconstruct multiple spectral component images to obtain the multispectral image of the target object;
[0038] The expression for the superposition of the spectra of each color channel is as follows:
[0039]
[0040] Using a single color channel, such as the red channel:
[0041]
[0042] From Euler's formula The above equation can be transformed into:
[0043]
[0044] Performing a Fourier transform on the above equation, and obtaining from the phase shift theorem:
[0045]
[0046] The second technical solution: A system including a multi-spectral single-pixel computational imaging method based on cosine coding multiplexing as described in any one of the above, comprising:
[0047] An image generation module, configured to generate a set of Hadamard basis patterns and cosine coding, and construct a multi-spectral illumination pattern by using a cosine coding multiplexing strategy;
[0048] A single-pixel detector module, configured to receive the backscattered signal reflected by the target object;
[0049] An operation module, configured to restore the aliased spectral image of the target object according to an algorithm, perform a Fourier transform on the aliased spectral image to obtain a plurality of spectral component images, and fuse and reconstruct the plurality of spectral component images to obtain the multi-spectral image of the target object.
[0050] As a further solution of the present invention: The operation module restores the aliased spectral image of the target object through an algorithm.
[0051] The third technical solution: A device further comprising:
[0052] At least one processor;
[0053] At least one memory, configured to store at least one program;
[0054] When the at least one program is executed by the at least one processor, the at least one processor implements a multi-spectral single-pixel computational imaging method based on cosine coding multiplexing as described in any one of the above first technical solutions.
[0055] Fourth technical solution: A storage medium stores instructions executable by a processor. When executed by the processor, the executable instructions are used to implement a multi-spectral single-pixel computational imaging method based on cosine coding multiplexing as described in any one of the above first technical solutions.
[0056] Compared with the prior art, the present invention has the following technical effects:
[0057] Adopting the above technical solution, a multi-spectral illumination pattern is generated based on the generated Hadamard basis pattern and cosine coding, and after obtaining the aliased spectral image, the multi-spectral image of the target object is fused and reconstructed. In the computational imaging method of the present invention, a non-spatial resolution detector is used to obtain the spatial information of the scene, and the image is restored through correlation calculation. The multi-spectral single-pixel imaging technology is an organic combination of spectral technology and imaging technology. By extracting the multi-spectral channel information, the multi-spectral image of the target scene can be effectively restored. A new coding multiplexing strategy is proposed, which uses the frequency shift characteristic of cosine coding in the Fourier frequency domain to well modulate and correlate the object information into the coding matrix frequency. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] The following will describe in detail the specific embodiments of the present invention with reference to the accompanying drawings:
[0059] Figure 1 It is a schematic diagram of the steps of a multi-spectral single-pixel computational imaging method according to some embodiments disclosed in the present application;
[0060] Figure 2 It is a flow chart of cosine coding multiplexing multi-spectral single-pixel imaging according to some embodiments disclosed in the present application;
[0061] Figure 3 It is a schematic diagram of the experimental results of multi-spectral computational single-pixel imaging according to some embodiments disclosed in the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0063] Please refer to Figure 1 and Figure 2 , in the embodiments of the present invention, a multi-spectral single-pixel computational imaging method based on cosine coding multiplexing specifically includes the following steps:
[0064] S1. Generate a set of Hadamard basis patterns and cosine encodings, and construct a multispectral illumination pattern using the cosine encoding multiplexing strategy. The specific steps are as follows:
[0065] First, implement multispectral imaging by encoding a broadband light source using the cosine encoding multiplexing strategy. In this embodiment, three spectral channels are used as an example for illustration.
[0066] Construct three N×N two-dimensional grayscale cosine structure encoding matrices F red 、F green and F blue , corresponding to the red, green, and blue color channels respectively. The expressions are:
[0067]
[0068] where f x 、f y are the frequencies of the encoding matrix, and is the initial phase.
[0069] Then, based on the deterministic orthogonal basis Hadamard basis pattern H i generated by the computer, construct the multispectral illumination pattern P i :
[0070] P i = F red * H i + F green * H i + F blue * H i ;
[0071] where F red * H i 、F green * H i and F blue * H i correspond to the illumination patterns of the red, green, and blue spectral channels respectively.
[0072] S2. Modulate according to the constructed multispectral illumination pattern, and irradiate the target object with the modulated broadband light source to obtain a backscattering signal. The specific steps are as follows:
[0073] Load the multispectral illumination pattern into the modulation system to modulate the broadband light source. After irradiating the target object with the modulated broadband light source, obtain a backscattering signal.
[0074] S3. Use a single-pixel detector to receive the backscattering signal, and restore the aliased spectral image of the target object according to the evolutionary compressive sensing algorithm. The specific steps are as follows:
[0075] The single-pixel detector is used to receive and capture the backscattering signal and save it to the computer, and the backscattering signal D i has the following expression:
[0076]
[0077] where the parameters r, g, and b are the response coefficients of the single-pixel detector to the red, green, and blue spectral ranges, respectively, which can be obtained through calibration in this embodiment. M red , M green and M blue correspond to the three spectral information of the target object respectively;
[0078] The expression of the backscattering signal D i is simplified to:
[0079]
[0080] where M represents the aliased spectral image of the imaging object, which contains the spectral information of three color channels and can be expressed as:
[0081] M = r * F red * M red + g * F green * M green + b * F blue * M blue ;
[0082] For the deterministic orthogonal basis Hadamard basis pattern H i , the aliased spectral image M of the target object with N×N dimensions can be restored by an iterative algorithm as:
[0083]
[0084] S4. The specific steps of performing a Fourier transform on the aliased spectral image to obtain multiple spectral component images and fusing and reconstructing the multiple spectral component images to obtain the multispectral image of the target object are as follows:
[0085] The aliased spectral image M is transformed from the spatial domain to the Fourier frequency domain for spectral component recombination. The transformation process of the cosine encoding frequency according to the spectral channel is:
[0086] Specifically, a Fourier transform (FT) is performed on the aliased spectral image M to process it from the spatial domain to the Fourier frequency domain. Since the cosine encoding frequencies of the three spectral channels are different, the phase shifts in the frequency spectrum are also different. The specific proof process is as follows:
[0087]
[0088] where f x , fy is the frequency of the encoding matrix, is the initial phase, and the parameters r, g, and b are the response coefficients of the single-pixel detector to the red, green, and blue spectral bands respectively;
[0089] Then, the spectral information of each color channel is filtered through a low-pass filter, and multiple spectral component images can be obtained after two-dimensional inverse Fourier transform;
[0090] The multiple spectral component images are fused and reconstructed to obtain the multispectral image of the target object;
[0091] The Fourier spectrum of the aliased spectral image M is the superposition of the spectra of each color channel, and the expression for the superposition of the spectra of each color channel is:
[0092]
[0093] In this embodiment, taking a single red channel as an example, we get:
[0094]
[0095] According to Euler's formula The above formula can be transformed into:
[0096]
[0097] Performing Fourier transform on the above formula, according to the phase shift theorem, we get:
[0098]
[0099] During the test of this embodiment, the imaging performance of the proposed cosine coding multiplexing strategy is experimentally evaluated:
[0100] The experimental setup mainly consists of a projection system (DLP digital projector), an imaging object, an optical receiving system (a focusing lens and a PMT single-pixel detector), and a computer system. The computer program generates the same set (256×256) of multispectral illumination patterns as in the numerical simulation. The projector projects the corresponding patterns onto the object, and the projection frequency is about 20 Hz to obtain a higher signal-to-noise ratio. The backscattered light is collected by a single-pixel detector (Thorlabs PMM02) through the lens and converted into an electrical signal, and then stored in the computer system by a data acquisition card (Advantech PCI-9816H), and the sampling rate is set to 40 KHz.
[0101] As Figure 3As shown in the figure, it is a schematic diagram of experimental results, corresponding to sampling rates of 6.25%, 12.5%, 25%, 50%, 75% and 100% respectively. Similar to the numerical simulation results, as the sampling rate increases, the quality of the spectral component image and the reconstructed spectral image also improves. The preliminary experimental results also show that after setting the optimal parameter ratio of the experimental system, this method can successfully achieve multispectral imaging.
[0102] Beneficial effects:
[0103] In the present invention, a cosine coding multiplexing strategy is proposed and applied to the multispectral computational single-pixel imaging technology. This method uses the Fourier frequency shift characteristic of cosine coding to distinguish the spectral information of different color channels in the frequency domain. With only one single-pixel detector, multispectral images can be obtained simultaneously, simplifying the multispectral imaging system and improving the imaging efficiency.
[0104] A system including a multispectral single-pixel computational imaging method based on cosine coding multiplexing as described in any one of the above, comprising:
[0105] An image generation module, configured to generate a set of Hadamard basis patterns and cosine coding, and construct a multispectral illumination pattern by using the cosine coding multiplexing strategy;
[0106] A single-pixel detector module, configured to receive the backscattered signal reflected by the target object;
[0107] An operation module, configured to restore the aliased spectral image of the target object according to the algorithm, perform Fourier transform on the aliased spectral image to obtain multiple spectral component images, and fuse and reconstruct the multiple spectral component images to obtain the multispectral image of the target object.
[0108] In the specific implementation manner, the operation module restores the aliased spectral image of the target object through the evolutionary compressive sensing algorithm.
[0109] A device further includes:
[0110] At least one processor;
[0111] At least one memory, configured to store at least one program;
[0112] When the at least one program is executed by the at least one processor, the at least one processor implements a multispectral single-pixel computational imaging method based on cosine coding multiplexing as described in any one of the above first technical solutions.
[0113] A storage medium, in which instructions executable by a processor are stored, and the instructions executable by the processor are used to implement a multispectral single-pixel computational imaging method based on cosine coding multiplexing as described in any one of the above first technical solutions when executed by the processor.
[0114] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and all of them should be included within the protection scope of the present invention.
Claims
1. A multi-spectral single-pixel computational imaging method based on cosine coding multiplexing, characterized in that The specific steps include: S1. Generate a set of Hadamard basis patterns and cosine encodings, and construct a multispectral illumination pattern by using a cosine encoding multiplexing strategy. The specific steps are as follows: First, encode a broadband light source by using a cosine encoding multiplexing strategy; Construct three N×N two-dimensional grayscale cosine structure encoding matrices F red , F green and F blue , and the expression is: Among them, f x , f y are the frequencies of the encoding matrix. x and y respectively represent the positions of each pixel of the corresponding red, green, and blue colors in the encoding matrix. x1, y1, x2, y2, and x3, y3 respectively represent the frequency subscripts that distinguish the corresponding red, green, and blue colors in the encoding matrix. is the initial phase; Then, based on the generated deterministic orthogonal basis Hadamard basis pattern H i , a multispectral illumination pattern P i is constructed as follows: P i = F red * H i + F green * H i + F blue * H i ; Among them, F red *H i 、F green *H i and F blue *H i correspond to the illumination patterns of the red, green, and blue spectral channels respectively; S2. Modulate according to the constructed multispectral illumination pattern, and irradiate a target object with the modulated broadband light source to obtain a backscattering signal; S3. Use a single-pixel detector to receive the backscattering signal, and restore the aliased spectral image of the target object according to an algorithm. The specific steps are as follows: Use a single-pixel detector to receive and save the captured backscattering signal, and the backscattering signal D i has the following expression: Among them, the parameters r, g, and b are the response coefficients of the single-pixel detector to the red, green, and blue spectral bands respectively, and M red , M green , and M blue correspond to the three spectral information of the target object respectively; The backscattered signal D i is simplified to the expression: Where M represents the aliased spectral image of the imaging object, which contains spectral information of three color channels and can be expressed as: M = r * F red * M red + g * F green * M green + b * F blue * M blue ; For the deterministic orthogonal basis Hadamard basis pattern H i , the aliased spectral image M of the target object with N×N dimensions is restored by the iterative algorithm as follows: S4. Perform a Fourier transform on the aliased spectral image to obtain multiple spectral component images, and fuse and reconstruct the multiple spectral component images to obtain the multispectral image of the target object.
2. The multispectral single-pixel computational imaging method based on cosine coding multiplexing according to claim 1, wherein The specific steps of S2 are as follows: Load the multispectral illumination pattern into a modulation system to modulate the broadband light source. After irradiating the target object with the modulated broadband light source, a backscattering signal is obtained.
3. The multispectral single-pixel computational imaging method based on cosine coding multiplexing according to claim 1, wherein The specific steps of S4 for performing a Fourier transform on the aliased spectral image are as follows: Convert the aliased spectral image M from the spatial domain to the Fourier frequency domain for spectral component recombination. The transformation process of the cosine encoding frequency of the spectral channel is as follows: Among them, f x and f y are the frequencies of the encoding matrix. x and y respectively represent the positions of each pixel of the corresponding red, green, and blue colors in the encoding matrix. x1, y1, x2, y2, and x3, y3 respectively represent the frequency subscripts that distinguish the corresponding red, green, and blue colors in the encoding matrix. M red and M green and M blue respectively correspond to the three spectral information of the target object. is the initial phase. The parameters r, g, and b are respectively the response coefficients of the single-pixel detector to the red, green, and blue spectral colors.
4. A multispectral single-pixel computational imaging method based on cosine coding multiplexing according to claim 1, characterized in that, The specific steps of S4 for fusing and reconstructing multiple spectral component images to obtain the multispectral image of the target object are as follows: Filter the spectral information of each color channel through a low-pass filter respectively, and multiple spectral component images can be obtained after two-dimensional inverse Fourier transform; Fuse and reconstruct the multiple spectral component images to obtain the multispectral image of the target object; The expression for the superposition of the spectra of each color channel is: Use a single color channel. Among them, the calculation formula for the red channel is: From Euler's formula The above equation can be transformed into: Perform a Fourier transform on the above formula, and obtain according to the phase shift theorem: Among them, the parameters r, g, and b are the response coefficients of the single-pixel detector to the red, green, and blue spectral ranges respectively, i is the imaginary unit, is the initial phase, M red 、M green and M blue correspond to the three spectral information of the target object respectively.
5. A system adopting a multi-spectral single-pixel computational imaging method based on cosine coding multiplexing according to any one of claims 1-4, characterized in that, Including: An image generation module, which is used to generate a set of Hadamard basis patterns and cosine encodings, and construct a multispectral illumination pattern by using a cosine encoding multiplexing strategy; A single-pixel detector module, which is used to receive the backscattering signal reflected by the target object; An operation module, which is used to restore the aliased spectral image of the target object according to an algorithm, perform a Fourier transform on the basis of the aliased spectral image to obtain multiple spectral component images, and fuse and reconstruct the multiple spectral component images to obtain the multispectral image of the target object.
6. The multispectral single-pixel computational imaging method based on cosine coding multiplexing according to claim 5, wherein The operation module restores the aliased spectral image of the target object through an algorithm.
7. A device, characterized in that, It further includes: At least one processor; At least one memory, which is used to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a multispectral single-pixel computational imaging method based on cosine encoding multiplexing as described in any one of claims 1-4.
8. A storage medium storing instructions executable by a processor, characterized in that: The instructions executable by the processor are used to implement a multispectral single-pixel computational imaging method based on cosine encoding multiplexing as described in any one of claims 1-4 when executed by the processor.