Multi wavelength imaging sensor

Abstract

The present disclosure concerns an imaging sensor comprising an optical encoder comprising a layer of at least four transducers stochastically arrayed on a substrate, the four transducers configured to each independently absorb a different wavelength of electromagnetic wave; and a charge-coupled device (CCD) in electromagnetic communication with the optical encoder for receiving visible light from the at least four transducers and outputting a combined image.

Description

[0001] Multi Wavelength Imaging Sensor

[0002] Technical Field

[0003] The present invention relates, in general terms, to an imaging sensor capable of simultaneously detecting and distinguishing multiple wavelengths of electromagnetic waves, spanning from X-rays to near-infrared region II.

[0004] Background

[0005] Multi-wavelength, multi-channel, and multi-depth imaging technologies have revolutionized scientific research and industrial applications, enabling precise three- dimensional reconstruction and comprehensive information representation of target objects. These technologies capture the unique characteristics of target objects across various wavelength bands such as ultraviolet, visible light, and infrared. By employing different-wavelength light to illuminate different parts of a sample, simultaneous multiwavelength excitation enables the acquisition of multiple depth images. However, sequential switching of excitation wavelengths and corresponding filters present limitations in terms of image acquisition time and is unsuitable for applications requiring simultaneous observation of synergistic and multi-depth information. Current solutions involving complex beam splitting based on multiple optical components, diffractive optical elements, metasurfaces, or multispectral filters tend to increase system size, complexity, and cost. Therefore, despite rapid advances in optical imaging and dispersive systems, their direct combination results in a bulky system that hinders the practical implementation of multi-wavelength and multi-channel imaging. Moreover, most existing methods are limited to the visible band, and the realization of a multiwavelength imaging system spanning near-infrared (regions I and II), visible, ultraviolet (UV), and X-rays remains beyond the reach of traditional optical components.

[0006] Multi-wavelength images for studying tissue interactions and spectral analysis typically necessitates spectroscopic systems with bulky components or passive filters constrained by limited bandwidth.

[0007] It would be desirable to overcome or ameliorate at least one of the above-described problems. Summary

[0008] The present disclosure concerns an imaging sensor, comprising: a) an optical encoder comprising a layer of at least four transducers stochastically arrayed on a substrate, the four transducers configured to each independently absorb a different wavelength of electromagnetic wave; and b) a charge-coupled device (CCD) in electromagnetic communication with the optical encoder for receiving visible light from the at least four transducers and outputting a combined image.

[0009] In some embodiments, the at least four transducers each independently output a different wavelength of visible light.

[0010] In some embodiments, the at least four transducers is at least four different types of lanthanide-doped luminescent phosphors transducers.

[0011] In some embodiments, the at least four transducers each independently comprises phosphors having a particle size of about 50 nm to about 5 pm.

[0012] In some embodiments, the optical encoder is characterised by a resolution of about 10 pm to about 300 pm.

[0013] In some embodiments, each of the at least four transducers is characterised by a pixel size of about 10 pm to about 300 pm.

[0014] In some embodiments, the substrate is a transparent elastomer.

[0015] In some embodiments, the substrate is selected from polydimethylsiloxane (PDMS).

[0016] In some embodiments, each of the at least four electromagnetic waves is independently selected from X-rays, ultraviolet, near-infrared region I, and near-infrared region II.

[0017] In some embodiments, the at least four electromagnetic waves are selected from X- rays with a wavelength of about 0.089 nm, ultraviolet with a wavelength of about 375 nm, near-infrared region I with a wavelength of about 808 nm, and near-infrared region II with a wavelength of about 1532 nm.

[0018] In some embodiments, the layer of at least four transducers is characterised by a thickness of about 0.01 mm to about 5 mm. In some embodiments, the layer of at least four transducers is characterised by a thickness of about 0.2 mm to about 2 mm.

[0019] In some embodiments, the optical encoder further comprises a second layer of at least four transducers stochastically arrayed and adjacent to the layer on the substrate.

[0020] In some embodiments, the optical encoder comprises a plurality of layers adjacent to each other on a substrate, each layer having at least four transducers stochastically arrayed.

[0021] In some embodiments, the CCD is a monochrome CCD or RGB CCD.

[0022] In some embodiments, the imaging sensor further comprises a filter configured to remove excitation light.

[0023] In some embodiments, the imaging sensor further comprises a deep learning algorithm configured to receive the combined image from the CCD and reconstruct it into at least four reconstructed images.

[0024] The present disclosure concerns a method of fabricating an imaging sensor, comprising : a) screen printing at least four transducers stochastically arranged as a layer on a substrate in order to form an optical encoder, the four transducers configured to each independently absorb a different wavelength of electromagnetic wave; and b) electromagnetically communicating a charge-coupled device (CCD) with the optical encoder for receiving visible light emitted from the at least four transducers.

[0025] The present disclosure concerns a method for reconstructing a combined image from an imaging sensor as disclosed herein into at least four reconstructed images using a deep learning algorithm, the image analysis method comprising: a) obtaining the combined image from the imaging sensor as disclosed herein; b) generating a measurement matrix based on parameters of the optical encoder; c) inputting the combined image and a measurement matrix to the deep learning algorithm; and d) generating at least four reconstructed images.

[0026] In some embodiments, the deep learning algorithm uses an iterative shrinking process to generate the at least four reconstructed images.

[0027] In some embodiments, the parameters comprise a transducer arrangement.

[0028] The present disclosure concerns a method of imaging a plurality of objects using the imaging sensor as disclosed herein, comprising: a) positioning the objects in electromagnetic communication with the imaging sensor; b) exposing the objects to at least four electromagnetic waves absorbable by the optical encoder of the imaging sensor; and c) obtaining a combined image from the CCD of the imaging sensor.

[0029] The present disclosure concerns a method of imaging an object at multiple depths using the imaging sensor as disclosed herein, comprising: a) positioning the object in electromagnetic communication with the imaging sensor; b) exposing the object to at least four electromagnetic waves absorbable by the optical encoder of the imaging sensor and focusing the at least four electromagnetic waves at different depths and / or magnification of the object; and c) obtaining a combined image from the CCD of the imaging sensor.

[0030] In some embodiments, the method further comprises reconstructing the combined image into at least four reconstructed images.

[0031] Brief description of the drawings

[0032] Embodiments of the present invention will now be described, by way of non-limiting example, with reference to the drawings in which:

[0033] Figure 1. A, Design of the multi-wavelength coded imaging sensor based on lanthanide transducer array. The optical encoder encodes and converts images captured by X-rays, ultraviolet, and near-infrared light to be detectable by conventional Si-based CCDs. A filter is to remove excitation light. B, Conceptual scheme of the multi-wavelength coded imaging, taking four wavelength channels as an example.

[0034] Figure 2. A, Luminescence photograph of active optical encoders excited by various wavelengths (375 nm, 808 nm, and 1532 nm, and X-rays) with unique random coding schemes. B, Schematic of imaging principle. The encoding process involves linear multiplication between the measurement matrix <p (encoder representation) and image vector x, resulting in a compressed measurement vector y. Reconstruction of the original image x from y and cp is accomplished using a machine learning algorithm.

[0035] Figure 3. Imaging results of the four wavelength channels, displaying encoded and reconstructed images.

[0036] Figure 4. Schematic comparison of a single-layer and a three-layer active optical encoder

[0037] Figure 5. A, Schematic diagram of the experimental setup for the multi-channel imaging system. B, Reconstructed images of dragonfly wings and partial torso simultaneously imaged at four wavelengths, with a compression ratio of 25% for each wavelength channel. The white and yellow dotted boxes mark the typical complementary structural information obtained from different channel images.

[0038] Figure 6. A, The relationship between photoelectric absorption coefficient of scintillator and X-ray photon energy. B, Calculated light output of scintillator layers with different thicknesses under X-ray and gamma-ray radiation. C, An optical encoder specifically designed for imaging both X-rays (with a pixel thickness of ~200 pm) and gamma-rays (with a pixel thickness of ~2.5 mm). In thin pixels, the ratio between radioluminescence output of X-ray and gamma-ray is approximately 105. Similarly, in thick pixels, the ratio of gamma-ray radioluminescence output to X-ray radioluminescence output is approximately 105. D, Reconstruction images of a stencil at X-rays and spot shape at gamma rays, and the compression ratio for each channel is 50%.

[0039] Figure 7. Processing of the multi-wavelength compressed imaging sensor composed of arrays of four distinct transducers arranged in a random but complementary configuration. The process begins with designing a stencil containing a random hole array for each wavelength channel. A flat glass substrate coated with transparent high- quality double-sided adhesive serves as the processing substrate. Microparticles and nanopowders are uniformly spread onto the template and pressed with force to fill the pattern and adhere to the adhesive on the glass. After curing, the template is removed, leaving behind the imprinted pattern. The other three materials are then printed in a similar manner, ensuring precise alignment each time.

[0040] Detailed description

[0041] The present disclosure is predicated on a multi-wavelength imaging sensor through stochastic photoluminescence and compressed encoding. Leveraging lanthanide-doped luminescent materials, a vastly broad spectrum of wavelengths may be efficiently converted into the visible range, enabling simultaneous detection using conventional cameras. Lanthanide ions have recently sparked considerable interest due to their luminescence properties, which allow them to sensitize and emit light across a broad spectrum of wavelengths. Moreover, their ability to coordinate with dyes and organic compound makes them highly versatile as upconversion and downshifting transducers for various applications. By leveraging stochastic screen printing and employing compressed imaging based on deep learning algorithms, it is believed that an ultrabroad band multi-wavelength compressed imaging sensor can be developed.

[0042] Accordingly, the present disclosure concerns an imaging sensor, comprising : a) an optical encoder comprising a layer of at least four transducers stochastically arrayed on a substrate, the four transducers configured to each independently absorb a different wavelength of electromagnetic wave; and b) a charge-coupled device (CCD) in electromagnetic communication with the optical encoder for receiving the visible light emitted from the at least four transducers and outputting a combined image.

[0043] In certain scenarios, such as space imaging and industrial inspection, it is necessary to utilize a variety of electromagnetic wavelengths, ranging from X-rays to mid-infrared light, for imaging objects. However, commercially available sensors capable of simultaneously capturing images across these wavelengths are currently lacking. The presently disclosed imaging sensor is able to image multiple wavelengths based on stochastically positioned photoluminescence materials. The imaging sensor thus possess the capability to image ultra-broadband wavelengths depending on the transducer used, such as gamma rays, X-rays, ultraviolet, and NIR I and II, through seamlessly converting them into the visible light detected by a traditional camera. By using different phosphors as transducers, different wavelengths may be targeted. The response wavelength may also be adjusted by employing different luminescent materials. In contrast, achieving these features through traditional optical components requires intricate adjustments to various processing parameters.

[0044] In some embodiments, the at least four transducers are at least four different types of lanthanide-doped luminescent phosphors transducers. Luminescent materials (or phosphors) are solid inorganic and / or organic materials consisting of a lattice and emit light under excitation. Phosphors are usually made from a host material with an added activator. The best known type is a copper-activated zinc sulfide (ZnS) and the silver- activated zinc sulfide (zinc sulfide silver). The host materials are typically oxides, nitrides and oxynitrides, sulfides, selenides, halides or silicates of zinc, cadmium, manganese, aluminium, silicon, or various rare-earth metals. The activators prolong the emission time (afterglow). In turn, other materials (such as nickel) can be used to quench the afterglow and shorten the decay part of the phosphor emission characteristics. Examples of phosphors include calcium sulfide with strontium sulfide with bismuth as activator ((Ca,Sr)S: Bi), zinc sulfide with about 5 ppm of a copper activator, a mix od zinc sulfide and cadmium sulphide, and strontium aluminate activated by europium, (SrAl2O4: Eu(II):Dy(III)).

[0045] The phosphors may be intentionally doped to alter its absorption and emission wavelength. The absorption of energy takes place via either the host lattice or on impurities. In addition, transfer of energy through the lattice can take place. In almost all cases, the emission takes place on intentionally doped impurities, like rare-earth ions, which are present in relatively low concentrations (a few mole percent or less). The lanthanide-doped luminescent phosphors may comprise a lanthanide selected from La, Ce, Pr, Nd, Pm, Sm, Eu, Gd, Tb, Dy, Ho, Er, Tm, Yb, Lu, or a combination thereof. The lanthanide may be in any proportion relative to the phosphors. For example, the lanthanide may be about 0.01 wt% to about 30 wt% relative to the phosphors.

[0046] The lanthanide-doped luminescent phosphors may each be specifically designed to convert electromagnetic waves of different wavelengths into visible light detectable by a Si-based CCD. For example, the lanthanide-doped luminescent phosphors may be doped with specific lanthanide emitters to target electromagnetic wave of different wavelength. For example, doping with Eu2+may emit a blue light, Eu3+emits red light, Tb3+emits green light, Ce3+emits blue light, Dy3+emits blue and / or yellow light, and Sm3+emits orange-red light.

[0047] When at least four different types of lanthanide-doped luminescent phosphors transducers are used, the output wavelength of visible light may be substantially the same or overlap with each other. In other embodiments, at least two different wavelength of visible light is output, or at least three different wavelength of visible light is output, or at least four different wavelength of visible light is output.

[0048] In some embodiments, the at least four transducers each independently comprises nanoparticles having a particle size of about 50 nm to about 5 pm. In some embodiments, the nanoparticles have a particle size of about 50 nm to about 4 pm, about 50 nm to about 3 pm, about 50 nm to about 2 pm, about 50 nm to about 1 pm, about 50 nm to about 900 nm, about 50 nm to about 800 nm, about 50 nm to about 700 nm, about 50 nm to about 600 nm, about 50 nm to about 500 nm, about 50 nm to about 400 nm, about 50 nm to about 300 nm, about 50 nm to about 200 nm, or about 50 nm to about 100 nm.

[0049] In some embodiments, the optical encoder is characterised by a resolution of about 10 pm to about 300 pm. In some embodiments, the optical encoder is characterised by a resolution of about 10 pm to about 200 pm, about 10 pm to about 150 pm, about 10 pm to about 100 pm, about 10 pm to about 80 pm, or about 10 pm to about 60 pm.

[0050] The at least four transducers are stochastically arrayed such that there are no gaps between them. However, as the transducers are stochastically arrayed, for a single type of transducer, one transducer may be spaced apart from another transducer of the same type. It was found that stochastically (or randomly) arrayed transducers provide better encoding than a regular array, such as a Bayer mosaic. In this regard, it was found that the random nature of the distance between the same type of transducer provides an additional dimension of information, such that when the coded mask is decoded, a better reconstruction may be obtained.

[0051] In some embodiments, each of the at least four transducers is characterised by a pixel size of about 10 pm to about 300 pm. The pixel size refers to the physical size of each transducer. Thus, each transducer may have a size of about 10 |jm to about 300 pm, and arrayed stochastically to form a surface of the optical encoder. In some embodiments, each of the at least four transducers is characterised by a pixel size of about 50 pm to about 300 pm, about 100 pm to about 300 pm, about 150 pm to about 300 pm, about 200 pm to about 300 pm, or about 250 pm to about 300 pm.

[0052] It was found that the wt loading in each pixel is less critical, as the system compensates for differences in excitation and emission efficiencies between the phosphors to ensure effective performance.

[0053] The at least four transducers may be four to ten transducers.

[0054] In some embodiments, the at least four transducers are repeatedly and stochastically arrayed on the substrate such that they are repeatedly and randomly positioned to form an arrayed layer on a surface of the optical encoder. In this regard, a plurality of at least four transducer may be present.

[0055] In some embodiments, the substrate is a transparent elastomer. The substrate may be an elastomer with a transparency of at least 90%. In some embodiments, the substrate is selected from polydimethylsiloxane (PDMS).

[0056] In some embodiments, each of the at least four electromagnetic waves is independently selected from X-rays, ultraviolet, near-infrared region I, and near-infrared region II.

[0057] In some embodiments, the at least four electromagnetic waves are selected from X- rays with a wavelength of about 0.089 nm, ultraviolet with a wavelength of about 375 nm, near-infrared region I with a wavelength of about 808 nm, and near-infrared region II with a wavelength of about 1532 nm.

[0058] The number of wavelength channels may be expanded by creating multilayer optical encoders based on lanthanide transducers with different emission colors, or by creating optical encoders with pixels having different thickness. Luminescent materials exhibit a variety of characteristics, including material thickness, luminescent color, lifetime, and polarization. These attributes can all be leveraged to expand wavelength channels. In contrast, achieving these features through traditional optical components requires intricate adjustments to various processing parameters.

[0059] In some embodiments, the layer of at least four transducers is characterised by a thickness of about 0.01 mm to about 5 mm. In some embodiments, the layer of at least four transducers is characterised by a thickness of about 0.2 mm to about 2 mm.

[0060] In some embodiments, the optical encoder further comprises a second layer of at least four transducers stochastically arrayed and adjacent to the first layer on the substrate. The second layer may be screen printed on top of the first layer. In this regard, the second layer may contact with the first layer. The second layer may comprise at least four transducers which are different from those in the first layer. The pixels in the second layer may align with the first layer. In other embodiments, the pixels in the second layer are offset by a predetermined angle or distance relative to the pixels in the first layer. The angle may be about 1 ° to about 45 ° . The distance may be about 1 pm to about 200 pm.

[0061] The optical encoder may comprise one to three layers of transducers. In some embodiments, the optical encoder comprises a plurality of layers adjacent to each other on a substrate, each layer having at least four transducers stochastically arrayed. The layers may contact with each other. Each of the layers may comprise at least four transducers which are different from the other layer. The pixels in each layer may align with its neighbouring layer.

[0062] A charge-coupled device (CCD) is an integrated circuit containing an array of linked, or coupled, capacitors. Under the control of an external circuit, each capacitor can transfer its electric charge to a neighboring capacitor. In a CCD for capturing images, there is a photoactive region (an epitaxial layer of silicon), and a transmission region made out of a shift register.

[0063] In some embodiments, the CCD is further configured to combine the at least four wavelengths of visible light to output a combined image.

[0064] In some embodiments, the CCD is directly coupled to an output surface of the optical encoder. In this regard, a photoactive region of the CCD is coupled to the layer of transducers such that the visible light may be received. Accordingly, the array size of the optical encoder may be sized to be substantially similar to the photoactive region of the CCD.

[0065] In some embodiments, the CCD is a monochrome CCD or RGB CCD.

[0066] In some embodiments, the imaging sensor further comprises a filter configured to remove excitation light.

[0067] In some embodiments, the imaging sensor further comprises a deep learning algorithm configured to receive the combined image from the CCD and reconstruct it into at least four reconstructed images. The deep learning algorithm may be implemented by a computer program. The deep learning algorithm may be a convolutional neural network (CNN) with multiple layers between the input and output layers.

[0068] A compressed encoding approach may be used, wherein the images from different wavelength channels can be reconstructed using only NxN / m pixels to generate an NxN pixel image, and m represents the number of wavelength channels. In traditional multiwavelength cameras, reconstructing m images of size NxN requires a camera with mxNxN pixels. In our design, however, reconstructing m images of size NxN only requires NxN pixels, significantly reducing camera costs.

[0069] The present disclosure concerns a method of fabricating an imaging sensor, comprising : a) screen printing at least four transducers stochastically arranged as a layer on a substrate in order to form an optical encoder, the four transducers configured to each independently absorb a different wavelength of electromagnetic wave; and b) electromagnetically communicating a charge-coupled device (CCD) with the optical encoder for receiving visible light from the at least four transducers and outputting a combined image.

[0070] The method is suitable for large-scale production and on-chip integrated systems.

[0071] The present disclosure concerns a method for reconstructing a combined image from an imaging sensor as disclosed herein into at least four reconstructed images using a deep learning algorithm, the image analysis method comprising: a) obtaining the combined image from the imaging sensor as disclosed herein; b) generating a measurement matrix based on parameters of the optical encoder; c) inputting the combined image and a measurement matrix to the deep learning algorithm; and d) generating at least four reconstructed images.

[0072] In some embodiments, the step of obtaining the combined image (step a)) comprises simultaneously irradiating the imaging sensor with at least four different wavelengths of electromagnetic wave.

[0073] In some embodiments, the step of obtaining the combined image (step a)) comprises sequentially irradiating the imaging sensor with at least four different wavelengths of electromagnetic wave. Each wavelength may be applied individually to the sample, and the emitted light is captured separately by the imaging system. This approach ensures that the system can accurately measure and distinguish the emission response for each specific wavelength, minimizing any cross-talk or interference between different electromagnetic waves.

[0074] In some embodiments, the combined image is undersampled. Undersampling is where one samples a signal at a sample rate below its Nyquist rate (twice the upper cutoff frequency), but is still able to reconstruct the signal.

[0075] In some embodiments, the deep learning algorithm uses an iterative shrinking process to generate the at least four reconstructed images.

[0076] In some embodiments, the parameters comprises a transducer arrangement. The transducer arrangement indicates the distribution of the transducer in the random coded matrix (as 0 or 1).

[0077] The present disclosure concerns a method of imaging a plurality of objects using the imaging sensor as disclosed herein, comprising: a) positioning the objects in electromagnetic communication with the imaging sensor; b) exposing the objects to at least four electromagnetic waves absorbable by the optical encoder of the imaging sensor; and c) obtaining a combined image from the CCD of the imaging sensor.

[0078] The present disclosure concerns a method of imaging an object at multiple depths using the imaging sensor as disclosed herein, comprising: a) positioning the object in electromagnetic communication with the imaging sensor; b) exposing the object to at least four electromagnetic waves absorbable by the optical encoder of the imaging sensor and focusing the at least four electromagnetic waves at different depths and / or magnification of the object; and c) obtaining a combined image from the CCD of the imaging sensor.

[0079] In some embodiments, the method further comprises reconstructing the combined image into at least four reconstructed images.

[0080] Examples

[0081] Fabrication of multi-wavelength compressed imaging sensors

[0082] Screen printing is a printing method to transfer intricate designs onto various substrates, including textiles, ceramics, glass, and other materials. Referring to Figure 7, first, the stencil is designed with a random hole array for every wavelength channel. Using computer-aided design (CAD) software, four different digital stencils are created for four different materials. The stencils are fabricated using a precise laser cutting process and have a thickness of 30 micrometers. Then, a flat glass substrate coated with transparent, high-quality double-sided adhesive is used as a processing base. The stencil is carefully positioned on the glass substrate to ensure proper alignment. Subsequently, a uniform layer of 100% powder is evenly spread onto the stencil. The powder is then pressed firmly to fill the template pattern and adhere to the double-sided tape on the glass substrate. Following the purging of the template with nitrogen to remove any loose powder residue, the stencil is gently removed.

[0083] After screen-printing a single array of optical materials onto a glass substrate coated with double-sided tape, the process was repeated to create the second, third, and fourth arrays of optical materials, each designed to respond to different wavelengths. To ensure proper alignment of subsequent arrays, the layer deposited on the glass substrate is placed on the mask holder of a mask aligner (URE-2000 / 17). The arrays are manually aligned using the x, y, and tilt axes. The aligned array is then pressed to ensure that the plane is flat, and finally the sample is removed from the aligner. A 200 pm-thick layer of PDMS is applied to the sample and heated to 60°C after vacuuming. Once solidified, the PDMS, featuring four imprinted patterns, is peeled off from the glass substrate, resulting in an approximately powder-to-PDMS ratio of 4: 1 in each pixel.

[0084] Fundamental principle of stochastic photoluminescence and compressed encoding

[0085] To illustrate the process of how a luminescent material array samples an intensity image, we can use an example of a specific wavelength channel, such as SrAl2O4: Eu2+ / Dy3+material array responding to 375 nm. The complete intensity image with N x N pixels encoded with the 375-nm radiation can be represented by a two- dimensional matrix leNxN. When the intensity image is incident on the randomly arranged luminescent material array, the array samples the image in a random and undersampled manner. This sampling process can be described by a sampling matrix <p, which represents the locations of the luminescent materials in the array. The image collected by the CCD is the result of linear multiplication of the complete intensity image I and the measurement matrix <p, which represents the sampling process of the luminescent material array. This process can be described as a matrix equation:

[0086] H = I ■ <p (1) where H is the measured image by CCD.

[0087] In the array of luminescent materials, the pixels that do not respond to the 375nm excitation light do not emit light. Therefore, the measurement matrix p has a value of 0 at those positions and 1 at the pixel positions that respond to the 375 nm excitation. If the N x N pixels in the intensity image are equally divided into four wavelength channels, then the number of pixels with a value of 1 in the measurement matrix cp is M = N2 / 4 (Supplementary Fig. S3). This means that in the image H measured by the CCD, only M pixels are useful and contain the image information, while the other values are all 0 and can be removed. By removing the redundant 0 values, we can effectively reduce the size of the image and improve the efficiency of image processing.

[0088] In practice, a large number of images need to be processed. To simplify the calculations, the image matrix I is treated as a vector x e N2x l by stacking the columns of the matrix I. The measurement matrix <p is transformed into JjeM x N2, where each row has only one pixel with a value of 1 and the others are all 0. Accordingly, the image H collected by the CCD can be represented as a vector yeM x 1, and expressed as: y = 5 ■ x + e (2) where e is the noise. The objective of various compressed sensing reconstruction algorithms is to solve for complete image x in terms of y and J;.

[0089] Iterative Shrinkage-Thresholding Algorithm-Network (ISTA-Net)

[0090] Given the measurements y, the traditional CS reconstruction algorisms usually reconstruct the original image x by solving the following (generally convex) optimization problem:

[0091] Here, iPx denotes the transform coefficients of x with respect to some transform ip, and the sparsity of the vector iPx is promoted by the fi norm. is a generally predefined regularization parameter that controls the trade-off between the sparsity of 4»x and the fidelity of the reconstructed image.

[0092] The iterative shrinkage-thresholding algorithm (ISTA) is a first-order proximal method that solves the compressed sensing reconstruction problem in Eq. (3) by iterating between the following update steps:

[0093] Here, k denotes the ISTA iteration index, and p is the step size.

[0094] The choice of a sparse transformation IP plays a critical role in the performance of the CS reconstruction. The commonly used transforms include the wavelet transform, the discrete cosine transform, and the Fourier transform. In the ISTA-Net approach, a general nonlinear transform function was used to sparsify natural images, denoted by H( • ), with learnable parameters. H( ■ ) was designed as a combination of two linear convolution operators (without bias terms) separated by a rectified linear unit (ReLU). By replacing ip in Eq. (5) with H( • ), it becomes xMcan be efficiently computed in closed-form as: where, 9 = Aa, a is a scalar that is only related to the parameters of H( • ). 0, as a shrinkage threshold, is a learnable parameter. • ) is designed to exhibit a structure symmetrical to H( • ).

[0095] Each module in each phase of ISTA-Net corresponds to the update steps in an ISTA iteration. The learnable parameter set in ISTA-Net consists of the step size p(k)in the r<k)module, the parameters of the forward and backward transforms H( • ), and the shrinkage threshold 0(k) in the x(k) module.

[0096] The end-to-end loss function for ISTA-Net is as follows: where Np, N , N, and y are the total number of ISTA-Net phases, the total number of training blocks, the size of each block Xi, and the regularization parameter, respectively.

[0097] Configurations of all algorithms

[0098] We began by downloading the T91 Image Dataset from the Kaggle platform, which consists of 91 images. Each image was enlarged by a factor of 3 and then cropped into approximately 60,000 sub-images with a pixel size of 100 x 100. Adjacent sub-images have an overlap of 75 pixels. These 60,000 sub-images were used as the training dataset for ISTA-Net. ISTA-Net was implemented in Python 3.7 using Tensorflow 1.13.1 as the operating environment, and Pycharm as the integrated development environment. All experiments were conducted on a workstation equipped with an E5- 2680V3 CPU and an RTX 2080 Ti GPU. The standard dataset Setll was used for testing, which contains 11 grayscale images with multi-wavelength channels. The orthogonal matching pursuit algorithm was compiled using Matlab2020a.

[0099] Training and reconstruction process

[0100] Specific training steps involve generating a corresponding measurement matrix <p based on the given compressive sensing ratio and the designed pattern of a randomly distributed encoding matrix. The size of the matrix <p is MxN, where N = 10,000, meaning that each sub-image has a size of 100x 100. M = CxN, where C represents the compression ratio. Applying y = ipx produces a set of CS measurements, where x is the vectorized version of sub-image blocks. The original image x and the corresponding sparse measurement data y constitute the training dataset. The network architecture used and the definition of the loss function are described in the section 'Iterative Shrinkage-Thresholding Algorithm-Network (ISTA-Net)'. The training dataset, i.e., the original image x and sparse measurement data y, are provided as inputs to the neural network. The network maps the y to a reconstructed image x through forward propagation. The network employs a loss function to evaluate the quality of the reconstructed image x compared with x, and utilizes the backpropagation algorithm to compute gradients. Subsequently, the network's weights and biases are updated using the gradient descent to minimize the loss function. These steps are repeated iteratively to train the neural network, with a training duration of 100 epochs. Once the training is complete, we can use the trained neural network to perform compressive sensing reconstruction on new measurement data y, generating high-quality reconstructed images x. Unlike random initialization of x, a linear mapping matrix is used here, which is learned from training data pairs of CS measurement data and the corresponding image blocks. This matrix is then employed to compute the initial estimate of the image for the compressive sensing reconstruction process.

[0101] Compact all-in-one sensor employing stochastic photoluminescence and compressed encoding (SPACE) for multi-wavelength, multi-channel imaging. In one embodiment, an optical encoder was developed through screen printing, using a stochastically distributed array of various lanthanide transducers that can converting a broad range of excitation light wavelengths into detectable visible light. The resulting device is directly integrated with a charge-coupled device (CCD) to create a multi-wavelength encoded imaging sensor. The active optical encoder, utilizing various lanthanide transducers, converts multiple wavelengths into the visible range while encoding wavelength-tagged images into a single image detected by the CCD. To reconstruct wavelength-tagged images from the encoded image, we utilize an end-to-end compressed sensing deep network. The currently demonstrated imaging sensor covers four wavelength channels of X-rays (0.089 nm), ultraviolet (375 nm), and near-infrared region I (808 nm) and region II (1532 nm), which cannot be achieved by traditional passive optical component-based methods. We demonstrated the multi-channel imaging using different wavelengths of light to expose different depths of the sample. Moreover, it is possible to fabricate multilayer active optical encoders based on lanthanide transducers with different emission colors and control the pixel thickness to extend the number of wavelength channels. We simultaneously acquired two images of gamma-rays (6 MeV) and X-rays (~15 KeV) by controlling the thickness of the encoder material layer. Compressive coded imaging combined with advanced nanofabrication techniques and abundant luminescent materials enables large-area, ultra-broad spectral response range, and on-chip optical system-compatible multi-wavelength imaging schemes, with significant potential for applications in multi-channel bioimaging, multispectral imaging, encrypted communication, and among other.

[0102] Lanthanide-doped luminescent materials were selected as transducers. These transducers were randomly arranged in complementary manner (side-by-side), and then integrated with a commercial CCD, constructing an active multi-wavelength encoded imaging sensor (Fig. 1A). In multi-wavelength encoded imaging sensor, each randomly arranged transducer array undersampled the tissue image and convert its specific wavelength into the visible range. The compressed sensing reconstruction algorithm was applied to recover structural information from the undersampled image. Specifically, the designed optical encoder, consisting of random arrays of four types of lanthanide transducers, captured four images tagged with different wavelengths. The CCD integrated under the encoder collected the luminescence from the transducers, resulting in a coded image containing the relevant information of the four images (Fig. IB). The designed random patterns of the encoder and the collected encoded images were then fed into a trained machine learning reconstruction network, generating four complete reconstructed images.

[0103] An active optical encoder was fabricated using a screen-printing process, consisting of four types of randomly arranged lanthanide transducers. Under the excitation of X-rays and specific wavelengths (375 nm, 808 nm, and 1532 nm), these transducers emitted visible light (Fig. 2A). An image can be represented as a two-dimensional matrix. Stacking the columns of the matrix gives a vector x e / V2x l, where N is the height and width of the image in pixels. For one type of transducer, undersampling an image by an NxN random array can be mathematically equivalent to linearly multiplying the vector x by a measurement matrix <PeMx.N2, where M / N2is the compressed sampling rate. We obtained a vector y with only M pixels. Compressed sensing reconstruction involves reconstructing the original image x from y and (Fig. 2B). An iterative shrinkage-thresholding Algorithm (ISTA) network (ISTA-Net) was used to reconstruct final images. The network takes compressed measurement images corresponding to four wavelengths and the measurement matrix corresponding to the encoder as inputs. Subsequently, it produces four complete reconstructed images (Fig. 3).

[0104] The abundant energy levels of lanthanide ions enable effective adjustment of the emission wavelength of lanthanide transducers. This adjustment, when combined with a color RGB CCD, allows for the realization of a multi-layer, multi-wavelength coded imaging sensor, theoretically increasing the number of wavelength channels by threefold (Fig. 4). For example, a three-layer encoder with 10 types of transducers in each layer, and a sampling ratio of 10%, can encode 30 wavelength channel images.

[0105] We next developed a system for multiple-depth imaging by tagging different portions of a sample with different-wavelength light. This enables the acquisition of depth images simultaneously using multi-wavelengths. As a proof-of-concept, we constructed an imaging system that utilizes three wavelengths of light to image various depths of a sample and X-rays to capture interior images (Fig. 5A). When imaging a dragonfly's wings and partial torso, different wavelengths of light can focus on and magnify structures at different depths. However, the torso remains impenetrable to light. Conversely, X-rays can visualize the interior of the torso, but the wing structure lacks clear imaging due to insufficient absorption contrast (Fig. 5B).

[0106] We also discovered that by controlling the thickness of the layer, the optical encoder can image high-energy radiation with varying energies. The attenuation thickness differs for KeV-X-rays and MeV-gamma rays, measuring at submillimeter and centimeter levels, respectively (Fig. 6A and B). An optical encoder comprising randomly arranged pixel arrays with two thicknesses was fabricated, including 2 mm thickness (for 6 MeV) and 0.2 mm thickness (for 15 KeV). By utilizing a sampling rate of 50%, we reconstructed the X-ray image of a stencil and the gamma-ray beam shape from medical radiotherapy equipment (Fig. 6C and D).

[0107] In summary, an active optical encoder was manufactured using a screen-printing process, consisting of four randomly arranged lanthanide transducers. This system demonstrates simultaneous imaging capabilities for X-rays and multiple wavelengths (375 nm, 808 nm, and 1532 nm). Multi-channel imaging with distinct wavelength tagging has been validated. Furthermore, the use of a multilayer encoder and encoder with varying pixel thicknesses has also been demonstrated to expand the number of wavelength channels. The imaging sensor may be used for multi-organelle collaborative observation, multi-component protein analysis, and multi-wavelength space imaging.

[0108] It will be appreciated that many further modifications and permutations of various aspects of the described embodiments are possible. Accordingly, the described aspects are intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.

[0109] Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

[0110] Throughout this specification and the claims which follow, unless the context requires otherwise, the phrase "consisting essentially of", and variations such as "consists essentially of" will be understood to indicate that the recited element(s) is / are essential i.e. necessary elements of the invention. The phrase allows for the presence of other non-recited elements which do not materially affect the characteristics of the invention but excludes additional unspecified elements which would affect the basic and novel characteristics of the method defined.

[0111] The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that that prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.

Claims

Claims1. An imaging sensor, comprising : a) an optical encoder comprising a layer of at least four transducers stochastically arrayed on a substrate, the four transducers configured to each independently absorb a different wavelength of electromagnetic wave; and b) a charge-coupled device (CCD) in electromagnetic communication with the optical encoder for receiving visible light from the at least four transducers and outputting a combined image.

2. The imaging sensor according to claim 1, wherein the at least four transducers each independently output a different wavelength of visible light.

3. The imaging sensor according to claim 1 or 2, wherein the at least four transducers is at least four different types of lanthanide-doped luminescent phosphors transducers.

4. The imaging sensor according to any one of claims 1 to 3, wherein the at least four transducers each independently comprises phosphors having a particle size of about 50 nm to about 5 pm.

5. The imaging sensor according to any one of claims 1 to 4, wherein the optical encoder is characterised by a resolution of about 10 pm to about 300 pm.

6. The imaging sensor according to any one of claims 1 to 5, wherein each of the at least four transducers is characterised by a pixel size of about 10 pm to about 300 pm.

7. The imaging sensor according to any one of claims 1 to 6, wherein the substrate is a transparent elastomer.

8. The imaging sensor according to any one of claims 1 to 7, wherein the substrate is selected from polydimethylsiloxane (PDMS).

9. The imaging sensor according to any one of claims 1 to 8, wherein each of theat least four electromagnetic waves is independently selected from X-rays, ultraviolet, near-infrared region I, and near-infrared region II.

10. The imaging sensor according to any one of claims 1 to 9, wherein the at least four electromagnetic waves are selected from X-rays with a wavelength of about 0.089 nm, ultraviolet with a wavelength of about 375 nm, near-infrared region I with a wavelength of about 808 nm, and near-infrared region II with a wavelength of about 1532 nm.

11. The imaging sensor according to any one of claims 1 to 10, wherein the layer of at least four transducers is characterised by a thickness of about 0.01 mm to about 5 mm.

12. The imaging sensor according to any one of claims 1 to 11, wherein the optical encoder further comprises a second layer of at least four transducers stochastically arrayed and adjacent to the layer on the substrate.

13. The imaging sensor according to any one of claims 1 to 12, wherein the optical encoder comprises a plurality of layers adjacent to each other on a substrate, each layer having at least four transducers stochastically arrayed.

14. The imaging sensor according to any one of claims 1 to 13, wherein the CCD is a monochrome CCD or RGB CCD.

15. The imaging sensor according to any one of claims 1 to 14, wherein the imaging sensor further comprises a filter configured to remove excitation light.

16. The imaging sensor according to any one of claims 1 to 15, wherein the imaging sensor further comprises a deep learning algorithm configured to receive the combined image from the CCD and reconstruct it into at least four reconstructed images.

17. A method of fabricating an imaging sensor, comprising : a) screen printing at least four transducers stochastically arranged as a layer on a substrate in order to form an optical encoder, the four transducers configured to each independently absorb a different wavelength of electromagnetic wave; andb) electromagnetically communicating a charge-coupled device (CCD) with the optical encoder for receiving visible light emitted from the at least four transducers.

18. A method for reconstructing a combined image from an imaging sensor according to any one of claims 1 to 16into at least four reconstructed images using a deep learning algorithm, the image analysis method comprising: a) obtaining the combined image from the imaging sensor; b) generating a measurement matrix based on parameters of the optical encoder; c) inputting the combined image and a measurement matrix to the deep learning algorithm; and d) generating at least four reconstructed images.

19. The method according to claim 18, wherein the deep learning algorithm uses an iterative shrinking process to generate the at least four reconstructed images.

20. The method according to claim 18 or 19, wherein the parameters comprises a transducer arrangement.

21. A method of imaging a plurality of objects using the imaging sensor according to any one of claims 1 to 16, comprising: a) positioning the objects in electromagnetic communication with the imaging sensor; b) exposing the objects to at least four electromagnetic waves absorbable by the optical encoder of the imaging sensor; and c) obtaining a combined image from the CCD of the imaging sensor.

22. A method of imaging an object at multiple depths using the imaging sensor according to any one of claims 1 to 16, comprising : a) positioning the object in electromagnetic communication with the imaging sensor; b) exposing the object to at least four electromagnetic waves absorbable by the optical encoder of the imaging sensor and focusing the at least four electromagnetic waves at different depths and / or magnification of the object; and c) obtaining a combined image from the CCD of the imaging sensor.

23. The method according to claim 21 or 22, further comprising reconstructing the combined image into at least four reconstructed images.

Images