Lensless holographic multispectral reconstruction method, equipment and medium

Abstract

The invention is suitable for the technical field of optics and computers, and particularly relates to a lensless holographic multispectral reconstruction method, equipment and a medium. The method comprises the following steps: acquiring first original holographic image information of a target sample; extracting vibration information in the first original holographic image information; and determining second spectral information of the target sample according to the vibration information, preset first spectral information of a first reference sample and preset vibration information of the reference sample.

Description

Technical Field

[0001] This application belongs to the fields of optical technology and computer technology, and particularly relates to a lensless holographic multispectral reconstruction method, device, and medium. Background Art

[0002] Lensless holographic imaging is an innovative method that breaks through the traditional microscopic imaging mode. It has the imaging advantages of a large field of view and high resolution, and at the same time avoids the complex and expensive optical elements required by the microscopic lens system. It is particularly suitable for observing samples such as microorganisms, cell tissues, and microparticles, and can provide a wider field of view and higher resolution than traditional microscopes.

[0003] For transparent or semi-transparent objects such as microorganisms, cell tissues, and microparticles, the transmittance spectrum is also of great significance as an identification feature in microscopic imaging. Different molecular structures of an object will affect the absorption and reflection characteristics of the object to the light source, and these changes can be reflected through spectral information. Summary of the Invention

[0004] Embodiments of this application provide a lensless holographic multispectral reconstruction method, device, and medium, which can reconstruct the transmittance spectrum of an object using lensless holographic imaging technology.

[0005] In a first aspect, embodiments of this application provide a lensless holographic multispectral reconstruction method, including:

[0006] Obtain the first original holographic image information of a target sample;

[0007] Extract the vibration information from the first original holographic image information;

[0008] Determine the second spectral information of the target sample according to the vibration information, the first spectral information of a preset first reference sample, and the vibration information of a preset reference sample.

[0009] In an implementation manner, the vibration information includes amplitude and phase; the extracting the vibration information from the original holographic image information includes:

[0010] Determine the phase in the first original holographic image information according to the wavelength in the first original holographic image information and the frequency-domain coordinates of each point in the first original holographic image information;

[0011] Determine the amplitude in the first original holographic image information according to the light intensity distribution in the first original holographic image information and the phase in the first original holographic image information.

[0012] In one embodiment, when the vibration information includes the amplitude, determining the second spectral information of the target sample according to the vibration information, the first spectral information of a preset first reference sample, and the vibration information of a preset second reference sample includes:

[0013] Determining the correlation between the vibration information and the first spectral information according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample;

[0014] Determining the second spectral information of the target sample according to the correlation between the vibration information and the first spectral information.

[0015] In one embodiment, determining the correlation between the vibration information and the first spectral information according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample includes:

[0016] Determining an estimation matrix according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample;

[0017] Determining the correlation between the vibration information and the first spectral information according to the phase of the first original holographic image information and the estimation matrix.

[0018] In one embodiment, determining the estimation matrix according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample includes:

[0019] Determining the covariance matrix of the transmittance of the first reference sample according to the first spectral information of the first reference sample;

[0020] Determining the spectral function of the second reference sample according to the vibration information of the second reference sample;

[0021] Determining the estimation matrix according to the spectral function of the light irradiating the first reference sample, the response function of the camera for obtaining the second original holographic image of the second reference sample, and the covariance matrix.

[0022] In one embodiment, the covariance matrix of the transmittance is obtained by discretizing the transmittance spectrum of the first reference sample into a preset number of bands and then taking the average.

[0023] In one embodiment, the second reference sample includes a preset number of preset items; the second reference sample is disposed in the light projection area; the target item is also disposed in the light projection area; obtaining the first original holographic image information of the target sample includes:

[0024] Project light with a preset color onto the light projection area to obtain the first original holographic image information of the target sample and the second original holographic image information of the second reference sample.

[0025] In one implementation, the first original holographic image information includes the original holographic image information under illumination of a set wavelength; the first spectral information of the pre-set first reference sample includes the spectral information when the first reference sample is illuminated by white light.

[0026] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method provided by any embodiment of the present application is implemented.

[0027] According to another aspect of the present application, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, the computer is made to execute the method provided by the relevant embodiments of the first aspect above.

[0028] As can be seen from the above, the embodiments of the present application can pre-measure the spectral information of the first reference sample and determine the spectral information of the target sample according to the first original holographic image information of the target sample to be measured. Thus, for the phenomenon that the target sample is prone to overlap and irregular position in the natural scene, the spectral information of the target sample can be quickly obtained. The method provided by the embodiments of the present application reconstructs the transmittance spectrum of the sample through the complex amplitude image reconstructed from the hologram, overcoming the shortcomings of lack of spectral information in lensless holography and poor real-time performance in multispectral imaging, and showing great application potential in multiple fields such as environmental monitoring and medical detection. Description of the Drawings

[0029] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 is a schematic flowchart of the method provided by an embodiment of the present application;

[0031] Figure 2 is a schematic structural diagram of obtaining the original holographic image in the method provided by an embodiment of the present application;

[0032] Figure 3 is a schematic structural diagram of obtaining the original holographic image in the method provided by the embodiment of the present application;

[0033] Figure 4 Another flowchart of the method provided by the embodiment of the present application;

[0034] Figure 5 A schematic structural diagram for obtaining the first spectral information of the first reference sample in the embodiment of the present application. Detailed implementation manners

[0035] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures and technologies are presented in order to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0036] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0037] It should also be understood that the term "and / or" as used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0038] As used in the specification of the present application and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" according to the context.

[0039] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but rather mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0041] The embodiments of the present application will be exemplarily described below with reference to the accompanying drawings.

[0042] Figure 1 A lensless holographic multispectral reconstruction method in an embodiment of the present application includes steps as Figure 1 shown.

[0043] Step S11: Obtain the first original holographic image information of the target sample.

[0044] The target sample may include microorganisms, microplastics, pollen, dust, etc. The target sample is an object whose spectral information is not known in advance, and the target sample may also be referred to as the sample to be measured. When obtaining the first original holographic image of the target sample, the optical path composition structure may refer to Figure 2 shown. The target sample 21 can be placed on a glass slide, and the light from the light source 22 is used to irradiate the glass slide, and the camera 23 is used to capture and obtain the first original holographic image information of the target sample.

[0045] In the case where the target sample includes microorganisms, the microorganisms may include seaweeds. The seaweed sample can be placed on a glass slide, and a small amount of seawater or buffer solution is used to keep it moist to avoid drying and deformation. In the case where the target sample includes microplastics, the microplastic particles are dispersed in deionized water or a transparent medium to ensure uniform distribution.

[0046] The first original holographic image information can be obtained based on the lensless holographic imaging technique. The basic principle of the lensless holographic imaging technique is to use a coherent or partially coherent light source (such as a laser diode, LED, etc.) to illuminate the sample, and directly obtain the hologram of the sample on the sensor of the camera through the way of in-line holography. Through digital calculation and reconstruction algorithms, multi-dimensional information such as the amplitude, phase map, and three-dimensional position of the sample is extracted from the hologram. The lensless holographic imaging technique can not only provide high-resolution two-dimensional images, but also obtain three-dimensional structure information, helping scientists achieve more accurate measurements and analyses at the microscopic scale. Its characteristics of large field of view and high resolution show great application potential in the field of microscopic imaging, especially in real-time imaging, rapid detection, and the development of portable devices. With the continuous development of technology, lensless holography is expected to play a greater role in the fields of biomedicine, environmental monitoring, material analysis, etc.

[0047] A laser beam can be split by a beam splitter into a reference beam and an object beam. The object beam passes through the sample and then interferes with the reference beam to form a hologram. Adjust the optical path so that the optical path difference between the reference beam and the object beam is close to zero to ensure clear interference fringes. Then, the sensor of a CCD / CMOS (Charge-coupled Device / Complementary Metal-Oxide-Semiconductor) camera can be used to record the holographic image information to obtain the first original holographic image information.

[0048] Step S12: Extract the vibration information from the first original holographic image information.

[0049] When the target sample is irradiated by light, the phase and amplitude of the light scattered by the target sample change with time, and these changes are recorded in the holographic image information. The first holographic image information contains the spatial distribution and frequency information of the light scattered by the target sample during vibration.

[0050] The vibration information can include at least one of amplitude, frequency, and phase. The vibration information extracted in step S12 can be the vibration information of the light scattered when the target sample is irradiated by light.

[0051] Step S13: Determine the second spectral information of the target sample according to the vibration information, the first spectral information of the preset first reference sample, and the vibration information of the preset second reference sample.

[0052] The preset first reference sample can be a light-transmitting thin sheet with an area significantly larger than that of the target sample. For example, a rectangular PVC (Polyvinyl chloride) thin sheet can be used as the preset reference sample.

[0053] A pre-set second reference sample, which can be particles of the first reference sample. The order of magnitude of the size of the second reference sample can be consistent with the order of magnitude of the size of the target sample.

[0054] In the embodiments of the present application, spectral information may refer to information of a spectrum. A spectrum may refer to an image of the distribution of light or other electromagnetic waves according to wavelength or frequency, and is generally divided into a continuous spectrum and a discrete spectrum. The spectral information may include: (1) wavelength / frequency, the wavelength or frequency distribution of light; (2) intensity, the intensity of light at different wavelengths or frequencies; (3) polarization, the polarization state of light; (4) time characteristics, the change of the spectrum over time.

[0055] Accurate spectral analysis methods have been widely used for qualitative analysis of object components and are of great significance for the study of object components. Traditional methods for measuring the transmittance spectrum of an object rely on spectral instruments. By sequentially irradiating the sample with light sources of different single wavelengths and measuring the change in the intensity of the transmitted light, the transmittance spectrum is calculated. This process usually requires precise optical instruments and long-term data acquisition, and the operation of multi-wavelength measurement is complex and time-consuming, which limits its application in dynamic monitoring.

[0056] Traditional lensless holographic imaging generally uses monochromatic light as the light source to generate light rays to irradiate the target sample, and reconstructs information such as the refractive index, two-dimensional shape map, and three-dimensional topography of the target sample through the obtained hologram. Although this information can be used to preliminarily identify the type of the sample, it lacks specificity. For example, many microalgae have circular or quasi-elliptical contour features, or when multiple target samples of different types overlap, the contour features of the overlapping multiple target samples become blurred or stacked, and it will be difficult for the image-based classification and recognition method to identify the target sample. For target samples such as biological sols like microalgae and pollen, the aggregation and overlap of target samples are very common, which are caused by factors such as static electricity, molecular force, surface affinity, and hydrodynamic force. Therefore, how to achieve more effective identification and classification through lensless holographic imaging technology when the two-dimensional image features fail is still a challenging problem faced by this technology currently.

[0057] The transmittance characteristic is one of the optical characteristics of the target sample. The transmittance characteristic of the target sample is affected by the molecular structure, and the transmittance spectra of different target samples will exhibit obvious specificities. Traditional lensless holographic imaging treats the target sample as a transparent object, ignores the absorption effect of the target sample, and obtains its refractive index representing the phase delay. However, the refractive index of the target sample is actually a complex number a + bi. Traditional lensless holographic imaging only focuses on its real part a and ignores its imaginary part b. Although the imaginary part b can be obtained during the reconstruction calculation, the absorption rate at a single wavelength cannot display the specificity of the sample and is not very meaningful. Although traditional multispectral imaging techniques can obtain transmittance data at multiple wavelengths, there are still problems such as long processing time and large data volume. In addition, traditional multispectral imaging techniques usually process spectral data separately from the two-dimensional image of the target sample and cannot fully combine spectral information with the two-dimensional characteristics of the sample for comprehensive identification. This lack of fusion of spectral and spatial characteristics limits the accuracy and efficiency of multispectral imaging in material recognition and classification, especially in applications with complex samples and high-precision requirements. Therefore, how to effectively fuse lensless holographic imaging technology and multispectral imaging technology remains an important challenge to be overcome in the field of optical technology.

[0058] Through the method provided by the embodiments of the present application, the spectral information of the first reference sample can be measured in advance, and the spectral information of the target sample can be determined according to the first original holographic image information of the target sample to be measured. Thus, in view of the phenomenon that the target sample is prone to overlap and irregular positions in the natural scene, the spectral information of the target sample can be obtained quickly. The method provided by the embodiments of the present application reconstructs the transmittance spectrum of the sample through the complex amplitude image reconstructed from the hologram, overcomes the shortcomings of the lack of spectral information in lensless holography and the poor real-time performance in multispectral imaging, and shows great application potential in multiple fields such as environmental monitoring and medical detection.

[0059] In one implementation manner, the vibration information includes amplitude and phase; extracting the vibration information from the original holographic image information includes:

[0060] Determining the phase in the first original holographic image information according to the wavelength in the first original holographic image information and the frequency-domain coordinates of each point in the first original holographic image information;

[0061] Determining the amplitude in the first original holographic image information according to the light intensity distribution in the first original holographic image information and the phase in the first original holographic image information.

[0062] The phase in the first original holographic image information may be the phase of the light reflected by the target object in the first original holographic image information.

[0063] In a possible implementation manner, the architecture for obtaining the first original holographic image information is as follows Figure 3 As shown, the first original holographic image information is obtained by means of lensless in-line holography. The laser light source 31 can emit RGB (red, green, blue) monochromatic lights with central wavelengths of 658 nm, 520 nm, and 450 nm respectively. A group of PVC sheets 33 with pre-determined transmittance spectra and a sample to be measured 34 (a variety of marine microalgae) are placed on the glass slide 32. The background reference light directly passing through the glass slide in the original light field interferes with the transmitted light modulated by the PVC sheet and the target sample, and a diffraction intensity pattern is obtained on the camera sensor. The part of the diffraction intensity pattern related to the marine microorganisms is the first original holographic image information. By calculating the first original holographic image information, the two-dimensional image and the transmittance spectrum of the sample can be obtained simultaneously.

[0064] In an implementation manner, when the vibration information includes amplitude, determining the second spectral information of the target sample according to the vibration information, the first spectral information of a pre-set first reference sample, and the vibration information of a pre-set second reference sample includes:

[0065] Determining the correlation relationship between the vibration information and the first spectral information according to the first spectral information of the pre-set first reference sample and the vibration information of the pre-set second reference sample;

[0066] Determining the second spectral information of the target sample according to the correlation relationship between the vibration information and the first spectral information.

[0067] In an implementation manner, determining the correlation relationship between the vibration information and the first spectral information according to the first spectral information of the pre-set first reference sample and the vibration information of the pre-set second reference sample includes:

[0068] Determining an estimation matrix according to the first spectral information of the pre-set first reference sample and the vibration information of the pre-set second reference sample;

[0069] Determining the correlation relationship between the vibration information and the first spectral information according to the phase of the first original holographic image information and the estimation matrix.

[0070] In an implementation manner, determining the estimation matrix according to the first spectral information of the pre-set first reference sample and the vibration information of the pre-set second reference sample includes:

[0071] Determining the covariance matrix of the transmittance of the first reference sample according to the first spectral information of the first reference sample;

[0072] Determine the spectral function of the second reference sample according to the vibration information of the second reference sample;

[0073] Determine the estimation matrix according to the spectral function of the first reference sample irradiated, the response function of the camera for obtaining the second original holographic image of the second reference sample, and the covariance matrix.

[0074] In one embodiment, the covariance matrix of the transmittance is obtained by averaging the transmittance spectrum of the first reference sample after discretization into a preset number of bands.

[0075] In a possible implementation, light sources of different wavelengths such as red, green, and blue (R, G, B) are used to irradiate the target sample and the second reference sample to obtain the first original holographic image information of the target sample and the second original holographic image information of the second reference sample, as the Figure 4 original holograms shown. The first original holographic image information and the second original holographic image information can be obtained by using the device shown in Figure 3 . Extract the amplitude information according to the first original holographic image information and the second original holographic image information, and obtain the monochromatic reconstruction image corresponding to each wavelength through a reconstruction algorithm. Referring to Figure 4 shown, the target sample is set in the measurement area gx, and the second reference sample is set in the reference area gc. Then, using the intensity distribution of these reconstruction images, estimate the spectral reflectance distribution of the target through a spectral estimation method.

[0076] After obtaining the original hologram, it is first necessary to reconstruct the original hologram to extract the complex amplitude information of the target sample and the second reference sample from the hologram. The complex amplitude information includes the amplitude and phase information of the sample, and the complex amplitude information can be used for subsequent spectral estimation. Exemplarily, the angular spectrum algorithm can be used to obtain the complex amplitude information. The angular spectrum algorithm is a common reconstruction algorithm widely used in holographic imaging. Its principle is based on the propagation characteristics of light waves. Assuming that the light waves propagate at a certain angle, the holographic image is processed in the frequency domain through Fourier transform, so as to realize the simulation of the light wave propagation from the object plane to the image plane. By calculating the propagation of light waves at different spatial positions, the angular spectrum algorithm can effectively restore the complex amplitude information of the target sample and the second reference sample. The main advantages of the angular spectrum method are its high computational efficiency, simple implementation, and high accuracy, and it can quickly process large-size holograms. The expression of the angular spectrum algorithm is:

[0077] g(x,y) = IFFT{FFT[I(x,y)]·H(u,v)};

[0078]

[0079] Among them, g(x, y) is the original light field obtained after reconstruction, that is, the above-mentioned amplitude. I(x, y) is the light intensity distribution of the hologram. u and v are the coordinates corresponding to x and y in the frequency domain after Cartesian transformation. FFT{} and IFFT{} represent the fast Fourier transform and the inverse fast Fourier transform respectively. H(u, v) is the above-mentioned phase. x and y are the coordinates of any point in the original light field. j represents the imaginary unit in electromagnetics, k is the reciprocal of the wavelength (k = 2π / λ), and z is the diffraction distance.

[0080] To obtain the spectral transmittance result from the reconstructed light field intensity g(x, y) and perform multispectral reconstruction, the spectral estimation technique in the dashed box is required. Its essence is to fit the transmittance data to be measured from the known light field intensity-transmittance relationship through least squares estimation. The original light field g(x, y) is expressed using the wavelength λ as follows:

[0081] g = ∫I(λ)S(λ)T(λ)dλ.

[0082] Among them, I(λ) is the spectral function of the light source, S(λ) is the response function of the camera, and T(λ) is the spectral transmittance function at this point. In transmissive illumination imaging, the light field intensity at the object is actually composed of lights of multiple wavelengths, and each wavelength of light will be affected differently by the light source, the object, and the camera. Since the camera sensor can only respond to the light field intensity, the camera sensor will linearly integrate the energies of the lights of each wavelength that arrive. Therefore, this integral formula describes the physical process that the light source is modulated according to its transmittance characteristics after passing through the object, which conforms to the energy conservation and linear superposition principles of optics. For the convenience of calculation, in the embodiments of the present application, B(λ) is used to represent the quantity related to the imaging environment, that is, B(λ) = S(λ)T(λ).

[0083] For the conditions of using coherent or partially coherent light source illumination in lensless holography, it can be approximately regarded as monochromatic light illumination. When using monochromatic lights of different wavelengths for holographic imaging, it is equivalent to discretizing the above-mentioned continuous integral formula for wavelengths and converting the integral formula into a matrix form expression, that is, g = B·t. Assuming that the number of illumination wavelengths is n, and the spectrum is discretely sampled in m bands, g on the left side of the equation is the expression symbol of the original light field, and g is an n×1 vector. B is the expression symbol of B(λ), and B is an n×m matrix. t represents the spectral information, and t is an m×1 vector. Usually, m >> n, and t cannot be directly solved from g. Therefore, in the embodiments of the present application, determining the spectral information is converted into a problem of solving an underdetermined equation. The least squares method is a common technique for solving underdetermined equations, and the goal is to find the optimal estimate of t by minimizing the sum of the squares of the errors.

[0084] Specifically, in the process of solving the constrained equation, it is hoped to find a t such that H·t is closest to the known g, that is, to minimize the error:

[0085] For this reason, an estimation matrix A is introduced in the embodiments of the present application, and its calculation formula is:

[0086] A = C t H T (HC t H T ) -1 .

[0087] Among them, C t is the covariance matrix of the transmittance t, which is obtained by averaging the transmittance spectra of known samples after discretization in m bands. Its dimension is m×m and it contains the covariance information between the various bands of the transmittance. This matrix represents the statistical information of the reflectance of multiple samples. C t The specific calculation formula of is:

[0088]

[0089] And (HC t H T ) -1 is a weighting matrix, and C t can be regarded as a weighted prior of the transmittance t. By introducing C t into the equation, different bands can be "weighted" according to the statistical characteristics between the transmittances, and additional information (such as the change trend or smoothness of the transmittance) can be provided during the solution. Simply put, A is an estimation matrix obtained by weighted least squares, and the weight is provided by C t .

[0090] The first reference sample for calculating the covariance matrix can be an optical sample with standard transmittance parameters, such as plexiglass, PMMA (acrylic), etc., which all have extremely high transmittance in the visible light band and show obvious attenuation characteristics in the infrared or ultraviolet band; it can also be a known sample with a pre-determined transmittance curve, such as colored PVC, etc. Since the light source wavelengths used in holographic illumination are all within the visible light range, the latter is selected as the reference sample in this method. First, use a commercial spectrometer to measure the transmittance curve of a colored PVC thin sheet with a size of 1 cm×1 cm×0.1 mm under white light illumination. Since the transmittances of colored PVC thin sheets of different colors are different in each band, a specific transmittance curve can be obtained, and the covariance matrix C t is calculated therefrom; then the colored PVC thin sheet is cut into small squares with a size of 0.2 mm×0.2 mm×0.1 mm for use as a reference sample in the holographic field; finally, by calculating the estimation matrix A, the hologram intensity is mapped to the spectral transmittance space through t = A·g to obtain the spectral information t of the target sample.

[0091] In one embodiment, the second reference sample includes a preset number of preset articles; the second reference sample is disposed in the light projection area; the target article is also disposed in the light projection area; obtaining the first original holographic image information of the target sample includes:

[0092] Projecting light of a preset color onto the light projection area to obtain the first original holographic image information of the target sample and the second original holographic image information of the second reference sample.

[0093] The light of the preset color may include red, green, and blue light.

[0094] In one embodiment, the first original holographic image information includes the original holographic image information under illumination of a set wavelength; the first spectral information of the preset first reference sample includes the spectral information when the first reference sample is irradiated with white light.

[0095] Exemplarily, when obtaining the spectral information of the first reference sample, a measurement architecture as shown in Figure 5 may be adopted, including a light source, a first reference sample, a spectrometer, and a transmission integrating sphere. The first reference sample may be placed in a cuvette. Through the spectrometer, the light transmitted by the transmission integrating sphere can be received to obtain the spectral information of the first reference sample.

[0096] In one example of the present application, a colored PVC sheet is used as the reference sample. Utilizing its characteristics of rich color, wide spectral range, and easy customization, it can accurately reflect the transmission characteristics of the sample at a micro scale and can be conveniently integrated into the microscopic system, which is very suitable for the customization of microscopic multispectral imaging systems in different scenarios.

[0097] The spectral estimation technology proposed in the present application can reconstruct the multi-wavelength transmittance spectrum of the sample to be measured from a single-wavelength hologram. It overcomes the dependence of traditional spectral imaging technology on multi-wavelength illumination conditions, realizes high-precision spectral estimation under small sample conditions, and can meet the high-precision spectral reconstruction requirements of complex samples.

[0098] The lensless multi-spectral reconstruction method proposed in the present application combines high-resolution two-dimensional topography imaging with spectral feature extraction. Through the same imaging system, the spatial information and spectral information of the sample can be obtained simultaneously, providing richer multi-dimensional data for material analysis and precise classification of biological samples, and breaking through the limitation of traditional imaging technology that cannot obtain morphological and spectral information simultaneously.

[0099] By integrating lensless holography technology with hyperspectral technology, this method uses a reference sample to construct a mapping relationship between transmittance and hologram intensity. Then, an estimation matrix is established through numerical analysis methods, mapping the hologram intensity of the unknown region to the transmittance spectral space, realizing the transformation from single spatial information to multi-modal data containing spectral dimensions. For samples with similar two-dimensional shapes but different molecular characteristics, such as microplastics of different materials or different types of biological cells, specific spectral data is provided for more accurate classification;

[0100] The spectral estimation technology proposed in this method can use a small number of calibration samples to achieve high-precision, multi-dimensional spectral information extraction at the pixel level or region level of the hologram through a pre-established prior model. For the overlapping phenomenon of samples to be measured, such as aggregated microalgae or bioaerosols, etc., by extracting the spectral data in this region, the problem of difficult classification and identification using only single morphological data is solved.

[0101] This application integrates lensless holography and hyperspectral technology to establish a mapping relationship between holographic intensity and spectral data, and simultaneously realizes high-resolution two-dimensional imaging and transmittance spectral estimation, making up for the defect of lacking spectral data in the traditional microscopic scenario; This application uses the least squares method to establish an estimation matrix, realizing high-precision spectral information reconstruction under the condition of small samples, overcoming the difficulty of spectral estimation with a small number of calibration samples, and improving the classification and identification ability of complex samples.

[0102] The embodiment of this application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it realizes the steps executed by the server or any one of the kernels in the above-mentioned method embodiments.

[0103] The embodiment of this application provides a computer program product. When the computer program product runs on an electronic device, it enables the electronic device to execute the steps executed by the server or any one of the kernels in the above-mentioned method embodiments.

[0104] The embodiment of this application also provides a chip, which includes: a processing unit and a communication unit. The processing unit can be a processor, for example, and the communication unit can be an input / output interface, a pin, or a circuit, etc. The processing unit can execute computer instructions to enable the computer device to execute the steps executed by the server or any one of the kernels in any one of the method embodiments provided in this application.

[0105] Optionally, the computer instructions are stored in a storage unit.

[0106] Optionally, the storage unit is a storage unit within the chip, such as a register, cache, etc. The storage unit can also be a storage unit outside the chip within the terminal, such as a ROM or other types of static storage devices that can store static information and instructions, a random RAM, etc. Among them, the processor mentioned anywhere above can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above methods. The processing unit and the storage unit can be decoupled and respectively arranged on different physical devices, and connected by wired or wireless means to implement the respective functions of the processing unit and the storage unit, so as to support the system chip to implement various functions in the above embodiments. Alternatively, the processing unit and the memory can also be coupled on the same device.

[0107] An embodiment of the present application further provides a lensless holographic multispectral reconstruction device, including: a holographic image acquisition module, a vibration information extraction module, and a spectral information determination module.

[0108] Among them, the holographic image acquisition module is used to acquire the first original holographic image information of the target sample.

[0109] The vibration information extraction module is used to extract the vibration information in the first original holographic image information.

[0110] The spectral information determination module is used to determine the second spectral information of the target sample according to the vibration information, the first spectral information of the preset first reference sample, and the vibration information of the preset reference sample.

[0111] The lensless holographic multispectral reconstruction device provided by the embodiment of the present application can also implement the steps in the lensless holographic multispectral reconstruction method provided by any embodiment of the present application.

[0112] In the above embodiments, the descriptions of each embodiment have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0113] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0114] In the embodiments provided in this application, it should be understood that the disclosed device / equipment and method can be implemented in other ways. For example, the device / equipment embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling, direct coupling, or communication connection shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0115] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in various embodiments of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0117] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the electronic file generation device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0118] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A lensless holographic multispectral reconstruction method, characterized in that Including: Obtaining first original holographic image information of a target sample; Extracting vibration information from the first original holographic image information; Determining second spectral information of the target sample according to the vibration information, first spectral information of a preset first reference sample, and vibration information of a preset reference sample.

2. The method according to claim 1, characterized in that, The vibration information includes amplitude and phase; the extracting of the vibration information from the original holographic image information includes: Determining the phase in the first original holographic image information according to the wavelength in the first original holographic image information and the frequency domain coordinates of each point in the first original holographic image information; Determining the amplitude in the first original holographic image information according to the light intensity distribution in the first original holographic image information and the phase in the first original holographic image information.

3. The method according to claim 1 or 2, characterized in that, When the vibration information includes amplitude, the determining of the second spectral information of the target sample according to the vibration information, first spectral information of a preset first reference sample, and vibration information of a preset second reference sample includes: Determining the correlation relationship between the vibration information and the first spectral information according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample; Determining the second spectral information of the target sample according to the correlation relationship between the vibration information and the first spectral information.

4. The method according to claim 3, wherein The determining of the correlation relationship between the vibration information and the first spectral information according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample includes: Determining an estimation matrix according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample; Determining the correlation relationship between the vibration information and the first spectral information according to the phase of the first original holographic image information and the estimation matrix.

5. The method according to claim 4, wherein The determining of the estimation matrix according to the first spectral information of the preset first reference sample and the vibration information of the preset second reference sample includes: Determining the covariance matrix of the transmittance of the first reference sample according to the first spectral information of the first reference sample; Determining the spectral function of the second reference sample according to the vibration information of the second reference sample; Determining the estimation matrix according to the spectral function of the first reference sample irradiated, the response function of the camera for obtaining the second original holographic image of the second reference sample, and the covariance matrix.

6. The method according to claim 5, wherein The covariance matrix of the transmittance is obtained by averaging the transmittance spectrum of the first reference sample discretized into a preset number of bands.

7. The method according to claim 1, characterized in that, The second reference sample includes a preset number of preset articles; the second reference sample is arranged in the light projection area; The target article is also arranged in the light projection area; The obtaining of the first original holographic image information of the target sample includes: Projecting light of a preset color onto the light projection area to obtain the first original holographic image information of the target sample and the second original holographic image information of the second reference sample.

8. The method according to claim 1, wherein The first original holographic image information includes the original holographic image information under the irradiation of a set wavelength; the first spectral information of the pre-set first reference sample includes the spectral information when the first reference sample is irradiated with white light.

9. An electronic device, characterized in that, Comprising: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, the method described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method described in any one of claims 1-8 is implemented.

Images