Lensless microscopic system of super-resolution reconstruction algorithm based on double-amplitude gradient descent

A lensless microscopy system employing array illumination and scattering multiplexing technology utilizes the relative displacement of the scattering layer and the array light source for secondary diffraction modulation. Combined with a super-resolution reconstruction algorithm, this system solves the problem of limited imaging resolution in traditional lensless microscopy systems, enabling the application of super-resolution imaging and miniaturized diagnostic tools.

CN119024667APending Publication Date: 2024-11-26TSINGHUA SHENZHEN INTERNATIONAL GRADUATE SCHOOL
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411169786.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-25
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The imaging resolution of traditional lensless microscopy systems is limited by the pixel size of the image sensor, making it difficult to observe and analyze subcellular structures.

Method used

A lensless microscopy system based on array light source illumination and scattering multiplexing technology is adopted. By inserting a scattering layer between the sample and the image sensor, secondary diffraction modulation is performed using the relative displacement of the array light source and the scattering layer. Combined with a super-resolution reconstruction algorithm with dual amplitude gradient descent, the amplitude and phase information of the sample under test are reconstructed.

Benefits of technology

It achieves super-resolution imaging, improves imaging resolution, and provides miniaturized, integrated point-of-care diagnostic tools suitable for early diagnosis and timely treatment of diseases in resource-limited areas.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119024667A_ABST
    Figure CN119024667A_ABST
Patent Text Reader

Abstract

A lensless microscopic system is used for carrying out large-view-field and high-resolution microscopic imaging on a sample and comprises an array light source, a sample table, a scattering layer and an image sensor array light source, the array light source is provided with a plurality of sub-light sources at different positions, and the sub-light sources provide incident light at different angles and can be sequentially lightened and illuminate the sample to be detected; the scattering layer is used for carrying out secondary modulation on the diffracted light from the sample to generate a diffracted intensity pattern with multiple diffracted patterns; and the image sensor records the diffracted light secondarily modulated by the scattering layer to form a hologram. Secondary diffraction modulation is carried out on the diffraction light of the sample to be detected through the scattering layer, the diffraction angle of the high-frequency diffraction light is reduced, and more high-frequency information can be collected by the image sensor; the relative displacement modulation effect generated by light emitted by different sub light sources and secondary diffraction modulation are utilized to provide additional data constraint for super-resolution imaging, high-resolution amplitude and phase information of a sample can be reconstructed more accurately, and super-resolution imaging is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of optical imaging, and in particular relates to a lens-free microscope system. Background Art

[0002] Traditional optical microscopes usually cannot meet the imaging requirements of large field of view and high resolution. At the same time, the huge optical path system design makes it difficult to achieve integration and portability, and the expensive lens price also increases the cost of observation. The lensless microscopic imaging system that has been widely popular in recent years has proposed an effective solution to the above problems. It does not require the use of expensive optical lenses, but directly places the sample to be tested on top of the image sensor and illuminates it with a coherent light source, and then uses the image sensor to collect the diffraction intensity map of the sample to be tested. Then, the phase recovery algorithm is used to reconstruct the amplitude and phase distribution of the sample to be tested. Because it has a series of advantages such as large field of view, portability, low cost, phase imaging, etc., it is widely used in large-scale sample statistical analysis such as sample slice observation, blood smear detection, and identification of rare sperm movement trajectories.

[0003] However, the imaging resolution of lensless microscopy systems is limited by the pixel size of image sensors, which restricts the observation and analysis of subcellular structures. Therefore, how to break through the limitation of pixel size and achieve super-resolution microscopy has become an urgent problem to be solved. Summary of the invention

[0004] The main purpose of the present invention is to solve the above-mentioned problems existing in the background technology and to propose a lens-free microscopy system based on array light source illumination and scattering multiplexing technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A lensless microscopic system for performing large-field-of-view, high-resolution microscopic imaging of a sample, comprising an imaging system and a computing device, wherein the imaging system is used to perform large-field-of-view, high-resolution microscopic imaging of the sample; the computing device is used to process image data captured by the imaging system and reconstruct amplitude and phase information of the sample;

[0007] The imaging system includes: an array light source having a plurality of sub-light sources at different positions, each of which provides incident light at a different angle, and can sequentially light up and illuminate the sample to be tested; a sample stage, which is used to carry the sample to be tested and is located below the array light source; a scattering layer, which is located between the sample stage and the image sensor and is used to perform secondary modulation on the diffracted light from the sample to generate a diffraction intensity map with a variety of diffraction patterns; and an image sensor, which is used to record the diffracted light secondary modulated by the scattering layer to form a hologram reflecting the diffraction intensity information of the sample.

[0008] The computing device receives hologram data from the image sensor, and reconstructs amplitude and phase information of the sample to be measured according to the hologram data to achieve super-resolution imaging.

[0009] Furthermore, the array light source LED array or Micro-LED array includes a plurality of light-emitting chips, the size of the light-emitting chips is between 10 μm and 200 mm, and the spacing between the light-emitting chips is between 10 μm and 100 mm.

[0010] Furthermore, the plurality of sub-light sources have a plurality of different wavelengths, and the wavelength range is between 380 μm and 760 μm.

[0011] Furthermore, the scattering layer includes a plurality of modulation units, the characteristic size of the modulation units is between 0.5 μm and 500 μm, and the modulation units are used to modulate the diffracted light of the sample to be measured to generate a second diffraction.

[0012] Furthermore, a scattering medium is coated on the protective glass surface of the image sensor to form the scattering layer, or a scattering sheet is arranged between the sample stage and the image sensor to provide the scattering layer.

[0013] Furthermore, the scattering medium is a polystyrene microsphere coating or a blood cell smear.

[0014] Furthermore, the distance between the light emitting surface of the array light source and the sample stage is between 5 mm and 800 mm.

[0015] Furthermore, the distance between the sample stage and the scattering layer is between 1 μm and 100 mm.

[0016] Furthermore, the distance between the scattering layer and the image sensor is between 1 μm and 100 mm.

[0017] Furthermore, the computing device reconstructs the amplitude and phase information of the sample to be tested by a super-resolution reconstruction algorithm based on dual amplitude gradient descent, and the super-resolution reconstruction algorithm based on dual amplitude gradient descent includes the following processing steps:

[0018] S1, obtaining a diffraction intensity map of the scattering layer and a diffraction intensity map of the sample after being modulated by the scattering layer;

[0019] S2, updating the distribution function of the scattering layer using the diffraction intensity map of the scattering layer;

[0020] S3. Reconstruct the target function and update the amplitude and phase information of the sample to be tested by using the updated distribution function of the scattering layer and the diffraction intensity diagram of the sample after being modulated by the scattering layer.

[0021] An imaging system of a lensless microscopic system, used for performing large-field-of-view, high-resolution microscopic imaging of a sample, comprising:

[0022] An array light source, having multiple sub-light sources at different positions, each of which provides incident light at a different angle, and can sequentially light up and illuminate the sample to be tested;

[0023] A sample stage, used for carrying a sample to be tested and located below the array light source;

[0024] A scattering layer, located between the sample stage and the image sensor, for performing secondary modulation on the diffracted light from the sample to generate a diffraction intensity map having a variety of diffraction patterns;

[0025] The image sensor is used to record the diffracted light secondarily modulated by the scattering layer to form a hologram reflecting the diffraction intensity information of the sample.

[0026] Compared with the prior art, the present invention has the following technical effects:

[0027] The present invention proposes a lensless microscopy system based on array light source illumination and scattering multiplexing technology, which uses an array light source and inserts a scattering layer between the sample platform and the image sensor. The scattering layer performs secondary modulation on the diffracted light from the sample to generate a diffraction intensity map with a variety of diffraction patterns, and then the image sensor records the diffracted light that has been secondary modulated by the scattering layer to form a hologram reflecting the diffraction intensity information of the sample. Since there is a certain height interval between the scattering layer and the sample, the light-emitting chips at different positions can also cause the scattering layer and the sample to be relatively displaced, thereby realizing a variable modulation mode of the scattering layer on the sample without the aid of any precise displacement platform.

[0028] The present invention performs secondary diffraction modulation on the diffracted light of the sample to be tested by adding a scattering layer, reduces the diffraction angle of the high-frequency diffracted light, and enables more high-frequency information to be collected by the image sensor; at the same time, with the illumination of an array light source (such as an LED array light source), the light emitted by the sub-light sources (such as LED light-emitting chips) at different positions will cause a relative displacement between the sample to be tested and the scattering layer, thereby achieving a modulation effect similar to that of a variable mask without using any displacement device, and utilizing the relative displacement modulation effect generated by the light emitted from different sub-light sources, the high-resolution amplitude and phase information of the sample to be tested can be reconstructed more accurately to achieve super-resolution imaging. Preferably, a super-resolution reconstruction algorithm based on dual amplitude gradient descent is adopted, and the secondary diffraction modulation of the diffracted light of the sample to be tested by the scattering layer provides additional data constraints for super-resolution imaging, making the reconstructed image clearer and richer in details. Therefore, the lensless microscopy system based on array light source illumination and scattering multiplexing technology of the present invention is expected to provide a miniaturized, integrated instant diagnostic tool, which can provide more powerful assistance for early diagnosis and timely treatment of diseases in areas with limited resource conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a schematic structural diagram of a lensless microscopy system based on array light source illumination and scattering multiplexing technology according to an embodiment of the present invention;

[0030] Figure 2 This is an image reconstructed from a USFA1951 resolution positive calibration plate by a lensless microscope system based on the traditional multi-wavelength phase retrieval algorithm;

[0031] Figure 3 This is an image reconstructed by a lensless microscope system based on array light source illumination and scattering multiplexing technology of an embodiment of the present invention on a USFA1951 resolution positive calibration plate. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0033] The lensless microscope system based on array light source illumination and scattering multiplexing technology in the embodiment of the present invention is as follows Figure 1 As shown, it is used for super-resolution wide-field microscopic imaging of the sample to be tested, including an imaging system and a computing device. The imaging system includes an array light source 1, a sample stage 2, a scattering layer 3 and an image sensor 4 arranged in sequence. The array light source 1 can be placed on the top of the entire imaging system, and has a plurality of sub-light sources 11 at different positions. Each sub-light source 11 provides incident light at different angles, which can be sequentially lit and illuminated to the sample to be tested. The light emitted by each sub-light source 11 sequentially illuminates the sample to be tested placed on the sample stage 2; the diffracted light of the sample to be tested reaches the scattering layer 3, is modulated by the modulation unit and undergoes a second diffraction; the secondary diffracted light reaches the image sensor 4, forms a hologram and is transmitted to the computing device (not shown). The computing device receives hologram data from the image sensor 4, and reconstructs the amplitude and phase information of the sample to be tested according to the hologram data to achieve super-resolution imaging.

[0034] In another embodiment, the present invention also provides the above imaging system alone.

[0035] In the present invention, a scattering layer is inserted between the sample and the image sensor 4, which can play a secondary modulation role on the diffracted light. Different diffraction patterns can be generated by the relative movement of the light source and the scattering layer, providing more constraint information for the iterative reconstruction process. At the same time, the scattering layer can effectively reduce the incident angle of the diffracted light from the sample, so that the image sensor can collect more high-frequency information.

[0036] In some embodiments, the array light source is an LED array or a Micro-LED array, comprising a plurality of light-emitting chips, the size of the light-emitting chips includes but is not limited to between 10 μm-200 mm, and the spacing between the light-emitting chips includes but is not limited to between 10 μm-100 mm.

[0037] In some embodiments, the LED array or Micro-LED array includes a plurality of light-emitting chips with different wavelengths, and the wavelength range includes but is not limited to 380 μm-760 μm.

[0038] In some embodiments, the scattering layer is placed between the sample to be tested and the image sensor, and can also be formed by directly applying particles on the protective glass surface of the image sensor.

[0039] In some embodiments, the characteristic size of the modulation unit of the scattering layer includes but is not limited to between 0.5 μm and 500 μm.

[0040] In some embodiments, the distance between the light emitting surface of the LED array light source and the sample stage includes but is not limited to between 5 mm and 800 mm.

[0041] In some embodiments, the distance between the sample stage and the scattering layer includes but is not limited to between 1 μm and 100 mm.

[0042] In some embodiments, the distance from the scattering layer to the image sensor includes but is not limited to between 1 μm and 100 mm.

[0043] In some embodiments, the computing device reconstructs the amplitude and phase information of the sample to be tested by a super-resolution reconstruction algorithm based on dual amplitude gradient descent. The super-resolution reconstruction algorithm based on dual amplitude gradient descent includes the following processing steps:

[0044] S1, obtaining a diffraction intensity map of the scattering layer and a diffraction intensity map of the sample after being modulated by the scattering layer;

[0045] S2, updating the distribution function of the scattering layer using the diffraction intensity map of the scattering layer;

[0046] S3. Reconstruct the target function and update the amplitude and phase information of the sample to be tested by using the updated distribution function of the scattering layer and the diffraction intensity diagram of the sample after being modulated by the scattering layer.

[0047] In a preferred embodiment, the super-resolution reconstruction algorithm based on dual amplitude gradient descent includes the following processing steps:

[0048] S1. Obtain the diffraction intensity map y of the scattering layer 1 And the diffraction intensity diagram y of the sample after being modulated by the scattering layer 2 ;

[0049] S2, using the diffraction intensity map y of the scattering layer 1 Update the distribution function P of the scattering layer:

[0050]

[0051]

[0052] represents the amplitude-based error function, D represents the downsampling process, and D * represents the upsampling process, B 1 is the first reverse diffraction process, t is the number of iterations, τ t is the modified step size of the tth iteration, and the initial value can usually be set to 2. Preferably, in order to ensure the convergence speed and prevent the step size from being too large and causing the optimal solution to be missed, τ t is set to be an adaptive adjustment step size related to the reconstruction error;

[0053] S3, using the updated distribution function P of the scattering layer and the diffraction intensity graph y of the sample after being modulated by the scattering layer 2 To reconstruct the objective function and update the complex amplitude information O of the sample to be tested:

[0054]

[0055]

[0056] B 2 It is the second reverse diffraction process, and α is an adjustment factor. In one embodiment, the adjustment factor is set to 0.25.

[0057] The secondary diffraction modulation of the diffracted light of the sample to be tested by the scattering layer provides more constraint support for reconstructing the sample to be tested based on the super-resolution reconstruction algorithm, making the reconstructed image clearer and richer in details.

[0058] The function of the LED array light source is to provide incident light at different angles through light-emitting chips at different positions. At the same time, the light-emitting chips at different positions can contain illumination light of multiple wavelengths. For example, in this embodiment, illumination light of three wavelengths of red, green and blue is used, and its typical wavelength is red light λ R =635nm, green light λ G =525nm and blue light λ B =475nm. The size L of the light-emitting surface of each light-emitting chip on the LED array light source and the distance Δx between the light-emitting surfaces can be learned from the manufacturer's manual.

[0059] The function of the sample stage 2 is to carry the sample, and the distance between the sample stage 2 and the LED array module is generally, but not limited to, between 10 mm and 500 mm. It is best that the lateral position of the sample stage 2 is adjustable so that the microscope can observe different areas of the sample.

[0060] The scattering layer 3 performs secondary modulation on the diffracted light from the sample, reducing the divergence angle of the large-angle diffracted light, so that more high-frequency information can be collected by the image sensor, thereby improving the imaging resolution of the system. In order not to increase the diffraction distance from the sample to the image sensor, in one embodiment, polystyrene microspheres are directly applied to the protective glass surface of the image sensor to prepare the scattering layer. In other embodiments, the scattering layer can also be obtained by other preparation methods such as adding an external scattering sheet and applying a blood cell smear on the protective glass surface.

[0061] The function of the image sensor 4 is to record the hologram generated after the diffracted light of the sample is modulated by the scattering layer. The sample stage is generally close to the image sensor so that the diffraction distance between them is much smaller than the illumination distance to ensure good spatial coherence. However, there is a scattering layer above the image sensor, so the distance from the sample stage to the image sensor generally includes but is not limited to between 0.5mm and 2mm. The hologram obtained by the image sensor is transmitted to the computer, and then the amplitude and phase information of the sample to be tested are reconstructed by a super-resolution reconstruction algorithm based on double amplitude gradient descent to obtain clear, high-resolution wide-field microscopic imaging.

[0062] In order to verify the imaging effect of the lensless microscope system based on array light source illumination and scattering multiplexing technology of the present invention, an imaging test was conducted on a USFA1951 resolution positive calibration plate. Figure 2 and Figure 3 As shown, Figure 2 This is an image reconstructed from the USFA1951 resolution positive calibration plate using a lensless microscope system without adding a scattering layer, based on a traditional multi-wavelength phase recovery algorithm. Figure 3 This is an image reconstructed from the USFA1951 resolution positive calibration plate by a lensless microscope system based on array light source illumination and scattering multiplexing technology with a scattering layer added. It can be seen that due to the limitation of the image sensor pixel size, the traditional reconstruction method cannot achieve an imaging resolution higher than the sensor pixel size, and can only observe the first element in the 8th group, and the corresponding half-pitch resolution is 1.95 microns. The lensless microscope system based on array light source illumination and scattering multiplexing technology can observe the 5th element in the 8th group, and the corresponding half-pitch resolution is 1.23 microns. This result shows that the present invention can improve the imaging resolution of the lensless microscope imaging system and achieve clear, high-resolution wide-field microscope imaging while ensuring a low-cost and compact optical path structure.

[0063] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A lensless microscopy system, characterized in that: The invention comprises an imaging system and a computing device, wherein the imaging system is used to perform large-field-of-view, high-resolution microscopic imaging of a sample; and the computing device is used to process image data captured by the imaging system and reconstruct amplitude and phase information of the sample; Wherein, the imaging system comprises: An array light source, having multiple sub-light sources at different positions, each of which provides incident light at a different angle, and can sequentially light up and illuminate the sample to be tested; A sample stage, used for carrying a sample to be tested and located below the array light source; A scattering layer, located between the sample stage and the image sensor, for performing secondary modulation on the diffracted light from the sample to generate a diffraction intensity map having a variety of diffraction patterns; An image sensor is used to record the diffracted light secondarily modulated by the scattering layer to form a hologram reflecting the diffraction intensity information of the sample; The computing device receives hologram data from the image sensor, and reconstructs amplitude and phase information of the sample to be measured according to the hologram data to achieve super-resolution imaging.

2. The lensless microscopy system according to claim 1, wherein: The array light source LED array or Micro-LED array includes a plurality of light-emitting chips, the size of the light-emitting chips is between 10 μm and 200 mm, and the spacing between the light-emitting chips is between 10 μm and 100 mm.

3. The lensless microscopy system according to claim 1 or 2, characterized in that: The plurality of sub-light sources have a plurality of different wavelengths, and the wavelength range is between 380 μm and 760 μm.

4. The lensless microscopy system according to any one of claims 1 to 3, characterized in that: The scattering layer includes a plurality of modulation units, the characteristic size of the modulation units is between 0.5 μm and 500 μm, and the modulation units are used to modulate the diffracted light of the sample to be measured to generate a second diffraction.

5. The lensless microscopy system according to claim 4, wherein: The scattering layer is formed by coating a scattering medium on the protective glass surface of the image sensor, or by providing a scattering sheet between the sample stage and the image sensor.

6. The lensless microscopy system according to claim 5, wherein: The scattering medium is a polystyrene microsphere coating or a blood cell smear.

7. The lensless microscopy system according to any one of claims 1 to 6, characterized in that: The distance between the light emitting surface of the array light source and the sample stage is between 5mm and 800mm.

8. The lensless microscopy system according to any one of claims 1 to 7, characterized in that: The distance between the sample stage and the scattering layer is between 1 μm and 100 mm; the distance between the scattering layer and the image sensor is between 1 μm and 100 mm.

9. The lensless microscopy system according to any one of claims 1 to 8, characterized in that: The amplitude and phase information of the sample to be tested is reconstructed by a super-resolution reconstruction algorithm based on dual amplitude gradient descent, and the super-resolution reconstruction algorithm based on dual amplitude gradient descent includes the following processing steps: S1, obtaining a diffraction intensity map of the scattering layer and a diffraction intensity map of the sample after being modulated by the scattering layer; S2, updating the distribution function of the scattering layer using the diffraction intensity map of the scattering layer; S3. Reconstruct the target function and update the amplitude and phase information of the sample to be tested by using the updated distribution function of the scattering layer and the diffraction intensity diagram of the sample after being modulated by the scattering layer.

10. An imaging system of a lensless microscopic system, used for performing large-field, high-resolution microscopic imaging of a sample, characterized in that: include: An array light source, having multiple sub-light sources at different positions, each of which provides incident light at a different angle, and can sequentially light up and illuminate the sample to be tested; A sample stage, used for carrying a sample to be tested and located below the array light source; A scattering layer, located between the sample stage and the image sensor, for performing secondary modulation on the diffracted light from the sample to generate a diffraction intensity map having a variety of diffraction patterns; The image sensor is used to record the diffracted light secondarily modulated by the scattering layer to form a hologram reflecting the diffraction intensity information of the sample.

Citation Information

Cited By

  • Micro-LED array measurement method and system based on phase reconstruction

    CN120194914A