A real-time three-dimensional microscopic imaging device and method
Through the spatial coding and multi-focus lens technology of real-time three-dimensional microscopy imaging device, combined with computer reconstruction, the problem that traditional three-dimensional scanning imaging methods cannot record three-dimensional structures in real time is solved, real-time monitoring of tiny objects and high-precision three-dimensional image presentation is realized , suitable for real-time detection in the fields of biology and medicine.
Patent Information
- Application Number
- CN202011156312.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-10-26
AI Technical Summary
Traditional three-dimensional scanning imaging methods have a long scanning cycle and cannot record and present the three-dimensional structure of the target object in real time, especially in the fields of medical and biology, real-time measurement of living cells cannot be met.
A real-time three-dimensional microscopic imaging device is designed, including a coding plate, a multi-focus lens, an optical microscopic imaging system and an imaging plane receiving mechanism. Through spatial encoding and multi-focus lens imaging, combined with computer three-dimensional reconstruction technology, the microscopic objects are realized. real-time recording and presentation.
Real-time three-dimensional detection of tiny transparent or translucent objects is achieved, with high sensitivity, low computational pressure and high accuracy, which can truly reflect the three-dimensional changes of cells and is suitable for real-time monitoring in the fields of biology and medicine.
Smart Images

Figure CN112509111B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to three-dimensional microscopic imaging technology, and in particular to a real-time three-dimensional microscopic imaging device and method, targeting transparent and semi-transparent objects, especially cells and cell clusters. Background Art
[0002] Three-dimensional microscopic imaging detection plays an important role in medicine, biology, and micro-processing. At present, the traditional three-dimensional scanning imaging method generally adopts layer-by-layer scanning imaging, which images the target object (cells, tissues or other tiny samples) at different depths, and then reconstructs the three-dimensional structure of the target object using a computer. For example, the three-dimensional imaging mode of the laser confocal microscope obtains microscopic photos of different depths through high-precision Z-axis movement (the Z-axis direction is the depth direction of the target object), and then forms a three-dimensional structural image of the target object through computer synthesis technology.
[0003] Traditional 3D scanning imaging methods have a long scanning cycle due to the scanning process, and cannot record and present the 3D structure of the target object in real time. In medicine and biology, they cannot meet the measurement of tiny objects that change over time, such as living cells. Summary of the invention
[0004] In order to solve the technical problems of existing three-dimensional scanning and imaging methods, such as long scanning cycles and inability to record and present the three-dimensional structure of target objects in real time, the present invention provides a real-time three-dimensional microscopic imaging device and method, which can record the three-dimensional structure of tiny objects in real time, and realize real-time recording of tiny objects, such as cell structures, through solution and computer three-dimensional reconstruction technology.
[0005] To achieve the above purpose, the technical solution provided by the present invention is:
[0006] A real-time three-dimensional microscopic imaging device, which is special in that it includes a coding plate, a multi-focus lens, an optical microscopic imaging system and an imaging surface receiving mechanism arranged in sequence along an optical path, and a computer connected to the imaging surface receiving mechanism;
[0007] The coding plate is used to spatially encode the three-dimensional information of the object to be measured;
[0008] The multi-focal lens includes N lens areas, where N is an integer greater than or equal to 2, the optical axes of the N lens areas are parallel, and the focal lengths of the N lens areas are different; the N lens areas of the multi-focal lens are respectively used to image the encoded three-dimensional information according to different depths of field to obtain two-dimensional images of the object to be measured at different depths;
[0009] The optical microscopic imaging system is used to optimize two-dimensional images at different depths, and the optimization includes achromatic processing and aberration elimination processing;
[0010] The imaging surface receiving mechanism is used to receive the two-dimensional image optimized by the optical microscopic imaging system, and different depths are imaged at different positions of the imaging surface receiving mechanism to obtain overlapping images of different depths of the object to be measured;
[0011] The computer is used to demodulate and reconstruct the overlapping images acquired by the imaging surface receiving mechanism in real time to obtain a reconstructed three-dimensional image.
[0012] Furthermore, the imaging surface receiving mechanism is a CCD or a CMOS or a fluorescent screen.
[0013] Furthermore, the encoding plate is a spatial encoder composed of light-transmitting areas and light-opaque areas overlapping each other, and its encoding unit is square.
[0014] Furthermore, the multifocal lens is formed by splicing and gluing N lens areas.
[0015] At the same time, the present invention provides a real-time three-dimensional microscopic imaging method based on the above-mentioned real-time three-dimensional microscopic imaging device, which is special in that it includes the following steps:
[0016] Step 1: Place the object to be measured in front of the encoder plate;
[0017] Step 2: The encoding plate encodes the light signal emitted by the object to be measured;
[0018] Step 3: N lens areas of the multi-focal lens image the encoded information according to different depths of field;
[0019] Step 4: The optical microscopic imaging system optimizes the imaging of the multi-focal lens to form clear two-dimensional images at different positions on the imaging surface receiving mechanism;
[0020] Step 5: The computer obtains a three-dimensional image by demodulating and reconstructing all the two-dimensional images obtained by the imaging surface receiving mechanism.
[0021] Furthermore, in step 2, the light signal emitted by the object to be measured is represented by different depths d n The two-dimensional image set S, S[S(x,y,d1,t),S(x,y,d2,t),…S(x,y,d n ,t)], n=1,2,…; t is the recording point time.
[0022] Further, in step 3, the overlapping graph of all two-dimensional images acquired by the imaging surface receiving mechanism is represented as T(x, y);
[0023]
[0024] In the formula, S k is the depth of the object to be measured dk The original information two-dimensional image, k = 1, 2, ... n; l k is the depth d k The offset of the formed two-dimensional image on the imaging surface receiving mechanism; A is the mathematical expression of the two-dimensional matrix of the encoding plate.
[0025] Compared with the prior art, the advantages of the present invention are:
[0026] 1. The multi-focal lens of the imaging device of the present invention comprises a plurality of lens regions. By designing the focal lengths and principal axes of different lens regions of the multi-focal lens, imaging of different depths of the object to be measured can be realized. The encoding plate and the multi-focal lens are used to record different depths d in real time on the imaging surface receiving mechanism. n By decompressing the image, we can get overlapping images of different depths d n The original image is taken and the target object is reconstructed using computer 3D reconstruction technology, realizing real-time 3D detection of tiny transparent or translucent objects. It has the advantages of real-time and continuity, can truly reflect the 3D changes of cells, and provides necessary detection means for scientific research in the biological and medical fields.
[0027] 2. The imaging device of the present invention has the advantages of high detection sensitivity, low calculation pressure, and low reconstruction difficulty under multi-depth detection conditions.
[0028] 3. The imaging method of the present invention adjusts the focal length of the microscope and the depth of field through a multi-focal lens, and can simultaneously obtain multiple images of different depths, and simultaneously optimizes and compresses the multiple images through an optical microscopic imaging system. The imaging surface receiving mechanism obtains the optimized multiple images of different depths, and restores them into multiple images through computer processing, thereby realizing real-time three-dimensional image detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a schematic diagram of the principle of the real-time three-dimensional microscopic imaging device of the present invention;
[0030] Figure 2 It is a schematic diagram of the structure of the encoding plate in the real-time three-dimensional microscopic imaging device of the present invention;
[0031] Figure 3a It is a schematic diagram of the structure of a multi-focal lens in the real-time three-dimensional microscopic imaging device of the present invention;
[0032] Figure 3b yes Figure 3a Left view of;
[0033] Figure 4a Schematic diagram of the structure of the real-time three-dimensional microscopic imaging device of the present invention (optical microscopic imaging system is not shown);
[0034] Figure 4b, Figure 4c , Figure 4d They are Figure 4a Schematic diagram of the cross section of the object to be measured at depth d3, depth d2, and depth d1;
[0035] Figure 5 It is a schematic diagram of image demodulation formed by the imaging surface receiving mechanism in the real-time three-dimensional microscopic imaging device of the present invention; wherein a is the overlapping compressed image formed by three lens areas, b is the two-dimensional image formed by the lens area with a focal length of f1 at the depth d1 of the object to be measured at the imaging surface receiving mechanism, c is the two-dimensional image formed by the lens area with a focal length of f2 at the depth d2 of the object to be measured at the imaging surface receiving mechanism, and d is the two-dimensional image formed by the lens area with a focal length of f3 at the depth d3 of the object to be measured at the imaging surface receiving mechanism.
[0036] The reference numerals are as follows:
[0037] 1-object to be measured, 2-encoding plate, 3-multi-focus lens, 4-optical microscopic imaging system, 5-imaging surface receiving mechanism, 6-computer. DETAILED DESCRIPTION
[0038] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0039] like Figure 1 As shown, a real-time three-dimensional microscopic imaging device is composed of a computer 6 and a coding plate 2, a multi-focal lens 3, an optical microscopic imaging system 4, and an imaging surface receiving mechanism 5 which are arranged in sequence; the core process is that the coding plate 2 spatially encodes the three-dimensional information of the object to be measured 1, and uses the multi-focal lens 3 to perform two-dimensional imaging at different positions of the imaging surface receiving mechanism 5 according to the depth of field difference, and then the image is demodulated and reconstructed by the demodulation algorithm of the computer 6 to form a real-time three-dimensional image. The imaging device of the present invention can be used for three-dimensional real-time recording and perception of microscopic objects, and realizes three-dimensional real-time detection of tiny objects, such as transparent and semi-transparent objects such as cells.
[0040] After the optical information of the object to be measured 1 (three-dimensional object) passes through the encoding plate 2, the encoding of the spatial three-dimensional information is realized. After passing through the multi-lens system (the multi-lens system includes a multi-focal lens 3, an optical microscopic imaging system 4, and an imaging surface receiving mechanism 5), two-dimensional compressed images of the object to be measured 1 at different depths, which are encoded and overlapped with each other, are obtained at the imaging surface of the imaging surface receiving mechanism 5.
[0041] The main components of the three-dimensional microscopic imaging device of this embodiment are as follows:
[0042] 1. Encoding board 2
[0043] The coding plate 2 is a spatial encoder composed of overlapping light-transmitting areas and opaque areas. Figure 2 As shown, the black part is the opaque area, the white part is the translucent area, and each square unit is called a coding unit. In other embodiments, the coding unit may also be other graphics, such as a hexagon. The translucent area is defined as 1, and the opaque area is defined as 0, then the three-dimensional imaging information of the object to be measured 1 is modulated by a two-dimensional array composed of 1 and 0, and a modulated image is formed on the imaging surface receiving mechanism 5 behind. Since the coding plate 2 will block part of the information of the object to be measured 1, the resolution of the final three-dimensional real-time image is affected by the unit size of the coding plate 2, and the theoretical maximum resolution can reach 2 times the unit size of the coding plate 2.
[0044] 2. Multifocal lens 3
[0045] The multifocal lens 3 is a lens composed of parts with different focal lengths, that is, the multifocal lens 3 includes multiple lens areas. The multifocal lens 3 has the following characteristics:
[0046] 1) Different focal lengths
[0047] Different lens areas have different focal lengths. When a beam of parallel light enters the multi-focal lens 3, multiple focal points may appear (each lens area corresponds to a focal point, and multiple lens areas correspond to multiple focal points). Through design, the focal lengths of the multiple focal points can be different, namely f1, f2, f3, ... From the imaging theorem, it can be known that the shape of the lens does not affect the imaging of the lens. Therefore, the use of the multi-focal lens 3 to image the object 1 to be measured will not change the morphology of the object 1 to be measured.
[0048] 2) Different focal positions
[0049] The lateral positions of the focal points of different lens areas are different, that is, the optical axes of different lens areas are parallel and do not overlap. Through the design of the multi-focal lens 3, it is possible to achieve different positions of the principal axes of the lens areas, that is, the focal positions of different lens areas are different in the direction parallel to the optical axis of any lens area. It can be seen from the imaging theorem that under the premise of the same distance, the difference in focal length will lead to a change in object distance, so the multi-focal lens 3 can realize the detection of different object distances (that is, different depths of field of the object to be measured 1). At the same time, due to the different positions of the principal axes (optical axes) of the lens areas, the images of different depths of field have a certain deviation on the imaging surface.
[0050] The multifocal lens 3 can be realized by manufacturing it on a lens through the method of microlens processing technology, or by manufacturing multiple lenses with different focal points separately and cutting the multiple lenses. The cutting principle is to ensure that the multiple lenses after cutting can be assembled into a complete lens. The multiple lenses after cutting are glued together to complete the production of the multifocal lens 3.
[0051] 3. Optical microscopy imaging system 4
[0052] The optical microscopic imaging system 4 plays an optical shaping role in the imaging device of this embodiment, and is used to perform optical shaping effects such as achromatization and aberration elimination on the image obtained by the multifocal lens 3, so that the image processed by the optical microscopic imaging system 4 can form a clear overlapping compressed image on the imaging surface receiving mechanism 5.
[0053] 4. Imaging surface receiving mechanism 5
[0054] The imaging surface receiving mechanism 5 plays an image acquisition role in the imaging device of this embodiment, and is used to collect the image processed by the optical microscope imaging system 4. The imaging surface receiving mechanism 5 of this embodiment can be implemented by CCD, CMOS or fluorescent screen, etc., and mainly completes the collection and transmission function of overlapping compressed images.
[0055] 5. Computer 6
[0056] The computer 6 plays the role of data processing, image demodulation, and graphic reconstruction in the imaging device of this embodiment. It mainly demodulates, three-dimensionally reconstructs, and displays 3D images of the collected compressed images through pre-designed algorithms (specific algorithms are used for demodulation for different encoding plates 2 to restore multiple images from a compressed image), so as to restore the three-dimensional information of the object to be measured 1 and obtain a three-dimensional image.
[0057] like Figure 4a and Figure 5 As shown, based on the above-mentioned real-time three-dimensional microscopic imaging device, this embodiment provides a real-time three-dimensional microscopic imaging method, which compresses and encodes the three-dimensional image through image compression sensing technology and reads it through a two-dimensional image sensing device, and realizes real-time three-dimensional image detection through a computer reconstruction method. The method is used to obtain the three-dimensional structure of the target object in real time, and the imaging method includes the following steps:
[0058] Step 1: The object to be measured 1 emits or projects illumination light, which can be regarded as a set of two-dimensional images at different depths;
[0059] The optical signal of the object 1 to be measured at any time t can be expressed as S, S[S(x,y,d1,t),S(x,y,d2,t),…
[0060] S(x,y,d n ,t)], where n = 1, 2, ...; t is the time of recording point, i.e., for different depths d n To achieve real-time three-dimensional reconstruction of the object 1 to be measured, it is necessary to record the optical signals of the object 1 to be measured at different depths at any time t; x, y represent the plane coordinates perpendicular to the depth direction;
[0061] Step 2: Place the object 1 to be tested in front of the encoding plate 2; use mask technology to encode the optical signal emitted by the object 1 through the encoding plate 2;
[0062] Step 3: The encoded signal light passes through a multi-intersection lens so that images of the object 1 at different depths are presented at different positions on the imaging surface receiving mechanism 5;
[0063] During the imaging process, the optical microscopic imaging system 4 optimizes the encoded image so that a clear compressed image is formed on the imaging surface receiving mechanism 5;
[0064] Step 4: After passing through the multi-focal lens 3, the two-dimensional images of the object 1 at different depths form real images with overlapping areas on the imaging surface receiving mechanism 5, i.e., an integration effect;
[0065] Step 5: Record the compressed image recorded by the imaging surface receiving mechanism 5 to record the two-dimensional image information at different depths in the three-dimensional image at the time t;
[0066] What is recorded on the imaging surface receiving mechanism 5 is the encoded overlapping graph T(x, y, t) of two-dimensional images of different depths. Due to the optical axis deviation of the multi-focal lens 3, each image will generate l on the imaging surface receiving mechanism 5 in turn. k The overall offset of
[0067]
[0068] in, k is a certain depth d of the object 1 to be measured k The original information two-dimensional image, k = 1, 2, ... n; A is the mathematical expression of the two-dimensional matrix of the encoding plate 2;
[0069] Step 6: Using image compression and demodulation technology, all two-dimensional compressed images recorded by the imaging surface receiving mechanism 5 are subjected to data processing, image demodulation, and graphic reconstruction, and the existing compressed sensing demodulation method is used to obtain a three-dimensional stereoscopic image of the object to be measured 1.
[0070] The information of a three-dimensional object 1 to be measured can be described as a collection of two-dimensional information at different depths n, where n is infinite. That is, the three-dimensional information of the object 1 to be measured is formed by superimposing two-dimensional information at different depths. In the imaging device of the present invention, N different depths are selected for the object 1 to be measured (the number of depths of the object 1 to be measured is equal to the number of lens areas of the multifocal lens 3). N is a limited number, and an approximate three-dimensional image is obtained by the imaging method of this embodiment. The larger N is, the higher the accuracy is. In this embodiment, the structure of the multifocal lens 3 is as follows: Figure 3a and Figure 3b As shown, the multi-focal lens 3 includes three lens areas, and the focal lengths of the three lens areas are f1, f2, and f3 respectively. Then, three different depths are selected for the object to be measured 1, such as Figure 4a , 4b 4b, 4c, which are depths d1, d2, and d3, respectively. The three lens areas of the multifocal lens 3 respectively image the depths d1, d2, and d3 of the object 1 to be measured at different positions of the imaging surface receiving mechanism 5. The imaging surface receiving mechanism 5 obtains an overlapping diagram of the three depth images, as shown in FIG. Figure 5 As shown in a, Figure 5 b in the figure is the two-dimensional image of the depth d1 of the object 1 to be measured formed by the lens area with a focal length of f1 on the imaging surface receiving mechanism 5, as shown in FIG. Figure 5 Where c is the two-dimensional image of the depth d2 of the object 1 to be measured formed by the lens area with a focal length of f2 on the imaging surface receiving mechanism 5, as shown in Figure 5 d in the figure is the two-dimensional image of the depth d3 of the object 1 to be measured formed by the lens area with a focal length of f3 on the imaging surface receiving mechanism 5; the target images at three different depths are obtained by solving, and then the images are reconstructed into three-dimensional stereo images using the reconstruction technology of computer 6. The depth detection accuracy of the three-dimensional image is Δd, and the horizontal detection accuracy is affected by the coding pattern of the coding plate 2. The theoretical upper limit is one times the unit size a of the coding plate 2, that is, 2a.
[0071] The above is only a description of the preferred implementation mode of the present invention, and the technical solution of the present invention is not limited thereto. Any known deformation made by those skilled in the art on the basis of the main technical concept of the present invention belongs to the technical scope to be protected by the present invention.
Claims
1. A real-time three-dimensional microscopic imaging device, characterized in that: It comprises a coding plate (2), a multi-focus lens (3), an optical microscopic imaging system (4), and an imaging surface receiving mechanism (5) which are sequentially arranged along an optical path, and a computer (6) connected to the imaging surface receiving mechanism (5); The encoding plate (2) is used to spatially encode the three-dimensional information of the object to be measured (1); The multi-focal lens (3) comprises N lens areas, where N is an integer greater than or equal to 2, the optical axes of the N lens areas are parallel, and the focal lengths of the N lens areas are different; the N lens areas of the multi-focal lens (3) are used to image the encoded three-dimensional information according to different depths of field, so as to obtain two-dimensional images of the object to be measured (1) at different depths; The optical microscopic imaging system (4) is used to optimize two-dimensional images at different depths, wherein the optimization includes achromatic processing and aberration elimination processing; The imaging surface receiving mechanism (5) is used to receive the two-dimensional image optimized by the optical microscopic imaging system (4), and different depths are imaged at different positions of the imaging surface receiving mechanism (5) to obtain overlapping images of different depths of the object to be measured (1); The computer (6) is used to demodulate and reconstruct the overlapping images acquired by the imaging surface receiving mechanism (5) in real time to obtain a reconstructed three-dimensional image.
2. The real-time three-dimensional microscopic imaging device according to claim 1, characterized in that: The imaging surface receiving mechanism (5) is a CCD or a CMOS or a fluorescent screen.
3. The real-time three-dimensional microscopic imaging device according to claim 2, characterized in that: The encoding plate (2) is a spatial encoder composed of light-transmitting areas and light-impermeable areas overlapping each other, and its encoding unit is square.
4. The real-time three-dimensional microscopic imaging device according to any one of claims 1 to 3, characterized in that: The multi-focal lens (3) is formed by splicing and gluing N lens areas.
5. A real-time three-dimensional microscopic imaging method based on the real-time three-dimensional microscopic imaging device of claim 1, characterized in that: The following steps are involved: Step 1: placing the object to be measured (1) in front of the encoding plate (2); Step 2: The encoding plate (2) encodes the light signal emitted by the object to be detected (1); Step 3: The N lens areas of the multi-focal lens (3) image the encoded information according to different depths of field; Step 4: The optical microscopic imaging system (4) optimizes the imaging of the multi-focal lens (3) to form clear two-dimensional images at different positions on the imaging surface receiving mechanism (5); Step 5: The computer (6) obtains a three-dimensional image by demodulating and reconstructing all the two-dimensional images acquired by the imaging surface receiving mechanism (5).
6. The real-time three-dimensional microscopic imaging method according to claim 5, characterized in that: In step 2, the light signal emitted by the object to be measured (1) is represented by different depths d n The two-dimensional image set S, S = [S (x, y, d1, t), S (x, y, d2, t), ... S (x, y, d n ,t)], n=1,2,…; t is the recording point time.
7. The real-time three-dimensional microscopic imaging method of the real-time three-dimensional microscopic imaging device according to claim 6, characterized in that: In step 3, the overlapping graph of all two-dimensional images acquired by the imaging surface receiving mechanism (5) is represented as T(x, y); In the formula, S k is the depth d of the object to be measured (1) k The original information two-dimensional image, k = 1, 2, ... n; l k is the depth d k The resulting two-dimensional image is offset on the imaging surface receiving mechanism (5); A is the mathematical expression of the two-dimensional matrix of the encoding plate (2).
