A holographic tomography three-dimensional microscope

Abstract

The present application discloses a holographic tomographic three-dimensional microscope, including: a holographic optical path and an FPGA module; the holographic optical path includes a light source and an optical sensor, the light source is used to generate a light beam, a part of the light beam passes through the observed object to obtain the object light, and the other part of the light beam is used as the reference light, the object light and the reference light are superimposed to interfere, generate interference fringes, and the optical sensor records the interference fringes to obtain a holographic image; the FPGA module includes a holographic reconstruction unit and a tomographic three-dimensional reconstruction unit; the holographic reconstruction unit uses a diffraction propagation algorithm to reconstruct the holographic image into an amplitude map and a phase map; the tomographic three-dimensional reconstruction unit uses a tomographic reconstruction algorithm to perform tomographic three-dimensional reconstruction according to the amplitude map and the phase map to obtain a three-dimensional image. Compared with the prior art of using a computer for three-dimensional reconstruction, the present application uses FPGA to realize three-dimensional reconstruction based on holographic images, which solves the problems of slow reconstruction speed, large volume, and high power consumption of holographic microscopes in the related art.

Description

Technical Field

[0001] The present application belongs to the technical field of holographic microscopic imaging, and more specifically, relates to a holographic tomographic three-dimensional microscope. Background Art

[0002] Microscopes are common tools in fields such as life sciences and have a wide range of applications, such as live cell imaging. Currently, most microscopes are laser confocal 3D microscopes, which require point-by-point and surface-by-surface scanning, which is time-consuming, labor-intensive, and inefficient. In addition, when performing live cell imaging, cells usually need to be stained and marked before observation, which is not friendly to live cell imaging.

[0003] Digital holographic microscopes avoid the shortcomings of laser confocal microscopes, such as point-by-point scanning and the need for cell staining and labeling, and improve imaging efficiency. When observing living cells, it is not necessary to stain and label cells. However, holographic microscopes need to be reconstructed by computers, and the reconstruction speed is limited by the computer. In addition, holographic microscopes are generally equipped with a large host, which results in a large size.

[0004] As can be seen from the above, the related art has the problems of slow reconstruction speed and large size of the holographic microscope. Summary of the invention

[0005] In view of the defects of the related art, the present application provides a holographic tomography three-dimensional microscope, which aims to solve the problems of slow reconstruction speed and large size of the holographic microscope in the related art.

[0006] The technical solution is as follows:

[0007] According to one aspect of the present application, a holographic tomographic three-dimensional microscope includes: a holographic optical path and an FPGA module; the holographic optical path includes a light source and an optical sensor, the light source is used to generate a light beam, a part of the light beam passes through an observed object to obtain object light, and the other part of the light beam is used as a reference light, the object light and the reference light are superimposed and interfere with each other to generate interference fringes, and the optical sensor records the interference fringes to obtain a holographic image of the observed object; the FPGA module includes a holographic reconstruction unit and a tomographic three-dimensional reconstruction unit; the holographic reconstruction unit and the tomographic three-dimensional reconstruction unit are both implemented based on a hardware description language; the holographic reconstruction unit uses a diffraction propagation algorithm to reconstruct the holographic image into an amplitude map and a phase map; the tomographic three-dimensional reconstruction unit uses a tomographic reconstruction algorithm to perform tomographic three-dimensional reconstruction on the observed object according to the amplitude map and the phase map to obtain a three-dimensional image of the observed object.

[0008] Optionally, the holographic reconstruction unit includes a transformation subunit; the transformation subunit is used to perform Fourier forward transform on the holographic image to obtain the holographic image that has completed Fourier forward transform; and perform inverse Fourier transform on the holographic image that has completed Fourier forward transform to obtain the amplitude map of each projection angle.

[0009] Optionally, the reconstruction subunit includes a data indexing unit, a filtering unit and an interpolation back-projection unit; the indexing module is used to divide the observed object into a number of sections to obtain the section intensity data corresponding to each section; the filtering unit is used to filter each section intensity data to remove noise; the interpolation back-projection unit is used to filter and back-project each section intensity data to obtain a slice image corresponding to each section, and obtain the three-dimensional image of the observed object based on the slice image corresponding to each section.

[0010] Optionally, the diffraction propagation algorithm is an angular spectrum method.

[0011] Optionally, the tomographic reconstruction algorithm is a two-dimensional filtered back-projection algorithm.

[0012] Optionally, the holographic optical path includes a spatial light modulation device, and the spatial light modulation device is used to reflect the laser light beam to achieve high-speed modal change.

[0013] Optionally, the FPGA module includes a modulation device control unit, and the modulation device control unit is used to control the spatial light modulation device.

[0014] Optionally, the spatial light modulation device is a digital micromirror device.

[0015] Optionally, the holographic optical path includes a reflector for adjusting the direction of the laser beam.

[0016] Optionally, the holographic optical path comprises a beam splitter for forming coherent light.

[0017] According to one aspect of the present application, a three-dimensional image generation method based on the holographic tomography three-dimensional microscope as described above is applied to the FPGA module, and the method includes: using a diffraction propagation algorithm to reconstruct the holographic image of the observed object into an amplitude map and a phase map; based on the amplitude map and the phase map, using a tomographic reconstruction algorithm to perform tomographic three-dimensional reconstruction on the observed object to obtain the three-dimensional image of the observed object.

[0018] According to one aspect of the present application, a three-dimensional image generating device based on the holographic tomographic three-dimensional microscope as described above is applied to the FPGA module, and the device includes: an amplitude-phase reconstruction module, which is used to reconstruct the holographic image of the observed object into an amplitude map and a phase map using a diffraction propagation algorithm; and a tomographic three-dimensional reconstruction module, which is used to perform tomographic three-dimensional reconstruction on the observed object according to the amplitude map and the phase map using a tomographic reconstruction algorithm to obtain the three-dimensional image of the observed object.

[0019] According to one aspect of the present application, an electronic device includes: at least one processor, at least one memory, and at least one communication bus, wherein a computer program is stored in the memory, and the processor reads the computer program in the memory through the communication bus; when the computer program is executed by the processor, the three-dimensional image generation method as described above is implemented.

[0020] According to one aspect of the present application, a storage medium stores a computer program, and when the computer program is executed by a processor, the three-dimensional image generation method described above is implemented.

[0021] According to one aspect of the present application, a computer program product includes a computer program, the computer program is stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device implements the three-dimensional image generation method as described above when executing the computer program.

[0022] This application has the following beneficial effects:

[0023] The holographic optical path generates and records the holographic image of the observed object. The holographic image is transmitted to the FPGA through the FPGA high-speed interface. Then, the FPGA is used to perform holographic tomography reconstruction based on the holographic image to obtain a three-dimensional image of the observed object. The calculation involved in the field of holographic tomographic three-dimensional reconstruction is essentially image processing, and tomographic reconstruction will perform the same operation on different two-dimensional sections. FPGA has a wide range of applications in the field of image processing and has parallel pipeline processing capabilities. Using FPGA can greatly accelerate tomographic three-dimensional reconstruction. In addition, FPGA has the advantage of being small in size compared to computers.

[0024] In summary, compared with the prior art that uses computers to achieve three-dimensional reconstruction based on holographic images, the present application uses FPGA to achieve three-dimensional reconstruction based on holographic images, which solves the problems of slow reconstruction speed and large size of holographic microscopes in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a structural schematic diagram of a holographic tomography three-dimensional microscope provided in an embodiment of the present application;

[0026] Figure 2 It is a schematic structural diagram of another holographic tomography three-dimensional microscope provided in an embodiment of the present application;

[0027] Figure 3 It is a flow chart of a method for generating a three-dimensional image based on a holographic tomography three-dimensional microscope provided in an embodiment of the present application;

[0028] Figure 4 is a structural block diagram of a three-dimensional image generating device according to an exemplary embodiment;

[0029] Figure 5 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. Implementation

[0030] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0031] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0032] Before describing the various embodiments shown in the present application, several concepts involved in the present application are first introduced.

[0033] Holographic technology is a technology based on the principle of light wave interference. It records all the light information of each point on the surface of an object into a holographic image. The amplitude and phase of the light waves recorded in the holographic image can encode the surface information and depth information of the object. Holographic imaging can be achieved by inverting this information.

[0034] A digital holographic microscope is a new type of microscope developed using holographic technology. It combines optical microscopy, digital holography, computational imaging and other technologies, uses holographic technology to obtain surface information of objects, and retains the phase information reflecting the height that is lost during plane imaging in the interference fringes, thus obtaining three-dimensional information of the object.

[0035] FPGA (Field Programmable Gate Array) is a type of programmable logic device (PLD). Compared with fixed application-specific chips (ASICs), its characteristic is that it can be programmed based on hardware description language (HDL). FPGA HDL programming is to map the functions described by the code into hardware circuits.

[0036] As mentioned above, the related art has the problems of slow reconstruction speed and large size of holographic microscope.

[0037] In the prior art, holographic microscopes are all computer-based, that is, current holographic microscopes are all equipped with a large host (computer), which makes the holographic microscope large and difficult to carry; tomographic three-dimensional reconstruction based on holographic images is essentially image processing, which involves large and repetitive computational processing. For example, tomographic reconstruction will perform the same operation on different two-dimensional sections, while the computer's CPU is serial processing and is not good at image processing, resulting in slow and inefficient computer three-dimensional reconstruction.

[0038] Some researchers also use GPUs (graphics processing units) to accelerate 3D reconstruction, but this will increase costs, power consumption, and volume, and there are still major problems.

[0039] From the above, it can be seen that the relevant technology still has defects such as slow three-dimensional reconstruction speed, large size, high power consumption, etc. of holographic microscope.

[0040] To this end, the FPGA-based holographic tomography three-dimensional microscope provided in this application can greatly accelerate the reconstruction speed and has a small size.

[0041] See also Figure 1 The embodiment of the present application provides a holographic tomographic 3D microscope 100, which includes a holographic optical path 101 and an FPGA module 102. The holographic optical path 101 includes a light source 1011, a beam splitter 1012, a reflector 1015, and an optical sensor 1014.

[0042] The holographic optical path 101 generates a holographic image of the observed object 1013 based on holographic technology (the principle of light wave interference). Specifically, the observed object 1013 forms a diffuse object beam under laser irradiation, and another part of the laser (from the same light source) is used as a reference beam to be projected onto the holographic film, and superimposed with the object beam to produce interference, thereby converting the phase and amplitude of each point on the object light wave into an intensity that varies in space, thereby using the contrast and interval between the interference fringes to record all the information of the observed object 1013 light wave. After the film recording the interference fringes undergoes processing procedures such as development and fixing, it becomes a holographic image.

[0043] For example, in the holographic optical path 101, the light beam generated by the light source 1011 is split into two beams by the beam splitter 1012. One beam passes through the observed object 1013 to obtain object light, and the other beam is used as reference light. The reference light passes through the reflector 1015 to adjust the propagation direction. The reference light and the object light are superimposed and interfere to generate interference fringes. The optical sensor 1014 records the interference fringes to obtain a holographic image.

[0044] It is worth explaining. Figure 1 The holographic optical path 101 shown is only an example, and the specific structure of the actual holographic optical path may be in different forms.

[0045] The holographic image records all the information of the surface light waves of the observed object 1013, so the three-dimensional image of the observed object can be reconstructed based on the holographic image. In other words, the information such as the amplitude and phase of the light waves recorded by the holographic image can encode the surface information and depth information of the object, and holographic imaging can be achieved by inverting this information.

[0046] The FPGA module 102 is used to generate a three-dimensional image of the observed object 1013 based on the holographic image. For example, a corresponding holographic image reconstruction algorithm can be implemented on the FPGA module 102 using a hardware description language, and the holographic image can be reconstructed into a three-dimensional image based on the holographic image reconstruction algorithm.

[0047] Holographic image reconstruction algorithms include diffraction propagation algorithms such as convolution method, angular spectrum method and Fresnel method, as well as tomographic reconstruction algorithms such as filtered back projection, direct back projection and iterative reconstruction algorithm.

[0048] In a possible implementation, the FPGA module 102 communicates with the optical sensor 1014 via the FPGA high-speed interface, and acquires a holographic image from the optical sensor via the FPGA high-speed interface.

[0049] In a possible implementation, the light source 1011 is a laser light source, and the optical sensor 1014 is a CCD / CMOS camera.

[0050] In a possible implementation, the FPGA module 102 includes a holographic reconstruction unit and a tomographic 3D reconstruction unit. The holographic reconstruction unit and the tomographic 3D reconstruction unit are both implemented based on a hardware description language. The holographic reconstruction unit uses a diffraction propagation algorithm to reconstruct the holographic image into an amplitude map and a phase map; the tomographic 3D reconstruction unit uses a tomographic reconstruction algorithm to perform tomographic 3D reconstruction on the observed object according to the amplitude map and the phase map to obtain a 3D image of the observed object.

[0051] Among them, the diffraction propagation algorithm includes the convolution method, the angular spectrum method and the Fresnel method, and the tomographic reconstruction algorithm includes filtered back projection, direct back projection, iterative reconstruction algorithm, etc., which are not limited here.

[0052] In one possible implementation, the holographic reconstruction unit includes a transformation subunit; the transformation subunit is used to perform Fourier forward transform on the holographic image to obtain a holographic image that has completed Fourier forward transform; and perform inverse Fourier transform on the holographic image that has completed Fourier forward transform to obtain the amplitude map of each projection angle.

[0053] In one possible implementation, the tomographic three-dimensional reconstruction unit includes a data indexing unit, a filtering unit and an interpolation back-projection unit; the indexing module is used to divide the observed object into a number of sections to obtain section intensity data corresponding to each section; the filtering unit is used to filter the section intensity data to remove noise; the interpolation back-projection unit is used to filter and back-project the section intensity data to obtain slice images corresponding to each section, and obtain a three-dimensional image of the observed object based on the slice images corresponding to each section.

[0054] The holographic reconstruction unit is used to reconstruct the holographic image into an amplitude map and a phase map; the tomographic 3D reconstruction unit is used to perform tomographic 3D reconstruction on the observed object according to the amplitude map and the phase map to obtain a 3D image of the observed object. Possibly, the holographic reconstruction unit is implemented based on a diffraction propagation algorithm (such as an angular spectrum method), and the tomographic 3D reconstruction unit is implemented based on a tomographic reconstruction algorithm (such as a two-dimensional filtered back projection algorithm), which does not constitute a specific limitation here.

[0055] It is worth mentioning that FPGA programming can realize parallel pipeline operation, which has a natural advantage in processing large but similar data such as images. When the image is not fully loaded, the FPGA can start image processing. As the image data is loaded, the FPGA pipeline programming can process multiple image blocks in parallel at the same time. For some image operations, the FPGA can almost process the image when the image loading is completed. Therefore, using FPGA for tomographic three-dimensional reconstruction can greatly speed up the reconstruction speed. And FPGA has the advantage of small size, that is, the FPGA-based holographic tomographic three-dimensional microscope provided in this application is smaller in size than the traditional holographic microscope and is easy to carry. In summary, it can be seen that the FPGA-based holographic tomographic three-dimensional microscope provided in this application solves the problems of slow reconstruction speed and large size of holographic microscopes existing in related technologies.

[0056] In an exemplary embodiment, the holographic optical circuit 101 includes a spatial light modulation device.

[0057] Spatial Light Modulator (SLM) is a type of device that can load information onto a one-dimensional or two-dimensional optical data field in order to effectively utilize the inherent speed, parallelism, and interconnection capabilities of light. Such devices can change the amplitude, phase, wavelength, and polarization state of light distribution in space, or convert incoherent light into coherent light, under the control of a time-varying electrical drive signal or other signal.

[0058] Spatial light modulation devices are used to reflect laser beams and achieve high-speed modal changes, thereby meeting the requirements of multi-angle holographic imaging.

[0059] In a possible implementation, the spatial light modulation device may be a digital micromirror device (DMD). A digital micromirror device (DMD) is a type of optical switch that uses a rotating reflector to realize the opening and closing of the optical switch.

[0060] In a possible implementation, the FPGA module 102 includes a modulation device control unit, which is used to control the spatial light modulation device of the holographic optical path 101. Possibly, the modulation device control unit is implemented based on a hardware description language.

[0061] Through the above embodiments, the holographic optical path 102 based on the spatial light modulation device can meet the requirements of multi-angle holographic imaging. The digital micromirror device DMD has a high conversion frequency and can achieve a very high control rate, thereby obtaining a very high frame rate, thereby realizing real-time acquisition of multiple frames of data in three-dimensional imaging; and using the FPGA module 102 to control the spatial light modulation device is beneficial to the unification of the holographic microscope system and simplifies the system structure.

[0062] In an exemplary embodiment, the holographic optical path 101 may further include any one of a laser light source, a reflector, an optical sensor, and a spectroscope.

[0063] Among them, the laser light source is used to generate a laser beam.

[0064] Reflectors are used to adjust the direction of the laser beam.

[0065] An optical sensor is used to record the holographic image formed by interference. The optical sensor can be a CMOS / CCD sensor.

[0066] Beam splitter, used to form coherent light.

[0067] See also Figure 2 The embodiment of the present application provides another holographic tomographic three-dimensional microscope 200, which includes a holographic optical path 201 and an FPGA module 202, wherein the holographic optical path 201 includes a laser light source 2012, a reflector 2013, an optical sensor 2014, a spectroscope 2015, and a spatial light modulation device 2016; the FPGA module 202 includes a holographic reconstruction unit 2021, a tomographic three-dimensional reconstruction unit 2022, and a modulation device control unit 2023.

[0068] Wherein, 2011 refers to the observed object, for example, it can be a living cell, which is not limited here. The spatial light modulation device 2016 is a digital micromirror device (DMD), and correspondingly, the modulation device control unit 2023 is a DMD control unit, which is not specifically limited here.

[0069] Figure 2 In the figure, the solid arrow represents the propagation path of the laser beam, and the dotted arrow represents the data transmission path. For example, the dotted arrow from the optical sensor 2014 to the holographic reconstruction unit 2021 represents the transmission path of the holographic image, the dotted arrow from the DMD control unit 2023 to the spatial light modulation device (DMD) 2016 represents the transmission path of the control signal, the dotted arrow from the holographic reconstruction unit 2021 to the tomographic three-dimensional reconstruction unit 2022 represents the transmission path of the amplitude diagram and the phase diagram, and the dotted arrow pointing outward from the tomographic three-dimensional reconstruction unit 2022 represents the transmission path (i.e., output) of the three-dimensional image reconstructed for the observed object 2011.

[0070] In the holographic optical path 201, the light beam emitted by the laser light source 2012 is split into two light beams by the beam splitter 2015. One of the light beams passes through the reflector 2013 and is projected onto the optical sensor 2014. This light beam is called the reference light. The other light beam passes through the spatial light modulation device (DMD) 2016 and irradiates the observed object 2011. The light beam modulated by the observed object 2011 (called the object light) is projected onto the optical sensor 2014. The reference light and the object light are superimposed at the optical sensor 2014, and interfere with each other to form interference fringes, which are recorded by the optical sensor 2014 to obtain a holographic image.

[0071] The optical sensor 2014 may be a CMOS / CCD sensor.

[0072] The holographic reconstruction unit 2021, the tomographic three-dimensional reconstruction unit 2022, and the modulation device control unit 2023 are all implemented based on the hardware description language. For example, the holographic reconstruction unit 2021 is obtained by implementing the angular spectrum method on the FPGA using Verilog HDL, and is used to reconstruct the holographic image obtained by the holographic optical path 201 into an amplitude map and a phase map; the tomographic three-dimensional reconstruction unit 2022 is obtained by implementing the tomographic three-dimensional reconstruction based on the projection slice theorem using Verilog HDL. Specifically, the two-dimensional filtering back projection algorithm is first implemented using Verilog HDL, and then the two-dimensional filtering back projection algorithm is used in parallel for data of different layers to synchronously reconstruct multi-layer images, and finally the multi-layer images are spliced ​​into three-dimensional data output to obtain a reconstructed three-dimensional image; the modulation device control unit 2023 uses Verilog HDL to implement the control of the spatial light modulation device (DMD) 2016 on the FPGA, and the control signal is transmitted to the spatial light modulation device (DMD) 2016 through the FPGA high-speed interface.

[0073] In one possible implementation, the holographic reconstruction unit 2021 includes a transformation subunit; the transformation subunit is used to perform Fourier forward transform on the holographic image to obtain a holographic image that has completed the Fourier forward transform; and perform inverse Fourier transform on the holographic image that has completed the Fourier forward transform to obtain the amplitude map of each projection angle.

[0074] In one possible implementation, the tomographic three-dimensional reconstruction unit 2022 includes a data indexing unit, a filtering unit and an interpolation back-projection unit; the indexing module is used to divide the observed object into a number of sections to obtain section intensity data corresponding to each section; the filtering unit is used to filter the section intensity data to remove noise; the interpolation back-projection unit is used to filter and back-project the section intensity data to obtain slice images corresponding to each section, and obtain a three-dimensional image of the observed object based on the slice images corresponding to each section.

[0075] Through the above embodiments, based on the general conventional holographic optical path, the present application uses a spatial light modulation device (DMD) 2016 for reflecting laser light, and the spatial light modulation device (DMD) 2016 realizes high-speed modal changes through the control of FPGA, thereby satisfying multi-angle holographic imaging; the control of the spatial light modulation device (DMD) 2016 is realized based on FPGA, which simplifies the structure of the holographic tomography 3D microscope 200. Using FPGA for tomography 3D reconstruction can greatly speed up the reconstruction speed, and FPGA has the advantage of small size. Therefore, compared with the traditional holographic microscope, the FPGA-based holographic tomography 3D microscope provided by the present application has faster reconstruction speed, smaller size, lower cost and power consumption, and is easy to carry. That is, the FPGA-based holographic tomography 3D microscope provided by the present application solves the problems of slow reconstruction speed, large size and high power consumption of holographic microscopes in the related art.

[0076] See also Figure 3 The embodiment of the present application provides a method for generating a three-dimensional image based on a holographic tomography three-dimensional microscope. The method is applied to an FPGA module. The method may include the following steps:

[0077] Step 300, using a diffraction propagation algorithm to reconstruct the holographic image of the observed object into an amplitude map and a phase map.

[0078] The diffraction propagation algorithms can be convolution method, angular spectrum method and Fresnel method.

[0079] Step 320 , performing tomographic three-dimensional reconstruction on the observed object using a tomographic reconstruction algorithm according to the amplitude map and the phase map, to obtain a three-dimensional image of the observed object.

[0080] The tomographic reconstruction algorithm may be a two-dimensional filtered back projection algorithm.

[0081] The following is an embodiment of the device of the present application, which can be used to execute the three-dimensional image generation method involved in the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the three-dimensional image generation method involved in the present application.

[0082] See also Figure 4 In an embodiment of the present application, a three-dimensional image generation device 900 is provided, which is applied to an FPGA module. The device 900 includes but is not limited to: an amplitude-phase reconstruction module 910 and a tomographic three-dimensional reconstruction module 930.

[0083] The amplitude-phase reconstruction module 910 is used to reconstruct the holographic image of the observed object into an amplitude map and a phase map by using a diffraction propagation algorithm.

[0084] The tomographic 3D reconstruction module 930 is used to perform tomographic 3D reconstruction on the observed object according to the amplitude map and the phase map using a tomographic 3D reconstruction algorithm to obtain a 3D image of the observed object.

[0085] It should be noted that the three-dimensional image generating device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing three-dimensional image generation. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the three-dimensional image generating device will be divided into different functional modules to complete all or part of the functions described above.

[0086] In addition, the 3D image generation device and the 3D image generation method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments, which will not be repeated here.

[0087] See also Figure 5 In an embodiment of the present application, an electronic device 4000 is provided, and the electronic device 4000 may include: a desktop computer, a laptop computer, a server, etc.

[0088] exist Figure 5 In the embodiment, the electronic device 4000 includes at least one processor 4001, at least one communication bus 4002 and at least one memory 4003.

[0089] The processor 4001 and the memory 4003 are connected, such as through a communication bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0090] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0091] The communication bus 4002 may include a path to transmit information between the above components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0092] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0093] The memory 4003 stores a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002 .

[0094] When the computer program is executed by the processor 4001 , the three-dimensional image generating method in the above-mentioned embodiments is implemented.

[0095] In addition, a storage medium is provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the three-dimensional image generation method in the above embodiments is implemented.

[0096] In an embodiment of the present application, a computer program product is provided, the computer program product includes a computer program, the computer program is stored in a storage medium. A processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device executes the three-dimensional image generation method in each of the above embodiments.

[0097] Compared with the related art, this application has the following beneficial effects:

[0098] 1. FPGA programming can realize parallel pipeline operation, which has a natural advantage in processing large but similar data such as images. When the image is not fully loaded, the FPGA can start image processing. As the image data is loaded, the FPGA pipeline programming can process multiple image blocks in parallel at the same time. For some image operations, the FPGA can almost complete the image processing when the image loading is finished. Therefore, using FPGA for tomographic three-dimensional reconstruction can greatly speed up the reconstruction speed. And FPGA has the advantage of small size, so the FPGA-based holographic tomographic three-dimensional microscope provided in this application is smaller in size than the traditional holographic microscope and is easy to carry. That is, the FPGA-based holographic tomographic three-dimensional microscope provided in this application solves the problems of slow reconstruction speed and large size of holographic microscopes existing in related technologies.

[0099] 2. Based on the general conventional holographic optical path, the present application uses a spatial light modulator (DMD) 2016 to reflect laser light. The spatial light modulator (DMD) 2016 realizes high-speed modal changes through the control of FPGA, thereby satisfying multi-angle holographic imaging; the control of the spatial light modulator (DMD) 2016 is realized based on FPGA, which simplifies the structure of the holographic tomography 3D microscope 200. Using FPGA for tomography 3D reconstruction can greatly speed up the reconstruction speed, and FPGA has the advantage of small size. Therefore, compared with the traditional holographic microscope, the FPGA-based holographic tomography 3D microscope provided by the present application has faster reconstruction speed, smaller size, lower cost and power consumption, and is easy to carry. That is, the FPGA-based holographic tomography 3D microscope provided by the present application solves the problems of slow reconstruction speed, large size and high power consumption of holographic microscopes in related technologies.

[0100] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0101] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A holographic tomography three-dimensional microscope, characterized in that: include: Holographic optical path and FPGA module; The holographic optical path includes a light source and an optical sensor. The light source is used to generate a light beam. A portion of the light beam passes through an observed object to obtain object light, and another portion of the light beam is used as reference light. The object light and the reference light are superimposed to interfere with each other to generate interference fringes. The optical sensor records the interference fringes to obtain a holographic image of the observed object. The FPGA module includes a holographic reconstruction unit and a tomographic 3D reconstruction unit; the holographic reconstruction unit uses a diffraction propagation algorithm to reconstruct the holographic image into an amplitude map and a phase map; the tomographic 3D reconstruction unit uses a tomographic reconstruction algorithm to perform tomographic 3D reconstruction on the observed object according to the amplitude map and the phase map to obtain a 3D image of the observed object; The holographic reconstruction unit comprises a transformation subunit; the transformation subunit is used to perform Fourier forward transformation on the holographic image to obtain the holographic image after the Fourier forward transformation is completed; Performing inverse Fourier transformation on the holographic image that has completed the forward Fourier transformation to obtain the amplitude map at each projection angle; The tomographic three-dimensional reconstruction unit includes a data index unit, a filtering unit and an interpolation back-projection unit; the data index unit is used to divide the observed object into a number of sections to obtain section intensity data corresponding to each of the sections; the section intensity data includes an amplitude map of each of the projection angles corresponding to the section; the filtering unit is used to filter each of the section intensity data to remove noise; the interpolation back-projection unit is used to filter and back-project each of the section intensity data to obtain a slice image corresponding to each of the sections, and obtain the three-dimensional image of the observed object based on the slice image corresponding to each of the sections.

2. The holographic tomography three-dimensional microscope according to claim 1, characterized in that: The tomographic reconstruction algorithm is a two-dimensional filtered back-projection algorithm.

3. The holographic tomography three-dimensional microscope according to claim 1, characterized in that: The diffraction propagation algorithm is the angular spectrum method.

4. The holographic tomography three-dimensional microscope according to claim 1, characterized in that: The holographic optical path includes a spatial light modulation device, and the spatial light modulation device is used to reflect the light beam to achieve high-speed modal change.

5. The holographic tomography three-dimensional microscope according to claim 4, characterized in that: The FPGA module includes a modulation device control unit, and the modulation device control unit is used to control the spatial light modulation device.

6. The holographic tomographic three-dimensional microscope according to any one of claims 4 to 5, characterized in that: The spatial light modulation device is a digital micromirror device.

7. The holographic tomography three-dimensional microscope according to claim 1, characterized in that: The holographic optical path includes a reflector for adjusting the direction of the laser beam.

8. The holographic tomography three-dimensional microscope according to claim 1, characterized in that: The holographic optical path includes a beam splitter for forming coherent light.

Images