Optical diffraction tomography solution method, device, electronic equipment and storage medium

Abstract

The present invention provides an optical diffraction tomography solution method, device, electronic device and storage medium, the method comprising: obtaining multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of the environment in which the target sample is located; determining the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images; performing spectrum mapping on the light field distribution information to obtain the imaging solution result corresponding to the target sample. The method makes full use of the parallel computing capability of a graphics processor, optimizes the solution process of optical diffraction tomography, accelerates the calculation of a large amount of data (multiple sample images and multiple background images), can quickly obtain the imaging solution result corresponding to the target sample, effectively improves the solution efficiency, and ensures the real-time performance of imaging.

Description

Technical Field

[0001] The present invention relates to the technical field of optical diffraction tomography, and in particular to an optical diffraction tomography solution method, device, electronic equipment and storage medium. Background Art

[0002] Optical diffraction tomography (ODT) is a new type of three-dimensional label-free microscopic imaging technology with the characteristics of high spatial resolution and weak phototoxicity, so it can perform long-term super-resolution three-dimensional imaging of living cells. Compared with the traditional "what you see is what you get" optical microscopic imaging system, the optical diffraction tomography system can observe the life activities of cells from a longer time, higher dimension, and higher resolution. It is of great significance to explore the biological and chemical nature of life activities such as cell growth and differentiation, metabolism and reproduction, movement and communication, aging and apoptosis, inheritance and evolution, and also provides a more reliable scientific basis for exploring the causes, development and treatment of major diseases. In order to achieve three-dimensional super-resolution imaging, the optical diffraction tomography system takes multi-angle shots of the target sample (such as a cell sample) and the environment in which the target sample is located, and obtains multiple sample images of the target sample and multiple background images of the environment in which the target sample is located.

[0003] The existing optical diffraction tomography solution method is based on the central processing unit (CPU) to solve multiple sample images of the target sample and multiple background images of the environment in which the target sample is located. Due to the complexity of the solution process and the huge amount of data, the CPU needs to consume a lot of computing resources and time, resulting in low solution efficiency and difficulty in ensuring the real-time imaging. Summary of the invention

[0004] The present invention provides an optical diffraction tomography solution method, device, electronic device and storage medium, which are used to solve the defects that the CPU needs to consume a lot of computing resources and time in the existing optical diffraction tomography solution method, resulting in low solution efficiency and difficulty in ensuring the real-time performance of imaging. The method makes full use of the parallel computing capability of a graphics processor, optimizes the solution process of the optical diffraction tomography, accelerates the calculation of a large amount of data (multiple sample images and multiple background images), and can quickly obtain the imaging solution result corresponding to the target sample, effectively improve the solution efficiency, and ensure the real-time performance of imaging.

[0005] The present invention provides an optical diffraction tomography solution method, which is applied to an electronic device, wherein the electronic device is provided with a graphics processor, and the graphics processor is equipped with a unified computing device architecture. The method comprises:

[0006] Acquire multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of an environment in which the target sample is located;

[0007] Determining light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images;

[0008] The light field distribution information is subjected to spectrum mapping to obtain an imaging solution result corresponding to the target sample.

[0009] According to an optical diffraction tomography solution method provided by the present invention, the light field distribution information corresponding to the target sample is determined based on the multiple background images and the multiple sample images, including: determining a first light field distribution matrix based on the multiple background images; determining a second light field distribution matrix based on the multiple sample images; and determining the light field distribution information based on the first light field distribution matrix and the second light field distribution matrix.

[0010] According to an optical diffraction tomography solution method provided by the present invention, the first light field distribution matrix is ​​determined according to the multiple background images, including: performing Fourier transform on the multiple background images to obtain a first result; determining the off-axis holographic parameter array corresponding to the multiple background images according to the first result; intercepting the first result based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images to obtain a second result; performing inverse Fourier transform on the second result to obtain a third result; and determining the first light field distribution matrix according to the third result.

[0011] According to an optical diffraction tomography solution method provided by the present invention, the first light field distribution matrix includes a first amplitude matrix and a first phase matrix, and the second light field distribution matrix includes a second amplitude matrix and a second phase matrix; determining the light field distribution information according to the first light field distribution matrix and the second light field distribution matrix includes: determining a target amplitude matrix according to the first amplitude matrix and the second amplitude matrix; determining a target phase matrix according to the first phase matrix and the second phase matrix; and determining the light field distribution information according to the target amplitude matrix and the target phase matrix.

[0012] According to an optical diffraction tomography solution method provided by the present invention, based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images, the first result is intercepted to obtain the second result, including: determining the interception center according to the off-axis holographic parameter array; determining the interception side length according to the shooting parameters and the side length corresponding to each background image in the multiple background images, the shooting parameters including: shooting camera pixel size parameters, objective lens numerical aperture parameters, objective lens magnification parameters and shooting light wavelength parameters; intercepting the first result according to the interception center and the interception side length to obtain the second result.

[0013] According to an optical diffraction tomography solution method provided by the present invention, the shooting parameters also include the refractive index of the environment in which the objective lens is located; correspondingly, the spectrum mapping of the light field distribution information to obtain the imaging solution result corresponding to the target sample includes: performing Fourier transform on the light field distribution information to obtain a fourth result; determining the spectrum stacking z-axis parameter according to the off-axis holographic parameter array, the shooting light wavelength parameter and the refractive index; based on the off-axis holographic parameter array and the spectrum stacking z-axis parameter, spectrum mapping the fourth result to obtain a spectrum stacking matrix; and determining the imaging solution result according to the spectrum stacking matrix.

[0014] According to an optical diffraction tomography solution method provided by the present invention, the first light field distribution matrix includes a first amplitude matrix and a first phase matrix, and determining the first light field distribution matrix according to the third result includes: determining the complex elements in the third result; determining the first amplitude matrix according to the real part of the complex elements; and determining the first phase matrix according to the imaginary part of the complex elements.

[0015] The present invention also provides an optical diffraction tomography solution device, which is applied to an electronic device, wherein the electronic device is provided with a graphics processor, and the graphics processor is equipped with a unified computing device architecture, and the device comprises:

[0016] An acquisition module, used to acquire multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of the environment in which the target sample is located;

[0017] A solution module is used to determine the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images; and perform spectrum mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample.

[0018] The present invention also provides an electronic device, comprising a memory, a graphics processor, and a computer program stored in the memory and executable on the graphics processor, wherein the graphics processor implements any of the above-described optical diffraction tomography solution methods when executing the computer program.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a graphics processor, implements any of the above-mentioned optical diffraction tomography solution methods.

[0020] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a graphics processor, the optical diffraction tomography solution method as described above is implemented.

[0021] The optical diffraction tomography solution method, device, electronic device and storage medium provided by the present invention obtain multiple sample images of the target sample taken at multiple angles by the optical diffraction tomography system, as well as multiple background images of the environment in which the target sample is located; determine the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images; perform spectrum mapping on the light field distribution information to obtain the imaging solution result corresponding to the target sample. The method makes full use of the parallel computing capability of the graphics processor, optimizes the solution process of the optical diffraction tomography, accelerates the calculation of a large amount of data (multiple sample images and multiple background images), can quickly obtain the imaging solution result corresponding to the target sample, effectively improves the solution efficiency, and ensures the real-time performance of the imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a schematic diagram of the process of the optical diffraction tomography solution method provided by the present invention;

[0024] Figure 2 is a schematic diagram of a scenario for determining a first sub-result and a second sub-result provided by the present invention;

[0025] Figure 3 It is a comparative schematic diagram of the solution time of different solutions provided by the present invention;

[0026] Figure 4 is a schematic diagram comparing imaging solution results of different schemes provided by the present invention;

[0027] Figure 5 It is a schematic structural diagram of the optical diffraction tomography solution device provided by the present invention;

[0028] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be noted that the execution subject involved in the embodiment of the present invention may be an optical diffraction tomography solution device or an electronic device. Optionally, the electronic device may include: a computer and a mobile terminal.

[0031] The embodiment of the present invention is further described below by taking an electronic device as an example, wherein the electronic device is provided with a graphics processing unit (GPU), and the graphics processor is equipped with a unified computing device architecture (CUDA).

[0032] Among them, the Unified Computing Device Architecture is a programming platform developed by a certain company to support its self-produced graphics processors, which is used to solve the problem of graphics processor programming difficulties.

[0033] like Figure 1 FIG. 1 is a flow chart of the optical diffraction tomography solution method provided by the present invention, which may include:

[0034] 101. Acquire multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of an environment in which the target sample is located.

[0035] Exemplarily, the target sample may be a biological sample. Optionally, the biological sample may be a living cell, or a fixed or dead cell.

[0036] In the microscopic shooting area of ​​the optical diffraction tomography system, the optical diffraction tomography system respectively shoots the target sample and the environment in which the target sample is located at multiple angles, and correspondingly obtains multiple sample images of the target sample and multiple background images of the environment in which the target sample is located. Based on this, the electronic device obtains the multiple sample images and multiple background images sent by the optical diffraction tomography system, so as to subsequently determine the light field distribution information corresponding to the target sample.

[0037] 102. Determine light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images.

[0038] The light field distribution information is used to characterize the light field distribution characteristics corresponding to the target sample after removing the background influence of the background image on the sample image.

[0039] After acquiring multiple sample images of the target sample and multiple background images of the environment in which the target sample is located, the electronic device can determine the light field distribution information corresponding to the target sample based on the multiple sample images and the multiple background images. The light field distribution information is relatively accurate, so that the subsequently determined imaging solution results are also relatively accurate.

[0040] In some embodiments, the electronic device determines the light field distribution information corresponding to the target sample based on multiple background images and multiple sample images, which may include: the electronic device determines a first light field distribution matrix based on the multiple background images; the electronic device determines a second light field distribution matrix based on the multiple sample images; the electronic device determines the light field distribution information based on the first light field distribution matrix and the second light field distribution matrix.

[0041] It should be noted that the timing of the electronic device determining the first light field distribution matrix and the electronic device determining the second light field distribution matrix is ​​not limited.

[0042] In the process of determining the light field distribution information, the electronic device can determine a first light field distribution matrix based on multiple background images, and can determine a second light field distribution matrix based on multiple sample images; then, the electronic device can determine the light field distribution information corresponding to the target sample based on the first light field distribution matrix and the second light field distribution matrix, and the light field distribution information is relatively accurate.

[0043] In some embodiments, the electronic device determines a first light field distribution matrix based on multiple background images, which may include: the electronic device performs Fourier transform on the multiple background images to obtain a first result; the electronic device determines the off-axis holographic parameter array corresponding to the multiple background images based on the first result; the electronic device intercepts the first result based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images to obtain a second result; the electronic device performs inverse Fourier transform on the second result to obtain a third result; and the electronic device determines the first light field distribution matrix based on the third result.

[0044] The first result refers to the frequency spectra corresponding to the multiple background images, which can be represented by Back_fft.

[0045] The third result refers to the initial light field distribution matrix corresponding to the multiple background images, which can be represented by Back_ifft.

[0046] The side length corresponding to each background image in multiple background images can be represented by size.

[0047] It should be noted that the above-mentioned Fourier transform refers to a two-dimensional Fourier transform; and the above-mentioned inverse Fourier transform refers to a two-dimensional inverse Fourier transform.

[0048] After acquiring multiple background images, the electronic device may perform a two-dimensional Fourier transform on the multiple background images to obtain the frequency spectra corresponding to the multiple background images, i.e., the first result Back_fft. Specifically, the electronic device performs a two-dimensional Fourier transform from the real domain to the complex domain on the background image matrix Back composed of the multiple background images to obtain the first result Back_fft. Wherein, Back_fft[i] represents the frequency spectra corresponding to the i-th background image among the multiple background images, i=1,…,n, and n represents the number of the multiple background images.

[0049] Then, after determining the first result, the electronic device may perform a modulus operation on the first result to obtain a fifth result, and then determine the position corresponding to the maximum modulus value of the non-central area from the fifth result, and then determine the off-axis holographic parameter array corresponding to the multiple background images. The off-axis holographic parameter array can be represented by kScan. Among them, kScan[i].row represents the holographic off-axis information of the i-th background image in the x direction, and kScan[i].col represents the holographic off-axis information of the i-th background image in the y direction.

[0050] Then, the electronic device can intercept the first result Back_fft based on the above-mentioned off-axis holographic parameter array kScan, the side length size corresponding to each background image in the multiple background images, and the shooting parameters of the optical diffraction tomography system to obtain the interception result, that is, the second result; then, the electronic device performs a two-dimensional inverse Fourier transform on the second result to obtain the initial light field distribution matrix corresponding to the multiple background images, that is, the third result Back_ifft; then, the electronic device can determine the first light field distribution matrix according to the third result Back_ifft, and the first light field distribution matrix is ​​relatively accurate, so that the subsequently determined light field distribution information is also relatively accurate.

[0051] In some embodiments, the electronic device intercepts the first result to obtain the second result based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images, which may include: the electronic device determines the interception center according to the off-axis holographic parameter array; the electronic device determines the interception side length according to the shooting parameters and the side length corresponding to each background image in the multiple background images, the shooting parameters including: the shooting camera pixel size parameter pixel, the objective lens numerical aperture parameter NA, the objective lens magnification parameter mag and the shooting light wavelength parameter lambda; the electronic device intercepts the first result according to the interception center and the interception side length to obtain the second result.

[0052] The intercepted side length can be represented by kBoundPixel.

[0053] It should be noted that there is no limit to the timing of the electronic device determining the clipping center and the electronic device determining the clipping side length.

[0054] Specifically, the electronic device determines the mean value of kScan[i].row as the shooting position parameter meanRow in the x direction, and determines the mean value of kScan[i].col as the shooting position parameter meanCol in the y direction, and further determines (meanRow, meanCol) as the interception center.

[0055] Exemplarily, the calculation formula for the intercepted side length is: kBoundPixel=2×NA×size×pixel / lambda / mag.

[0056] Based on this, the electronic device can determine the clipping area according to the clipping center (meanRow, meanCol) and the clipping side length kBoundPixel, and then clip the first result based on the clipping area to obtain the second result.

[0057] Exemplarily, for a background image among the multiple background images, the electronic device performs a two-dimensional Fourier transform on the background image to obtain a first sub-result, and the first sub-result belongs to the first result; the electronic device intercepts the first sub-result based on the interception center and the interception side length to obtain a second sub-result, and the second sub-result belongs to the second result. Figure 2 FIG. 1 is a schematic diagram of a scenario for determining a first sub-result and a second sub-result provided by the present invention. Figure 2 It can be seen that the clipped area can be regarded as a square area, which covers the bright spot in the upper left corner area of ​​the first sub-result. The bright spot in the upper left corner area is the clipping center of the clipped area, and the clipped area does not include the bright spot in the central area.

[0058] In some embodiments, the first light field distribution matrix may include a first amplitude matrix and a first phase matrix. The electronic device determines the first light field distribution matrix based on the third result, which may include: the electronic device determines the complex elements in the third result; the electronic device determines the first amplitude matrix based on the real part of the complex elements; the electronic device determines the first phase matrix based on the imaginary part of the complex elements.

[0059] Among them, the first amplitude matrix can be represented by BackAmp; the first phase matrix can be represented by BackPhs.

[0060] It should be noted that the timing of the electronic device determining the first amplitude matrix and the electronic device determining the first phase matrix is ​​not limited.

[0061] In the process of determining the first light field distribution matrix, the electronic device may first determine the complex elements in the third result Back_ifft, and construct a first initial amplitude matrix and a first initial phase matrix. Then, the electronic device may copy the real part of the complex elements to the first initial amplitude matrix, and determine the first initial amplitude matrix at this time as the first amplitude matrix BackAmp. Similarly, the electronic device may copy the imaginary part of the complex elements to the first initial phase matrix, and determine the first initial phase matrix at this time as the first phase matrix BackPhs.

[0062] It should be noted that the process of the electronic device determining the second light field distribution matrix is ​​the same as the process of the electronic device determining the first light field distribution matrix, which will not be described in detail here.

[0063] In some embodiments, the first light field distribution matrix may include a first amplitude matrix and a first phase matrix, and the second light field distribution matrix may include a second amplitude matrix and a second phase matrix; the electronic device determines the light field distribution information based on the first light field distribution matrix and the second light field distribution matrix, which may include: the electronic device determines the target amplitude matrix based on the first amplitude matrix and the second amplitude matrix; the electronic device determines the target phase matrix based on the first phase matrix and the second phase matrix; the electronic device determines the light field distribution information based on the target amplitude matrix and the target phase matrix.

[0064] The second phase matrix can be represented by SamplePhs; the second amplitude matrix can be represented by SampleAmp.

[0065] The target amplitude matrix is ​​calculated as:

[0066] The calculation formula of the target phase matrix is: target phase matrix = SamplePhs-BackPhs.

[0067] The light field distribution information can be expressed as result, and the calculation formula of the light field distribution information is: Where i represents an imaginary unit. It can be seen that the light field distribution information result removes the background influence of the background image on the sample image.

[0068] It should be noted that there is no limitation on the timing of the electronic device determining the target amplitude matrix and the electronic device determining the target phase matrix.

[0069] In the process of determining the light field distribution information result, the electronic device can determine the target amplitude matrix according to the first amplitude matrix BackAmp and the second amplitude matrix SampleAmp, and can determine the target phase matrix according to the first phase matrix BackPhs and the second phase matrix SamplePhs. Based on this, the electronic device can determine the light field distribution information result according to the target amplitude matrix and the target phase matrix. The light field distribution information result removes the background influence of the background image on the sample image, and can effectively reflect the light field distribution characteristics corresponding to the target sample.

[0070] 103. Perform spectrum mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample.

[0071] After determining the light field distribution information result, the electronic device can perform three-dimensional spectrum mapping on the light field distribution information result to obtain the imaging solution result corresponding to the target sample. The entire process fully utilizes the parallel computing capability of the graphics processor, optimizes the solution process of optical diffraction tomography, and accelerates the calculation of a large amount of data (multiple sample images and multiple background images), which can quickly obtain the imaging solution result corresponding to the target sample, effectively improve the solution efficiency, and ensure the real-time imaging.

[0072] In some embodiments, the shooting parameters may also include the refractive index of the environment in which the objective lens is located;

[0073] Correspondingly, the electronic device performs spectrum mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample, which may include: the electronic device performs Fourier transform on the light field distribution information to obtain a fourth result; the electronic device determines the spectrum ball stacking z-axis parameter based on the off-axis holographic parameter array, the shooting light wavelength parameter and the refractive index; the electronic device performs spectrum mapping on the fourth result based on the off-axis holographic parameter array and the spectrum ball stacking z-axis parameter to obtain a spectrum ball stacking matrix; the electronic device determines the imaging solution result based on the spectrum ball stacking matrix.

[0074] Among them, the refractive index of the environment in which the objective lens is located can be represented by para_nm; the z-axis parameter of the spectrum stacking ball can be represented by kScanZ.

[0075] The fourth result refers to the frequency spectrum matrix corresponding to the target sample after removing the background influence at multiple shooting angles, which can be represented by result_fft.

[0076] It should be noted that there is no limitation on the timing between the electronic device determining the fourth result and the electronic device determining the z-axis parameter kScanZ of the spectrum stacking ball.

[0077] The electronic device may perform Fourier transform on the light field distribution information result to obtain a frequency spectrum matrix corresponding to the target sample after removing the background influence at multiple shooting angles, that is, a fourth result result_fft.

[0078] Exemplarily, the calculation formula of the spectrum ball z-axis parameter is:

[0079] Among them, Para_km=2×π×(para_nm / lambda).

[0080] Next, the electronic device can construct an initial spectrum ball stacking matrix f_Joint and a counting matrix f_count, and then the electronic device uses a kernel function to complete the spectrum ball stacking. Specifically, the holographic off-axis information kScan[i].row in the x direction and the holographic off-axis information kScan[i].col in the y direction provided by the off-axis holographic parameter array kScan are respectively determined as the x-coordinate and y-coordinate of each layer result_fft[i] in the fourth result result_fft. At the same time, the spectrum ball stacking z-axis parameter kScanZ[i] is determined as the z-coordinate of each layer result_fft[i] in the fourth result result_fft; based on the three-dimensional coordinates composed of the x-coordinate, the y-coordinate and the z-coordinate, the fourth result result_fft is realized. i] mapping from two dimensions to three dimensions, that is, mapping each layer result_fft[i] of the fourth result result_fft to the initial spectrum stacking matrix f_Joint, and adding 1 to the corresponding position of the counting matrix f_count matrix each time the corresponding position of the initial spectrum stacking matrix f_Joint is mapped once; then, the electronic device divides the elements at the corresponding positions of the initial spectrum stacking matrix f_Joint by the elements at the corresponding positions of the counting matrix f_count, and saves the result in the initial spectrum stacking matrix f_Joint, and determines the initial spectrum stacking matrix f_Joint at this time as the spectrum stacking matrix f_Joint'.

[0081] Then, the electronic device performs a three-dimensional inverse Fourier transform on the spectrum stacking matrix f_Joint' to obtain the refractive index matrix RI. Next, a square root operation is performed on the refractive index matrix RI to obtain a complex result, and the real part of the complex result is multiplied by the refractive index para_nm of the environment in which the objective lens is located to obtain the final refractive index matrix RI'; then, the final refractive index matrix RI' is determined as the imaging solution result corresponding to the target sample.

[0082] It should be noted that after determining the imaging solution result corresponding to the target sample, if the next target sample is in the same environment as the target sample, it means that the next target sample and the target sample are in the same background environment. The electronic device can directly reuse the first light field distribution matrix corresponding to multiple background images of the environment in which the target sample is located. The subsequent solution process is the same as the solution process involved above, and will not be described in detail here.

[0083] In an embodiment of the present invention, multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system and multiple background images of the environment in which the target sample is located are obtained; the light field distribution information corresponding to the target sample is determined based on the multiple background images and the multiple sample images; the light field distribution information is spectrum mapped to obtain the imaging solution result corresponding to the target sample. This method makes full use of the parallel computing capability of the graphics processor, optimizes the solution process of the optical diffraction tomography, accelerates the calculation of a large amount of data (multiple sample images and multiple background images), and can quickly obtain the imaging solution result corresponding to the target sample, effectively improving the solution efficiency to ensure the real-time performance of the imaging.

[0084] The experimental results of the embodiments of the present invention are further described below:

[0085] For example, Figure 3 , which is a schematic diagram comparing the solution time of different solutions provided by the present invention. Figure 3 In the embodiment, the first solution is a CPU-based solution method in the prior art; the second solution is an optical diffraction tomography solution method proposed in the present invention. Figure 3 It can be seen that under different image sizes, the solution time of the second solution is much lower than that of the first solution, which shows that the second solution can effectively improve the solution efficiency.

[0086] For example, Figure 4 , which is a schematic diagram comparing the imaging solution results of different schemes provided by the present invention. Figure 4 (a) in the figure represents the imaging solution of the first scheme. Figure 4 (b) in the figure shows the imaging solution of the second solution. Figure 4 It can be seen that compared with the imaging solution results of the first solution, the imaging solution results of the second solution have basically no error in the cell structure, and the background in the imaging solution results of the second solution has better brightness and darkness uniformity.

[0087] The optical diffraction tomography solving device provided by the present invention is described below. The optical diffraction tomography solving device described below and the optical diffraction tomography solving method described above can be referred to each other.

[0088] like Figure 5 FIG. 1 is a schematic diagram of the structure of an optical diffraction tomography solution device provided by the present invention, which is applied to an electronic device, wherein a graphics processor is provided in the electronic device, and the graphics processor is equipped with a unified computing device architecture. The device may include:

[0089] An acquisition module 501 is used to acquire multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of the environment in which the target sample is located;

[0090] The solution module 502 is used to determine the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images; and perform spectrum mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample.

[0091] Optionally, the solving module 502 is specifically configured to determine a first light field distribution matrix according to the multiple background images; determine a second light field distribution matrix according to the multiple sample images; and determine the light field distribution information according to the first light field distribution matrix and the second light field distribution matrix.

[0092] Optionally, the solution module 502 is specifically used to perform Fourier transform on the multiple background images to obtain a first result; determine the off-axis holographic parameter array corresponding to the multiple background images based on the first result; intercept the first result based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images to obtain a second result; perform inverse Fourier transform on the second result to obtain a third result; and determine the first light field distribution matrix based on the third result.

[0093] Optionally, the first light field distribution matrix includes a first amplitude matrix and a first phase matrix, and the second light field distribution matrix includes a second amplitude matrix and a second phase matrix; the solution module 502 is specifically used to determine a target amplitude matrix according to the first amplitude matrix and the second amplitude matrix; determine a target phase matrix according to the first phase matrix and the second phase matrix; and determine the light field distribution information according to the target amplitude matrix and the target phase matrix.

[0094] Optionally, the solution module 502 is specifically used to determine the interception center according to the off-axis holographic parameter array; determine the interception side length according to the shooting parameters and the side length corresponding to each background image in the multiple background images, the shooting parameters including: shooting camera pixel size parameters, objective lens numerical aperture parameters, objective lens magnification parameters and shooting light wavelength parameters; intercept the first result according to the interception center and the interception side length to obtain the second result.

[0095] Optionally, the shooting parameters also include: the refractive index of the environment in which the objective lens is located; correspondingly, the solution module 502 is specifically used to perform Fourier transform on the light field distribution information to obtain a fourth result; determine the spectrum ball z-axis parameter according to the off-axis holographic parameter array, the shooting light wavelength parameter and the refractive index; perform spectrum mapping on the fourth result based on the off-axis holographic parameter array and the spectrum ball z-axis parameter to obtain a spectrum ball matrix; determine the imaging solution result according to the spectrum ball matrix.

[0096] Optionally, the first light field distribution matrix includes a first amplitude matrix and a first phase matrix, and the solving module 502 is specifically used to determine the complex elements in the third result; determine the first amplitude matrix according to the real part of the complex elements; and determine the first phase matrix according to the imaginary part of the complex elements.

[0097] like Figure 6 As shown, it is a structural schematic diagram of an electronic device provided by the present invention, and the electronic device may include: a graphics processor (GPU) 610, a communication interface (Communications Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the graphics processor 610, the communication interface 620, and the memory 630 complete mutual communication through the communication bus 640. The graphics processor 610 can call the logic instructions in the memory 630 to execute the optical diffraction tomography solution method, which includes: obtaining multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of the environment in which the target sample is located; determining the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images; performing spectrum mapping on the light field distribution information to obtain the imaging solution result corresponding to the target sample.

[0098] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a graphics processor, the computer can execute the optical diffraction tomography solution method provided by the above methods, and the method includes: obtaining multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of the environment in which the target sample is located; determining light field distribution information corresponding to the target sample based on the multiple background images and the multiple sample images; and performing spectral mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample.

[0100] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a graphics processor, it is implemented to execute the optical diffraction tomography solution method provided by the above-mentioned methods. The method includes: acquiring multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of the environment in which the target sample is located; determining light field distribution information corresponding to the target sample based on the multiple background images and the multiple sample images; and performing spectral mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An optical diffraction tomography solution method, characterized in that: Applied to an electronic device, wherein the electronic device is provided with a graphics processor, and the graphics processor is equipped with a unified computing device architecture, the method comprises: Acquire multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of an environment in which the target sample is located; Determining light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images; Performing spectrum mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample; Wherein, determining the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images includes: Determining a first light field distribution matrix according to the multiple background images; Determining a second light field distribution matrix according to the plurality of sample images; Determining the light field distribution information according to the first light field distribution matrix and the second light field distribution matrix; The step of determining a first light field distribution matrix according to the plurality of background images comprises: Performing Fourier transform on the multiple background images to obtain a first result; Determine, according to the first result, an off-axis holographic parameter array corresponding to the plurality of background images; Based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images, the first result is intercepted to obtain a second result; the shooting parameters include: a shooting camera pixel size parameter, an objective lens numerical aperture parameter, an objective lens magnification parameter and a shooting light wavelength parameter; Performing an inverse Fourier transform on the second result to obtain a third result; Determining the first light field distribution matrix according to the third result; The shooting parameters also include the refractive index of the environment in which the objective lens is located. The spectrum mapping of the light field distribution information to obtain the imaging solution result corresponding to the target sample includes: Performing Fourier transform on the light field distribution information to obtain a fourth result; Determine the z-axis parameter of the spectrum stacking ball according to the off-axis holographic parameter array, the shooting light wavelength parameter and the refractive index; Based on the off-axis holographic parameter array and the spectrum ball stacking z-axis parameter, performing spectrum mapping on the fourth result to obtain a spectrum ball stacking matrix; The imaging solution result is determined according to the spectrum balling matrix.

2. The method according to claim 1, characterized in that The first light field distribution matrix includes a first amplitude matrix and a first phase matrix, and the second light field distribution matrix includes a second amplitude matrix and a second phase matrix; The determining the light field distribution information according to the first light field distribution matrix and the second light field distribution matrix includes: Determining a target amplitude matrix according to the first amplitude matrix and the second amplitude matrix; Determining a target phase matrix according to the first phase matrix and the second phase matrix; The light field distribution information is determined according to the target amplitude matrix and the target phase matrix.

3. The method according to claim 1, characterized in that The first result is intercepted based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images to obtain the second result, including: Determining a capture center according to the off-axis holographic parameter array; Determine a cut side length according to the shooting parameters and the side length corresponding to each background image in the plurality of background images; The first result is intercepted according to the interception center and the interception side length to obtain the second result.

4. The method according to claim 1, characterized in that: The first light field distribution matrix includes a first amplitude matrix and a first phase matrix, and determining the first light field distribution matrix according to the third result includes: determining a plurality of elements in the third result; determining the first amplitude matrix according to the real parts of the complex elements; The first phase matrix is ​​determined according to the imaginary parts of the complex elements.

5. An optical diffraction tomography solution device, characterized in that: Applied to an electronic device, wherein the electronic device is provided with a graphics processor, and the graphics processor is equipped with a unified computing device architecture, and the device comprises: An acquisition module, used to acquire multiple sample images of a target sample taken at multiple angles by an optical diffraction tomography system, and multiple background images of the environment in which the target sample is located; A solution module, used to determine the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images; perform spectrum mapping on the light field distribution information to obtain an imaging solution result corresponding to the target sample; The solving module is specifically used to determine the light field distribution information corresponding to the target sample according to the multiple background images and the multiple sample images, including: Determining a first light field distribution matrix according to the multiple background images; Determining a second light field distribution matrix according to the plurality of sample images; Determining the light field distribution information according to the first light field distribution matrix and the second light field distribution matrix; The solving module is specifically used to determine the first light field distribution matrix according to the multiple background images, including: Performing Fourier transform on the multiple background images to obtain a first result; Determine, according to the first result, an off-axis holographic parameter array corresponding to the plurality of background images; Based on the off-axis holographic parameter array, the shooting parameters of the optical diffraction tomography system and the side length corresponding to each background image in the multiple background images, the first result is intercepted to obtain a second result; the shooting parameters include: a shooting camera pixel size parameter, an objective lens numerical aperture parameter, an objective lens magnification parameter and a shooting light wavelength parameter; Performing an inverse Fourier transform on the second result to obtain a third result; Determining the first light field distribution matrix according to the third result; The shooting parameters also include the refractive index of the environment in which the objective lens is located. The solution module is specifically used to perform spectrum mapping on the light field distribution information to obtain the imaging solution result corresponding to the target sample, including: Performing Fourier transform on the light field distribution information to obtain a fourth result; Determine the z-axis parameter of the spectrum stacking ball according to the off-axis holographic parameter array, the shooting light wavelength parameter and the refractive index; Based on the off-axis holographic parameter array and the spectrum ball stacking z-axis parameter, performing spectrum mapping on the fourth result to obtain a spectrum ball stacking matrix; The imaging solution result is determined according to the spectrum balling matrix.

6. An electronic device comprising a memory, a graphics processor, and a computer program stored in the memory and executable on the graphics processor, characterized in that: When the graphics processor executes the computer program, the optical diffraction tomography solution method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a graphics processor, the optical diffraction tomography solution method according to any one of claims 1 to 4 is implemented.

Images