A 3D reconstruction method based on a single phase image

Abstract

This invention provides a 3D reconstruction method based on a single phase image. Training and testing datasets are constructed. The training dataset consists of single phase images of polystyrene microsphere samples treated in Olympus immersion oil, sodium chloride solutions of varying concentrations, and ultrapure water. The testing dataset contains single phase image data from polystyrene microsphere samples and red blood cell samples that were not included in the training dataset. The training dataset is then imported into a machine learning model for training, and the test dataset is used for prediction to obtain the 3D structural morphology of substructured biological cells. This method solves the problems of existing technologies, such as large data volumes, excessive time consumption, and complex implementation processes.

Description

Technical Field

[0001] The present invention belongs to the technical field of phase microscopy, and in particular relates to a method for rapidly reconstructing the three-dimensional morphology of a phase body based on a single phase image and machine learning. Background Art

[0002] Biological cells are the fundamental units of biological structure and life. Cell morphology is closely linked to their physiological activities and functions. For example, cell morphology undergoes changes during proliferation, division, metabolism, and cancer. Morphology can, to a large extent, intuitively characterize a cell's state. Therefore, the study of cell morphology is of great significance to health and cell biology. However, most cells are colorless and transparent, representing phase objects, making them difficult to image with traditional microscopes. Quantitative phase microscopy (QPM) modulates the intangible phase information of light waves into easily observable and collectable amplitude information. Because light passing through transparent phase objects induces a phase shift closely related to the refractive index, QPM has been applied to obtain phase distribution data of biological cells. Phase data reflects the cumulative thickness and refractive index of a sample in a specific illumination direction, but cannot accurately capture the sample's morphology and structure. Traditional methods for reconstructing the three-dimensional topography of biological cells require phase images from multiple angles of the sample. This requires significant resources, both in terms of data collection and reconstruction time, and cannot meet the demand for rapid reconstruction of the three-dimensional topography of biological cells in the life sciences and medical fields. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the present invention provides a three-dimensional reconstruction method based on a single phase image, which overcomes the shortcomings of the existing phase volume three-dimensional reconstruction method, such as large data volume, long time consumption, and complex implementation process.

[0004] The present invention achieves the above technical objectives through the following technical means.

[0005] A 3D reconstruction method based on a single phase image is specifically as follows:

[0006] Constructing a machine learning model, importing a training data set into the machine learning model for training, and using a test data set to predict and obtain the three-dimensional structural morphology of the substructured biological cells;

[0007] The training dataset consists of single phase images of polystyrene microsphere samples treated in Olympus immersion oil, sodium chloride solutions of different concentrations, and ultrapure water. The test dataset consists of single phase image data of polystyrene microsphere samples and red blood cell samples that do not appear in the training dataset.

[0008] According to a further technical solution, the processed single phase image includes edge extraction, normalization and feature engineering processing.

[0009] A further technical solution is that the edge extraction is specifically as follows: gradient calculation is performed on the same row or column of pixels in a single phase image to obtain a gradient curve for each pixel row or column, and the jump points on the gradient curve are used as sample boundary points.

[0010] A further technical solution is to assume that the phase value of the sample boundary point is 0, and use this as a standard to normalize the phase map to obtain a phase value distribution map based on a unified standard.

[0011] As a further technical solution, the feature engineering process is to extract features from the phase value distribution map to obtain the phase gradient value.

[0012] According to a further technical solution, the features of the training data set include phase values, pixel coordinates, incident light wavelength, refractive index of the ambient liquid, and phase gradient values ​​obtained from a sample optical experiment, and the label of the training data set is spatial thickness.

[0013] A further technical solution is to use the test data set for prediction and then evaluate the model based on the absolute error between the prediction result of the test data set and the real space thickness obtained manually.

[0014] According to a further technical solution, the spatial thickness is obtained by manually calibrating the spatial thickness of the polystyrene microsphere sample.

[0015] According to a further technical solution, the machine learning model is a linear regression model, a random forest model, an SVM model, or a ridge regression model.

[0016] The beneficial effects of the present invention are:

[0017] (1) The single-amplitude phase based on machine learning implemented by the present invention Figure 3 3D reconstruction innovatively combines the machine learning algorithm in the computer field with the 3D morphology reconstruction of the phase body, achieving a relatively accurate 3D morphology of the sample without knowing the refractive index of the sample;

[0018] (2) The present invention requires less data to be collected and is easy to store. Compared with traditional phase volume three-dimensional reconstruction, the present invention does not need to collect multiple phase images, but only needs a single phase image to achieve three-dimensional morphological reconstruction. It does not require a complex optical path system to collect multiple phase images in a specific direction of the sample, which is easier to implement and store data;

[0019] (3) The present invention takes about 0.2 seconds to reconstruct 256*256 (pixel) phase image data, which has a huge advantage in terms of time consumption compared to traditional phase volume 3D morphology reconstruction algorithms;

[0020] (4) The present invention performs feature engineering processing on the optical experimental data, so that the contour of the three-dimensional reconstruction is smoother and the noise is smaller. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the 3D reconstruction method based on a single phase image according to the present invention;

[0022] Figure 2 This is the quantitative phase imaging image of the red blood cells of the present invention;

[0023] Figure 3 The present invention Figure 2 Gradient map processed by Laplace gradient operator;

[0024] Figure 4 The present invention Figure 3 The gradient curve of the pixel y-axis midline;

[0025] Figure 5 This is the three-dimensional reconstructed contour morphology of the red blood cells described in the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0027] like Figure 1 As shown, a 3D reconstruction method based on a single phase image specifically includes the following steps:

[0028] Step (1): edge extraction, normalization and feature engineering of the sample phase map

[0029] Polystyrene microsphere samples whose environmental fluids were sodium chloride solution, ultrapure water, and Olympus immersion oil, and red blood cell samples whose environmental fluid was physiological saline, were prepared. Single phase images of the four samples were obtained using Biophase's quantitative phase microscopy technology.

[0030] The Laplace operator is used to extract the edges of the obtained phase image. Specifically, the gradient of the phase image is calculated on the same row or column of pixels to obtain the gradient curve of each pixel row or column, and the jump points on the gradient curve are used as the boundary points of the sample.

[0031] Based on the above boundary points, the experimental phase value range of the sample can be determined. Due to the influence of the experimental environment (incident light size, uniform distribution of environmental liquid, impurities on the slide), the experimental phase values ​​obtained are not on the same standard.

[0032]

[0033] Where: φ(x,y) represents the phase distribution of the sample, represents the wave number, n c (x, y, z) represents the refractive index inside the sample, n m represents the refractive index of the liquid in which the sample is located, h(x,y) represents the spatial thickness of the sample, and (x,y) represents the pixel coordinates.

[0034] According to formula (1), the phase value is the coupling of the object's refractive index and spatial thickness. In fact, the refractive index difference at the object boundary cannot be 0. Theoretically, only when the spatial thickness of the object boundary is 0, the corresponding phase value is also 0. Assuming that the phase value of the sample boundary point is 0, the phase map is normalized based on this standard to obtain a phase value distribution map based on a unified standard.

[0035] Feature extraction is performed on the phase value distribution map to obtain the phase gradient value. The specific operation of feature extraction is: first-order gradient processing and second-order gradient processing of the phase value in the same row and column of the phase map on the pixel are performed to increase the dimension of the subsequent data set.

[0036] Step (2): Construct machine learning training data set and test data set

[0037] The training dataset consists of single phase images of polystyrene microsphere samples with sizes of 4, 7, and 8 μm after being treated in Olympus immersion oil, sodium chloride solutions of different concentrations, and ultrapure water. The features of the training dataset include the phase value, pixel coordinates, incident light wavelength, refractive index of the ambient liquid, and phase gradient value obtained from the sample optical experiment. The label of the training dataset is spatial thickness. Spatial thickness is the manual calibration of the spatial thickness of the polystyrene microsphere samples.

[0038] The test set data are single phase image data of polystyrene microsphere samples and red blood cell samples that do not appear in the training data set.

[0039] Step (3) constructs a machine learning model, imports the training data set of step (2) into the machine learning model for training, uses the test data set for prediction, performs model evaluation, and obtains the three-dimensional structural morphology of the substructure biological cell.

[0040] The machine learning models include linear regression model, random forest model, SVM (Support Vector Machine) model, ridge regression model, etc.

[0041] The model is evaluated as the absolute error between the predicted results and the manually obtained real-space thickness of the test dataset.

[0042] Example

[0043] The training process of this embodiment is:

[0044] In this embodiment, the training data set comes from 80 single phase images of polystyrene microsphere samples in different environmental liquids, and the pixel size of a single phase image is 256*256; in this embodiment, the machine learning model is a random forest model, the training loss value is less than 0.001, and the training time is 100 minutes.

[0045] The testing process of this embodiment is as follows:

[0046] The test data set sample of this embodiment is: red blood cells whose environmental fluid is physiological saline. The pixel size of a single phase image of the test data set sample is 96*96, and the predicted reconstruction process is about 0.2s.

[0047] In step (1) of this embodiment, the sample phase imaging is as follows Figure 2 As shown in Figure 2, the sample phase image obtained by BioPhase quantitative phase microscopy imaging; Figure 2 The gradient of the red blood cell phase image extracted by the Laplace operator gradient in step (1) is as follows Figure 3 shown.

[0048] The gradient curve in the edge extraction process of step (1) of this embodiment is as follows Figure 4 As shown, Figure 4 for Figure 3 The gradient curve of the pixel y-axis centerline.

[0049] In step (3) of this embodiment, the three-dimensional reconstruction result of the test data set sample is obtained as follows: Figure 5 As shown, the basic morphology of the red blood cells is basically consistent. The minimum height of the reconstructed red blood cell outline in the direction of incident light is 0.61μm, the maximum height is 1.49μm, and the average length of the sample perpendicular to the incident light direction is 8.10μm.

[0050] The embodiments described are preferred implementations of the present invention, but the present invention is not limited to the above implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention are within the scope of protection of the present invention.

Claims

1. A 3D reconstruction method based on a single phase image, characterized by: Constructing a machine learning model, importing a training data set into the machine learning model for training, and using a test data set to predict and obtain the three-dimensional structural morphology of the substructured biological cells; The training data set consists of single phase images of polystyrene microsphere samples after being treated in Olympus immersion oil, sodium chloride solutions of different concentrations, and ultrapure water; the test data set consists of single phase image data of polystyrene microsphere samples and red blood cell samples that do not appear in the training data set; The processed single phase image is obtained by edge extraction, normalization and feature engineering processing; the feature engineering processing is to extract features from the phase value distribution map to obtain a phase gradient value; The features of the training data set include phase values, pixel coordinates, incident light wavelength, refractive index of the ambient liquid, and phase gradient values ​​obtained from a sample optical experiment, and the label of the training data set is spatial thickness.

2. The 3D reconstruction method based on a single phase image according to claim 1, characterized in that: The edge extraction is specifically as follows: performing gradient calculation on the same row or column of pixels in a single phase image to obtain a gradient curve for each pixel row or column, and using the transition points on the gradient curve as sample boundary points.

3. The 3D reconstruction method based on a single phase image according to claim 2, characterized in that: Assuming that the phase value of the sample boundary point is 0, the phase map is normalized based on this standard to obtain a phase value distribution map based on a unified standard.

4. The 3D reconstruction method based on a single phase image according to claim 1, characterized in that: After prediction using the test dataset, the model is evaluated based on the absolute error between the prediction results of the test dataset and the artificially obtained real space thickness.

5. The 3D reconstruction method based on a single phase image according to claim 4, characterized in that: The spatial thickness is obtained by manually calibrating the spatial thickness of a polystyrene microsphere sample.

6. The 3D reconstruction method based on a single phase image according to claim 1, characterized in that: The machine learning model is a linear regression model, a random forest model, a SVM model, or a ridge regression model.

Images