Microscopic light field reconstruction method and device based on implicit expression and electronic equipment

Abstract

The invention provides a microscopic light field reconstruction method and device based on implicit expression and electronic equipment, and relates to the technical field of microscopic imaging, and the method comprises the steps: distributing a corresponding feature vector for each time-space coordinate based on a hash function, each feature vector being a vector stored in a hash table; inputting feature vectors corresponding to the space-time coordinates in the hash table into a multilayer perceptron module to obtain a reconstructed initial refractive index intensity graph; determining loss information based on the initial refractive index intensity map and a light field image stack of the biological sample; and performing iterative adjustment on the feature vector and parameters of the multi-layer perceptron module based on the loss information, and obtaining a reconstructed target refractive index intensity graph of the biological sample based on the feature vector after iterative adjustment and the multi-layer perceptron module after iterative adjustment. According to the method, implicit expression is carried out through the feature vectors, compression of redundant information in the microscopic light field is achieved, the resource space occupied in the reconstruction process is small, and the reconstruction efficiency of the microscopic light field is improved.

Description

Technical Field

[0001] The present invention relates to the field of microscopic imaging technology, and in particular to a microscopic light field reconstruction method, device and electronic device based on implicit expression. Background Art

[0002] High-speed visualization of dynamic processes in large three-dimensional fields of view is crucial for many applications in neuroscience. Another important challenge in biology is to extract more spatio-temporal information from the target. To capture neuronal activities distributed throughout the brain, imaging methods capable of resolving these dynamic processes in the millisecond and hundreds-of-micron ranges are required. An attractive candidate for high-speed three-dimensional imaging in biology is light field microscopy, because light field microscopy can instantaneously capture three-dimensional spatial information and can image biological samples on a large three-dimensional field of view in the range of hundreds of microns with a time resolution of milliseconds, which opens up new avenues for the further development of neurobiology.

[0003] In related technologies, microscopic light field reconstruction technology mainly uses wave optics to model the point spread function of the light field system, and recovers the three-dimensional structure of the sample from the light field image through a deconvolution method.

[0004] However, in the above related technologies, microscopic light field reconstruction by the deconvolution method requires a large amount of resource space, resulting in a long overall reconstruction time, thereby reducing the efficiency of microscopic light field reconstruction. Summary of the Invention

[0005] The present invention provides a microscopic light field reconstruction method, device and electronic device based on implicit expression to solve the defect of reducing the efficiency of microscopic light field reconstruction in the prior art.

[0006] The present invention provides a microscopic light field reconstruction method based on implicit expression, including the following steps.

[0007] Input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, so that the spatio-temporal hash joint encoding module assigns corresponding feature vectors to the spatio-temporal coordinates of each spatio-temporal position based on the hash function, and each of the feature vectors is a vector stored in the hash table; Input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module to obtain the initial refractive index intensity map reconstructed by the multi-layer perceptron module; Determine the loss information based on the initial refractive index intensity map and the light field image stack of the biological sample; Iteratively adjust the feature vectors and the parameters of the multi-layer perceptron module based on the loss information, and obtain the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module.

[0008] A microscopic light field reconstruction method based on implicit expression provided by the present invention, wherein the light field image stack is obtained by imaging the biological sample through a light field microscope; Determining loss information based on the initial refractive index intensity map and the light field image stack of the biological sample, including: Performing a Fourier transform on the initial refractive index intensity map through a forward projection module to obtain a first Fourier transform result, and performing a Fourier transform on the point spread function of the light field microscope to obtain a second Fourier transform result; Multiplying the first Fourier transform result and the second Fourier transform result, and performing an inverse Fourier transform on the multiplication result to obtain the forward projection of the initial refractive index intensity map in the light field microscopy system; Determining loss information based on the forward projection and the light field image stack of the biological sample.

[0009] A microscopic light field reconstruction method based on implicit expression provided by the present invention, wherein the loss information includes mean square error loss, edge detection loss, and regularization loss; Determining loss information based on the forward projection and the light field image stack of the biological sample, including: Rearranging the spatial pixels belonging to the same angle in the light field image stack to obtain four-dimensional light field data; Determining mean square error loss, edge detection loss, and regularization loss based on the four-dimensional light field data and the forward projection.

[0010] A microscopic light field reconstruction method based on implicit expression provided by the present invention, wherein obtaining the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vector and the iteratively adjusted multi-layer perceptron module includes: Inputting the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, and mapping the spatio-temporal coordinates of each spatio-temporal position to the corresponding iteratively adjusted feature vector in the hash table based on the hash function through the spatio-temporal hash joint encoding module; Inputting the iteratively adjusted feature vectors corresponding to the spatio-temporal coordinates in the hash table into the iteratively adjusted multi-layer perceptron module to obtain a newly reconstructed refractive index intensity map output by the iteratively adjusted multi-layer perceptron module. Until a preset convergence condition is reached, the refractive index intensity map output by the iteratively adjusted multi-layer perceptron module in the last iteration is determined as the target refractive index intensity map.

[0011] A microscopic light field reconstruction method based on implicit expression provided by the present invention, wherein the target refractive index intensity map includes a three-dimensional target refractive index intensity map and / or a four-dimensional target refractive index intensity map.

[0012] A microscopic light field reconstruction method based on implicit expression provided by the present invention, the multi-layer perceptron module includes a first neuron layer and a second neuron layer; The step of inputting the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module to obtain the initially reconstructed refractive index intensity map output by the multi-layer perceptron module includes: Inputting the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module, and decoding each of the feature vectors through the first neuron layer and the second neuron layer to obtain the initially refractive index intensity map.

[0013] The present invention also provides a microscopic light field reconstruction device based on implicit expression, including: An encoding unit, configured to input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, so as to allocate corresponding feature vectors to the spatio-temporal coordinates of each spatio-temporal position based on a hash function through the spatio-temporal hash joint encoding module, and each of the feature vectors is a vector stored in the hash table; A reconstruction unit, configured to input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module to obtain the initially reconstructed refractive index intensity map output by the multi-layer perceptron module; A determination unit, configured to determine loss information based on the initially refractive index intensity map and the light field image stack of the biological sample; An adjustment unit, configured to iteratively adjust the feature vectors and the parameters of the multi-layer perceptron module based on the loss information, and obtain the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the microscopic light field reconstruction method based on implicit expression as described in any one of the above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the microscopic light field reconstruction method based on implicit expression as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the microscopic light field reconstruction method based on implicit expression as described in any one of the above is implemented.

[0017] The microscopic light field reconstruction method, device, and electronic device provided by the present invention based on implicit representation input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module. Through the spatio-temporal hash joint encoding module, the spatio-temporal coordinates of each spatio-temporal position are mapped to the corresponding feature vectors in the hash table based on the hash function. The feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table are input into the multi-layer perceptron module to obtain the initially reconstructed refractive index intensity map. Then, based on the initially reconstructed refractive index intensity map and the light field image stack of the biological sample, the loss information is determined. Based on the loss information, the feature vectors and the parameters of the multi-layer perceptron module are iteratively adjusted. Finally, based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the reconstructed biological sample is obtained. It can be seen that the present invention can map the spatio-temporal coordinates of each spatio-temporal position to the corresponding feature vectors in the hash table based on the hash function through the spatio-temporal hash joint encoding module, perform implicit representation through the feature vectors, realize the compression of redundant information in the microscopic light field, and decode and reconstruct each feature vector through the multi-layer perceptron module, so that the resource space occupied by the reconstruction process is small, thereby improving the efficiency of microscopic light field reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 FIG. 1 is one of the flow diagrams of the microscopic light field reconstruction method based on implicit representation provided by an embodiment of the present invention.

[0020] Figure 2 FIG. 2 is another flow diagram of the microscopic light field reconstruction method based on implicit representation provided by an embodiment of the present invention.

[0021] Figure 3 FIG. 3 is yet another flow diagram of the microscopic light field reconstruction method based on implicit representation provided by an embodiment of the present invention.

[0022] Figure 4 FIG. 4 is the overall framework diagram of the microscopic light field reconstruction method based on implicit representation provided by an embodiment of the present invention.

[0023] Figure 5 FIG. 5 is a schematic diagram of the microscopic light field reconstruction system based on implicit representation provided by an embodiment of the present invention.

[0024] Figure 6 FIG. 6 is a schematic diagram of the structure of the microscopic light field reconstruction device based on implicit representation provided by an embodiment of the present invention.

[0025] Figure 7 It is a schematic diagram of the physical structure of the electronic device provided by the embodiment of the present invention. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0027] Although light field microscopy has made great improvements in terms of spatial resolution, signal-to-noise ratio, and image reconstruction artifacts, the application of light field microscopy in life sciences is still limited by the high computational requirements and complex iterative image reconstruction processes in traditional algorithms, especially in long-term recordings. In recent years, deep learning has become a powerful framework for three-dimensional image reconstruction. Aiming at the limitation of the slow reconstruction speed in the traditional iterative reconstruction algorithm of light field microscopy on the ability to completely extract dynamic spatio-temporal information in the sample, traditional deep learning reconstruction is based on training a convolutional neural network on a large dataset to learn the mapping from two-dimensional light field images to three-dimensional volume images. Although deep learning has significantly accelerated image reconstruction in many fields, due to the difficulty of obtaining high-quality real three-dimensional refractive index intensity maps in experiments, traditional deep learning methods still have limitations in practical applications. Neural fields are a deep learning framework that has gained popularity in computer vision and graphics. Current research shows that neural fields can learn high-quality representations of complex scenes from sparse datasets without any external training datasets.

[0028] Against this background, the present invention proposes a microscopic light field reconstruction method based on implicit representation. The spatio-temporal coordinates of each spatio-temporal position are mapped to the corresponding feature vectors in the hash table based on the hash function through the spatio-temporal hash joint encoding module, and implicit representation is achieved through the feature vectors, realizing the compression of redundant information in the microscopic light field. The multilayer perceptron module decodes and reconstructs each feature vector, making the resource space occupied by the reconstruction process smaller, thereby improving the efficiency of microscopic light field reconstruction, and enabling instant encoding of long-time captured massive microscopic data with a high compression ratio.

[0029] The following Figures 1-5 describes the microscopic light field reconstruction method based on implicit representation of the present invention. The execution subject of the microscopic light field reconstruction method based on implicit representation can be an electronic device such as a terminal, a tablet computer, or a computer, or it can be a microscopic light field reconstruction device based on implicit representation provided in the electronic device. The microscopic light field reconstruction device based on implicit representation can be implemented through software, hardware, or a combination of both.

[0030] Figure 1 It is one of the flow diagrams of the microscopic light field reconstruction method based on implicit expression provided by an embodiment of the present invention. As Figure 1 shown, the microscopic light field reconstruction method based on implicit expression includes the following steps: Step 101: Input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, so as to allocate corresponding feature vectors to the spatio-temporal coordinates of each spatio-temporal position based on the hash function by the spatio-temporal hash joint encoding module, and each of the feature vectors is a vector stored in the hash table.

[0031] Among them, the spatio-temporal coordinates of each spatio-temporal position are set based on requirements. For example, if it is necessary to reconstruct the four-dimensional data of the frame, then the spatio-temporal coordinates of each spatio-temporal position are determined based on the four-dimensional data. For example, the spatio-temporal coordinate (100, 200, 10, 50) means that at the 50th frame, at the position with a depth of 10, the pixel coordinates are (100, 200).

[0032] Exemplarily, input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, and map the spatio-temporal coordinates of each spatio-temporal position to the corresponding feature vectors in the hash table based on the hash function by the spatio-temporal hash joint encoding module. Among them, multiple feature vectors are pre-initialized in the hash table, the hash function is the index that maps the spatio-temporal coordinates to the hash table, and each spatio-temporal coordinate corresponds to a feature vector respectively.

[0033] Step 102: Input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module, and obtain the reconstructed initial refractive index intensity map output by the multi-layer perceptron module.

[0034] Exemplarily, when obtaining the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table, input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module, map the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position to the corresponding refractive index intensity values through the multi-layer perceptron module, and finally output the reconstructed initial refractive index intensity map. That is, the multi-layer perceptron module provides a continuous representation of the initial refractive index intensity map. The initial refractive index intensity map here can be a three-dimensional initial refractive index intensity map representing spatial information, or a four-dimensional initial refractive index intensity map representing spatial information and time information.

[0035] Step 103: Determine the loss information based on the initial refractive index intensity map and the light field image stack of the biological sample.

[0036] Among them, the light field image stack is obtained by imaging the biological sample with a light field microscope.

[0037] Exemplarily, when obtaining the reconstructed initial refractive index intensity map output by the multi-layer perceptron module, loss information is calculated based on the initial refractive index intensity map and the light field image stack of the biological sample.

[0038] Step 104: Iteratively adjust the feature vectors and the parameters of the multi-layer perceptron module based on the loss information, and obtain the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module.

[0039] Exemplarily, when obtaining the loss information, the feature vectors and the parameters of the multi-layer perceptron module are iteratively adjusted by means of gradient backpropagation, that is, the feature vectors and the parameters of the multi-layer perceptron module are updated until the convergence condition is reached, and finally, based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the reconstructed biological sample is obtained.

[0040] It should be noted that the present invention can also be applied to any hardware device for measuring the refractive index of a sample from multiple angles in the field of microscopic observation. For example, an intensity diffraction tomography system uses a programmable illumination array to irradiate a biological sample from different angles, and a camera records the intensity measurements of the scattered light. The present invention can reconstruct a high-quality three-dimensional or four-dimensional refractive index intensity map from only intensity and limited-angle measurements.

[0041] The microscopic light field reconstruction method based on implicit representation provided by the present invention inputs the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module. The spatio-temporal hash joint encoding module maps the spatio-temporal coordinates of each spatio-temporal position to the corresponding feature vectors in the hash table based on a hash function. The feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table are input into the multi-layer perceptron module to obtain the reconstructed initial refractive index intensity map. Then, based on the initial refractive index intensity map and the light field image stack of the biological sample, loss information is determined. The feature vectors and the parameters of the multi-layer perceptron module are iteratively adjusted based on the loss information. Finally, based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the reconstructed biological sample is obtained. It can be seen that the present invention can map the spatio-temporal coordinates of each spatio-temporal position to the corresponding feature vectors in the hash table based on a hash function through the spatio-temporal hash joint encoding module, perform implicit representation through the feature vectors, compress the redundant information in the microscopic light field, and decode and reconstruct each feature vector through the multi-layer perceptron module, so that the resource space occupied by the reconstruction process is small, thereby improving the efficiency of microscopic light field reconstruction.

[0042] In one embodiment, Figure 2 is the second schematic flowchart of the microscopic light field reconstruction method based on implicit representation provided by the embodiment of the present invention. As Figure 2As shown, the above step 103 determines loss information based on the initial refractive index intensity map and the light field image stack of the biological sample, which can be specifically implemented through the following steps: Step 1031: Perform Fourier transform on the initial refractive index intensity map through a forward projection module to obtain a first Fourier transform result, and perform Fourier transform on the point spread function of the light field microscope to obtain a second Fourier transform result.

[0043] Exemplarily, a point light source is photographed by a light field microscope, and the point spread function (PSF) of the light field microscope is calculated based on the image of the point light source, or the point spread function of the light field microscope is calculated according to the Fresnel diffraction approximation. When the reconstructed initial refractive index intensity map output by the multi-layer perceptron module is obtained, Fourier transform is performed on the initial refractive index intensity map to obtain a first Fourier transform result, and Fourier transform is performed on the point spread function of the light field microscope to obtain a second Fourier transform result. Among them, the specific method of Fourier transform can refer to related technologies, and the present invention will not elaborate here.

[0044] Step 1032: Multiply the first Fourier transform result and the second Fourier transform result, and perform inverse Fourier transform on the multiplication result to obtain the forward projection of the initial refractive index intensity map in the light field microscopy system.

[0045] Exemplarily, when the first Fourier transform result and the second Fourier transform result are obtained, the first Fourier transform result and the second Fourier transform result are multiplied to obtain a multiplication result, and inverse Fourier transform is performed on the multiplication result to obtain the forward projection of the initial refractive index intensity map in the light field microscopy system. Among them, the specific method of inverse Fourier transform can refer to related technologies, and the present invention will not elaborate here.

[0046] Step 1033: Determine loss information based on the forward projection and the light field image stack of the biological sample.

[0047] Exemplarily, a biological sample is imaged by a light field microscope. The specific process is as follows: An objective lens with a certain magnification is used, and a microlens array is placed on the imaging plane to modulate the optical path, converting the wide-field acquisition method into a light field acquisition method, and then imaging is performed to obtain a light field image stack of the biological sample. The light field image stack is a set of light field images at multiple different angles. When obtaining the forward projection of the initial refractive index intensity map in the light field microscopy system, the forward projection and the light field image stack of the biological sample are input into the rendering loss function module. The rendering loss function module calculates loss information based on the forward projection and the light field image stack of the biological sample, and then iteratively adjusts the feature vectors of the spatio-temporal hashing joint encoding module and the parameters of the multi-layer perceptron module based on the loss information. Based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the reconstructed biological sample is obtained.

[0048] In this embodiment, the initial refractive index intensity map is Fourier-transformed by the forward projection module, and the point spread function of the light field microscope is Fourier-transformed. The first Fourier transform result and the second Fourier transform result are multiplied, and the multiplied result is inverse Fourier-transformed to obtain the forward projection of the initial refractive index intensity map in the light field microscopy system. Finally, loss information is determined based on the forward projection and the light field image stack of the biological sample. The loss information is used to measure the difference between the forward projection and the light field image stack, facilitating subsequent iterative adjustment of the feature vectors of the spatio-temporal hashing joint encoding module and the parameters of the multi-layer perceptron module to improve the accuracy of microscopic light field reconstruction.

[0049] In one embodiment, the loss information includes mean square error loss, edge detection loss, and regularization loss; the above step 1033 determines loss information based on the forward projection and the light field image stack of the biological sample, and can be specifically implemented in the following manner: The spatial pixels belonging to the same angle in the light field image stack are rearranged to obtain four-dimensional light field data; based on the four-dimensional light field data and the forward projection, mean square error loss, edge detection loss, and regularization loss are determined.

[0050] Exemplarily, a light field image stack is usually composed of multiple light field images at different angles. Each light field image contains two-dimensional spatial information (x, y) and one-dimensional angular information u. Therefore, the light field image stack itself is a four-dimensional data (x, y, u, t). In the light field image stack, all spatial pixels (x, y) belonging to the same angle u are selected. These spatial pixels are extracted to form a two-dimensional spatial image. The spatial images corresponding to all angles are rearranged according to the angle u. Each angle corresponds to a two-dimensional spatial image. After rearrangement, a three-dimensional data (x, y, u) is formed. On the basis of the rearrangement, the time dimension is further introduced to construct four-dimensional light field data, including two-dimensional spatial information (x, y), which represents the position of the light ray on the imaging plane, one-dimensional angular information (u), which represents the direction of the light ray, and one-dimensional time information (t), which represents the change of the light field data at different time points. The spatial information, angular information, and time information are combined to form four-dimensional light field data (x, y, u, t).

[0051] Calculate the mean squared error loss MSE based on the following formula (1): where, = , representing the total number of pixels. The two-dimensional spatial information (x, y) represents the position of a pixel. represents the total number of pixels in the width direction. represents the total number of pixels in the height direction. represents the intensity value of the four-dimensional light field data at (x, y). represents the intensity value of the forward projection at (x, y).

[0052] Furthermore, the first edge feature corresponding to the four-dimensional light field data is extracted through an edge detection operator, and the second edge feature corresponding to the forward projection is extracted. The edge detection loss is calculated based on the first edge feature and the second edge feature. The edge detection loss is used to measure the difference in edge information between the image corresponding to the four-dimensional light field data and the forward projection. Among them, the edge detection operator includes Sobel, Canny, or Laplacian, and the present invention does not limit this.

[0053] Furthermore, calculate the regularization loss based on the following formula (2) and formula (3): where, represents the difference between the intensity value of the four-dimensional light field data at (x, y) and the intensity value of the forward projection at (x, y). represents at The rate of change in the represents the rate of change in direction, represents the total variation regularization, and the regularization loss is represented by the total variation regularization.

[0054] It should be noted that the mean squared error loss is used to reconstruct the low-frequency components of the refractive index intensity map, that is, the main part. The edge detection loss is used to recover the high-frequency details of the refractive index intensity map, and the regularization loss is used to reduce the noise and outliers of the refractive index intensity map.

[0055] In this embodiment, the spatial pixels belonging to the same angle in the light field image stack are rearranged to obtain four-dimensional light field data. Based on the four-dimensional light field data and forward projection, the mean squared error loss, the edge detection loss, and the regularization loss are determined. Furthermore, based on the mean squared error loss, the edge detection loss, and the regularization loss, the feature vector of the spatio-temporal hashing joint encoding module and the parameters of the multi-layer perceptron module are iteratively adjusted to improve the accuracy of microscopic light field reconstruction.

[0056] In one embodiment, Figure 3 is the third flowchart of the microscopic light field reconstruction method based on implicit expression provided by the embodiments of the present invention. As Figure 3 shown, in the above step 104, based on the iteratively adjusted feature vector and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the biological sample to be reconstructed is obtained. Specifically, it can be implemented through the following steps: Step 301: Input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hashing joint encoding module. Through the spatio-temporal hashing joint encoding module, the spatio-temporal coordinates of each spatio-temporal position are mapped to the corresponding iteratively adjusted feature vector in the hash table based on the hash function.

[0057] Exemplarily, after the first iterative adjustment of the feature vector of the spatio-temporal hashing joint encoding module, the spatio-temporal coordinates of each spatio-temporal position are input into the spatio-temporal hashing joint encoding module. Through the spatio-temporal hashing joint encoding module, the spatio-temporal coordinates of each spatio-temporal position are mapped to the corresponding feature vector after the first iterative adjustment in the hash table based on the hash function.

[0058] Step 302: Input the iteratively adjusted feature vectors corresponding to the spatio-temporal coordinates in the hash table into the iteratively adjusted multi-layer perceptron module to obtain a newly reconstructed refractive index intensity map output by the iteratively adjusted multi-layer perceptron module. Until the preset convergence condition is reached, the refractive index intensity map output by the multi-layer perceptron module after the last iterative adjustment is determined as the target refractive index intensity map.

[0059] Exemplarily, after the parameters of the multi-layer perceptron module are iteratively adjusted for the first time, the feature vectors after the first iterative adjustment corresponding to the spatio-temporal coordinates at each spatio-temporal position in the hash table are input into the multi-layer perceptron module after the first iterative adjustment, and a reconstructed new refractive index intensity map output by the multi-layer perceptron module after the first iterative adjustment is obtained. Based on the new refractive index intensity map and the light field image stack of the biological sample, new loss information is determined. Based on the loss information, a second iteration is performed, and so on, until a preset convergence condition is reached. Finally, the refractive index intensity map output by the multi-layer perceptron module after the last iterative adjustment is determined as the target refractive index intensity map of the biological sample, realizing the reconstruction of the biological sample in the microscopic light field. Here, the target refractive index intensity map can be a three-dimensional target refractive index intensity map including spatial information, or a four-dimensional target refractive index intensity map including spatial information and time information. The present invention does not limit this.

[0060] In this embodiment, the spatio-temporal coordinates at each spatio-temporal position are input into the spatio-temporal hash joint encoding module. Through the spatio-temporal hash joint encoding module, the spatio-temporal coordinates at each spatio-temporal position are mapped to the corresponding feature vectors after iterative adjustment in the hash table based on a hash function, and the feature vectors after iterative adjustment corresponding to the spatio-temporal coordinates at each spatio-temporal position in the hash table are input into the multi-layer perceptron module after iterative adjustment, and a reconstructed new refractive index intensity map output by the multi-layer perceptron module after iterative adjustment is obtained. Until a preset convergence condition is reached, the refractive index intensity map output by the multi-layer perceptron module after the last iterative adjustment is determined as the target refractive index intensity map, that is, the accuracy of the target refractive index intensity map reconstruction is improved by means of multiple iterations.

[0061] In one embodiment, the multi-layer perceptron module includes a first neuron layer and a second neuron layer. In step 102 above, the feature vectors corresponding to the spatio-temporal coordinates at each spatio-temporal position in the hash table are input into the multi-layer perceptron module, and an initial refractive index intensity map reconstructed by the multi-layer perceptron module is obtained. Specifically, it can be realized in the following way: The feature vectors corresponding to the spatio-temporal coordinates at each spatio-temporal position in the hash table are input into the multi-layer perceptron module, and the first neuron layer and the second neuron layer decode each of the feature vectors to obtain the initial refractive index intensity map.

[0062] Among them, the activation function of the first neuron layer includes a rectified linear unit function, and the activation function of the second neuron layer includes a leaky rectified linear unit function.

[0063] Exemplarily, the multi-layer perceptron module consists of two neuron layers, namely the first neuron layer and the second neuron layer. Both the first neuron layer and the second neuron layer include 64 neurons. The activation function of the first neuron layer is the Rectified Linear Unit (ReLU), and the activation function of the second neuron layer is the Leaky Rectified Linear Unit (Leaky ReLU). The weight of each neuron can be of the 32-bit floating-point number type or the 16-bit floating-point number type, which can accelerate the training speed of the network.

[0064] It should be noted that the multi-layer perceptron module can also include three or more neuron layers, and the present invention does not limit this.

[0065] In this embodiment, the multi-layer perceptron module includes two neuron layers, which can not only learn the non-linear mapping between the input features and the refractive index intensity through the hidden layer, but also avoid overfitting and computational redundancy caused by an overly deep network, so as to efficiently and flexibly generate the refractive index intensity map. Figure 4 It is the overall framework diagram of the microscopic light field reconstruction method based on implicit representation provided by the embodiment of the present invention. As Figure 4 shown, the spatio-temporal coordinates (x, y, z, t) of each spatio-temporal position are input into the spatio-temporal hash joint encoding module, where (x, y) represents the coordinates of the pixel points in the two-dimensional image, z represents the depth, and t represents the time information. Through the spatio-temporal hash joint encoding module, the spatio-temporal coordinates of each spatio-temporal position are mapped to the corresponding feature vectors in the hash table based on the hash function; the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table are input into the multi-layer perceptron module to obtain the reconstructed initial refractive index intensity map output by the multi-layer perceptron module; the initial refractive index intensity map is subjected to Fourier transform through the forward projection module to obtain the first Fourier transform result, and the point spread function of the light field microscope is subjected to Fourier transform to obtain the second Fourier transform result. The first Fourier transform result and the second Fourier transform result are multiplied, and the multiplied result is subjected to inverse Fourier transform to obtain the forward projection of the initial refractive index intensity map in the light field microscopic system. Based on the forward projection and the light field image stack of the biological sample, the loss information is determined, and the feature vectors of the spatio-temporal hash joint encoding module and the parameters of the multi-layer perceptron module are iteratively adjusted by means of gradient backpropagation, that is, the feature vectors and the parameters of the multi-layer perceptron module are updated until the convergence condition is reached. Finally, based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the reconstructed biological sample is obtained. It can be seen that by decoupling the representation from the explicit voxel grid, the microscopic light field reconstruction method based on implicit representation can be used to effectively store the refractive index intensity distribution map of large three-dimensional biological samples.

[0066] Figure 5 It is a schematic diagram of a microscopic light field reconstruction system based on implicit expression provided by an embodiment of the present invention. As Figure 5 shown, the microscopic light field reconstruction system based on implicit expression includes a spatio-temporal hash joint encoding module 100, a multi-layer perceptron module 200, a forward projection module 300, and a rendering loss function module 400. Among them, the spatio-temporal hash joint encoding module 100 includes spatio-temporal coordinates 110, feature vectors 120, and a hash table 130 at each spatio-temporal position. Among them, the feature vectors 120 are stored in the hash table 130. The spatio-temporal coordinates 110 at each spatio-temporal position are input into the spatio-temporal hash joint encoding module 100, and the spatio-temporal coordinates 110 at each spatio-temporal position are mapped to the corresponding feature vectors in the hash table 130 based on a hash function through the spatio-temporal hash joint encoding module 100, so as to compress redundant information (unorganized cell regions or regions with very little dynamic change in time series) in the microscopic light field; in addition, the spatio-temporal coordinates 110 of the present invention can be equally spaced sampling matching the point spread function, or random sampling or more dense sampling methods, which are not limited herein; the multi-layer perceptron module 200 includes a first neuron layer 210 and a second neuron layer 220. The activation function of the first neuron layer 210 includes a rectified linear unit function, and the activation function of the second neuron layer 220 includes a leaky rectified linear unit function. The feature vectors corresponding to the spatio-temporal coordinates 110 at each spatio-temporal position in the hash table are input into the multi-layer perceptron module 200 to obtain a reconstructed refractive index intensity map 310 output by the multi-layer perceptron module 200; the forward projection module 300 is composed of a refractive index intensity map 310 and a point spread function 320. The point spread function 320 and the refractive index intensity map 310 of the biological sample calculated by the multi-layer perceptron module 200 are respectively Fourier-transformed and multiplied, and the multiplied result is Fourier-inverted again to obtain the forward projection of the reconstructed refractive index intensity map in the light field microscopy system; the rendering loss function module 400 is composed of a mean square error loss 410, an edge detection loss 420, a regularization loss 430, and a stack of real-shot light field images 440. The loss information between the forward projection and the stack of light field images 440 is calculated by the mean square error loss 410, the edge detection loss 420, and the regularization loss 430 respectively, and the feature vectors of the spatio-temporal hash joint encoding module and the parameters of the multi-layer perceptron module are iteratively adjusted by means of gradient backpropagation. Among them, the mean square error loss is used to reconstruct the low-frequency components of the refractive index intensity map, that is, the main part, the edge detection loss is used to restore the high-frequency details of the refractive index intensity map, and the regularization loss is used to reduce the noise and outliers of the refractive index intensity map.

[0067] The microscopic light field reconstruction system based on implicit representation provided by the present invention inputs the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module. Through the spatio-temporal hash joint encoding module, the spatio-temporal coordinates of each spatio-temporal position are mapped to the corresponding feature vectors in the hash table based on the hash function. The feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table are input into the multi-layer perceptron module to obtain the reconstructed initial refractive index intensity map. Then, based on the initial refractive index intensity map and the light field image stack of the biological sample, the loss information is determined. Based on the loss information, the feature vectors and the parameters of the multi-layer perceptron module are iteratively adjusted. Finally, based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the reconstructed biological sample is obtained. It can be seen that the present invention can map the spatio-temporal coordinates of each spatio-temporal position to the corresponding feature vectors in the hash table based on the hash function through the spatio-temporal hash joint encoding module, perform implicit representation through the feature vectors, realize the compression of redundant information in the microscopic light field, and decode and reconstruct each feature vector through the multi-layer perceptron module, so that the resource space occupied by the reconstruction process is small, thereby improving the efficiency of microscopic light field reconstruction, and thus enabling instant encoding of long-time captured massive microscopic data with a high compression ratio; in addition, the present invention determines the loss information based on the initial refractive index intensity map and the light field image stack of the biological sample, without a supervised training process, thereby expanding the application of the light field microscope in in-vivo imaging.

[0068] The microscopic light field reconstruction device based on implicit representation provided by the present invention will be described below. The microscopic light field reconstruction device based on implicit representation described below can be mutually referred to corresponding to the microscopic light field reconstruction method described above.

[0069] Figure 6 is a schematic structural diagram of the microscopic light field reconstruction device based on implicit representation provided by an embodiment of the present invention, as Figure 6 shown, the microscopic light field reconstruction device 600 based on implicit representation includes an encoding unit 601, a reconstruction unit 602, a determination unit 603, and an adjustment unit 604; wherein: The encoding unit 601 is configured to input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, so as to allocate corresponding feature vectors for the spatio-temporal coordinates of each spatio-temporal position based on the hash function through the spatio-temporal hash joint encoding module, and each of the feature vectors is a vector stored in the hash table; The reconstruction unit 602 is configured to input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module to obtain the reconstructed initial refractive index intensity map output by the multi-layer perceptron module; The determination unit 603 is configured to determine loss information based on the initial refractive index intensity map and the light field image stack of the biological sample; An adjustment unit 604 is configured to iteratively adjust the feature vector and the parameters of the multi-layer perceptron module based on the loss information, and obtain the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vector and the iteratively adjusted multi-layer perceptron module.

[0070] The microscopic light field reconstruction device based on implicit representation provided by the present invention inputs the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module. The spatio-temporal coordinates of each spatio-temporal position are mapped to the corresponding feature vector in the hash table by the spatio-temporal hash joint encoding module based on the hash function. The feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table are input into the multi-layer perceptron module to obtain the initial reconstructed refractive index intensity map. Then, based on the initial refractive index intensity map and the light field image stack of the biological sample, the loss information is determined. The feature vector and the parameters of the multi-layer perceptron module are iteratively adjusted based on the loss information. Finally, based on the iteratively adjusted feature vector and the iteratively adjusted multi-layer perceptron module, the target refractive index intensity map of the reconstructed biological sample is obtained. It can be seen that the present invention can map the spatio-temporal coordinates of each spatio-temporal position to the corresponding feature vector in the hash table by the spatio-temporal hash joint encoding module, perform implicit representation through the feature vector, compress the redundant information in the microscopic light field, and decode and reconstruct each feature vector through the multi-layer perceptron module, so that the resource space occupied by the reconstruction process is small, thereby improving the efficiency of microscopic light field reconstruction.

[0071] Based on any of the above embodiments, the light field image stack is obtained by imaging the biological sample with a light field microscope; the determining unit 603 is specifically configured to: Perform Fourier transform on the initial refractive index intensity map through the forward projection module to obtain the first Fourier transform result, and perform Fourier transform on the point spread function of the light field microscope to obtain the second Fourier transform result; Multiply the first Fourier transform result and the second Fourier transform result, and perform inverse Fourier transform on the multiplication result to obtain the forward projection of the initial refractive index intensity map in the light field microscopy system; Determine the loss information based on the forward projection and the light field image stack of the biological sample.

[0072] Based on any of the above embodiments, the loss information includes mean square error loss, edge detection loss, and regularization loss; the determining unit 603 is further specifically configured to: Rearrange the spatial pixels belonging to the same angle in the light field image stack to obtain four-dimensional light field data; Determine the mean square error loss, edge detection loss, and regularization loss based on the four-dimensional light field data and the forward projection.

[0073] Based on any of the above embodiments, the adjustment unit 604 is specifically configured to: Input the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, and map the spatio-temporal coordinates of each spatio-temporal position to the corresponding iteratively adjusted feature vectors in the hash table through the spatio-temporal hash joint encoding module based on the hash function; Input the iteratively adjusted feature vectors corresponding to the spatio-temporal coordinates in the hash table into the iteratively adjusted multi-layer perceptron module, obtain the reconstructed new refractive index intensity map output by the iteratively adjusted multi-layer perceptron module, and until the preset convergence condition is reached, determine the refractive index intensity map output by the multi-layer perceptron module in the last iteration adjustment as the target refractive index intensity map.

[0074] Based on any of the above embodiments, the target refractive index intensity map includes a three-dimensional target refractive index intensity map and / or a four-dimensional target refractive index intensity map.

[0075] Based on any of the above embodiments, the multi-layer perceptron module includes a first neuron layer and a second neuron layer, and the reconstruction unit 602 is specifically configured to: Input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module, and decode each of the feature vectors through the first neuron layer and the second neuron layer to obtain the initial refractive index intensity map.

[0076] Figure 7 It is a schematic physical structure diagram of an electronic device provided by an embodiment of the present invention. As Figure 7 shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 complete mutual communication through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute the microscopic light field reconstruction method based on implicit expression. The method includes: inputting the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module to allocate corresponding feature vectors for the spatio-temporal coordinates of each spatio-temporal position through the spatio-temporal hash joint encoding module based on the hash function, and each of the feature vectors is a vector stored in the hash table; Input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module to obtain the reconstructed initial refractive index intensity map output by the multi-layer perceptron module; Determine loss information based on the initial refractive index intensity map and the light field image stack of the biological sample; Iteratively adjust the feature vector and the parameters of the multi-layer perceptron module based on the loss information, and obtain the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vector and the iteratively adjusted multi-layer perceptron module.

[0077] In addition, when the logic instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0078] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for microscopic light field reconstruction based on implicit expression provided by the above-mentioned various methods. The method includes: inputting the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, so that the spatio-temporal hash joint encoding module assigns corresponding feature vectors to the spatio-temporal coordinates of each spatio-temporal position based on a hash function, and each of the feature vectors is a vector stored in a hash table; Input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into the multi-layer perceptron module to obtain the initial refractive index intensity map output by the multi-layer perceptron module; Determine loss information based on the initial refractive index intensity map and the light field image stack of the biological sample; Iteratively adjust the feature vector and the parameters of the multi-layer perceptron module based on the loss information, and obtain the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vector and the iteratively adjusted multi-layer perceptron module.

[0079] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the microscopic light field reconstruction method based on implicit expression provided by the above-mentioned various methods. The method includes: inputting the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, so that the spatio-temporal hash joint encoding module assigns corresponding feature vectors to the spatio-temporal coordinates of each spatio-temporal position based on a hash function, and each of the feature vectors is a vector stored in a hash table; Inputting the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into a multi-layer perceptron module to obtain an initial refractive index intensity map reconstructed by the multi-layer perceptron module; Determining loss information based on the initial refractive index intensity map and the light field image stack of the biological sample; Iteratively adjusting the feature vectors and the parameters of the multi-layer perceptron module based on the loss information, and obtaining a target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module.

[0080] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0081] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A microscopic light field reconstruction method based on implicit expression, characterized in that, Including: Inputting the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, so as to allocate corresponding feature vectors for the spatio-temporal coordinates of each spatio-temporal position based on a hash function by the spatio-temporal hash joint encoding module, and each of the feature vectors is a vector stored in a hash table; Inputting the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into a multi-layer perceptron module to obtain a reconstructed initial refractive index intensity map output by the multi-layer perceptron module; Determining loss information based on the initial refractive index intensity map and the light field image stack of the biological sample; Iteratively adjusting the feature vectors and the parameters of the multi-layer perceptron module based on the loss information, and obtaining a target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module.

2. The microscopic light field reconstruction method based on implicit expression according to claim 1, wherein The light field image stack is obtained by imaging the biological sample with a light field microscope; The determining loss information based on the initial refractive index intensity map and the light field image stack of the biological sample includes: Performing a Fourier transform on the initial refractive index intensity map through a forward projection module to obtain a first Fourier transform result, and performing a Fourier transform on the point spread function of the light field microscope to obtain a second Fourier transform result; Multiplying the first Fourier transform result and the second Fourier transform result, and performing an inverse Fourier transform on the multiplication result to obtain a forward projection of the initial refractive index intensity map in the light field microscopy system; Determining loss information based on the forward projection and the light field image stack of the biological sample.

3. The microscopic light field reconstruction method based on implicit expression according to claim 2, wherein The loss information includes mean square error loss, edge detection loss, and regularization loss; The determining loss information based on the forward projection and the light field image stack of the biological sample includes: Rearranging the spatial pixels belonging to the same angle in the light field image stack to obtain four-dimensional light field data; Determining mean square error loss, edge detection loss, and regularization loss based on the four-dimensional light field data and the forward projection.

4. The microscopic light field reconstruction method based on implicit expression according to claim 1, characterized in that The obtaining a target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module includes: Inputting the spatio-temporal coordinates of each spatio-temporal position into the spatio-temporal hash joint encoding module, and mapping the spatio-temporal coordinates of each spatio-temporal position to the iteratively adjusted feature vectors corresponding in the hash table based on the hash function by the spatio-temporal hash joint encoding module; Inputting the iteratively adjusted feature vectors corresponding to each spatio-temporal coordinate in the hash table into the iteratively adjusted multi-layer perceptron module to obtain a reconstructed new refractive index intensity map output by the iteratively adjusted multi-layer perceptron module, until a preset convergence condition is reached, and determining the refractive index intensity map output by the multi-layer perceptron module in the last iterative adjustment as the target refractive index intensity map.

5. The microscopic light field reconstruction method based on implicit expression according to claim 1, characterized in that The target refractive index intensity map includes a three-dimensional target refractive index intensity map and / or a four-dimensional target refractive index intensity map.

6. The method for microscopic light field reconstruction based on implicit expression according to any one of claims 1-5, characterized in that, The multi-layer perceptron module includes a first neuron layer and a second neuron layer; Inputting the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into a multi-layer perceptron module to obtain the reconstructed initial refractive index intensity map output by the multi-layer perceptron module, including: Inputting the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into a multi-layer perceptron module, and decoding each of the feature vectors through the first neuron layer and the second neuron layer to obtain the initial refractive index intensity map.

7. A microscopic light field reconstruction device based on implicit expression, characterized in that, Including: An encoding unit configured to input the spatio-temporal coordinates of each spatio-temporal position into a spatio-temporal hash joint encoding module, so as to allocate corresponding feature vectors for the spatio-temporal coordinates of each spatio-temporal position based on a hash function by the spatio-temporal hash joint encoding module, and each of the feature vectors is a vector stored in the hash table; A reconstruction unit configured to input the feature vectors corresponding to the spatio-temporal coordinates of each spatio-temporal position in the hash table into a multi-layer perceptron module to obtain the reconstructed initial refractive index intensity map output by the multi-layer perceptron module; A determination unit configured to determine loss information based on the initial refractive index intensity map and the light field image stack of the biological sample; An adjustment unit configured to iteratively adjust the feature vectors and the parameters of the multi-layer perceptron module based on the loss information, and obtain the target refractive index intensity map of the reconstructed biological sample based on the iteratively adjusted feature vectors and the iteratively adjusted multi-layer perceptron module.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, wherein, When the processor executes the computer program, it implements the implicit-expression-based microscopic light field reconstruction method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements the implicit-expression-based microscopic light field reconstruction method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the implicit-expression-based microscopic light field reconstruction method according to any one of claims 1 to 6.

Images