A Multi-color Synchronous Reconstruction Method for a Light Field Microscopy System

Abstract

The present invention discloses a multi-color synchronous reconstruction method for a light field microscopy system, belonging to the fields of computer vision and digital image processing. The method includes: A1: extracting spectral information from multi-wavelength fluorescence light field images and performing spectral correction according to the wavelength response function of the image acquisition system; A2: separating single-wavelength light field images and performing reconstruction one by one using the single-wavelength light field microscopy reconstruction algorithm; A3: coloring the single-wavelength reconstruction results and fusing all the reconstruction results according to the principle of linear superposition. The present invention interprets the synchronous multi-color light field reconstruction method as three sub-methods: spectral information extraction and correction, monochromatic reconstruction, and image coloring and fusion, and proposes a reliable multi-color reconstruction method that is easy to implement and expand, which is of great significance for promoting the research of neuroscience and the development of synchronous multi-color light field microscopy technology.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and digital image processing, and in particular to a method for multi-color synchronous reconstruction of a light field microscopy system. Background Art

[0002] Light field microscopy is an optical microscopy imaging technology that does not require scanning. By placing a microlens array at a specific position in a traditional optical microscope, three-dimensional volume information of the target can be quickly obtained at a single frame rate. Compared with other microscopy techniques that require scanning imaging, it has better temporal resolution, providing the possibility for the complete observation of high-speed life activities, especially neural activities. Currently, the vast majority of light field microscopy research only considers monochromatic light. However, with the increasing demand for multi-color fluorescence observation, the research and development of multi-color light field microscopy are imperative. According to the imaging method, multi-color light field microscopy can be classified into asynchronous multi-color imaging and synchronous multi-color imaging. Asynchronous multi-color imaging refers to the imaging process in which signal lights of different wavelengths do not appear simultaneously in the time domain. As long as the data of each monochromatic light is reconstructed and fused separately, a multi-color three-dimensional reconstruction result can be obtained. This method is relatively easy to implement and is the main multi-color light field microscopy method currently used, but it sacrifices temporal resolution. Synchronous multi-color imaging means that multiple wavelengths are imaging at the same time and in the same space, which is a research hotspot in recent years. However, due to high cost, difficulty in expansion, etc., it has not been more widely applied and is still in the early exploration stage and is not yet mature. Neither of them has achieved a reliable multi-color reconstruction method that is both easy to implement and easy to expand. Summary of the Invention

[0003] The purpose of the present invention is to solve the problems of high cost and difficulty in expansion of synchronous multi-color imaging in light field microscopy, and to provide a method for multi-color synchronous reconstruction of a light field microscopy system.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] A method for multi-color synchronous reconstruction of a light field microscopy system includes the following steps:

[0006] A1: Extract spectral information from the multi-wavelength fluorescence light field image according to the wavelength response function of the image acquisition system and perform correction;

[0007] A2: Separate the single-wavelength light field images according to the corrected spectral information, and perform reconstruction one by one using the single-wavelength light field microscopy reconstruction algorithm;

[0008] A3: Assign corresponding colors to the single-wavelength reconstruction results, and fuse all the reconstruction results according to the principle of linear superposition.

[0009] In some embodiments of the present invention, step A1 includes:

[0010] A1-1: Extract preliminary estimated spectral information from a multi-wavelength fluorescence light field image;

[0011] A1-2: Perform spectral correction on the spectral information.

[0012] In some embodiments of the present invention, in step A1-1, the method for extracting spectral information from a multi-wavelength fluorescence light field image includes the following steps:

[0013] A1-1-1: Using the RGB triple of each pixel in the image as input and combining with the wavelength response function of the image acquisition device, construct an optimization model for estimating the spectral information of each pixel, that is, the estimation model; A1-1-2: Traverse the estimation model in step A1-1-1 for each pixel position in the light field image to obtain preliminary spectral information.

[0014] In some embodiments of the present invention, the specific operation of step A1-1-1 is as follows:

[0015]

[0016] Among them, S ij represents the spectral information at the pixel coordinates (i, j), C ij represents the RGB triple value of the multi-color fluorescence light field image at the pixel coordinates (i, j), R T represents the matrix formed by the wavelength response functions of the RGB three channels of the image acquisition device, represents the square of the l2-norm, represents the spectral information at the pixel coordinates (i, j) estimated according to the optimization model, and argmin represents the variable value when the objective function f(x) takes the minimum value.

[0017] In some embodiments of the present invention, in step A1-2, the method for performing spectral correction on the spectral conversion result includes the following steps:

[0018] A1-2-1: Apply the spectral correction function to each pixel of each band of the preliminary estimated spectral information obtained in step A1-1, and then multiply by the wavelength-related weight to obtain the corrected spectral information; A1-2-2: Perform step A1-2-1 on the wavelength information of all pixels to achieve spectral correction.

[0019] In some embodiments of the present invention, the specific operation of step A1-2-1 is as follows:

[0020]

[0021]

[0022] Among them, V ij(k) represents the intensity information of wavelength k at the corrected pixel coordinates (i, j), and W(k) represents the weight value of wavelength k. represents the spectral information intensity of wavelength k at the pixel coordinates (i, j) initially estimated according to the optimization model, and r(x) represents the spectral correction function, whose function value is equal to the independent variable when the independent variable is non - negative, and zero otherwise.

[0023] In some embodiments of the present invention, step A2 includes the following steps:

[0024] A2 - 1: Select specific wavelengths from the corrected spectral information obtained in step A1 - 2 to form several single - wavelength light - field images;

[0025] A2 - 2: Use existing single - wavelength light - field microscopy algorithms to perform three - dimensional reconstruction one by one to obtain several single - wavelength reconstruction results.

[0026] In some embodiments of the present invention, step A3 includes the following steps:

[0027] A3 - 1: Assign colors to the single - wavelength reconstruction results;

[0028] A3 - 2: Fuse all single - wavelength reconstruction results according to the principle of linear superposition.

[0029] In some embodiments of the present invention, step A3 - 1 includes the following steps:

[0030] A3 - 1 - 1: For the single - wavelength reconstruction results obtained in step A2 - 2, according to the wavelength response function of the image acquisition device, re - assign the response color corresponding to its wavelength;

[0031] A3 - 1 - 2: Execute step A3 - 1 - 1 for all voxels of all single - wavelength reconstruction results obtained in step A2 - 2 to complete the color assignment of the single - wavelength reconstruction results.

[0032] In some embodiments of the present invention, the specific operation of step A3 - 1 - 1 is as follows:

[0033]

[0034] Among them, C xyz (k) represents the RGB color information at the voxel coordinates (x, y, z) of the reconstruction result of wavelength k, and R T (k) represents the matrix formed by the wavelength response function of the image acquisition device at wavelength k, represents the intensity information at the voxel coordinates (x, y, z) of the reconstruction result of wavelength k.

[0035] In some embodiments of the present invention, step A3 - 2 includes the following steps:

[0036] A3-2-1: Use the set of colored single-wavelength reconstruction results obtained in step A3-1, and linearly sum the RGB color information of each reconstruction result voxel by voxel to obtain composite color information, thereby obtaining the final multi-color reconstruction result;

[0037] A3-2-2: Perform step A3-2-1 on all the single-wavelength reconstruction results obtained in step A3-1 to obtain a multi-color light field microscopy three-dimensional reconstruction image.

[0038] In some embodiments of the present invention, the specific operation of step A3-2-1 is as follows:

[0039]

[0040] Among them, represents the RGB color information of the final fusion result at the voxel coordinate (x, y, z), C xyz (k) represents the RGB color information of the reconstruction result at the voxel coordinate (x, y, z) with wavelength k, K represents the number of single-wavelength three-dimensional reconstruction results, and θ(k) represents the weight value with wavelength k.

[0041] The present invention has the following beneficial effects:

[0042] The present invention introduces a modular idea, interprets the synchronous multi-color light field reconstruction method as three sub-methods: spectral information extraction and correction, monochromatic reconstruction, and image coloring and fusion, bridges the monochromatic light field microscopy reconstruction method and the multi-color light field microscopy reconstruction method, and the sub-methods can be replaced when they have the same function, and proposes a reliable multi-color reconstruction method that is easy to implement and expand, which is of great significance for promoting the research of neuroscience and the development of synchronous multi-color light field microscopy technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of the multi-color synchronous reconstruction method of the light field microscopy system in the embodiment of the present invention;

[0044] Figure 2A is the real image of the sample in the embodiment of the present invention;

[0045] Figure 2B is the asynchronous multi-color imaging reconstruction image in the embodiment of the present invention;

[0046] Figure 2C is the reconstruction image of the method in the embodiment of the present invention DETAILED DESCRIPTION OF THE INVENTION

[0047] Before introducing the specific embodiments of the present invention, the main idea of the present invention is introduced as follows:

[0048] In view of the problems of the above-mentioned existing multi-color light field microscopy reconstruction methods, the present invention proposes a multi-color synchronous reconstruction method for a light field microscopy system. The main innovation is that by introducing the modular idea, the present invention interprets the synchronous multi-color light field reconstruction method as three sub-methods: spectral information extraction and correction, monochromatic reconstruction, and image coloring and fusion. It bridges the monochromatic light field microscopy reconstruction method and the multi-color light field microscopy reconstruction method, and the sub-methods can be replaced when they have the same function. Compared with the existing methods, this method effectively guarantees the time resolution of multi-color light field microscopy, and proposes a reliable multi-color reconstruction method that is easy to implement and expand, which is of great significance for promoting the research of neuroscience and the development of synchronous multi-color light field microscopy technology.

[0049] The following further describes the present invention with reference to the accompanying drawings and in combination with preferred embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0050] It should be noted that the azimuth terms such as left, right, up, down, top, bottom, etc. in this embodiment are only relative concepts to each other, or are referenced based on the normal use state of the product, and should not be considered restrictive.

[0051] The present invention proposes a multi-color synchronous reconstruction method for a light field microscopy system, as Figure 1 shown, the method is as follows:

[0052] A1: Extract spectral information from the multi-wavelength fluorescence light field image according to the wavelength response function of the image acquisition system and perform correction;

[0053] A2: Separate the single-wavelength light field images according to the corrected spectral information, and perform reconstruction one by one using the single-wavelength light field microscopy reconstruction algorithm;

[0054] A3: Assign the corresponding colors to the single-wavelength reconstruction results, and fuse all the reconstruction results according to the principle of linear superposition.

[0055] When performing the above steps in a specific implementation scheme, the following operations can be carried out. It should be noted that the specific methods adopted during the implementation are only for illustrative purposes, and the scope covered by the present invention includes but is not limited to the following methods listed.

[0056] A1: Extract spectral information from the multi-wavelength fluorescence light field image according to the wavelength response function of the image acquisition system and perform correction.

[0057] A11: Extract spectral information from the multi-wavelength fluorescence light field image;

[0058] Specifically, for the multi-wavelength fluorescence light field image, the present invention takes the RGB (Red, Green, Blue, three primary colors) triple of each pixel of the image as the input, combines the wavelength response function of the image acquisition device, and constructs an optimization model for estimating the spectral information of each pixel. After estimation, the intensity information of the signal light of different wavelengths at the corresponding pixel spatial position can be obtained, so as to separate multiple monochromatic light field images as needed. The specific operations are as follows:

[0059]

[0060] In the above formula:

[0061] S ij represents the spectral information at the pixel coordinates (i, j);

[0062] C ij represents the RGB triple value of the multi-color fluorescence light field image at the pixel coordinates (i, j);

[0063] R T represents the matrix composed of the wavelength response functions of the RGB three channels of the image acquisition device;

[0064] represents the square of the l2-norm;

[0065] represents the spectral information at the pixel coordinates (i, j) estimated according to the optimization model;

[0066] argmin represents the variable value when the objective function f(x) takes the minimum value;

[0067] In the present invention, by traversing the estimation model at each pixel position of the light field image, preliminary spectral information can be obtained. Among them, RT needs to be constructed based on the wavelength response function of the image acquisition device. The methods for obtaining the wavelength response function include, but are not limited to, laboratory calibration, querying manuals, etc. The wavelength range should at least cover the spectral range of the target fluorescence. For the discrete response function, the method of linear interpolation can be used to increase the wavelength resolution. In addition, in order to make the extracted spectral information more accurate, the influence of the spectral curve of the light source, such as the fluorescence emission spectrum, can be considered on the basis of the wavelength response function. The operation method is as follows: obtain the fluorescence emission spectrum of the target through prior information, regard it as a weight, multiply the wavelength response function of the acquisition device wavelength by wavelength, construct a new response function, and then use this response function to extract spectral information.

[0068] A12: Perform spectral correction on the spectral information;

[0069] Specifically, spectral correction is performed on the preliminary estimation information obtained by A11 to ensure non-negativity of the data, and the single-wavelength light field image is separated therefrom. The method of the present invention applies a spectral correction function to the preliminary estimated spectral information of each pixel in each band, and then multiplies by a wavelength-dependent weight to obtain the corrected spectral information. According to the above results, the information of a specific wavelength can be selected from each pixel and recombined into a single-wavelength light field image. The specific operations are as follows:

[0070]

[0071]

[0072] In the above formula:

[0073] V ij (k) - The intensity information of wavelength k at the pixel coordinates (i, j) after correction;

[0074] W(k) - The weight value of wavelength k;

[0075] - The spectral information intensity of wavelength k at the pixel coordinates (i, j) estimated according to the optimization model;

[0076] r(x) - The spectral correction function, whose function value is equal to the independent variable when the independent variable is non-negative, and zero otherwise;

[0077] The weight value W(k) is a function of wavelength k. One method of selection is to sum and average the non-negative intensity values of all pixels of this wavelength, and then normalize and use it as the weight value. The specific operation is as follows:

[0078]

[0079] In the above formula:

[0080] ∑ k (·) - Sum wavelength by wavelength for all wavelengths;

[0081] ∑ i，j (·) - Sum pixel by pixel for all pixels;

[0082] W(k), has the same meaning as in formulas (2) and (3).

[0083] Another method that can be selected is to set the weight value W(k) to a constant.

[0084] After applying this method to the information of each wavelength of all pixels, spectral correction can be achieved. This step processes potential negative value information to ensure non-negativity of the spectrum, so as not to affect subsequent reconstruction, such as avoiding unpredictable artifacts and reconstruction failures.

[0085] A2: Separate the single-wavelength light field images according to the calibrated spectral information, and perform reconstruction one by one using the single-wavelength light field microscopy reconstruction algorithm.

[0086] Specifically, select specific wavelengths from the spectral calibration results obtained in step A12 to form several single-wavelength light field images, and use the existing single-wavelength light field microscopy algorithms to perform three-dimensional reconstruction one by one to obtain multiple single-wavelength reconstruction results. At this time, the multi-wavelength light field microscopy reconstruction problem has been transformed into several single-wavelength light field microscopy reconstruction problems. The reconstruction algorithms that can be used include but are not limited to the Richard-Lucy algorithm (an iterative light field depth estimation algorithm based on the point spread function). When using this method for reconstruction, this step can also be refined into a light field calibration and deconvolution reconstruction process; when using other methods for reconstruction, this step needs to be adjusted according to the selected reconstruction algorithm to achieve the goal of single-wavelength light field microscopy reconstruction.

[0087] A3: Assign corresponding colors to the single-wavelength reconstruction results, and fuse all the reconstruction results according to the principle of linear superposition.

[0088] A31: Assign colors to the monochromatic reconstruction results;

[0089] Specifically, use the several single-wavelength reconstruction results obtained in step A2, and re-assign the response colors corresponding to their wavelengths to each reconstruction result according to the wavelength response function of the image acquisition device. The specific operations are as follows:

[0090]

[0091] In the above formula:

[0092] C xyz (k) - The RGB color information at the voxel coordinates (x, y, z) of the reconstruction result with wavelength k;

[0093] R T (k) - The matrix formed by the wavelength response functions of the RGB three channels of the image acquisition device at wavelength k;

[0094] - The intensity information at the voxel coordinates (x, y, z) in the single-wavelength reconstruction result with wavelength k.

[0095] Apply the above formula to all voxels of all single-wavelength reconstructed three-dimensional results to complete the color assignment of the single-wavelength reconstruction results. Among them, the wavelength response function used to construct R T (k) is preferably the same as the wavelength response function used in method A1.

[0096] A32: Fuse all single-wavelength reconstruction results according to the principle of linear superposition;

[0097] Specifically, using a set of colored single-wavelength reconstruction results obtained in step A31, linearly adding the RGB color information of each reconstruction result voxel by voxel to obtain composite color information, the final multi-color reconstruction result can be obtained. The specific operation is as follows:

[0098]

[0099] In the above formula:

[0100] - RGB color information of the final fusion result at the voxel coordinate (x, y, z);

[0101] C xyz (k) - RGB color information of the voxel coordinates (x, y, z) at the reconstruction result with wavelength k;

[0102] K – the number of single-wavelength 3D reconstruction results;

[0103] θ(k) – weight value for wavelength k.

[0104] The weight value θ(k) is a function of the wavelength k. One method of selecting the weight value is to select the weight value as a normalization factor. The specific operation is:

[0105]

[0106] In the above formula:

[0107] C xyz The meanings of (k), K, and θ(k) are consistent with those in formula (6).

[0108] Another selection method is to construct weights based on the corrected spectral information and regard them as normalization factors. The specific operation is:

[0109]

[0110] K, θ(k) have the same meaning as formula (6); V ij (k) has the same meaning as formula (2).

[0111] By applying the above method to all single-wavelength reconstruction results, a multi-color light field microscopy 3D reconstructed image can be obtained.

[0112] The advantages of the present invention are: the temporal resolution of polychromatic light field microscopy is guaranteed; the method is easy to implement and sufficiently reliable; and the method is easy to expand.

[0113] The temporal resolution of polychromatic light field microscopy is guaranteed because the method does not need to sacrifice temporal resolution in exchange for the ability to collect polychromatic information of the observed target.

[0114] Temporal resolution is an indicator that measures the minimum time interval between two consecutive observations of a target. The smaller the time interval between observations, the larger the temporal resolution; conversely, the smaller the temporal resolution. The larger the temporal resolution, the more finely the system can capture the activities of the observed target on the time scale. The current main multi-color light field microscopy method is asynchronous multi-color imaging. Since it cannot effectively process multi-wavelength light field images, it can only obtain the multi-wavelength information of the observed target by taking multiple single-wavelength light field images. Each shot consumes time, so its temporal resolution is lower than the frame rate of the image acquisition device. In contrast, this method can effectively process multi-wavelength light field images, eliminating the need for the acquisition device to take single-wavelength light field images one by one. The temporal resolution of multi-color observation is equal to the frame rate of the image acquisition device, effectively guaranteeing the temporal resolution of multi-color light field microscopy. If the frame rate of the image acquisition device is 60 Hz (about one image is taken every 16.7 milliseconds) and the target is a three-color fluorescently stained sample, the frame rate of the asynchronous multi-color imaging method for acquiring the complete multi-color information of the target is 60÷3 Hz = 20 Hz (about one round is acquired every 50 milliseconds). When using this method, the frame rate for acquiring the complete multi-color information of the target is 60 Hz, saving 66% of the time compared to the former. In addition, since asynchronous multi-color imaging often requires a timing circuit to assist in controlling the sequential excitation of fluorescent groups by lasers, the actual frame rate for acquiring the multi-color information of the target will be even lower. Under this hypothetical scenario, when the activity frequency of the target is between 10 Hz and 30 Hz (i.e., the target activity period is between 100 milliseconds and 33.3 milliseconds), in the sense of the Nyquist sampling theorem (the sampling frequency should be at least twice the true frequency of the signal, otherwise all information of the signal cannot be acquired), this method can still be used as the reconstruction method to observe the target, but the asynchronous multi-color imaging method will not be able to acquire the true activity information of the observed target.

[0115] This method is easy to implement and reliable enough because it has no strict requirements for hardware and the reconstruction effect is similar to that of asynchronous multi-color imaging.

[0116] The input of this method is a multi-wavelength fluorescence light field image, and there is no restriction on the image acquisition device used for this image in light field microscopy. Compared with other synchronous multi-color imaging methods, such as achromatic metalens imaging method, the cost of this method is lower. In addition, the multi-wavelength fluorescence light field image is the result of the interaction between signal lights of different wavelengths and the image acquisition device system, which can be regarded as the linear superposition combination of the results of the interaction between countless monochromatic lights and the system independently. Therefore, the results of the interaction of each monochromatic light can be separated by certain means to obtain multiple single-wavelength light field images. At this time, synchronous multi-color imaging is transformed into asynchronous multi-color imaging. When the separation of single-wavelength light field images is accurate, the reconstruction results of the two should be consistent. And through experimental verification, the results reconstructed by this method are indeed similar to those reconstructed by asynchronous multi-color imaging methods. The experimental observation samples are blue fluorescent spheres and green fluorescent spheres, and the steps are as follows: First, collect the real image of the observation sample; then, take a multi-wavelength light field image and use this method for reconstruction; then, take two single-wavelength light field images and reconstruct them respectively using the monochromatic light field microscopy reconstruction method, and sum the two monochromatic reconstruction results with a certain weight; finally, compare the reconstruction results. The reconstruction results are as shown in the attached Figure 2A , Figure 2B , Figure 2C . Figure 2A is the real image of the sample, Figure 2B is the reconstruction image of asynchronous multi-color imaging, Figure 2C is the reconstruction image of this method. Among them, except for the different methods of image acquisition, the rest of the reconstruction algorithms are the same. Figure 2A In, the observation target appears in grayish white. Among them, the two circular spots with larger radii are green fluorescent spheres, and the smaller ones are blue fluorescent spheres, and the rest of the area is the background. Comparing Figure 2A with Figure 2B , it can be seen that at the position where the Figure 2A observation target appears, Figure 2B there also appears a similar bright spot shape, indicating that the reconstruction is correctly realized; similarly, at the corresponding position of Figure 2C there also appears a bright spot, indicating that the reconstruction is also correctly realized. Using Figure 2B and Figure 2C to compare the similarity with Figure 2A respectively, it can be seen that Figure 2C and Figure 2A have an even higher similarity, indicating that the reconstruction effect of this method is better. However, if mainly considering the similarity of the position and shape of the reconstruction results of the observation sample, it can be considered that the reconstruction effect of this method is equivalent to that of asynchronous multi-color imaging.

[0117] This method is easy to expand because this method interprets synchronous multi-color light field reconstruction as three sub-methods, and the sub-methods can be replaced and adjusted.

[0118] This method is based on the modular idea and makes a new interpretation of synchronous multi-color light field microscopy. The spectral information extraction and correction sub-method takes multi-wavelength fluorescence light field images as input and multiple single-wavelength light field images as output; the monochromatic reconstruction sub-method takes multiple single-wavelength light field images as input and multiple single-wavelength three-dimensional volume information as output; the image coloring and fusion sub-method takes multiple single-wavelength three-dimensional volume information as input and a single-color three-dimensional volume information as output. Given the input and output of the sub-methods, how the data is processed within each sub-method of this method can be ignored. For example, the Fourier light field microscopy reconstruction algorithm (another single-wavelength light field microscopy reconstruction method with better effects) can be used for monochromatic reconstruction instead of the Richard-Lucy method. Based on this feature, this method is more flexible than other synchronous multi-color imaging methods such as single-pixel imaging, and the internal data processing method of the sub-method can be adjusted according to actual convenience requirements to obtain higher reconstruction performance.

[0119] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those skilled in the technical field to which the present invention belongs, without departing from the concept of the present invention, several equivalent substitutions or obvious variations can be made, and as long as the performance or use is the same, they should all be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for multi-color synchronous reconstruction of a light field microscopy system, characterized in that, It includes the following steps: A1: According to the wavelength response function of the image acquisition system; wherein, step A1 further includes the following steps: A1-1: Extract the preliminary estimated spectral information from the multi-wavelength fluorescence light field image; A1-2: Perform spectral correction on the spectral information; apply the spectral correction function to the preliminary estimated spectral information pixel by pixel for each band, and then multiply by the wavelength-dependent weight to obtain the corrected spectral information; according to the above results, the information of a specific wavelength can be selected from each pixel and recombined into a single-wavelength light field image, and the specific operation is as follows: In the above formula: V ij (k) — The intensity information of wavelength k at the corrected pixel coordinates (i, j); W(k) — The weight value at wavelength k; — The spectral information intensity at wavelength k at pixel coordinates (i, j) estimated according to the optimized model; r(x') — The spectral correction function, whose function value is equal to the independent variable when the independent variable is non-negative, and zero otherwise; A2: Separate the single-wavelength light field images according to the corrected spectral information, and perform reconstruction one by one using the single-wavelength light field microscopy reconstruction algorithm; A3: Step A3 specifically includes the following steps: A3-1: Assign colors to the single-wavelength reconstruction results; Specifically, use the several single-wavelength reconstruction results obtained in step A2, and according to the wavelength response function of the image acquisition device, re-assign the response color corresponding to its wavelength to each reconstruction result, and the specific operation is as follows: In the above formula: C xyz (k) — RGB color information at the voxel coordinates (x, y, z) of the reconstruction result with wavelength k; R T (k) — The matrix formed by the wavelength response functions of the RGB three channels of the image acquisition device at wavelength k; — The intensity information at the voxel coordinates (x, y, z) in the single-wavelength reconstruction result with a wavelength of k; Apply the above formula to all voxels of all the three-dimensional single-wavelength reconstruction results to complete the color assignment of the single-wavelength reconstruction results; A3-2: Fuse all the single-wavelength reconstruction results according to the principle of linear superposition.

2. The method according to claim 1, wherein In step A1-1, the method for extracting the preliminary estimated spectral information from the multi-wavelength fluorescence light field image includes the following steps: A1-1-1: Take the RGB triple of each pixel of the image as the input, and combine with the wavelength response function of the image acquisition device to construct an optimization model for estimating the spectral information of each pixel, that is, the estimation model; A1-1-2: Traverse the estimation model in step A1-1-1 for each pixel position of the light field image to obtain the preliminary estimated spectral information.

3. The method according to claim 2, wherein The specific operation of step A1-1-1 is as follows: Among them, S ij represents the spectral information at the pixel coordinates (i, j), and C ij represents the RGB triple values of the multi-color fluorescence light field image at the pixel coordinates (i, j), and R T represents the matrix formed by the wavelength response functions of the RGB three channels of the image acquisition device, represents the square of the l2-norm, represents the spectral information at the pixel coordinates (i, j) estimated according to the optimization model, and argmin represents the variable value when the objective function takes the minimum value.

4. The method according to claim 1, characterized in that In step A1-2, the method for performing spectral correction on the spectral conversion result includes the following steps: A1-2-1: Apply the spectral correction function pixel by pixel for each band of the preliminary estimated spectral information obtained in step A1-1, and then multiply by the wavelength-dependent weight to obtain the corrected spectral information; A1-2-2: Perform step A1-2-1 on the information of each wavelength of all pixels to achieve spectral correction.

5. The method according to claim 1, characterized in that Step A2 includes the following steps: A2-1: Select specific wavelengths from the corrected spectral information obtained in step A1-2 to form several single-wavelength light field images; A2-2: Use the existing single-wavelength light field microscopy algorithm to perform three-dimensional reconstruction one by one to obtain several single-wavelength reconstruction results.

6. The method according to claim 1, wherein Step A3-2 includes the following steps: A3-2-1: Use the set of single-wavelength reconstruction results with colors assigned in step A3-1, and linearly add the RGB color information of each reconstruction result voxel by voxel to obtain the composite color information to obtain the final multi-color reconstruction result; A3-2-2: Perform step A3-2-1 on all the single-wavelength reconstruction results obtained in step A3-1 to obtain the multi-color light field microscopy three-dimensional reconstruction image.

7. The method according to claim 6, characterized in that, The specific operation of step A3-2-1 is: Among them, represents the RGB color information of the final fusion result at the voxel coordinates (x, y, z), C xyz (k) represents the RGB color information at the voxel coordinates (x, y, z) of the reconstruction result with wavelength k, K represents the number of single-wavelength three-dimensional reconstruction results, and θ(k) represents the weight value with wavelength k.

Images