33 results about "Light field microscopy" patented technology

A modality for compressed light field measurement to capture 3D fluorescent signals at kilohertz volume rate, is described. This modality, termed Squeezed Light field Microscopy (SQLFM), acquires the light field with an array of rotated 2D sub-aperture images. An anamorphic relay system applies anisotropic scaling, effectively "squeezing" the image along one axis. This allows the camera sensor to detect the light field with a letterbox-shaped region of interest (ROI). Each squeezed sub-aperture image is complementary to the others, facilitating successful 3D reconstruction from compressed measurement while preserving the nominal FOV and spatial resolution. SQLFM can achieve over a fivefold increase in acquisition rate compared to traditional LFM.

A device for light field microscopy, comprising: a two-dimensionally spatially resolving detector for detecting light radiated by a sample; a detection beam path having at least one microscope objective lens; and at least one multi-lens array for imaging the light radiated by the sample onto the detector, wherein the multi-lens array is arranged in a plane which is optically conjugated with respect to a rear focal plane of the microscope objective lens, or in the vicinity of such a plane; and a control and evaluation unit for activating the detector and for evaluating the measurement data from the detector. The device also includes an adjustable relay optical system in order to image the rear focal plane of the microscope objective lens into the plane in which the multi-lens array is arranged, or in the vicinity of this plane. A method for light field microscopy is also provided.

The invention provides a visual field depth expanding method based on sparse scanning of a light field microscope, which comprises the following steps of: performing sparse scanning on a sample in X, Y and Z axial directions by utilizing a micro-lens array light field imaging system, enabling the overlapping rate between adjacent visual fields to be lower than a preset value, and acquiring multi-visual-field two-dimensional original light field data; performing pixel rearrangement and deconvolution three-dimensional reconstruction on the two-dimensional original light field image of each view to obtain a reconstructed three-dimensional body of each view; calculating the theoretical overlapping region width in the XYZ axial direction according to the sparse scanning step length, and carrying out consistency calibration on the overlapping region in the XYZ axial direction of the adjacent visual fields; and carrying out recursive fusion on the calibrated reconstructed three-dimensional body along the X, Y and Z axial directions to obtain a full-view-field and full-depth-of-view three-dimensional body image. According to the invention, not only can a larger imaging depth range be provided, but also the image definition at different depth levels can be ensured, and the inherent depth-of-field limitation of a traditional optical system is overcome.

The invention discloses an automatic Gaussian light sheet center positioning method based on a confocal scanning light field microscope, and relates to the technical field of optical imaging, and the method comprises the steps: setting an initial offset center of a line scanning module based on a target sample, and adjusting the confocal scanning light field microscope; shooting a two-dimensional light field image of the target sample through the adjusted confocal scanning light field microscope, and performing two-dimensional slicing on the two-dimensional light field image according to a preset interval to obtain sub-view-angle images; on the basis of the sub-view-angle images, full-view-field longitudinal energy distribution is obtained, and the full-view-field longitudinal energy distribution is corrected; and calibrating the initial offset center based on the full-view-field longitudinal energy distribution before and after correction. Compared with manual calibration, parameters do not need to be adjusted manually, and automatic Gaussian line light center position positioning and calibration of different systems and different samples are achieved.

The present invention discloses a Fourier light field microscope super-resolution imaging method and system. The method comprises the following steps: (1) inputting a single multi-view light field image with a low angular sampling rate obtained by a Fourier light field microscope into an angle difference network to obtain multiple single-view light field images with a high angular sampling rate; (2) reconstructing the multiple single-view light field images with a high angular sampling rate obtained in step (1) into three-dimensional images on a perspective basis, and superimposing them in the same order as the multiple single-view images in the training sample sampling data of the angle difference network to obtain the Fourier light field microscope super-resolution imaging. The present invention provides a Fourier light field microscope super-resolution imaging method, which interpolates more perspective images in the actual captured light field image through a deep learning algorithm without generating crosstalk, and simultaneously obtains a light field PSF that matches the light field image, and then uses a three-dimensional reconstruction algorithm to obtain a high-quality reconstruction result.

This efficiently provides a wide range of transfer functions for light-field microscopy. [Solution] A sample volume to be imaged with multiple sub-images (1~12xy) from different capture angles (Φ) is captured, and for the sub-images (1~12xy) intended for 3D reconstruction, the respective point image distribution function (PSF) is identified. The sub-images (1~12xy) are examined for optical symmetry, the symmetry is captured, and stored along with a calculation rule. The calculation rule can start with the point image distribution function (PSF) and / or processed point image distribution function of the first sub-image (1~12) and generate the corresponding point image distribution function (PSF) and / or processed point image distribution function of further sub-images (2x~12xy) that are symmetric with respect to the first sub-image (1~12), and the point image distribution function (PSF) or processed point image distribution function of further sub-images (2x~12xy) is generated only when necessary.

The invention relates to a method for providing a transfer function in light field microscopy, in which a sample volume to be imaged is determined using a plurality of sub-images; determining a corresponding point image transfer function for the sub-image, and distributing the point image transfer function to the sub-image; checking the symmetry of the basic optical technical data of the sub-image; the confirmed symmetry is detected and saved along with a calculation rule that allows a point image transfer function and / or a processed point image transfer function of a further sub-image symmetrical to the first sub-image to be generated from the point image transfer function and / or the processed point image transfer function of the first sub-image. The invention also relates to a reconstruction method for virtually reconstructing a sample volume detected using a plurality of sub-images from different detection angles in light field microscopy.

A lightfield imaging system is described herein. The lightfield imaging system includes a display device for displaying three-dimensional (3D) video images of a volume of interest and a lightfield microscope assembly. The lightfield microscope assembly includes a microscope housing, a Multiple Angle Capture (MAC) sensor array including a plurality of sensors mounted to the microscope housing, an objective lens assembly mounted to the microscope housing and configured to direct light rays from a volume of interest to each of the sensors, and a microscope controller coupled to the display device and the sensors. The microscope controller includes a memory device for storing computer-executable instructions thereon and one or more processors for executing the computer-executable instructions for performing an algorithm for rendering 3D video images of the volume of interest onto the display device.

The invention discloses a large-view-field light field microscopic image distortion correction method and system, and belongs to the technical field of computer vision, and the method comprises the steps: obtaining a calibration image, and extracting a center coordinate of a central micro lens as a feature point; constructing an ideal circle center grid and an actual circle center grid according to the feature points, and obtaining grid coordinates; the two-dimensional nonlinear distortion model comprises an external parameter matrix, an internal parameter matrix, radial distortion and tangential distortion, and elements in the external parameter matrix comprise image rotation angle parameters and translation parameters; elements in the internal reference matrix comprise scaling parameters and optical center position parameters; substituting the normalized ideal circle center grid coordinates and the actual circle center grid coordinates into the two-dimensional nonlinear distortion model, and solving to obtain distortion parameters of the light field microscope; and performing correction and remapping based on the obtained distortion parameters. According to the method and the system, the distortion correction precision of the large-view-field light field microscopic image is improved.

The invention relates to a method for providing an overview image of an object to be imaged in light-field microscopy, in which image data of at least one image of a region of the object is detected by means of a detection optics and a detector; and from the image data, reconstructing a plurality of z-planes of the region to be detected in a computational manner, using a portion of the detected image data for reconstruction, and / or reducing the number of z-planes to be reconstructed. A z-section around and above and below the focal plane is defined along the z-direction. A channel is assigned to each z-section, and image data of the structure of the object detected in the respective z-section is stored in correspondence with the respective channel. According to the invention, a control command is generated by the control device, by means of which the output device is controlled, the structure being output in accordance with the associated channel in each case with a different, however, graphic feature specific to the channel. The invention also relates to a light field microscope for carrying out the method according to the invention.

To automatically three-dimensionally track an object in a microscope system using a light field camera.SOLUTION: A light field three-dimensional tracking control device 110 controls a motor-driven stage 90 movable in XYZ directions to three-dimensionally track an object observed by a light field camera 10 through an objective lens 70 even if the object is in a sample holder 80 mounted on the motor-driven stage 90, and comprises: a light field image analysis unit 30 that analyzes a light field image obtained by imaging the object with the light field camera 10 in a time-continuous manner to track a three-dimensional spatial position of the object in the light field image; and a stage control unit 90 that controls the movement in the XYZ directions of the motor-driven stage 90 so that a relative positional relation between the objective lens 70 and the three-dimensional spatial position of the object tracked by the light field image analysis unit 30 is maintained constant.SELECTED DRAWING: Figure 2

The invention relates to a light-field microscope and a light-field microscopy image capturing method, including the steps of providing an illumination radiation from at least one light source; modulating the illumination radiation by virtue of adjusting the intensity distribution thereof; generating a dynamic light sheet in a sample space, in which a sample to be imaged can be arranged; scanning a region of the sample to be imaged using the light sheet; collecting a detection radiation and detecting image signals using light-field technology. The illumination radiation is modulated by virtue of the light source being controlled and an intensity of the illumination radiation provided by the light source being altered, especially in a controlled fashion over time. Image recordings of the sample are triggered for different positions of the light sheet.

The invention provides a confocal light field microscope, which comprises an objective lens, a dichroic mirror, a microlens array positioned on a conjugate rear pupil plane of the objective lens, a mask positioned on a rear focal plane of the microlens array, a double-sided scanning galvanometer and a reflector group, the double-sided scanning galvanometer comprises a first reflecting surface and a second reflecting surface which are respectively positioned on the front and back surfaces, the first reflecting surface is used for reflecting the exciting light introduced by the dichroic mirror to an imaging object in front of the objective lens, and the reflecting mirror group reflects the excited exciting light to the second reflecting surface so as to be reflected to the photosensitive element through the second reflecting surface; wherein the angle of the double-sided scanning galvanometer is changed, so that the exciting light scans the imaging object through the first reflecting surface and the excited light enters the photosensitive element through the second reflecting surface. According to the confocal light field microscope provided by the invention, the influence of scanning distortion can be eliminated during scanning through a double-sided scanning mechanism, and a series of problems caused by a synchronization error of two times of scanning are avoided.

A method provides an overview image of an object in light field microscopy, in which image data of at least one image of a region of the object are acquired by a detection optical unit and a detector and, on the basis of the image data, a number of z planes (Zn) of the region are computationally reconstructed. Z sections (uhF, F, ohF) around and above and below the focal plane (F) are defined in a z direction and a channel is assigned to each of the z sections (Zn) and image data of structures of the object detected in the respective z sections (Zn) are stored assigned to the respective channel. By a controller, control commands by which an output device is activated are generated, the structures being output according to the respectively assigned channels with different, but channel-specific graphical characteristics. A light field microscope carries out the method.

The application discloses an automatic positioning Gauss light sheet center method based on a confocal scanning light field microscope, and relates to the technical field of optical imaging, and comprises the following steps: based on a target sample, setting an initial offset center of a line scanning module, and adjusting a confocal scanning light field microscope; through the adjusted confocal scanning light field microscope, shooting a two-dimensional light field graph of the target sample, and carrying out two-dimensional slicing on the two-dimensional light field graph according to a preset interval to obtain a sub-view angle graph; based on the sub-view angle graph, obtaining a full-view field longitudinal energy distribution, and correcting the full-view field longitudinal energy distribution; based on the full-view field longitudinal energy distribution before and after correction, calibrating the initial offset center. Compared with manual calibration, the application does not need to manually adjust parameters, and realizes automatic Gauss line light center position positioning and calibration of different systems and different samples.

A method for automatically determining whether to perform denoising includes the following steps: S1: scanning a sample using a light field microscopy system to obtain a light field microscopic image containing 4-dimensional information, where the sample is a sample to be 3D reconstructed; S2: rearranging the light field microscopic image to obtain a multi-view rearranged image; S3: calculating a cleanliness score of the multi-view rearranged image; and determining whether to perform denoising process on the multi-view rearranged image, based on the cleanliness score of the multi-view rearranged image.

The present invention discloses a three-dimensional microscale velocity field measurement method, device, and storage medium. The three-dimensional microscale velocity field measurement method includes: obtaining a particle light field image under light field microscopy imaging; calculating a reconstructed particle distribution of the obtained particle light field image within a measurement volume; obtaining a particle center distribution of the reconstructed particle distribution; calculating a low-resolution three-dimensional displacement field within the measurement volume using a cross-correlation algorithm based on the reconstructed particle distribution; using the calculated low-resolution three-dimensional displacement field as the predicted displacement, performing motion compensation on the obtained particle center distribution to obtain a shifted particle center distribution; and performing cross-validation matching on the obtained shifted particle center distribution to obtain a high-resolution three-dimensional microscale velocity field. Based on the low-resolution three-dimensional displacement field obtained by the cross-correlation algorithm, the present invention uses a cross-validation matching algorithm to match two frames of particles. This method can accurately track individual particles at high concentrations and effectively improve the resolution of the three-dimensional microscale velocity field.

This invention provides a light field microscopy imaging system based on rare-earth upconversion materials, comprising: a first light source, a second light source, a third light source, a spatial filtering module, a first dichroic mirror, a second dichroic mirror, a third dichroic mirror, an aperture, a multi-channel filtering module, an objective lens, a tube lens, a microlens array, and a sensor. Light of different wavelengths emitted from the first, second, and third light sources is transmitted through the spatial filtering module to the corresponding first, second, and third dichroic mirrors. The first, second, and third dichroic mirrors are spatially aligned and beam-combined through the aperture. The combined three-color light sources are focused into the objective lens to achieve wide-field illumination. The rare-earth upconversion material-labeled sample is excited and transmitted through the multi-channel filtering module to the tube lens to form an image. The microlens array is located at the focal point of the tube lens, and the sensor is located at the focal length of the microlens array for recording the three-color imaging of the sample.

The present invention discloses a multi-color synchronous reconstruction method for a light field microscopy system, belonging to the fields of computer vision and digital image processing. The method includes: A1: extracting spectral information from multi-wavelength fluorescence light field images and performing spectral correction according to the wavelength response function of the image acquisition system; A2: separating single-wavelength light field images and performing reconstruction one by one using the single-wavelength light field microscopy reconstruction algorithm; A3: coloring the single-wavelength reconstruction results and fusing all the reconstruction results according to the principle of linear superposition. The present invention interprets the synchronous multi-color light field reconstruction method as three sub-methods: spectral information extraction and correction, monochromatic reconstruction, and image coloring and fusion, and proposes a reliable multi-color reconstruction method that is easy to implement and expand, which is of great significance for promoting the research of neuroscience and the development of synchronous multi-color light field microscopy technology.

The present invention discloses a method, device and medium for calibrating the step size of a two-dimensional scanning module in a scanning light field imaging system, which relates to the technical field of light field microscopy imaging. The method includes: collecting two-dimensional original scanning light field data with an initial scanning step size and rearranging it into four-dimensional phase space data to obtain a multi-view rearranged image; selecting the middle view of all views in the multi-view rearranged image to obtain a non-scanning center view sub-image; calculating a sub-pixel displacement matrix of all non-scanning center view sub-images relative to the first non-scanning center view sub-image; calculating an average displacement amount according to the sub-pixel displacement matrix, and calculating a step size adjustment coefficient according to the average displacement amount; updating the correct scanning step size according to the step size adjustment coefficient. The present invention calibrates the scanning step size of two-dimensional scanning imaging with higher speed and higher precision, and improves the spatial resolution of scanning imaging.

A method for automatically determining whether to perform denoising includes the following steps: S1: scanning a sample using a light field microscopy system to obtain a light field microscopic image containing 4-dimensional information, where the sample is a sample to be 3D reconstructed; S2: rearranging the light field microscopic image to obtain a multi-view rearranged image; S3: calculating a cleanliness score of the multi-view rearranged image; and determining whether to perform denoising process on the multi-view rearranged image, based on the cleanliness score of the multi-view rearranged image.

The invention concerns a method for the provision of transfer functions in the field of light field microscopy, where a sample volume to be imaged is captured using a multitude of partial images (1 bis 12xy) from different angles of capture (Φ) and for which the respective point spread functions (PSF) for the partial images (1 through 12xy) intended for a 3D reconstruction of the sample volume are determined and assigned to the partial images (1 through 12xy).A characteristic is that the partial images (1 through 12xy) are examined for existing symmetries of the optical-technical data they are based on and any found symmetries are captured and stored together with a computation formula. The computation formula makes it possible, starting from a point spread function (PSF) and / or a processed point spread function of a first partial image (1 through 12), to generate corresponding point spread functions (PSF) and / or processed point spread functions of other partial images (2x through 12xy) that are symmetric relative to the first partial image (1 through 12), where the point spread functions (PSF) and the processed point spread functions (PSF), respectively, of the other partial images (2x through 12xy) are only generated if needed and are only stored temporarily. A processed point spread function is obtained by executing at least one mathematical operation on the relevant point spread function. In addition, or alternatively, further symmetric subsets within the transmission function of at least one partial image (1 through 12xy) can be generated based on a first subset of the transmission function of a partial image (1 through 12xy), when and if needed.

Light field microscope-based image acquisition apparatuses and methods are provided. The light field microscope-based image acquisition apparatus includes a lenslet synthesis processing unit configured to acquire a virtual lenslet image using a multi-lenslet image group; a sub-aperture image conversion unit configured to convert the virtual lenslet image generated in the lenslet synthesis processing unit into a sub-aperture image; and a 3D conversion unit configured to convert the sub-aperture image generated in the sub-aperture image conversion unit into a focal-stack image.

The present invention provides a three-dimensional reconstruction method, device, and storage medium based on a microlens array, belonging to the field of light field microscopy. The method comprises: importing multiple light field raw data, processing white images of the multiple light field raw data to obtain a target white image; screening the coordinates with the maximum brightness value in the target white image, obtaining multiple microlens center coordinates after screening, and constructing the microlens array surface coordinates through all the microlens center coordinates; obtaining a sub-aperture image through a CCD camera, importing point light source data, and analyzing the sub-aperture image, point light source data, and microlens array surface coordinates to obtain a three-dimensional reconstruction result. The present invention can obtain light field microscopy imaging results at different imaging depths, and the three-dimensional reconstruction effect is relatively good, which is suitable for three-dimensional reconstruction of light field microscopy.

This invention provides a light-field microscope 3D image reconstruction system and method integrated with an event camera, belonging to the field of optical technology. This system synthesizes raw light-field data into event data for network training. The data is then input into an event image reconstruction network module capable of converting a continuous event stream into a series of intensity images to produce a light-field event image. The 3D image reconstruction network module then generates a 3D reconstruction of the target area. This system provides a new technical reference for improving the quality and resolution of 3D reconstruction using light-field microscopes.

A method including the following: a) a light field microscope records at least one image of a sample, said image consisting of a set of partial images, b) at least one aberration of the imaging system of the light field microscope is specified by a user and / or established from a set of partial images recorded in step a), c) one or both of steps d) and e) are performed using the aberrations of the imaging system specified and / or established in step b): d) reconstructing a three-dimensional image of the sample from the set of partial images, wherein the aberrations of the imaging system which are specified and / or established in step b) are at least partially corrected; e) establishing improved settings of adjustable components of the imaging system which influence wavefronts of the propagated light, on the basis of the aberrations specified and / or established in step b). The invention also relates to a microscopy apparatus.

The invention provides a microlens array for a light field microscope and the light field microscope. A microlens array for a light field microscope is located on a rear pupil plane of an objective lens, each microlens of the microlens array having optimization parameters independently designed based on respective incident light characteristics for correcting aberrations and chromatic aberrations.

This invention relates to the field of computational optical imaging and microscopic image processing technology, and discloses a method and system for joint reconstruction and denoising of light field microscopy based on self-supervised learning. In the pre-training stage, the method performs random masking operations in the spatial and angular domains on large-scale light field data, guiding the encoder to learn general features through a completion task. In the model training stage, the acquired light field images are randomly divided into input and supervision viewpoints. Features are extracted using the encoder and aligned in three-dimensional space using a physical point spread function. A three-dimensional volume is generated through viewpoint pooling and a decoding network, and a self-supervised loss is established using the supervision viewpoint, achieving implicit denoising based on the independence of noise distribution between viewpoints. In the testing stage, high signal-to-noise ratio three-dimensional reconstruction results can be obtained by inputting all viewpoints. This invention does not rely on noise-free ground truth images, can adapt to different optical system configurations, and improves the reconstruction quality and denoising capability of light field microscopy imaging.

The invention discloses a light field self-supervised three-dimensional reconstruction method and system based on per-view aberration correction, and the method comprises the steps: carrying out the self-supervised training of a per-view aberration prediction module through employing a multi-view aberration-estimation body joint loss function, independently reasoning the aberration between a perfect light field image and an original light field image of a sample at each view, and carrying out the self-supervised three-dimensional reconstruction. And three-dimensional reconstruction is carried out based on the corrected per-view-angle image to obtain a three-dimensional body, so that the axial resolution is improved, and artifacts are reduced. Therefore, the technical problem that an ideal imaging effect of a three-dimensional body obtained through three-dimensional reconstruction is difficult to obtain due to aberration introduced by an optical system and a sample of an existing light field microscope is solved. According to the method, on the premise that training data is not needed, the phenomenon that artifacts are obvious during three-dimensional reconstruction due to the fact that aberrations of all view angles are inconsistent is avoided, the artifacts are reduced while the axial resolution is effectively improved, and a good light field image three-dimensional reconstruction effect is achieved.

The invention relates to a method for providing an overview image of an object (1) to be imaged in light field microscopy, in which image data of at least one image of an area of ​​the object (1) are acquired by means of a detection optic (3) and a detector (11); and on the basis of the image data a number of z-planes (Zn) of the acquired area are computationally reconstructed. According to the invention, a portion of the acquired image data is used for reconstruction and / or the number of z-planes (Zn) to be reconstructed is reduced. In a z-direction extending towards an optical axis of the detection optics (3), z-segments (uhF, F, ohF) are defined around the focal plane (F) as well as above and below the focal plane (F). Each of the z-segments (Zn) is assigned a channel, and image data of structures (2) of the object (1) detected in the respective z-segments (Zn) are stored and assigned to the respective channel. Control commands are generated by means of a controller (6), which actuates an output device (4), whereby the structures (2) are output with different, but channel-specific, graphic features corresponding to the respective assigned channels. The invention also relates to a light field microscope (5) for carrying out the method according to the invention.