24 results about "Light field microscopy" patented technology

A device for light field microscopy, comprising: a two-dimensionally spatially resolving detector for detecting light radiated by a sample; a detection beam path having at least one microscope objective lens; and at least one multi-lens array for imaging the light radiated by the sample onto the detector, wherein the multi-lens array is arranged in a plane which is optically conjugated with respect to a rear focal plane of the microscope objective lens, or in the vicinity of such a plane; and a control and evaluation unit for activating the detector and for evaluating the measurement data from the detector. The device also includes an adjustable relay optical system in order to image the rear focal plane of the microscope objective lens into the plane in which the multi-lens array is arranged, or in the vicinity of this plane. A method for light field microscopy is also provided.

The invention provides a visual field depth expanding method based on sparse scanning of a light field microscope, which comprises the following steps of: performing sparse scanning on a sample in X, Y and Z axial directions by utilizing a micro-lens array light field imaging system, enabling the overlapping rate between adjacent visual fields to be lower than a preset value, and acquiring multi-visual-field two-dimensional original light field data; performing pixel rearrangement and deconvolution three-dimensional reconstruction on the two-dimensional original light field image of each view to obtain a reconstructed three-dimensional body of each view; calculating the theoretical overlapping region width in the XYZ axial direction according to the sparse scanning step length, and carrying out consistency calibration on the overlapping region in the XYZ axial direction of the adjacent visual fields; and carrying out recursive fusion on the calibrated reconstructed three-dimensional body along the X, Y and Z axial directions to obtain a full-view-field and full-depth-of-view three-dimensional body image. According to the invention, not only can a larger imaging depth range be provided, but also the image definition at different depth levels can be ensured, and the inherent depth-of-field limitation of a traditional optical system is overcome.

The invention discloses an automatic Gaussian light sheet center positioning method based on a confocal scanning light field microscope, and relates to the technical field of optical imaging, and the method comprises the steps: setting an initial offset center of a line scanning module based on a target sample, and adjusting the confocal scanning light field microscope; shooting a two-dimensional light field image of the target sample through the adjusted confocal scanning light field microscope, and performing two-dimensional slicing on the two-dimensional light field image according to a preset interval to obtain sub-view-angle images; on the basis of the sub-view-angle images, full-view-field longitudinal energy distribution is obtained, and the full-view-field longitudinal energy distribution is corrected; and calibrating the initial offset center based on the full-view-field longitudinal energy distribution before and after correction. Compared with manual calibration, parameters do not need to be adjusted manually, and automatic Gaussian line light center position positioning and calibration of different systems and different samples are achieved.

This efficiently provides a wide range of transfer functions for light-field microscopy. [Solution] A sample volume to be imaged with multiple sub-images (1~12xy) from different capture angles (Φ) is captured, and for the sub-images (1~12xy) intended for 3D reconstruction, the respective point image distribution function (PSF) is identified. The sub-images (1~12xy) are examined for optical symmetry, the symmetry is captured, and stored along with a calculation rule. The calculation rule can start with the point image distribution function (PSF) and / or processed point image distribution function of the first sub-image (1~12) and generate the corresponding point image distribution function (PSF) and / or processed point image distribution function of further sub-images (2x~12xy) that are symmetric with respect to the first sub-image (1~12), and the point image distribution function (PSF) or processed point image distribution function of further sub-images (2x~12xy) is generated only when necessary.

The invention relates to a method for providing a transfer function in light field microscopy, in which a sample volume to be imaged is determined using a plurality of sub-images; determining a corresponding point image transfer function for the sub-image, and distributing the point image transfer function to the sub-image; checking the symmetry of the basic optical technical data of the sub-image; the confirmed symmetry is detected and saved along with a calculation rule that allows a point image transfer function and / or a processed point image transfer function of a further sub-image symmetrical to the first sub-image to be generated from the point image transfer function and / or the processed point image transfer function of the first sub-image. The invention also relates to a reconstruction method for virtually reconstructing a sample volume detected using a plurality of sub-images from different detection angles in light field microscopy.

The invention discloses a large-view-field light field microscopic image distortion correction method and system, and belongs to the technical field of computer vision, and the method comprises the steps: obtaining a calibration image, and extracting a center coordinate of a central micro lens as a feature point; constructing an ideal circle center grid and an actual circle center grid according to the feature points, and obtaining grid coordinates; the two-dimensional nonlinear distortion model comprises an external parameter matrix, an internal parameter matrix, radial distortion and tangential distortion, and elements in the external parameter matrix comprise image rotation angle parameters and translation parameters; elements in the internal reference matrix comprise scaling parameters and optical center position parameters; substituting the normalized ideal circle center grid coordinates and the actual circle center grid coordinates into the two-dimensional nonlinear distortion model, and solving to obtain distortion parameters of the light field microscope; and performing correction and remapping based on the obtained distortion parameters. According to the method and the system, the distortion correction precision of the large-view-field light field microscopic image is improved.

The invention relates to a method for providing an overview image of an object to be imaged in light-field microscopy, in which image data of at least one image of a region of the object is detected by means of a detection optics and a detector; and from the image data, reconstructing a plurality of z-planes of the region to be detected in a computational manner, using a portion of the detected image data for reconstruction, and / or reducing the number of z-planes to be reconstructed. A z-section around and above and below the focal plane is defined along the z-direction. A channel is assigned to each z-section, and image data of the structure of the object detected in the respective z-section is stored in correspondence with the respective channel. According to the invention, a control command is generated by the control device, by means of which the output device is controlled, the structure being output in accordance with the associated channel in each case with a different, however, graphic feature specific to the channel. The invention also relates to a light field microscope for carrying out the method according to the invention.

The invention relates to a light-field microscope and a light-field microscopy image capturing method, including the steps of providing an illumination radiation from at least one light source; modulating the illumination radiation by virtue of adjusting the intensity distribution thereof; generating a dynamic light sheet in a sample space, in which a sample to be imaged can be arranged; scanning a region of the sample to be imaged using the light sheet; collecting a detection radiation and detecting image signals using light-field technology. The illumination radiation is modulated by virtue of the light source being controlled and an intensity of the illumination radiation provided by the light source being altered, especially in a controlled fashion over time. Image recordings of the sample are triggered for different positions of the light sheet.

A method provides an overview image of an object in light field microscopy, in which image data of at least one image of a region of the object are acquired by a detection optical unit and a detector and, on the basis of the image data, a number of z planes (Zn) of the region are computationally reconstructed. Z sections (uhF, F, ohF) around and above and below the focal plane (F) are defined in a z direction and a channel is assigned to each of the z sections (Zn) and image data of structures of the object detected in the respective z sections (Zn) are stored assigned to the respective channel. By a controller, control commands by which an output device is activated are generated, the structures being output according to the respectively assigned channels with different, but channel-specific graphical characteristics. A light field microscope carries out the method.

The application discloses an automatic positioning Gauss light sheet center method based on a confocal scanning light field microscope, and relates to the technical field of optical imaging, and comprises the following steps: based on a target sample, setting an initial offset center of a line scanning module, and adjusting a confocal scanning light field microscope; through the adjusted confocal scanning light field microscope, shooting a two-dimensional light field graph of the target sample, and carrying out two-dimensional slicing on the two-dimensional light field graph according to a preset interval to obtain a sub-view angle graph; based on the sub-view angle graph, obtaining a full-view field longitudinal energy distribution, and correcting the full-view field longitudinal energy distribution; based on the full-view field longitudinal energy distribution before and after correction, calibrating the initial offset center. Compared with manual calibration, the application does not need to manually adjust parameters, and realizes automatic Gauss line light center position positioning and calibration of different systems and different samples.

A method for automatically determining whether to perform denoising includes the following steps: S1: scanning a sample using a light field microscopy system to obtain a light field microscopic image containing 4-dimensional information, where the sample is a sample to be 3D reconstructed; S2: rearranging the light field microscopic image to obtain a multi-view rearranged image; S3: calculating a cleanliness score of the multi-view rearranged image; and determining whether to perform denoising process on the multi-view rearranged image, based on the cleanliness score of the multi-view rearranged image.

The present invention discloses a three-dimensional microscale velocity field measurement method, device, and storage medium. The three-dimensional microscale velocity field measurement method includes: obtaining a particle light field image under light field microscopy imaging; calculating a reconstructed particle distribution of the obtained particle light field image within a measurement volume; obtaining a particle center distribution of the reconstructed particle distribution; calculating a low-resolution three-dimensional displacement field within the measurement volume using a cross-correlation algorithm based on the reconstructed particle distribution; using the calculated low-resolution three-dimensional displacement field as the predicted displacement, performing motion compensation on the obtained particle center distribution to obtain a shifted particle center distribution; and performing cross-validation matching on the obtained shifted particle center distribution to obtain a high-resolution three-dimensional microscale velocity field. Based on the low-resolution three-dimensional displacement field obtained by the cross-correlation algorithm, the present invention uses a cross-validation matching algorithm to match two frames of particles. This method can accurately track individual particles at high concentrations and effectively improve the resolution of the three-dimensional microscale velocity field.

This invention provides a light field microscopy imaging system based on rare-earth upconversion materials, comprising: a first light source, a second light source, a third light source, a spatial filtering module, a first dichroic mirror, a second dichroic mirror, a third dichroic mirror, an aperture, a multi-channel filtering module, an objective lens, a tube lens, a microlens array, and a sensor. Light of different wavelengths emitted from the first, second, and third light sources is transmitted through the spatial filtering module to the corresponding first, second, and third dichroic mirrors. The first, second, and third dichroic mirrors are spatially aligned and beam-combined through the aperture. The combined three-color light sources are focused into the objective lens to achieve wide-field illumination. The rare-earth upconversion material-labeled sample is excited and transmitted through the multi-channel filtering module to the tube lens to form an image. The microlens array is located at the focal point of the tube lens, and the sensor is located at the focal length of the microlens array for recording the three-color imaging of the sample.

A method for automatically determining whether to perform denoising includes the following steps: S1: scanning a sample using a light field microscopy system to obtain a light field microscopic image containing 4-dimensional information, where the sample is a sample to be 3D reconstructed; S2: rearranging the light field microscopic image to obtain a multi-view rearranged image; S3: calculating a cleanliness score of the multi-view rearranged image; and determining whether to perform denoising process on the multi-view rearranged image, based on the cleanliness score of the multi-view rearranged image.

The invention concerns a method for the provision of transfer functions in the field of light field microscopy, where a sample volume to be imaged is captured using a multitude of partial images (1 bis 12xy) from different angles of capture (Φ) and for which the respective point spread functions (PSF) for the partial images (1 through 12xy) intended for a 3D reconstruction of the sample volume are determined and assigned to the partial images (1 through 12xy).A characteristic is that the partial images (1 through 12xy) are examined for existing symmetries of the optical-technical data they are based on and any found symmetries are captured and stored together with a computation formula. The computation formula makes it possible, starting from a point spread function (PSF) and / or a processed point spread function of a first partial image (1 through 12), to generate corresponding point spread functions (PSF) and / or processed point spread functions of other partial images (2x through 12xy) that are symmetric relative to the first partial image (1 through 12), where the point spread functions (PSF) and the processed point spread functions (PSF), respectively, of the other partial images (2x through 12xy) are only generated if needed and are only stored temporarily. A processed point spread function is obtained by executing at least one mathematical operation on the relevant point spread function. In addition, or alternatively, further symmetric subsets within the transmission function of at least one partial image (1 through 12xy) can be generated based on a first subset of the transmission function of a partial image (1 through 12xy), when and if needed.

Light field microscope-based image acquisition apparatuses and methods are provided. The light field microscope-based image acquisition apparatus includes a lenslet synthesis processing unit configured to acquire a virtual lenslet image using a multi-lenslet image group; a sub-aperture image conversion unit configured to convert the virtual lenslet image generated in the lenslet synthesis processing unit into a sub-aperture image; and a 3D conversion unit configured to convert the sub-aperture image generated in the sub-aperture image conversion unit into a focal-stack image.

The present invention provides a three-dimensional reconstruction method, device, and storage medium based on a microlens array, belonging to the field of light field microscopy. The method comprises: importing multiple light field raw data, processing white images of the multiple light field raw data to obtain a target white image; screening the coordinates with the maximum brightness value in the target white image, obtaining multiple microlens center coordinates after screening, and constructing the microlens array surface coordinates through all the microlens center coordinates; obtaining a sub-aperture image through a CCD camera, importing point light source data, and analyzing the sub-aperture image, point light source data, and microlens array surface coordinates to obtain a three-dimensional reconstruction result. The present invention can obtain light field microscopy imaging results at different imaging depths, and the three-dimensional reconstruction effect is relatively good, which is suitable for three-dimensional reconstruction of light field microscopy.

A method including the following: a) a light field microscope records at least one image of a sample, said image consisting of a set of partial images, b) at least one aberration of the imaging system of the light field microscope is specified by a user and / or established from a set of partial images recorded in step a), c) one or both of steps d) and e) are performed using the aberrations of the imaging system specified and / or established in step b): d) reconstructing a three-dimensional image of the sample from the set of partial images, wherein the aberrations of the imaging system which are specified and / or established in step b) are at least partially corrected; e) establishing improved settings of adjustable components of the imaging system which influence wavefronts of the propagated light, on the basis of the aberrations specified and / or established in step b). The invention also relates to a microscopy apparatus.

This invention relates to the field of computational optical imaging and microscopic image processing technology, and discloses a method and system for joint reconstruction and denoising of light field microscopy based on self-supervised learning. In the pre-training stage, the method performs random masking operations in the spatial and angular domains on large-scale light field data, guiding the encoder to learn general features through a completion task. In the model training stage, the acquired light field images are randomly divided into input and supervision viewpoints. Features are extracted using the encoder and aligned in three-dimensional space using a physical point spread function. A three-dimensional volume is generated through viewpoint pooling and a decoding network, and a self-supervised loss is established using the supervision viewpoint, achieving implicit denoising based on the independence of noise distribution between viewpoints. In the testing stage, high signal-to-noise ratio three-dimensional reconstruction results can be obtained by inputting all viewpoints. This invention does not rely on noise-free ground truth images, can adapt to different optical system configurations, and improves the reconstruction quality and denoising capability of light field microscopy imaging.

The invention discloses a light field self-supervised three-dimensional reconstruction method and system based on per-view aberration correction, and the method comprises the steps: carrying out the self-supervised training of a per-view aberration prediction module through employing a multi-view aberration-estimation body joint loss function, independently reasoning the aberration between a perfect light field image and an original light field image of a sample at each view, and carrying out the self-supervised three-dimensional reconstruction. And three-dimensional reconstruction is carried out based on the corrected per-view-angle image to obtain a three-dimensional body, so that the axial resolution is improved, and artifacts are reduced. Therefore, the technical problem that an ideal imaging effect of a three-dimensional body obtained through three-dimensional reconstruction is difficult to obtain due to aberration introduced by an optical system and a sample of an existing light field microscope is solved. According to the method, on the premise that training data is not needed, the phenomenon that artifacts are obvious during three-dimensional reconstruction due to the fact that aberrations of all view angles are inconsistent is avoided, the artifacts are reduced while the axial resolution is effectively improved, and a good light field image three-dimensional reconstruction effect is achieved.

The invention relates to a method for providing an overview image of an object (1) to be imaged in light field microscopy, in which image data of at least one image of an area of ​​the object (1) are acquired by means of a detection optic (3) and a detector (11); and on the basis of the image data a number of z-planes (Zn) of the acquired area are computationally reconstructed. According to the invention, a portion of the acquired image data is used for reconstruction and / or the number of z-planes (Zn) to be reconstructed is reduced. In a z-direction extending towards an optical axis of the detection optics (3), z-segments (uhF, F, ohF) are defined around the focal plane (F) as well as above and below the focal plane (F). Each of the z-segments (Zn) is assigned a channel, and image data of structures (2) of the object (1) detected in the respective z-segments (Zn) are stored and assigned to the respective channel. Control commands are generated by means of a controller (6), which actuates an output device (4), whereby the structures (2) are output with different, but channel-specific, graphic features corresponding to the respective assigned channels. The invention also relates to a light field microscope (5) for carrying out the method according to the invention.

The invention discloses a light field microscopic image super-resolution method and device based on deep learning, equipment and a medium. The method comprises the following steps: acquiring a light field image to be super-resolved of a sample; the light field image to be subjected to super-resolution is input into a convolutional neural network framework, super-resolution light field image data are obtained, the convolutional neural network framework comprises a parallax estimation network and a view fusion network, the parallax estimation network is used for estimating parallax between a target view and other views, and the view fusion network is used for fusing the target view and the other views. And the view fusion network is used for carrying out view fusion on the distorted image and outputting a super-resolution target view, and other views are distorted to the target view so as to realize alignment of spatial features, and the view fusion network is used for carrying out view fusion on the distorted image and outputting the super-resolution target view. Therefore, the light field image resolution can be improved under the condition that other hardware is not required to be changed, and the long-time observation phototoxicity is reduced.

The invention relates to a method for providing transfer functions in light field microscopy, in which a sample volume to be imaged is captured with a plurality of partial images (1 to 12xy) from different acquisition angles (Φ), and the respective point image transfer functions (PSF) are determined for the partial images (1 to 12xy) intended for a 3D reconstruction of the sample volume and are assigned to the partial images (1 to 12xy). A key feature is that the sub-images (1 to 12xy) are examined for existing symmetries in their underlying optical-technical data, and any symmetries found are recorded and stored along with a calculation formula. This formula allows the generation of corresponding point image transfer functions (PSFs) and / or processed point image transfer functions for further sub-images (2x to 12xy) that are symmetrical to the first sub-image (1 to 12), starting from a point image transfer function (PSF) and / or a processed point image transfer function of a first sub-image (1 to 12). The point image transfer functions (PSFs) or the processed point image transfer functions (PSFs) of the further sub-images (2x to 12xy) are generated only when needed and are stored temporarily.A processed point-image transfer function is obtained by subjecting each point-image transfer function to at least one mathematical operation. Additionally or alternatively, if required, symmetric subsets within the transfer function of at least one sub-mapping (1 to 12xy) can be generated from a first subset of the transfer function of that sub-mapping.

The application discloses a snapshot type light field microscope with large depth of field and a preparation method thereof, and belongs to the technical field of photoelectric devices and microscopic imaging, and comprises the following steps: design of a large-depth curved micro-lens array; femtosecond laser direct writing preparation of the curved micro-lens array; assembly and integration of the curved micro-lens array; preparation and image calibration of the snapshot type light field microscope; the method of the application is to replace the plane configuration with the curved micro-lens array to enhance the parallax, and replace the conventional spherical lens with a specially designed lens surface to expand the depth of focus, thus fundamentally solving the baseline limitation problem of the conventional plane micro-lens array, greatly improving the imaging depth of field and depth perception ability of the optical microscope, and retaining the imaging resolution at the level of the diffraction limit. The high-precision preparation of the special function surface curved micro-lens array can be realized by using the femtosecond laser additive manufacturing technology, and the prepared curved micro-lens array is integrated with a commercial objective through a customized mechanical adapter, so that the preparation of the snapshot type light field microscope with large depth of field is finally realized.