Image processing apparatus and method for enhancing phase distribution - Patents.com

Abstract

Input phase distribution (I(x i ) ... and the apparatus (1 and the method Obtain the input phase distribution and calculate the baseline (I2(x i )) as a baseline estimate (f(x i )), where the apparatus and method are configured to calculate an output phase distribution (O(x i )).

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus and method for enhancing the phase distribution of a digital input. [Background technology]

[0002] Highly transparent objects are difficult to optically interrogate with an observation device, e.g. a microscope. Since only a very small amount of light is absorbed and / or scattered, the contrast of a bright-field image of a highly transparent object is very small. Therefore, instead of using the light intensity, the phase shift imparted to the light passing through a highly transparent object is used for optical interrogation. The phase shift at different positions or pixels of the image is represented by a phase distribution.

[0003] Some methods for visualizing phase shifts, such as phase contrast and differential interference contrast, do not work in well plates. Furthermore, post-processing of phase shift images or phase distributions, such as cell counting or confluence measurements, requires, for example, segmentation of the recorded images, which is difficult to perform with differential interference contrast, phase contrast, or modulation contrast images.

[0004] Furthermore, interference based on the generation of phase distributions requires expensive hardware such as coherent light sources and careful separation and combination of optical paths, which prevents rapid integration into existing optical observation devices such as microscopes.

[0005] Quantitative phase imaging based on differential phase contrast or reconstruction from out-of-focus images requires knowledge of the phase transfer function, in particular the optical imaging system used to record the input image from which the phase contrast is derived. Since the phase transfer function has zero frequency, compensating the phase transfer function introduces low-frequency artifacts or noise. In the past, attempts have been made to compensate for these errors by varying the illumination of the object, for example by using annular illumination. However, this requires longer exposure times and cannot be used for well plates. Summary of the Invention [Problem to be solved by the invention]

[0006] It is therefore necessary to provide an apparatus and method by which the phase distribution can be reconstructed quickly and reliably, i.e. with as few artifacts as possible. [Means for solving the problem]

[0007] This need is addressed by an apparatus for noise reduction in an input phase distribution, where the apparatus is configured to obtain an input phase distribution and calculate a baseline estimate as an estimate of a baseline in the input phase distribution, where the apparatus is further configured to obtain an output phase distribution based on the baseline estimate and the input phase distribution.

[0008] This need is further addressed by a method for noise reduction or baseline estimation in an input phase distribution, the method including the steps of obtaining an input phase distribution, estimating a baseline in the input phase distribution to obtain a baseline estimate, and obtaining an output phase distribution from the baseline estimate and the input phase distribution.

[0009] Furthermore, the problem is solved by a non-transitory computer readable medium storing a program causing a computer to perform the claimed method by a computer program having a program code for performing the claimed method when the computer program is run on a processor, by an output phase distribution resulting from performing the claimed method, and / or by a neural network device trained by an input phase distribution and an output phase distribution, where the output phase distribution has been generated from input phase distribution data by the claimed method.

[0010] According to the claimed device and method, a baseline of the input phase distribution is estimated. This baseline represents the components of the input phase distribution that have low spatial frequencies and large spatial structure. By estimating the baseline, low spatial frequency artifacts introduced by zeros in the phase transfer function are estimated. The baseline estimate can thus be used to obtain a less noisy output phase distribution, for example by removing, in particular subtracting, the baseline estimate from the input phase distribution.

[0011] The input phase distribution is preferably represented as an N-dimensional digital array I(x i ), where N is an integer greater than 1, and x 1 ~x N indicates the dimension or coordinate of the array. For digital arrays, x i is an integer. In particular, the input phase distribution may be a two-dimensional real-valued array, represented digitally, for example by an array of floating-point numbers. The term "spatial" refers to the dimension of the array.

[0012] term x i is a discrete location x in an array that contains N location values. i (or a position vector for the location) 1 ;…;x N}. Location x imay be represented by a pixel or preferably a coherent set of pixels in the input phase distribution data. i For example, in the case of two-dimensional input phase distribution data, 1 ;x 2}, and in the case of three-dimensional input phase distribution data, a triplet of discrete location variables {x 1 ;x 2 ;x 3 In the i-th dimension, the array is i locations, i.e., xi={x i,1 ,…,x iMi In total, I(x i ) is (M 1 ×…×M N ) elements. In what follows, we will not mention specific locations or specific dimensions, so a location is simply x i As shown by:

[0013] The above devices and methods may be further improved by adding one or more of the features described below. Each of the following features may be added to the method and / or device of the present invention without relying on the other features. In particular, a person skilled in the art, knowing the device of the present invention, can configure the method of the present invention such that the method of the present invention can operate the device of the present invention. Moreover, as described below, each feature has an advantageous technical effect in itself.

[0014] The claimed apparatus and method start from the assumption that due to its low spatial frequency, the noise contribution in the input phase distribution results in mainly gradual changes that spread over a wider area of ​​the input phase distribution array. Starting from this assumption, the changes across the input phase distribution are, in one embodiment, calculated as the high spatial frequency content component I as follows: 1 (x i ) and the low spatial frequency noise component I 2 (x i ) may additionally be separated into I(x i )=I 1 (x i )+I 2 (x i )

[0015] The content component corresponds to an (unknown) phase distribution without (additive) large structure artifacts. Due to its low spatial frequency, the noise component I 2 (x i ) can be seen as a more or less smooth baseline of the input phase distribution, onto which the content components are superimposed as features with high spatial frequencies.

[0016] A baseline estimate is determined, thus I 2 (x i ) for the baseline estimate f(x i ) is obtained, for example, as a digital array of discrete values, then the output phase distribution O(x i ) may be obtained from the baseline estimate and the input phase distribution. In particular, the output phase distribution may be calculated by subtracting the baseline estimate from the input phase distribution as follows: O(x i )=I(x i )-f(x i )

[0017] Output phase distribution O(x i ) is preferably a function of dimensions N and M 1 ×…×M N The input phase distribution and / or baseline estimate may also be represented by a discrete digital data array having elements and thus preferably has the same dimensions as the input phase distribution and / or baseline estimate.

[0018] Preferably, the baseline estimate is calculated using a fit to the input phase distribution. In one particular example, the fit may be a polynomial fit to the input phase distribution. In particular, the baseline estimate may be represented by a K-th degree polynomial in any N dimensions i as follows:

number

[0019] The optimal value of the maximum polynomial order K depends on the smoothness required for the baseline estimate: for a smooth baseline, the polynomial order should be set as low as possible, while a higher order may be needed to fit a highly irregular background.

[0020] For polynomial fits, the baseline estimates are the polynomial coefficients a i,k However, the polynomial fit may be difficult to control and inaccurate since the only parameter that allows tuning to the input phase distribution data is the maximum polynomial order. The polynomial order can only take integer values. Therefore, it may not always be possible to find an optimal baseline estimate. A non-optimal polynomial fit may exhibit local minima in the baseline estimate, which may lead to annoying artifacts.

[0021] Therefore, according to another preferred embodiment, the fitting to the input phase distribution data may be a spline fit, in particular a smoothed spline fit. A spline fit usually provides more reliable results than a polynomial fit, since it is easier to control, for example in terms of smoothness, is more robust to noise and produces fewer artifacts. On the other hand, a spline fit is computationally more complex than a polynomial fit, since each pixel, or more generally the input phase distribution data, must be modified in order to minimize a least-squares minimization criterion.

[0022] A least-squares minimization criterion may be used to ensure that the baseline estimate is an accurate representation of the noise or baseline contribution in the input phase distribution data and to avoid fitting the baseline estimate to the content contribution.

[0023] The least-squares minimization criterion may include a penalty term that is used to penalize any undesirable behavior of the baseline estimate, such as belonging to content components that represent components of the input phase distribution data that have high spatial frequency content and / or that represent small scale structures.

[0024] According to one embodiment, the least squares minimization criterion M(f(x i )) can have the following forms: M(f(x i ))=C(f(x i ))+P(f(x i )) where C(f(x i )) is the cost function, P(f(x i )) is a penalty term. The least squares minimization criterion, the cost function, and the penalty term are preferably scalar values.

[0025] In one particular example, the cost function is i ) and the baseline estimate f(x i For example, ε(x i ) represents the difference term between the input phase distribution and the baseline estimate, ε(x i )=I(x i )-f(x i ) Cost function C(f(x i )) is L 2 Norm ||ε(x i )|| 2which is used here as a shorthand for the sum over all dimensions of the root-mean-square value of the sum of squared differences between the input phase distribution and the baseline estimate for the i-th dimension:

number

[0026] L 2 Norm ||ε(x i )|| 2 is a scalar value. An example of a cost function is: C(f(x i ))=||ε(x i )|| 2

[0027] To compute the baseline estimate of the phase input distribution, it is preferable to have a symmetric difference term so that representing small structural deviations, both positive and negative, from the baseline leads to an increase in the cost function.

[0028] To improve the accuracy of the baseline estimate, it may be advantageous to truncate the difference between the input phase distribution and the baseline estimate, for example, by using a truncated difference term. The truncated difference term reduces the influence that peaks in the input phase distribution have on the baseline estimate. Such a reduction occurs when the content component is greater than I(x i ) peaks. Due to the truncated difference term, peaks in the input phase distribution that deviate from the baseline estimate by more than a certain predefined threshold s are weighted less in the cost function due to the threshold fit, in particular the truncation of its penalty based on the spline fit. Thus, the baseline estimate only tracks such peaks to a limited extent.

[0029] The truncated difference term is φ(ε(x i )) As an example, the symmetric truncated quadratic term may have the form:

number

[0030] By using a truncated quadratic term, the cost function C(f(x i )) may preferably be expressed as follows:

number

[0031] The least squares minimization criterion M(f(x i )) i )) is the baseline estimate of the content component I 1 (x i ), a penalty is created that increases the value of the penalty term if the content components in the input phase distribution are represented by the baseline estimate.

[0032] For example, noise component I 2 (x i ) is assumed to be considered to have low spatial frequency, the penalty term may include a term that increases as the spatial frequency of the baseline estimate decreases.

[0033] Such a penalty term, in one embodiment, may be a roughness penalty term that penalizes non-smooth baseline estimates that deviate from a smooth baseline, thus effectively penalizing the fitting of data with high spatial frequencies.

[0034] According to another aspect, deviations from a smooth baseline may lead to increased values ​​in at least one of the first derivative, i.e., steepness or gradient, and the second derivative, i.e., curvature, of the baseline estimate. Thus, the roughness penalty term may include at least one of the first spatial derivative of the baseline estimate, in particular the square and / or absolute value of the first spatial derivative, and the second derivative of the baseline estimate, in particular the square and / or absolute value of the second spatial derivative. More generally, the penalty term may include any order of spatial derivative of the baseline estimate, or any linear combination of the spatial derivatives of the baseline estimate. Different penalty terms may be used in different dimensions.

[0035] In one embodiment, the penalty term P(f(x i )) is the baseline estimate f(x i ) and the characteristic length f l . Different characteristic lengths may be used for different dimensions.

[0036] For example, the roughness penalty term P(f(x i )) may be formed as follows:

number

number

[0037] Regularization parameter γ j is set depending on the spatial structure of the input phase distribution. It represents the maximum length scale that is considered as part of the content component. The feature scale f lj Structures in the input phase distribution with length scales larger than are considered to belong to the noise component. The regularization parameter can be predefined by the user and is preferably larger than zero.

[0038] The units of the second derivative are (x i (Unit of -2 , i.e. the length -2 or time -2 Therefore, γ j is the fourth power of the characteristic length

number

[0039] The first derivative ∂ j The penalty term based on

number

number

[0040] For various combinations of derivatives such as

number

number

[0041] ∂ i ∂ j f(x i,j ), i ≠ j, etc., the corresponding combination of characteristic lengths, e.g., fl i f l j may also be used.

[0042] In the discrete case, the derivatives may be efficiently calculated using convolution. For example,

number

number

[0043] However, the roughness penalty term P(f(x i )) is advantageously formed as follows:

number

[0044] This is a roughness penalty term that penalizes small-scale features and large gradients in the baseline estimate. The sum over j can use different penalty terms and regularization parameters in different dimensions. Here, x j and f(x i ) are both discrete, so the differential array ∂ j Note that differentiation can be performed by convolution with the operator ∂ j represents the discrete first derivative or gradient operator in dimension j, which may be represented by an array.

[0045] Instead of or in addition to the derivative or a linear combination of derivatives of the baseline estimate, the penalty term may include a feature extraction filter, in particular a linear filter or a linear combination of such filters. The feature extraction filter may be a Sobel filter, a Laplace filter, and / or an FIR filter, e.g., a high-pass or band-pass spatial filter with a passband for high spatial frequencies.

[0046] In such a general formulation, the penalty term for the jth dimension is given by the general operator ζ (j) and may be expressed as follows:

number

[0047] The least squares minimization criterion M(f(x i ) may be minimized using known methods. In one example, an iterative minimization scheme may be preferably used for the calculation of the minimization criterion. To perform the minimization, the baseline estimator engine may include a minimization engine. The minimization may include an iterative mechanism having two iteration stages.

[0048] The minimization scheme may, for example, include at least a portion of the computationally efficient LEGEND algorithm, which is described in Idier, J. (2001) "Convex Half-Quadratic Criteria and Interacting Variables for Image Restoration," IEEE Transactions on Image Processing, 10(7), pp. 1001-1009, and Mazet, V., Carteret, C., Bire, D, Idier, J., and Humbert, B. (2005) "Background Removal from Spectra by Designing and Minimizing a Non-Quadratic Cost Function," Chemometrics and Intelligent Laboratory Systems, 76, pp. 121-133. Both articles are incorporated herein by reference in their entirety.

[0049] The LEGEND algorithm preferably uses discrete auxiliary data d l (xi ) is introduced. The auxiliary data is updated at each iteration depending on the latest initial baseline estimate, the truncated quadratic term, and the input phase distribution data.

[0050] In the LEGEND algorithm, a least-squares minimization criterion, which contains only a cost function and no penalty term, is minimized using two iterative steps until a convergence criterion is met.

[0051] A suitable convergence criterion may be, for example, that the sum of the differences over all locations xi between the current baseline estimate and the previous baseline estimate is less than a predefined threshold.

[0052] In a further refinement, the convergence criterion may be expressed as:

number

[0053] As a starting step in the LEGEND algorithm, an initial set of baseline estimates is defined.

[0054] The LEGEND algorithm uses a first baseline estimate when polynomial fitting is used.

number

[0055] When spline fitting is used, the initial condition to start the LEGEND algorithm is d l (x i )=0,f(x i )=I(x i ), where the iteration begins by inputting in the second iteration step.

[0056] In the first iteration step, the auxiliary data may be updated as follows:

number

[0057] In the second iteration step, the previously calculated auxiliary data d l (x i ), the baseline estimate f from the previous iteration l-1 l-1 (x i ), and the penalty term P(x i ), the baseline estimate f l (x i ) is updated.

[0058] Baseline estimate f l (x i ) is the minimization criterion M(f(x i )) may be minimized.

[0059] In particular, the updated baseline estimate may be calculated in the second iterative LEGEND step using a formula such as:

number

[0060] In the second iterative step, the baseline estimate can be updated using a matrix calculation such as:

number

number

number

number

[0061] d l (x i ) and f l (x i ) are repeated until a convergence criterion is met.

[0062] According to another aspect, the second step of the LEGEND algorithm is modified using convolutions instead of matrix calculations, which significantly reduces the computational effort.

[0063] More specifically, preferably, the updated baseline estimate f l (x i ) is calculated directly by convolving the Green's function with the sum of the input phase distribution and the updated auxiliary data.

[0064] According to a more specific embodiment, the second iteration step of the LEGEND algorithm may be replaced by an iteration step as follows, where the updated baseline estimate f l (x i) is expressed as the Green function G(x i ) at the l-th iteration. f l (x i )=G(x i )*(I(x i )+d l (x i ))

[0065] This step significantly reduces the computational burden compared to the conventional LEGEND algorithm.

[0066] The reduction in computational load results from the fact that the convolution is calculated according to the second iteration step described above. This calculation can be efficiently performed using the FFT algorithm. Furthermore, the second iteration step can make full use of the array processor, such as a graphics processing unit or an FPGA, for the FFT algorithm. If the input phase distribution data and all other arrays are two-dimensional, the computational problem is reduced to (M x ×M y ) 2 From M x ×M y In the general N-dimensional case, the computational load is reduced to (M 1 ×…×M N ) 2 From the dimensional matrix calculation, (M 1 ×…×M N )-dimensional array to FFT calculation.

[0067] Therefore, baseline estimation can be performed very quickly, preferably in real-time, for two-dimensional input phase distribution data (2k×2k) image data arrays, where an image may be processed in under 50 ms.

[0068] In one particular embodiment, the Green's function may have the following formula:

number

number

[0069] In general, a discrete penalty array

number

number

number

number

number

[0070] The big advantage of the above Green's function is that it allows the penalty term P(f(x i)) can benefit from fast computation of the second iterative step in the minimization engine. Therefore, in embodiments using Green's functions, an optional penalty term to obtain a good baseline estimate may be used.

[0071] The general formula for the penalty term is as follows:

number

number

number

[0072] For example, if the penalty term is:

number

number

[0073] Output phase distribution O(x i) may, in one embodiment, already constitute an output image ready for further post-processing or display. However, it may be necessary to convert the output phase distribution into an image format of a color or monochrome image, for example by converting the real-valued phase distribution into an integer form within the value range of the respective standardized image format. The apparatus may include a display for displaying the output phase distribution.

[0074] According to another embodiment, the device comprises: k (x i ) and obtain the phase point spread function H(x i ) may be obtained. These two entities J k (x i ) and H(x i ), involves obtaining an entity in a form that allows this entity to be computed. For example, H(x i ) instead of its Fourier transform F(H(x i )), the phase transfer function may be obtained. The digital input images may be obtained, for example, from a camera connected to or part of the device. The digital images and the phase transfer function may be stored in a recording section, which may be connected to or part of the device, from which these images may be obtained. The device receives at least two digital input images J k (x i ) and the phase transfer function to obtain the input phase distribution I(x i ) The two input images should not be identical, but should preferably be slightly different.

[0075] In particular, the apparatus may be configured to record two or more input images using one of a different focal length, a different illumination direction, a different distance of the objective lens from the probe volume, and a different length of the optical path from the image sensor and / or detector to the probe volume for each of the two or more digital input images.

[0076] The two or more input images may be bright-field images, in particular monochrome bright-field images. According to one embodiment, an even number of input images are used, where the input images are recorded in pairs with equal changes in opposite directions, for example equal defocus amounts or equal illumination asymmetries. The device may be configured to automatically record the input images in pairs with equal changes in opposite directions. If two input images are used to obtain a phase distribution, an effective artifact reduction in the output phase distribution may already be achieved.

[0077] The objective lens and / or the image sensor / detector may be part of the apparatus.

[0078] To facilitate recording of two or more input images, the apparatus may include at least one of: a controllable aperture in the back focal plane; an illumination system configured to illuminate the probe volume from at least two different directions, preferably asymmetrically sequentially or simultaneously; an imaging system including at least one image sensor and / or one camera, the imaging system configured to record two or more digital input images, each of the two or more images being recorded at a different distance from the probe volume; an objective lens having at least one optical axis and movable relative to the probe volume along the optical axis; and a lens having a variable focal length.

[0079] The device and / or its sample volume may be configured to receive at least one well plate, which may contain one or more objects within one or more wells of the well plate.

[0080] The apparatus may further comprise an illumination system having a plurality of light sources, at least some of the light sources being configured to be controlled independently of one another, in particular the apparatus may be configured to automatically control the light sources to obtain different, preferably asymmetric, illumination for each of the two or more input images.

[0081] The illumination system may further comprise a collecting lens. The light source may be located at or near the back focal plane of the collecting lens. Alternatively or additionally, a variable aperture, iris or diaphragm may be located at or at least near the back focal plane of the collecting lens. The device may be configured to automatically control the variable aperture, for example to automatically set the aperture to two different operating states for recording each input image. Each operating state produces a different asymmetric illumination of the probe volume.

[0082] To obtain two or more input images with different amounts of defocus, one or more of the following embodiments of the apparatus may be used: A lens with a variable focal length may be located at the back focal plane of the objective lens or at a plane conjugate to the focal plane; An image sensor / detector may be movable to at least two positions with respect to the sample volume, each position being located at a different distance from the sample volume; The apparatus may include two or more image sensors / detectors, each located at a different distance from the probe volume and selectively used to record the input images.

[0083] In one embodiment, the intermediate image I I (x i ) can be used to convert two or more digital input images J, for example by subtracting two or more input images from each other. k (x i Correspondingly, the apparatus, and in particular its image processor, may be configured to subtract two or more input images from each other.

[0084] For thin and highly translucent objects such as objects on well plates, intermediate image I I (x i )Fourier transform F(I I (x i ) is expressed as the phase distribution I(x i )Fourier transform F(I(x i )) and the phase point spread function H(x i )Fourier transform F(H(x i )) and is related to. F(I I (x i ))=F(H(x i ))·F(I(x i ))

[0085] The phase transfer function may be known, for example, from experiments, numerical simulations and / or analysis, and may be obtained, for example, from a recording section. The phase transfer function is zero at zero frequency in Fourier space, so that the above H(x i ) is preferably solved using a regularization method, such as minimizing the expression: |F(I I (x i ))-F(H(x i ))·F(I(x i ))| 2 +R Here, R is the regularization function. The regularization function R is expressed as R = α|F(I(x i ))| 2 Formulations such as Tikhonov regularization may be used, which preferably represents the noise intensity and / or length scales in the intermediate images. The "true" phase distribution is generally better fitted with a smaller α, but in this case strong low-frequency (large structure) artifacts occur. If a larger α is chosen, the low-frequency artifacts will be smaller, but the overall representation of the phase distribution will not be of high quality.

[0086] The baseline removal as described above is applied to the phase distribution I(x i), we can unregularize the phase distribution using smaller values ​​for a, and thus obtain a very accurate representation of small spatial structures in the phase distribution. Artifacts in large-scale, low spatial frequency structures are eliminated if they are too small for the baseline estimate f(x i ) and can be removed from the input phase distribution, so it is acceptable. Therefore, the phase distribution calculated by the above regularization, preferably with small α, is used as the input phase distribution for baseline estimation and removal.

[0087] The apparatus may include a recording section for at least temporarily storing at least one digital input image, the intermediate image, the input phase distribution, the phase transfer function and the output phase distribution.

[0088] The device may include an input section, which may include, for example, one or more standard connectors and / or one or more standardized data exchange protocols. The input section may be adapted to receive an input phase distribution and / or two or more input images via one or more standard connectors and / or one or more data exchange protocols. For example, a recording section and / or a camera may be connected to the image input section.

[0089] The device may include an image output section, which may include, for example, one or more standard connectors and / or one or more standardized data exchange protocols. The image output section may be adapted to output an output image via one or more standard connectors and / or one or more data exchange protocols. For example, another computer, a network, and / or a display may be connected to the image output section.

[0090] The apparatus may further include an image processor, which may be configured to perform a baseline estimation.

[0091] The image processor may include a deblurring section, which may be adapted to subtract a baseline contribution, e.g. a baseline estimate, from the input image to calculate the output image.

[0092] The image processor may include a baseline estimator engine configured to calculate a baseline estimate by fitting to at least a subset of the input images. The baseline estimator engine may be configured to calculate a baseline estimate by fitting a least squares minimization criterion M(x i ) may include discrete representations of

[0093] The image processor may be configured to calculate an intermediate image from at least two digital input images, for example by subtracting two or more digital input images from one another. The device may be configured to obtain the intermediate image from the recording section, for example if the intermediate image has been calculated outside the device.

[0094] The image processor may further include a regularization engine that determines regularization parameters α and / or α|F(I(x i ))| 2 The input phase distribution may be calculated by minimizing a Tikhonov regularization, which includes a regularization function such as

[0095] The image processor, baseline estimator engine, deblurring section, and minimization engine may each be implemented in hardware, software, or a combination of hardware and software. For example, at least one of the image processor, baseline estimator engine, deblurring section, and minimization engine may be implemented, at least in part, by subroutines, sections of a general-purpose processor such as a CPU, and / or a special-purpose processor such as a CPU, GPU, FPGA, vector processor, and / or ASIC.

[0096] Another way of implementing one embodiment of the above method is to train an artificial neural network, for example a convolutional neural network, using a pair of input and output phase distribution data, where the output phase distribution data was generated using an embodiment of the method described above. A neural network device trained in this way can be considered as an implementation of the method used to generate the training pairs of input and output phase distribution data.

[0097] The apparatus may be part of an observation device such as a microscope. The input image does not have to be recorded by a camera and does not have to be a visible light image. For example, the apparatus may be part of and / or the method may be performed on two or more input images or input phase distributions from an X-ray microscope, an electron microscope, and / or a microscope using a coherent or non-coherent light source.

[0098] The invention will now be further described, by way of example only, using sample embodiments which are also illustrated in the drawings, in which like reference signs are used for features which correspond to one another in terms of at least one of function and design, and in which:

[0099] The combinations of features illustrated in the accompanying embodiments are for illustrative purposes only and may be modified. For example, features of the embodiments may be omitted if the technical effect associated with the feature is not required for a particular application. Similarly, a feature not illustrated as part of the embodiments may be added if the technical effect associated with that feature is required for a particular application.

[0100] As will be apparent from the above, any type of image data may be used. [Brief description of the drawings]

[0101] [Figure 1] FIG. 1 is a schematic diagram of one embodiment of an apparatus for baseline removal in an input phase distribution. [Diagram 2] FIG. 1 is a schematic diagram of a flow chart for baseline removal of an input phase distribution. [Diagram 3] FIG. 3 shows detail III of FIG. 2. [Figure 4] FIG. 1 is a schematic diagram of the device 1. [Diagram 5] FIG. 2 is another schematic diagram of the device 1. [Figure 6] FIG. 2 is another schematic diagram of the device 1. [Figure 7] FIG. 2 is another schematic diagram of the device 1. [Figure 8] FIG. 2 is another schematic diagram of the device 1. [Figure 9] FIG. 13 illustrates a sample input phase distribution. [Figure 10] FIG. 10 shows the output phase distribution obtained after removing the baseline estimate from the input phase distribution of FIG. [Figure 11] FIG. 2 is a schematic diagram showing input and output phase distributions together with noise (baseline estimate) and content components. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0102] The structure of the apparatus 1 will be described with initial reference to Fig. 1. The apparatus 1 may be an observation device 2, such as an endoscope or a microscope 2a. For the sake of explanation only, a microscope 2a is shown as an example of the apparatus 1. The microscope 2a may be any type of microscope, such as a bright field microscope, a SPIM microscope, a SCAPE microscope, a scanning microscope or a confocal microscope.

[0103] The apparatus 1 may include an imaging system 4 that receives an input image J(x i The system is adapted to capture input image data 6, preferably in digital format, representative of the input image J(x), for example by means of a camera 8. The camera 8 may comprise at least one image sensor and / or detector 9. More specifically, the camera 8 may be a monochrome camera, a CCD camera, a multispectral or hyperspectral camera, a photon counter or a photosensor. In the following, the input image J(x i ) is considered to be a monochrome image, although the input image may represent a single color channel of a color image, or may represent light intensities across all or a subset of the color channels of a color image.

[0104] Of course, other types of input image data 6 may be recorded using devices or sensors other than a camera, such as one or more microphones, vibration sensors, accelerometers, velocity sensors, antennas, pressure transducers, temperature sensors, capacitive sensors, magnetic sensors, radiographic, tomographic, ultrasonic, and any combination thereof.

[0105] The input image data 6 is stored at location x in the array. i It may be a digital array representing the measured physical quantity I, which depends on the term x i is a tuple {x 1 ;…;x N}. Location x imay be a pixel / voxel, or preferably a coherent collection of pixels / voxels, in the input image data 6. i For example, in the case of two-dimensional input image data 6, the discrete location variables of pixels {x 1 ;x 2}, and in the case of three-dimensional input image data, the discrete location variables {x 1 ;x 2 ;x 3 In the i-th dimension, the array is i It can contain x locations, i.e. i ={x i,1 ,…,x i,Mi In total, J(x i ) in the N-dimensional case, (M 1 ×…×M N ) elements. An input image J(x i ) may be represented by an array of three or more dimensions, where one dimension is the index of the color channels. Alternatively, each channel may be viewed separately and considered as a two-dimensional input image.

[0106] The object 10 is arranged in a probe volume 12 of the imaging system 4. The probe volume 12 is configured to receive the object 10 to be inspected by the device 1. For this purpose, the probe volume 12 is preferably arranged in a field of view 13 of the imaging system 4. An optical path 14 of the device 1 or the imaging system 4, respectively, extends from the probe volume 12 to at least one image sensor / detector 9.

[0107] The object 10 is preferably thin and highly translucent. The object 10 may be placed on a well plate 15. The object 10 may include living and / or non-living objects.

[0108] Due to the high translucency of the object 10, the input image J(x i) does not translate to a sufficient level of intensity for interrogation of the object. Therefore, it is preferable to interrogate the object 10 using phase information, which represents the difference in refractive index across the object 10, rather than intensity information, which represents the difference in transmittance across the object 10.

[0109] The apparatus 1 may include an illumination system 16 located on an opposite side of the probe volume 12 from the imaging system 4. Light from the illumination system 16 is directed through the object 10 and, if present, through a well plate 15 into the imaging system 4, which may be part of the apparatus 1, and guided along an optical path 14 to an image sensor / detector 9. The imaging system 4 includes an objective lens 18 and a tube lens 19.

[0110] The apparatus 1 may further include a storage section 20 adapted to at least temporarily contain the input image data 6. This storage section 20 may include a volatile memory, such as a cache memory of a CPU 22 of a computing device, such as a PC and / or a GPU 26, or a non-volatile memory. The storage section 20 may include a RAM, a disk drive, or a replaceable storage device, such as a USB stick or an SD card. The storage section 20 may include any combination of these types of memory.

[0111] Input phase distribution I(x i An image input section 28 may be provided to acquire an image signal (e.g., from a camera 8) using a standardized input connection device 30 such as a standardized data exchange protocol, a hardware connector connection and / or a wireless connection, or any combination thereof. Examples of standardized input connection devices 30 to which the camera 8 may be connected are an HDMI connector, a USB connector, and an RJ45 connector.

[0112] The apparatus 1 may further include an image output section 32 which may include standardized output connection devices 34, such as standardized data exchange protocols, hardware connectors and / or wireless connections, or any combination thereof, each configured to output output phase distribution data 36 to one or more displays 37. The output phase distribution data 36 is preferably expressed as an output phase distribution O(x i ) is represented by a discrete array of discrete values ​​representing

[0113] From input image data 6 to output phase distribution O(x i An image processor 38 may be provided to compute the image of the image captured by the image processor 38. The image processor 38 and its components may be at least partially hardware, at least partially software, and / or a combination of both hardware and software. For example, the image processor 38 may include at least one of the CPU 22 and the GPU 26 of the computing device 24, as well as sections that are coded in software and exist temporarily as structural entities as operational states in the CPU 22 and / or the GPU 26. The image processor 38 may also include additional hardware, such as one or more ASICs and / or FPGAs, specifically designed to perform the operations required for the apparatus and methods according to the invention.

[0114] The device 1 receives K different input images J 1…k (x i For example, the camera 8 may be configured to obtain two input images J 1 (x i ) and J 2 (x i ) can be recorded (preferably automatically) where at least one of illumination, exposure and focus is changed during recording, and preferably automatically.

[0115] The apparatus 1, and in particular the image processor 38, receives a number of input images J k (x i) and the phase point spread function H(x i ) to the input phase distribution I(x i ) may be further configured to obtain a phase point spread function H(x i ), i.e. its Fourier transform F(H(x i )), the phase transfer function may be stored in the recording section 20.

[0116] Input phase distribution I(x i To obtain the image of the at least two digital input images J, the device 1, in particular the image processor 38, may, for example, subtract two or more digital input images from each other to obtain the image of the at least two digital input images J. k (x i ) to intermediate image I I (x i ) In the simplest case, I I (x i )=J 1 (x i )-J 2 (x i Alternatively or cumulatively, the device 1 may, for example, receive an intermediate image I I (x i ) is calculated outside the device 1, the intermediate image I I (x i ) may be configured to obtain

[0117] The apparatus 1, and in particular the image processor 38, may further include a regularization engine 40, which may be a hardware section, a software section, or a combination of hardware and software sections of the image processor 38.

[0118] The regularization engine 40 calculates the intermediate image I I (x i ) and the phase point spread function H(x i ) based on, for example, I(x i ) by minimizing the regularization function over |F(I I (x i ))-F(H(x i))·F(I(x i ))| 2 +α|F(I(x i ))| 2 Input phase distribution I(x i ), where α is a regularization parameter, which is determined by the intermediate image I I (x i ) represents the noise level and / or length scale of the noise structure in the input phase distribution I(x i To calculate α, α is set to a user-determined value or a predetermined value based on the measured noise structure. Typically, α is set to a value between 10 -10 ~10 -6 Using values ​​of α in this range typically results in a phase distribution I(x i ) but introduces artifacts at low frequencies.

[0119] Below, we explain how to reduce these artifacts by the phase distribution I(x i ), thereby providing an accurate representation for low frequencies.

[0120] Alternatively or cumulatively, the device 1 may receive the input phase distribution I(x) from the recording section 20 if the input phase information has been calculated outside the device 1. i The acquired input phase information may be preferably arranged to acquire two or more bright field images J k (x i ) and / or contains mainly low (spatial) frequency noise components.

[0121] Input phase distribution I(x i ) is the input phase distribution I(x i ) which represents an improved estimate of the output phase distribution O(xi) 1 (x i ) and a noise component I, which contains artifacts and noise that are not part of the true image. 2(x i ) and, additionally, 1 (x i ) and I 2 (x i ) are unknown and must be estimated.

[0122] Output phase distribution O(x i ) to calculate the noise component I 2 (x i ) may be assumed to consist mainly of or contain components with low spatial frequencies. Thus, the noise component I 2 (x i ) represents a smooth baseline around which the content component I 1 (x i ) fluctuates at higher spatial frequencies. In contrast, content component I 1 (x i ) is not smooth and contains at least one peak and valley, and the noise component I 2 (x i ) are assumed to be composed of structures or features with length scales smaller than that of the noise component I 2 (x i ), i.e. by subtracting the baseline, noise is reduced and the content component I 1 (x i ) structure appears more clearly. The output phase distribution O(x i ) may be displayed as an image, so the noise component I 2 (x i ) makes the structure of the object 10 easier to see.

[0123] To reduce noise, noise component I 2 (x i ) is calculated. This estimate is the baseline estimate f(x i ) The baseline estimate f(x i ) is preferably the input phase distribution I(x i) and / or a discrete digital array having the same dimensions as the output phase distribution data 36. i ) is represented in FIG. 1 by baseline estimate data 44. The baseline estimate f(x i ) may be stored, at least temporarily, in the recording section 20. i ) is calculated, then for each location x i The input phase distribution I(x i ) to the baseline estimate f(x i ) to obtain the output phase distribution O(x i ) is obtained.

[0124] According to FIG. 1, the image processor 38 may include a baseline estimator engine 42 that receives input phase distribution data I(x i ) to obtain a baseline estimate f(x i ) Preferably, the fit to at least a subset of the input phase distribution data is a spline fit.

[0125] For a computationally efficient spline fit, the baseline estimator engine 42 may include a minimization engine 46, which may be, for example, a subroutine or a combination of hardwired algorithms and matching software. The minimization engine 46 may be configured to execute a minimization scheme, and to this end may have two iteration stages 48, 50.

[0126] In accordance with the present invention, the minimization engine 46 uses a convolution for the calculation of the baseline estimate 44 in the second iteration stage 50. This convolution can be calculated more efficiently on an array processor using an FFT, so the image processor 38 advantageously includes an array processor such as the GPU 26.

[0127] Referring to FIG. 2, the input phase distribution I(x i ), or optionally from a number K of input images J k (x i ), the output phase distribution O(x i ) will be described as they are performed by the apparatus 1. In operation, the image processor 38 includes a minimization engine 46.

[0128] In a first step 200, various parameters of the baseline estimator engine 42 that may need to be preset may be defined by a user, for example using the graphical user interface 62 (FIG. 1). These parameters define the input phase distribution I(x) to be executed by the baseline estimator engine 42. i ) can include the type of fit to the input phase distribution I(x i ) or a spline fit of the baseline estimate 44. The user can choose between a polynomial fit of the baseline estimate 44 to the content component I 1 (x i ) as noise component I 2 (x i ) can also be used to set a length scale.

[0129] The user also uses the camera to capture the input image J k (x i ) or to obtain the input image from the recording section 20. The user can further select between recording the intermediate image as the input image J k (x i ) or retrieved from the recording section 20. The user can further select whether the input phase distribution I(x i ) should be obtained from the recording section 20, or multiple input images J k (x i ) from the input image J k (x i ) and the number of input images J k (xi ) you can set parameters (defocus amount, lighting, exposure settings) that should be automatically changed between

[0130] In addition, the user can set the regularization parameters α and γ j We can choose a value for the regularization parameter γ j may be chosen differently for each dimension, as described further below.

[0131] In addition, the user may wish to consider the various penalty terms P(f(x i )) can be chosen between these penalty terms P(f(x i )) is the content component I at the baseline estimate of 44 1 The shape of the baseline estimate 44 is determined by penalizing the component representation of (xi). For example, the user may be presented with a selection of various penalty terms that penalize non-smooth features of the baseline estimate 44. For example, the penalty term may be a high-pass spatial frequency filter for the baseline estimate 44, which is large if the baseline estimate 44 contains components with high spatial frequencies. Other penalty terms may include the gradient of the baseline estimate 44. Another example of a penalty term may be the curvature of the baseline estimate 44. Additionally, feature extraction filters such as Sobel filters, Laplace filters, and / or FIR band-pass filters, high-pass or low-pass filters may be selected by the user as the penalty term. Additionally, any linear combination of the above may be selected. Different penalty terms may be selected for different dimensions.

[0132] The general expression for the penalty term is:

number

[0133] In the following, we consider a baseline estimate f(x i,m ) to select a gradient-based roughness penalty term.

number

[0134] This penalty term penalizes large gradients in the baseline estimate. j represents the first derivative or gradient in dimension j.

[0135] When using the gradient-based penalty terms above, the parameters to be specified by the user are the array of regularization parameters γ j The regularization parameter represents the square of the length scale under which structures in the input phase distribution are considered content.

number

[0136] Additionally, when selecting parameters for the baseline estimator engine 42, the user may also select the baseline estimate f(x i,m ) to determine the effect of large peaks on the baseline estimate f(x i,m ) also determines the shape of the symmetric quadratic term φ(ε(x i )) can be selected.

[0137] Finally, the user may select the convergence criteria and / or the threshold at which the convergence criteria must be reached.

[0138] For example, the user may select a symmetric truncated quadratic term such as:

number

[0139] A user uses device 1 to capture input image J k (x i ), you can select a set number of input images J 1…k (x i ) is recorded in step 202. Otherwise, the input image J k (x i ) is taken from Records Section 20.

[0140] The user selects intermediate image I I (x i ), we choose to compute the intermediate image I I (x i ) is input to a set number of input images J(x i ) by the device 1, for example by subtraction. If the user has chosen to use a stored intermediate image, it is downloaded from the recording section 20 in this step.

[0141] In step 206, the input phase distribution I(x i ) is calculated by the device 1, for example using Tikhonov regularization, or obtained from the recording section 20 if the user chooses to use an already calculated input phase distribution. In this step, the phase point spread function H(x i ) may be obtained from the records section 20.

[0142] In step 208, the baseline estimate is calculated based on the input phase distribution I(x i ) is calculated for

[0143] In particular, the calculation of the baseline estimate may involve an iterative quadratic minimization scheme 210 as described above until a convergence criterion 212 is met. In an embodiment, the convergence criterion used is as follows:

number

[0144] If the convergence criterion 212 is met, the baseline estimate f(x i ) is presumed to have been successfully calculated. Therefore, in step 214, the output phase distribution O(x i ) is, for example, the input phase distribution data I(x i ) to the baseline estimate f(x i ) is obtained by subtracting

[0145] Output phase distribution O(x i After the output phase distribution O(x) is calculated, post-processing operations 216 such as deconvolution may be performed on the output phase distribution data 36 and / or the output phase distribution O(x i ) may be displayed on the display 37 with or without post-processing.

[0146] Figure 3 shows the baseline estimate f(x i 2. In order to explain in more detail the minimization scheme 210 for computing θ(θ), detail III of FIG. 2 is shown. The minimization scheme 210 includes a first iteration stage 48 and a second iteration stage 50.

[0147] In principle, the minimization scheme 210 executed by the minimization engine 46 could be the LEGEND algorithm. However, according to the invention, it is advantageous to modify the second step of the LEGEND algorithm in order to significantly reduce the computational load.

[0148] In the illustrated embodiment, the second iteration stage 50 begins after initializing the data in step 300. At this point, the baseline estimate f(x i ) first estimate f 1 (x i ) is the Green function G(x i ) using the input phase distribution data I(x i ) convolution. f 1 (x i )=G(x i )*I(x i )

[0149] For the gradient-based penalty term used in this embodiment, the Green's function is defined as follows:

number

number

[0150] Then, in a first iteration stage 48, the auxiliary data d l (x i ) is the updated version of the current baseline estimate f l-1 (x i ) may be calculated as follows:

number

[0151] The parameter β is a constant that may be specified by the user. Typically, β is in the range of 0.4 to 0.499.

[0152] Then, in the second iteration stage 50, an updated baseline estimate f(x i ) is the updated auxiliary data d l (x i ) is calculated as follows: f(x i )=G(x i )*(I(x i )+d l (x i ))

[0153] In the next step, it is checked whether the convergence criterion 212 is met. If the convergence criterion 212 is not met, the minimization scheme 210 updates the updated baseline estimate f l (x i ) and proceed to iterative step 48 using

[0154] Next, various exemplary embodiments of the device 1 may be implemented to generate a plurality of input images J k (x i ) can be used. These various embodiments can be used individually or in any combination of the various features.

[0155] Figure 4 shows multiple input images J k (x i ) for each of the input images J using different illuminations. k (x i FIG. 1 shows a schematic diagram of an apparatus 1 configured to record an input image J ). In particular, the illumination may be asymmetric. Differential illumination may be achieved for example by using a preferably automatically controlled aperture 400 in the back focal plane 402 of a focusing lens 404. According to this embodiment, the apparatus 1 focuses an input image J , as indicated by the arrow 405 . k (x i) for each illumination, the apparatus is configured to automatically move the aperture 400 to a different position. The aperture 400 preferably comprises at least one asymmetrically arranged semi-transparent sector 406 and at least one complementary asymmetrically arranged opaque sector 408. The aperture 400 may comprise a magazine 410 of different asymmetric apertures 400. The apparatus may be adapted to move the different apertures 400 of the magazine 410 into the optical path 14 for recording a digital input image using the image sensor / detector 9.

[0156] 5, alternatively or additionally, the illumination system 16 may comprise individually controllable illumination elements 500, which are preferably also arranged in the back focal plane 402 of the collecting lens 404. According to this embodiment, the device 1 is configured to automatically switch on and / or off different sets of illumination elements 500 and / or to automatically switch to different illumination intensities, and thus to automatically control the illumination intensity of the input image J. k (x i ) different lighting settings are achieved for each of the

[0157] 6, alternatively or additionally, objective lens 18 and / or tube lens 19 may be supported by apparatus 1 for movement along its optical axis 600 or optical path 14. Apparatus 1 may include a drive mechanism 602, such as a ball screw drive, having an actuator 604, such as an electric motor. In this embodiment, apparatus 1 may move input image J, for example, by operating drive mechanism 602 as indicated by arrow 606. k (x i ) is configured to automatically move the objective lens 18 and / or the tube lens 19 to different positions relative to the probe volume 12.

[0158] Alternatively or additionally, the apparatus 1, and in particular the objective lens 18, may include a lens 700 having a variable focal length at a back focal plane 702 of the objective lens 18 or at a conjugate thereof. This is shown in FIG. 7. In this embodiment, the apparatus 1 may include a lens 700 having a variable focal length at a back focal plane 702 of the objective lens 18 or at a conjugate thereof. k (x i ) is configured to automatically change the focal length of lens 700.

[0159] 8, alternatively or additionally, the apparatus 1 may include two or more image sensors / detectors 9 arranged at different distances d1, d2 from the image plane 800 or probe volume 12. The image sensors / detectors 9 may be held fixedly or may be movable within the apparatus 1. A beam splitting device 802 may be used to simultaneously or alternatively direct light to at least two image sensors / detectors 9 and split the light path 14.

[0160] FIG. 9 shows the input phase distribution I(x i ) is shown. Low frequency artifacts are visible as very bright and very dark areas.

[0161] Figure 10 shows the baseline estimate f(x i The output phase distribution O(x i ) is shown. The dynamics of the image are more uniform than in FIG. 9 and the details of the object 10 are not obscured by low frequency artifacts.

[0162] As used in this specification, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0163] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step. Similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, microprocessor, programmable computer, or electronic circuitry. In some embodiments, any one or more of the crucial steps may be performed by such an apparatus.

[0164] Depending on certain implementation requirements, the embodiments of the present invention can be implemented in hardware or software. The implementation can be performed by a non-transitory recording medium, such as a digital recording medium, for example a floppy disk, a DVD, a Blu-ray, a CD, a ROM, a PROM and EPROM, an EEPROM or a FLASH memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to implement the respective methods. Thus, the digital recording medium can be computer readable.

[0165] Some embodiments of the present invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform any of the methods described herein.

[0166] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code which is operable to perform any of the methods when the computer program product is run on a computer, the program code may for example be stored on a machine readable carrier.

[0167] Another embodiment comprises the computer program for performing any of the methods described herein, stored on a machine readable carrier.

[0168] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing any of the methods described herein, when the computer program runs on a computer.

[0169] Therefore, another embodiment of the present invention is a recording medium (or data carrier or computer readable medium) containing a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital recording medium or recording medium is typically tangible and / or non-transitory. Another embodiment of the present invention is an apparatus as described herein, including a processor and a recording medium.

[0170] A further embodiment of the invention is therefore also a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or the sequence of signals being for example adapted to be transmitted via a data communication connection, for example the Internet.

[0171] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.

[0172] Another embodiment comprises a computer having the computer program installed thereon for performing any of the methods described herein.

[0173] Another embodiment of the invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, include a file server to transfer the computer program to the receiver.

[0174] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus. [Explanation of symbols]

[0175] 1 device 2 Observation Device 2a Microscope 4 Imaging System 6 Input image data I(x i ) 8. Camera 9 Detectors / Image Sensors 10. Object 12 Probe Volume 13 Field of view 14 Light path 15-well plate 16 Lighting System 18 Objective Lens 19 Tube Lens 20 Record Section 22 CPU 24 Computing Devices 26 GPU 28 Image Input Section 30 Input Connection Devices 32 Image Output Section 34 Output Connection Device 36 Output phase distribution data, O(x i ) 37 Display 38 Image Processor 40 Regularization Engine 42 Baseline Estimator Engine 44 Baseline estimate f(x i ) 46 Minimization Engine 48 First Iteration Stage 50 Second iteration stage 62 Graphical User Interface 200 Record or capture an input image 202 Setting baseline estimate parameters 204 Calculate or obtain intermediate images 206 Calculate or get input phase distribution 208 Calculate the baseline estimate 210 Minimization Scheme 212 Convergence Criteria 214 Obtaining the output phase distribution 216 Post-processing operations 300 Initialize baseline regularization 400,400a,400b aperture 402 Back focal plane 404 Condenser Lens 405 Arrow 406 Semi-transparent sector 408 Opaque Sector 410 Magazine 500 Lighting Elements 600 Optical axis 602 Drive mechanism 604 Actuator 606 Arrow 700 Lens with variable focal length 702 Back focal plane of objective lens 800 image planes 802 Beam splitter

Claims

1. Input phase distribution (I(x ｉ A device (1) for highlighting a The device (1) comprises: The input phase distribution (I(x ｉ )) and The input phase distribution (I(x ｉ )) at baseline (I ２ (x ｉ )) as the baseline estimate (f(x ｉ ) to calculate It is configured as follows: The device (1) comprises: - the baseline estimate (f(x ｉ )) and the input phase distribution (I(x ｉ Based on the output phase distribution (O(x ｉ ) and the baseline estimate (f(x ｉ ) by an iterative minimization scheme (210), The iterative minimization scheme (210) involves a least-squares minimization criterion (M(f(x ｉ ))) and the least squares minimization criterion includes a truncated quadratic term (φ(ε(x ｉ ))) Device (1).

2. The device (1) calculates the input phase distribution (I(x ｉ ) to the baseline estimate (f(x ｉ )) further configured to subtract 2. The device (1) according to claim 1.

3. The truncated quadratic term (φ(ε(x ｉ ))) is symmetric, 3. Apparatus (1) according to claim 1 or 2.

4. The iterative minimization scheme (210) includes a first iteration stage (48) and a second iteration stage (50); In the first iteration stage (48), auxiliary data (d ｌ (x ｉ ) is the baseline estimate (f ｌ－１ (x ｉ )) and the truncated quadratic term (φ(ε(x ｉ )) and the input phase distribution (I(x ｉ )) and, depending on, updated, In the second iteration stage (50), the baseline estimate (f(x ｉ ) is the input phase distribution (I(x ｉ ) and the updated auxiliary data (d(x ｉ )) and the discrete Green's function (G(x ｉ )) is calculated using the convolution of The device (1) according to any one of claims 1 to 3.

5. The device (1) comprises: - Two or more digital input images (J ｋ (x ｉ )) and - Phase transfer function F (H(x ｉ ) to get The method further comprises: The device (1) comprises: The two or more digital input images (J ｋ (x ｉ )) and the phase transfer function F(H(x ｉ )) and the input phase distribution (I(x ｉ )) The device (1) according to any one of claims 1 to 4.

6. The device (1) comprises: - the input phase distribution (I(x ｉ ) 6. Apparatus (1) according to claim 5.

7. The device (1) comprises: - said two or more digital input images (J ｋ (x ｉ )) are configured to subtract from each other, 7. Apparatus (1) according to claim 5 or 6.

8. The two or more digital input images (J ｋ (x ｉ ) is a bright field image, Device (1) according to any one of claims 5 to 7.

9. The device (1) comprises: - said two or more digital input images (J ｋ (x ｉ For each of the two or more digital input images (J), one of a different focal length, a different illumination, a different distance from the probe volume (12) to the objective lens (18), and a different length of the optical path (14) from the image sensor (9) to the probe volume (12) is used to obtain the two or more digital input images (J). ｋ (x ｉ )) configured to record Device (1) according to any one of claims 5 to 8.

10. The device (1) comprises: - a controllable aperture (400) in the back focal plane (702) of the objective lens (18); - an illumination system (16) configured to be switched between at least two different illumination settings, in which the probe volume (12) is illuminated differently in each of said at least two illumination settings; - an optical path (14) extending from said probe volume (12) to at least one image sensor (9) and having a variable length; - an imaging system (4) comprising at least one image sensor (9) which receives two or more digital input images (J ｋ (x ｉ a digital input image (J) that is movable relative to the imaging system (4) configured to record the two or more digital input images (J ｋ (x ｉ )) an imaging system (4) each of which is recorded at a different distance from said probe volume (12); at least two image sensors (9) arranged at different distances from said probe volume (12); an objective lens (18), and The objective lens (18) an optical axis (600) movable relative to said probe volume (12) along said optical axis (600); - a lens (700) with a variable focal length, having at least one of: The device (1) according to any one of claims 1 to 9.

11. A microscope (2), comprising a device (1) according to any one of claims 1 to 10.

12. Input phase distribution (I(x ｉ )) a method for highlighting The method is preferably arranged to operate an apparatus (1) according to any one of claims 1 to 10 or to operate a microscope according to claim 11, The method comprises: - Input phase distribution (I(x ｉ ) and - Baseline estimate (f(x ｉ ) to obtain the input phase distribution (I(x ｉ )) at baseline (I ２ (x ｉ ) estimating - the baseline (I ２ (x ｉ )) and the input phase distribution to obtain the output phase distribution (O(x ｉ ) and The method includes:

13. The input phase distribution (I(x ｉ )) the step of obtaining - at least two digital input images (J ｋ (x ｉ )) and the phase transfer function F(H(x ｉ ) and said at least two digital input images (J ｋ (x ｉ )) and the phase transfer function F(H(x ｉ ) from the input phase distribution (I(x ｉ ) and Further comprising:

13. The method of claim 12.

14. The method comprises: said at least two digital input images (J ｋ (x ｉ )) as a bright field microscope image, 14. The method of claim 13.

15. The method comprises: - Using a regularization function R, the phase transfer function F(H(x ｉ )) and said at least two digital input images (J ｋ (x ｉ ) from the input phase distribution (I(x ｉ ) 14. The method of claim 13.

16. The method comprises: - Using a minimization scheme (210), we calculate the baseline estimate (f(x ｉ ) 16. The method according to any one of claims 12 to 15.

17. A computer program comprising a program code for performing the method according to any one of claims 12 to 16, when the computer program is run on a processor.

18. Input phase distribution (I(x ｉ ) and causing a computer to execute the steps of the method according to any one of claims 12 to 16 to obtain the input phase distribution (I(x ｉ )) ｉ )) and the neural network program having parameters trained using training data corresponding to the

Images