Signal processing apparatus and method for enhancing a digital input signal - Patents.com

The signal processing device and method address noise introduced by signal recording systems by calculating a baseline estimate with a smaller length scale than the system response, effectively separating noise and content components to enhance signal quality and resolution.

JP7577696B2Active Publication Date: 2024-11-05LEICA MICROSYSTEMS CMS GMBH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2021576699
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-06-24
Filing Date
2020-06-19
Publication Date
2024-11-05
Estimated Expiration
2040-06-19

AI Technical Summary

Technical Problem

Existing signal recording systems introduce noise due to their non-ideal response to point sources, affecting the quality of one-dimensional, two-dimensional, and three-dimensional digital signals, including images and tomographic images, which existing methods struggle to effectively remove.

Method used

A signal processing device and method that calculates a baseline estimate with a length scale smaller than the system response's characteristic length, removing spatial features larger than this scale to enhance digital input signals, using techniques like polynomial and spline fitting, and least squares minimization with penalty terms to separate noise and content components.

Benefits of technology

This approach significantly reduces noise and improves signal quality by effectively separating noise from content components, enhancing the resolution and clarity of digital input signals.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007577696000047
    Figure 0007577696000047
  • Figure 0007577696000048
    Figure 0007577696000048
  • Figure 0007577696000049
    Figure 0007577696000049
Patent Text Reader

Abstract

The present invention calculates the system response (H(x i The digital input signal (I(x)) is recorded by a recording system (4) having i )), wherein the device receives a digital input signal and measures a characteristic length (f l ) containing a baseline estimate of the digital input signal (f(x i )) and output signals containing spatial features smaller than the feature length (O(x i ))), wherein the apparatus is configured to remove a baseline estimate from the digital input signal to obtain a characteristic length (c l ) and use a characteristic length smaller than the characteristic length of the system response to obtain the baseline estimate (f(x i )).
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a signal processing apparatus and method for enhancing a digital input signal. [Background technology]

[0002] When a signal is recorded using a recording system, the non-ideal response of the recording system to a point-like source generates an additive noise described by its impulse response or system response. This is independent of whether the signal is a time-dependent signal, such as a radar signal or an acoustic signal, and / or a location-dependent signal. It is also independent of the dimensionality of the signal. Noise is introduced in one-dimensional signals as well as two-dimensional signals, such as images, or multidimensional signals, such as three-dimensional data, for example tomographic images or images with multiple color channels. Much effort has been made to provide devices and methods that remove any artifacts from the signal. Summary of the Invention [Problem to be solved by the invention]

[0003] It is therefore an object of the present invention to provide an apparatus and method which are able to reduce noise in a signal and thereby improve the signal quality. [Means for solving the problem]

[0004] This problem is solved by a signal processing device for enhancing a digital input signal recorded by a recording system having a system response, where the device is configured to obtain the digital input signal, calculate a baseline estimate of the digital input signal that represents a baseline and includes spatial features of the digital input signal that are greater than or longer than a characteristic length, and remove the baseline estimate from the digital input signal to obtain an output signal that includes spatial features that are smaller than the characteristic length, where the device is configured to obtain a characteristic length of the system response and calculate the baseline estimate using a characteristic length that is smaller than the characteristic length of the system response.

[0005] Furthermore, the problem is solved by a signal processing method for enhancing a digital input signal recorded by a recording system having a system response, the system response having at least one characteristic length in at least one dimension or direction, in which spatial features having a characteristic length larger or longer than the at least one characteristic length of the system response are removed from the digital input signal to obtain an output signal.

[0006] The problem is further solved by a non-transitory computer readable medium storing a program causing a computer to execute the method as claimed; by a computer program with a program code for executing the method as claimed, when the computer program is executed on a processor; by an output signal which is a result of executing the method as claimed; and / or by a neural network device trained with input signal data and output signal data generated from the input signal data by the method as claimed.

[0007] Calculating and then removing the baseline estimate by using a length scale smaller than the characteristic length of the system response, i.e. smaller than the smallest feature expected in the input signal, unexpectedly improves noise reduction. This characteristic length may be expressed in spatial coordinates or in time. For example, if a time signal, such as a microphone signal, is recorded, the characteristic length of the system response, here the impulse response, may be measured in a time dimension, such as seconds. In a recording system, such as an imaging system that records digital images, where the system response is, for example, a point spread function, the characteristic length may be measured in spatial coordinates.

[0008] As practical examples of the input signal, the input signal may preferably include or consist of one of digital input image data, input sonar, acoustic and / or ultrasonic data, input radar data, spectral data including input spectroscopy and / or cepstrum, input microwave data, input vibration data such as seismogram data, input tomographic data of any kind of tomography, and statistical data such as stock exchange data, and may include or consist of any combination thereof. All of these may be integer-valued, real-valued, or complex-valued arrays of digital data. The input signal may be one of one-dimensional, two-dimensional, three-dimensional, and N-dimensional, where N≧1.

[0009] The output signal may suitably include or consist of one of digital output image data, output sonar, acoustic or ultrasonic data, output radar data, spectral data including output spectroscopy and / or cepstral, output microwave data, output vibration data such as seismogram data, statistical data such as stock trading data, and any combination thereof. The output signal may be real or integer valued. The output signal data may be one of one-dimensional, two-dimensional, three-dimensional and N-dimensional. The output signal data may be output for further processing.

[0010] For an input signal, a baseline estimate as described herein may be used to remove artifacts introduced from the system response.

[0011] In more general terms, the baseline estimation method described herein is not only for removing a baseline estimate from an input signal, but also for estimating the content I 1 (x i ) to the baseline (noise) component I 2 (x i ) which may then be processed and ultimately analyzed separately. For example, with spectral data, particularly hyperspectral data, large-scale baseline spectral features may be separated from small-scale spectral features and examined independently.

[0012] term x i is a discrete location x in an array that contains N location values. i (or a position vector for the location) 1 ;…;x N}. Location x i may be represented by data in an array or a suitable coherent data set representing the input signal data. i For example, in the case of two-dimensional input signal data, 1 ;x 2}, and in the case of three-dimensional input signal data, a triplet of discrete location variables {x 1 ;x 2 ;x 3 In the i-th dimension, the array is i locations, i.e., x i ={x i,1 ,…,x iMi In total, I(x i ) is (M 1 ×…×M N) elements. In what follows, we will not mention specific locations or specific dimensions, so a location is simply x i Denoted by x i can represent spatial and / or temporal dimensions. A combination of temporal and spatial dimensions may be present, for example, in a series of image time frames or a sequence of input signal data.

[0013] I(x i ) is a value representing the intensity and / or amplitude of electromagnetic radiation or sound at a location x i can be any value or combination of values ​​in I(x i ) can represent the intensity of the color R in a color space, for example the RGB space, or the combined intensity of two or more colors, for example (R+G+B) / 3 in the RGB color space. An input signal recorded as an input image, for example by a multispectral or hyperspectral camera, or by a microscope, in particular a scanning microscope, can contain four or more channels. The same is true for other types of input signals.

[0014] A two-dimensional input signal or a three-color RGB format input image is expressed as two-dimensional input signal data I(x i )={I R (x i );I G (x i );I B (x i )}, where I R (x i ) represents a value such as the intensity of color R, and I G (x i ) represents a value such as the intensity of color G, and I B (x i) represents a value such as the intensity of color B. Alternatively, each color may be considered as constituting a separate input signal and therefore separate input signal data. In the case where the input signal data is recorded as an input image using a multispectral or hyperspectral camera, four or more channels may be represented by the input signal data. Each channel may represent a different spectrum or spectral range of the light spectrum. For example, four or more channels may be used to represent the visible light spectrum. In the case where the imaged object includes a fluorescent material such as at least one fluorophore or at least one autofluorescent material, each channel may represent a different fluorescence spectrum. For example, in the case where multiple fluorescent fluorophores are present in the input signal, each fluorescence spectrum of one fluorophore may be represented by a different channel of the input signal. Furthermore, different channels may be used for the fluorescence selectively triggered by the illumination on the one hand and the autofluorescence that may be generated as a by-product or secondary effect of the triggered fluorescence on the other hand. The additional channels may cover the NIR and IR ranges. The channels may not necessarily include intensity data but may represent other types of data related to the image of the object, for example phase. In another example, a channel may contain fluorescence lifetime data representing the fluorescence lifetime after being triggered at a particular location in an image. Thus, in general, the input signal data may have the form: I(x i )={I 1 (x i );I 2 (x i );…;I C (x i )} where C is the total number of channels in the input signal data. Preferably, all channels have the same dimensions.

[0015] The above apparatus and method may be further improved by adding one or more of the features described below. Each of the following features may be added to the method and / or the apparatus without relying on the other features. In particular, a person skilled in the art, knowing the apparatus of the present invention, may configure the method of the present invention such that the method of the present invention operates the apparatus of the present invention. Moreover, as described below, each feature has an advantageous technical effect in itself.

[0016] In one embodiment, the input signal may be an input image, in particular a digital input image, and the output signal may be an output image, in particular a digital output image.

[0017] According to one embodiment, the apparatus may be an observation device, in particular a medical observation device such as an endoscope or a microscope, and the digital input signal may be a digital input image. The apparatus may be configured to at least temporarily capture and / or store the input image, to output the output signal as an output image, to remove, in particular subtract, a baseline estimate from the input signal to calculate the output signal, and to calculate the baseline estimate representing the noise component by fitting to at least a subset of the input signal using a least-squares minimization criterion including a penalty term.

[0018] Furthermore, the method may be a method for enhancing or improving the resolution of an input signal, in particular automatically, comprising the steps of estimating a baseline component of the input signal, obtaining a baseline estimate representative of a baseline in the image signal, and removing, in particular subtracting, the baseline estimate from the input signal to obtain an output signal, wherein the step of estimating the noise component preferably comprises a step of calculating the baseline estimate as a fit to at least a subset of the input signal by minimization with a least-squares minimization criterion, in which the least-squares minimization criterion preferably includes a penalty term.

[0019] For example, content components in the input signal, i.e. components to be separated for further processing, may be assumed to have high spatial or temporal frequencies, e.g., responsible for changes in the input signal that occur over short distances or time periods. Noise components are assumed to have low frequencies, i.e., resulting in mainly gradual intensity changes that spread over a wider area of ​​the input signal. Thus, noise components are reflected in the baseline of the input signal.

[0020] Starting from this assumption, the change over the input signal is the high frequency content component I 1 (x i ) and the low-frequency noise component I 2 (x i ) may additionally be separated into I(x i )=I 1 (x i )+I 2 (x i )

[0021] Due to low temporal or spatial frequencies, noise component I 2 (x i ) can be considered as a more or less smooth baseline, on which the content components I 1 (x i ) is superimposed as a high frequency feature.

[0022] In particular for images, the frequencies to be considered for separating the baseline component from the content component may be spatial frequencies. Of course, the same considerations apply if temporal frequencies are considered instead of spatial frequencies. In this case, the input signal may represent, for example, a spectrum, a cepstrum, or multiple spectra or cepstra.

[0023] Thus, the baseline estimate may be used equally to extract and / or remove small-scale (baseline) or large-scale (baseline) signal content in the spatial or frequency domain.

[0024] Input signal I(x i ) is the system response H(x i ) is assumed to have a recording system. This recorded input signal I(x i ) is the true image I T (x i ) results from I(x i )=I T (x i )*H(x i )

[0025] If the recording system is an imaging system, the system response corresponds to the point spread function of the imaging system. In other systems that record a time trace of a signal, such as an acoustic recording system, the system response corresponds to the impulse response.

[0026] The system response is given by the characteristic length c l and this characteristic length c l may be, for example, the half-width at half maximum (HWHM), the full-width at half maximum (FWHM), or any other ratio of width to maximum. In the time domain, the characteristic length corresponds to the duration. The system response and / or its characteristic length may be predetermined, i.e., known prior to the apparatus described herein, for example, by experiment, numerical simulation, analysis, or any combination thereof, and may be stored within the apparatus. Alternatively or additionally, the system response or its characteristic length may be calculated, for example, by the Lucy-Richardson algorithm.

[0027] A baseline estimate is determined, thus I 2 (x i ) for the baseline estimate f(x i ) is obtained, the output signal O(x i ) may be obtained from the baseline estimate and the input signal. In particular, the output signal may be calculated by subtracting the baseline estimate from the input signal as follows: O(x i )=I(x i )-f(x i )

[0028] Output signal O(x i ) is preferably a function of dimensions N and M 1 ×…×M N The baseline estimate may also be represented by a discrete digital data array having N elements and, therefore, preferably, has the same dimensions as the input signal and / or the baseline estimate. The baseline estimate may have N dimensions and (M 1 ×…×M N ) elements, and thus has the same dimensions as the input signal.

[0029] The baseline may be estimated using a fit to the input signal. Computationally, this fit, i.e., the baseline estimate, is calculated as a discrete baseline estimate f(x i )

[0030] In one embodiment, the apparatus may be configured to calculate the baseline estimate using a regularization length scale, e.g., as further described below. For example, the baseline estimate may be calculated using a regularization, such as Tikhonov regularization, where the regularization length scale may be used as a regularization parameter. The regularization length scale is preferably a characteristic length f l Depends only on

[0031] The apparatus may be configured to calculate the baseline estimate using a least-squares minimization criterion, the minimization criterion preferably including the baseline estimate and the feature length, and in particular a penalty function of the least-squares minimization criterion may include the feature length.

[0032] In another embodiment, the least squares minimization criteria can include a suitable dimensionless combination of the characteristic length and at least one derivative of the baseline estimate, preferably of dimension x iis a derivative or combination of derivatives with respect to

[0033] In one particular example, the fit may be a polynomial fit to the input signal. In particular, the baseline estimate may be represented by a K-th degree polynomial in any N dimensions i as follows:

number

[0034] The optimal value of the maximum polynomial order K depends on the smoothness required for the baseline estimate: for a smooth baseline, the polynomial order should be set as low as possible, while a higher order may be needed to fit a highly irregular background.

[0035] For polynomial fits, the baseline estimate data is used to estimate the polynomial coefficients α i,k However, the polynomial fit can be difficult to control and inaccurate since the only parameter that allows adjustment to the input signal data is the maximum polynomial order. The polynomial order can only take integer values. Therefore, it may not always be possible to find an optimal baseline estimate. A non-optimal polynomial fit may exhibit local minima in the baseline estimate, which can lead to annoying artifacts.

[0036] Therefore, according to another preferred embodiment, the fitting to the input signal data may be a spline fit, in particular a smoothed spline fit. A spline fit usually provides more reliable results than a polynomial fit, since it is easier to control, for example in terms of smoothness, is more robust to noise, and produces fewer artifacts. On the other hand, a spline fit is computationally more complex than a polynomial fit.

[0037] A least squares minimization criterion is preferably used to calculate the baseline estimate, which should be minimized for the fit. The exact formulation of the least squares minimization criterion determines the properties of the fit, and therefore the properties of the baseline estimate data. An inappropriate selection of the least squares minimization criterion may cause the baseline estimate to not represent the noise component with sufficient accuracy.

[0038] To ensure that the baseline estimate data is an accurate representation of the noise or baseline components in the input signal data and to avoid the baseline estimate fitting to content components, the least-squares minimization criterion can include a penalty term that is used to penalize undesirable behavior in the baseline estimate, such as representing components of the input signal data that have high frequency content and therefore are believed to belong to the content components.

[0039] According to one embodiment, the least squares minimization criterion M(f(x i )) can have the following forms: M(f(x i ))=C(f(x i ))+P(f(x i )) where C(f(x i )) is the cost function, P(f(x i )) is a penalty term. The least squares minimization criterion, the cost function and the penalty term are preferably scalar values.

[0040] In one particular example, the cost function is j ) and the baseline estimate f(x i For example, ε(x i ) represents the difference term between the input signal and the baseline estimate, ε(x i )=I(x i )-f(x i ) Cost function C(f(x i )) is L 2 Norm ||ε(x i )|| 2 which is used here as a shorthand for the sum over all dimensions of the root-mean-square value of the sum of squared differences between the input signal and the baseline estimate in the i-th dimension:

number

[0041] L 2 Norm ||ε(x i )|| 2 is a scalar value. One example of a cost function is a quadratic difference term: C(f(x i ))=||ε(x i )|| 2

[0042] To improve the accuracy of the baseline estimate, it may be advantageous to truncate the difference between the input signal and the baseline estimate, for example, by using a truncated difference term. The truncated difference term reduces the influence that peaks in the input signal data have on the baseline estimate data. Such a reduction occurs when the content component is greater than I(x i) is assumed to lie at the peaks of the truncated difference term. Because of the truncated difference term, peaks in the input signal data that deviate from the baseline estimate by more than a predefined constant threshold s are "ignored" in the cost function by truncating the fit to a threshold, in particular its penalty based spline fit. Thus, the baseline estimate only follows such peaks to a limited extent. The truncated quadratic expression may be symmetric or asymmetric. The truncated difference term is hereafter referred to as φ(ε(x i )).

[0043] In some applications, the content components may be contained only, or at least primarily, in peaks in the input signal, e.g., bright spots in an image. This may be reflected by choosing truncated quadratic terms that are asymmetric and allow the fit, particularly the spline fit, to follow the valleys rather than the peaks in the input signal data. For example, an asymmetric truncated quadratic term φ(ε(x i )) may be in the following form:

number

[0044] In certain other applications, where valleys, i.e. dark regions or regions with low values ​​in the input signal, are also to be considered as content components, a symmetric truncated quadratic expression may be used instead of an asymmetric truncated quadratic expression. For example, the symmetric truncated quadratic expression may have the following form:

number

[0045] When using a truncated quadratic, the cost function C(f(x i )) may preferably be expressed as follows:

number

[0046] The least squares minimization criterion M(f(x i )) i )) is the baseline estimate of the content component I 1 (x i ) is applied to data that is considered to belong to the baseline estimate. A penalty is created that increases the value of the penalty term when the content components in the input signal are represented by the baseline estimate.

[0047] For example, noise component I 2 (x i ) is assumed to be considered to have low spatial frequency, the penalty term may include a term that increases as the spatial frequency of the baseline estimate increases.

[0048] Such a penalty term, in one embodiment, may be a roughness penalty term that penalizes non-smooth baseline estimate data that deviates from a smooth baseline, thus effectively penalizing the fitting of data with high spatial frequencies.

[0049] In particular, the baseline estimate is the characteristic length f l This characteristic length f l represents the length scale above which features in the input signal are considered noise and below which features in the input signal are considered content. For example, the length of one feature, f l If is set to 0.01 mm, then all features smaller than 0.01 mm are considered content and therefore penalized if included in the baseline estimate. This feature length may be used for the regularization length scale of regularizations such as Tikhonov regularization to compute the baseline estimate.

[0050] According to another aspect, deviations from a smooth baseline may lead to increased values ​​in at least one of the first derivative, i.e., steepness or gradient, and the second derivative, i.e., curvature, of the baseline estimate. Thus, the roughness penalty term may include at least one of the first spatial derivative of the baseline estimate, in particular the square and / or absolute value of the first spatial derivative, and the second derivative of the baseline estimate, in particular the square and / or absolute value of the second spatial derivative. More generally, the penalty term may include any order of spatial derivative of the baseline estimate, or any linear combination of the spatial derivatives of the baseline estimate. Different penalty terms may be used in different dimensions.

[0051] In one embodiment, the penalty term P(f(x i )) is the baseline estimate f(x i ) and the characteristic length f l Different characteristic lengths may be used for different dimensions.

[0052] For example, the roughness penalty term P(f(x i )) may be formed as follows:

number

[0053] This roughness penalty term penalizes large rates of change in the slope of the baseline estimate, i.e., high curvature, and thus favors smooth estimates. Here, γ j is the regularization parameter,

number

number

[0054] The first derivative ∂ j The penalty term based on

number

number

[0055] For various combinations of derivatives such as

number

number

[0056] ∂ i ∂ j f(x i,j ), i ≠ j, etc., the corresponding combination of characteristic lengths, e.g., f l i f l j may also be used.

[0057] Regularization length scale γ jdepends on the structure of the input signal, i.e. the maximum length scale that is considered to be part of the content component. Structures of the input signal with a length scale larger than the characteristic length represented by the regularization length scale are considered to belong to the noise component. The regularization length scale is preferably larger than zero. However, to reach a better resolution, the regularization length scale γ j is the characteristic length f l is the system response H(x i ) characteristic length c l It is preferable to select the frequency f so that it is smaller than f l i <c l j , where c l j is the characteristic length of the system response in the direction of dimension j. If the characteristic length is less than about 40%-50% of the characteristic length of the system response, or equivalently, f l i ≦0.4…0.5c l j Even better results in terms of resolution may be obtained if: Note that both the characteristic length and the feature length may be constant in all dimensions.

[0058] In the discrete case, the derivatives may be efficiently calculated using convolution. For example,

number

number

[0059] However, the roughness penalty term P(f(x i )) is advantageously formed as follows:

number

[0060] This is a roughness penalty term that penalizes small scale features and large gradients in the baseline estimate. The sum over j can use different penalty terms in different dimensions. Here, x j and f(x i ) are both discrete, so the differential array ∂ j Note that differentiation can be performed by convolution with the operator ∂ j represents the discrete first derivative or gradient operator in dimension j, which may be represented by an array.

[0061] Instead of or in addition to the derivative or a linear combination of derivatives of the baseline estimate, the penalty term may include a feature extraction filter, in particular a linear filter or a linear combination of such filters. The feature extraction filter may be a Sobel filter, a Laplace filter and / or an FIR filter, e.g., a high-pass or band-pass spatial filter with a passband for high spatial frequencies.

[0062] In such a general formulation, the penalty term for the jth dimension is given by the general operator ζ (j) and may be expressed as follows:

number

[0063] The least squares minimization criterion M(f(x i) may be minimized using known methods. In one example, an iterative quadratic or semi-quadratic minimization scheme may be preferably used. To perform the minimization, the baseline estimator engine may include a minimization engine. The minimization may include an iterative mechanism having two iteration stages.

[0064] The minimization scheme may, for example, include at least a portion of the computationally efficient LEGEND algorithm, which is described in Idier, J. (2001) "Convex Half-Quadratic Criteria and Interacting Variables for Image Restoration," IEEE Transactions on Signal Processing, 10(7), pp. 1001-1009, and Mazet, V., Carteret, C., Bire, D, Idier, J., and Humbert, B. (2005) "Background Removal from Spectra by Designing and Minimizing a Non-Quadratic Cost Function," Chemometrics and Intelligent Laboratory Systems, 76, pp. 121-133. Both articles are incorporated herein by reference in their entirety.

[0065] The LEGEND algorithm preferably uses discrete auxiliary data d(x i ) These auxiliary data are updated at each iteration depending on the latest initial baseline estimate, the truncated quadratic term and the input signal data.

[0066] In the LEGEND algorithm, a least-squares minimization criterion, which contains only a cost function and no penalty term, is minimized using two iterative steps until a convergence criterion is met.

[0067] A suitable convergence criterion is, for example,i The sum of the differences between the current baseline estimate and the previous baseline estimate over a period of time may be less than a predetermined threshold.

[0068] In a further refinement, the convergence criterion may be expressed as:

number

[0069] As a starting step in the LEGEND algorithm, an initial set of baseline estimation data is defined.

[0070] The LEGEND algorithm uses a first baseline estimate when polynomial fitting is used.

number

[0071] When spline fitting is used, the initial condition to start the LEGEND algorithm is d(x i )=0,f(x i )=I(x i ), where the iteration begins by inputting in the second iteration step.

[0072] In the first iteration step, the auxiliary data may be updated as follows:

number

[0073] In the second iteration step, the previously calculated auxiliary data d l (x i ), the baseline estimate f from the previous iteration l-1 l-1 (x i ) and the penalty term P(x i ), the baseline estimate f l (x i ) is updated.

[0074] Baseline estimate f l (x i ) is the minimization criterion M(f(x i )) may be minimized.

[0075] In particular, an updated baseline estimate may be calculated in the second iterative LEGEND step using a formula such as:

number

[0076] In the second iterative step, the baseline estimate data can be updated using a matrix calculation such as:

number

number

number

number

[0077] d l (x i ) and f l (x i ) are repeated until a convergence criterion is met.

[0078] According to another aspect, the second step of the LEGEND algorithm is modified using convolutions instead of matrix calculations, which significantly reduces the computational effort.

[0079] More specifically, preferably, the updated baseline estimate f l (x i ) is calculated directly by convolving the Green's function with the sum of the input signal and the updated auxiliary data.

[0080] According to a more specific embodiment, the second iteration step of the LEGEND algorithm may be replaced by an iteration step as follows, where updated baseline estimate data f l (x i ) is expressed as the Green function G(x i ) at the l-th iteration. f l (x i )=G(x i )*(I(x i )+d l (x i ))

[0081] This step significantly reduces the computational burden compared to the conventional LEGEND algorithm.

[0082] A reduction in the computational load results from the fact that the convolution is calculated according to the second iteration step described above. This calculation can be efficiently performed using the FFT algorithm. Furthermore, the second iteration step can make full use of array processors such as graphics processing units or FPGAs for the FFT algorithm. If the input signal data and all other arrays are two-dimensional, the computational problem is reduced to (M x ×M y ) 2 From M x ×M y In the general N-dimensional case, the computational load is reduced to (M 1 ×…×M N ) 2 From the dimensional matrix calculation, (M 1 ×…×M N )-dimensional array to FFT calculation.

[0083] Therefore, baseline estimate removal may be performed very quickly, preferably in real-time, for two-dimensional input signal data: for a (2k x 2k) data array, baseline estimate removal may be performed in 50 ms or less.

[0084] In one particular embodiment, the Green's function may have the following form:

number

number

[0085] Preferably, a discrete penalty array

number

number

number

number

number

[0086] The big advantage of the above Green's function is that it allows the penalty term P(f(x i )) can benefit from fast computation of the second iterative step in the minimization engine. Therefore, in embodiments using Green's functions, an optional penalty term to obtain a good baseline estimate may be used.

[0087] The general formula for the penalty term is as follows:

number

number

number

[0088] For example, if the penalty term is:

number

number

[0089] The apparatus may include a storage section, which may be configured to at least temporarily store the input signal.

[0090] The device may include a signal input section, which may include, for example, one or more standard connectors and / or one or more standardized data exchange protocols, such as HDMI, USB, RGB, DVI, etc. The signal input section may be adapted to receive an output signal via one or more standard connectors and / or one or more data exchange protocols. For example, a storage device and / or a sensor, such as a camera, may be connected to the signal input section.

[0091] The device may include a signal output section, which includes, for example, one or more standard connectors and / or one or more standardized data exchange protocols, such as HDMI, USB, RGB, DVI, etc. The signal output section may be adapted to output an output signal via the one or more standard connectors and / or the one or more data exchange protocols. For example, another computer, a network and / or a display may be connected to the signal output section.

[0092] The apparatus may further include a signal processor, which may be configured to calculate the baseline estimate.

[0093] The signal processor may include a baseline removal section. The baseline removal section may be adapted to subtract a baseline component, such as a baseline estimate, from the input signal to calculate an output signal. In some applications, assuming that the content component is present in the low frequency components of the input signal, the baseline estimate already represents the content component. In this case, it is not necessary to remove the baseline estimate from the input signal. Rather, the baseline estimate may be output for display or further processing.

[0094] The signal processor may include a baseline estimation engine configured to calculate a baseline estimate by fitting to at least a subset of the input signal. The baseline estimation engine may be configured to calculate a baseline estimate by fitting a least squares minimization criterion (M(x i )) can include discrete representations of

[0095] The signal processor, baseline estimation engine, baseline removal section, and minimization engine may each be implemented in hardware, software, or a combination of hardware and software. For example, at least one of the signal processor, baseline estimator engine, baseline removal section, and minimization engine may be implemented, at least in part, by subroutines, sections of a general purpose processor such as a CPU, and / or a special purpose processor such as a CPU, GPU, FPGA, vector processor, and / or ASIC.

[0096] Another way of implementing any of the above embodiments of the apparatus and method is to train an artificial neural network, such as a convolutional neural network, using pairs of input and output signal data, where the output signal data was generated using an embodiment of the method described above. A neural network device trained in this way can be considered as an implementation of the method used to generate the training pairs of input and output signal data.

[0097] According to another aspect, the neural network device generates an output signal O(x i ), where the characteristic length f is smaller than at least one characteristic length of the system response. l and a characteristic length f that is greater than at least one characteristic length of the system response. l and contained in the digital input signal, and

[0098] Input signal I(x i Note that if the input signal I(x) is not pre-convolved or deconvolved, then calculating and / or removing the baseline provides the best results. i Deconvolution provides the best results if ) has been preprocessed by baseline removal according to the invention.

[0099] According to one embodiment, the signal processor is adapted to detect different characteristic lengths f l 1…K and the corresponding penalty term to decompose a single digital input signal I(x i ) to generate two or more output signals O 1…K (x i ) and calculate the deconvoluted output signal J(x i ) to obtain multiple output signals O 1…K (x i ) for each different feature length, a different regularization length scale and therefore a different penalty term is obtained. Thus, the baseline estimate is l 1…K Different baseline estimates f l 1…K (x i ) is removed from a separate copy of the input signal to obtain multiple output signals: k =I(x i )-f k (x i ), where K denotes the total number of output signals and therefore the total number of different characteristic lengths.

[0100] Therefore, the first output signal O 1 (x i ) is the first characteristic length f l 1 and the second output signal O 2 (x i ) is the first characteristic length f l 1 The second characteristic length f l 2 Then, the multi-image deconvolution may be calculated using 1 (x i ) and O 2 (x i ) can be implemented.

[0101] Multi-image deconvolution is an image fusion technique that reduces shadows and blurring caused by scattering between the sample to be analyzed and the light sheet (see for example the publication "Efficient Bayan-based multiview deconvolution" (Stephan Preibisch et al., Nat Methods., 2014 June, 11(6): 645-648)). Furthermore, multi-image deconvolution has been described in the publications "Multi-image deconvolution improves signal-to-noise ratio on gated stimulated depletion microscopy" (Castello, Diaspro, Vicidomini,(2014)), "Unified blind method for multi-image super resolution and single / multi-image blur deconvolution" (Applied Physics Letters 105.23: 234106; in Faramarzi, Rajan, Christensen (2013)), "Richardson-Lucy deconvolution as a general tool for combining images with complementary strength" (Ingaromo, York, Hoogendorn et al. (2014)), "Multiframe deconvolution, super-resolution, and saturation correction via incremental deconvolution" (Wiley online library, and Harmeling, Sra, Hirsch, Schoenkopf et al.). EM" (IEEE 17th International Conference on Signal Processing, 2010).

[0102] Multi-image deconvolution has a characteristic length c l Characteristic length f smaller than l, especially 0.4…0.5c l Characteristic length f smaller than l At least one output signal O(x i ) and characteristic length c l Characteristic length f l At least another output signal O(x i ) and, unexpected improvements in input signal resolution may result.

[0103] In addition, the individual system responses H 1…K (x i ) are multiple output signals O k (x i Further improvement in resolution may be obtained if the input signal I(x i ) may be used. However, since the effective system response changes after baseline removal, the output signal O k (x i ) a new estimate of the system response for each of

[0104] Preferably, the system response is calculated using a maximum a posteriori (MAP) distribution to estimate the output signal O k (x i ) is estimated from

[0105] More specifically, the following MAP may be maximized: P(T k (x i ),H k (x i )|O k (x i ))

[0106] This is an unknown property T k (x i ), i.e. the true original image and a given output signal O, which is used as input for multi-image deconvolution. k (x i) and the probability for . Assuming Poisson probabilities, the solution to this maximization is an iterative algorithm known as blind Richardson-Lucy deconvolution. This algorithm is described in the publication (Fish, Brinicombe, Pike (1995)) "Blind deconvolution by means of the Richardson-Lucy algorithm", J. Opt. Soc. Am. A. (12) 1.

[0107] An embodiment will now be further described, by way of example only, using a sample embodiment which is also shown in the drawings, in which like reference signs are used for features which correspond to one another in terms of at least one of function and design.

[0108] The combinations of features illustrated in the accompanying embodiments are for illustrative purposes only and may be modified. For example, features of the embodiments may be omitted if the technical effect associated with the feature is not required for a particular application. Similarly, a feature not illustrated as part of the embodiments may be added if the technical effect associated with that feature is required for a particular application.

[0109] As will be apparent from the above, any type of image data may be used. [Brief description of the drawings]

[0110] [Figure 1] 1 is a schematic diagram of one embodiment of an apparatus for reducing noise in an input signal. [Diagram 2] FIG. 1 is a schematic diagram of a flow chart for baseline removal in an input signal. [Diagram 3] FIG. 3 shows detail III of FIG. 2. [Figure 4] 1 is a schematic diagram of a method for improving the resolution of an input signal. [Diagram 5] FIG. 2 shows a sample of an input signal. [Figure 6]FIG. 6 is a diagram showing an output signal based on the input signal of FIG. 5. [Figure 7] FIG. 7 shows detail VII of FIG. 5. [Figure 8] FIG. 8 shows detail VIII of FIG. [Figure 9] FIG. 9 shows the intensity distribution along the sections of FIGS. 7 and 8. [Figure 10] FIG. 6 illustrates a deconvoluted output signal based on the multi-image deconvolution based on the input signal of FIG. 5. [Figure 11] FIG. 11 shows detail XI of FIG. [Figure 12] 2 is a schematic diagram of input signal data, content components in the input signal data, noise components in the input signal data, baseline estimate data, and output signal data; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0111] The structure of the apparatus 1 will be described with initial reference to Fig. 1. This apparatus 1 may be a medical observation device 2, such as an endoscope or a microscope 2a, or any other device configured to capture high quality images, such as an aerial or astronomical observation device. For the purposes of explanation only, a microscope 2a is shown as an example of the apparatus 1. For the purposes of the description of the following embodiments, no distinction is made between an endoscope or a microscope.

[0112] The device 1 may include a recording system 4, which is adapted to capture input signal data 6. If the recording system 4 is an imaging system, it may for example be provided with a camera 8 arranged to record a digital input image, preferably in digital format. The camera may include an image sensor 9. The camera 8 may be a CCD camera, a multispectral camera or a hyperspectral camera which records the input signal data 6 in a number of channels 10, where each channel 10 preferably represents a different light spectral range, from infrared to ultraviolet. In the following, the input signal data 6 will also be referred to as the input signal I(xi ) is written as

[0113] Of course, other types of input signal data 6 may be recorded using devices or sensors other than a camera, for example a point detector as in the case of a confocal microscope, one or more microphones, vibration sensors, accelerometers, velocity sensors, antennas, pressure transducers, temperature sensors, capacitive sensors, magnetic sensors, by radiographic, tomographic, ultrasonic techniques, and any combination thereof.

[0114] In the case of a CCD camera recording in RGB color space, for example, three channels 10 may be provided, e.g., an R channel, a G channel and a B channel, to represent the visible light input signal of the object 12. In the case of a multispectral or hyperspectral camera, a total of four or more channels 10 may be used in at least one of the visible, infrared, near infrared and ultraviolet light ranges.

[0115] The recording system 4 records the system response H(x i ), and the system response H(x i The system response H(x) may be known, for example from measurements, or may be estimated using a MAP algorithm such as the Richardson-Lucy algorithm. i ) is the characteristic length c l and this characteristic length c l may be known and / or directionally dependent. Alternatively, as shown in FIG. 1 Directional c l 1 and x 2 Directional c l 2 As shown, different dimensions have different characteristic lengths c l may be present. This characteristic length may be the HWHM length or the FWHM length. i can indicate a spatial or a temporal dimension.

[0116] In the case of an imaging system as a recording system 4, the object 12 is placed in a probe volume 13. This probe volume may be configured to receive the object 12 to be inspected by the device 1. In the case of an imaging system as a recording system, the probe volume is preferably placed in a field of view 14 of the imaging system. The object 12 may include animate and / or inanimate objects. The object 12 may further include one or more fluorescent materials, such as at least one fluorophore 15.

[0117] A multispectral or hyperspectral camera may have a channel 10 for each different fluorescence spectrum of a fluorescent material in the object 12. For example, each fluorophore 15 may be represented by at least one channel 10 that matches the fluorescence spectrum triggered by the illumination system 16. Alternatively or additionally, separate channels 10 may be provided for autofluorescence spectra, or for spectra of secondary fluorescence triggered by fluorescence excited by the illumination system 16, or for lifetime fluorescence data. Of course, the illumination system 16 may also emit white light or light of any other composition, or alone, without triggering fluorescence in the object 12.

[0118] The microscope 2 may be adapted to excite the fluorescence of fluorophores 15 in the object 12 with light having a suitable fluorescence excitation wavelength, for example by means of an illumination system 16. The illumination system 16 may be arranged on an opposite side of the probe volume 13 to the recording system 4 and / or on the same side as the recording system 4.

[0119] If the illumination system 16 is arranged on the same side as the recording system 4, its light may be guided through a lens 17 through which the input signal I(x i) is also obtained. The illumination system 16 may include or consist of one or more flexible light guides for directing light onto the object 12 from one or more different directions. A suitable blocking filter (not shown) may be placed in the light path in front of the camera 8, for example to suppress glare. In case of fluorescence, the blocking filter preferably blocks only the illumination wavelength, allowing the fluorescence of the fluorophores 15 in the object 12 to pass through to the camera 8.

[0120] If the illumination system is placed opposite the probe volume 13 , its light can pass through the probe volume 13 .

[0121] It will be apparent that the input signal data 6 can be captured by any kind of camera or photodetector, without being limited in general.

[0122] The input signal data 6 is two-dimensional if a single channel 10 is included in a two-dimensional image. The input signal may have higher dimensions, such as three or more dimensions, if more than two channels 10 are included and / or if the input signal data 6 represents a three-dimensional array, such as a three-dimensional image. A signal may be one-dimensional if it represents, for example, a time trace or a one-dimensional spatial measurement.

[0123] The three-dimensional input signal data 6 may be recorded by the device 1 using, for example, light field techniques, Z-stacking in a microscope, three-dimensional reconstruction of images obtained by a SCAPE microscope and / or by a SPIM microscope. Another source of three-dimensional input signal data may be tomographic images. In the case of three-dimensional images, each plane of the three-dimensional input signal data 6 may be considered as a two-dimensional input signal 6. Again, each plane may include multiple channels 10.

[0124] Each channel 10 may be considered as a separate two-dimensional image or signal, or alternatively, multiple channels may be interpreted collectively as a multi-dimensional array.

[0125] The input signal data 6 is a discrete real-valued quantity such as intensity or phase or an integer-valued quantity I(x i ), where x i represents a location in the input signal data 6, and I is the quantity at that position that constitutes the input signal. i is a vector of n dimensions (N≧1) and represents discrete locations x i A tuple {x 1 ;…;x N}. Location x i may be a pixel or preferably a coherent collection of pixels in the input signal data. i For example, in the case of two-dimensional input signal data, 1 ;x 2}, and in the case of three-dimensional input signal data, a triplet of discrete location variables {x 1 ;x 2 ;x 3 In the i-th dimension, the array is i locations, i.e., x i ={x i,1 ,…,x iMi In total, I(x i ) is (M 1 ×…×M N ) elements.

[0126] The apparatus 1 may further comprise a storage section 20 adapted to at least temporarily contain the input signal data 6. This storage section 20 may comprise a volatile memory, such as a cache memory of a CPU 22 of a computing device 24, such as a PC and / or a GPU 26, or a non-volatile memory. The storage section 20 may further comprise a RAM, a hard disk drive, or a replaceable storage section, such as a USB stick or an SD card. The storage section 20 may comprise any combination of these types of memory.

[0127] A signal input section 28 may be provided for acquiring input signal data 6, for example from a camera 8. This signal input section 28 may include standardized connection means 30, such as a standardized data exchange protocol, a hardware connector and / or a wireless connection, or any combination thereof. Examples of standardized connectors to which the camera 8 may be connected are an HDMI connector, a USB connector and an RJ45 connector.

[0128] The device 1 may further comprise a signal output section 32 which may include standardized connection means 34, such as a standard data exchange protocol, a hardware connector and / or a wireless connection, each configured to output output signal data 36 to one or more displays 37. The output signal data 36 preferably has the same dimensions as the input signal data 6 and is expressed as the output signal O(x i ) is a discrete array of discrete values ​​forming

[0129] Input signal I(x i ) to the output signal O(x i A signal processor 38 may be provided to compute the signal (signal, signal ...

[0130] Before continuing with further description of FIG. 1, with reference to FIG. 12, a step of estimating the baseline and, if necessary, removing the baseline will be described. iFor baseline removal, the signal processor 38 may include a baseline removal section 40.

[0131] Input signal I(x i ) is a content component I that contains the content of interest and that may correspond in particular to the unknown “true” input signal. 2 (x i ) and a noise component I, which contains artifacts and noise that are not part of the true input signal. 1 (x i ) and the noise component I 2 (x i ) consists of or contains components with mainly low spatial frequencies. The noise component therefore represents a smooth baseline around which the content component I 1 (x i ) fluctuates at higher spatial frequencies. 2 (x i ) is assumed to be smooth and have large length scales, in contrast to the content component I 1 (x i ) is not smooth, contains peaks and / or valleys, and has a length scale or characteristic length f smaller than the noise component. l It is assumed that the image is composed of structures or features having the following characteristics: content, noise, and baseline. As shown in FIG. 12, subtracting the noise component, i.e., the baseline, enhances image contrast and reduces noise. However, in some situations, the situation may be reversed, with content present in large-scale structures and noise present in small-scale structures. 1 (x i ) also I 2 (x i ) is also not known and must be estimated.

[0132] To emphasize the input signal, the noise component I 2 (x i ) is calculated. This estimate is the baseline estimate f(x i), i.e., the data that represents the baseline estimate. The baseline estimate f(x i ) is a discrete, preferably real-valued array having the same dimensions as the input signal data 6 and / or the output signal data 36. i ) is represented in FIG. 1 by baseline estimate data 44. The baseline estimate f(x i ) may be at least temporarily resident in the storage section 20. Once the baseline estimates have been calculated, the input signal I(x i ) to the baseline estimate f(x i ) to obtain an output signal (herein O(x i ) is obtained.

[0133] According to FIG. 1, the signal processor 38 may include a baseline estimator engine 42 that derives a baseline estimate f(x) by fitting to at least a subset of the input signal data 6. i ) Preferably, the fit to at least a subset of the input signal data is a spline fit.

[0134] For a computationally efficient spline fit, the baseline estimator engine 42 may include a semiquadratic or quadratic minimization engine 46, which may be, for example, a subroutine or a combination of hardwired algorithms and software. The minimization engine 46 may be configured to perform a quadratic or semiquadratic minimization scheme and, to this end, may include two iteration stages 48, 50.

[0135] Preferably, the minimization engine 46 uses a convolution for the calculation of the baseline estimate data 44 in the second iteration stage 50. Because this convolution can be more efficiently calculated on an array processor using an FFT, the signal processor 38 advantageously includes an array processor such as the GPU 26. In operation, the signal processor includes the minimization engine 46.

[0136] Referring to FIG. 2, the input signal I(x i ) to the output signal O(x i ) are described as they are performed by the device 1. It should be noted that in the case of an input image as an input signal, preferably each channel 10 is processed separately.

[0137] In a first step 60, various parameters of the baseline estimator engine 42 that need to be preset may be defined by a user, for example using a graphical user interface 62 (FIG. 1). These parameters may include the type of fit to the input signal data 6 that should be performed by the baseline estimator engine 42. For example, the user may choose between a polynomial fit and a spline fit of the baseline estimate data 44 to the input signal data 6. The user may also define a characteristic length f that may be used to separate the content components from the noise components. l You can also set it.

[0138] In addition, the user may wish to consider the various penalty terms P(f(x i )) can be chosen between these penalty terms. 1 (x 1 ) to determine the shape of the baseline estimate.

[0139] For example, the user may be presented with a selection of various penalty terms that penalize non-smooth characteristics of the baseline estimate data 44. For example, the penalty term may be a high-pass spatial frequency filter for the baseline estimate data 44, which is larger if the baseline estimate data 44 contains components with high spatial frequencies. Other penalty terms may include the gradient of the baseline estimate data 44. Another example of a penalty term may be the curvature of the baseline estimate data 44. Additionally, feature extraction filters such as Sobel, Laplace and / or FIR band-pass, high-pass or low-pass filters may be selected by the user as the penalty term. Additionally, any linear combination of the above may be selected. Different penalty terms may be selected for different dimensions or different channels of the input signal data 6.

[0140] The general expression for the penalty term is:

number

[0141] In the following, a user may provide baseline estimate data f(x i,m ) or 44 to select a gradient-based roughness penalty term.

number

[0142] This penalty term penalizes large gradients in the baseline estimate data.j represents the first derivative or gradient in dimension j.

[0143] When using the gradient-based penalty terms above, the parameters to be specified by the user are the sequence of regularization length scales γ j The regularization length scale may further include a length scale, i.e., a characteristic length

number

number

[0144] Surprisingly, the characteristic length f used as or in the regularization length scale in the penalty function l But the characteristic length c l is smaller than f l <c l , f l i <c l i When the input signal resolution is significantly improved.

[0145] These results indicate that the characteristic length f l is the characteristic length of the system response, c l If it is set to less than half or 40 percent of f l ≦0.4…0.5c l , f l i ≦0.4…0.5c l i Further improvement could be achieved in the case of

[0146] When selecting parameters for the baseline estimator engine, the user may select a symmetric quadratic term φ(ε(x i )) and.

[0147] For example, the user may select an asymmetric truncated quadratic term such as:

number

[0148] Finally, the user can select the convergence criterion and / or the threshold t above which the convergence criterion must be reached.

[0149] After the initial parameters for the baseline estimator engine 42 are set, data is initialized for an iterative minimization scheme 66 in step 64 .

[0150] Thereafter, an iterative minimization scheme 66 is performed by the minimization engine 46 until a convergence criterion 68 is met. In an embodiment, the convergence criterion used is as follows:

number

[0151] If the convergence criteria 68 are met, then the baseline estimate data 44 is presumed to have been successfully calculated. Thus, in step 70, the output signal data O(x i) to obtain the baseline estimate data f(x i ) is the input signal data I(x i ) is subtracted.

[0152] Output signal O(x i ) is calculated, post-processing operations 72, such as deconvolution, may be performed on the output signal data 36.

[0153] More specifically, the above calculation of the baseline estimate is done for the same input signal I(x i ) may be performed at least twice, each time with a different characteristic length f l , and hence the different regularization length scales y j is used. This results in K different baseline estimates for K different regularization length scales. These K different baseline estimates f k (x i ) are separately removed, in particular subtracted, from the input signal, thereby producing a plurality of output signals O k (x i ) is obtained (where k = 1…K). That is, O k (x i )=I(x i )-f k (x i ).

[0154] K output signals O k (x i ) with different regularization length scales, y j,k or equivalent characteristic length f l j,k At least one output signal is preferably a characteristic length c l Characteristic length f smaller than 40% to 50% l and more preferably, at least one output signal is calculated using a regularization length scale including a characteristic length f l >c l , f l i >c l iThe characteristic lengths are calculated using a regularized length scale with ±20% deviation between the characteristic length and the feature length may still be considered equal.

[0155] Output signal O k (x i ), an estimate of each modified system response is calculated using a MAP algorithm, such as the Richardson-Lucy algorithm. The deconvolved output signal J(x i A multi-signal or multi-image deconvolution is performed on the output signal O to obtain deconvolved output signal data 36 representative of k (x i ) for k=1...K. The deconvoluted output signal J(xi) is preferably i ) is a discrete digital real-valued or integer-valued array with the same dimensions as

[0156] Output signal O k (x i ) and the deconvolved output signal J(x i ) may be displayed on the display 37 with or without post-processing.

[0157] 3, detail III of FIG. 2 is shown to explain in more detail the minimization scheme 66. The minimization scheme 66 includes a first iteration stage 48 and a second iteration stage 50.

[0158] In principle, the minimization scheme 66 executed by the minimization engine 46 may be the LEGEND algorithm. However, it is advantageous to modify the second step of the LEGEND algorithm, which may be modified to significantly reduce the computational load.

[0159] In the illustrated embodiment, the second iteration stage 50 is entered after initializing the data in step 64. At this point, the first estimate of the baseline estimate data, f 1 (x i) is the Green function G(x i ) is calculated using a convolution of the input signal data with f 1 (x i )=G(x i )*I(x i )

[0160] For the gradient-based penalty term used in this embodiment, the Green's function is defined as follows:

number

number

[0161] Then, in a first iteration stage 48, the auxiliary data d l (x i An updated version of ) may be calculated using the current baseline estimate data 44 as follows:

number

[0162] The parameter α is a constant that may be specified by the user.

[0163] Then, in the second iteration stage 50, the updated baseline estimate data 44 is added to the updated auxiliary data d l (x i ) is calculated as follows: f(x i )=G(x i )*(I(x i )+d l (x i ))

[0164] In the next step, it is checked whether the convergence criterion 68 is met. If the convergence criterion 68 is not met, the minimization scheme 66 uses the updated baseline estimate data f l (x i ) and proceed to iterative step 48 using

[0165] FIG. 4 shows the input signal I(x i ) to obtain the deconvoluted output signal J(x i 4 shows a sample process flow for obtaining an output signal O. 1 (x i ) is the first characteristic length f l 1,i In this case, preferably, f l 1,i <0.4…0.5c l 1,i In step 402, another output signal O 2 (x i ) has a second characteristic length f l 2,i For this, it is preferably calculated using f l 2,i ≧c l 2,i In optional step 404, another output signal O 3 (x i ) is calculated using a different characteristic length from the characteristic lengths in other output signals using further different characteristic lengths. Step 404 may be repeated to obtain more output signals s. The steps performed in steps 400 to 404 correspond to those described above with reference to Figures 1 to 3.

[0166] In step 406, the system response is determined based on the output signal O k (x i ) are estimated for each of

[0167] In step 408, the deconvoluted output signal J(x i), a multi-image deconvolution is performed using the multiple output signals calculated in steps 400-404 and the estimated system response in step 406.

[0168] 5 to 9 show resolution enhancement using baseline removal for a real image of a paramecium.

[0169] FIG. 5 shows the input signal I(x i ) and an input image recorded using a DMi8 widefield microscope equipped with an HCX FLUOTAR 100x / 1.30 oil lens with a numerical aperture of 1.3 and peak emission at 520 nm.

[0170] FIG. 6 shows the output signal O(x i ) and the output image, where the characteristic length f l , i.e. f l =0.4c l The feature length f is used in the regularization length scale of the penalty term involving the squared first derivative of the baseline estimate as described above. The improvement in resolution enhancement is clearly visible.

[0171] 7 and 8 show detail VII of FIG. 5 and detail VIII of FIG. 6, respectively. Here, the noise component I, which is considered as blur, 2 (x i ) is removed in the output signal.

[0172] This is verified by FIG. 9, which shows the intensity distribution along the line 700 in each of FIGS. 7 and 8. The baseline estimate f(x i ), the intensity peaks that represent the content are preserved, and the output signal O(x i ) is seen to be better.

[0173] FIG. 10 shows the output signal O(x i) as the first input. As the second input to the multi-image deconvolution, the second output signal is a signal with a characteristic length f l =c l The deconvoluted output signal J(x i A multi-image deconvolution was performed on these images to obtain the imaging described above to obtain . The results obtained are improved in that the signal-to-noise ratio is not as drastically reduced as when using output signals with small characteristic lengths. This can be seen in FIG. 11, which shows detail XI of FIG. 10.

[0174] As a general note on the application of baseline removal and multi-image deconvolution, the dimensionality of the data may be changed by rearranging the array. For example, two-dimensional data may be rendered as one or more sets of one-dimensional data. This may be achieved by concatenating successive rows or columns backwards to each other. Additionally, three-dimensional data may be reduced to two-dimensional data by concatenating one of the successive faces backwards to each other. By using this principle recursively, any N-dimensional data may be reduced to one- or two-dimensional data to which the above-mentioned schemes may be applied.

[0175] Conversely, any one-dimensional array may be arranged as a two- or higher-dimensional array by simply breaking it down into smaller one-dimensional arrays, preferably of the same length, and indexing those smaller arrays in a two- or higher-dimensional scheme. Additionally, any type of data may be viewed and displayed as image data or an image, for example, by assigning a grayscale intensity to each value of the input signal data and displaying it in a two- or three-dimensional arrangement, as described above.

[0176] As used in this specification, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0177] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step. Similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, microprocessor, programmable computer, or electronic circuitry. In some embodiments, any one or more of the crucial steps may be performed by such an apparatus.

[0178] Depending on certain implementation requirements, the embodiments of the present invention can be implemented in hardware or software. The implementation can be performed by a non-transitory storage medium, such as a digital storage medium, for example a floppy disk, DVD, Blu-ray, CD, ROM, PROM and EPROM, EEPROM or FLASH memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to implement the respective methods. Thus, the digital storage medium can be computer readable.

[0179] Some embodiments of the present invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform any of the methods described herein.

[0180] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code which is operable to perform any of the methods when the computer program product is run on a computer, the program code may for example be stored on a machine readable carrier.

[0181] Another embodiment comprises the computer program for performing any of the methods described herein, stored on a machine readable carrier.

[0182] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing any of the methods described herein, when the computer program runs on a computer.

[0183] Therefore, another embodiment of the invention is a storage medium (or data carrier or computer readable medium) containing a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital storage medium or stored medium is typically tangible and / or non-transitory. Another embodiment of the invention is an apparatus as described herein, including a processor and a storage medium.

[0184] A further embodiment of the invention is therefore also a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or the sequence of signals being for example adapted to be transmitted via a data communication connection, for example the Internet.

[0185] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.

[0186] Another embodiment comprises a computer having the computer program installed thereon for performing any of the methods described herein.

[0187] Another embodiment of the invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, include a file server to transfer the computer program to the receiver.

[0188] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus. [Explanation of symbols]

[0189] 1 device 2 Observation Device 2a Microscope 4. System of Record 6 Input signal data I(x i ) 8. Camera 9. Image Sensor 10 Channels 12 Object 13 Probe Volume 14 Field of view 15 Phosphors 16 Lighting System 17 Objective Lens 20 Memory Section 22 CPU 24 Computing Devices 26 GPU 28 Signal Input Section 30 Signal input section connections 32 Signal Output Section 34 Signal output section connections 36 Output signal data O(x i ) 37 Display 38 Signal Processor 40 Baseline Removal Section 42 Baseline Estimator Engine 44 Baseline estimate f(x i ) 46 Quadratic or semi-quadratic minimization engines 48 First Iteration Stage 50 Second iteration stage 60 Setting baseline estimation parameters 62 Graphical User Interface 64 Initialization of the minimization engine and / or scheme 66 Quadratic or semiquadratic minimization schemes 68 Convergence Criteria 70 Calculation of output signal data 72 Post-processing operations 400~404 Remove blur 406 Estimating System Response 408 Multi-Image Deconvolution 700 Line

Claims

1. System response (H(x i A digital input signal (I(x)) is recorded by a recording system (4) having a i )) for enhancing a given signal, the signal processing device (1) comprising: - receiving a digital input signal; represents the baseline of the digital input signal and has a characteristic length (f l ) a baseline estimate of the digital input signal (f(x i )) and - an output signal (O(x i and removing the baseline estimate from the digital input signal to obtain It is configured as follows: The signal processing device (1) - the system response (H(x i ) to obtain the characteristic length (cl) of - the system response (H(x i )) using a characteristic length smaller than the characteristic length of the baseline estimate (f(x i )) and - calculating a number of output signals (O 1 . . . K (x i )) from said digital input signals (I(x i )); - calculating each of said plurality of output signals based on a baseline estimate (f(x i )) having a different characteristic length (f l ); - estimating a system response (H(x i )) for each of said plurality of output signals (O 1 . . . K (x i )); performing a multi-image deconvolution of said plurality of output signals (O1...K(xi)) to obtain a deconvolved output signal (J(xi)); It is configured as follows: Signal processing device (1).

2. The characteristic length (f l ) is the system response (H(x i) is smaller than half of the characteristic length (cl); A signal processing device (1) according to claim 1.

3. The plurality of output signals (O 1…K (x i A subset of the system response (H(x i )) l ) is based on A signal processing device (1) according to claim 1 or 2.

4. The plurality of output signals (O 1…K (x i A subset of the system response (H(x i The characteristic length (c l ) l ) is based on A signal processing device (1) according to any one of the preceding claims.

5. The signal processing device (1) - the digital input signal (6, I(x i )) in at least two dimensions, different characteristic lengths (f l ) is configured to apply A signal processing device (1) according to any one of the preceding claims.

6. The system response (H(x i The characteristic length (c l ) is the digital input signal (I(x i ) differ in at least two dimensions (i) of A signal processing device (1) according to any one of the preceding claims.

7. The characteristic length (f l ) is the least squares minimization criterion (M(f(x i ))) is included in A signal processing device (1) according to any one of the preceding claims.

8. The least squares minimization criterion (M(f(x i ))) is the characteristic length (f l ) including the penalty function (P(f(x i ))) A signal processing device (1) according to claim 7.

9. The least squares minimization criterion (M(f(x i ))) is the characteristic length (f l ), and the baseline estimate (f(x i ) at least one derivative of [0010] and, A signal processing device (1) according to claim 7 or 8.

10. The characteristic length (f l ) is the regularization length scale (γ j ) Signal processing device (1) according to any one of the preceding claims.

11. The signal processing device (1) is configured to implement a quadratic minimization scheme (66) having two iteration stages (50, 60), A signal processing device (1) according to any one of the preceding claims.

12. An observation device (1), in particular a medical observation device such as a microscope (2) or an endoscope, comprising a signal processing device (1) according to any one of claims 1 to 11. Observation device (1).

13. System response (H(x i A digital input signal (I(x)) is recorded by a recording system (4) having a i )) a signal processing method for enhancing a signal, the signal processing method comprising: Preferably, the device is adapted to operate a signal processing device (1) according to one of claims 1 to 11 or to operate an observation device according to claim 12, The system response (H(x i )) has at least one characteristic length (c) in at least one dimension (i). l ), and the output signal (O(x i To obtain the system response (H(x i )) a characteristic length (f l ) is removed from the digital input signal; Signal processing methods.

14. At least one further output signal (O 2 (x i ) is calculated, and the step of calculating the at least one further output signal comprises calculating the system response (H(x i )) the at least one characteristic length (c l ) or more characteristic length (f l ) from the digital input signal, and a deconvolution output signal (J(x i To obtain the output signal (O(x i )) and said at least one further output signal (O 2 (x i ) is calculated from 14. The signal processing method according to claim 13.

15. The baseline estimate (f(x i ) is the characteristic length (f l ) depends on the regularization length scale (γ j ), 15. A signal processing method according to claim 13 or 14.

16. A computer program comprising a program code for performing the method according to any one of claims 13 to 15, when the computer program is run on a processor. Computer program.

17. A non-transitory computer-readable medium storing a computer program for causing a computer to execute the signal processing method according to any one of claims 13 to 15. A non-transitory computer-readable medium.

Citation Information

Patent Citations

  • Wave field microscope with sub-wavelength resolution and methods for processing microscopic images to detect objects with sub-wavelength dimensions

    US20090263002A1