Fourier ptychographic generation of an image

Abstract

The Fourier ptychographic generation of images is intended to be simplified. To this end, an object having a multiplicity of illumination elements which are arranged in distributed fashion at a corresponding multiplicity of locations in space is illuminated. A plurality of spatial frequency patterns (16), which result from the respective illumination of the object by means of one or more of the multiplicity of illumination elements, are detected. Each spatial frequency pattern is centered at a respective position in Fourier space, which corresponds to a nominal spatial frequency (8) of the respective illumination element or respective illumination elements. The image is reconstructed using a totality of all respective centered spatial frequency patterns.

Description

[0001]Description Fourier ptychographic generation of an image The present invention relates to a method for the Fourier ptychographic generation of an image of an object using color-corrected optics by illuminating the object with a plurality of illumination elements that are arranged at a corresponding plurality of locations in space. Furthermore, the present invention relates to a corresponding device for the Fourier ptychographic generation of an image of an object using color-corrected optics and an illumination device. Ptychography is a microscopic method. The sample to be examined is scanned by an electromagnetic beam or a particle beam, which is scattered by the material and forms a diffraction and / or interference pattern. The beam changes its position slightly each time (e.g.different locations of the light sources) so that it hits the sample at different angles and accordingly generates different diffraction and / or interference images. From the diffraction and / or interference images from many locations on the sample, an image of the entire sample can be calculated using an algorithm (e.g., Fourier inverse transformation). The advantage of this technique is that the diffraction and / or interference pattern also contains information about the phases of the waves and can therefore, for example, also image transparent structures. In Fourier ptychography (FP), a wavelength parameter is required for image reconstruction in order to perform a correct reconstruction. Technically speaking, the spatial frequency spectrum of the diffraction pattern must be linked to the lateral spatial frequency pattern of the image, as captured by the detector and generated by a subsequent Fourier transform of the image.Imaging typically does not use a single wavelength, but rather a narrower or broader emission spectrum of a light source. In current implementations of Fourier ptychography, the spectral profile is not taken into account, and the wavelength parameter is conversely determined as an (effective or averaged) value in the emission spectrum of the light source, for example, by optimizing the reconstructed image by varying the value of the wavelength parameter. In the literature and in evaluation codes available in software packages for Fourier ptychography (e.g., BG Zheng: “Fourier ptychographic imaging,” 2016, Morgan & Claypool Publishers, ISBN 978-1-64327-860-5), there is currently only one parameter, the wavelength, which can be optimized for each reconstruction to optimize image quality. However, there is no approach that takes the spectral profile of the light source into account.Everything is indirectly obscured by the single wavelength parameter, which corresponds approximately to the weighted average of the effective wavelength transmitted through the entire optical system. The object of the present invention is therefore to simplify the Fourier ptychographic reconstruction of an image of an object. According to the invention, this object is achieved by a method and a device according to the independent patent claims. Furthermore, a corresponding computer program or computer-readable storage medium is provided. Advantageous developments of the invention arise from the subclaims. According to the present invention, a method for Fourier ptychographic generation of an image of an object using color-corrected optics is provided.The present invention takes advantage of the fact that the image of an imaging system is typically color-corrected to a very high degree. This is a prerequisite for all imaging systems, from microscopes to macroscopic imaging systems, photo cameras, and the like. Even in critical applications such as lithography, the respective optics are color-corrected for the optical spectrum used. Even in chromatic confocal imaging systems (e.g., confocal white-light sensors or color scanning microscopes such as the Zeiss CSM with a Nipkow disk), the lateral color aberration is compensated, and only an axial color aberration with specific properties is incorporated into the system. In the method according to the invention, the object is illuminated with a plurality of illumination elements arranged at a corresponding plurality of locations in space.The illumination source for the object is a lighting device with many individual lighting elements that can be located at predetermined locations in space. For example, this is a matrix spotlight. Each individual lighting element is preferably considered to be point-shaped and can be located accordingly at the respective point. In a further step of the method according to the invention, several spatial frequency patterns resulting from illuminating the object with one or more of the plurality of lighting elements are recorded. For example, the object is illuminated separately with individual lighting elements of the lighting device. However, the illumination can also be achieved with groups of lighting elements from this plurality of lighting elements. The object is therefore illuminated one after the other with one or more of the plurality of lighting elements.At the same time, either a single illumination element or a group of illumination elements is switched on for illumination. Subsequently, each spatial frequency pattern is centered at a (respective) position in Fourier space that corresponds to a nominal spatial frequency of the respective illumination element(s). The nominal spatial frequency is the point in Fourier space that represents the center of the spatial frequency pattern of a lighting element depending on its location. Each spatial frequency pattern or interference pattern is thus centered at a specific location in Fourier space that corresponds to its nominal spatial frequency. Finally, the image is reconstructed using the totality of all centered spatial frequency patterns. For example, all spatial frequency patterns are "summed" (i.e.spatially combined) and an image is obtained from this by inverse Fourier transformation. Due to the color-corrected optics, there is no dispersion, so that reconstruction can take place regardless of the wavelength. In an advantageous embodiment of the method according to the invention, it is provided that the plurality of locations in space at which the lighting elements are distributed lie on a surface that is flat, ellipsoidal, or in the shape of a spherical shell. For example, the lighting elements can be evenly distributed over a rectangular surface. This results in a matrix radiator, for example. Alternatively, the lighting elements can also be distributed over the surface of a sphere (or another body). In particular, they can be uniformly distributed over a section of a spherical shell. In particular, this distribution can also be given on a hemisphere. Both a distribution on a e.g.B. rectangular flat surface as well as a distribution on a spherical segment enable a simple reconstruction algorithm for reconstructing an image. In one embodiment of the method according to the invention, the respective nominal spatial frequency is determined (directly) from an angle of incidence of the respective lighting element onto the object or a position of the respective lighting element. For example, with a rectangular arrangement of the lighting elements it is advantageous to determine the nominal spatial frequency of the respective lighting element directly from its position (e.g. x and y coordinates). With a spherical (i.e. spherical shell (segment) shaped) arrangement of the lighting elements, on the other hand, it is advantageous if the nominal spatial frequency of a respective lighting element is determined directly from its angle of incidence onto the object.Thus, the coordinates in K-space (Fourier space) can be determined directly from the polar angle and the azimuthal angle of each illumination element. As already indicated above, with a rectangular arrangement of the numerous illumination elements, which are arranged at a uniform distance P from each other in orthogonal spatial directions x and y, x and P y have, the nominal spatial frequencies are calculated using the formulas: K x,i = atan(i*P x / D) and K y,j = atan(j*P y / D), where D corresponds to a distance between an object plane in which the object is located and an illumination plane in which the plurality of illumination elements are arranged. Alternatively, the nominal spatial frequencies in a spherical arrangement of the plurality of illumination elements arranged at azimuthal angles φ ij in rings i and polar angles ϑ iarranged to an optical axis of the optics, can be calculated using the formulas: K x,ij = ϑ i cos(φ ij +δφ i ) and K y,ij = ϑ i sin(φ ij +δφ i ), where δφ ia uniform azimuthal spacing of the illumination elements on the respective ring i. In the case of a sphere, the nominal spatial frequencies can thus be calculated from the spherical coordinates that describe the locations of the light sources, and in the case of a planar array of illumination elements, from the Cartesian coordinates of the respective light source locations. The simple equations result in the case that the sphere or array is also "centered" on the optical axis of the imaging system and the locations of the light sources for this orientation of the illumination system are known. If there is no centering, "offsets" would have to be taken into account in order to achieve the centering mathematically. As mentioned, the reconstruction of the image can involve an inverse Fourier transformation of the totality of all respectively centered spatial frequency patterns. This means that all the spatial frequencies associated with the individual illumination elements or groups ofThe spatial frequency patterns obtained from the illumination elements are taken into account in the inverse transformation. The inverse transformation can thus be implemented using a simple algorithm. In a particularly preferred embodiment, it is provided that a spatial frequency space or Fourier space used for reconstructing the image is limited on the basis of a maximum spatial frequency defined by a numerical aperture of the color-corrected optics. The spatial frequency for the resolution limit is therefore derived directly from the numerical aperture. However, there is a wavelength dependency in that the numerical aperture is divided by the wavelength to calculate the spatial frequency. Furthermore, each spatial frequency pattern can be limited using a gradient or apodization filter, which is selected according to the optical and / or wavelength-dependent transmission properties, e.g., based on the modulation transfer function, of the optical system.is configured. By limiting the spatial frequency spectrum for each of the individual images, a reduction in the computational effort during reconstruction can be achieved. In a further advantageous embodiment of the method according to the invention, it is provided that each spatial frequency pattern is corrected with an inverse modulation transfer function of the color-corrected optics. This correction leads to the influence of the color-corrected optics on the image content being reduced and, in the best case, completely compensated. In this way, almost undistorted spatial frequency patterns can be determined. In a further development, it can be provided that during reconstruction, each individual image obtained with the individual or several of the plurality of illumination elements is freed from the modulation transfer function of the color-corrected optics by deconvolution, and each individual image thus corrected is subsequently converted into a respective corrected spatial frequency patternis transformed, and only then are the corrected spatial frequency patterns combined. In the present case, a correction of the individual images already takes place in the spatial domain before the transformation into the spatial frequency domain. Alternatively, during reconstruction, each individual image obtained with the individual or several of the plurality of illumination elements can be transformed into a respective spatial frequency pattern by Fourier transformation. Each spatial frequency pattern is then subsequently freed from the influences of the optics and / or the illumination elements on a respective transfer function for Fourier components of the respective spatial frequency pattern, and only then are the spatial frequency patterns thus corrected combined. Here, the correction takes place in the spatial frequency domain and not in the spatial domain. This can also bring advantages with regard to computing speed. In a further advantageous embodiment of the method according to the invention,that during the reconstruction of the image, all spatial frequency patterns are combined using an iterative optimization algorithm. In this way, it is possible to combine the spatial frequency patterns in such a way that ultimately a high-resolution image can be obtained. The above-mentioned object is also achieved according to the invention by a device for the Fourier ptychographic generation of an image of an object, comprising - a color-corrected optical system and - an illumination device for illuminating the object with a plurality of illumination elements arranged at a corresponding plurality of locations in space, as well as comprising - a detection device for detecting a plurality of spatial frequency patterns resulting from illuminating the object with a single one or more of the plurality of illumination elements, and - a computing device (designed) for centering each spatial frequency pattern at a position in Fourier space which corresponds to aNominal spatial frequency of the respective illumination element(s), and for reconstructing the image using a totality of all respectively centered spatial frequency patterns. The computing device can be integrated into the optical sensor or form a separate unit. The advantages and developments presented above in connection with the method according to the invention also apply mutatis mutandis to the device according to the invention. Accordingly, the method features presented in the device are to be viewed as functional features of corresponding means. According to the invention, a computer program is also provided which comprises instructions which, when the program is executed by an above-mentioned device, cause the device to carry out the method also described above. Furthermore, a computer-readable storage medium can be provided which comprises instructions which, when executed by the above-mentionedDevice to cause them to carry out the said method. The storage medium can, for example, be designed at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can further be implemented in a data memory of a processor circuit. However, the storage medium can also be operated, for example, as a so-called app store server on the Internet. A processor circuit with at least one microprocessor can be provided by a computer or computer network. The instructions can be provided as binary code or assembler and / or as source code of a programming language (e.g. C). For applications or application situations that may arise during the method and which are not explicitly described here, it can be provided that according to the method aAn error message and / or a request to enter user feedback is output and / or a default setting and / or a predetermined initial state is set. The invention also encompasses combinations of the features of the described embodiments. Exemplary embodiments of the invention are described below. These are shown in: FIG. 1 shows a structure of an FP microscope; FIG. 2 shows an enlargement of the illumination device of the FP microscope; FIG. 3 shows an enlarged view of the object plane of the FP microscope; FIG. 4 shows a schematic view of a rectangular illumination arrangement; FIG. 5 shows a schematic plan view of a spherical illumination arrangement; FIG. 6 shows the Fourier space of the arrangement of FIG. 4; FIG. 7 shows the Fourier space of the arrangement of FIG. 5; FIG. 8 shows a spectrum of an illumination element; FIG. 9 shows resolution limits in Fourier space; FIG. 10 shows an amplitude distribution in Fourier space; FIG. 11 shows a schematic representation of a reconstruction process; FIG.12 shows a transfer function of an optical system; FIG. 13 shows a normalized transfer function; FIG. 14 shows a spectral distribution taking the optical system into account; FIG. 15 shows a modulation transfer function of a real optical system; FIG. 16 shows a flowchart of a reconstruction algorithm; FIG. 17 shows a flowchart of an alternative reconstruction algorithm; FIG. 18 shows the Fourier space of the rectangular illumination arrangement with K-spaces of the image sensor; and FIG. 19 shows the Fourier space of a spherical illumination arrangement with K-spaces of the image sensor. The exemplary embodiments described in more detail below represent preferred embodiments of the present invention. FIG. 1 shows a conventional structure of a Fourier ptychography microscope 1. It has an object plane in which an object 2 to be imaged is arranged. The object 2 is illuminated from below using an illumination device 3. FIG. 2 shows an enlarged section of theLighting device 3. It has a plurality of regularly arranged lighting elements 4. In the present example, the lighting elements are arranged in a matrix in orthogonal directions x and y. Each lighting element can, for example, be implemented as an LED that emits an illumination beam 5 onto the object 2. The arrows 6 in FIG. 2 indicate the direction in which the individual lighting elements 4 can be switched on and off one after the other. For example, a switching-on sequence can consist of the lighting elements 4 in a first row being switched on one after the other, followed by those in a second row, and so on. In principle, any sequence can be selected. Furthermore, several lighting elements 4 in a group can be switched on and off together. The illumination beam of each lighting element 4 is diffracted by the object 2.and collected by an optic 6. Beyond the optic 6, the Fourier space 7 results for the image of the object 2. This Fourier space 7 is shown in FIG 3 in an enlarged plan view. It has the coordinates K x and K yFor each individual illumination element 4, a nominal spatial frequency 8 results in the Fourier space 7. This nominal spatial frequency 8 represents, for example, the center of the circle shown and moves analogously to the arrow 9 in FIG 2. The Fourier space 7 is recorded or scanned by an optical sensor 10 of the FP microscope 1. With this FP microscope (here a transmitted-light microscope), it is possible to record an image sequence of an object with various collimated or very low-aperture spatial illuminations in bright-field and / or dark-field mode. Instead of the flat arrangement of the illumination elements 4, a (hemi-)spherical arrangement is also possible. It can be assumed that the microscope is optically corrected so that it records good-quality images of the sample without any significant distortions or chromatic aberrations.Optionally, the illumination wavelength can be selected by the LED used or in combination with bandpass filters with a half-width of 10, 50, or 80 nm, for example, in front of the black-and-white camera. The tube lens of the FP microscope can cause imaging errors. These effects can occur particularly in the paraxial regions of the object space, e.g., < + / - 500 µm to the optical axis. Figures 4 to 7 illustrate rectangular and spherical arrangements of illumination elements in spatial domain and their effects in spatial frequency domain in more detail. Figure 4 shows a rectangular arrangement of illumination elements 4. The illumination device here has 81 illumination elements 4 (e.g., LEDs). They are spaced P in the horizontal direction x. x and in the vertical direction y a distance P yEach lighting element has corresponding coordinates (i, j). The position of each lighting element 4 in a lighting angular coordinate system can be specified as: K x,i = atan(i*P x / D) and K y,j = atan(j*P y / D), where i is, for example, an integer from the range -m to m and j is an integer from the range -n to n. D corresponds to the distance from the object plane of the object 2 to the illumination plane of the illumination device 3. FIG. 6 shows a corresponding illumination coordinate system in Fourier space for the rectangular illumination arrangement of FIG. 4. Each point represents a nominal spatial frequency θ for this rectangular arrangement. The sampling is accordingly not equidistant and the symmetry axes may exhibit aliasing effects. The spot size for the illumination elements is determined by the numerical aperture of the illumination of each LED as viewed from the object. For flat arrangements, the spot size decreases with increasing distance from the optical axis. FIG. 5 shows a plan view of a spherical arrangement of illumination elements 4.The illumination elements, or LEDs, have a constant radial distance R from the intersection of the optical axis (+) and the object plane. The rings have a fixed angular increment Δϑ. The LEDs in the rings have a constant angular spacing within the respective ring. The individual azimuthal displacement δφ. i per ring serves to avoid crosstalk effects. The arrangement shown in FIG. 5 has, purely as an example, 76 lighting elements 4 in four rings around the center. φ ij represents the azimuthal angle in ring i. ϑ i corresponds to the polar angle to the optical axis. The center position of the lighting elements in the illumination angle coordinate system can be represented as follows: K x,ij = ϑ i cos(φ ij +δφ i ) and K y,ij = ϑ i sin(φ ij +δφ i ) or explicitly K x,ij = i*∆ϑ cos(j*∆φ i +δφ i ) and K y,ij = ϑi sin(j*∆φ i +δφ i ) where i is an integer between 0 and N R -1 and j is an integer between 1 and N R1 or 1 and N R2 or 1 and N R depending on the ring number. N R is the maximum number of illumination elements on the respective ring. The distance in angular coordinates between the individual illumination elements is uniform due to the spherical structure with constant angular increments. FIG. 7 shows the corresponding illumination coordinate system in Fourier space for the spherical arrangement of the illumination elements 4 in FIG. 5. Here, equidistant scanning in the radial and circumferential directions is possible. Due to the azimuthal displacement ∆φ iThere is no axis of symmetry in each ring. The following shows the combination of the illumination coordinate system and the sampling coordinate system of the optical sensor in Fourier space. The illumination coordinate system extends over a larger area in the Fourier space coordinate system than the image coordinate system (also called the sampling coordinate system). In order to combine both coordinate systems, their dimensions or physical units must match. The dimension in the illumination coordinate system is the unitless numerical aperture, i.e., the sine of an angle. The dimension in the sampling coordinate system is, for example, 1 / mm or 1 / µm, i.e., the inverse length. The link between the two coordinate systems is the wavelength λ used, since the diffraction formula defines the structure width Λ, the diffraction angle α. n and the respective wavelength λ coupled together by sin α n= n * λ / Λ where n indicates the diffraction order (usually n = 1). This implies some general aspects. Firstly, the diffraction pattern is highly dispersive (sin α ~ λ) for a given feature size Λ. Secondly, the wavelength and bandwidth of the illumination are of great importance in standard reconstruction methods. In particular, the effective wavelength plays a crucial role in converting the illumination coordinate space into the Fourier coordinate space of the sampling. With regard to bandwidth, dispersion is of great importance in two respects. The higher the numerical aperture of the optics, the greater the effects can be. This is only unproblematic if the optics are well corrected. With regard to diffraction, the bandwidth of the illuminating light directly correlates with the “blur” in the diffraction angles for the aperture plane.The effective wavelength is calculated as the convolution of the LED emission plus the filter and sensor sensitivity and the modulation transfer function at the respective wavelength. This has the following consequences and advantages for current reconstruction algorithms: - For a given wavelength / effective wavelength and a given numerical aperture, there is a deterministic connection transformation for the illumination and sampling coordinate systems if the illumination system is calibrated at angles relative to the sensor coordinate axis. - The bandwidth of the imaging system only needs to be controlled or limited to avoid image degradation due to chromatic aberration during imaging if a high numerical aperture and high-resolution, detailed images are desired. As long as the camera optics and the image are of high quality, the wavelength bandwidth is not a problem.- Different imaging wavelengths can be used to generate Fourier patterns even with slight sampling overlap for each wavelength. Compared to conventional reconstruction algorithms, the new inventive concept is based on the observation that the optics, and thus the images captured by the image sensor, are of high quality in a plane conjugate to the object plane. With fully color-corrected optics, there is therefore no dispersion and thus no wavelength effect in the image. Therefore, the invention provides a dispersion-free—non-wavelength-dependent—approach for reconstructing ptychography images. This requires linking the numerical aperture of the illumination to the spatial image frequency without having to introduce the concept of diffraction angles.To clarify this difference, we will first briefly explain, using FIG. 11, how a state-of-the-art FP reconstruction process is carried out using wavelength-based angle transformation. The starting point is, for example, an object pattern 11 in spatial domain. After a Fourier transformation (FFT), a corresponding overtone spectrum 11' with line spacings and conjugate complex components results in the spatial frequency domain or Fourier domain. For the reconstruction, it should be noted that a blue wavelength (λ1) is smaller than a green wavelength (λ2), which in turn is smaller than a red wavelength (λ3). The angle of incidence for each illumination setting is wavelength-independent and is α. i and β j , where α i the polar angle and β j is the azimuthal angle. Each light source or lighting element can thus be assigned angular coordinates (α i , β j). Using the wavelength as a parameter, the respective angle can be converted into a lattice constant, e.g. simplified for light incidence perpendicular to the lattice plane: G1= λ1 / sin(α1), G2= λ2 / sin(α2) and G3= λ3 / sin(α3). Each Fourier transform pattern then lies in the Fourier space with its center at the point (g i , β j ), where g i = 1 / G i is the respective grating frequency in a common coordinate system for the Fourier transform and the grating spatial frequency. A maximum filter for g i can be realized as g i,max = 2π * NA / λ i . By filtering the Fourier space to the maximum diffraction angle, which is defined by the numerical aperture of the optics and the wavelength, it follows that the intensity in the Fourier space is a function of the polar angle α i , the azimuthal angle β j , the wavelength λ iand the numerical aperture, namely: I1(g i , β j ) = f(α i , β j , λ1, NA), I2(g i , β j ) = f(α i , β j , λ2, NA) and I3(g i , β j ) = f(α i , β j , λ3, NA). Using an iterative algorithm 12, a high-resolution image with a composite amplitude and phase of sub-images (self-consistency in overlapping regions of the Fourier space) is obtained from a set of images as input with boundary conditions. This results in a corresponding monochrome output image 14 of the corresponding color as well as a phase image 15 for each color. Another possibility to couple the aperture angle of the illumination and the spatial frequencies of the image plane is to use the concept of the resolution limit of a minimal structure d min by using an optical system with numerical aperture NA and wavelength λ: d min= k1* λ / NA where k1 is a parameter that depends on the applied resolution criterion. For example, the Rayleigh criterion k1= 0.61 or the Sparrow criterion k1= 0.5 = k 1,min used. For a modern camera sensor with linear intensity scaling, k1 = 0.5 applies. The resolution limit d min (λ, k1, NA) for a given imaging setup with fixed k1 and NA can be directly translated into a spatial frequency like the camera pixels: k(λ) = 1 / (2 * d min (λ)) = NA / (2 * k1* λ) Alternatively, for a given camera sensor and k1, there is a cutoff wavelength λ that can transmit the entire information content of the image: d min = 2 * d pixel = k1* λ / NA This leads to: λ min = 2 * NA * d pixel / k1where all parameters on the right-hand side of the last equation are fixed and determined by the parameters of the imaging setup. λ minis the shortest wavelength for the imaging setup, which uses the full numerical aperture NA of the lens to image structures with the detector of a specific pixel pitch d pixel so that the resolution can be limited. The corresponding spatial frequency is calculated as follows: k NA = 2 * π / λ min = 2 * π / (2 * NA * d pixel / k1) = π * k1 / (NA * d pixel ) In a two-dimensional plane, the spatial frequency can be determined according to the azimuthal angle β j the illumination and split into the Cartesian parts with: k x,i,j = k NA,ij cos(β j ) k y,i,j = k NA,ij sin(β j ) For each light source (ij), the Fourier transform of the respective image is applied to the position (K x,i,j; K y,i,j) in the resulting Fourier space for image reconstruction. For a given optical system with maximum numerical aperture and various wavelengths in the illumination spectrum, the spatial frequency limit to be calculated is wavelength-dependent at all wavelengths and a fixed k1, according to the formula: k(λ) = 2 * π * NA / (2 * k1 * λ) = π * NA / (k1 * λ). The factor 2 * π in the above formulas can depend on the way in which normalization is performed during the Fourier transform. (If necessary, the normalization / calibration of a Fourier transform is also performed in other ways.) This offers the opportunity to natively incorporate wavelength and bandwidth into the image reconstruction process. The longest wavelength used has the lowest resolution limit that can be achieved with a related spatial frequency. The shortest wavelength has the greatest spatial frequency content in the Fourier plane.Assuming that all wavelengths used are in the range of λ. min to λ max have the same intensity (see FIG 8), the constellation shown in FIG 9 results for the Fourier space. Accordingly, the resolution limits for short-wave light λ min in Fourier space significantly further out than the resolution limits for long-wave light λ max . In addition, FIG 10 shows a radial section through the Fourier space. While the contribution of the spatial frequencies below k (λ max ) is constant (all frequencies contribute), decreases outside of k (λ max ) the contribution of the individual frequencies. Above the spatial frequencies k (λ max ), the long-wavelength components of the light can no longer contribute. Thus, the proportion of contributions decreases continuously until finally, even with the shortest wavelength λ minno further contribution can be made. This insight leads to a new approach for reconstruction (see FIG. 11), according to which the angle of incidence is determined or calibrated for each illumination source or each illumination element. This results, for example, in a list of light source positions: LS k = (α i , β j ) or (x i , y j ) Here, α i the polar angle and β j the azimuthal angle. The spatial frequency component for all used (α i , β j ) or (x i , y j ) can be calculated as follows: K x,i,j = cos(β j )sin(α i ); K y,i,j = sin(β j )sin(α i ) or alternatively K x,i,j = K x,i = atan(x i / D); K y,i,j = K y,j = atan(y j / D) Each Fourier transform pattern then lies in the Fourier space with its center at the point (K x,i,j , K y,i,j ) or (K x,i , K y,j). Further reconstruction is carried out according to FIG 11 in the iterative algorithm 12. In an extended version of the reconstruction process, a filtering of the Fourier space to the maximum spatial frequency, which is defined by the numerical aperture of the optics, can be carried out. Spectral bandwidth information can be taken into account in the algorithm. Furthermore, the new approach to the reconstruction process offers further potential for improvement. Firstly, the new approach based on the spatial vector or the illumination makes spectral information based on the color or wavelength band spectrum possible and useful, which is not possible in the prior art. Furthermore, a filtering of the Fourier transform for a raw image can be carried out with an apodization filter according to the spectral profile / wavelength profile of the light source used to acquire the respective image.The wavelength spectrum is usually an idealized hat or rectangular profile, which depends on λ. min to λ maxis sufficient. For a square pixel size, the Fourier plane has a square shape. The sampling step size in the Fourier plane in the direction of the sensor's coordinate axis depends on the number of sensor pixels in the respective axis. For a square array, the sampling is the same in both axes. The circles (see FIG. 9) show the area of ​​the image's Fourier space covered by the respective wavelength. The area of ​​the Fourier plane not covered by the circles refers to spatial frequencies in the image that are not accessible for the contribution of a specific wavelength to the overall image. These are sampling frequencies that exceed the Nyquist theorem. Of particular interest is the cross-sectional view of FIG. 10 of the Fourier plane mentioned above. It shows how the wavelength spectrum appears for correct reconstruction. The triangular shape between λ max and λ minwith negative slope is a direct consequence of the assumption that the light intensity is the same for each wavelength. In real cases, light sources usually do not have such uniform emission profiles, and the optics and the sample do not have such ideal transmission performance. In a realistic scenario, one has an effective transmission profile of the optical system with a certain wavelength characteristic, which is shown as an example in FIG 12. This curve of the spectral power distribution (which is considered as a density function) leads to the weighted distribution function according to FIG 13 by taking the integral from a certain wavelength to λ min It is determined by the maximum value for the area under the curve between λ max and λ minnormalized to 1. Taking this fact into account, the sketch for the radial section of the Fourier plane (cf. FIG. 10) changes according to FIG. 14. There, the spectral distribution is included in the transfer function for the Fourier components in order to filter the higher spatial frequency contributions. This transfer function can be interpreted as a wavelength-dependent pupil function corresponding to a pupil diaphragm in the pupil of an optical imaging system. From a functional point of view, this representation corresponds exactly to the optical function of a physical diaphragm in the aperture plane with spectrally corrected pupil imaging with a diameter matching the maximum aperture NA. maxof the lens. Real optical systems do not have a constant transfer function across the entire frequency range from 0 to the maximum transmitted spatial frequency. In the real world, the frequency response is described by a modulation transfer frequency. It has the value 1 or its maximum value for the frequency 0 and drops to 0 at the maximum spatial frequency transmitted by the system (cf. FIG 15). When reconstructing Fourier ptychographic images, this effect must be compensated for when combining the different spatial frequency patterns from the Fourier transforms of the different illumination directions. The modulation transfer function (MTF) can be determined using any known method, e.g., from imaging test structures and evaluating the contrast or by determining parameters for the imaging performance (e.g.,The optical system's coefficients (e.g., Seidel coefficients, Zernike coefficients) are extracted and the MTF is calculated from them using a suitable formalism. The image or the Fourier spectrum of the image can then be corrected for the MTF to obtain an image of much higher quality, preferably as an ideal optical system would have imaged the object. The corrected Fourier spectra can then be merged into a common spectrum with greater spatial frequency content to produce a high-quality reconstructed image. For the merging process, well-known (iterative) optimization algorithms 12 for image search, such as those of Gerchberg-Saxton or Levenberg-Marquardt, or comparable methods for the iterative solution of multidimensional and / or nonlinear problems, can be used.In general, the entire optimization could also be performed in a single step, as is the case with the state-of-the-art Fourier ptychography reconstruction. In this sense, the functional flow diagrams of FIGS. 16 and 17 show only a generic strategy for implementing the entire reconstruction process, which can be restructured without departing from the scope of the reconstruction goal. In the example of FIG. 16, an image is provided for an optimization algorithm 12 for image generation (e.g., that of Gerchberg-Saxton or Levenberg-Marquardt) in a step S1. Within the framework of this algorithm 12, a deconvolution of the image and MTF occurs in a step S2. In a subsequent step S3, a Fourier transform (FFT) into Fourier space is performed. In the subsequent step S4, the Fourier images are combined. In the subsequent step S5, an inverse Fourier transform is performed to generate the image.Steps S3 to S5 can be performed iteratively, whereby an iteration cycle can also include a step for comparing the current image with at least one of the input images from S1 or an optimized input image after step S2. Finally, a high-quality output image can be provided in a step S6. Alternatively, the reconstruction process can also be carried out according to the example in FIG 17. Here, too, an image is first provided for the optimization algorithm 12 in a step S1. Within this, however, a Fourier transformation according to step S3 now takes place immediately. This is followed by a step S7 in which the contributions of the imaging system and the object are separated in Fourier space.Subsequently, in step S8, a real / Fourier image is combined, before an inverse Fourier transformation follows in step S5 to provide a high-quality output image in step S6. The concept of combining illumination and scanning coordinate systems in Fourier space is clearly explained using FIGS. 18 and 19, with appropriate scaling being chosen for the purpose of illustration. FIGS. 18 and 19 are based on the Fourier spaces of FIGS. 6 and 7 for a flat, rectangular illumination device and a spherical illumination device, respectively. In the center of each of the two Fourier spaces is a spatial frequency pattern of the optical sensor for illumination on the optical axis.Symbolically, additional spatial frequency patterns 16 (equivalent to the overtone spectrum 11' from FIG. 11) of the optical sensor are plotted in the Fourier spaces, each centered at a nominal spatial frequency of the illumination device. The illumination coordinate system extends over a larger area in the Fourier space coordinate system than the imaging coordinate system. In addition to the central image with coaxial illumination on the optical axis of the imaging coordinate system, multiple images can be acquired at different illumination angles. After combining the images in Fourier space, the inverse transformation can take place, resulting in the high-resolution image.

Claims

Claims 1. A method for Fourier ptychographically generating an image of an object using color-corrected optics (6) by illuminating the object with a plurality of lighting elements (4) distributed at a corresponding plurality of locations in space, characterized by acquiring several spatial frequency patterns (16) resulting from illuminating the object with one or more of the plurality of lighting elements (4), centering each spatial frequency pattern (16) at a position in Fourier space (4) corresponding to a nominal spatial frequency (8) of the respective lighting element(s) (4), and reconstructing the image using an entirety of all respective centered spatial frequency patterns (16). 2.

1. A method according to claim 1, wherein the plurality of locations in space where the lighting elements (4) are distributed are arranged lie on a surface that is planar, ellipsoidal, or segment-shaped.

2. A method according to claim 1 or 2, wherein the respective nominal spatial frequency (8) is determined from an angle of incidence of the respective lighting element (4) on the object or from a position of the respective lighting element (4).

3. A method according to claim 3, wherein the respective nominal spatial frequency (8) is calculated a) in the case of a rectangular arrangement of the plurality of lighting elements (4) arranged in orthogonal spatial directions. x and y are at a uniform distance P from each other x and P y possess, with the formulas: K x,i = atan(i*P x / D) and K y,j = atan(j*P y / D), where D corresponds to a distance between an object plane (2) in which the object is located and a lighting plane in which the plurality of lighting elements (4) is arranged, or b) in the case of a spherical or ellipsoidal arrangement, the plurality of lighting elements which are arranged at azimuthal angles φ ij in rings i and polar angles ϑ i are arranged to an optical axis of the optics (6), with the formulas: K x,ij = ϑ i cos(φ ij +δφ i ) and K y,ij = ϑ i sin(φ ij +δφ i ), where δφ i5. A method according to any one of the preceding claims, wherein the reconstruction of the image includes a Fourier inverse transform of the entirety of all respective centered spatial frequency patterns (16).

6. A method according to any one of the preceding claims, wherein a spatial frequency space (7) used for reconstructing the image is limited to that image based on a maximum spatial frequency defined by a numerical aperture of the color-corrected optics (6).

7. A method according to any one of the preceding claims, wherein each spatial frequency pattern (16) is limited by a gradient or apodization filter configured according to a wavelength profile of the illumination elements (4).

8. A method according to any one of the preceding claims, wherein each spatial frequency pattern (16) is corrected by an inverse modulation transfer function of the color-corrected optics (6).

9. The method of claim 8, wherein, during the reconstruction, each individual image obtained with the single or multiple illumination elements (4) is deconvolutionized to remove the modulation transfer function of the color-corrected optics (6), each corrected individual image is subsequently transformed into a respective corrected spatial frequency pattern, and only then are the corrected spatial frequency patterns combined.

10. The method of claim 8, wherein, during the reconstruction, each individual image obtained with the single or multiple illumination elements (4) is transformed into a respective spatial frequency pattern (16) by Fourier transformation, each spatial frequency pattern is subsequently freed from the influence of the optics (6) and / or the illumination elements (4) on a respective transfer function for Fourier components of the respective spatial frequency pattern, and only then are the corrected spatial frequency patterns combined.

11. A method according to any of the preceding claims, wherein, in reconstructing the image, all spatial frequency patterns are combined using an iterative optimization algorithm.

12. A device for Fourier-ptychographic generation of an image of an object comprising: - a color-corrected optic (6) and - a lighting device (3) for illuminating the object with a plurality of lighting elements (4) distributed at a corresponding plurality of locations in space, characterized by: - ​​a detection device (10) for detecting multiple spatial frequency patterns resulting from illuminating the object with one or more of the plurality of lighting elements (4). - A computing device for centering each spatial frequency pattern (16) at a position in Fourier space corresponding to a nominal spatial frequency (8) of the respective illumination element(s) (4), and for reconstructing the image using the entirety of all centered spatial frequency patterns (16).

13. Computer program comprising instructions which, when executed by a device according to claim 12, cause the device to execute the method according to any one of claims 1 to 11.

14. Computer-readable storage medium comprising instructions which, when executed by a device according to claim 12, cause the device to execute the method according to any one of claims 1 to 11.

Images