Method for optimizing a fourier ptychographic illumination system

Abstract

The application relates to an optimization method of a Fourier ptychographic imaging illumination system, comprising the following steps: determining that vignetting does not exist in an image collected by the Fourier ptychographic imaging illumination system, and the collected image comprises multiple bright-field images; and the optimal value of the center distance between object plane fields when adjacent light source units in an illumination device are lighted; according to the optimal value of the center distance between object plane fields when the adjacent light source units are lighted, adjusting the positions of part of the light source units in the illumination device or the distances between part of the light source units and a sample, so as to eliminate the vignetting. The application avoids the influence of vignetting on the basis of the original system structure, and no redundant hardware is added; the means that the FPM experiment must rely on GPU to block and parallelly process the collected large-field pictures due to the vignetting phenomenon is avoided, the cost of the system is greatly reduced, and the data recovery efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of optical information acquisition and processing technology, and more specifically, to an optimization method for a Fourier layered imaging illumination system. Background Technology

[0002] Fourier ptychographic microscopy (FPM or FP) combines optical phase retrieval technology with microwave synthetic aperture technology. It enables high-resolution and quantitative phase imaging with a large field of view using low-NA objectives, resulting in high imaging throughput. However, in the application of FPM technology, a problem arises: during reconstruction, the acquired large field-of-view images must be segmented. These high-resolution fragments are then stitched together using GPU parallel computing to create a high-resolution full-field-of-view image. There are several reasons for using segmentation in FPM imaging, the most important being that system hardware limitations prevent the pupil from being fully filled with light, leading to vignetting. This causes FPM to deviate from a linear space-invariant model, resulting in "wrinkled" artifacts in the reconstructed image. Segmentation avoids vignetting, thus preserving the linear space-invariant properties of the model.

[0003] However, block processing presents the following challenges for FPM technology: Firstly, Fourier Transform Imaging Systems are primarily used in biomedical diagnostics and industrial inspection to help doctors or researchers better analyze and judge cells, pathology, and industrial manufacturing defects. However, high-performance GPU devices are difficult to widely adopt in non-research laboratory settings such as hospitals. Block parallel computing implemented using GPUs would significantly increase the cost of FPM systems, hindering their market application. Secondly, block processing requires an image overlap rate of more than 10%, which increases the overall computational load and time cost.

[0004] To eliminate vignetting, existing technologies have designed light source surface shapes and illumination modes in FPM, but none of these have been able to eliminate vignetting, and imaging still relies on the GPU. Summary of the Invention

[0005] To overcome at least one deficiency in the prior art, this application provides an optimization method for a Fourier stacked imaging illumination system.

[0006] Firstly, an optimization method for a Fourier layered imaging illumination system is provided, including:

[0007] The optimal center distance between the object plane fields of view when adjacent light source units in the illumination device are lit, provided that the images acquired by the Fourier stacked imaging illumination system do not have vignetting and the acquired images contain multiple bright field images.

[0008] Based on the optimal center distance between the object surface fields of view when adjacent light source units are lit, adjust the position of some light source units in the lighting device or the distance between some light source units and the sample to eliminate vignetting.

[0009] In one embodiment, the method further includes:

[0010] The Fourier layered imaging illumination system acquires images that do not exhibit vignetting, and the acquired images contain multiple bright-field images. The constraints are expressed by the following formula:

[0011]

[0012] Where X is the side length of the object plane's field of view, n is the number of rows or columns when multiple brightfield images are distributed, n is an odd number, D is the pupil diameter on the object plane, and L is the center distance between the object plane's field of view when adjacent light source units are lit.

[0013] In one embodiment, the optimal center distance between the object plane fields of view when adjacent light source units in the lighting device are lit is calculated using the following formula:

[0014]

[0015] Among them, L TOV The optimal center distance between the fields of view of the object surface is given when adjacent light source units are lit, where the acquired image contains n×n brightfield images. D is the pupil diameter on the object surface.

[0016] In one embodiment, adjusting the position of some light source units in the lighting device to eliminate vignetting, based on the optimal center distance between the object plane fields of view when the adjacent light source units are lit, includes:

[0017] The optimal spacing between the sample and the lighting device is determined using the following formula;

[0018]

[0019] Among them, h TOV L is the optimal spacing between the sample and the lighting device. TOV This refers to the optimal center distance between the objects in the field of view when adjacent light source units are lit, given that the acquired image contains n×n brightfield images. NA is the numerical aperture of the objective lens, is the pupil diameter on the object surface, and d... LED This refers to the distance between adjacent light source units in a lighting device;

[0020] Adjust the spacing between the sample and the lighting device to achieve the optimal spacing h. TOV ;

[0021] Identify the light source units in the lighting device that need to be moved;

[0022] When the side length of the object plane's field of view is adjusted to X... Based on the optimal value h of the distance between the sample and the lighting device TOV The optimal value of L between the centers of the object plane in the field of view when adjacent light source units are lit, when the acquired image contains n×n brightfield images. TOV Determine the moving distance and direction of each light source unit that needs to be moved;

[0023] Adjust the position of each light source unit that needs to be moved according to its moving distance and direction.

[0024] In one embodiment, determining the light source unit that needs to be moved in the lighting device includes:

[0025] For each light source unit, if the image acquired when the light source unit is lit is a bright field image, if there exists p (p = 1, 2, 3, 4), where p is the vertex number of the field of view, such that... The light source unit needs to be moved, where C (center) is the center point of the object surface field of view when the light source unit at the center position of the lighting device is lit. p (m) represents the vertex p in the field of view of the object surface when the light source unit numbered m is lit; D is the pupil diameter on the object surface.

[0026] If the image captured when the light source unit is lit is a dark field image, and if there exists p (p = 1, 2, 3, 4), where p is the vertex number of the field of view, such that... Then the light source unit needs to be moved.

[0027] In one embodiment, when the side length of the object plane's field of view is adjusted to X... D is the pupil diameter on the object surface, and h is the optimal value based on the distance between the sample and the illumination device. TOV The optimal value of L between the centers of the object plane in the field of view when adjacent light source units are lit, when the acquired image contains n×n brightfield images. TOV Determine the moving distance and direction of each light source unit that needs to be moved, including:

[0028] Determine the direction of movement for each light source unit that needs to be moved: The image acquired when the light source unit is lit is a bright field image, using the following formula:

[0029]

[0030] Wherein, sgp(m) is the moving direction of the light source unit numbered m, C(center) is the center point of the object surface field of view when the light source unit at the center position of the lighting device is lit, and C(m) is the center point of the object surface field of view when the light source unit numbered m is lit.

[0031] The image captured when the light source unit is lit is a dark field image, and the following formula is used:

[0032]

[0033] Based on the coordinates of the center point of the object surface field of view when each light source unit that needs to be moved is lit, determine the moving distance of the object surface field of view when each light source unit that needs to be moved is lit.

[0034] Based on the movement distance of the object's field of view when each light source unit that needs to be moved is lit, and the optimal value of the spacing h between the sample and the lighting device. TOV The optimal value of L between the centers of the object plane in the field of view when adjacent light source units are lit, when the acquired image contains n×n brightfield images. TOV Calculate the moving distance of each light source unit that needs to be moved.

[0035] In one embodiment, determining the movement distance of the object surface field of view when each light source unit that needs to be moved is lit, based on the coordinates of the center point of the object surface field of view when each light source unit that needs to be moved is lit, includes:

[0036] When the light source unit is lit, the acquired image is a bright field image. The coordinates of C(m) are (x, y), and C(m) is the center point of the object's field of view when the light source unit numbered m is lit. Then:

[0037] If |x|=|y|, x≠0, y≠0 N(m) is the distance the object plane's field of view moves when the light source unit numbered m is lit, and C(center) is the center point of the object plane's field of view when the light source unit at the center of the lighting device is lit. max () is to make The C corresponding to the p value that holds true for (p=1,2,3,4) p (), C p (m) is the vertex with number p in the field of view of the object surface when the light source unit with number m is lit. If the number of p values ​​that satisfy the condition is greater than 1, then any p value is selected.

[0038] If x = 0 or y = 0 X is the side length of the field of view of the object plane;

[0039] Other situations Among them, C max (Center) refers to the field of view of the object surface and C when the light source unit at the center of the lighting device is lit.max () Vertices with the same p value, b1() is V max (Center) and C max () The intersection of the line connecting the two points and the edge of the pupil;

[0040] When the light source unit is lit, the image acquired is a dark field image, and the coordinates of C(m) are (x, y). Then:

[0041] If |x|=|y|, x≠0, y≠0 Among them, C min () is to make The C corresponding to the p value that holds true for (p=1,2,3,4) p ();

[0042] If x = 0 or y = 0

[0043] Other situations Among them, C min (Center) refers to the field of view of the object surface and C when the light source unit at the center of the lighting device is lit. min () Vertices with the same p value, b2() is C min (Center) and C min () is the intersection of the line and the edge of the pupil.

[0044] In one embodiment, the optimal distance h between the sample and the illumination device is determined based on the distance the object plane field of view moves when each light source unit that needs to be moved is illuminated. TOV When the acquired image contains n×n brightfield images, the optimal value of the center distance L between adjacent light source units when they are lit is the object surface field of view. TOV The moving distance of each light source unit that needs to be moved is calculated using the following formula:

[0045]

[0046] Where, d m h is the moving distance of the light source unit numbered m. TOV The optimal distance between the sample and the illumination device is given by N(m), where N(m) is the distance the field of view on the object plane moves when the m-numbered light source unit is lit, D is the pupil diameter on the object plane, NA is the numerical aperture of the objective lens, and L is the distance between the sample and the illumination device. TOV d represents the optimal center distance between the object plane fields of view when adjacent light source units are illuminated, given that the acquired image contains n×n brightfield images. LED C is the distance between adjacent light source units in the lighting device, C(center) is the center point of the object surface field of view when the light source unit at the center position of the lighting device is lit, and C(m) is the center point of the object surface field of view when the light source unit numbered m is lit.

[0047] In one embodiment, adjusting the distance between some light source units and the sample based on the optimal center distance between the object plane fields of view when adjacent light source units are lit, in order to eliminate vignetting, includes:

[0048] After determining the position of some light source units in the lighting device, the new center point of the object surface field of view when each light source unit in the partial light source unit is lit;

[0049] Based on the new object plane field of view center point and the optimal center distance between the object plane field of view when adjacent light source units are lit, the distance between each light source unit and the sample in some light source units is determined using the following formula:

[0050]

[0051] Among them, h m Let m be the distance between the light source unit and the sample, D be the pupil diameter on the object surface, NA be the numerical aperture of the objective lens, and d be the distance between the light source unit and the sample. LED C is the distance between adjacent light source units in a lighting device. new (m) is the new center point of the object plane field of view when the light source unit numbered m is lit, C(center) is the center point of the object plane field of view when the light source unit at the center of the lighting device is lit, and C(m) is the center point of the object plane field of view when the light source unit numbered m is lit; L TOV This represents the optimal center distance between the object surface fields of view when adjacent light source units are lit.

[0052] Secondly, an optimized device for a Fourier layered imaging illumination system is provided, comprising:

[0053] The optimal value determination module for the center distance of the field of view is used to determine the optimal value of the center distance between the object surface fields of view when adjacent light source units in the illumination device are lit, in the case that the image acquired by the Fourier stacked imaging illumination system does not have vignetting and the acquired image contains multiple bright field images.

[0054] The adjustment module is used to adjust the position of some light source units in the lighting device or the distance between some light source units and the sample based on the optimal value of the center distance between the fields of view of the probe when adjacent light source units are lit, so as to eliminate vignetting.

[0055] Compared with the prior art, this application has the following advantages: The optimized method of this application avoids the vignetting effect on the basis of the original system structure without the addition of extra hardware; it avoids the means that FPM experiments must rely on GPU to process the large field of view images in blocks in parallel due to the vignetting phenomenon, which greatly reduces the cost of the system, reduces the complexity of the data, and improves the data recovery efficiency. Attached Figure Description

[0056] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0057] Figure 1 The diagram illustrates the generation of vignetting, where (a) is a simulation of vignetting in an optical system, (b) is the original image acquired when vignetting is present in the system, (c1-1) is the high-resolution intensity image recovered from a small field-of-view FPM without vignetting, (c1-2) is the high-resolution phase image recovered from a small field-of-view FPM without vignetting, (c1-3) is the spectral image recovered from a small field-of-view FPM without vignetting, (c2-1) is the intensity recovered image of FPM affected by vignetting, (c2-2) is the phase recovered image of FPM affected by vignetting, and (c2-3) is the spectral recovered image of FPM affected by vignetting.

[0058] Figure 2 A flowchart illustrating an optimization method for a Fourier stacked imaging illumination system according to an embodiment of this application is shown.

[0059] Figure 3 The diagram shows a system modeling scheme using constraints. (a) is a diagram showing the change in pupil position on the object surface when adjacent LEDs are lit; (b) is a diagram illustrating the vignetting coefficient; (c) is a diagram showing the relative position of the object surface field of view when the LED units are lit one by one, assuming the pupil position is stationary; (d) is a curve showing the relationship between the distance L between two adjacent object surface fields of view and the field size X, obtained under constraints; and (e) is a curve showing the relationship between the distance L between two adjacent object surface fields of view and the distance h between the sample and the LED.

[0060] Figure 4 The following diagrams illustrate the "tolerance" of the experimental system to vignetting: (a) is a curve showing the relationship between X and h; (b1) is the result obtained by FPM recovery using the theoretical maximum object field of view X when h = 58 mm; (b2) is the result obtained by changing the object field of view size in the experiment when h = 58 mm, under the maximum recoverable field of view; (c1) is the result obtained by FPM recovery using the theoretical maximum object field of view X when h = 70 mm; (c2) is the result obtained by changing the object field of view size in the experiment when h = 70 mm, under the maximum recoverable field of view; (d1) is the result obtained by FPM recovery using the theoretical maximum object field of view when h = 76 mm; and (d2) is the result obtained by changing the object field of view size in the experiment when h = 76 mm.

[0061] Figure 5The diagram shows the optimized LED surface design scheme, where (a) is a schematic diagram of realizing the planar movement effect of the LED using height movement (for experimentation, to reduce manufacturing costs), (b) is a schematic diagram of setting the LED number and the center coordinates of the object surface field of view, (c-1) is a schematic diagram of the position movement of LED units numbered 3, 11, 15, and 23 corresponding to the object surface field of view, (c-2) is a schematic diagram of the position movement of LED units numbered 7, 9, 17, and 19 corresponding to the object surface field of view, (c-3) is a schematic diagram of the position movement of LED units numbered 8, 12, 14, and 18 corresponding to the object surface field of view, (c-4) is a schematic diagram of the position movement of LED units numbered 1, 5, 21, and 25 corresponding to the object surface field of view, and (c-5) is a schematic diagram of the position movement of LED units numbered 2, 4, 6, 10, 16, 20, 22, and 24 corresponding to the object surface field of view.

[0062] Figure 6 The diagram shows the verification of the optimal LED surface design and the search for the maximum field of view edge length of the object surface in a single FPM recovery, where (a) is the experimental diagram for finding the intensity “soft boundary” and (b) is the experimental diagram for finding the phase “soft boundary”.

[0063] Figure 7 The optimal light source surface shape obtained using this design is shown. (a) is the light source surface shape calculation after converting different heights during acquisition to a fixed height. (b) is the light source surface shape when X = 2.6 mm. The single light source marked in red is the light source that does not need to change position, and the single light source marked in other colors is the light source that needs to be moved. The specific movement parameters of each light source and the corresponding numbers of the light sources of different colors are given. (c) is a schematic diagram of the scanning scheme for imaging the sample when X = 2.6 mm using a single light source. Detailed Implementation

[0064] Exemplary embodiments of this application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment in order to implement a specific sample for the developer, and these decisions may vary as the embodiments differ.

[0065] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0066] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0067] The Fourier layered imaging illumination system of this application has the same hardware structure as the traditional FPM experimental system, including: an illumination device, an objective lens, a tube lens, and a CCD image sensor, where the CCD image sensor is also the detector. Here, the illumination device adopts a periodic distributed arrangement of light sources, including multiple light source units, such as flat panel LEDs, or partially or fully coherent light sources such as laser dot arrays (LDs), laser dot scanning, LED scanning (miniLEDs, microLEDs), and LCDs. When the lighting device is a flat LED, the flat LED consists of a 13×13 array of single tri-color LEDs. The side length of each LED unit is 1mm, the spacing between adjacent LED units is 4mm, and the center wavelength is 628.63nm (red), 518.08nm (green), and 63.46nm (blue). In the FPM experiment, the flat LED serves as a light source to achieve multi-angle illumination. The objective lens used is a Nikon objective lens (4×0.1NA, FN: 22mm). The sample is imaged using a tube lens with a focal length of 18mm. The CCD used is an Image Source image sensor (DMK23U445, 1280×960, 3.75μm) and a HAMAMATSU DIGITAL CAMERA (C13440, 2048×2048, 6.5μm). In the Fourier in-plane imaging illumination system, each LED unit in the flat panel LED lights up one by one to illuminate the sample. After the sample is illuminated, the imaging unit and the CCD image sensor acquire images of the sample. Once all images of the sample have been acquired, the FPM method is used to restore the images and obtain the final restored sample image.

[0068] Figure 1 The diagram illustrates the generation of vignetting, where (a) is a simulation of vignetting in an optical system, (b) is the original image acquired when vignetting is present in the system, (c1-1) is the high-resolution intensity image recovered from a small field-of-view FPM without vignetting, (c1-2) is the high-resolution phase image recovered from a small field-of-view FPM without vignetting, (c1-3) is the spectral image recovered from a small field-of-view FPM without vignetting, (c2-1) is the intensity recovered image of the FPM affected by vignetting, (c2-2) is the phase recovered image of the FPM affected by vignetting, and (c2-3) is the spectral recovered image of the FPM affected by vignetting.

[0069] This application provides an optimization method for a Fourier layered imaging illumination system. Figure 2A flowchart illustrating an optimization method for a Fourier layered imaging illumination system according to an embodiment of this application is shown. See also... Figure 2 The methods include:

[0070] Step 1: Determine the optimal center distance between the object plane fields of view when adjacent light source units in the illumination device are lit, provided that the image acquired by the Fourier stacked imaging illumination system does not have vignetting and the acquired image contains multiple bright field images.

[0071] Here, assuming the pupil position is stationary, when each light source unit in the lighting device is lit up one by one using the concept of relative motion, the object surface field position is different when adjacent light source units are lit.

[0072] Step 2: Based on the optimal center distance between the object surface fields of view when adjacent light source units are lit, adjust the position of some light source units in the lighting device or the distance between some light source units and the sample to eliminate vignetting.

[0073] In one embodiment, the method further includes:

[0074] The Fourier layered imaging illumination system acquires images that do not exhibit vignetting, and the acquired images contain multiple bright-field images. The constraints are expressed by the following formula:

[0075]

[0076] Where X is the side length of the object plane field of view, which represents the size of each field of view. When the field of view is a square, X is the side length of the square. n is the number of rows or columns when multiple bright field images are distributed. n is an odd number. D is the pupil diameter on the object plane. L is the center distance between the object plane fields of view when adjacent light source units are lit.

[0077] Furthermore, by solving formula (1), the maximum field of view achievable by the Fourier stacked imaging illumination system is obtained, calculated using the following formula:

[0078]

[0079] Among them, X TOV The maximum field of view of the object plane when the acquired image contains n×n brightfield images (n is an odd number), where D is the pupil diameter on the object plane;

[0080] The optimal center distance between the object plane fields of view when adjacent light source units in a lighting device are lit is calculated using the following formula:

[0081]

[0082] Among them, L TOVThe optimal center distance between the fields of view of the object surface is given when adjacent light source units are lit, where the acquired image contains n×n brightfield images. D is the pupil diameter on the object surface.

[0083] Here, according to formulas (2) and (3), when D = 5.5 mm, X TOV =0.759, L TOV =1.565.

[0084] Figure 3 The diagram shows a system modeling scheme using constraints. (a) is a diagram showing the change in pupil position on the object surface when adjacent LEDs are lit; (b) is a diagram illustrating the vignetting coefficient; (c) is a schematic diagram showing the relative position of the object surface field of view when the LED units are lit one by one, assuming the pupil position is stationary; (d) is a curve showing the relationship between the distance L between two adjacent object surface fields of view and the field size X, obtained under constraints; and (e) is a curve showing the relationship between the distance L between two adjacent object surface fields of view and the distance h between the sample and the LED.

[0085] Figure 4 This is a graph showing the "tolerance" of the experimental system to vignetting. (a) is a curve showing the relationship between X and h; (b1) is the result of FPM recovery using the theoretical maximum object plane field of view when h = 58 mm; (b2) is the result of recoverable maximum object plane field of view obtained by changing the size of the object plane field of view in the experiment when h = 58 mm; (c1) is the result of FPM recovery using the theoretical maximum object plane field of view when h = 70 mm; (c2) is the result of recoverable maximum object plane field of view obtained by changing the size of the object plane field of view in the experiment when h = 70 mm; (d1) is the result of FPM recovery using the theoretical maximum object plane field of view when h = 76 mm; and (d2) is the result of recoverable maximum object plane field of view obtained by changing the size of the object plane field of view in the experiment when h = 76 mm. Figure 4As shown in (a), X reaches its maximum value when h is approximately 70 mm, which is consistent with the theoretical analysis. However, the actual value of X at this time is 0.984375 mm, while the theoretical value is 0.795 mm. This verifies that the system does indeed have a certain "tolerance" for vignetting. Analysis using data from 58-68 mm shows that when the vignetting coefficient is 90%-100%, FPM can be used to obtain wrinkle-free high-throughput images. Analysis of data from h=70-76 mm shows that when the vignetting coefficient is 0-10%, the FPM-reconstructed image has no obvious wrinkles. It is worth noting that at a height of 74-76 mm, due to the scattering effect within the system, vignetting still exists under the theoretical field of view, and the vignetting coefficient is greater than 10%. Therefore, the maximum field of view under experimental conditions will be lower than the maximum field of view obtained from the theoretical analysis. Combining simulation and experiment, it can be concluded that under the experimental conditions, the maximum recoverable field of view without vignetting in a single test is h=70.174mm. That is, when the distance between the LED and the sample should be maintained at 70.174mm, the maximum recoverable field of view on the object plane in a single test is approximately 1mm. 2 When the field of view increases further, the acquired image will exhibit significant vignetting, leading to severe artifacts in the FPM-reconstructed image. Based on the above research, step 2, adjusting the positions of some light source units in the lighting device according to the optimal center distance between the object plane's field of view when adjacent light source units are lit, to eliminate vignetting, may include:

[0086] Step 21: Determine the optimal spacing between the sample and the flat LED using the following formula;

[0087]

[0088] Among them, h TOV L is the optimal spacing between the sample and the lighting device. TOV This refers to the optimal center distance between the objects in the field of view when adjacent light source units are lit, given that the acquired image contains n×n brightfield images. NA is the numerical aperture of the objective lens, is the pupil diameter on the object surface, and d... LED L is the distance between adjacent light source units in a lighting device; when L ToV When h = 1.565, TOV =70.174. When the field of view is further increased, the acquired image will show obvious vignetting, resulting in severe artifacts in the FPM-reconstructed image.

[0089] Step 22: Adjust the spacing between the sample and the lighting device to achieve the optimal spacing h. TOV .

[0090] Step 23: Determine the light source unit in the lighting device that needs to be moved.

[0091] Step 24, when the side length of the object plane's field of view is adjusted to X, Based on the optimal value h of the distance between the sample and the lighting device TOV The optimal value of L between the centers of the object plane in the field of view when adjacent light source units are lit, when the acquired image contains n×n brightfield images. TOV Determine the moving distance and direction of each light source unit that needs to be moved;

[0092] Step 25: Adjust the position of each light source unit that needs to be moved according to the moving distance and moving direction of each light source unit.

[0093] In this embodiment, the purpose of eliminating vignetting can be achieved by adjusting the position of each light source unit in the entire light source unit on the lighting device.

[0094] Specifically, when the field of view size is equal to the side length of the inscribed square of the pupil (3.88), the number of LED units in the determined flat LED is 24. Each LED unit in the determined portion of the LED, as well as the LED unit at the center of the flat LED (a total of 25 LEDs), are numbered as follows: Figure 5 As shown, the LED unit at the center is numbered 13. Figure 5 The diagram shows the optimized LED surface design scheme, where (a) is a schematic diagram of realizing the planar movement effect of the LED using height movement (for experimentation, to reduce manufacturing costs), (b) is a schematic diagram of setting the LED numbers and the center coordinates of the object plane field of view, (c-1) is a schematic diagram of the position movement of the corresponding object plane field of view for LED units numbered 3, 11, 15, and 23, (c-2) is a schematic diagram of the position movement of the corresponding object plane field of view for LED units numbered 7, 9, 17, and 19, (c-3) is a schematic diagram of the position movement of the corresponding object plane field of view for LED units numbered 8, 12, 14, and 18, (c-4) is a schematic diagram of the position movement of the corresponding object plane field of view for LED units numbered 1, 5, 21, and 25, and (c-5) is a schematic diagram of the position movement of the corresponding object plane field of view for LED units numbered 2, 4, 6, 10, 16, 20, 22, and 24.

[0095] In one embodiment, step 23, determining the light source unit that needs to be moved in the lighting device, may include:

[0096] For each light source unit, if the image acquired when the light source unit is lit is a bright field image, if there exists p (p = 1, 2, 3, 4), where p is the vertex number of the field of view, such that... The light source unit needs to be moved, where C (center) is the center point of the object surface field of view when the light source unit at the center position of the lighting device is lit. p(m) represents the vertex p in the field of view of the object surface when the light source unit numbered m is lit; D is the pupil diameter on the object surface.

[0097] If the image captured when the light source unit is lit is a dark field image, and if there exists p (p = 1, 2, 3, 4), where p is the vertex number of the field of view, such that... Then the light source unit needs to be moved.

[0098] Specifically, in step 24, when the side length of the object plane's field of view is adjusted to X, Based on the optimal value h of the distance between the sample and the lighting device TOV The optimal value of L between the centers of the object plane in the field of view when adjacent light source units are lit, when the acquired image contains n×n brightfield images. TOV Determine the moving distance and direction of each light source unit that needs to be moved, including:

[0099] Step 241, determine the moving direction of each light source unit that needs to be moved: the image acquired when the light source unit is lit is a bright field image, using the following formula:

[0100]

[0101] Wherein, sgp(m) is the moving direction of the light source unit numbered m, C(center) is the center point of the object surface field of view when the light source unit at the center position of the lighting device is lit, and C(m) is the center point of the object surface field of view when the light source unit numbered m is lit.

[0102] The image captured when the light source unit is lit is a dark field image, and the following formula is used:

[0103]

[0104] Step 242: Based on the coordinates of the center point of the object plane field of view when each light source unit that needs to be moved is lit, determine the moving distance of the object plane field of view when each light source unit that needs to be moved is lit; specifically, the following methods can be used:

[0105] (1) When the light source unit is lit, the image acquired is a bright field image, and the coordinates of C(m) are (x, y). C(m) is the center point of the object surface field of view when the light source unit numbered m is lit. Then:

[0106] If |x|=|y|, x≠0, y≠0 N(m) is the distance the object plane's field of view moves when the light source unit numbered m is lit, and C(center) is the center point of the object plane's field of view when the light source unit at the center of the lighting device is lit. max () is to make The C corresponding to the p value that holds true for (p=1,2,3,4)p (), C p (m) represents the vertex p in the object plane field of view when the light source unit numbered m is lit. If the number of p values ​​satisfying the condition is greater than 1, then any p value is chosen. For example, if p = 1, then... Then C max (m)=1(), if p=1 and p=2, both can make Then V max (m) = 1 or C max (m)=2().

[0107] If x = 0 or y = 0 X is the side length of the object plane's field of view, which is selected as needed.

[0108] Other situations Among them, C max (Center) refers to the field of view of the object surface and C when the light source unit at the center of the lighting device is lit. max () Vertices with the same p value, b1() is C max (Center) and C max () The intersection of the line connecting the two points and the edge of the pupil;

[0109] (2) When the light source unit is lit, the image acquired is a dark field image, and the coordinates of C(m) are (x, y). Then:

[0110] If |x|=|y|, x≠0, y≠0 Among them, C min () is to make The C corresponding to the p value that holds true for (p=1,2,3,4) p ();

[0111] If x = 0 or y = 0

[0112] Other situations Among them, C min (Center) refers to the field of view of the object surface and C when the light source unit at the center of the lighting device is lit. min () Vertices with the same p value, b2() is C min (Center) and C min () is the intersection of the line and the edge of the pupil.

[0113] Step 243: Based on the distance the object plane field of view moves when each light source unit that needs to be moved is lit, and the optimal value h of the distance between the sample and the lighting device... TOV The optimal value of L between the centers of the object plane in the field of view when adjacent light source units are lit, when the acquired image contains n×n brightfield images. TOVCalculate the moving distance for each light source unit that needs to be moved. Specifically, the following formula can be used:

[0114]

[0115] Where, d m h represents the moving distance of the light source unit numbered m, i.e., the moving distance on the lighting device. TOV The optimal distance between the sample and the illumination device is given by N(m), where N(m) is the distance the field of view on the object plane moves when the m-numbered light source unit is lit, D is the pupil diameter on the object plane, NA is the numerical aperture of the objective lens, and L is the distance between the sample and the illumination device. TOV Let d be the optimal center distance between the object plane's field of view when adjacent light source units are lit, given that the acquired image contains n×n brightfield images. LED C is the distance between adjacent light source units in the lighting device, C(center) is the center point of the object surface field of view when the light source unit at the center position of the lighting device is lit, and C(m) is the center point of the object surface field of view when the light source unit numbered m is lit.

[0116] Figure 6 This diagram illustrates the verification of the optimized LED surface design and the search for the maximum field of view of the object plane in a single FPM recovery. (a) shows the experimental diagram for finding the intensity "soft boundary," and (b) shows the experimental diagram for finding the phase "soft boundary." Theoretically, the achievable maximum field of view of the object plane in a single recovery is 3.8 mm (the side length of the inscribed square of the pupil). However, due to 1. unavoidable noise in the experimental environment; and 2. changes in the overlap rate caused by the expansion of the object plane field of view, a "soft boundary" exists for the maximum field of view of the object plane in a single high-throughput imaging. This boundary was sought through multiple experiments. Regarding intensity recovery, calculations and comparison with the USAF table yielded a theoretical resolution of approximately 8-5. Considering experimental errors, the theoretical resolution boundary was set to 8-4. Figure 6 As shown in (a), when the edge length of the object plane field of view is greater than 2.6 mm, the resolution drops rapidly, indicating that the "intensity soft boundary" is 2.6 mm. Using the same concept, the "phase soft boundary" is experimentally determined to be 1.4 mm, as shown in (a). Figure 6 As shown in (b) of the diagram.

[0117] Figure 7The optimal light source surface shape obtained using this design is shown. (a) is the calculation of the light source surface shape at a fixed height after converting different heights during acquisition. (b) is the light source surface shape when X = 2.6 mm. Single light sources marked in red are those that do not need to change position, while those marked in other colors are those that need to be moved. Specific movement parameters for each light source and the corresponding numbers for different colored light sources are given. (c) is a schematic diagram of the scanning scheme for imaging a sample at X = 2.6 mm using a single light source. The arrows in the diagram indicate the point scanning order, which is not unique and can take other forms; this diagram only provides an example. In the experiment, a variable height strategy was used to move a single light source on the plane to save time and cost. However, in actual manufacturing, the optimal light source surface shape can be designed by fixing the distance h between the light source and the sample (the optimal h in this design is 70.174 mm). Furthermore… Figure 7 In the light source arrangement shown in (b), the red squares represent single light sources that do not need to be moved. These light sources can be arranged in any way that meets the overlap ratio requirements.

[0118] In one embodiment, step 2, adjusting the distance between some light source units and the sample based on the optimal center distance between the object plane fields of view when adjacent light source units are lit, in order to eliminate vignetting, may include:

[0119] First, after determining the position of some light source units in the lighting device, the new center point of the object surface field of view when each light source unit in the partial light source unit is lit;

[0120] Then, based on the new object plane field of view center point and the optimal center distance between the object plane field of view when adjacent light source units are lit, the distance between each light source unit and the sample in some light source units is determined using the following formula:

[0121]

[0122] Among them, h m Let m be the distance between the light source unit and the sample, D be the pupil diameter on the object surface, NA be the numerical aperture of the objective lens, and d be the distance between the light source unit and the sample. LED C is the distance between adjacent light source units in a lighting device. new (m) is the new center point of the object plane field of view when the light source unit numbered m is lit, C(center) is the center point of the object plane field of view when the light source unit at the center of the lighting device is lit, and C(m) is the center point of the object plane field of view when the light source unit numbered m is lit; L ToV This represents the optimal center distance between the object surface fields of view when adjacent light source units are lit.

[0123] Based on the same inventive concept as the optimization method for Fourier layered imaging illumination systems, embodiments of this application also provide an optimization apparatus for Fourier layered imaging illumination systems, the apparatus comprising:

[0124] The optimal value determination module for the center distance of the field of view is used to determine the optimal value of the center distance between the object surface fields of view when adjacent light source units in the illumination device are lit, in the case that the image acquired by the Fourier stacked imaging illumination system does not have vignetting and the acquired image contains multiple bright field images.

[0125] The adjustment module is used to adjust the position of some light source units in the lighting device or the distance between some light source units and the sample based on the optimal value of the center distance between the object surface field of view when adjacent light source units are lit, so as to eliminate vignetting.

[0126] The optimization device for the Fourier stacked imaging illumination system in this embodiment has the same inventive concept as the optimization method for the Fourier stacked imaging illumination system described above. Therefore, the specific implementation of this device can be found in the embodiment section of the optimization method for the Fourier stacked imaging illumination system described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0127] In summary, the method of this application has the following technical effects: it avoids the vignetting effect on the basis of the original system structure without the addition of extra hardware; it avoids the need for FPM experiments to rely on GPUs to process the large field-of-view images in blocks in parallel due to the vignetting phenomenon, which greatly reduces the system cost, reduces the complexity of the data, and improves the data recovery efficiency.

[0128] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An optimization method for a Fourier layered imaging illumination system, characterized in that, include: The optimal center distance between the object plane fields of view when adjacent light source units in the illumination device are lit, provided that the images acquired by the Fourier stacked imaging illumination system do not have vignetting and the acquired images contain multiple bright field images. Based on the optimal center distance between the object surface fields when the adjacent light source units are lit, adjust the position of some light source units in the lighting device or the distance between some light source units and the sample to eliminate vignetting.

2. The method as described in claim 1, characterized in that, The method further includes: The Fourier layered imaging illumination system constructs images that do not exhibit vignetting and contains multiple bright-field images, subject to the following constraints: Where X is the side length of the object plane's field of view, n is the number of rows or columns when multiple bright-field images are distributed, and n is an odd number. The diameter of the pupil on the object surface. The center distance between the object surface fields of view when adjacent light source units are lit.

3. The method as described in claim 2, characterized in that, The optimal center distance between the object surface fields of view when adjacent light source units in the lighting device are lit is calculated using the following formula: in, The acquired images contain The optimal center distance between the object surface fields of view when adjacent light source units are lit in Zhang Ming's field image. Let be the diameter of the pupil on the object surface.

4. The method as described in claim 1, characterized in that, Based on the optimal center distance between the object plane fields of view when adjacent light source units are lit, the positions of some light source units in the lighting device are adjusted to eliminate vignetting, including: The optimal spacing between the sample and the lighting device is determined using the following formula; in, The optimal spacing between the sample and the lighting device. The acquired images contain The optimal center distance between the object surface fields of view when adjacent light source units are lit in Zhang Ming's field image. The numerical aperture of the objective lens. The diameter of the pupil on the object surface. This refers to the distance between adjacent light source units in a lighting device; Adjust the spacing between the sample and the lighting device to achieve the optimal spacing value. ; Determine the light source unit in the lighting device that needs to be moved; When the side length of the object plane's field of view is adjusted to X... Based on the optimal spacing between the sample and the lighting device The acquired images contain Zhang Ming's field image shows the optimal center distance between the object plane's field of view when adjacent light source units are lit. The moving distance and direction of each of the light source units that need to be moved are determined. The position of each light source unit that needs to be moved is adjusted according to the moving distance and moving direction of each light source unit.

5. The method as described in claim 4, characterized in that, The step of determining the light source unit that needs to be moved in the lighting device includes: For each light source unit, if the image acquired when the light source unit is lit is a bright field image, if there exists p Let p be the vertex number of the field of view, such that Then the light source unit needs to be moved, wherein, The center point of the object surface field of view when the light source unit at the center of the lighting device is lit. The vertex with the number p in the field of view of the object surface when the light source unit numbered m is lit; The diameter of the pupil on the object surface; If the image captured when the light source unit is lit is a dark field image, if p exists Let p be the vertex number of the field of view, such that The light source unit then needs to be moved.

6. The method as described in claim 4, characterized in that, in, When the side length of the object plane's field of view is adjusted to X... , The pupil diameter on the object surface is determined based on the optimal value of the distance between the sample and the illumination device. The acquired images contain Zhang Ming's field image shows the optimal center distance between the object plane's field of view when adjacent light source units are lit. Determining the moving distance and direction of each of the light source units that need to be moved includes: Determine the moving direction of each of the light source units that needs to be moved: When the light source unit is lit, the image acquired is a bright field image, using the following formula: in, Let m be the direction of movement of the light source unit. The center point of the object surface field of view when the light source unit at the center of the lighting device is lit. The center point of the object surface field of view when the light source unit numbered m is lit; When the light source unit is lit, the captured image is a dark field image, and the following formula is used: Based on the coordinates of the center point of the object surface field of view when each of the light sources that need to be moved is lit, determine the moving distance of the object surface field of view when each of the light sources that need to be moved is lit. Based on the distance the object's field of view moves when each of the light source units that need to be moved is lit, and the optimal distance between the sample and the lighting device... The acquired images contain Zhang Ming's field image shows the optimal center distance between the object plane's field of view when adjacent light source units are lit. Calculate the moving distance of each of the light source units that need to be moved.

7. The method as described in claim 6, characterized in that, in, Based on the coordinates of the center point of the object plane field of view when each of the light sources that need to be moved is illuminated, the movement distance of the object plane field of view when each of the light sources that need to be moved is illuminated is determined, including: The image captured when the light source unit is lit is a bright field image. The coordinates are (x, y). Let m be the center point of the object's field of view when the light source unit is lit. Then: like , , , Let m be the distance the object plane's field of view moves when the light source unit numbered m is lit. The center point of the object surface field of view when the light source unit at the center of the lighting device is lit. In order to make The p-value that is true , When the light source unit numbered m is lit, the vertex with the field of view numbered p on the object surface is selected. If the number of p values ​​that satisfy the condition is greater than 1, then any p value is selected. If x=0 or y=0 X is the side length of the field of view of the object plane; Other situations ,in, When the light source unit at the center of the lighting device is lit, the field of view of the object surface and Vertices with the same p value for and The intersection of the line and the edge of the pupil; The image captured when the light source unit is lit is a dark field image. If the coordinates are (x, y), then: like , , ,in, In order to make The p-value that is true ; If x=0 or y=0 ; Other situations ,in, When the light source unit at the center of the lighting device is lit, the field of view of the object surface and Vertices with the same p value for and The point where the line intersects with the edge of the pupil.

8. The method as described in claim 6, characterized in that, in, Based on the distance the object's field of view moves when each of the light source units that need to be moved is lit, and the optimal distance between the sample and the lighting device... The acquired images contain The optimal center distance between the object surface fields of view when adjacent light source units are lit in Zhang Ming's field image. The moving distance of each of the light source units that needs to be moved is calculated using the following formula: in, Let m be the distance the light source unit moves. The optimal spacing between the sample and the lighting device. Let m be the distance the object plane's field of view moves when the light source unit numbered m is lit. The diameter of the pupil on the object surface. The numerical aperture of the objective lens. The acquired images contain The optimal center distance between the object plane's field of view when adjacent light source units are lit in Zhang Ming's field image. The distance between adjacent light source units in a lighting device. The center point of the object surface field of view when the light source unit at the center of the lighting device is lit. The center point of the object's field of view when the light source unit numbered m is lit.

9. The method as described in claim 1, characterized in that, Based on the optimal center distance between the object plane fields of view when adjacent light source units are lit, the distance between some light source units and the sample is adjusted to eliminate vignetting, including: After determining the positions of some light source units in the lighting device, determine the new center point of the object surface field of view when each light source unit in the partial light source unit is lit; Based on the new object plane field of view center point and the optimal center distance between the object plane fields of view when adjacent light source units are lit, the distance between each light source unit and the sample in a subset of light source units is determined using the following formula: in, Let m be the distance between the light source unit numbered m and the sample. The diameter of the pupil on the object surface. The numerical aperture of the objective lens. The distance between adjacent light source units in a lighting device. The new center point of the object plane field of view when the light source unit numbered m is lit. The center point of the object surface field of view when the light source unit at the center of the lighting device is lit. The center point of the object surface field of view when the light source unit numbered m is lit; This represents the optimal center distance between the object surface fields of view when adjacent light source units are lit.

10. An optimization device for a Fourier layered imaging illumination system, characterized in that, include: The optimal value determination module for the center distance of the field of view is used to determine the optimal value of the center distance between the object surface fields of view when adjacent light source units in the illumination device are lit, in the case that the image acquired by the Fourier stacked imaging illumination system does not have vignetting and the acquired image contains multiple bright field images. The adjustment module is used to adjust the position of some light source units in the lighting device or the distance between some light source units and the sample based on the optimal value of the center distance between the fields of view of the probe when the adjacent light source units are lit, so as to eliminate vignetting.

Images