Rapid focusing method and system for full-automatic microscopic imaging of high-throughput sample

Abstract

The invention discloses a rapid focusing method and system for full-automatic microscopic imaging of a high-throughput sample, and relates to the technical field of microscopic imaging, the type of a container is selected, and a positioning point and a central point of at least two non-collinear holes are selected; the method comprises the following steps: placing a container on an objective table, moving imaging equipment to perform automatic focusing on imaging units of a positioning point and a central point of the container, and collecting focus coordinates of at least six imaging units; performing error evaluation on the collected focus coordinates to obtain an error result; selecting a corresponding focus model according to an error result and solving to obtain a model coefficient; and obtaining the focus of the new position according to the focus model and the coefficient thereof in combination with the coordinates of the new position. According to the method, a plurality of positioning points are systematically selected, automatic focusing is carried out, and error evaluation and model solving are combined, so that a new position focus of high-throughput sample microscopic imaging can be quickly and accurately determined, the imaging efficiency and the image quality are remarkably improved, and the operation of a high-throughput experiment is more efficient and reliable.

Description

Technical Field

[0001] The present invention relates to the field of microscopic imaging technology, and more particularly to a fast focusing method and system for fully automatic microscopic imaging of high-throughput samples. Background Art

[0002] At present, the fully automatic microscopic imaging instruments on the market can perform microscopic imaging of any area of ​​a single specimen in a large area or of different specimens arranged on the stage at a time. Some instruments can even perform imaging of cell samples in a 384-well plate or hundreds of pathological tissues (tissue chips) at one time. However, due to the different levelness of the stage of the instrument itself and the thickness of the specimen carrier, the thickness of the specimen itself may cause different specimens and different imaging points to have different Z-axis focal heights. This requires the use of autofocus technology to solve this problem. However, for high-throughput samples, since autofocusing needs to be completed through algorithms, and even more so, it requires the use of additional focusing components. No matter how advanced the focusing algorithm is, how it is optimized, and no matter how sensitive the focusing components are, if each sample is fully autofocused, it will take a lot of time. In today's society, whether in scientific research or clinical practice, it is necessary to quickly and automatically image high-throughput samples in order to improve efficiency.

[0003] Therefore, how to save focusing time is a problem that those skilled in the art need to solve urgently. Summary of the invention

[0004] In view of this, the present invention provides a fast focusing method and system for fully automatic microscopic imaging of high-throughput samples to solve the problems existing in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] In one aspect, a fast focusing method for fully automated microscopic imaging of high-throughput samples is provided, comprising:

[0007] Select the container type, and use the center positions of the four vertex holes in the container as the positioning points, and select the center points of at least two other non-collinear holes;

[0008] The container is placed on the stage, and the mobile imaging device automatically focuses on the imaging units of the positioning point and the center point of the container, and collects the focal coordinates of at least six imaging units;

[0009] Perform error evaluation on the collected focus coordinates to obtain error results;

[0010] Select a corresponding focus model according to the error result, solve the focus model, and obtain the corresponding model coefficients;

[0011] According to the selected focus model and its coefficients, combined with the coordinates of the new position, the focus of the new position is obtained.

[0012] Optionally, the error evaluation is specifically:

[0013] Calculating the error between the actual surface and the ideal plane or curved surface according to the focal coordinates of at least six imaging units, and selecting whether to use a plane focal model or a curved surface focal model according to the calculated error result and an error threshold;

[0014] When the plane error is less than the error threshold, the plane focus model is selected;

[0015] When the error result is not less than the error threshold, it is determined whether the surface error is less than the error threshold;

[0016] When the surface error is less than the error threshold, the surface focus model is selected;

[0017] When the surface error result is not less than the error threshold, the installation errors of each installation stage of the correction device are checked, and then the focal coordinates of the imaging unit are re-collected.

[0018] Optionally, the error threshold is calculated according to the magnification of the imaging device, using the formula:

[0019]

[0020] Where λ is the wavelength of the illumination light; n is the refractive index of the optical medium; NA is the numerical aperture of the objective lens, which indicates the focusing ability of the optical system; n M is the adjustment coefficient, which is an empirical value after measurement and is 1 by default; e is the minimum resolvable distance.

[0021] Optionally, the difference between the actual z coordinate and the predicted z coordinate of each imaging unit is calculated, and the plane error and the curved surface error are calculated respectively; then the error set standard error SE is calculated, and the formula is:

[0022]

[0023] Here, s is the sample standard deviation and N is the sample size.

[0024] Optionally, the equation of the plane focus model is Ax+By+Cz+D=0.

[0025] Optionally, the equation of the surface focus model is z=Ax 2 +By 2 +Cxy+Dx+Ey+F.

[0026] Optionally, the imaging device uses a 4X lens, and the calculated error threshold is 0.015 mm.

[0027] Optionally, the calculation process of the model coefficient is:

[0028] According to the selected plane or surface focus model, the corresponding fitting plane equation formula or quadratic surface equation formula is used; in the fitting plane equation formula Ax+By+Cz+D=0, C defaults to -1; the quadratic surface equation formula z=Ax 2 +By 2 +Cxy+Dx+Ey+F, calculate the corresponding plane coefficients (A, B, D) or curved surface coefficients (A, B, C, D, E, F) based on the provided experimental data.

[0029] In another aspect, a fast focusing system for fully automated microscopic imaging of high-throughput samples is provided, comprising:

[0030] Point selection module, select the container type, and use the center positions of the four vertex holes in the container as the positioning points, and select the center points of at least two non-collinear holes;

[0031] A focusing module is used to place the container on the stage, and the mobile imaging device automatically focuses on the imaging units of the positioning point and the center point of the container, and collects the focal coordinates of at least six imaging units;

[0032] The error evaluation module performs error evaluation on the collected focus coordinates to obtain error results;

[0033] The model selection module selects the corresponding focus model according to the error result, solves the focus model, and obtains the corresponding model coefficients;

[0034] The focus output module obtains the focus of the new position according to the selected focus model and its coefficients combined with the coordinates of the new position.

[0035] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a fast focusing method and system for fully automatic microscopic imaging of high-throughput samples, which can quickly and accurately determine the new position focus of high-throughput sample microscopic imaging by systematically selecting multiple positioning points and performing automatic focusing, combined with error evaluation and model solving, thereby significantly improving imaging efficiency and image quality, reducing the need for manual intervention, and making the operation of high-throughput experiments more efficient and reliable. And the higher the sample throughput targeted by the present invention, the more time it saves, achieving the purpose of fast automatic focusing. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A flow chart of the method provided by the present invention;

[0038] Figure 2 A schematic diagram of orifice plate point selection provided by the present invention;

[0039] Figure 3 A plane fitting diagram provided by the present invention;

[0040] Figure 4 This is a surface fitting diagram provided by the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The present invention discloses a fast focusing method for fully automatic microscopic imaging of high-throughput samples. Figure 1 As shown, including:

[0043] Select the container type, and use the center positions of the four vertex holes in the container as the positioning points, and select the center points of at least two other non-collinear holes;

[0044] The container is placed on the stage, and the mobile imaging device automatically focuses on the imaging units of the positioning point and the center point of the container, and collects the focal coordinates of at least six imaging units;

[0045] Perform error evaluation on the collected focus coordinates to obtain error results;

[0046] Select a corresponding focus model according to the error result, solve the focus model, and obtain the corresponding model coefficients;

[0047] According to the selected focus model and its coefficients, combined with the coordinates of the new position, the focus of the new position is obtained.

[0048] Among them, Figure 2As shown, on the SBS 96-well plate, near the center of the four apex wells<A1,A12,H1,H12> For the positioning point, select the center points of 2 (or more) non-collinear holes.<C6,F7> , the X and Y positions of these 6+ points can be calculated according to the specifications of the orifice plate, which are known conditions. Place the orifice plate on the stage, and then move it to the position of the selected point, so that the fully automatic microscope imager can focus clearly and form an image <judged by the focus algorithm that calculates the image clarity in real time>, and record the Z-axis values ​​of the 6+ points respectively. Among them, H4 is the real-time position of the device, not the experimental data position.

[0049] Description of image clarity calculation and judgment: The Laplace transform variance is calculated for real-time video frames, and the optimal focus Z of the current point can be quickly found through the hill climbing algorithm and its set hill climbing threshold (10) combined with manual assistance.

[0050] In a specific embodiment, the error evaluation is specifically:

[0051] Calculating the error between the actual surface and the ideal plane or curved surface according to the focal coordinates of at least six imaging units, and selecting whether to use a plane focal model or a curved surface focal model according to the calculated error result and an error threshold;

[0052] When the plane error is less than the error threshold, the plane focus model is selected;

[0053] When the error result is not less than the error threshold, it is determined whether the surface error is less than the error threshold;

[0054] When the surface error is less than the error threshold, the surface focus model is selected;

[0055] When the surface error result is not less than the error threshold, the installation errors of each installation stage of the correction device are checked, and then the focal coordinates of the imaging unit are re-collected.

[0056] In a specific embodiment, the error threshold is calculated according to the magnification of the imaging device, and the formula is:

[0057]

[0058] Where λ is the wavelength of the illumination light; n is the refractive index of the optical medium; NA is the numerical aperture of the objective lens, which indicates the focusing ability of the optical system; n M is the adjustment coefficient, which is an empirical value after measurement and is 1 by default; e is the minimum resolvable distance.

[0059] In a specific embodiment, the difference between the actual z coordinate and the predicted z coordinate of each imaging unit is calculated, and the plane error and the curved surface error are calculated respectively;

[0060] Then calculate the error set standard error SE, the formula is:

[0061]

[0062] Here, s is the sample standard deviation and N is the sample size.

[0063] In a specific embodiment, the equation of the plane focus model is Ax+By+Cz+D=0.

[0064] In a specific embodiment, the equation of the surface focus model is z=Ax 2 +By 2 +Cxy+Dx+Ey+F.

[0065] In a specific embodiment, the imaging device uses a 4X lens, and the calculated error threshold is 0.015 mm.

[0066] In a specific embodiment, the calculation process of the model coefficient is:

[0067] According to the selected plane or surface focus model, the corresponding fitting plane equation formula or quadratic surface equation formula is used; for example, Figure 3 and Figure 4 As shown; in the plane fitting formula Ax+By+Cz+D=0, C defaults to -1; the quadratic surface equation formula z=Ax 2 +By 2 +Cxy+Dx+Ey+F, calculate the corresponding plane coefficients (A, B, D) or curved surface coefficients (A, B, C, D, E, F) based on the provided experimental data.

[0068] At the new position coordinate (x, y), the focus z of the new position (x, y) is quickly calculated according to the formula of the selected model and the coefficients obtained.

[0069] Plane: z = Ax + By + D

[0070] Surface: z = Ax 2 +By 2 +Cxy+Dx+Ey+F.

[0071] A specific embodiment is introduced below to further illustrate the method adopted by the present invention.

[0072] In fully automatic microscope imagers, the focal planes of each of our devices are inconsistent due to manufacturing errors in the component generation process, assembly errors in the accessory assembly process, measurement errors, and environmental errors caused by environmental changes. Each point needs to be focused. In actual use, especially in the use environment of high-throughput well plates, it involves focusing on hundreds of points, which is very time-consuming. Long-term focusing also affects the growth of living cells. We are thinking about fitting a highly matched focal plane (surface) with as little focusing as possible, and focusing on multiple points very quickly based on the depth of field of the device.

[0073] The following is the actual operation process (for a 96-well culture experiment).

[0074] Experimental preparation:

[0075] After the experimenter completes the preliminary work of the experiment, the cultured cells are planted into each well of a 96-well plate (standard SBS well plate), loaded onto the CellImage Mini device, and the supporting software is started.

[0076] Select 6 observation points

[0077] Select the holes at the four corners, and randomly select two points in the middle, requiring that any three points are not collinear. (Why do we need to select 6 points in total: because fitting a surface requires at least 6 points; fitting a plane requires at least 3 points. The more points, the higher the accuracy.) The result is that these six points are selected:

[0078] The corresponding coordinate records are shown in Table 1:

[0079] Table 1 Coordinate record table

[0080]

[0081] Record the focus position.

[0082] According to the serial number, each point is manually focused using the Laplace transform focus clarity evaluation method to record the focus Z coordinate. The higher the Laplace transform value, the clearer the clarity. Combined with the experimenter's observation and confirmation, the focus z values ​​of the following points are finally recorded, as shown in Table 2.

[0083] Table 2 Focus z value record table

[0084]

[0085] Fitting focal plane or focal surface

[0086] According to the lens parameters of the fully automatic microscopic imaging instrument, the depth of field is calculated to be 0.015mm, so the error threshold is 0.015mm. In the plane fitting formula Ax+By+Cz+D=0, C defaults to -1. Six formulas are listed for each hole to calculate the plane coefficients A, B, and D; through the surface fitting formula z=Ax 2 +By 2 +Cxy+Dx+Ey+F lists 6 formulas to calculate the surface coefficients A, B, C, D, E, and F. Through the recorded 6 points, the newly obtained plane coefficients A, B, D, and the surface coefficients A, B, C, D, E, and F, the plane standard error and surface standard error are calculated respectively. Among them, the coefficients of the plane equation: A = -0.0002612342259156082, B = -0.0006085718682672797, D = 6.646844467528469; the average value of the plane model error: 0.021222222222222418mm; the standard deviation of the plane model error: 0.023058164273717454mm; the coefficient of the quadratic surface equation A = -0.031290347217024 , B=0.010412.5575112282; C=-1.539201539245e-05; D=2.40440189094115, E=-1.215334138535305, F=1.3963484643087907; the average value of the quadratic surface model error: 0.003935826408123806mm; the standard deviation of the quadratic surface model error: 0.004995669870142741mm.

[0087] The calculation results are: plane standard error = 0.023058164273717454mm, surface standard error = 0.004995669870142741mm. Based on the standard error and the error threshold = 0.015mm, it is concluded that the computing power required for the plane is less than that required for the surface due to the limited computing power of the computer equipment. Therefore, start with the plane selection. Whether to select the plane depends on whether the standard error of the plane is less than the error threshold. If the standard error of the plane is greater than the error threshold, it means that the plane model is not applicable, and the surface model selection judgment is entered. The surface model judgment is the same. If the standard error of the surface is less than the error threshold, it is applicable, otherwise it is not applicable. If neither model is applicable, it is necessary to check and correct the installation errors of the equipment at each installation stage, and then re-collect the focus information of the point. In this example, the plane standard error 0.023 (rounded)> the error threshold 0.015; according to the rule, enter the surface selection stage. According to the surface standard error 0.005 (rounded) is less than the error threshold 0.015. So I decided to choose the surface.

[0088] The focal surface fitted according to the 6 points provided is as follows Figure 4 shown.

[0089] The focus of other points can be obtained by the surface coefficients A, B, C, D, E, F and the surface fitting formula: z = Ax 2 +By 2 +Cxy+Dx+Ey+F quickly calculates the focus z provided by the (x,y) point;

[0090] Optimization effect

[0091] Before optimization: 96 points, each point takes 4 minutes to focus, and the time required for 96 points is 96*4=384.

[0092] After optimization: 96 points, at least 6 non-collinear points are provided, and the focusing time is 6*4=24. The 24 minutes are for automatic focus; in the actual manual focus mode, the time for 6 points is within 6 minutes, which involves the determination of the clarity evaluation value and the actual clarity. The remaining 96 points are obtained through calculation, and the basic time is at the millimeter level and can be ignored.

[0093] Time saved: 384-6=378, increased 384 / 6=64 times.

[0094] This method is superior to other high-throughput well plates such as 96-well plates on a flat surface, can improve efficiency, save more time, and provide good support for cell culture and observation.

[0095] In another aspect, a fast focusing system for fully automated microscopic imaging of high-throughput samples is provided, comprising:

[0096] Point selection module, select the container type, and use the center positions of the four vertex holes in the container as the positioning points, and select the center points of at least two non-collinear holes;

[0097] A focusing module is used to place the container on the stage, and the mobile imaging device automatically focuses on the imaging units of the positioning point and the center point of the container, and collects the focal coordinates of at least six imaging units;

[0098] The error evaluation module performs error evaluation on the collected focus coordinates to obtain error results;

[0099] The model selection module selects the corresponding focus model according to the error result, solves the focus model, and obtains the corresponding model coefficients;

[0100] The focus output module obtains the focus of the new position according to the selected focus model and its coefficients combined with the coordinates of the new position.

[0101] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0102] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fast focusing method for fully automatic microscopic imaging of high-throughput samples, characterized in that: include: Select the container type, and use the center positions of the four vertex holes in the container as the positioning points, and select the center points of at least two other non-collinear holes; The container is placed on the stage, and the mobile imaging device automatically focuses on the imaging units of the positioning point and the center point of the container, and collects the focal coordinates of at least six imaging units; Perform error evaluation on the collected focus coordinates to obtain error results; Select a corresponding focus model according to the error result, solve the focus model, and obtain the corresponding model coefficients; According to the selected focus model and its coefficients, combined with the coordinates of the new position, the focus of the new position is obtained.

2. A fast focusing method for fully automatic microscopic imaging of high-throughput samples according to claim 1, characterized in that: The error evaluation is specifically as follows: Calculating the error between the actual surface and the ideal plane or curved surface according to the focal coordinates of at least six imaging units, and selecting whether to use a plane focal model or a curved surface focal model according to the calculated error result and an error threshold; When the plane error is less than the error threshold, the plane focus model is selected; When the error result is not less than the error threshold, it is determined whether the surface error is less than the error threshold; When the surface error is less than the error threshold, the surface focus model is selected; When the surface error result is not less than the error threshold, the installation errors of each installation stage of the correction device are checked, and then the focal coordinates of the imaging unit are re-collected.

3. A fast focusing method for fully automatic microscopic imaging of high-throughput samples according to claim 2, characterized in that: According to the magnification of the imaging device, the error threshold is calculated using the formula: Where λ is the wavelength of the illumination light; n is the refractive index of the optical medium; NA is the numerical aperture of the objective lens, which indicates the focusing ability of the optical system; n M is the adjustment coefficient; e is the minimum resolvable distance.

4. A fast focusing method for fully automatic microscopic imaging of high-throughput samples according to claim 2, characterized in that: Calculate the difference between the actual z coordinate and the predicted z coordinate of each imaging unit, and calculate the plane error and the surface error respectively; then calculate the error set standard error SE, the formula is: Here, s is the sample standard deviation and N is the sample size.

5. The fast focusing method for fully automatic microscopic imaging of high-throughput samples according to claim 2, characterized in that: The equation of the plane focus model is Ax+By+Cz+D=0.

6. A fast focusing method for fully automatic microscopic imaging of high-throughput samples according to claim 2, characterized in that: The equation of the surface focus model is z = Ax 2 +By 2 +Cxy+Dx+Ey+F.

7. The fast focusing method for fully automatic microscopic imaging of high-throughput samples according to claim 3, characterized in that: The imaging device uses a 4X lens, and the calculated error threshold is 0.015mm.

8. The fast focusing method for fully automatic microscopic imaging of high-throughput samples according to claim 1, characterized in that: The calculation process of the model coefficients is: According to the selected plane or surface focus model, the corresponding fitting plane equation formula or quadratic surface equation formula is used; in the fitting plane equation formula Ax+By+Cz+D=0, C defaults to -1; the quadratic surface equation formula z=Ax 2 +By 2 +Cxy+Dx+Ey+F, calculate the corresponding plane coefficients (A, B, D) or curved surface coefficients (A, B, C, D, E, F) based on the provided experimental data.

9. A fast focusing system for fully automatic microscopic imaging of high-throughput samples, characterized in that: A fast focusing method for fully automatic microscopic imaging of high-throughput samples according to any one of claims 1 to 8, comprising: Point selection module, select the container type, and use the center positions of the four vertex holes in the container as the positioning points, and select the center points of at least two non-collinear holes; A focusing module is used to place the container on the stage, and the mobile imaging device automatically focuses on the imaging units of the positioning point and the center point of the container, and collects the focal coordinates of at least six imaging units; The error evaluation module performs error evaluation on the collected focus coordinates to obtain error results; The model selection module selects the corresponding focus model according to the error result, solves the focus model, and obtains the corresponding model coefficients; The focus output module obtains the focus of the new position according to the selected focus model and its coefficients combined with the coordinates of the new position.

Images