Image based autofocus of optical systems

Abstract

The present disclosure describes illumination methods and systems for illumination as well as methods and systems for autofocusing the systems. The systems can be used for, for example, microscopy and sequencing platforms. The methods and systems of the present disclosure can provide fast and accurate autofocusing, which can reduce error and improve system throughput.

Description

W O 2 0 2 4 / 1 7 A 2 (12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property Organization International Bureau (10) International Publication Number VITE, Michael; 10055 Barnes Canyon Road, San Diego, California 92121 (US). (74) Agent: SWABECK, Joseph et al.; Wilson Sonsini Goodrich & Rosati, 650 Page Mill Road, Palo Alto, Cali- fornia 94304 (US). (43) International Publication Date 22 August 2024 (22.08.2024) WO 2024 / 173403 A2 WIPO PCT (51) International Patent Classification: H04N 23 / 951 (2023.01) (21) International Application Number: PCT / US2024 / 015602 (22) International Filing Date: 13 February 2024 (13.02.2024) (81) (25) Filing Language: English (26) Publication Language: English 63 / 484,723 13 February 2023 (13.02.2023) US (30) Priority Data: (71) Applicant: ELEMENT BIOSCIENCES, INC. [US / US]; 10055 Barnes Canyon Road, San Diego, California 92121 (US). (72) Inventors: JIANG, Yanfei; 10055 Barnes Canyon Road. San Diego, California 92121 (US). CHEN, Steve Xi- angling: 10055 Banes Canyon Road, San Diego, Cali- fornia 92121 (US). NEYSMITH, Jordan; 10055 Barnes Canyon Road, San Diego, California 92121 (US). PRE- Designated States (unless otherwise indicated, for every kind of national protection available): AE, AG, AL, AM, AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, BZ, CA, CH, CL, CN, CO, CR, CU, CV, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, HN, HR, HU, ID, IL, IN, IO. IR, IS, IT. JM, JO, JP. KE, KG. KH, KN, KP, KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, MG, MK, MN, MU, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, OM, PA, PE, PG, PH, PL, PT, QA, RO, RS, RU. RW. SA, SC, SD, SE, SG. SK, SL. ST. SV. SY, TH. TJ. TM, TN. TR. TT, TZ, UA, UG, US, UZ, VC, VN, WS. ZA, ZM, ZW. (84) Designated States (unless otherwise indicated, for every kind of regional protection available): ARIPO (BW, CV, GH, GM, KE, LR, LS, MW, MZ, NA, RW, SC, SD, SL, ST, SZ, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, (54) Title: IMAGE BASED AUTOFOCUS OF OPTICAL SYSTEMS In focus z axis Out of focus z axis In focus Out of focus z axis 609 z axis 609 x shift x axis -- Sample Stage Focal Plane of the objective FIG. 6A 608 center O (57) Abstract: The present disclosure describes illumination methods and systems for illumination as well as methods and systems for autofocusing the systems. The systems can be used for, for example, microscopy and sequencing platforms. The methods and systems of the present disclosure can provide fast and accurate autofocusing, which can reduce error and improve system throughput. [Continued on next page] WO 2024 / 173403 A2 RU, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, LV, MC, ME, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Published: - without international search report and to be republished upon receipt of that report (Rule 48.2(g)) WO 2024 / 173403 1 PCT / US2024 / 015602 IMAGE BASED AUTOFOCUS OF OPTICAL SYSTEMS CROSS-REFERENCE

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 484,723, filed February 13, 2023, which is incorporated herein by reference in its entirety. BACKGROUND

[0002] In many fluorescence-based genomic testing assays, e.g., genotyping or nucleic acid sequencing, dye molecules that are attached to nucleic acid molecules tethered on a substrate are excited using an excitation light source, a fluorescent signal is generated in spatially-localized position(s) on the substrate, and the fluorescence is subsequently imaged through an optical system onto an image sensor. An analysis process is then used to analyze the images, find the positions of labeled molecules (or clonally amplified clusters of molecules) on the substrate, and quantify the fluorescence photon signal in terms of wavelength and spatial coordinates. This process may then be correlated with the degree to which a specific chemical reaction, e.g., a hybridization event or base addition event, occurred in the specified locations on the substrate. Imaging-based methods provide large scale parallelism and multiplexing capabilities, which help to drive down the cost and accessibility of such technologies. SUMMARY

[0003] Described herein are methods and systems for autofocusing optical systems, e.g., optical systems for imaging sequencing reactions, so that optical signals can be acquired in-focus and relied upon for generating accurate sequencing analysis results. The systems and methods described herein can utilize a single image to conveniently and accurately determine a z shift for autofocusing the optical system. The single image can be acquired using an image sensor of the optical system after tilting the sample stage relative to the image sensor, without the need for any dedicated hardware, e.g., an autofocus (AF) laser or an AF sensor, which are used for autofocusing purposes only. The image-based autofocusing methods and systems described herein advantageously save machinery costs and reduce complexity of the optical system compared to existing autofocusing methods using AF lasers and / or AF sensors. Additionally, the methods and systems herein require only a single image, which reduces time consumption and computational complexity when compared with existing autofocusing methods that rely on at least two images and / or machine learning algorithms.

[0004] The present disclosure provides for a method for focusing an optical system, comprising: receiving an image of a substrate of the optical system, wherein a portion and less than all of the image is in focus, and wherein the portion of the image in focus is offset from a center of the WO 2024 / 17340 2 PCT / US2024 / 015602 image; determining, using at least a distance from the portion of the image in focus and the center of the image, an amount of defocus in the image; and adjusting a parameter of the optical system to adjust for the defocus. In some embodiments, the image is an image of a flow cell, and wherein the substrate is a flow cell. In some embodiments, the adjusting of (c) is an automated adjusting. In some embodiments, the image is received from an autofocus element. In some embodiments, the determining is done in at most about 600 milliseconds (ms). In some embodiments, the determining is done within at most about 100 ms. In some embodiments, the method further comprises, prior to (a), imaging a substrate using a light source and a detector to generate the image. In some embodiments, the determining is performed using the image and no additional images. In some embodiments, the image comprises a length or width that is in a range from about 0.1 millimeters (mm) to about 5 centimeters (cm). In some embodiments, the image comprises a length or width that is in a range from about 0.5 mm to about 9 mm. In some embodiments, an error in the amount of defocus from a true amount of defocus is at most about 400 nanometers (nm). In some embodiments, an error in the amount of defocus from a true amount of defocus is at most about 100 nanometers (nm). In some embodiments, an error in the amount of defocus from a true amount of defocus is at most about 50 nanometers (nm). In some embodiments, a center of the in focus region is determined using an image processing algorithm. In some embodiments, the image processing algorithm comprises determining the center of the in focus region by separating the image into a predetermined number of regions and using a sum or average intensity of each region to identify the location of the in focus region. In some embodiments, image intensity or spatial frequency information of the location of the in focus region is used to locate the center of the in focus region. In some embodiments, information about a geometrical pattern in the image determines the image processing algorithm.

[0005] The present disclosure provides for a method of focusing an optical system, comprising: imaging, using a detector, a substrate tilted at a tilt angle, wherein an image of the substrate comprises an in focus portion and an out-of-focus portion; determining, using a processor, a defocus of the optical system based at least in part on the tilt angle and a distance of the in focus portion from a center of the image; adjusting the substrate to remove the tilt angle; and adjusting the substrate by the defocus, thereby focusing the optical system. In some embodiments, the determining of (b) further comprising using a vector of the in focus portion from the center of the image. In some embodiments, the method further comprises a motor coupled to the substrate configured to impart the tilt angle. In some embodiments, the detector is a portion of an autofocusing element. In some embodiments, the optical system further comprises an additional detector configured to image the substrate. In some embodiments, the method further comprises, prior to (a), tilting the substate to the tilt angle. In some embodiments, the method further WO 2024 / 17340 PCT / US2024 / 015602 3 comprises, subsequent to (d), de-tilting the substrate. In some embodiments, the tilting is tilting of a plane orthogonal to an optical axis of the optical system. In some embodiments, the tilt angle is from about 0.01 to about 89 degrees. In some embodiments, the tilt angle is from about 0.05 to about 15 degrees. In some embodiments, an angular resolution of the tilt angle is from about 0.001 degrees to about 0.2 degrees. In some embodiments, an angular resolution of the tilt angle is from about 0.01 degrees to about 0.1 degrees. In some embodiments, an angular resolution of the tilt angle is from about 0.01 degrees to about 0.08 degrees. In some embodiments, the determining is performed using the image and no additional images. In some embodiments, the substrate comprises a flow cell comprising: one or more surfaces; at least one hydrophilic polymer coating layer; a plurality of oligonucleotide molecules attached to the at least one hydrophilic polymer coating layer; and at least one discrete region of the one or more surfaces that comprises a plurality of clonally-amplified nucleic acid molecules immobilized to the plurality of attached oligonucleotide molecules, wherein the plurality of immobilized clonally amplified sample nucleic acid molecules are present at a distance less than / (2*NA), wherein λ is the center wavelength of an excitation energy source and NA is the numerical aperture of the optical system. In some embodiments, the substrate comprises a beaded flow cell. In some embodiments, the beaded flow cell comprises a surface comprising fluorescent beads chemically immobilized to the substrate. In some embodiments, the fluorescent beads are randomly distributed on the surface. In some embodiments, the fluorescent beads comprise at least about 4 types of beads configured to emit different colors in response to excitation from a laser. In some embodiments, an error in the distance from a focal plane to a true distance from the focal plane is at most about 400 nanometers (nm). In some embodiments, an error in the distance from the focal plane to a true distance from the focal plane is at most about 100 nanometers (nm). In some embodiments, an error in the distance from the focal plane to a true distance from the focal plane is at most about 50 nanometers (nm). In some embodiments, (d) occurs prior to the optical system imaging a nucleic acid molecule immobilized to the substrate in a first flow cycle. In some embodiments, the method further comprises repeating (a) - (d) to refocus the optical system for a second flow cycle.

[0006] The present disclosure provides a method of focusing an optical system, comprising: imaging, using a detector tilted at a tilt angle, a substrate, wherein an image of the substrate comprises an in focus portion and an out-of-focus portion; determining, using a processor, a defocus of the optical system based at least in part on the tilt angle and a distance of the in focus portion from a center of the image; and adjusting the substrate by the defocus, thereby focusing the optical system. In some embodiments, the method further comprises adjusting the substrate by the defocus, thereby placing the substrate into focus. In some embodiments, the method WO 2024 / 173403 4 PCT / US2024 / 015602 further comprises, prior to (a), tilting the detector to the tilt angle. In some embodiments, the method further comprises, subsequent to (c), de-tilting the detector. In some embodiments, the tilting is tilting of a plane orthogonal to an optical axis of the optical system. In some embodiments, the tilt angle is from about 0.01 to about 89 degrees. In some embodiments, the tilt angle is from about 0.05 to about 15 degrees. In some embodiments, the determining is performed using the image and no additional images. In some embodiments, an error in the amount of defocus from a true amount of defocus is at most about 400 nanometers (nm). In some embodiments, an error in the amount of defocus from a true amount of defocus is at most about 100 nanometers (nm). In some embodiments, an error in the amount of defocus from a true amount of defocus is at most about 50 nanometers (nm). In some embodiments, the method further comprises calibrating a pivot point of the optical system. In some embodiments, the calibrating of the pivot point comprises de-tilting the substrate, the detector, or an autofocus sensor.

[0007] The present disclosure provides for a method for autofocus of an optical system, comprising: tilting a sample stage of the optical system by a tilt angle, wherein a sample is immobilized on the sample stage; obtaining, by an image sensor of the optical system, an image of the sample on the tilted sample stage; determining, by a processor, a z shift based on: the tilt angle; and a x-y plane shift from a center of the image, wherein the x-y plane shift is determined based on an in-focus region of the image; and moving the sample stage relative to a focal plane of an objective lens of the optical system by the determined z shift thereby bringing the sample in-focus.

[0008] The present disclosure provides for a method for autofocus of an optical system, comprising: tilting an image sensor of the optical system by a tilt angle; obtaining, by the tilted image sensor of the optical system, an image of a sample, wherein the sample is immobilized on a sample stage; determining, by a processor, a z shift based on: the tilt angle; and a x-y plane shift from a center of the image, wherein the x-y plane shift is determined based on an in-focus region of the image; and moving the sample stage relative to a focal plane of an objective lens of the optical system by the determined z shift thereby bringing the sample in-focus.

[0009] The present disclosure provides for a method for autofocus of an optical system, comprising: tilting a sample stage of the optical system by a tilt angle, wherein a sample is immobilized on the sample stage; obtaining, by an autofocus (AF) sensor of the optical system, an image of the sample on the tilted sample stage, wherein the AF sensor is different from an image sensor of the optical system; determining, by a processor, a z shift based on the tilt angle and a x-y plane shift from a center of the image, wherein the x-y plane shift is determined based on an in-focus region of the image; and moving the sample stage relative to a focal plane of an WO 2024 / 173403 5 PCT / US2024 / 015602 objective lens of the optical system by the determined z shift thereby bringing the sample in- focus.

[0010] The present disclosure provides for a method for autofocus of an optical system, comprising: tilting an AF sensor of the optical system by a tilt angle, wherein the AF sensor is different from an image sensor of the optical system; obtaining, by the AF sensor of the optical system, an image of the sample, wherein the sample is immobilized on a sample stage; determining, by a processor, a z shift based on the tilt angle and a x-y plane shift from a center of the image, wherein the x-y plane shift is determined based on an in-focus region of the image; and moving the sample stage relative to a focal plane of an objective lens of the optical system by the determined z shift thereby bring the sample in-focus with the optical system. In some embodiments, the method further comprises: calibrating a pivot point of the optical system. In some embodiments, calibrating the pivot point of the optical system comprises: tilting the sample stage, the image sensor, or the AF sensor by the tilt angle or a second tilt angle; acquiring, by the AF sensor or the image sensor, a calibration image of the sample immobilized on the sample stage; determining, by the processor, a pivot point offset based on a region center of an in-focus region of the calibration image and an image center of the calibration image; and de-tilting the sample stage, the image sensor, or the AF sensor by the tilt angle or the second tilt angle. In some embodiments, the method further comprises: de-tilting the tilted sample stage by the tilt angle. In some embodiments, the method further comprises: de-tilting the tilted image sensor by the tilt angle. In some embodiments, the method further comprises: de-tilting the tilted AF sensor by the tilt angle. In some embodiments, tilting the sample stage of the optical system by the tilt angle is about a x or y axis. In some embodiments, tilting the sample stage of the optical system by the tilt angle is within a x-z plane or y-z plane. In some embodiments, tilting the AF sensor or image sensor of the optical system by the tilt angle is about x or y axis. In some embodiments, tilting the AF sensor or image sensor of the optical system by the tilt angle is within a x-z plane or y-z plane. In some embodiments, the tilt angle is in a range from 0.01 degrees to 89 degrees. In some embodiments, the tilt angle is in a range from 0.05 degrees to 15 degrees. In some embodiments, the tilt angle is clockwise about the x or y axis. In some embodiments, the tilt angle is counter-clockwise about the x or y axis. In some embodiments, the image of the sample obtained by the AF sensor or image sensor comprises a single image. In some embodiments, the AF sensor is only used for acquiring signals for autofocusing the optical system. In some embodiments, the image sensor is used for autofocusing the optical system and for imaging using the optical system after autofocusing. In some embodiments, the optical system lacks an AF illumination source that is only used for autofocusing but not for imaging. In some embodiments, the method for autofocus of the optical system is completed in 100 to 990 本公開內容描述了照明方法和照明系統，以及用於自動對焦系統的方法和系統。 該系 統可用於例如顯微鏡和測序平台。 本發明公開的方法和系統能夠提供快速、准確的自 動對焦，從而減少誤差並提高系統吞吐量。 摘要

Claims

CLAIMSWhat is claimed is:

1. A method for focusing an optical system, comprising:(a) receiving an image of a substrate of said optical system, wherein a portion and less than all of said image is in focus, and wherein said portion of said image in focus is offset from a center of said image;(b) determining, using at least a distance from said portion of said image in focus and said center of said image, an amount of defocus in said image; and(c) adjusting a parameter of said optical system to adjust for said defocus.

2. The method of claim 1, wherein said image is an image of a flow cell, and wherein said substrate is a flow cell.

3. The method of claim 1, wherein said adjusting of (c) is an automated adjusting.

4. The method of claim 1, wherein said image is received from an autofocus element.

5. The method of claim 1, wherein said determining is done in at most about 600 milliseconds(ms).

6. The method of claim 1, wherein said determining is done within at most about 100 ms.

7. The method of claim 1, further comprising, prior to (a), imaging a substrate using a light source and a detector to generate said image.

8. The method of claim 1, wherein said determining is performed using said image and no additional images.

9. The method of claim 1, wherein said image comprises a length or width that is in a range from about 0.1 millimeters (mm) to about 5 centimeters (cm).

10. The method of claim 1, wherein said image comprises a length or width that is in a range from about 0.5 mm to about 9 mm.

11. The method of claim 1, wherein an error in said amount of defocus from a true amount of defocus is at most about 400 nanometers (nm).

12. The method of claim 1, wherein an error in said amount of defocus from a true amount of defocus is at most about 100 nanometers (nm).

13. The method of claim 1, wherein an error in said amount of defocus from a true amount of defocus is at most about 50 nanometers (nm).

14. The method of claim 1, wherein a center of said in focus region is determined using an image processing algorithm.

15. The method of claim 14, wherein said image processing algorithm comprises determining said center of said in focus region by separating said image into a predetermined number ofregions and using a sum or average intensity of each region to identify the location of said in focus region.

16. The method of claim 1, wherein image intensity or spatial frequency information of said location of said in focus region is used to locate said center of said in focus region.

17. The method of claim 1, wherein information about a geometrical pattern in said image determines said image processing algorithm.

18. A method of focusing an optical system, comprising:(a) imaging, using a detector, a substrate tilted at a tilt angle, wherein an image of said substrate comprises an in focus portion and an out-of-focus portion;(b) determining, using a processor, a defocus of the optical system based at least in part on said tilt angle and a distance of said in focus portion from a center of said image;(c) adjusting said substrate to remove said tilt angle; and(d) adjusting said substrate by said defocus, thereby focusing said optical system.

19. The method of claim 18, wherein said determining of (b) further comprises using a vector of said in focus portion from said center of said image.

20. The method of claim 18, further comprising a motor coupled to said substrate configured to impart said tilt angle.

21. The method of claim 18, wherein said detector is a portion of an autofocusing element.

22. The method of claim 18, wherein said optical system further comprises an additional detector configured to image said substrate.

23. The method of claim 18, further comprising, prior to (a), tilting said substate to said tilt angle.

24. The method of claim 18, further comprising, subsequent to (d), de-tilting said substrate.

25. The method of claim 23, wherein said tilting is tilting of a plane orthogonal to an optical axis of said optical system.

26. The method of claim 18, wherein said tilt angle is from about 0.01 to about 89 degrees.

27. The method of claim 18, wherein said tilt angle is from about 0.05 to about 15 degrees.

28. The method of claim 18, wherein an angular resolution of said tilt angle is from about 0.001 degrees to about 0.2 degrees.

29. The method of claim 18, wherein an angular resolution of said tilt angle is from about 0.01 degrees to about 0.1 degrees.

30. The method of claim 18, wherein an angular resolution of said tilt angle is from about 0.01 degrees to about 0.08 degrees.

31. The method of claim 18, wherein said determining is performed using said image and no additional images.

32. The method of claim 18, wherein said substrate comprises a flow cell comprising:(a) one or more surfaces;(b) at least one hydrophilic polymer coating layer;(c) a plurality of oligonucleotide molecules attached to said at least one hydrophilic polymer coating layer; and(d) at least one discrete region of said one or more surfaces that comprises a plurality of clonally-amplified nucleic acid molecules immobilized to said plurality of attached oligonucleotide molecules, wherein said plurality of immobilized clonally amplified sample nucleic acid molecules are present at a distance less than X / (2*NA), wherein X is the center wavelength of an excitation energy source and NA is the numerical aperture of said optical system.

33. The method of claim 32, wherein said substrate comprises a beaded flow cell.

34. The method of claim 33, wherein said beaded flow cell comprises a surface comprising fluorescent beads chemically immobilized to said substrate.

35. The method of claim 34, wherein said fluorescent beads are randomly distributed on said surface.

36. The method of claim 34, wherein said fluorescent beads comprise at least about 4 types of beads configured to emit different colors in response to excitation from a laser.

37. The method of claim 18, wherein an error in a distance from said focal plane to a true distance from said focal plane is at most about 400 nanometers (nm).

38. The method of claim 18, wherein an error in said distance from said focal plane to a true distance from said focal plane is at most about 100 nanometers (nm).

39. The method of claim 18, wherein an error in said distance from said focal plane to a true distance from said focal plane is at most about 50 nanometers (nm).

40. The method of claim 32, wherein (d) occurs prior to said optical system imaging a nucleic acid molecule immobilized to said substrate in a first flow cycle.

41. The method of claim 32, further comprising repeating (a) - (d) to refocus said optical system for a second flow cycle.

42. A method of focusing an optical system, comprising:(a) imaging, using a detector tilted at a tilt angle, a substrate, wherein an image of said substrate comprises an in focus portion and an out-of-focus portion;(b) determining, using a processor, a defocus of the optical system based at least in part on said tilt angle and a distance of said in focus portion from a center of said image; and(c) adjusting said substrate by said defocus, thereby focusing said optical system.

43. The method of claim 42, further comprising adjusting said substrate by said defocus, thereby placing said substrate into focus.

44. The method of claim 42, further comprising, prior to (a), tilting said detector to said tilt angle.

45. The method of claim 42, further comprising, subsequent to (c), de-tilting said detector.

46. The method of claim 44, wherein said tilting is tilting of a plane orthogonal to an optical axis of said optical system.

47. The method of claim 42, wherein said tilt angle is from about 0.01 to about 89 degrees.

48. The method of claim 42, wherein said tilt angle is from about 0.05 to about 15 degrees.

49. The method of claim 42, wherein said determining is performed using said image and no additional images.

50. The method of claim 42, wherein an error in said amount of defocus from a true amount of defocus is at most about 400 nanometers (nm).

51. The method of claim 42, wherein an error in said amount of defocus from a true amount of defocus is at most about 100 nanometers (nm).

52. The method of claim 42, wherein an error in said amount of defocus from a true amount of defocus is at most about 50 nanometers (nm).

53. The method of claim 42, further comprising calibrating a pivot point of the optical system.

54. The method of claim 53, wherein said calibrating of said pivot point comprises de-tilting said substrate, said detector, or an autofocus sensor.

Images