Real-time cytopathological analysis methods and systems for cell diagnostics

A multi-view user interface with AI and multiple cameras in a digital microscope addresses latency and usability issues in cytology systems, enabling real-time, efficient, and accurate cytopathological analysis by concurrently processing cellular samples at varying magnification levels.

WO2026236163A1PCT designated stage Publication Date: 2026-11-12ASP HEALTH INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/US2026/026621
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-06-19
Filing Date
2026-05-04
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Current cytology systems face challenges with latency in cytopathological analysis, poor usability for remote pathologists, and inefficient scanning times, leading to delayed diagnoses and misdiagnoses due to inadequate specimen preparation and non-automated systems.

Method used

A multi-view user interface with machine-learning and artificial intelligence techniques, utilizing a digital microscope with multiple cameras and cameras of different magnification powers, enables real-time analysis by concurrently processing cellular samples at varying magnification levels, allowing for rapid identification and detailed examination of cellular structures.

Benefits of technology

Enables real-time, efficient, and accurate cytopathological analysis by reducing latency and improving usability for pathologists, facilitating rapid identification of areas of interest and reducing scanning times, thereby enhancing diagnostic accuracy and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2026026621_12112026_PF_FP_ABST
    Figure US2026026621_12112026_PF_FP_ABST
Patent Text Reader

Abstract

Methods and systems directed to real-time on-site imaging analysis of a cellular sample on a substrate are described. An example method includes generating a digitized macroscopic view using a digital optical microscope at an initial magnification level, and identifying a portion of the view based on a cell identification operation. A first digitized representation of the portion is generated at a higher magnification, and an image processing operation generates non-overlapping tiles that are overlaid on the digitized macroscopic view. A tile is selected prior to completion of the image processing operation, and a second digitized representation of the selected tile is generated at an even higher magnification. The method enables efficient, real-time analysis by progressively focusing on regions of interest. A system for implementing the method includes a digital optical microscope, a user interface, and processors configured to simultaneously implement multiple processing threads for image analysis and display.
Need to check novelty before this filing date? Find Prior Art

Description

International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOREAL-TIME CYTOPATHOLOGICAL ANALYSIS METHODS AND SYSTEMS FOR CELL DIAGNOSTICSCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to U.S. Provisional Application Nos.63 / 800,078 and 63 / 826,713, filed on May 5, 2025 and June 19, 2025, respectively, which are incorporated by reference herein in their entireties.TECHNICAL FIELD

[0002] This document generally relates to laboratory procedures, and more specifically, to real-time imaging analysis for laboratory cytology procedures.BACKGROUND

[0003] Cytology involves examining cells from bodily tissues or fluids to determine a diagnosis. This field has developed minimally invasive techniques that have revolutionized medical practice by allowing quick sample collection with little patient discomfort. As cytology procedures have advanced, imaging systems have become an integral part of the analysis process.

[0004] Modern cytology laboratories typically incorporate digital imaging capabilities into their workflow. These imaging systems allow technicians and pathologists to capture high-resolution images of cellular samples, facilitating detailed examination and documentation. Digital imaging in cytology enables archiving of sample images, remote consultation between experts, and the application of image analysis algorithms to assist in diagnosis.SUMMARY

[0005] Methods and systems for real-time cytopathological analysis of cellular samples collected on a substrate are described. The described methods and systems provide a clinician with a multi-view user interface (Ul) that advantageously provides a real-time technological solution for cytopathological analysis of a cellular sample on a substrate. The multi-view user interface, which is communicatively coupled to a digital microscope with an objective lens and three cameras (with different magnification powers) configured to provide digitized representations at different magnification levels, includes three views and bidirectional switching therebetween. A first view isInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOconfigured to enable an image acquisition operation and a first classification operation, a second view is configured to enable a first analysis operation and a second classification operation, and a third view is configured to enable a second analysis operation. Herein, the first classification operation and second classification operation are implemented using machine-learning (ML) and artificial intelligence (Al) techniques that are selected for their low latency.

[0006] In these methods, the acquisition operation of the first view provides the first classification operation with digitized representations of the substrate, which are processed to provide the clinician with an initial assessment of a portion of the cellular sample. The first classification operation may be implemented using a convolutional neural network (CNN) classifier that acts as a low-complexity gating function and only uses further more complex processing to analyze portions of the cellular substrate that have been identified as suspicious, malignant, etc.

[0007] In these methods, the clinician can then select a sub-portion from the initially assessed portion for the first analysis operation in the second view. In the described embodiments, the acquisition operation and the first classification operation are configured to continue executing concurrently with the first analysis operation by the clinician in the second view. This enables the entire substrate comprising the cellular sample to be imaged and analyzed in real-time, thereby overcoming the cytopathological analysis latency bottleneck in existing implementations.

[0008] In these methods, the clinician (or a remote pathologist with access to a live feed from the digital microscope) can further perform a more detailed second analysis operation in the third view at a higher magnification level than in the first view or the second view. When the user interface switches to the third view for the second analysis operation, the acquisition operation is automatically paused. The acquisition operation is resumed when the third view is exited. Herein, positional information related to the digital microscope during the acquisition operation is saved to enable the acquisition operation to immediately resume.

[0009] In these methods, the user interface includes components and widgets that enable either the clinician or the remote pathologist to review specific portions of substrate that have been identified by the first and / or second classification operations. Alternatively, the user can traverse a trajectory (either in a predetermined manner or dynamically) over the substrate to analyze groups or clusters of cells. Herein, theInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOtrajectory can be saved as metadata. The components and widgets include directional controls, crosshairs that track the user’s field of view and / or current position, coarse focus and fine focus capabilities, autofocus capabilities, and annotation capabilities.

[0010] The described methods and systems enable real-time analysis of a cellular sample on a substrate by implementing, in a first view, an acquisition operation and a first classification operation that operate at a first magnification level that is equal to or lower than a second magnification level at which the first analysis operation is performed in the second view. Herein, the digital optical microscope includes an objective lens that is coupled to three cameras: a macro camera (e.g., operating at 1x magnification power), a first camera (e.g., at 5x magnification power), and a second camera (e.g., at 20x magnification power). In some examples, the operations in both the first view and the second view rely on the first camera of the digital microscope (with a first magnification power) and operations in the third view rely on the first camera and the second camera (with a second magnification power). The digitized images from the first camera are used in both the first and second views, thereby enabling the acquisition operation to continue while the clinician performs the first analysis operation. The second analysis operation in the third view relies on both the first camera and the second camera that operates at a second magnification power that is higher than the first magnification power, thereby enabling the second analysis operation to be performed at a third magnification level higher than the first and second magnification levels.

[0011] In an example aspect, a method for analyzing a cellular sample on a substrate is disclosed. The method includes generating, using a digital optical microscope comprising an objective lens and a plurality of cameras, digitized representations of a surface of the substrate comprising the cellular sample. Herein, the digitized representations comprise a digitized macroscopic view of a surface of the substrate comprising the cellular sample at an initial magnification level. The method then includes identifying, using a user interface, a portion of the digitized macroscopic view based on a cell identification operation, generating, using the digital optical microscope at a first magnification level greater than the initial magnification level, a first digitized representation of the portion, and performing, on the first digitized representation, an image processing operation to generate a plurality of nonoverlapping tiles covering the first digitized representation, wherein each of theInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOplurality of non-overlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion. The method further includes causing the user interface to provide a mosaic view, a tile view, and a live view. The mosaic view comprises overlaying the plurality of non-overlapping tiles on the digitized macroscopic view such that each tile is visually indicated according to its associated cell type. The tile view comprises displaying a second digitized representation of a user-selected tile of the plurality of non-overlapping tiles, where the second digitized representation is generated by the digital optical microscope at a second magnification level greater than or equal to the first magnification level. The tile view further comprises automatically overlaying one or more bounding boxes on one or more clusters of cells within the second digitized representation. The live view comprises a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level. In this method, in response to one or more inputs received at the user interface, bidirectional switching among the mosaic view, the tile view, and the live view is enabled. Furthermore, in response to a first input switching from the mosaic view to the tile view, the second digitized representation is displayed while a generation of the first digitized representation is ongoing, and further, in response to a second input engaging the live view while the image processing operation is ongoing, the image processing operation is automatically paused and automatically resumed in response to a third input exiting the live view to the mosaic view or the tile view.

[0012] In another example aspect, a system for analyzing a cellular sample on a substrate is disclosed. The system includes a digital optical microscope, comprising an objective lens and a plurality of cameras, configured to generate digitized representations of a surface of the substrate comprising the cellular sample at a plurality of magnification levels, a user interface configured to receive one or more inputs and to display, based thereon, the digitized representations, and one or more processors communicatively coupled to the user interface and to the digital optical microscope. In this system, the one or more processors are configured to generate a digitized macroscopic view of the surface of the substrate using the digital optical microscope at an initial magnification level, identify a portion of the digitized macroscopic view based on a cell identification operation, and generate, using theInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOdigital optical microscope at a first magnification level greater than the initial magnification level, a first digitized representation of the portion of the digitized macroscopic view. The one or more processors are further configured to perform, on the first digitized representation, an image processing operation to generate a plurality of non-overlapping tiles covering the first digitized representation, wherein each of the plurality of non-overlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion, and cause the user interface to provide the above-described mosaic view, tile view, and live view.

[0013] In yet another example aspect, an apparatus for analyzing a cellular sample on a substrate is disclosed. The apparatus includes a user interface configured to receive one or more inputs, and display digitized representations of a surface of the substrate comprising the cellular sample. Herein, the digitized representations are generated, by a digital optical microscope at multiple magnification levels. Herein, the digital optical microscope comprises an objective lens, a macro camera, a first camera, and a second camera. The apparatus further includes one or more processors, communicatively coupled to the user interface and the digital optical microscope, configured to cause the user interface to provide a first view, a second view, and a third view. The first view comprises an image acquisition operation configured to produce a first digitized representation of the surface of the substrate at a first magnification level, and a first classification operation configured to process each of a plurality of non-overlapping tiles covering the first digitized representation and identify a cell type therein. The second view comprises a first analysis operation configured to display a second digitized representation of the surface of the substrate at a second magnification level greater than or equal to the first magnification level, and a second classification operation configured to automatically overlay at least one bounding box on one or more clusters of cells within the second digitized representation. The third view comprises a second analysis operation configured to display a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level. In this apparatus, each of the image acquisition operation, the first classification operation, the first analysis operation, and the second classification operation is further configured to process a digitized output associated with the firstInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOcamera, and each of the first classification operation and the second classification operation is further configured to implement an artificial intelligence (Al)-based model. Further, the second analysis operation is configured to process digitized outputs associated with the first camera and the second camera.

[0014] In yet another example aspect, a method for analyzing a cellular sample on a substrate is disclosed. The method includes configuring a user interface, coupled to a digital optical microscope, to operate in each of a first mode, a second mode, and a third mode. The method further includes receiving, while operating in the first mode, the second mode, or the third mode, one or more inputs from the user interface, and displaying, based on the one or more inputs, one or more digitized representations of a surface of the substrate comprising the cellular sample. In this method, the one or more digitized representations are generated, by the digital optical microscope, at multiple magnification levels. The first mode, the second mode, and the third mode are similar to those in the above-described apparatus, as is the configuration of the digital optical microscope.

[0015] In yet another example aspect, an apparatus comprising a memory and a processor that implements the above-described methods is disclosed.

[0016] In yet another example aspect, the above-described methods may be embodied as processor-executable code and may be stored on a non-transitory computer-readable program medium.

[0017] The above and other aspects and features of the disclosed technology are described in greater detail in the drawings, the description and the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIGS. 1 and 2 show examples of systems that functionally integrate a benchtop instrument for sample deposition and staining with artificial intelligence (Al)-based imaging and analysis.

[0019] FIG. 3 shows an example of a system that both functionally and physically integrates Al-based imaging and analysis into the benchtop instrument.

[0020] FIG. 4 shows a block diagram of a cellular sample analysis system.

[0021] FIG. 5 shows an example of an ML-based classification system.

[0022] FIGS. 6A-6C show an example of a stained slide, the slide after being processed by the Specimen Region Identification (SRI) algorithm, and the slide after being processed by the Relevant Tile Identification (RTI) algorithm, respectively.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0023] FIGS. 7A-7C show another example of a stained slide, the slide after being processed by the SRI algorithm, and the slide after being processed by the RTI algorithm, respectively.

[0024] FIG. 8 shows an example of an original image used in training data.

[0025] FIGS. 9A-9K show examples of processed images used in training data.

[0026] FIGS. 10A-10C show screenshots of an example user interface (III).

[0027] FIGS. 11 A-11 E shows screenshots of another example III.

[0028] FIGS. 12A-12D show screenshots of yet another example III.

[0029] FIGS. 13A-13D show screenshots of yet another example III.

[0030] FIGS. 14 and 15 show flowcharts of example methods for processing and visualizing a cellular sample on a substrate, respectively.DETAILED DESCRIPTION

[0031] To make the purposes, technical solutions and advantages of this disclosure more apparent, various embodiments are described in detail below with reference to the drawings. Unless otherwise noted, embodiments and features in embodiments of the present document may be combined with each other. Section headings are used in the present document to improve readability of the description and do not in any way limit the discussion or the embodiments to the respective sections only.

[0032] 1 Introduction

[0033] Biological tissue samples are collected from patients for microscopic and molecular diagnostic analysis for clinical, diagnostic and research applications. These samples are collected in a variety of laboratory, medical clinic and other health-care or medical research settings. For example, cells / tissue can be collected from a patient using a collection device, such as a brush, swab or cutting tool for biopsies.

[0034] Rapid on-site evaluation (ROSE) is a cytopathologic diagnostic adequacy assessment of individual biopsy passes performed during a biopsy procedure in order to optimize the procedure itself and inform subsequent patient management. One of the goals of ROSE is to optimize the quantity of diagnostic material procured during a biopsy procedure (e.g., a cellular sample) including triage in anticipation of any ancillary studies (e.g., flow cytometry, molecular testing, etc.). In an example, ROSE specimens consist of aspirate smears from fine needle aspiration (FNA) biopsies or touch preparation slides from needle core (NO) biopsies.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0035] The subsequent histological or cytological analysis of these cell samples depends greatly on the quality of prepared specimens. Inadequate preparation of specimens can result in inaccurate data that causes an error in interpretation of results and misdiagnosis. In addition, complicated, non-automated systems lead to backlogs, delayed diagnoses, unintended types of artifacts that are due to the complexity of procedures, and intolerable run-to-run variability.

[0036] Current ROSE implementations support remote viewing of stained specimen slides, but there still remain significant drawbacks. For example, in existing systems, pathologists use their keyboard or mouse to control X, Y, and Z actuation stages, as well as to operate other microscope controls. The major challenge with existing remotely-operable microscopes is poor usability for pathologists, who have to physically move their mouse or use multiple keyboard buttons to perform the corresponding movement in the remotely-operable microscope. This procedure requires ultra low-latency communication capabilities and a learning curve that takes a significant amount of time, which has consequently led to poor uptake of remotely-operable microscopes by pathologists.

[0037] Furthermore, remotely-operable microscopes that examine the specimen slide are used by cytologists in non-standardized ways. That is, a pathologist remotely examining the specimen slide in real time is likely following his or her own methods or techniques to determine whether the cellular sample includes cells of a certain size and / or type (e.g., based on past experience or pedagogical methods). The alternative remote-viewing approach is a more systematic procedure for examining the slide, which can be implemented using pathology scanners and ML-based algorithms.

[0038] Digital slide (or pathology) scanners work in conjunction with the remote-operable microscope to allow for the complete scanning of the specimen slide using automated microscope systems. These microscopes quickly digitize the entire specimen slide or a region within a specimen slide. The digitized images are then uploaded to a cloud for it to be accessed by the pathologist / cytopathologist at a later time. Alternatively, these images can also be quickly emailed or shared via other means with the clinicians. Since a remote user only needs access to the digital representation of the specimen slide (or a portion thereof), there is no longer any need for the remote user to engage with the mouse- or keyboard-interfaced actuator controls.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0039] However, in existing systems, the amount of time it takes the digital slide scanner (or alternatively, a digital optical microscope) to scan a single specimen slide is a major drawback. Typically, it takes about 20 minutes to scan an area of 25mm x 50mm with a 40X objective lens. Although it takes less time (about 5-7 minutes) to scan the same slide area with a 4X or a 10X objective lens, these objective lenses do not provide the necessary resolution to obtain the definite diagnosis. Even if the pathologist were to wait for the higher-resolution scan to be available, his or her own techniques to examine the slide would then be employed, which might not necessarily be the optimal approach for reading the specimen slide.

[0040] Thus, there remains a need for a system that enables slide specimens to be remotely examined by a cytologist in real-time. Embodiments of the disclosed technology address these drawbacks by providing an ML-based software system that combines the advantages of the remote microscope and the digital slide scanner while resolving the challenges associated with the current use of each of the component technologies.

[0041] In some examples, the ML-based software can systematically scan the specimen slide at a lower resolution (e.g., 5x-10x), while allowing the (remote) cytologist to examine the specimen slide in real time. The lower resolution scans are analyzed, and smaller portions of the scan are identified if they are determined to include any cells of interest. Once identified, these portions can be scanned at a higher resolution (e.g., 20x-40x), and provided to the cytologist. Thus, a systematic scan of the specimen slide can be performed, which can lead to more rapid identification of any cells of interest; first by the ML-based algorithms, and then by the cytologist.

[0042] FIGS. 1-3 show examples of different embodiments that integrate the ML-based described imaging and analysis systems with benchtop instruments for cellular sample deposition and staining. FIGS. 1 and 2 show examples of systems that functionally integrate a benchtop instrument for sample deposition and staining with ML-based imaging and analysis. In these examples, the slide is physically taken from the sample preparation device and loaded into the microscope.

[0043] In some embodiments, the sample preparation device contains a slide holder (e.g. , as shown in US D862,727) that can be easily taken out from the sample preparation device and inserted into the microscope.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0044] FIG. 3 shows an example of a system that both functionally and physically integrates ML-based imaging and analysis into a benchtop instrument for sample deposition and staining. As shown therein, the microscope and its corresponding optical components are placed around the components of the sample preparation system.

[0045] In some embodiments, the microscope and its optical instruments will be placed below the sample preparation unit of the device (e.g., as shown in FIG. 3). Here, after the sample is deposited onto the glass slide and is stained, the objective unit of the microscope will automatically move closer to the glass slide from below the slide to obtain an image of the relevant portion of the slide. In order to view the glass slide, the movable objective lens will move along the entire length of the slide to obtain the entire image of the slide.

[0046] In some embodiments, the microscope and its optical components are placed above the sample preparation unit of the device. Herein, the objective lens will move closer to the fully prepared glass slide to take the image of the entire glass slide.

[0047] In some embodiments, the imaging modes supported by the microscope shown in FIG. 3 include bright field, dark field, phase contrast, and fluorescent microscopy. In other embodiments, the field of view of the glass slide will be projected to the display unit that is present in the system.

[0048] In some embodiments, the microscope shown in FIG. 3 includes features and aspects of the microscope shown in FIGS. 1 and 2.

[0049] In some embodiments, an example workflow for the benchtop instrument shown in FIG. 3 includes generating a slide on which a cellular sample has been deposited (and optionally, stained). Subsequently, in the imaging mode, the slide will be imaged and projected onto the display that is attached to the instrument. In addition to the on-instrument display, the image will also be projected to a remote computer such that a clinician sitting at a remote site can visually look at the slides in real time. The described embodiments are directed to imaging and analysis techniques and systems (e.g., the system shown in FIG. 4) that can be deployed in the context of both the on-instrument display and the remote viewer, and using a user interface (e.g., the user interfaces shown in FIGS. 10A-10C, 11A-11 E, and 12A-12D).International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0050] 2 Examples of Al-based imaging and analysis

[0051] Herein, a digitized representation of the substrate (or slide) with the deposited and stained sample can be analyzed by the described embodiments for both on-site and remote analysis by the (cyto)pathologist and / or the medical team. In other embodiments, the imaging subsystem (described in the context of FIG. 4 next) is functionally integrated with (but physically separated from) the benchtop instrument, and also supports both on-site and remote analysis.

[0052] FIG. 4 illustrates a block diagram of an Al-based cellular analysis and imaging system 400 for analyzing a cellular sample on a substrate. The system 400 includes a digital optical microscope 402, a user interface 404, and a processor 406, and is configured to provide real-time, intraoperative analysis during Rapid On-Site Evaluation (ROSE) procedures. In some embodiments, the digital optical microscope 402 may be an integrated system (e.g., the Mateo FL Digital Fluorescence Microscope manufactured by Leica Microsystems), whereas in other embodiments, the digital optical microscope may include an optical microscope and a digital scanner that are communicatively coupled to provide the appropriate functionality.

[0053] In some embodiments, the digital optical microscope 402 is configured to provide a digitized representation of a portion of a surface of the substrate comprising the cellular sample. This may include generating digital images at various magnification levels for analysis of the cellular sample. For example, the digital optical microscope 402 includes at least one objective lens and multiple cameras, e.g., a macro camera, a first camera providing the lower resolution (e.g., 5x) images, and a second camera providing the higher-resolution (e.g., 20x) video feed and images.

[0054] As illustrated in FIG. 4, the system 400 includes a user interface 404 coupled to the digital optical microscope 402. This user interface 404 is configured to receive one or more inputs and display the digitized representation. In some cases, the user interface 404 allows operators to control various aspects of the imaging process and view the results in real time during a ROSE procedure.

[0055] The system 400 further includes one or more processors 406, communicatively coupled to the user interface 404 and the digital optical microscope 402. These processors 406 are configured to perform specific operations related to image analysis and system control. In some cases, the processors 406 mayInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOcoordinate the actions of the various components, process image data, and facilitate the display of results on the user interface 404.

[0056] As further depicted in FIG. 4, the processor 406 includes an image processing unit 408 that processes the digitized representations from the digital optical microscope 402, extracting relevant features and providing analytical results. As shown therein, the image processing unit 408 includes a you-only-look-once (YOLO) model 410 and an EfficientNet-based classifier 412, which may use artificial intelligence or machine learning algorithms to assist in the identification and classification of cellular structures.

[0057] By integrating these components as illustrated in FIG. 4, the cellular analysis system 400 may provide a comprehensive solution for real-time cellular sample analysis. The digital optical microscope 402 enables efficient imaging at various magnification levels, while the processing capabilities of the processor 406 may enable rapid interpretation and decision-making by pathologists or other medical professionals through the user interface 404.

[0058] In some embodiments, the imaging analysis process involves obtaining and analyzing representations of the cellular sample at different magnification levels. This process begins with generating a first digitized representation of a portion of the substrate comprising the cellular sample. The first digitized representation is generated by the digital optical microscope at a first magnification level using the first camera (e.g., with magnification power 5x).

[0059] The system then performs an image processing operation on the first digitized representation to generate a plurality of non-overlapping tiles covering the first digitized representation. Prior to completion of the image processing operation, a tile may be selected from the plurality of non-overlapping tiles.

[0060] In some embodiments, the tile may be selected by a user through the user interface, allowing for manual identification of regions of interest. Alternatively, the tile selection may be performed automatically by the Al model based on predefined criteria or learned patterns, such as the presence of specific cellular structures or abnormalities detected during the initial analysis of the first digitized representation.

[0061] Subsequently, the digital optical microscope generates a second digitized representation of the selected tile at a second magnification level. The secondInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOmagnification level is greater than or equal to the first magnification level, allowing for more detailed examination of specific areas of interest within the cellular sample. However, this detailed examination does not interrupt the scanning process since both operations are configured to process the output of the first camera.

[0062] This multi-level imaging approach enables efficient analysis of the entire sample at a lower magnification while providing the capability to examine specific regions at higher magnification as needed. The system utilizes machine learning algorithms, such as the YOLO model and the EfficientNet-based classifier, to assist in identifying and classifying cellular structures within these digitized representations.

[0063] In some embodiments, the macro camera (e.g., operating at 1x) is used to generate the underlying digitized representation in the first view. The first camera (e.g., operating at 5x) is used to generate the first digitized representation, which is processed by the ML algorithms and enables a first level of further analysis. The first camera is also used to generate the second digitized representations used in the first analysis, e.g. to analyze portions that were classified based on the first digitized representations. Then, the second camera is used, in conjunction with or in addition to the first camera, to generate the real-time video feed in the third view (live view). By configuring the first and second views to rely on the same camera output, the system enables the image acquisition operation to continue scanning the substrate while the user examines selected tiles in the second view, because no physical camera change is required. The third view, which engages multiple cameras, requires repositioning the optical path, which is why the image acquisition operation is paused when the third view is entered.

[0064] In the described embodiments, when a magnification level matches the magnification power of a camera, e.g., the clinician or remote pathologist selects a 10x magnification level and the digital optical microscope has a camera with a magnification power of 10x, then the image captured by the digital optical microscope is displayed to the user. Alternatively, if only 5x and 20x cameras are available, then the digitized representations at 5x or 20x (captured by respective cameras) are interpolated or extrapolated to provide a digitized representation at the required magnification level, e.g., 10x. It will be appreciated that any combination of interpolation and / or extrapolation, as well as uniform and non-uniform weightings, can be used to generate a digitized representation at a particular magnification level.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0065] The user interface allows operators to interact with the system, control the imaging process, and view results in real-time. This integration of advanced imaging techniques with Al-powered analysis tools supports rapid and accurate interpretation of cellular samples during ROSE procedures.

[0066] The described embodiments incorporate various features that advantageously enable the rapid on-site analysis of cytology slides:

[0067] - The imaging analysis process utilizes tiled image handling for partial fields-of-view with non-overlapping tiles. These tiles are obtained from live camera feeds or TIFF (Tagged Image File Format) captures. This approach allows for faster processing and analysis of the cellular sample.

[0068] - To facilitate rapid analysis, the system employs low-latency image processing techniques. These techniques are optimized to process each tile in less than one second, enabling quick turnaround times for analysis results.

[0069] -The system incorporates, in some embodiments, an event-driven architecture that supports real-time hooks. For example, the system triggers image capture after a Z-stack is complete or when frame stability is detected. Z-stack image capture is a microscopy technique that acquires multiple images at different focal planes (Z-positions) to create a fully focused two-dimensional (2D) projection of the specimen. In other embodiments, the described image processing techniques are configured to process captured images with a single layer of depth information, e.g., in contrast to existing implementations that use multiple depth layers. Herein, the single layer of depth information refers to a two-dimensional image corresponding to a single focal plane (Z-position) of the digital optical microscope. These architectures helps reduce lag and improve the overall efficiency of the imaging process.

[0070] In some embodiments, the imaging analysis process involves an interactive loop with clinicians. As slides are being scanned, the system provides updated outputs. This real-time feedback allows clinicians to make informed decisions during the examination process.

[0071] By combining these various aspects, the imaging analysis process provides a comprehensive and efficient approach to analyzing cellular samples at different magnification levels. The process allows for rapid identification of areas of interest and detailed examination of specific regions, supporting real-time decisionmaking in clinical settings. As discussed in this patent document, embodiments of theInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOAl (or ML)-based imaging analysis process can be implemented with (a) any slide deposition and staining system that produces a slide with a deposited (and optionally stained) cellular sample, and / or (b) any imaging system that produces a digitized representation of the slide at various magnification levels that can be controlled by the described embodiments.

[0072] 2.1 Examples of Al-based Models

[0073] In some embodiments, the system employs an artificial intelligence (Al)-based model to perform analysis of the first representation and / or second representation of the cellular sample. This Al-based model comprises various machine learning (ML) classification algorithms designed to identify and analyze specific features within the cellular sample images.

[0074] In the described embodiments, one or more machine-learning (ML)-based imaging and analysis algorithms are used to analyze the digitized images generated by the microscopy subsystem. In some examples, as shown in FIG. 5, an ML algorithm module 520 can include a model training engine, a model evaluation engine, and a result processing and distribution engine. As shown in FIG. 5, training / testing data that is generated (e.g., by performing one or more pre-processing algorithms described later) is stored in the training / testing dataset 510. The machine learning algorithm module 520 includes the model training engine (e.g., which is trained using the training / testing dataset 510), the model evaluation engine, and the result processing engine, which processes and formats the output such that it can be utilized and displayed in a user interface 530.

[0075] In some embodiments, an example workflow for the microscopy subsystem and the ML-based imaging and analysis algorithms includes:

[0076] - A specimen slide is placed on the microscope, and its presence is automatically sensed (e.g., using a weight sensor or an optical sensor), which triggers obtaining a macroscopic view of the entire slide image (at an initial magnification level, e.g., at 1x). Examples of the macroscopic view for two deposited and stained slides 610 and 710 are shown in FIGS. 6A and 7A, respectively.

[0077] - An ML-based Specimen Region Identification (SRI) algorithm is executed, which identifies the region within the specimen slide where the specimen is deposited. The SRI algorithm is an implementation of the cell identification operation. Examples of the region (620 and 720) within the specimen slide (610 and 710) beingInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOidentified are shown in FIGS. 6B and 7B, respectively. The coordinates on the slide where the specimen is located are automatically identified, and the image acquisition operation begins. In some embodiments, the SRI algorithm operates with a predefined probability of detection when identifying the region within the specimen slide that contains the deposited specimen.

[0078] As used herein, the cell identification operation refers to any operation that identifies, within one or more digitized views (e.g., the digitized macroscopic view, the first digitized representation, the second digitized representation) one or more regions that contain cells of interest — for example, by distinguishing specimenbearing regions from background regions of the substrate. In some embodiments, the cell identification operation is implemented using the described SRI algorithm. In other embodiments, the cell identification operation may comprise a user-directed selection of a region displayed on the user interface, an automated detection algorithm that identifies regions based on color, contrast, or morphological features, or a combination thereof. In yet other embodiments, the cell identification operation can include the processing techniques described in Section 2.2 and FIGS. 9A-9K. In yet other embodiments, the cell identification operation is based on the EfficientNet-based classifier described later in this section, and which can be configured to operate in conjunction with the YOLO-based model.

[0079] - As the specimen slide is getting scanned, the ML-based Relevant Tile Identification (RTI) algorithm analyzes the scanned region at a finer degree of granularity (referred to as a “tile”). This algorithm analyzes the first digitized representation to identify specific tiles or sections that warrant further examination at higher magnification levels. By focusing on relevant tiles, the system optimizes the analysis process and reduces processing time. In some embodiments, the output of the RTI algorithm is the tile view described herein.

[0080] In some embodiments, the RTI algorithm identifies whether a tile of interest contains any relevant cells (e.g., epithelial cells, lymphocytes, etc.). The RTI algorithm automatically flags any identified tiles and saves the location of these tiles in order for the pathologist or cytopathologist to examine those tiles in real time.Examples of identified tiles (620-1 , 620-2, 620-3, .... 620-N and 720-1, 720-2, 720-3, ..., 720-N) within the specimen slide (610 and 710) being identified are shown in FIGS. 6C and 7C, respectively.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0081] In some embodiments, the tile size used in the RTI algorithm is based on the magnification of the lens used and the field of the first camera. In an example, if the tile size is too small (e.g., due to a small field of view camera (or sensor size of the camera) or a large magnification objective lens), then the microscope will take a longer time to scan the entire specimen slide due to the number of steps it takes to scan a 25 mm x 50 mm area. In another example, if the tile size is too big (e.g., due to a very large field of view camera or a very small magnification objective lens), then the objects within the tile will be too small to be discerned by the RTI algorithm. The described embodiments typically use a tile size between 300pm and 1500pm, with the specific size being selected based on, for example, the size of the cells of interest.

[0082] An example system combines the YOLO-based model with additional classification techniques to enhance analysis capabilities. In some embodiments, the YOLO-based model is integrated with a convolutional neural network (CNN) ora transformer-based classifier. In some examples, the CNN is implemented as an EfficientNet-based classifier. This hybrid detection and classification pipeline leverages the strengths of multiple Al approaches to improve overall accuracy and performance.

[0083] To optimize processing speed and efficiency, the system implements various techniques to reduce the inference complexity of the Al-based model. In some examples, one or more parameters of either the YOLO-based model or the EfficientNet-based classifier are pruned or quantized. Pruning involves removing less important connections or neurons from the model, while quantization reduces the precision of the model’s weights and activations. These techniques help streamline the model’s performance, particularly when deployed on edge devices or in real-time analysis scenarios.

[0084] In some examples, the system includes at least one graphics processing unit (GPU) to implement the Al-based model. GPUs are particularly well-suited for parallel processing tasks involved in Al computations, thereby enabling faster analysis of cellular sample images.

[0085] By leveraging these Al-based analysis techniques, the system provides rapid and accurate identification of cellular structures and regions of interest within the first representation. This analysis guides subsequent examination steps, such asInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOdetermining which areas to analyze at higher magnification levels using the microscope.

[0086] 2.1.1 Implementations for reducing classification latency

[0087] In some implementations, an EfficientNet-based classifier (or EffNet) is used in combination with the YOLO model to provide the real-time capabilities of the disclosed technology. In the described embodiments, EffNet is used as a fast, frontend image triage model rather than as the primary diagnostic or detection model.

[0088] In the described system, EffNet functions as a tile-level gating model. As microscope images are captured during the ROSE workflow, each image tile is first passed through an EfficientNet-based classifier. The purpose of this step is to quickly determine whether the tile contains enough relevant cellular content to justify deeper downstream analysis, i.e., EffNet helps decide: “Should this tile be sent to the more computationally expensive detection / segmentation model, or can it be deprioritized?”

[0089] Structurally, EffNet is a convolutional neural network (CNN) classifier. It takes an image tile, or tile quadrant, as input, processes it through an EfficientNet backbone, and outputs a confidence score. That score represents the likelihood that the tile contains relevant cellular material or tissue patterns worth further review. The model is designed to be lightweight and fast, making it appropriate for real-time use during microscope scanning. In this context, an example workflow includes:

[0090] 1. A microscope image tile is acquired during the procedure.

[0091] 2. The tile is preprocessed and optionally divided into quadrants.

[0092] 3. Each tile or quadrant is passed through the EffNet classifier.

[0093] 4. EffNet outputs a confidence score.

[0094] 5. If the confidence score passes the gating threshold, the tile is sent to the downstream YOLO detection / segmentation model.

[0095] 6. If the tile does not pass the threshold, it is treated as low-priority or nodetection and can still be displayed for review depending on the Ul configuration.

[0096] This is different from using EffNet as a final diagnostic classifier. EffNet is not intended to make the final suspicious versus benign decision by itself. Instead, it serves as a rapid screening layer that improves system efficiency, reduces unnecessary YOLO inference, and helps maintain low-latency performance during live scanning. In the described embodiments, the downstream model performs the more detailed work: identifying suspicious or benign cellular regions, drawing overlays,International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOestimating counts, and generating the final tile-level output shown in the user interface. EffNet therefore acts as a real-time routing mechanism within the broader Al pipeline. This makes the live scanning pipeline faster, more efficient, and more practical for real-time ROSE use. It filters and prioritizes incoming microscope tiles so that the more detailed Al models can focus on the images most likely to contain clinically relevant cellular content.

[0097] 2.2 Examples of Training Data and Processing

[0098] In some embodiments, the training set for the Al-based model comprises, for each type of feature (e.g., atypical cells of a specific size, certain cellular structures, etc.), an original image associated with the cellular sample (e.g., shown in FIG. 8) and a processed image generated by applying one or more processing techniques to the original image (e.g., shown in FIGS. 9A-9K). This approach allows the model to learn from both unaltered and modified versions of the cellular sample images, thereby improving its ability to generalize and perform accurately across various imaging conditions.

[0099] In some embodiments, the system employs a customizable and modular architecture with plug-and-play capabilities for various models. This architecture supports the integration of different model types, such as Inception, Vision Transformer (ViT), oryou-only-look-once (YOLO) models. The modular design allows for experimentation and rapid iteration, enabling researchers and developers to test and implement various Al approaches for cellular sample analysis.

[0100] The processing techniques used to generate the processed images for the training set include a range of image augmentation and enhancement methods. In some examples, and corresponding to FIGS. 9A-9K, these techniques include:

[0101] 1. Random crop: This technique involves randomly selecting a smaller rectangular region from the original image. Random cropping simulates zoomed-in or cropped views, thereby improving the model’s robustness to partial observations of cellular structures.

[0102] 2. Red-Green-Blue (RGB) shift: This method involves randomly shifting the intensities of the red, green, and blue color channels independently. RGB shifting simulates lighting inconsistencies, sensor differences, or staining variability in medical slides, thereby enhancing the model’s ability to generalize across different imaging conditions.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0103] 3. Color jitter: This technique involves randomly altering brightness, contrast, saturation, and hue. Color jittering helps the model tolerate color distortions and variations in image capture conditions, which is particularly important for pathology applications.

[0104] 4. Random scaling of brightness and / or contrast: This method involves randomly adjusting the overall brightness and contrast of the image. Such scaling models camera exposure changes or slide fading, thereby making the model less sensitive to illumination variations or minor scan quality fluctuations.

[0105] 5. Sharpening operation: This technique involves applying an edge enhancement filter to increase detail intensity. Sharpening mimics high-resolution focus or digital enhancement used in clinical review, thereby preparing the model for overprocessed or highly focused inputs.

[0106] 6. Gaussian blur: This method involves applying a Gaussian kernel to blur the entire image. Gaussian blurring simulates defocus, motion blur, or low-quality scans, thereby improving the model’s robustness to imperfect inputs common in fast, intraoperative imaging.

[0107] 7. Gaussian noise: This technique involves injecting Gaussian-distributed noise into the image. Adding Gaussian noise simulates sensor noise, compression artifacts, or interference, thereby enhancing the model’s tolerance to noisy conditions or scan artifacts.

[0108] 8. Affine transform: This method involves applying a composite geometric transform, which includes rotation, scaling, shearing, and translation. Affine transforms model changes in slide orientation, zoom, or scanning angle, thereby training the model to recognize rotated or misaligned structures common in real-world slide scans or manual capture.

[0109] 9. Grayscale conversion: This technique involves converting the RGB image to grayscale while maintaining three channels. Grayscale conversion simulates scanning in grayscale mode or poor color reproduction, thereby encouraging the model to focus on morphology and boundaries rather than color.

[0110] 10. Contrast Limited Adaptive Histogram Equalization (CLAHE): This method involves improving local contrast in tiles across the image while limiting overenhancement. CLAHE enhances subtle features and textures that are poorly visible,International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOthereby boosting the model’s sensitivity to faint or low-contrast features such as small nuclei or cytoplasmic textures.

[0111] In some embodiments, the system incorporates augmented annotation and human-in-the-loop tools to accelerate training data generation. These tools allow for efficient labeling and annotation of cellular sample images, thereby reducing the manual effort required to create large, high-quality training datasets. The human-in-the-loop approach enables experts to provide input and corrections during the annotation process, thereby improving the accuracy and relevance of the training data. As described later, annotations are preserved across digitized representations regardless of the magnification level.

[0112] The system also includes capabilities for processing the second digitized representation. In some examples, this processing involves generating the second digitized representation, and then implementing one or more of the techniques described below. By applying these processing techniques to both the training data and the second representation, the system enhances its ability to accurately analyze and interpret cellular samples across a wide range of imaging conditions and sample preparations. Examples of these processing techniques include:

[0113] - Thresholding method. Can be size-based or intensity-based. A sizebased thresholding algorithm is used to remove debris and other tiny structures from the slide images, and an intensity-based thresholding algorithm is used to remove artifacts from the slide images.

[0114] - Opening and closing methods. These methods are used to identify clusters of cells within the slide images. In general, the cells of interest are present in groups or clusters while the red blood cells are present throughout the slide as individual cells. Thus, the opening and closing techniques are used to pre-process the slides to extract these groups of cells. In an example, the opening and closing methods use one or more of K-means clustering, density-based spatial clustering, a Gaussian mixture model based clustering, a Balance Iterative Reducing and Clustering using Hierarchies (BIRCH) clustering algorithm, an affinity propagation clustering algorithm, a mean-shift clustering algorithm, or an agglomerative hierarchy clustering algorithm.

[0115] As used herein, a "cluster of cells" (or "cell cluster") refers to a spatially proximate grouping of two or more cells that are identified, by the second classificationInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOoperation, as being located within a predefined distance of one another or within a contiguous region of the second digitized representation. In some embodiments, clusters are identified using morphological operations (e.g., opening and closing), clustering algorithms (e.g., density-based spatial clustering of applications with noise (DBSCAN) or K-Means), or the object detection model (e.g., the YOLO-based model), which groups spatially adjacent cell detections. The bounding boxes overlaid on clusters in the tile view delineate the spatial extent of each identified cluster to direct the user's attention thereto.

[0116] - Surface roughness methods. The surface roughness, which can be calculated as the variance of intensities within a group of pixels, is used to identify the types of cells. In general, red blood cells have a smoother cell surface which in turn has a lower surface roughness value compared to epithelial cells. Thus, this metric can be used to identify the cells of interest from a red blood cell background.

[0117] - Random flip method. Randomly flip the orientation of an input image (horizontally and vertically). This is primarily done to force the model to look at the entire input image instead of looking into the same spot in an image every time during the training process.

[0118] - Random rotation method. Randomly rotate the input image to prevent the training algorithm from looking at the same spot in an image.

[0119] - Rescaling method. Rescale input image values to [0,1] or [-1,1] depending on the transfer learning model’s desired input. This is done to rescale the image regardless of the types of imaging sources that are used to obtain the input image.

[0120] - Image sharpening method. This is done by passing a kernel over the image to “sharpen” the borders of cells / objects in the input image. This methodology is primarily used to overcome the fuzziness in an input image due to problems encountered with the imaging system.

[0121] - Resizing method. A single model or multiple parallel models are created to account for training images of different image sizes (256x256x3, 512x512x3, etc.). This is done to extract different details from the training image. The higher resolution model with a small image size will extract finer details like the size and shape of small cells (e.g., lymphocytes) while the lower resolution model with a large size will generalize details more.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0122] - Noise model. Add random noise to force the training model to not learn too quickly from inputs that have similar patterns but are not generalizable to the rest of the dataset. The purpose of this method is similar to the random flip method and the random rotation method to make sure that the model is forced to look at different features within the input image.

[0123] - Contrast normalization method. The global and local contrast normalization is performed to reduce the overall contrast in the input image. While this method is counterintuitive, the purpose of reducing the contrast of the entire training image or a subsection of the training image is so that smaller objects can be better visualized. For example, if there is a very small contrast between the small cells and its background, reducing the overall contrast of the image will help to better visualize the cells in real-time usage of the machine.

[0124] - Color channel method. Most images in the dataset contain only red or blue colors (or both). This is due to the type of staining that is used to stain the cells in the images. Hence, removing the green layer could force the machine to focus on red and blue colors more while also reducing input size for faster classification.

[0125] 2.3 Examples of Cell Classification

[0126] In some embodiments, the system performs classification of the cellular sample based on the analysis of the digitized macroscopic view, the first digitized representation, and / or the second digitized representation. In some embodiments, the output of the analysis comprises a classification of the cellular sample in the subset of the portion of the substrate. In some examples, and as shown in FIG. 10A, this classification includes categories such as benign (green tiles in FIGS. 10A, 11 A, 11 B, 11 E, 12A, or 12C), atypical (red tiles in FIGS. 10A, 11 A, 11B, 11E, 12A, or 12C), or indeterminate (blue tiles in FIGS. 10A, 11 B, 11 E, 12A, or 12C).

[0127] Herein, a benign classification indicates that the analyzed cells appear normal and do not show signs of abnormality or malignancy. An atypical classification suggests that some cellular features deviate from normal appearance, may be suspicious, but do not necessarily indicate malignancy. An indeterminate classification is used when the analysis does not provide sufficient information to confidently categorize the sample as either benign or atypical. As shown in FIGS. 10B, 10C, 11 C, 11 D, 12B and 12D, a benign or atypical classification in the mosaic view can be followed by an examination of that tile in the tile view, in which clusters of cells areInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOautomatically identified using red bounding boxes that enable the user to focus their attention thereto.

[0128] In some examples, the system generates visual explanations to aid in the interpretation of the classification results. These visual explanations utilize Gradient-weighted Class Activation Mapping (Grad-CAM) techniques. Grad-CAM highlights regions of the image that strongly influence the classification decision, thereby providing insights into which cellular features or structures contribute most significantly to the classification outcome.

[0129] The system also generates slide-level cell density heatmaps. These heatmaps provide a visual representation of the distribution and concentration of cells across the analyzed portion of the substrate. Areas with higher cell density are represented by warmer colors, while areas with lower cell density are represented by cooler colors. This visualization helps identify regions of interest or areas that require further examination.

[0130] In some embodiments, the system employs a custom sampling adequacy scoring system to quantify cellularity (e.g., the relative amount of cells within a specific tissue or area) in real time. This scoring system assesses the quantity and quality of cells present in the sample, thereby providing a numerical or categorical measure of sample adequacy. Real-time quantification of cellularity assists in determining whether sufficient cellular material has been collected for accurate diagnosis or further analysis.

[0131] The system performs per-frame cell detection and classification. In this process, individual frames or tiles of the image are analyzed to detect and classify individual cells. This approach allows for detailed characterization of cellular populations within the sample, thereby providing information about cell types, morphological features, or abnormalities at a cellular level.

[0132] The output of the analysis, including classification results, visual explanations, heatmaps, adequacy scores, and cell-level information, is presented through the user interface. In some examples, this information is displayed in real time as the analysis progresses, allowing for immediate feedback and decision-making during procedures such as Rapid On-Site Evaluation (ROSE).

[0133] The system presents the classification results alongside confidence scores or probability distributions, providing an indication of the certainty of theInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOclassification. Visual elements such as color-coding or graphical representations are used to convey this information clearly and intuitively.

[0134] In some embodiments, the system allows for interactive exploration of the analysis results. Users are able to zoom in on specific regions of interest (e.g., as shown in the progression from FIG. 10B to FIG. 10C), toggle between different visualization modes, or access detailed information about individual cells or regions upon request. For example, and as shown in FIG. 12A, the mosaic view illustrated therein includes tiles that have been classified as benign (green tiles), atypical (red tiles), or indeterminate (blue tiles), and any one of which can be selected to be examined in higher detail in the tile view, as shown in FIGS. 12B and 12D.Furthermore, from the tile view, the user can return to the mosaic view (e.g., as shown in FIGS. 12B and 12D) or use a higher magnification option to further examine the cells in that particular tile using the “live view” option (e.g., as shown in FIG. 12D). In other embodiments, the user has the option of canceling the generation of the mosaic view (e.g., as shown in FIG. 12C).

[0135] In some embodiments, and as illustrated in the screenshots shown in FIGS. 11A-11E, the image processing subsystem generates a digitized macroscopic view of the slide (shown in the upper right corner of the screen in FIGS. 11 A-11 E). A portion of the digitized macroscopic view can be selected, and that portion (e.g., the first digitized representation) is now displayed on the left of the user interface, with the generated non-overlapping tiles being overlaid on the digitized macroscopic view. FIG. 11 A shows a set of overlaid tiles, some of which have been classified after a first duration of time. After a second duration of time, additional portions of the first digitized representation and additional tiles have been generated, as shown in FIG.11B.

[0136] From the tiles illustrated in FIG. 11 B, one or more tiles can be selected for further examination. FIG. 11 C shows an example of three classified tiles being selected, and FIG. 11 D shows a user examining one of those tiles at an increased magnification, i.e., compared to the magnification of the digitized macroscopic view and the first digitized representation. Once the one or more selected tiles have been examined, the user can return to the generation of the first digitized representation, which is shown in FIG. 11 E. As illustrated therein, the generation of the first digitizedInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOrepresentation and the classified overlaid tiles has continued (in a background process) during the time the one or more selected tiles were being examined.

[0137] In some embodiments, a Live View Mode enables users to access a realtime video feed directly from the microscope. While in this mode, the microscope’s current position and the user’s field of view (FOV) are visually tracked using crosshairs displayed on the slide overview image. In some examples, users can manually focus the microscope and inspect the live sample at magnifications of up to 40x. Navigation is fully interactive, supporting both stepwise X, Y, Z movement controls and direct navigation by clicking within the overview image to reposition the stage accordingly. In other examples, Live View Mode is accessible at any time. If activated during an ongoing scanning process, the scan will pause automatically and resume once the user exits Live View and returns to either the mosaic or gallery interface.

[0138] The real-time aspects of the disclosed technology are evident in FIGS.11 A-11 E, which illustrate that the on-site cytopathologist has immediate access to the slide via the first digitized representation, which includes the overlaid classification of the tiled representation of the first digitized representation (e.g., FIGS. 11 A and 11 B). The on-site cytopathologist can then immediately choose to more closely examine or investigate certain tiles (e.g., FIGS. 11C and 11 D), and while this analysis is being performed, the generation of the first digitized representation and overlaid tile classification continues such that when the on-site cytopathologist has finished their initial analysis of the selected tiles, a larger portion of the first digitized representation and the overlaid tile classification is now available (e.g., FIG. 11 E).

[0139] The output of the analysis is used to guide further examination or inform clinical decision-making. For example, regions classified as atypical or indeterminate are flagged for more detailed microscopic examination or additional sampling. The real-time nature of the analysis allows for immediate adjustments to sampling or examination procedures based on the ongoing results.

[0140] In some examples, the system provides recommendations or suggestions based on the analysis results. These recommendations include suggestions for additional tests, sampling of specific regions, or consultation with specialists based on the detected cellular characteristics or patterns.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0141] The system also supports the generation of reports summarizing the analysis results. These reports include quantitative data, such as cell counts or density measurements, alongside qualitative assessments and classifications. The reports are customizable to meet specific clinical or research requirements.

[0142] By providing comprehensive, interpretable, and actionable output from the cellular sample analysis, the system supports efficient and accurate decision-making in clinical settings, thereby improving the speed and reliability of diagnostic processes.

[0143] 2.4 Examples of Cell Counting

[0144] In some embodiments, the system automates cell detection and counting using a combination of classical image processing, deep learning, and hybrid techniques. The system is designed to operate across a range of magnifications (e.g., 10x, 20x, 40x) and imaging conditions, with particular emphasis on flexibility, modularity, and real-time capability. The described embodiments for cell counting, which are configured to operate with magnification levels ranging from 5xto 40x, include one or more of preprocessing operations, detection and counting operations, and post-processing and aggregation operations.

[0145] Preprocessing operations. The preprocessing pipeline standardizes input by normalizing staining intensity and color across samples. It is further configured to apply magnification-specific downsampling or enhancement to match model requirements. In some examples, images are cropped into tiles or regions of interest (ROIs), which can be selected manually or Al can be used to focus processing. In other examples, preview scans or detection-based region proposals may be used to reduce computation.

[0146] Detection and counting operations. One or more of the following techniques can be used for the detecting and counting operations.

[0147] 1. Non-AI-based counting techniques

[0148] In some embodiments, thresholding and / or morphological filtering are applied to isolate cells and remove noise. In some examples, thresholding can be used to binarize images for foreground extraction, estimate cellular density in clusters, or filter non-target objects. In other examples, cell counts are estimated by dividing detected region area by average cell size, adjusted for class or magnification. Herein, the different cell size classes can be determined through various methods such as manual calibration, statistical averaging from previous data, real-time adaptiveInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOestimation using a subset of high-confidence segmentations, or calibration per imaging modality and magnification level. In yet other examples, techniques like watershed segmentation, Hough transforms (or Hough circle detection for round cell types), and connected component analysis can be used to separate and count individual cells.

[0149] In some embodiments, clustering algorithms (e.g., DBSCAN, K-Means) can be used to infer cell groupings from spatial or visual features, especially in regions with high cell densities. These techniques group spatial or feature-based representations of pixels or blobs to infer individual cells in dense or overlapping regions, and can be used for direct cell count estimation, seeded segmentation, or noise filtering.

[0150] In some embodiments, graph-based or hierarchical clustering techniques are configured to segment complex cellular structures or aggregates based on region proximity or morphological similarity.

[0151] 2. Al-based detection and segmentation

[0152] In some embodiments, object detection (e.g., YOLO models, RetinaNet, Faster R-CNN) is used to detect cells or regions with bounding boxes.

[0153] In some embodiments, instance segmentation (e.g., Mask R-CNN, StarDist, Cellpose) can be used to return per-cell masks for counting and morphology, which enables direct counting and shape analysis.

[0154] In some embodiments, semantic segmentation followed by postprocessing (e.g., Il-Net), which can be implemented as semantic segmentation for mask generation followed by blob detection, supports broader region-based analysis.

[0155] In some embodiments, density map regression (e.g., CSRNet, MCNN), which can be used for estimating cell counts from learned density distributions, i.e., without performing explicit segmentation.

[0156] In some embodiments, patch-wise count regression (e.g., patch-wise CNNs), which outputs count estimates per fixed-size region without performing explicit segmentation.

[0157] In some embodiments, transformer-based models (e.g., ViT-count, TranslINet), which learn global image context to improve count accuracy in crowded fields.

[0158] 3. Hybrid pipelinesInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0159] In some embodiments, fast region proposals (e.g., YOLO models) to find regions are combined with detailed segmentation (e.g., Il-Net) within those selected areas (or regions).

[0160] In some embodiments, segment anything model (SAM)-based mask generation with manual prompts or Al-derived prompts are used to define regions of interest (ROIs) followed by segmentation.

[0161] In some embodiments, attention-based aggregation is used to refine estimates based on surrounding cellular context.

[0162] Post-processing and aggregation operations. These operations remove duplicate detections using non-maximum suppression (NMS) and / or aggregate regional counts to compute overall cellularity across the specimen.

[0163] 2.5 Examples of System Components and Workflows

[0164] In some embodiments, the system for analyzing cellular samples includes a digital optical microscope and a user interface. The digital optical microscope is configured to generate digital (or digitized) images of cellular samples at various magnification levels. In some embodiments, the digital optical microscope is capable of scanning large portions of a substrate containing the cellular sample, providing an initial overview of the sample for analysis (e.g., the digitized macroscopic view shown in the upper right corner of FIGS. 11 A-11 E).

[0165] The digital optical microscope component of the system is designed to offer more detailed imaging capabilities. In some examples, the digital optical microscope is configured to support multiple imaging modes, including bright field, dark field, phase contrast, and fluorescent microscopy. This versatility allows for comprehensive examination of cellular samples under different lighting and contrast conditions, thereby revealing different aspects of cellular structure and composition.

[0166] In some embodiments, the system includes a Gallery Interface that organizes Al-processed image tiles into collapsible tabs, with each tab representing a distinct classification category (e.g., benign, suspicious, malignant). Once tiles are analyzed by the Al models, they are automatically sorted into their corresponding class-specific galleries (e.g., the left-hand panel in FIGS. 13A and 13C-13D), which remain accessible to the user at any point during or after scanning. Users can select individual tiles from any gallery for expanded inspection in the main display area. In this detailed view, tiles are shown with Al overlays (e.g., bounding boxes,International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOsegmentation masks) and can be digitally magnified up to 20x for fine review. A Live View Mode is also available, allowing the microscope to automatically move to the physical location corresponding to the selected tile. In this mode, users are presented with a real-time video feed from the microscope with up to 40x digital magnification. If the microscope is actively scanning when Live View is engaged, scanning is temporarily paused and will resume once the user exits Live View and returns to the mosaic or gallery view. The Gallery Interface also provides comprehensive detection metrics, displaying total Al-predicted detections by class and estimated cell counts both globally and per tile.

[0167] In some embodiments, the system incorporates a user interface (e.g., as shown in FIGS. 10A-10C and FIGS. 11A-11 E) that allows operators to control various aspects of the imaging and analysis process. In some embodiments, the user interface comprises several specific control elements, each serving a distinct function in the operation of the system.

[0168] One such control element is designed for selecting magnification levels. This element allows users to choose from a set of predefined magnification levels, thereby including both the lower magnification levels used by the digital scanner and the higher magnification levels available through the microscope.

[0169] Another control element of the user interface is dedicated to controlling the movement of the digital scanner. This element allows for precise control of the scanner’s position in three dimensions - the x-axis, y-axis, and z-axis. Such control enables users to navigate across the substrate and adjust focus as needed during the scanning process.

[0170] Yet another control element is designed to autofocus the microscope (e.g., “AF” in FIGS. 10A-10C and 12A-12D, and “Auto Focus” in FIGS. 11A-11E). This element uses the microscope’s camera to capture a series of images at different focal positions (z-axis), analyzes each of the images using a sharpness or contrast algorithm, assigns a “focus score” to each image, and selects the z-axis position with the highest focus score. In some examples, a laser or infrared (IR) beam is projected onto the slide as part of the autofocus operation. In other examples, the autofocus operation can be automatically performed when the type or number of cells being visualized changes. Alternatively, coarse focus (CF) and fine focus (FF) controls can be implemented, as shown in FIGS. 13A-13D.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0171] The user interface also includes a control element for annotating representations of the cellular sample. This annotation tool allows users to mark areas of interest, add notes, or highlight specific features within the digital or optical images generated by the system.

[0172] In some examples, the system employs lightweight user interface integrations to enhance usability and accessibility. These integrations utilize frameworks such as Gradio or Qt-based front-ends. The use of such lightweight III solutions allows for efficient interaction with the system without requiring complex software installations or extensive training.

[0173] The user interface is designed to display both the digital images from the scanner and the optical images from the microscope. In some embodiments, the interface allows for seamless switching between these different image types, thereby enabling users to compare low and high magnification views of the same sample area.

[0174] The control elements and display capabilities of the user interface are integrated to provide a cohesive and intuitive user experience. For example, selecting a higher magnification level using the magnification control automatically triggers the corresponding controls of the digital optical microscope for the selected area.

[0175] In some examples, the user interface also incorporates elements for controlling the various microscopy modes. Users are able to switch between bright field, dark field, phase contrast, and fluorescent microscopy modes through the interface, thereby allowing for comprehensive examination of samples without needing to manually adjust the digital optical microscope settings.

[0176] The annotation capabilities of the user interface extend across the multiple magnifications of the digital optical microscope views. For example, users are able to add annotations at low magnification and have these annotations persist when switching to higher magnification views of the same area. This feature facilitates the tracking and examination of specific regions of interest across different magnification levels.

[0177] By integrating these various components and user interface elements, the system provides a comprehensive and user-friendly platform for cellular sample analysis. The combination of digital scanning, versatile microscopy, and intuitive controls enables efficient and thorough examination of samples in various clinical and research contexts.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0178] 2.6 Examples of Al-based Frameworks and Implementations

[0179] In some embodiments, the system employs advanced artificial intelligence (Al) models for analyzing cellular samples, with a focus on lightweight deployment for edge devices optimized for edge computing scenarios. In some embodiments, these Al models are specifically configured for detecting individual nucleated cells within the cellular sample.

[0180] YOLO-based models offer advantages in terms of speed and efficiency, thereby enabling real-time detection of cellular structures even on devices with limited computational resources. In some examples, the YOLO-based model is combined with additional classification techniques to enhance analysis capabilities.

[0181] The system integrates the YOLO-based model with a convolutional neural network (CNN) or a transformer-based classifier, e.g., an EfficientNet-based classifier. This hybrid approach leverages the strengths of multiple Al architectures to improve overall accuracy and performance. For example, and as discussed above, the EfficientNet-based classifier rapidly identifies regions containing nucleated cells, which enables a more detailed analysis of the detected cells to be performed sooner.

[0182] To optimize these Al models for edge computing scenarios, the system employs various techniques to reduce inference complexity. In some embodiments, one or more parameters of the YOLO-based model or the EfficientNet-based classifier are pruned. Pruning involves selectively removing less important connections or neurons from the model, thereby reducing its computational requirements without significantly impacting performance.

[0183] Another optimization technique that is applied is quantization.Quantization involves reducing the precision of the model’s weights and activations, e.g., from 32-bit floating-point numbers to lower-precision formats such as 8-bit integers. This reduction in precision significantly decreases the model’s memory footprint and computational demands, thereby enabling faster inference on edge devices.

[0184] In some examples, the system utilizes model compression techniques to further optimize the Al models for edge deployment. These techniques include knowledge distillation, where a smaller, more efficient model is trained to mimic the behavior of a larger, more complex model. The resulting compressed model retainsInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOmuch of the performance of the original while requiring fewer computational resources.

[0185] The system also employs hardware-specific optimizations to enhance performance on edge devices. In some embodiments, this involves using specialized libraries or frameworks designed for efficient inference on specific types of hardware, such as mobile GPUs or dedicated Al accelerators.

[0186] To support real-time analysis in edge computing scenarios, the system implements techniques for efficient data handling and processing. In some examples, this includes methods for streaming partial results or implementing incremental processing strategies that allow for continuous updates as new data becomes available.

[0187] The Al-based model is configured to analyze processed representations of cellular samples to detect individual nucleated cells. In some embodiments, this analysis involves multiple stages, including initial detection of candidate regions, segmentation of individual cells, and detailed classification of cellular features.

[0188] The detection of individual nucleated cells relies on a combination of morphological features, intensity patterns, and contextual information. In some examples, the Al model is trained on diverse datasets representing various cell types, staining techniques, and imaging conditions to enhance its generalization capabilities.

[0189] To improve the accuracy and reliability of nucleated cell detection, the system incorporates domain-specific knowledge into the Al model architecture or training process. In some embodiments, this includes encoding biological constraints or prior knowledge about cellular structures into the model design.

[0190] The system also implements techniques for handling variations in cell appearance due to factors such as sample preparation, staining intensity, or imaging artifacts. In some examples, this involves using data augmentation techniques during model training or implementing adaptive normalization methods during inference.

[0191] To support the detection of individual nucleated cells across different magnification levels, the system employs multi-scale processing techniques. In some embodiments, this involves analyzing the cellular sample at multiple resolutions or using feature pyramid networks to capture both fine-grained details and broader contextual information.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0192] The Al-based model is designed to provide interpretable outputs in addition to raw detection results. In some examples, this includes generating attention maps or feature visualizations that highlight the specific cellular characteristics influencing the model’s decisions.

[0193] To facilitate continuous improvement of the Al model’s performance, the system incorporates mechanisms for collecting feedback and updating the model based on real-world usage. In some embodiments, this involves implementing active learning techniques or federated learning approaches that allow the model to adapt to new data while preserving privacy and security.

[0194] By leveraging these advanced Al models and optimization techniques, the system provides efficient and accurate detection of individual nucleated cells in cellular samples, even when deployed on edge devices with limited computational resources. This capability enables real-time analysis and decision support in various clinical and research contexts.

[0195] 2.7 Real-time capabilities of example embodiments

[0196] The disclosed technology advantageously enables the real-time cytopathological analysis of cellular samples. As used herein, “real-time” refers to a system’s ability to process, analyze, and present digitized representations of the cellular sample to the clinician within a timeframe that is compatible with the clinical workflow in which the system is deployed — i.e., fast enough that the clinician does not experience a perceptible delay between an action (e.g., selecting a tile, switching views, or initiating a scan) and the corresponding system response. In the context of Rapid On-Site Evaluation (ROSE) procedures, where a biopsy pass may last only a few minutes and each additional pass carries clinical risk, real-time performance means that the system’s imaging, classification, and display operations collectively proceed at a rate that keeps pace with the clinician’s decision-making. Real-time performance, as described herein, does not require that every operation complete instantaneously; rather, it requires that the end-to-end latency — from image capture to display of an analyzed result — be sufficiently low that the clinician can interactively examine the substrate and receive classification feedback without waiting for a batch process to complete.

[0197] In the described embodiments, the end-to-end latency of the imaging analysis pipeline is managed by decomposing the pipeline into discrete stages, eachInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOof which is independently optimized. These stages include, but are not limited to: (a) image capture, in which the digital optical microscope generates a digitized representation of a tile-sized portion of the substrate at the operative magnification level; (b) inference, in which the Al-based model (e.g., the YOLO-based model and the CNN or transformer-based classifier) processes the tile to detect cells and classify the tile’s cell type; and (d) rendering, in which the classification result is composited onto the mosaic view or tile view displayed on the user interface. By pipelining these stages — such that stage (a) for a subsequent tile begins while stages (b) and (c) for a preceding tile are still executing — the system achieves an effective per-tile throughput that is limited by the slowest individual stage rather than the sum of all stages. In some examples, the inference stage (b) is the throughput-limiting stage, and its latency is managed by the model optimization techniques described herein (e.g., pruning, quantization, and GPU acceleration).

[0198] The described embodiments further achieve real-time performance by exploiting concurrency between the acquisition operation, the classification operations, and the user’s analysis operations. As described herein, the acquisition operation (which captures and tiles the first digitized representation in the first view) and the first classification operation (which classifies each tile) are configured to execute concurrently with the first analysis operation performed by the clinician in the second view. This concurrency is enabled, in part, by configuring the acquisition operation and the first analysis operation to rely on the same objective lens of the digital optical microscope, so that no physical lens change is required when the user switches between the first view and the second view. The processor implements this concurrency using at least a first processing thread and a second processing thread, as described herein. The first processing thread continues the acquisition and classification operations in the background while the second processing thread services the user’s tile-level analysis in the foreground. This concurrent architecture ensures that the first digitized representation and overlaid tile classification continue to grow while the clinician examines selected tiles, thereby eliminating idle time in the scanning pipeline and enabling the entire substrate to be imaged and classified within the duration of a single clinical procedure.

[0199] The selection of tile size, as described herein, directly affects the real-time performance of the system. As noted above, the described embodiments typically useInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOa tile size between 300 m and 1500 pm. A smaller tile size increases the number of tiles required to cover the substrate, which increases the total number of acquisition-and-inference cycles and thereby increases the total scan time. A larger tile size reduces the number of tiles but also reduces the spatial resolution at which the RTI algorithm can distinguish cells of interest from background, potentially degrading classification accuracy. The tile size is therefore selected to balance these competing constraints — minimizing per-tile latency (by keeping each tile small enough to be processed by the Al-based model in less than one second) while also minimizing the total number of scanning steps (by keeping each tile large enough to cover a meaningful area of the substrate in each acquisition cycle). In some embodiments, the tile size is further constrained by the sensor size and field of view of the camera used with the objective lens, as described herein.

[0200] In embodiments that support remote analysis by a pathologist who is not co-located with the digital optical microscope, real-time performance additionally depends on the communication latency between the microscope system and the remote viewer. In some examples, the system transmits the digitized representations and classification overlays to the remote viewer using a low-latency streaming protocol (e.g., WebRTC or a comparable real-time communication protocol) over a network connection. In these examples, the live view (which comprises a real-time video feed from the digital optical microscope) is streamed to the remote viewer at a frame rate and resolution that are dynamically adjusted based on available network bandwidth and round-trip latency, so as to maintain interactive responsiveness.Herein, the user interface for the remote viewer mirrors the user interface of the onsite display, including the bidirectional switching among the mosaic view, the tile view, and the live view, so that the remote pathologist has the same analytical capabilities as the on-site clinician.

[0201] The selection of the Al-based model architecture is driven, in significant part, by latency constraints. As described herein, the first classification operation and the second classification operation are implemented using ML and Al techniques that are selected for their low latency. In the described embodiments, the YOLO-based model is selected for the detection stage because its single-pass architecture processes the entire input tile in a single forward pass through the network, as opposed to multi-stage detectors (e.g., Faster R-CNN) that require a separate regionInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOproposal stage followed by a classification stage. This single-pass architecture reduces inference latency per tile because it eliminates the overhead of generating and filtering region proposals. The CNN or transformer-based classifier that is combined with the YOLO-based model is further optimized for low latency by pruning (removing less important connections or neurons), quantization (reducing weight precision from, e.g., 32-bit floating-point to 8-bit integer), and, in some examples, knowledge distillation (training a smaller model to replicate the behavior of a larger, more accurate model). These optimizations collectively reduce the inference latency of the Al-based model to a level that supports the sub-one-second per-tile processing target described herein, even when deployed on edge devices with limited computational resources such as portable GPU boxes.

[0202] 3 Example embodiments of the disclosed technology

[0203] The described embodiments provide an imaging analysis system that is designed for real-time, on-site application, particularly in clinical settings where rapid diagnosis is crucial. In some embodiments, the system is specifically tailored for use during Rapid On-Site Evaluation (ROSE) procedures, where immediate feedback on specimen adequacy and cellularity is required.

[0204] 3.1 Technical benefits of the described embodiments

[0205] Advantageous aspects and features of the disclosed technology include one or more of the following.

[0206] - The system is optimized for low-latency image processing, enabling quick turnaround times for analysis results. In some examples, the system is capable of processing and analyzing image tiles in less than one second, allowing for near-instantaneous feedback during procedures.

[0207] - The real-time capabilities of the system are particularly beneficial in scenarios such as Endobronchial Ultrasound-Guided Transbronchial Needle Aspiration (EBUS-TBNA) procedures. During these procedures, the system provides immediate feedback on the adequacy of collected samples, thereby reducing the need for repeat biopsies and improving overall procedure efficiency.

[0208] - The system incorporates an interactive loop with clinicians, allowing for continuous updates as slides are being scanned. This real-time feedback mechanism enables intraoperative adequacy assessment, thereby improving decision-making during procedures.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0209] - The cellular sample analyzed by the system comprises a stained cellular sample on a surface of a cytology slide. The system is specifically designed to handle the unique characteristics of cytology slides, which differ structurally and diagnostically from formalin-fixed paraffin-embedded (FFPE) histopathology slides.

[0210] - The real-time analysis capabilities of the system extend to various aspects of sample evaluation. In some examples, the system provides immediate feedback on cel lularity, thereby quantifying the number and distribution of cells within the sample. The system also offers rapid classification of cellular structures, thereby aiding in the identification of abnormal or atypical cells.

[0211] - The on-site nature of the system eliminates the need for timeconsuming sample transport and processing typically associated with traditional pathology workflows. In some embodiments, this significantly reduces the time between sample collection and diagnosis, thereby leading to faster treatment decisions and improved patient outcomes.

[0212] - The system is designed for deployment on edge devices, thereby enabling its use in a wide range of clinical settings, including operating rooms, endoscopy suites, or outpatient clinics. In some examples, the system is optimized to run on portable devices or lightweight GPU boxes, providing flexibility in its deployment and use.

[0213] - The real-time analysis capabilities of the system also support ongoing quality assurance during sample collection. In some embodiments, the system provides immediate feedback on sample quality, thereby allowing clinicians to adjust their collection techniques in real-time to ensure optimal specimen adequacy.

[0214] - The system incorporates customizable thresholds and criteria for sample adequacy, thereby allowing for adaptation to different clinical contexts or specimen types. In some examples, these criteria are adjusted based on the specific requirements of the procedure or the preferences of the clinical team.

[0215] - The real-time nature of the system also facilitates collaborative decision-making in clinical settings. In some embodiments, the system allows for simultaneous viewing of analysis results by multiple team members, thereby enabling rapid consultation and consensus-building during procedures.

[0216] - The on-site deployment of the system reduces reliance on centralized pathology services, thereby improving access to rapid diagnostic capabilities inInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOremote or underserved areas. In some examples, this enables more efficient triage of patients and allocation of healthcare resources.

[0217] - The real-time analysis capabilities of the system extend beyond simple adequacy assessment. In some embodiments, the system provides preliminary classification or risk stratification of samples, thereby guiding immediate clinical decision-making or prioritizing samples for further analysis.

[0218] - The system is designed to integrate with existing clinical workflows and information systems. In some examples, this integration allows for seamless documentation of real-time analysis results, thereby improving the completeness and accuracy of procedural records.

[0219] - The real-time, on-site nature of the system also contributes to resource optimization in clinical settings. By providing immediate feedback on sample adequacy, the system helps reduce the number of unnecessary biopsies or repeat procedures, thereby leading to cost savings and improved patient experience.

[0220] 3.2 Advantages over existing cytopathological analyses

[0221] The described embodiments provide significant advantages and benefits over existing implementations for cytopathological analyses.

[0222] First, the disclosed system is designed around a real-time, microscopenative ROSE workflow rather than a primarily slide-level image analysis workflow. An existing implementation workflow appears to begin with generating or acquiring a slide image, analyzing a low-resolution overview, identifying areas of interest, and then transmitting selected tiles for remote review. In contrast, the described workflow is built around live procedural use, where images are acquired directly from the microscope during the ROSE procedure and analyzed tile-by-tile as scanning occurs. The goal is not only to review selected slide regions, but to assist adequacy assessment and triage during the active clinical encounter.

[0223] Second, the described Al pipeline is structured differently than the existing implementation. Rather than relying on multiple separate CNNs for quality, classification, area-of-interest localization, multilayer analysis, and semantic segmentation, the described workflow uses a staged inference process: an initial tilelevel gating model to decide whether a tile should proceed to deeper analysis, followed by object detection / segmentation models that localize suspicious or benign cellular regions, estimate counts, and package results for immediate III display. ThisInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOis a more operationally focused pipeline intended to reduce latency and avoid unnecessary downstream inference on non-informative tiles.

[0224] Third, the output from the disclosed system is action-oriented for the user interface. The referenced workflow emphasizes classification categories, heatmaps, contextual 3x3 tile review, and remote viewer feedback. The system produces realtime triage outputs such as suspicious, benign, or no-detection / low-priority tile states, along with overlays, confidence information, and cell-count-oriented summaries. This allows the user to quickly identify where attention is needed, rather than simply reviewing sorted tiles after a broader image-processing step.

[0225] Fourth, the disclosed system is designed for local deployment and onpremises inference. The existing implementation workflow includes transmission of selected tiles to a remote viewer and appears to center remote cytopathologist review as a core part of that workflow. The disclosed system can support remote review, but the Al inference itself is intended to run locally on the workstation connected to the microscope. This keeps the workflow independent of cloud connectivity, EHR / LIS / PACS integration, and network latency, which is important for real-time ROSE use in procedure rooms.

[0226] Fifth, the disclosed approach does not depend on the non-image data inputs described in the existing implementation workflow. That workflow incorporates biopsy site characteristics, CT density, ultrasound data, oxygen level, patient demographics, smoking history, and other clinical variables as inputs to the classification process. The described workflow is image-first and microscope-native. It focuses on real-time tile acquisition, cellular detection, classification, counting, and workflow presentation without requiring external patient or imaging metadata to produce useful intra-procedural outputs.

[0227] Finally, the disclosed remote review concept is different. In the existing implementation workflow, a remote cytopathologist review appears to be tightly integrated with the tile selection and feedback process. In the described workflow, remote review is an optional telecytology extension layered onto a local Al-assisted ROSE platform. The primary system provides an Al that assists the onsite workflow immediately, while remote cytopathologist review can be added when needed for collaboration, confirmation, or staffing flexibility.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0228] Different from existing implementations, the described workflow is differentiated by its real-time microscope-native architecture, local inference design, staged gate / detect / count pipeline, procedure-room usability, low-latency tile triage, and optional rather than foundational remote-review layer.

[0229] 3.3 Preferred solutions for the described embodiments

[0230] The following solutions may be preferably implemented by some of the described embodiments.

[0231] S1. A method (e.g., method 1400 in FIG. 14) of analyzing a cellular sample on a substrate, comprising: generating (1410), using a digital optical microscope comprising an objective lens and a plurality of cameras, digitized representations of a surface of the substrate comprising the cellular sample, the digitized representations comprising a digitized macroscopic view of a surface of the substrate comprising the cellular sample at an initial magnification level; identifying (1420), using a user interface, a portion of the digitized macroscopic view based on a cell identification operation; generating (1430), using the digital optical microscope at a first magnification level greater than the initial magnification level, a first digitized representation of the portion; performing (1440), on the first digitized representation, an image processing operation to generate a plurality of non-overlapping tiles covering the first digitized representation, wherein each of the plurality of nonoverlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion; and causing (1450) the user interface to provide: a mosaic view comprising the plurality of non-overlapping tiles overlaid on the digitized macroscopic view, each tile being visually indicated according to its associated cell type; a tile view comprising a second digitized representation of a user-selected tile of the plurality of nonoverlapping tiles, the second digitized representation being generated by the digital optical microscope at a second magnification level greater than or equal to the first magnification level, and including one or more bounding boxes automatically overlaid on one or more clusters of cells within the second digitized representation; and a live view comprising a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level, wherein, in response to one or more inputs received at the user interface, bidirectional switching among the mosaic view, the tile view, and the live view is enabled, wherein, inInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOresponse to a first input switching from the mosaic view to the tile view, the second digitized representation is displayed while a generation of the first digitized representation is ongoing, wherein, in response to a second input engaging the live view while the image processing operation is ongoing, the image processing operation is automatically paused and automatically resumed in response to a third input exiting the live view to the mosaic view or the tile view.

[0232] S2. A system (e.g., cellular sample analysis system 400 in FIG. 4) for analyzing a cellular sample on a substrate, comprising: a digital optical microscope (402), comprising an objective lens and a plurality of cameras, configured to generate digitized representations of a surface of the substrate comprising the cellular sample at a plurality of magnification levels; a user interface (404) configured to receive one or more inputs and to display, based thereon, the digitized representations; and one or more processors (406), communicatively coupled to the user interface and to the digital optical microscope, configured to: generate, using the digital optical microscope at an initial magnification level, a digitized macroscopic view of the surface of the substrate; identify, based on a cell identification operation, a portion of the digitized macroscopic view; generate, using the digital optical microscope at a first magnification level greater than the initial magnification level, a first digitized representation of the portion of the digitized macroscopic view; perform, on the first digitized representation, an image processing operation (e.g., implemented by image processing unit 408) to generate a plurality of non-overlapping tiles covering the first digitized representation, wherein each of the plurality of non-overlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion; and cause the user interface to provide: a mosaic view comprising the plurality of non-overlapping tiles overlaid on the digitized macroscopic view, each tile being visually indicated according to its associated cell type; a tile view comprising a second digitized representation of a user-selected tile of the plurality of non-overlapping tiles, the second digitized representation being generated by the digital optical microscope at a second magnification level greater than or equal to the first magnification level, and including one or more bounding boxes automatically overlaid on one or more clusters of cells within the second digitized representation; and a live view comprising a real-time video feed from the digital optical microscope at a third magnification level greaterInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOthan or equal to the second magnification level, wherein, in response to one or more inputs received at the user interface, bidirectional switching among the mosaic view, the tile view, and the live view is enabled, wherein, in response to a first input switching from the mosaic view to the tile view, the second digitized representation is displayed while a generation of the first digitized representation is ongoing, wherein, in response to a second input engaging the live view while the image processing operation is ongoing, the image processing operation is automatically paused and automatically resumed in response to a third input exiting the live view to the mosaic view or the tile view.

[0233] S3. The system of solution S2, wherein the image processing operation comprises an artificial intelligence (Al)-based model configured to analyze the first digitized representation or the second digitized representation.

[0234] S4. The system of solution S3, wherein the Al-based model comprises a machine-learning (ML) classification algorithm that includes at least one of a specimen region identification (SRI) algorithm ora relevant tile identification (RTI) algorithm.

[0235] S5. The system of solution S3, wherein a training set for the Al-based model comprises an original image associated with the cellular sample and at least one processed image generated by applying a processing technique to the original image.

[0236] S6. The system of solution S5, wherein the processing technique comprises at least one of a random crop, a red-green-blue (RGB) shift, a color jitter, a random scaling of a brightness and / or a contrast, a sharpening operation, a Gaussian blur, Gaussian noise, an affine transform, a grayscale conversion, or a Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm.

[0237] S7. The system of solution S3, wherein the Al-based model comprises a you-only-look-once (YOLO)-based model combined with an EfficientNet-based classifier.

[0238] S8. The system of solution S7, wherein the EfficientNet-based classifier is configured to identify, at a tile level, whether a tile contains relevant cellular material in the cellular sample on the substrate.

[0239] S9. The system of solution S7, wherein at least one parameter of the EfficientNet-based classifier is pruned or quantized, thereby reducing an inference complexity of the Al-based model.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0240] S10. The system of any of solutions S2 to S9, wherein the image processing operation comprises applying at least one of a thresholding operation, an opening and closing operation, a surface roughness operation, a random orientation operation, an image sharpening operation, ora contrast normalization operation.

[0241] S11. The system of any of solutions S2 to S10, wherein each nonoverlapping tile of the plurality of non-overlapping tiles is a rectangular tile or a hexagonal tile.

[0242] S12. The system of any of solutions S2 to S11 , wherein an indication of the cell type comprises a benign indication, an atypical indication, or an indeterminate indication.

[0243] S13. The system of any of solutions S2 to S12, wherein the digital optical microscope is configured to support bright field, dark field, phase contrast, and fluorescent microscopy imaging modes.

[0244] S14. The system of any of solutions S2 to S13, wherein the cellular sample on the substrate comprises a stained cellular sample on a surface of a cytology slide.

[0245] S15. The system of any of solutions S2 to S14, wherein the first digitized representation comprises a single layer of depth information.

[0246] S16. The system of any of solutions S2 to S15, wherein the image processing operation comprises a cell counting operation that includes at least one of: a preprocessing operation configured to normalize a stain intensity or a color of at least one of the digitized representations of the cellular sample; a detection and counting operation configured to determine a number of cells in a corresponding portion of at least one of the digitized representations; or an aggregation operation configured to determine the number of cells in at least one of the digitized representations of the cellular sample.

[0247] S17. The system of solution S16, wherein the cell counting operation further comprises: a post-processing operation, subsequent to the detection and counting operation, configured to update the number of cells by removing any duplicate detections.

[0248] S18. The system of any of solutions S2 to S17, wherein the digital optical microscope comprises an objective lens, a macro camera configured to produce the digitized macroscopic view, a first camera configured to produce the first digitizedInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOrepresentation, and a second camera, wherein the second digitized representation is produced based on the first camera, wherein the live view is produced based on the first camera and the second camera.

[0249] S19. An apparatus (e.g., cellular sample analysis system 400 in FIG. 4) for analyzing a cellular sample on a substrate, comprising: a user interface (404) configured to: receive one or more inputs, and display digitized representations of a surface of the substrate comprising the cellular sample, wherein the digitized representations are generated, by a digital optical microscope (402), at multiple magnification levels, wherein the digital optical microscope comprises an objective lens, a macro camera, a first camera, and a second camera; and one or more processors (406), communicatively coupled to the user interface and the digital optical microscope, configured to cause the user interface to provide: a first view comprising: an image acquisition operation configured to produce a first digitized representation of the surface of the substrate at a first magnification level, and a first classification operation configured to process each of a plurality of non-overlapping tiles covering the first digitized representation and identify a cell type therein; a second view comprising: a first analysis operation configured to display a second digitized representation of the surface of the substrate at a second magnification level greater than or equal to the first magnification level, and a second classification operation configured to automatically overlay at least one bounding box on one or more clusters of cells within the second digitized representation; and a third view comprising: a second analysis operation configured to display a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level, wherein each of the image acquisition operation, the first classification operation, the first analysis operation, and the second classification operation is further configured to process a digitized output associated with the first camera, wherein each of the first classification operation and the second classification operation is further configured to implement an artificial intelligence (Al)-based model, wherein the second analysis operation is further configured to process digitized outputs associated with the first camera and the second camera.

[0250] S20. The apparatus of solution S19, wherein the plurality of nonoverlapping tiles is overlaid on a digitized macroscopic view of the surface of theInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOsubstrate, the digitized macroscopic view being generated by the macro camera at an initial magnification level less than the first magnification level.

[0251] S21. The apparatus of solution S19, wherein the first analysis operation is configured to enable annotations on the second digitized representation.

[0252] S22. The apparatus of solution S21 , wherein the annotations are preserved across each of the multiple magnification levels.

[0253] S23. The apparatus of any of solutions S19 to S22, wherein the image acquisition operation is configured to execute concurrently with one or more of the first analysis operation, the first classification operation, or the second classification operation.

[0254] S24. The apparatus of any of solutions S19 to S23, wherein, in response to an input engaging the third view while the image acquisition operation is ongoing, the image acquisition operation is automatically paused and automatically resumed in response to another input exiting the third view to the first view or the second view.

[0255] S25. A method (e.g., method 1500 in FIG. 15) of analyzing a cellular sample on a substrate, comprising: configuring (1510) a user interface, coupled to a digital optical microscope, to operate in each of a first mode (e.g., the mosaic view), a second mode (e.g., the tile view), and a third mode (e.g., the live view); receiving (1520), while operating in the first mode, the second mode, or the third mode, one or more inputs from the user interface; and displaying (1530), based on the one or more inputs, one or more digitized representations of a surface of the substrate comprising the cellular sample, wherein the one or more digitized representations are generated, by the digital optical microscope, at multiple magnification levels, wherein the digital optical microscope comprises an objective lens, a macro camera, a first camera, and a second camera, wherein the first mode implements (a) an image acquisition operation configured to produce a first digitized representation of the surface of the substrate at a first magnification level, and (b) a first classification operation configured to process each of a plurality of non-overlapping tiles covering the first digitized representation and identify a cell type therein, wherein the second mode implements (a) a first analysis operation configured to display a second digitized representation of the surface of the substrate at a second magnification level greater than or equal to the first magnification level, and (b) a second classification operation configured to automatically overlay at least one bounding box on one or more clustersInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOof cells within the second digitized representation, wherein the third mode implements a second analysis operation configured to display a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level.

[0256] S26. The method of solution S25, wherein each of the first classification operation and the second classification operation is further configured to implement an artificial intelligence (Al)-based model.

[0257] S27. The method of solution S25 or S26, wherein each of the image acquisition operation, the first classification operation, the first analysis operation, or the second classification operation is further configured to process digitized outputs associated with the first camera.

[0258] S28. The method of any of solutions S25 to S27, wherein the second analysis operation is further configured to process digitized outputs associated with the first camera and the second camera.

[0259] S29. The method of any of solutions S25 or S28, wherein the image acquisition operation is configured to execute concurrently with one or more of the first analysis operation, the first classification operation, or the second classification operation.

[0260] S30. The method of any of solutions S25 or S29, wherein, in response to an input engaging the third mode while the image acquisition operation is ongoing, the image acquisition operation is automatically paused and automatically resumed in response to another input exiting the third mode to the first mode or the second mode.

[0261] The following additional solutions may be preferably implemented by some of the described embodiments.

[0262] P1. A method of processing a cellular sample on a substrate, comprising: generating, using a digital optical microscope at an initial magnification level, a digitized macroscopic view of a surface of the substrate comprising the cellular sample; identifying, based on a cell identification operation, a portion of the digitized macroscopic view of the surface of the substrate; generating, using the digital optical microscope at a first magnification level greater than the initial magnification level, a first digitized representation of the portion of the digitized macroscopic view; performing an image processing operation to generate a plurality of non-overlapping tiles covering the first digitized representation, wherein each of the plurality of non-International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOoverlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion of the first digitized representation; selecting, prior to a completion of the image processing operation, a tile from the plurality of non-overlapping tiles; and generating, using the digital optical microscope at a second magnification level greater than the first magnification level, a second digitized representation of the tile.

[0263] P2. The method of solution P1 , wherein the image processing operation uses an artificial intelligence (Al)-based model.

[0264] P3. The method of solution P2, wherein the Al-based model comprises a machine-learning (ML) classification algorithm that includes at least one of a specimen region identification (SRI) algorithm ora relevant tile identification (RTI) algorithm.

[0265] P4. The method of solution P2 or P3, wherein a training set for the Al-based model comprises an original image associated with the cellular sample and at least one processed image generated by applying a processing technique to the original image.

[0266] P5. The method of solution P4, wherein the processing technique comprises at least one of a random crop, a red-green-blue (RGB) shift, a color jitter, a random scaling of a brightness and / or a contrast, a sharpening operation, a Gaussian blur, Gaussian noise, an affine transform, a grayscale conversion, or a Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm.

[0267] P6. The method of solution P1 , wherein the cell identification operation or the image processing operation comprises one or more user-defined operations.

[0268] P7. A system for analyzing a cellular sample on a substrate, comprising: a digital optical microscope configured to provide a digitized representation of a portion of a surface of the substrate comprising the cellular sample; a user interface configured to receive one or more inputs and display the digitized representation; and one or more processors, communicatively coupled to the user interface and the digital optical microscope, configured to: simultaneously implement at least a first processing thread and a second processing thread; and display the digitized representation associated with the first processing thread on the user interface, wherein the one or more processors is further configured to: generate, using the first processing thread, a first digitized representation of the portion of the surface of the substrate at a first magnification level, wherein the user interface is configured to display the firstInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOdigitized representation; perform, using the first processing thread, an image processing operation on the first digitized representation to generate a plurality of nonoverlapping tiles covering the first digitized representation, wherein each tile of the plurality of non-overlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion of the first digitized representation, and wherein the user interface is configured to overlay the plurality of non-overlapping tiles on a digitized macroscopic view of the substrate; and perform, in response to an input identifying a tile from the plurality of non-overlapping tiles, operations that include: performing, using the second processing thread, the image processing operation on the first digitized representation; and generating, using the first processing thread, a second digitized representation of the tile at a second magnification level higher than the first magnification level, wherein the user interface is configured to display the second digitized representation, wherein the user interface is configured to receive the input prior to a completion of the image processing operation.

[0269] P8. The system of solution P7, wherein the one or more processors is further configured to: generate a digitized macroscopic view of the surface of the substrate comprising the cellular sample; and select, based on performing a cell identification operation on the digitized macroscopic view, the portion of the surface of the substrate.

[0270] P9. The system of solution P7 or P8, further comprising: at least one graphics processing unit (GPU) that implements an artificial intelligence (Al)-based model configured to analyze the first digitized representation or the second digitized representation.

[0271] P10. The system of solution P9, wherein the Al-based model comprises a you-only-look-once (YOLO)-based model configured to detect individual nucleated cells in the cellular sample on the substrate.

[0272] P11. The system of solution P10, wherein the YOLO-based model is combined with a convolutional neural network (CNN) or a transformer-based classifier.

[0273] P12. The system of solution P10, wherein at least one parameter of the YOLO-based model is pruned or quantized to reduce an inference complexity of the Al-based model.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO

[0274] P13. The method or system of any of the preceding solutions, wherein: the image processing operation comprises applying at least one of a thresholding operation, an opening and closing operation, a surface roughness operation, a random orientation operation, an image sharpening operation, or a contrast normalization operation; or an indication of the cell type comprises a benign indication, an atypical indication, or an indeterminate indication; or each non-overlapping tile of the plurality of non-overlapping tiles is a rectangular tile or a hexagonal tile; or the digital optical microscope is configured to support bright field, dark field, phase contrast, and fluorescent microscopy imaging modes; or the cellular sample on the substrate comprises a stained cellular sample on a surface of a cytology slide; or the user interface comprises: a first control element configured to select one of a set of magnification levels that includes the first magnification level and the second magnification level, a second control element configured to control a movement of the digital optical microscope in an x-axis direction, a y-axis direction, and a z-axis direction, a third control element configured to annotate the first digitized representation or the second digitized representation, or a fourth control element configured to perform an autofocus operation.

[0275] P14. The method or system of any of the preceding solutions, wherein: a mosaic view of the cellular sample on the substrate comprises overlaying the plurality of non-overlapping tiles onto a digitized macroscopic view of the substrate; a tile view of the cellular sample on the substrate comprises the second digitized representation of the tile at the second magnification level; and a live view of the cellular sample on the substrate comprises a third digitized representation of the tile at third magnification level greater than the second magnification level.

[0276] P15. The method or system of solution P14, wherein the digital optical microscope is configured to switch from (a) the mosaic view to the tile view or the live view, or (b) the tile view to the mosaic view or the live view.

[0277] P16. The method or system of solution P14, wherein the image processing algorithm comprises performing a cell counting operation at the first magnification level, the second magnification level, or the third magnification level.

[0278] P17. The method or system of any of the preceding solutions, wherein the image processing algorithm comprises performing a cell counting operation that includes at least one of: a preprocessing operation configured to normalize a stainInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOintensity or a color of the digitized macroscopic view, the first digitized representation, the second digitized representation, or the third digitized representation; a detection and counting operation configured to determine a number of cells in the corresponding portion of the first digitized representation; or an aggregation operation configured to determine the number of cells in the digitized macroscopic view or the first digitized representation.

[0279] P18. The method or system of solution P17, wherein the cell counting operation further comprises: a post-processing operation, subsequent to the detection and counting operation, configured to update the number of cells by removing any duplicate detections.

[0280] 4 Concluding Remarks

[0281] In this document, the term “cellular sample” refers to any biological sample containing cells. Cellular samples can be a tissue sample or samples (e.g., any collection of cells) removed from a subject. The tissue sample can be a collection of interconnected cells that perform a similar function within an organism. A cellular sample can also be any solid or fluid sample obtained from, excreted by, or secreted by any living organism, including, without limitation, single-celled organisms, such as bacteria, yeast, protozoans, and amebae, multicellular organisms (such as plants or animals, including samples from a healthy or apparently healthy human subject or a human patient affected by a condition or disease to be diagnosed or investigated, such as cancer). In some embodiments, a cellular sample is mountable on a microscope slide and includes, without limitation, a section of tissue, an organ, a tumor section, a smear, a frozen section, a cytology prep, or cell lines. An incisional biopsy, a core biopsy, an excisional biopsy, a needle aspiration biopsy (e.g., fine-needle aspiration (FNA)), a core needle biopsy, a stereotactic biopsy, an open biopsy, or a surgical biopsy can be used to obtain the sample.

[0282] Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangibleInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOand non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, ora combination of one or more of them. The term “data processing unit” or “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including byway of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0283] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0284] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0285] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOThe essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0286] While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0287] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0288] Only a few implementations and examples are described, and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOWHAT IS CLAIMED IS:

1. A method of analyzing a cellular sample on a substrate, comprising:generating, using a digital optical microscope comprising an objective lens and a plurality of cameras, digitized representations of a surface of the substrate comprising the cellular sample, the digitized representations comprising a digitized macroscopic view of a surface of the substrate comprising the cellular sample at an initial magnification level;identifying, using a user interface, a portion of the digitized macroscopic view based on a cell identification operation;generating, using the digital optical microscope at a first magnification level greater than the initial magnification level, a first digitized representation of the portion;performing, on the first digitized representation, an image processing operation to generate a plurality of non-overlapping tiles covering the first digitized representation, wherein each of the plurality of non-overlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion; andcausing the user interface to provide:a mosaic view comprising the plurality of non-overlapping tiles overlaid on the digitized macroscopic view, each tile being visually indicated according to its associated cell type;a tile view comprising a second digitized representation of a user- selected tile of the plurality of non-overlapping tiles, the second digitized representation being generated by the digital optical microscope at a second magnification level greater than or equal to the first magnification level, and including one or more bounding boxes automatically overlaid on one or more clusters of cells within the second digitized representation; anda live view comprising a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level,International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOwherein, in response to one or more inputs received at the user interface, bidirectional switching among the mosaic view, the tile view, and the live view is enabled,wherein, in response to a first input switching from the mosaic view to the tile view, the second digitized representation is displayed while a generation of the first digitized representation is ongoing, wherein, in response to a second input engaging the live view while the image processing operation is ongoing, the image processing operation is automatically paused and automatically resumed in response to a third input exiting the live view to the mosaic view or the tile view.

2. A system for analyzing a cellular sample on a substrate, comprising:a digital optical microscope, comprising an objective lens and a plurality of cameras, configured to generate digitized representations of a surface of the substrate comprising the cellular sample at a plurality of magnification levels;a user interface configured to receive one or more inputs and to display, based thereon, the digitized representations; andone or more processors, communicatively coupled to the user interface and to the digital optical microscope, configured to:generate, using the digital optical microscope at an initial magnification level, a digitized macroscopic view of the surface of the substrate; identify, based on a cell identification operation, a portion of the digitized macroscopic view;generate, using the digital optical microscope at a first magnification level greater than the initial magnification level, a first digitized representation of the portion of the digitized macroscopic view; perform, on the first digitized representation, an image processing operation to generate a plurality of non-overlapping tiles covering the first digitized representation, wherein each of the plurality of non-overlapping tiles is associated with (a) a corresponding portion of the first digitized representation and (b) a cell type of the cellular sample in the corresponding portion; andInternational Patent ApplicationAttorney Docket No.: 119157.8018.WOOOcause the user interface to provide:a mosaic view comprising the plurality of non-overlapping tiles overlaid on the digitized macroscopic view, each tile being visually indicated according to its associated cell type; a tile view comprising a second digitized representation of a user- selected tile of the plurality of non-overlapping tiles, the second digitized representation being generated by the digital optical microscope at a second magnification level greater than or equal to the first magnification level, and including one or more bounding boxes automatically overlaid on one or more clusters of cells within the second digitized representation; anda live view comprising a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level, wherein, in response to one or more inputs received at the user interface, bidirectional switching among the mosaic view, the tile view, and the live view is enabled,wherein, in response to a first input switching from the mosaic view to the tile view, the second digitized representation is displayed while a generation of the first digitized representation is ongoing, wherein, in response to a second input engaging the live view while the image processing operation is ongoing, the image processing operation is automatically paused and automatically resumed in response to a third input exiting the live view to the mosaic view or the tile view.

3. The system of claim 2, wherein the image processing operation comprises an artificial intelligence (Al)-based model configured to analyze the first digitized representation or the second digitized representation.

4. The system of claim 3, wherein the Al-based model comprises a machinelearning (ML) classification algorithm that includes at least one of a specimen region identification (SRI) algorithm ora relevant tile identification (RTI) algorithm.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO5. The system of claim 3, wherein a training set for the Al-based model comprises an original image associated with the cellular sample and at least one processed image generated by applying a processing technique to the original image.

6. The system of claim 5, wherein the processing technique comprises at least one of a random crop, a red-green-blue (RGB) shift, a color jitter, a random scaling of a brightness and / or a contrast, a sharpening operation, a Gaussian blur, Gaussian noise, an affine transform, a grayscale conversion, or a Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm.

7. The system of claim 3, wherein the Al-based model comprises a you-only-look-once (YOLO)-based model combined with an EfficientNet-based classifier.

8. The system of claim 7, wherein the EfficientNet-based classifier is configured to identify, at a tile level, whether a tile contains relevant cellular material in the cellular sample on the substrate.

9. The system of claim 7, wherein at least one parameter of the EfficientNet-based classifier is pruned or quantized, thereby reducing an inference complexity of the Al-based model.

10. The system of any of claims 2 to 9, wherein the image processing operation comprises applying at least one of a thresholding operation, an opening and closing operation, a surface roughness operation, a random orientation operation, an image sharpening operation, or a contrast normalization operation.

11. The system of any of claims 2 to 9, wherein each non-overlapping tile of the plurality of non-overlapping tiles is a rectangular tile or a hexagonal tile.

12. The system of any of claims 2 to 9, wherein an indication of the cell type comprises a benign indication, an atypical indication, or an indeterminate indication.

13. The system of any of claims 2 to 9, wherein the digital optical microscope is configured to support bright field, dark field, phase contrast, and fluorescent microscopy imaging modes.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO14. The system of any of claims 2 to 9, wherein the cellular sample on the substrate comprises a stained cellular sample on a surface of a cytology slide.

15. The system of any of claims 2 to 9, wherein the first digitized representation comprises a single layer of depth information.

16. The system of any of claims 2 to 9, wherein the image processing operation comprises a cell counting operation that includes at least one of:a preprocessing operation configured to normalize a stain intensity or a color of at least one of the digitized representations of the cellular sample;a detection and counting operation configured to determine a number of cells in a corresponding portion of at least one of the digitized representations; oran aggregation operation configured to determine the number of cells in at least one of the digitized representations of the cellular sample.

17. The system of claim 16, wherein the cell counting operation further comprises:a post-processing operation, subsequent to the detection and counting operation, configured to update the number of cells by removing any duplicate detections.

18. The system of any of claims 2 to 9, wherein the digital optical microscope comprises an objective lens, a macro camera configured to produce the digitized macroscopic view, a first camera configured to produce the first digitized representation, and a second camera, wherein the second digitized representation is produced based on the first camera, wherein the live view is produced based on the first camera and the second camera.

19. An apparatus for analyzing a cellular sample on a substrate, comprising:a user interface configured to:receive one or more inputs, anddisplay digitized representations of a surface of the substrate comprising the cellular sample,wherein the digitized representations are generated, by a digital optical microscope, at multiple magnification levels,International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOwherein the digital optical microscope comprises an objective lens, a macro camera, a first camera, and a second camera; and one or more processors, communicatively coupled to the user interface and the digital optical microscope, configured to cause the user interface to provide:a first view comprising:an image acquisition operation configured to produce a first digitized representation of the surface of the substrate at a first magnification level, anda first classification operation configured to process each of a plurality of non-overlapping tiles covering the first digitized representation and identify a cell type therein;a second view comprising:a first analysis operation configured to display a second digitized representation of the surface of the substrate at a second magnification level greater than or equal to the first magnification level, anda second classification operation configured to automatically overlay at least one bounding box on one or more clusters of cells within the second digitized representation; and a third view comprising:a second analysis operation configured to display a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level,wherein each of the image acquisition operation, the first classification operation, the first analysis operation, and the second classification operation is further configured to process a digitized output associated with the first camera,wherein each of the first classification operation and the second classification operation is further configured to implement an artificial intelligence (Al)-based model,International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOwherein the second analysis operation is further configured to process digitized outputs associated with the first camera and the second camera.

20. The apparatus of claim 19, wherein the plurality of non-overlapping tiles is overlaid on a digitized macroscopic view of the surface of the substrate, the digitized macroscopic view being generated by the macro camera at an initial magnification level less than the first magnification level.

21. The apparatus of claim 19, wherein the first analysis operation is configured to enable annotations on the second digitized representation.

22. The apparatus of claim 21 , wherein the annotations are preserved across each of the multiple magnification levels.

23. The apparatus of any of claims 19 to 22, wherein the image acquisition operation is configured to execute concurrently with one or more of the first analysis operation, the first classification operation, or the second classification operation.

24. The apparatus of any of claims 19 to 22, wherein, in response to an input engaging the third view while the image acquisition operation is ongoing, the image acquisition operation is automatically paused and automatically resumed in response to another input exiting the third view to the first view or the second view.

25. A method of analyzing a cellular sample on a substrate, comprising:configuring a user interface, coupled to a digital optical microscope, to operate in each of a first mode, a second mode, and a third mode;receiving, while operating in the first mode, the second mode, or the third mode, one or more inputs from the user interface; anddisplaying, based on the one or more inputs, one or more digitized representations of a surface of the substrate comprising the cellular sample, wherein the one or more digitized representations are generated, by the digital optical microscope, at multiple magnification levels,wherein the digital optical microscope comprises an objective lens, a macro camera, a first camera, and a second camera,International Patent ApplicationAttorney Docket No.: 119157.8018.WOOOwherein the first mode implements (a) an image acquisition operation configured to produce a first digitized representation of the surface of the substrate at a first magnification level, and (b) a first classification operation configured to process each of a plurality of non-overlapping tiles covering the first digitized representation and identify a cell type therein,wherein the second mode implements (a) a first analysis operation configured to display a second digitized representation of the surface of the substrate at a second magnification level greater than or equal to the first magnification level, and (b) a second classification operation configured to automatically overlay at least one bounding box on one or more clusters of cells within the second digitized representation, wherein the third mode implements a second analysis operation configured to display a real-time video feed from the digital optical microscope at a third magnification level greater than or equal to the second magnification level.

26. The method of claim 25, wherein each of the first classification operation and the second classification operation is further configured to implement an artificial intelligence (Al)-based model.

27. The method of claim 25 or 26, wherein each of the image acquisition operation, the first classification operation, the first analysis operation, or the second classification operation is further configured to process digitized outputs associated with the first camera.

28. The method of claim 25 or 26, wherein the second analysis operation is further configured to process digitized outputs associated with the first camera and the second camera.

29. The method of claim 25 or 26, wherein the image acquisition operation is configured to execute concurrently with one or more of the first analysis operation, the first classification operation, or the second classification operation.International Patent ApplicationAttorney Docket No.: 119157.8018.WOOO30. The method of claim 25 or 26, wherein, in response to an input engaging the third mode while the image acquisition operation is ongoing, the image acquisition operation is automatically paused and automatically resumed in response to another input exiting the third mode to the first mode or the second mode.