Systems and methods for brightfield cell detection

Abstract

Methods and systems are provided herein for automatic object detection. In an example, a system includes memory storing instructions and one or more processors configured to execute the instructions to: obtain a brightfield field of view (FOV) image of a sample including a plurality of cells; obtain a reference image of a cellular structure; and identify individual cells of the plurality of cells by performing template matching on the FOV image using the reference image.

Description

Docket No. ARL24303PCTSYSTEMS AND METHODS FOR BRIGHTFIELD CELL DETECTIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims priority to U.S. Provisional Patent Application No.63 / 746,838, filed January' 17, 2025. and entitled “SYSTEMS AND METHODS FOR BRIGHTFIELD CELL DETECTION.” The entire contents of the above-identified application is hereby incorporated by reference for all purposes.TECHNICAL FIELD

[0002] Embodiments of the subject matter disclosed herein relate generally to cell identification and segmentation, and more specifically to cell identification in brightfield images.BACKGROUND AND SUMMARY

[0003] Image analysis in high content screening relies on accurate detection of cells, and generally the nucleus is the primary target of initial detection. In practice, cell detection is usually accomplished using fluorescent stains that target the cell nucleus. However, staining the nucleus with fluorescent dyes can be problematic for live cell imaging due to potential cytotoxicity and photobleaching effects. Traditionally, this has been addressed by using dyes such as Hoechst or DAPI, which bind to DNA and fluoresce under UV light.

[0004] However, these methods have limitations, including the need for UV illumination, which can damage live cells, and the potential for non-specific binding, leading to background fluorescence. Also, capturing the fluorescent signal means using a fluorescent channel for nuclei that might otherwise be useful for detecting and measuring other cell constituents and / or activity.

[0005] Methods and systems for automatically detecting objects (e.g., cells) are provided herein that address the above issues. In one example, a system includes memory' storing instructions and one or more processors configured to execute the instructions to: obtain a brightfield field of view (FOV) image of a sample including a plurality of cells; obtain a reference image of a cellular structure; and identify individual cells of the plurality of cells by performing template matching on the FOV image using the reference image.

[0006] It should be understood that the brief description above is provided to introduce in simplified form a selection of concepts that are further described in the detailed description. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is defined uniquely by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure.Docket No. ARL24303PCTBRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 is a diagram of a computing device;

[0008] FIG. 2 is an example brightfield field of view (FOV) image of a sample including a plurality of cells;

[0009] FIG. 3 is an example reference image of a cellular structure;

[0010] FIG. 4 is a magnified portion of the FOV image of FIG. 2 with bounding boxes identifying cells overlaid on the FOV image;

[0011] FIG. 5 is a flowchart illustrating an example method for detecting cells in a brightfield FOV image; and

[0012] FIG. 6 is a flowchart illustrating an example method for performing template matching using a brightfield FOV image and a reference image to detect cells in the brightfield FOV image.DETAILED DESCRIPTION

[0013] The present description relates to identifying cells in brightfield images using a direct “template match” method that takes advantage of image Fourier transform-based correlation coefficient detection for extremely fast detection. Signals in brightfield images of cells have reliable proxies for nuclei that can be rapidly and robustly detected with extremely light processing compared to the smallest neural net. The resulting benefits include time and cost savings from not needing a nuclear stain, then instance brightfield processing, requiring far less sophistication, computational cost, and time.

[0014] Brightfield microscopy includes illumination of a sample (e g., cells on a slide or plate) with broad spectrum light (e g., white light) and detection of the light via a detector / camera to create brightfield images. The sample appears dark against a bright background. Brightfield microscopy has many advantages, including simplicity, low cost, and wide availability. However, brightfield microscopy is limited in that it is difficult to image samples that have low contrast with the backgroimd. For imaging samples such as cells, imaging is typically improved with cell stains, such as fluorescent stains, that bind specific cellular structure such as DNA and allow visualization of the cell nuclei, for example. However, some cell stains are cytotoxic and fluorescent stains are visualized with UV light that can induce cellular damage. Accordingly, obtaining high quality images of live cells that can be used in high content screening (HCS) applications is challenging.

[0015] Thus, according to embodiments disclosed herein, cell detection in brightfield images may be perfomred using template matching with a reference image. The reference image may include a high contrast, symmetrical cell structure that is visible in the brightfield images, such as the endoplasmic reticulum (ER). The template matching may generate a correlation coefficient map that includes correlation coefficient values for each pixel of the input image (e.g., a brightfield image) that indicate how well each pixel (or groups of pixels) matches the reference image. Local maxima in the correlationDocket No. ARL24303PCTcoefficient map may indicate the location of each ER, and bounding boxes may be placed based on the local maxima to denote the location of each ER. Filtering by correlation coefficient threshold, local luminance threshold, local contrast threshold, and / or nominal candidate object bounding box overlap threshold may be used to balance false positive and false negative detection. This results in total nuclei (ER) count and the respective bounding boxes for all detected objects (ER). While the ER may act as a reliable proxy for the nucleus and hence cell, accurate identification of the location of each individual cell may be increased by utilizing nuclear masks to re-center the bounding boxes around the cell nuclei. The nuclear masks may be generated using a neural network, for example. The neural network may utilize as input only image data extracted from the FOV image that represents each ER as defined by the initially -placed bounding boxes (e.g., a sub-image may be generated from each initial bounding box and entered as input to the neural network to generate a nuclear mask), rather than using the entire FOV image as input.

[0016] By using brightfield images, cell / nuclei staining can be avoided or reduced, thereby reducing reliance on potentially harmful fluorescent stains. By using template matching, computational time and cost may be reduced relative to other processes for cell identification, such as segmentation. For example, segmentation may rely on relatively large neural networks or other models to segment multiple cells in an image or image tile, which have large memory footprints and demand high computational power to train and / or execute. And tiles used in conventional approaches to limit neural network image sizes as a function of total system memory have inefficiencies, complexities, and inaccuracies stemming from tile overlap, overcounting, and / or missing nuclei / cells along tile edges. In scenarios where segmentation is desired for downstream tasks, by first detecting a cellular structure present in each cell (e.g., the ER) using template matching as disclosed herein, once a cellular structure or nucleus is detected, the associated cell (w hich may or may not have one or more additional nuclei) can be more easily detected, segmented, and further analyzed because the respective task requires attention to only one cell instead of an image with multiple cells in an image tile, for example. Thus, instance or semantic segmentation may be facilitated using particularly efficient methods such as very small deep neural networks that can be trained very rapidly with a relatively small dataset.

[0017] Turning now to the figures, FIG. 1 shows an exemplary computing system 100, according to an embodiment. The computing system 100 includes a computing device 110. which further includes a processor 112 and a memory 114. The processor 112 may comprise one or more computational components usable for executing machine-readable instructions. For example, the processor 112 may comprise a central processing unit (CPU) or may include, for example a graphics processing unit (GPU). The processor 112 may be positioned within the computing device 110 or may be communicatively coupled to the computing device 110 via a suitable remote connection.

[0018] The memory 114 may comprise one or more types of computer-readable media, including volatile and / or non-volatile memory. The volatile memory may comprise, for example, random-accessDocket No. ARL24303PCTmemory (RAM), and the non-volatile memory may comprise read-only memory (ROM). The memory 114 may include one or more hard disk drive(s) (HDDs), solid state drives (SSDs), flash memory', and the like. The memory 114 is usable to store machine-readable instructions, which may be executed by the processor 112. The memory 114 is further configured to store images 116, which may comprise digital images captured or created using a variety of techniques, including digital imaging, digital illustration, and more.

[0019] At least a portion of the images 116 may be acquired via an imager 106. The imager 106 may be one or more of a microscope (e.g.. a light microscope, a fluorescence microscope), a multi-well plate imager, and another type of bioassay imager, for example. The imager 106 may include one or more light sources, including broad and / or narrow spectrum light sources. Examples of broad spectrum light sources include light sources that emit light over a wide wavelength range, such as lamps (e.g., mercury lamps, halogen lamps) that emit light spanning the ultra-violet (UV) and visible ranges. Examples of narrow spectrum light sources include light sources that emit light from a narrow wavelength range or wavelength band, such as light-emitting diodes (LEDs) and lasers. The imager 106 may further include at least one image sensor, such as a charge-coupled device (CCD), an electron multiplying CCD (EMCCD), an active pixel sensor (e.g., a complementary metal-oxide-semiconductor, or CMOS, sensor), or another type of sensor that detects light in a location-specific manner, such as in an array -based fashion. Additionally, the imager 106 may include one or more optical coupling devices (e.g., lenses and mirrors), filters, beam splitters, and the like that may be used to direct light of a desired wavelength or wavelength range to a sample being imaged and receive light transmitted by, reflected by, or emitted by (e.g., depending on the imaging modality) the sample at the image sensor(s).

[0020] The memory 114 further includes a detection module 118, which comprises machine-readable instructions that may be executed by the processor 112 to perform template matching in order to identify objects in the images 116. The processor 112 may utilize the machine-readable instructions contained by the detection module 118 for obtaining or generating a reference image, obtaining a field of view (FOV) image that includes a plurality of objects to be detected (e.g., cells), performing template matching with the reference image on the FOV image to generate a correlation coefficient map, and outputting a total object count and / or bounding boxes indicative of the location of each object.

[0021] In some examples, the memory 114 may further include a classification module 120, which comprises machine-readable instructions that may be executed by the processor 112 to segment and / or classify objects (e.g., cells) in the images 116. In some examples, the classification module 120 may include one or more neural networks trained to classify cells based on morphology, such as whether the cells are alive or dead, mitotic phase of the cells, and / or other morphology. The one or more neural networks may be trained to use instance images of individual cells generated based on the output of the detection module as input to determine cell morphology.Docket No. ARL24303PCT

[0022] The computing system 100 further includes a user interface 102, which may comprise one or more peripherals and / or input devices, including, but not limited to, a keyboard, a mouse, a touchpad, or virtually any other input device technology that is communicatively coupled to the computing device 110. The user interface 102 may enable a user to interact with the computing device 110, such as to select one or more images to evaluate, to select one or more parameters of the classification model, and so forth.

[0023] The computing system 100 further includes a display device 104, which may be configmed to display outputs of the detection module 118 and / or the classification module 120, display the images themselves, and / or display other features / parameters. In some examples, the display device 104 may be a liquid crystal display (LCD), a light-emitting diode display (LED), or an organic light-emitting diode display (OLED). The user may select or otherwise input parameters (e.g., imaging protocols) via the user interface 102 based on options displayed via the display device 104.

[0024] In some examples, the computing device 110 may be implemented over a cloud or other computer network. For example, the computing device 110 is shown in FIG. 1 as constituting a single entity', but it is to be understood that the computing device 110 may be distributed across multiple devices, such as across multiple servers. Processors of the computing device may be single core or multicore, and the programs executed thereon may be configured for parallel or distributed processing. A logic subsystem of the computing device (e.g., memory) may optionally include individual components that are distributed throughout two or more devices, which may be remotely located and / or configured for coordinated processing. One or more aspects of the logic subsystem may be virtualized and executed by remotely accessible networked computing devices configured in a cloud-computing configuration. For example, the images 116 may be stored in the memory 114 of the computing device 110 and accessed by detection module 118 on a remote device. Output from the detection module on the remote device may be sent back to the computing device 110 for storage in the memory 114.

[0025] As used herein, the terms “system” or “module” may include a hardware and / or software system that operates to perform one or more functions. For example, a module or system may include a computer processor, controller, or other logic-based device that performs operations based on instructions stored on a tangible and non-transitory computer readable storage medium, such as a computer memory. Alternatively, a module or system may include a hard-wired device that performs operations based on hard-wired logic of the device. Various modules or systems shown in the attached figures may7represent the hardware that operates based on software or hardwired instructions, the software that directs hardware to perform the operations, or a combination thereof. For example, the detection module 118 and / or the classification module 120 may be implemented using hardware, softw are, firmw are, or a combination thereof. In some aspects, several portions of the subject matter described herein may be implemented via Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), semiconductor devices based around a matrix of configurableDocket No. ARL24303PCTlogic blocks (CLBs), connected via programmable interconnects. Some aspects of the embodiments disclosed herein, in whole or in part, can be equivalently implemented in standard integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as a combination thereof.

[0026] FIG. 2 illustrates an example brightfield FOV image 200. FOV image 200 may be an example of an image of images 116 obtained with imager 106. FOV image 200 may include a plurality of cells, such as human epithelial cells. Due to the use of broad spectrum light and lack of cell stain, the cells are relatively transparent other than at the edges of the cell membranes (where the index of refraction causes light to reflect or not based on the critical angle). The FOV image 200 may include hundreds or thousands of cells.

[0027] In order to detect each cell in the FOV image 200, template matching using a reference image, such as the reference image shown in FIG. 3, may be performed, which may result in bounding boxes being placed over each identified cell in the FOV image 200. A region 202 of FOV image 200 is shown in magnified view in FIG. 4 including bounding boxes placed based on the template matching.

[0028] FIG. 3 shows an example reference image 300 that may be used to perform template matching to identify areas of an input image (e.g., FOV image 200) that match the reference image 300. The reference image 300 may be an image of an ER or may approximate an average or expected ER. The ER may be used as the cell structure for matching via the template matching because ERs are present in most eukaryotic cells, ERs are associated with cell nuclei and so are a proxy for the nuclei, and ERs are visible in brightfield images. Further, ERs consistently modulate the brightfield image across cells. The mean intensity is less and the size and shape of the ER is somewhat low variance compared to the cell shape and size. Said another way, ERs are visible, can be differentiated from other cell structures visible in brightfield images, and have low variance in size and shape. While each ER in each cell may not be symmetrical, the reference image 300 may be symmetrical, which may avoid orientation-specific effects.

[0029] As will be explained in more detail below, after transformation to the frequency domain, the reference image 300 may be ‘ slid” across the FOV image 200 and the similarity between the reference image 300 and the underlying FOV image 200 at the region where the reference image is located may be calculated and represented as a correlation coefficient for each position of the reference image 300. The correlation coefficients may be represented as map of the same size / pixel resolution of the FOV image 200, with each pixel of the map corresponding to a location of the reference image 300 relative to the FOV image 200 and having value that represents the correlation coefficient calculated for that location. After thresholding and / or filtering to reduce false negatives and false positives, a bounding box may be placed at each local maxima of the map, and the bounding boxes may be overlaidDocket No. ARL24303PCTon the FOV image 200, as shown in FIG. 4. In some examples, the bounding boxes that are ultimately output (e.g., and overlaid on the FOV image) may be adjusted based on nuclear masks to more accurate identify each cell, as explained in more detail below.

[0030] FIG. 4 shows a bounding box image 400 that includes the region 202 of the FOV image 200 with bounding boxes overlaid thereon, such as bounding box 402. The region 202 in image 400 is magnified relative to the FOV image 200 to better visualize individual cells. When the bounding boxes are overlaid on the FOV image, the bounding boxes may include rectangles that are visible on the FOV image. However, it is to be appreciated that the bounding boxes may be represented as pixel coordinates of the FOV image that identify the location and size of each bounding box. For example, each bounding box may be represented as the coordinates of the FOV image for the top left corner and bottom right comer of the bounding box (or the top right comer and bottom left comer).

[0031] FIG. 5 is a flowchart that illustrates a method 500 for identifying cells in a FOV image using template matching via a detection module. The detection module may be a non-limiting example of the detection module 118 of FIG. 1. Method 500 may be carried out by a computing device, such as the computing device 110 of FIG. 1 (e.g., by executing instructions stored in memory of the computing device). Method 500 may be executed in response to a request to perform object detection on a selected FOV image. In some examples, the request may be a manual request initiated by a user (e.g., a user may select a FOV image and enter a request to detect objects in the FOV image by selecting a user interface element or menu option). In other examples, the request may be an automatic request initiated in response to execution of a particular imaging protocol. For example, a user may select an imaging protocol from among a plurality of stored imaging protocols. The selected imaging protocol may dictate drat object detection be performed on all or on selected FOV images obtained while executing the imaging protocol.

[0032] At 502. method 500 includes obtaining a brightfield FOV image. The brightfield FOV image may include a plurality of cells of a suitable fype(s) (e g., mammalian epithelial cells or other eukaryotic cells) obtained with a suitable imaging system, such as imager 106, with brightfield illumination. The brightfield FOV image (also referred to as a FOV image) may have a suitable resolution, such as 0.27 microns per pixel. However, other resolutions are possible, such as resolutions lower than 0.27 microns per pixel (e.g., up to half the resolution of 0.27 microns per pixel). FOV image 200 is an example brightfield FOV image that may be obtained and used for object detection as disclosed herein.

[0033] At 504, method 500 includes obtaining a reference image. The reference image may be an image of a cellular structure that is present in the cells of the FOV image and that can be detected via template matching. In some examples, the reference image may be an image of an endoplasmic reticulum (ER). In some examples, the reference image may not be an actual image of the cellular structure, but may be a representation of the cellular structure generated by averaging a plurality ofDocket No. ARL24303PCTimages of the cellular structure or via another suitable method. Reference image 300 of FIG. 3 is an example of a reference image that may be obtained and used for template matching to detect objects in the FOV image, described below.

[0034] At 506. method 500 includes performing template matching on the FOV image with the reference image. Additional details about performing the template matching are provided below with respect to FIG. 6. Briefly, the template matching may identify area(s) of the FOV image that are similar to the reference image (e.g., that match the reference image). Each area that is deemed to sufficiently match the reference image may be identified as an ER (or nucleus). The template matching process may result in output of a count of the total nuclei (or ER) count in the FOV image, as indicated at 508, and / or output of bounding boxes indicative the location of each ER / nucleus and hence cell in the FOV image. The total nuclei / ER / cell count output at 508 and / or the bounding boxes output at 510 may be displayed on a display device (e.g., display device 104) and / or saved in memory (e.g., memory 114) for use in or more downstream tasks. For example, the bounding boxes may be used to visually identify the boundary of each identified cell by overlaying the bounding boxes on the FOV image, as shown by the bounding box image 400 of FIG. 4, which may then be displayed on the display device. In some examples, the bounding boxes may be used to segment the FOV image into instance images, also referred to as subimages. For example, the FOV image may be partitioned / divided into a plurality of sub-images based on the coordinates of the bounding boxes, with each sub-image including a region of the FOV image contained within the coordinates of a respective bounding box.

[0035] Thus, as indicated at 512, method 500 may optionally include defining sub-images for further processing based on the bounding boxes. The processing of the sub-images may include, as indicated at 514, processing the sub-images for segmentation and / or direct cell classification. For example, the classification module 120 of FIG. 1 may be invoked to classify each cell in each subimage. The classification may include classifying cells as alive or dead, classifying by mitotic phase, and / or classifying by another suitable morphological feature. The model(s) of the classification module that are invoked to segment and / or classify the cells may be trained to use sub-images as input (e.g., where each sub-image includes only a single cell), rather than an entirety of the FOV image or image tiles of the FOV image that include multiple cells. Segmentation for each nucleus or cell may be used to create re-centered respective bounding boxes. For example, each originally -placed bounding box may be centered over a respective ER. Segmentation may be used to move one or more bounding boxes so the bounding boxes are centered around the nucleus or cell. This can result in significant improvements in measurement accuracy, for example, of the area of respective objects. Method 500 then ends.

[0036] FIG. 6 is a flowchart that illustrates a method 600 for performing template matching on a FOV image. Method 600 may be carried out by a computing device, such as the computing device 110Docket No. ARL24303PCTof FIG. 1 (e.g., by executing instructions stored in memory’ of the computing device). In some examples, method 600 may be carried out as part of method 500, such as at 506 of method 500.

[0037] At 602, method 600 includes validating the FOV image and the reference image. As explained above, the FOV image may be a brightfield image of a plurality of cells and the reference image may be an image of (or representation of) a cellular structure such as an ER. Validating the FOV image and the reference image may include confirming both the FOV image and reference image have been obtained and are 2D arrays. Further, in some examples, validating may include confirming that the size / resolution of the reference image matches the size / resolution of the FOV image. The reference image may be created with an expected magnification (and hence size) of the cells in the FOV image, and the reference image may be scaled (e.g., reduced or increased in size) to match the expected size of the ERs in the FOV image, if needed.

[0038] At 604, template matching on the FOV image with the reference image is performed to generate a correlation coefficient map. Template matching may utilize a 2D correlation-based method to determine how well each region of the FOV image matches the reference image, with the degree of matching represented by a correlation coefficient (e.g., Pearson’s linear correlation coefficient) for each region. For example, the reference image may be compared to a first region of the FOV image (that is the same size and shape as the reference image) at the upper-left corner of the FOV image, and a first correlation coefficient may be calculated for that region (and placed in the map at a location corresponding to the centroid pixel of the first region). The reference image may be “slid” one pixel to the right relative to the FOV image to compare the reference image to a second region of the FOV image. A second correlation coefficient may be calculated for the second region (and placed in the map at a location corresponding to the centroid pixel of the second region). The process may continue until the reference image has been slid over (and compared to) each respective region of the top of the FOV image; the reference image may then be slid down one pixel and compared to each respective region across the FOV image back toward the left; and so forth until a correlation coefficient has been calculated for each region of the FOV image.

[0039] The template matching may include Fourier domain-based correlation. As such, the FOV image and reference image are transformed to the frequency domain (e.g., using a Fourier transform) and the comparison described above takes place in the frequency domain. More specifically, when performing template matching in the frequency domain, the reference image is zero-padded to match the dimensions of the FOV image. Both the zero-padded reference image and the FOV image are then transformed to the frequency domain using a Fast Fourier Transform (FFT). The correlation is computed by performing element-wise multiplication of the Fourier transform of the FOV image with the complex conjugate of the Fourier transform of the zero-padded reference image. An inverse FFT is then applied to the result, producing a correlation coefficient map in the spatial domain. Each pixel location in this correlation coefficient map corresponds to a position in the FOV image where theDocket No. ARL24303PCTreference image could be centered, and the value at that pixel represents the correlation coefficient between the reference image and the corresponding region of the FOV image at that position. This approach efficiently computes correlation coefficients for all possible positions of the reference image within the FOV image simultaneously, which is equivalent to sliding the reference image across the FOV image in the spatial domain and computing correlation coefficients at each position, but is computationally more efficient, particularly for large FOV images.

[0040] In the context of identifying cells in brightfield images, Fourier domain correlation offers significant advantages over traditional time domain methods. By transforming images into the Fourier domain, the correlation coefficient detection can be leveraged to perform template matching more efficiently. This approach allows for rapid identification of cell patterns, as it focuses on frequency components rather than pixel intensity variations, which can be influenced by noise and lighting conditions.

[0041] Using Fourier transforms, the correlation process becomes computationally faster, enabling real-time analysis of large datasets. This is particularly beneficial in high-throughput microscopy, where quick and accurate cell detection is crucial. In contrast, time domain methods often demand extensive preprocessing and can be slower due to the need for pixel-wise comparisons, making them less suitable for large-scale applications. Overall, Fourier-based techniques enhance both the speed and robustness of cell identification in brightfield imaging.

[0042] At 606, method 600 may include filtering and / or thresholding the correlation coefficient map to create a filtered binary map. The filtering / thresholding may include filtering by correlation coefficient threshold, local luminance threshold, and / or local contrast threshold, which may act to reduce instances of false positives and false negatives. For example, for correlation coefficient threshold filtering, each correlation coefficient may be compared to a correlation coefficient threshold and the correlation coefficients that are greater than the correlation coefficient threshold may be identified as matching / passing and represented as a value of one on the map. The correlation coefficients that are less than the correlation coefficient threshold may be represented as a value of zero on the map. For local luminance and / or local contrast thresholding, a similar process may be applied. For example, a mean luminance of each region of the FOV image determined to match the reference image (e.g., the regions assigned a value of one based on the correlation coefficient) may be calculated and compared to a luminance threshold. If the mean luminance is above the luminance threshold, the region is confirmed as matching the reference image and the value of one is maintained; if the mean luminance is not above the threshold, the region is not confirmed as matching the reference image and the value is changed to zero. A similar process may be applied for local contrast thresholding using a contrast threshold. Each threshold may be a minimum allowable value. The thresholds may be selected manually via example image audits or selected automatically by comparing benchmarks of bounding box countsDocket No. ARL24303PCTand locations with respective bounding boxes found using nuclear seeding (e.g., nuclear stain detected in a fluorescent channel, which provides high contrast for easy detection, location, and counting).

[0043] At 608. method 600 includes performing local maxima detection in the filtered binary map. The local maxima detection may identify pixels of the filtered binary map that have a value higher than all of their immediate neighboring pixels. Each pixel / local maximum identified may be designated as an initial detected object (e.g., ER / nucleus). At 610, initial bounding boxes are calculated around the initial detected objects. The bounding boxes may be centered on the identified pixel and have a shape that matches the reference image and a size based on an expected size of the detected objects (e.g., an average or expected cell size).

[0044] At 612. the bounding boxes are filtered by a bounding box overlap threshold. The bounding box overlap threshold may be a percentage of overlap between neighboring bounding boxes, such as 50% overlap or 75% overlap. If neighboring bounding boxes overlap by more than the bounding box overlap threshold, one of the bounding boxes may be removed or a bounding box may not be placed to begin with. For example, bounding boxes are considered to be added for local maxima points. When a bounding box is in the process of being added, the bounding box is checked relative to the locations of any existing bounding boxes. If the bounding box to be added overlaps an existing bounding box by more than the bound box overlap threshold, the bounding box is not added. The bounding boxes that remain may be the final bounding boxes that may be used to generate the output (e.g., cell / nucleus / ER count, the bounding boxes themselves).

[0045] At 614, method 600 determines if mask processing is to be performed. Mask processing may be performed when it is desired to segment the detected objects in order to do morphology analysis or another desired task on the detected objects. If mask processing is to be performed, method 600 proceeds to 616 to perfonn mask processing. The mask processing may include normalizing and thresholding a mask for each detected object and optionally removing speckle noise. Each mask may be a binary image where each pixel is classified as part of a detected object or not, with pixels identified as part of the detected object given a value of 1 and pixels that are not part of the detected object given a value of 0. The mask processing may be performed with a suitable model, such as a neural network. The bounding boxes placed as described above may be initial bounding boxes that are centered on the initial detected object (e.g., cellular structure such as the ER). The initial bounding boxes may be used to divide the FOV image into sub-images / image patches and each sub-image / image patch may be processed via the mask processing model. In some examples, the mask processing may include using masks (e.g., nuclear or cellular masks) to identify the nucleus or cell in each sub-image / image patch. One or more or all of the initial bounding boxes may be adjusted in position so that the final bounding boxes are re-centered over the nucleus or cell. By using sub-images instead of the entire FOV image or tiles of the FOV image, the generation of the masks may be performed faster and with smaller neuralDocket No. ARL24303PCTnetworks that utilize less training data. Further, the template matching described herein is much more efficient than using a neural network to find cell (via ER) instances.

[0046] At 618, the output is generated. As mentioned above, the output may include a total cell / nucleus / ER count, the final bounding boxes, and / or an image that includes the final bounding boxes overlaid on the FOV image. If mask processing is perfonned, the output may include the binary image / mask for each detected object / sub-image. Method 600 then ends.

[0047] Thus, the template matching to detect cells in a brightfield FOV image as disclosed herein has many advantages. For example, utilizing a brightfield image allows for elimination of fluorescent dyes / stains, which reduces cytotoxicity and photobleaching issues associated with fluorescent dyes like DAPI and Hoechst. Thus, safer and more sustainable live cell imaging may be performed, preserving cell viability and integrity over longer periods. Additionally, the use of brightfield microscopy allows for utilization of existing brightfield microscopy setups, which are more widely available and less expensive than advanced fluorescence microscopy systems, thereby increasing accessibility and reducing costs for laboratories, making high-content screening more feasible for a broader range of research and clinical applications.

[0048] Further, the template matching to initially identify the cells allows for additional downstream processing to be performed with relatively smaller (so more efficient) Deep Neural Networks (DNNs). In this way, very small DNNs that can be trained rapidly with a relatively small dataset may be employed for morphological analysis or other tasks. Most any advancements in DNNs can be exploited with a smaller input and therefore total memory and nodes required. This reduces computational resources and training time, allowing for quicker deployment and iteration of models. Similarly, instance and semantic segmentation may be employed to facilitate accurate detection and segmentation of cell nuclei in brightfield images using advanced image processing techniques, which improves the accuracy and reliability of cell nucleus detection, leading to better data quality and more robust analysis in high-content screening.

[0049] Overall, the systems and methods disclosed herein provide cost and time savings by eliminating the need for nuclear stains and simplify ing the image processing workflow. As such, overall costs and processing time may be reduced, making high-content screening more efficient and cost-effective. Likewise, the systems and methods disclosed herein demand fewer computing resources and / or less complex computational infrastructure compared to traditional methods, which lowers the barrier to entry for laboratories with limited computational resources, enabling more widespread adoption of the technology. Thus, template matching on brightfield FOV images effectively addresses the limitations of traditional nuclear detection methods, providing a more efficient, cost-effective, and accessible solution for high-content screening and cell analysis.

[0050] It is to be appreciated that other cellular structures could be used as the basis for the reference image, as long as the cellular structure is present in each cell and has high contrast. OtherDocket No. ARL24303PCTexample cellular structures that could be used include portions of the cell membrane, for example. Depending on the cell type and / or other parameters defining the FOV image, other consistent aspects of the cells may be present that can be used to identify the cells. However, the ER is a readily -apparent organelle of commonly -used cell types in cell-based assays for cell-based therapies such as drug discovery, gene therapy, etc. Further, while the template matching disclosed herein is particularly advantageous for use on brightfield FOV images, the template matching may be performed on FOV images of samples that include stained cells, so long as the reference image represents a cellular structure as it would appear when stained.

[0051] The technical effect of identifying individual cells of a plurality of cells in a FOV image by performing template matching on the FOV image using a reference image is that the individual cells may be detected using a fast and efficient image processing technique that has a lower computational burden compared to prior methods for detecting individual cells in a FOV image.

[0052] In another representation, a system includes memory storing instructions, and one or more processors configured to execute the instructions to: obtain a brightfield field of view (FOV) image of a sample including a plurality of cells, obtain a reference image of a cellular structure, perform template matching on the FOV image using the reference image to place a plurality of initial bounding boxes, each initial bounding box indicating a location of an individual instance of the cellular structure in the FOV image, adjust a position of one or more of the plurality of initial bounding boxes using nuclear masks to form a plurality of final bounding boxes, and output the plurality of final bounding boxes. In a first example of the system, adjusting the position of one or more of the plurality’ of initial bounding boxes using nuclear masks to form the plurality' of final bounding boxes comprises: for a selected initial bounding box that indicates the location of the cellular structure in a selected individual cell, generating a selected nuclear mask that identifies the nucleus in the selected individual cell using a neural network, and re-centering the selected initial bounding box so that a resultant final bounding box is centered over the nucleus. In a second example of the system, optionally including the first example, only image data of the FOV image at the location defined by the selected initial bounding box is entered as input to the neural network in order to generate the selected nuclear mask.

[0053] The following claims particularly point out certain combinations and sub-combinations regarded as novel and non-obvious. These claims may refer to "an” element or "a first” element or the equivalent thereof. Such claims should be understood to include incorporation of one or more such elements, neither requiring nor excluding two or more such elements. Other combinations and subcombinations of the disclosed features, functions, elements, and / or properties may be claimed through amendment of the present claims or through presentation of new claims in this or a related application. Such claims, whether broader, narrower, equal, or different in scope to the original claims, also are regarded as included within the subject matter of the present disclosure.

Claims

Docket No. ARL24303PCTCLAIMS1. A system, comprising:memory storing instructions; andone or more processors configured to execute the instructions to:obtain a brightfield field of view (FOV) image of a sample including a plurality of cells; obtain a reference image of a cellular structure; andidentify individual cells of the plurality of cells by performing template matching on the FOV image using the reference image.

2. The system of claim 1. wherein the cellular structure is an endoplasmic reticulum.

3. The system of claim 1 or 2. wherein identifying individual cells of the plurality of cells by performing template matching on the FOV image using the reference image comprises performing the template matching to generate a correlation coefficient map, the correlation coefficient map including a respective correlation coefficient value for each region of the FOV image that indicates how well that region matches the reference image.

4. The system of claim 3, wherein identifying individual cells of the plurality of cells by performing template matching on the FOV image using the reference image further comprises filtering and / or thresholding the correlation coefficient map to generate a filtered binary map.

5. The system of claim 4, wherein filtering and / or thresholding the correlation coefficient map comprises filtering and / or thresholding using one or more of a correlation coefficient threshold, a luminance threshold, and a contrast threshold.

6. The system of claim 4 or 5, further comprising performing local maxima detection on the filtered binary map to identify a location of the cellular structure of each individual cell.

7. The system of any one of claims 1-6, wherein the one or more processors are configured to execute the instructions to output a total count of identified individual cells in the FOV image and / or a plurality of bounding boxes, each bounding box indicating a location of a respective identified individual cell in the FOV image.

8. The system of claim 7. wherein outputting the plurality' of bounding boxes comprises:generating a plurality of initial bounding boxes, each initial bounding box having a position centered on a position of a corresponding cellular structure as identified by the template matching;Docket No. ARL24303PCTadjusting one or more of the initial bounding boxes using nuclear masks to form the plurality of bounding boxes, wherein adjusting one or more of the initial bounding boxes using nuclear masks comprises re-centering the one or more of the initial bounding boxes.

9. The system of any one of claims 1-8, further comprising an imager, wherein the FOV image is captured with the imager using brightfield illumination.

10. A method, comprising:obtaining a brightfield field of view (FOV) image of a sample including a plurality of cells; obtaining a reference image of a cellular structure;identifying individual cells of the plurality of cells by performing template matching on the FOV image using the reference image; andoutputting a total count of the identified individual cells in the FOV image and / or a plurality of bounding boxes, each bounding box indicating a location of a respective identified individual cell in the FOV image.

11. The method of claim 10, wherein identifying individual cells of the plurality of cells by performing template matching on the FOV image using the reference image comprises identify ing individual instances of the cellular structure in the FOV image via the template matching and placing an initial bounding box centered over each identified cellular structure.

12. The method of claim 11, further comprising adjusting a position of one or more of or each initial bounding box using a respective mask to form the plurality of bounding boxes, each respective mask identifying a nucleus of each individual cell.

13. The method of any one of claims 10-12, wherein identifying individual cells of the plurality of cells by performing template matching on the FOV image using the reference image comprises performing template matching on a Fourier- transformed version of the FOV image to generate a correlation coefficient map, the correlation coefficient map including a respective correlation coefficient value for each region of the FOV image that indicates how well that region matches the reference image.

14. The method of claim 13, further comprising filtering and / or thresholding the correlation coefficient map using one or more of a correlation coefficient threshold, a luminance threshold, and a contrast threshold to create a filtered binary map and performing local maxima detection on the filtered binary map to identify a location of the cellular structure of each individual cell.Docket No. ARL24303PCT15. The method of any one of claims 10-14, wherein the cellular structure is an endoplasmic reticulum.

Images