Reference image generation for cell detection

Abstract

Methods and systems are provided herein for generating a reference image of a cellular structure for object detection in an image. In one example approach a method is provided that comprises obtaining a plurality of images of a cellular structure; training a deep neural network based on the plurality of images; generating a reference image via the deep neural network; calculating a Pearson's linear correlation coefficient between each image in the plurality of images and the reference image; refining the reference image based on the calculated Pearson's linear correlation coefficients; and outputting the reference image.

Description

Docket No. ARL25305PCTREFERENCE IMAGE GENERATION FOR CELL DETECTIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims priority to U.S. Provisional Patent Application No.63 / 775,892, filed March 21, 2025, and entitled “REFERENCE IMAGE GENERATION FOR CELL DETECTION.” The present application also claims priority to U.S. Provisional Patent Application No.63 / 746,838, filed January 17, 2025. and entitled “SYSTEMS AND METHODS FOR BRIGHTFIELD CELL DETECTION.” The entire contents of each of the above-identified applications is hereby incorporated by reference for all purposes.TECHNICAL FIELD

[0002] Embodiments of the subject matter disclosed herein relate generally to cell identification and segmentation, and more specifically to cell identification in brightfield images.BACKGROUND AND SUMMARY

[0003] Image analysis in high content screening relies on accurate detection of cells, and generally the nucleus is the primary’ target of initial detection. In practice, cell detection is usually accomplished using fluorescent stains that target the cell nucleus. However, staining the nucleus with fluorescent dyes can be problematic for live cell imaging due to potential cytotoxicity and photobleaching effects. Traditionally, this has been addressed by using dyes such as Hoechst or DAPI, which bind to DNA and fluoresce under UV light.

[0004] However, these methods have limitations, including the need for UV illumination, which can damage live cells, and the potential for non-specific binding, leading to background fluorescence. Also, capturing the fluorescent signal means using a fluorescent channel for nuclei that might otherwise be useful for detecting and measuring other cell constituents and / or activity.

[0005] In some approaches, methods and systems for automatically detecting objects may be used that identify cells in an image, e.g., a brightfield field of view (FOV) image of a sample, by performing template matching on the FOV image using a selected reference image. In such approaches, object detection methods may use similarity metrics such as correlation, against the reference signal or image. Often example reference images or signals may be manually selected for this purpose.

[0006] The inventors herein have recognized that template match selection in such approaches may rely on audits and trial and error to select an optimal reference image for object detection. For users of high content analysis software, this manual selection or creation of the reference image for template matching is typically a non-intuitivc, daunting task. The inventors herein have recognized that it may be advantageous to reduce or substantially eliminate the uncertainty of the optimality of the template reference selection or creation.Docket No. ARL25305PCT

[0007] In order to address these and other issues, methods and systems are provided herein for generating a reference image of a cellular structure for object detection in an image. In one example approach a method comprises obtaining a plurality of images of a cellular structure; training a deep neural network (DNN) based on the plurality of images; generating a reference image via the deep neural network; calculating a Pearson’s linear correlation coefficient between each image in the plurality' of images and the reference image; refining the reference image based on the calculated Pearson’s linear correlation coefficients; and outputting the reference image.

[0008] Such an approach takes advantage of modern DNN training optimizations to optimize the Pearson’s linear correlation coefficient (r) statistics for an ensemble of cell images. The result is an automated way to generate an optimized template reference image that has maximum Pearson’s linear correlation coefficient statistics for an ensemble of example input images. Further, such an approach may increase speed and accuracy of object detection and may reduce or substantially eliminate manual template reference selection or creation and the uncertainties surrounding that process.

[0009] It should be understood that the brief description above is provided to introduce in simplified form a selection of concepts that are further described in the detailed description. It is not meant to identity' key or essential features of the claimed subject matter, the scope of which is defined uniquely by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 shows an example diagram of a computing device.

[0011] FIG. 2 shows an example method for detecting cells in a brightfield FOV image.

[0012] FIG. 3 shows an example method for generating a reference image of a cellular structure for object detection in an image.

[0013] FIG. 4 shows an example method of using a deep neural network (DNN) for template reference generation.

[0014] FIG. 5 shows an example method for generating a reference template image.

[0015] FIG. 6 shows an example method for generating a brightfield nuclear reference template image.

[0016] FIG. 7 shows an example initial template match image.

[0017] FIG. 8 shows an example heatmap of bounding box (bbox) count per well using a reference template image for correlation coefficient based template matching.

[0018] FIG. 9 shows an example of bounding boxes overlaid on a nuclear channel after detecting nuclei with an initial reference template image.

[0019] FIG. 10 shows example sub-images used as a “positive” class for generating a reference template image.Docket No. ARL25305PCT

[0020] FIG. 11 shows an example nuclear stain channel generated brightfield reference template image.

[0021] FIG. 12 shows an example plot of accuracy versus image quality.DETAILED DESCRIPTION

[0022] The present description relates to methods and systems for generating a reference image of a cellular structure for object detection in an image. In some examples, such reference images may be used to identify cells in brightfield images using a direct “template match” method that takes advantage of image Fourier transform-based correlation coefficient detection for extremely fast detection. Signals in brightfield images of cells have reliable proxies for nuclei that can be rapidly and robustly detected with extremely light processing compared to the smallest neural net. The resulting benefits include time and cost savings from not needing a nuclear stain, then instance brightfield processing, requiring far less sophistication, computational cost, and time.

[0023] Brightfield microscopy includes illumination of a sample (e.g., cells on a slide or plate) with broad spectrum light (e.g., white light) and detection of the light via a detector / camera to create brightfield images. The sample appears dark against a bright background. Brightfield microscopy has many advantages, including simplicity, low cost, and wide availability. However, brightfield microscopy is limited in that it is difficult to image samples that have low contrast with the background. For imaging samples such as cells, imaging is typically improved with cell stains, such as fluorescent stains, that bind specific cellular structures such as DNA and allow visualization of the cell nuclei, for example. However, some cell stains are cytotoxic and fluorescent stains are visualized with UV light that can induce cellular damage. Accordingly, obtaining high quality images of live cells that can be used in high content screening (HCS) applications is challenging.

[0024] As remarked above, in some approaches cell detection in brightfield images may be performed using template matching with a reference image. The reference image may include a high contrast, symmetrical cell structure that is visible in the brightfield images, such as the endoplasmic reticulum (ER) or the cell nucleus. The template matching may generate a correlation coefficient map that includes correlation coefficient values for each pixel of the input image (e.g., a brightfield image) that indicate how well each pixel (or groups of pixels) matches the reference image. Local maxima in the correlation coefficient map may indicate the location of each ER. and bounding boxes may be placed based on the local maxima to denote the location of each ER. Filtering by correlation coefficient threshold, local luminance threshold, local contrast threshold, and / or nominal candidate object bounding box overlap threshold may be used to balance false positive and false negative detection. This results in total nuclei (ER) count and the respective bounding boxes for all detected objects (ER). While tire ER may act as a reliable proxy' for the nucleus and hence cell, accurate identification of the location of each individual cell may be increased by utilizing nuclear masks to re-center the bounding boxesDocket No. ARL25305PCTaround the cell nuclei. The nuclear masks may be generated using a neural network, for example. The neural network may utilize as input only image data extracted from the FOV image that represents each ER as defined by the initially -placed bounding boxes (e.g., a sub-image may be generated from each initial bounding box and entered as input to the neural network to generate a nuclear mask), rather than using the entire FOV image as input.

[0025] By using brightfield images, cell / nuclei staining can be avoided or reduced, thereby reducing reliance on potentially harmful fluorescent stains. By using template matching, computational time and cost may be reduced relative to other processes for cell identification, such as segmentation. For example, segmentation may rely on relatively large neural networks or other models to segment multiple cells in an image or image tile, which have large memory footprints and demand high computational power to train and / or execute. And tiles used in conventional approaches to limit neural network image sizes as a function of total system memory have inefficiencies, complexities, and inaccuracies stemming from tile overlap, over counting, and / or missing nuclei / cells along tile edges. In scenarios where segmentation is desired for downstream tasks, by first detecting a cellular structure present in each cell (e.g.. the ER of the nucleus) using template matching as disclosed herein, once a cellular structure or nucleus is detected, the associated cell (w hich may or may not have one or more additional nuclei) can be more easily detected, segmented, and further analyzed because the respective task requires attention to only one cell instead of an image with multiple cells in an image tile, for example. Thus, instance or semantic segmentation may be facilitated using particularly efficient methods such as very small deep neural netw orks that can be trained very rapidly w ith a relatively small dataset.

[0026] As remarked above, for object detection methods using similarity metrics such as correlation, a reference signal is used to correlated test signals against. Often example signals are manually selected as “exemplars" for this purpose. They may be evaluated in a trial and error basis to select the reference signal that results in highest correlation scores or highest classification benchmarks.

[0027] In the case of cell or cell organelle identification, for example, the nucleus is typically identified first using a “nuclear seed” image. The “nuclear seed” image is typically a field of view with many nuclei stained with fluorescent dye that only target the nuclei. Knowledge that the nuclei can be in any orientation, and that they are approximately oval implies that a good reference that has no rotational bias for detection will have rotational symmetry. And the range of nuclei size implies there will be a circular image with no abrupt edges as expected from integrating a continuous range of different size ovals at all angles.

[0028] However, for optimizing the reference template image for object detection, another approach is to use machine learning to generate optimal reference images for object detection. The inventors herein have recognized that it may be advantageous to generate an image with optimal detection properties as reflected in metrics and benchmarks such as the highest Pearson’s linearDocket No. ARL25305PCTcorrelation coefficient, r; and therefore what is needed is a method to generate a template reference image that has maximum Pearson’s linear correlation coefficient statistics for an ensemble of example input images.

[0029] The Pearson correlation coefficient (PCC), which is also referred to herein as r, r score, r value and the like, is a correlation coefficient that measures linear correlation betw een tw o sets of data. It is the ratio between the covariance of two variables and the product of their standard deviations; thus, it is essentially a normalized measurement of the covariance, such that the result has a value between -1 and 1.

[0030] The systems and methods disclosed herein may include taking a set of cell images as a dataset for training, validating and testing and sending through a deep neural network (DNN) that produces a reference image and individual respective r scores (i.e.. Pearson’s linear correlation coefficients). The systems and methods described herein may utilize DNN training optimizations to optimize the Pearson’s linear correlation coefficient statistics for an ensemble of cell images. The result is an automated way to generate an optimized template reference image.

[0031] In the systems and methods disclosed herein, training of the deep neural network may include comparing each image's r score with a respective ground truth label of 1 or 0. The dataset may be generated from images of an ensemble of target objects (e.g.. nuclei) where nuclei have already been identified by prior methods and sub-images taken via bounding boxes centered on each nucleus are labelled 1. and sub-images where bounding boxes are offset by over 1 typical diameter of a nucleus are labelled 0. Thus the dataset has examples of images that should correlate maximally and others that should correlate minimally (not including negative correlation).

[0032] The systems and methods for generating a reference image of a cellular structure for object detection in an image described herein may be used for template matching in brightfield images, as described above, and / or for other imaging techniques for detecting most commonly targeted cell image type objects including but not limited to endoplasmic reticula, nuclei, and / or other cellular structures.

[0033] The systems and methods for generating a reference image of a cellular structure for object detection in an image described herein provide many advantages. For example, a reference template image generated via the methods described herein may enable a template matching process that is computationally fast, efficient and accurate. As another example, the systems and methods described herein may enable a reduction in uncertainties by eliminating the need for manual template reference selection or creation. As remarked above, such manual template reference selection or creation may involve audits and trial and error for template match selection. For most users of high content analysis software, this manual selection or creation of the reference image for template matching is typically a non-intuitive, daunting task. In contrast the systems and methods described herein may enable a reduction of the uncertainty of the optimality of the template reference selection or creation.Docket No. ARL25305PCT

[0034] In the approaches described herein, after a DNN is trained to generate image references (referred to herein as inference mode after the DNN has been trained), the generation of template images may be performed very fast using a relatively small amount of computer processing resources. While training a DNN results in (at least approximate) optimality of the reference template image, it may take minutes depending on implementation details, dataset, and other considerations. But after training, inference can be used without training to provide optimal results relative to the closeness in feature space to the original training data. For example, if U2OS cells (i.e., cells from an osteosarcoma cell line derived from a human osteosarcoma patient) with relatively normal conditions are imaged on a particular high-content analysis (HCA) imaging system and most of the conditions are close to normal (the training dataset covered the important variance of data), then an approximately optimal reference can be generated very quickly via inference of the models described herein.

[0035] In the approaches described herein, filter parameter(s) can be determined as model weights during training instead of manual trial and error (correlation coefficient threshold is included herein as an example). Additionally, the approaches described herein may be used to handle difficult or non-intuitive images. Brightfield-based reference template generation is an important example included herein with verification for detecting and counting nuclei via a nuclear channel ground truth. In some approaches, endoplasmic reticulum (ER) have been used specifically as a target instead of directly targeting nuclei because the latter was too difficult; but ER is not always available and is not centered over nuclei, so the approaches described herein may be used to advantageously target nuclei. Moreover, if ER targeting is desired for particular applications, the systems and methods described herein can be used to optimize a reference template image to match ERs of cells or other cellular structures in the images.

[0036] The systems and methods described herein may be used in any suitable correlation-based detection process that uses a reference image for object detection and have example object “signals” (images, etc.) that can be used for DNN training. For high content analysis, this includes, but is not limited to: nuclei using typical fluorescent stains or in brightfield images, spots of all kinds, including mitochondria, liposomes, nucleoli, etc., endoplasmic reticulum (fluorescence stained, bright field, etc.), golgi, and various other cellular structures.

[0037] Turning now to the figures, FIG. 1 shows an example computing system 100, according to an embodiment. The computing system 100 includes a computing device 110, which further includes a processor 112 and a memory 114. The processor 112 may comprise one or more computational components usable for executing machine-readable instructions. For example, the processor 112 may comprise a central processing unit (CPU) or may include, for example a graphics processing unit (GPU). The processor 112 may be positioned within the computing device 110 or may be communicatively coupled to the computing device 110 via a suitable remote connection.Docket No. ARL25305PCT

[0038] The memory 114 may comprise one or more types of computer-readable media, including volatile and / or non-volatile memory. The volatile memory may comprise, for example, random-access memory (RAM), and the non-volatile memory may comprise read-only memory (ROM). The memory 114 may include one or more hard disk drive(s) (HDDs), solid state drives (SSDs), flash memory, and the like. The memory 114 is usable to store machine-readable instructions, which may be executed by the processor 112. The memory 114 is further configured to store images 116, which may comprise digital images captured or created using a variety of techniques, including digital imaging, digital illustration, and more.

[0039] At least a portion of the images 116 may be acquired via an imager 106. The imager 106 may be one or more of a microscope (e.g., a light microscope, a fluorescence microscope), a multi-well plate imager, and another type of bioassay imager, for example. The imager 106 may include one or more light sources, including broad and / or narrow spectrum light sources. Examples of broad spectrum light sources include light sources that emit light over a wide wavelength range, such as lamps (e g., mercury lamps, halogen lamps) that emit light spanning the ultra-violet (UV) and visible ranges. Examples of narrow spectrum light sources include light sources that emit light from a narrow wavelength range or wavelength band, such as light-emitting diodes (LEDs) and lasers. The imager 106 may further include at least one image sensor, such as a charge-coupled device (CCD), an electron multiplying CCD (EMCCD), an active pixel sensor (e.g., a complementary metal-oxide-semiconductor, or CMOS, sensor), or another type of sensor that detects light in a location-specific manner, such as in an array-based fashion. Additionally, the imager 106 may include one or more optical coupling devices (e.g., lenses and mirrors), filters, beam splitters, and the like that may be used to direct light of a desired wavelength or wavelength range to a sample being imaged and receive light transmitted by, reflected by, or emitted by (e.g., depending on the imaging modality) the sample at the image sensor(s).

[0040] The memory 114 further includes a detection module 118, which comprises machine-readable instructions that may be executed by the processor 112 to perform template matching in order to identify objects in the images 116. The processor 112 may utilize the machine-readable instructions contained by the detection module 118 for obtaining or generating a reference image, obtaining a field of view (FOV) image that includes a plurality of objects to be detected (e.g., cells), performing template matching with the reference image on the FOV image to generate a correlation coefficient map, and outputting a total object count and / or bounding boxes indicative of the location of each object. The processor 112 may utilize machine-readable instructions to implement various method and routines described herein to generate a reference image of a cellular structure for object detection in an image.

[0041] In some examples, the emory 114 may further include a classification module 120, which comprises machine-readable instructions that may be executed by the processor 112 to segment and / or classify objects (e.g., cells) in the images 116. In some examples, the classification module 120 may include one or more neural networks trained to classify cells based on morphology7, such as whether theDocket No. ARL25305PCTcells are alive or dead, mitotic phase of the cells, and / or other morphology. The one or more neural networks may be trained to use instance images of individual cells generated based on the output of the detection module as input to determine cell morphology.

[0042] The computing system 100 further includes a user interface 102, which may comprise one or more peripherals and / or input devices, including, but not limited to. a keyboard, a mouse, a touchpad, or virtually any other input device technology that is communicatively coupled to the computing device 110. The user interface 102 may enable a user to interact with the computing device 110, such as to select one or more images to evaluate, to select one or more parameters of the classification model, and so forth.

[0043] The computing system 100 further includes a display device 104, which may be configured to display outputs of the processor 112 and / or detection module 118 and / or the classification module 120. display the images themselves, and / or display other features / parameters. In some examples, the display device 104 may be a liquid crystal display (LCD), a light-emitting diode display (LED), or an organic light-emitting diode display (OLED). The user may select or otherwise input parameters (e.g., imaging protocols) via the user interface 102 based on options displayed via the display device 104.

[0044] In some examples, the computing device 110 may be implemented over a cloud or other computer network. For example, the computing device 110 is shown in FIG. 1 as constituting a single entity , but it is to be understood that the computing device 110 may be distributed across multiple devices, such as across multiple servers. Processors of the computing device may be single core or multicore, and the programs executed thereon may be configured for parallel or distributed processing. A logic subsystem of the computing device (e.g., memory) may optionally include individual components that are distributed throughout two or more devices, which may be remotely located and / or configured for coordinated processing. One or more aspects of the logic subsystem may be virtualized and executed by remotely accessible networked computing devices configured in a cloud-computing configuration. For example, the images 116 may be stored in the memory 114 of the computing device 110 and accessed by detection module 118 on a remote device. Output from the detection module on the remote device may be sent back to the computing device 110 for storage in the memory 114.

[0045] As used herein, the terms “system” or “module” may include a hardware and / or software system that operates to perform one or more functions. For example, a module or system may include a computer processor, controller, or other logic-based device that performs operations based on instructions stored on a tangible and non-transitory computer readable storage medium, such as a computer memory Alternatively, a module or system may include a hard-wired device that performs operations based on hard-wired logic of the device. Various modules or systems shown in the attached figures may represent the hardware that operates based on software or hardwired instructions, the software that directs hardware to perform the operations, or a combination thereof. For example, the detection module 118 and / or the classification module 120 may be implemented using hardware,Docket No. ARL25305PCTsoftware, firmware, or a combination thereof. In some aspects, several portions of the subject matter described herein may be implemented via Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), semiconductor devices based around a matrix of configurable logic blocks (CLBs), connected via programmable interconnects. Some aspects of the embodiments disclosed herein, in whole or in part, can be equivalently implemented in standard integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as a combination thereof.

[0046] FIG. 2 shows a flowchart that illustrates a method 200 for identifying cells in a FOV image using template matching via a detection module. The detection module may be a non-limiting example of the detection module 118 of FIG. 1. Method 200 may be carried out by a computing device, such as the computing device 110 of FIG. 1 (e.g., by executing instructions stored in memory of the computing device). Method 200 may be executed in response to a request to perform object detection on a selected FOV image. In some examples, the request may be a manual request initiated by a user (e.g., a user may select a FOV image and enter a request to detect objects in the FOV image by selecting a user interface element or menu option). In other examples, the request may be an automatic request initiated in response to execution of a particular imaging protocol. For example, a user may select an imaging protocol from among a plurality of stored imaging protocols. The selected imaging protocol may dictate that object detection be performed on all or on selected FOV images obtained while executing the imaging protocol.

[0047] At 202, method 200 includes obtaining a brightfield FOV image. The brightfield FOV image may include a plurality of cells of a suitable type(s) (e g., mammalian epithelial cells or other eukaryotic cells) obtained with a suitable imaging system, such as imager 106, with brightfield illumination. The brightfield FOV image (also referred to as a FOV image) may have a suitable resolution, such as 0.27 microns per pixel. However, other resolutions are possible, such as resolutions lower than 0.27 microns per pixel (e.g., up to half the resolution of 0.27 microns per pixel).

[0048] At 204, method 200 includes obtaining a reference image. The reference image may be an image of a cellular structure that is present in the cells of the FOV image and that can be detected via template matching. In some examples, the reference image may be an image of an endoplasmic reticulum (ER). In some examples, the reference image may not be an actual image of the cellular structure, but may be a representation of the cellular structure generated by averaging a plurality of images of the cellular structure or via another suitable method. In some approaches, the reference image may be generated via a deep neural network (or some other suitable machine learning process) as shown in FIG. 3 described below.Docket No. ARL25305PCT

[0049] In order to detect each cell in the FOV image, template matching using a reference image may be performed, which may result in bounding boxes being placed over each identified cell in the FOV image. Thus, at 206, method 200 includes performing template matching on the FOV image with the reference image. The template matching may identify area(s) of the FOV image that are similar to the reference image (e.g., that match the reference image). Each area that is deemed to sufficiently match the reference image may be identified as an ER (or nucleus or other cellular structure). The template matching process may result in output of a count of the total nuclei (or ER or other cellular structure) count in the FOV image, as indicated at 208, and / or output of bounding boxes indicative the location of each ER / nucleus and hence cell in the FOV image.

[0050] Performing template matching on the FOV image with the reference image may include validating the FOV image and the reference image. As explained above, the FOV image may be a brightfield image of a plurality of cells and the reference image may be an image of (or representation of) a cellular structure such as an ER or a nucleus. Validating the FOV image and the reference image may include confirming both the FOV image and reference image have been obtained and are 2D arrays. Further, in some examples, validating may include confirming that the size / resolution of the reference image matches the size / resolution of the FOV image. The reference image may be created with an expected magnification (and hence size) of the cells in the FOV image, and the reference image may be scaled (e.g., reduced or increased in size) to match the expected size of the ERs in the FOV image, if needed.

[0051] Performing template matching on the FOV image with the reference image may also include generating a correlation coefficient map. Template matching may utilize a 2D correlation-based method to determine how well each region of the FOV image matches tire reference image, with the degree of matching represented by a correlation coefficient (e.g., Pearson's linear correlation coefficient) for each region. For example, the reference image may be compared to a first region of the FOV image (that is the same size and shape as the reference image) at the upper-left corner of the FOV image, and a first correlation coefficient may be calculated for that region (and placed in the map at a location corresponding to the centroid pixel of the first region). The reference image may be “slid” one pixel to the right relative to the FOV image to compare the reference image to a second region of the FOV image. A second correlation coefficient may be calculated for the second region (and placed in the map at a location corresponding to the centroid pixel of the second region). The process may continue until the reference image has been slid over (and compared to) each respective region of the top of the FOV image; the reference image may then be slid down one pixel and compared to each respective region across the FOV image back toward the left; and so forth until a correlation coefficient has been calculated for each region of the FOV image.

[0052] The template matching may include Fourier domain-based correlation. As such, the FOV image and reference image may be transformed to the frequency domain (e.g., using a FourierDocket No. ARL25305PCTtransform) and the comparison described above may take place in the frequency domain. In the context of identify ing cells in brightfield images, Fourier domain correlation offers significant advantages over traditional spatial domain methods. By transforming images into the Fourier domain, the correlation coefficient detection can be leveraged to perform template matching more efficiently. This approach allows for rapid identification of cell patterns, as it focuses on frequency components rather than pixel intensity' variations, which can be influenced by noise and lighting conditions.

[0053] Using Fourier transforms, the correlation process becomes computationally faster, enabling real-time analysis of large datasets. This is particularly beneficial in high-throughput microscopy, where quick and accurate cell detection is crucial. In contrast, spatial domain methods often demand extensive preprocessing and can be slower due to the need for pixel-wise comparisons, making them less suitable for large-scale applications. Overall, Fourier-based techniques enhance both the speed and robustness of cell identification in brightfield imaging.

[0054] Performing template matching on the FOV image with the reference image may also include filtering and / or thresholding the correlation coefficient map to create a filtered binary' map. The filtering / thresholding may' include filtering by correlation coefficient threshold, local luminance threshold, and / or local contrast threshold, which may act to reduce instances of false positives and false negatives. For example, for correlation coefficient threshold filtering, each correlation coefficient may be compared to a correlation coefficient threshold and the correlation coefficients that are greater than the correlation coefficient threshold may be identified as matching / passing and represented as a value of one on the map. The correlation coefficients that arc less than the correlation coefficient threshold may be represented as a value of zero on the map. For local luminance and / or local contrast thresholding, a similar process may be applied. For example, a mean luminance of each region of the FOV image determined to match the reference image (e.g., the regions assigned a value of one based on the correlation coefficient) may be calculated and compared to a luminance threshold. If the mean luminance is above the luminance threshold, the region is confirmed as matching the reference image and the value of one is maintained; if the mean luminance is not above the threshold, the region is not confirmed as matching the reference image and the value is changed to zero. A similar process may be applied for local contrast thresholding using a contrast threshold. Each threshold may be a minimum allowable value. The thresholds may be selected manually via example image audits or selected automatically by comparing benchmarks of bounding box counts and locations with respective bounding boxes found using nuclear seeding (e.g., nuclear stain detected in a fluorescent channel, which provides high contrast for easy detection, location, and counting).

[0055] Performing template matching on the FOV image with the reference image may also include performing local maxima detection in the filtered binary map. The local maxima detection may identify pixels of the filtered binary map that have a value higher than all of their immediate neighboringDocket No. ARL25305PCTpixels. Each pixel / local maximum identified may be designated as an initial detected object (e.g., ER / nucleus / cellular structure).

[0056] Performing template matching on the FOV image with the reference image may also include calculating initial bounding boxes around the initial detected objects. The bounding boxes may be centered on the identified pixel and have a shape that matches the reference image and a size based on an expected size of the detected objects (e.g.. an average or expected cell size). The bounding boxes may be filtered by a bounding box overlap threshold. The bounding box overlap threshold may be a percentage of overlap between neighboring bounding boxes, such as 50% overlap or 75% overlap. If neighboring bounding boxes overlap by more than the bounding box overlap threshold, one of the bounding boxes may be removed or a bounding box may not be placed to begin with. For example, bounding boxes are considered to be added for local maxima points. When a bounding box is in the process of being added, the bounding box is checked relative to the locations of any existing bounding boxes. If the bounding box to be added overlaps an existing bounding box by more than the bound box overlap threshold, the bounding box is not added. The bounding boxes that remain may be the final bounding boxes that may be used to generate the output (e.g., cell / nucleus / ER count, the bounding boxes themselves).

[0057] Performing template matching on the FOV image with the reference image may also include performing mask processing when it is desired to segment the detected objects in order to do morphology analysis or another desired task on the detected objects. The mask processing may include normalizing and thresholding a mask for each detected object and optionally removing speckle noise. Each mask may be a binary image w here each pixel is classified as part of a detected object or not, w ith pixels identified as part of the detected object given a value of 1 and pixels that are not part of the detected object given a value of 0. The mask processing may be performed with a suitable model, such as a neural network. The bounding boxes placed as described above may be initial bounding boxes that are centered on the initial detected object (e.g., cellular structure such as the ER). The initial bounding boxes may be used to divide the FOV image into sub-images / image patches and each sub-image / image patch may be processed via the mask processing. In some examples, the mask processing may include using masks (e.g., nuclear or cellular masks) to identify the nucleus or cell in each sub-image / image patch. One or more or all of the initial bounding boxes may be adjusted in position so that the final bounding boxes are re-centered over the nucleus or cell. By using sub-images instead of the entire FOV image or tiles of the FOV image, the generation of the masks may be performed faster and with smaller neural networks that utilize less training data. Further, the template matching described herein is much more efficient than using a neural network to find cell (via ER) instances.

[0058] When the bounding boxes are overlaid on the FOV image, the bounding boxes may’ include rectangles that are visible on the FOV image. However, it is to be appreciated that the bounding boxes may be represented as pixel coordinates of the FOV image that identify the location and size of eachDocket No. ARL25305PCTbounding box. For example, each bounding box may be represented as the coordinates of the FOV image for the top left corner and bottom right corner of the bounding box (or the top right comer and bottom left corner). The total nuclei / ER / cell count output at 208 and / or the bounding boxes output at 210 may be displayed on a display device (e.g., display device 104) and / or saved in memory (e.g., memory 114) for use in or more downstream tasks. For example, the bounding boxes may be used to visually identify the boundary of each identified cell by overlaying the bounding boxes on the FOV image, which may then be displayed on the display device. In some examples, the bounding boxes may be used to segment the FOV image into instance images, also referred to as sub-images. For example, the FOV image may be partitioned / divided into a plurality of sub-images based on the coordinates of the bounding boxes, with each sub-image including a region of the FOV image contained within the coordinates of a respective bounding box.

[0059] As indicated at 212, method 200 may optionally include defining sub-images for further processing based on the bounding boxes. The processing of the sub-images may include, as indicated at 214, processing the sub-images for segmentation and / or direct cell classification. For example, the classification module 120 of FIG. 1 may be invoked to classify each cell in each sub-image. The classification may include classifying cells as alive or dead, classifying by mitotic phase, and / or classifying by another suitable morphological feature. The model(s) of the classification module that are invoked to segment and / or classify the cells may be trained to use sub-images as input (e.g., where each sub-image includes only a single cell), rather than an entirety of the FOV image or image tiles of the FOV image that include multiple cells. Segmentation for each nucleus or cell may be used to create re-centered respective bounding boxes. For example, each originally -placed bounding box may be centered over a respective ER. Segmentation may be used to move one or more bounding boxes so the bounding boxes are centered around the nucleus or cell. This can result in significant improvements in measurement accuracy, for example, of the area of respective objects.

[0060] Using template matching to detect cells in a brightfield FOV image has many advantages. For example, utilizing a brightfield image allows for elimination of fluorescent dyes / stains, which reduces cytotoxicity and photobleaching issues associated with fluorescent dyes like DAPI and Hoechst. Thus, safer and more sustainable live cell imaging may be performed, preserving cell viability and integrity over longer periods. Additionally, the use of brightfield microscopy allows for utilization of existing brightfield microscopy setups, which are more widely available and less expensive than advanced fluorescence microscopy systems, thereby increasing accessibility and reducing costs for laboratories, making high-content screening more feasible for a broader range of research and clinical applications.

[0061] Further, the template matching to initially identify the cells allows for additional downstream processing to be performed with relatively smaller (so more efficient) Deep Neural Networks (DNNs). In this way, very small DNNs that can be trained rapidly with a relatively smallDocket No. ARL25305PCTdataset may be employed for morphological analy sis or other tasks. Most any advancements in DNNs can be exploited with a smaller input and therefore total memory and nodes required. This reduces computational resources and training time, allowing for quicker deployment and iteration of models. Similarly, instance and semantic segmentation may be employed to facilitate accurate detection and segmentation of cell nuclei in brightfield images using advanced image processing techniques, which improves the accuracy and reliability of cell nucleus detection, leading to better data quality and more robust analysis in high-content screening.

[0062] FIG. 3 shows an example method 300 for generating a reference image of a cellular structure for object detection in an image. The method shown in FIG. 3 may be used to generate optimal reference images that may be used in template matching object detection in an image. For example, the reference image output(s) of method 300 may be used as the reference image in method 200 shown in FIG. 2 described above for object detection in an image. Method 300 may be carried out by a computing device, such as the computing device 110 of FIG. 1 (e.g., by executing instructions stored in memory of the computing device). The methods described herein are not limited to brightfield images and may work generally for generating a reference image for cellular object detection, including in fluorescence images. Any objects that are to be identified via template matching may benefit from the methods described herein.

[0063] At 302, method 300 includes obtaining images of a cellular structure. For example, a plurality' of images of a cellular structure may be obtained, wherein the plurality of images of a cellular structure comprises images of an ensemble of target objects wherein the target objects have already- been identified. For example, the cellular structure may comprise nuclei and the plurality- of images of a cellular structure may have been identified via fluorescent nuclear stained images. The cellular structure may comprise any suitable cellular structure such as nuclei, endoplasmic reticulum, or other cellular structures.

[0064] The plurality of images of the cellular structure may be obtained in any suitable way. For example, a data set comprising a plurality of cellular structures may be input by a user into a computing device via transferring from a memory storage component that may be local to the computing device or that may be remotely located, e.g.. on a remote server. The plurality of images of a cellular structure that are obtained may comprise a number of images that have already been identified as having the cell structure within the image as well as a number of images that have already been identified as not having the cell structure within the image. The identification of the cell structures in the plurality of images may be performed in various ways, such as via staining approaches in prior image analysis to identify the structures, by manual identification of structures in the images, by machine learning identification of the structures in the images and / or a combination of identification processes.

[0065] In some examples, the plurality of images of a cellular structure may comprise an ensemble of images packed as respective channels of an input image. As another example, the plurality ofDocket No. ARL25305PCTstructures may comprise a plurality of different input images. The input images may comprise a set of images that have been positively identified (using a suitable metric such as a correlation coefficient or some other similarity metric) and a set of images that have been negatively identified as substantially not having the cellular structure. These sets of input images may be used to train a computing device via a suitable machine learning approach. Examples of these input images are shown in FIGS. 10, 11, 16, and 17 described in more detail below.

[0066] In some examples, method 300 may further comprise applying bounding boxes centered on and offset from each target object and labeling bounding boxes centered on an object with a first indicium and labelling bounding boxes offset from an object by a threshold amount with a second indicium. The threshold may comprise a predetermined threshold that a similarity metric (such as correlation coefficient) may use to assign a degree of confidence to identifications of the structures in each the image and may be based on physical properties or metrics of the cellular structure being targeted. For example, if the target objects are nuclei, then the threshold amount may be an average diameter of a nucleus. As another example, for ground truth image set generation the number 1 may be used for positive and 0 for negative; and for prediction r may be thresholded by a given threshold (which may be empirically derived) value to label 1 for above threshold and 0 for below.

[0067] The indicium provided as labels may comprise any suitable label indicating a positive or negative identification of a cellular structure in the image. For example, the first indicium may comprise a label that specifies that the image positively has the cellular structure within it and the second indicium may comprise a label that specifics that the image docs not have the cellular structure within it. For example, the first indicium may comprise a label of 1 and / or “detected’’ to indicate that the image is one in which the cellular structure is positively detected in the image and the second indicium may comprise a label of 0 and / or “undetected” to indicate that the image is one in which the cellular structure is not detected in the image. These classifications may be applied prior to inputting the images into a machine learning protocol such as a deep neural network (DNN) for training. Examples of bounding boxes overlaid on cellular structures in an image are shown in FIGS 9,14, 15, 25- 33 described in more detail below

[0068] At 304. method 300 include training a deep neural network (DNN) or some other suitable machine learning protocol based on the images in the plurality of images obtained in step 302. An example of steps included in training a DNN are shown in more detail in FIG. 4 described below. Training a deep neural network (DNN) includes generating a reference image via the deep neural network at 306. For example, this may be an initial reference image output by the DNN. This initial reference image be iteratively updated (based on calculated correlation coefficients as described below) to optimize the reference image for use in subsequent object detection via template matching.

[0069] Training a deep neural network also includes calculating a Pearson’s linear correlation coefficient (or some other suitable similarity metric) between each image in the plurality of images andDocket No. ARL25305PCTthe reference image at 308. Training a deep neural netw ork also includes refining the reference image at 310. For example, the reference image may be refined or updated based on the calculated Pearson’s linear correlation coefficients. For example, the reference image may be updated to maximize the Pearson’s linear correlation coefficient between each image in the plurality of images and the reference image. For example, the model output may be used directly for detection is the binary classification per input image (classified as 1 for detected or 0 for not based on thresholding the respective Pearson's linear correlation coefficient), and so a standard loss (binary' cross-entropy, a.k.a. log loss) function may be used for training.

[0070] In some examples, refining the reference image may comprise classifying each image in the plurality of images relative to the reference image based on a threshold applied to the Pearson’s linear correlation coefficient calculated between the image and the reference image. For example, classifying each image in the plurality of images relative to the reference image may comprise assigning a classification of detected (1) or undetected (0) to the image based on the Pearson’s linear correlation coefficient calculated between the image and the reference image. This step may include comparing each Pearson’s linear correlation coefficient between each image in the plurality of images and the reference image with a respective ground truth label of 1 or 0, for example. Refining the reference image may comprise updating the reference image (via the DNN) based on the threshold applied to the Pearson’s linear correlation coefficient calculated between the image and the reference image. For example, refining the reference image may comprise generating a reference image that has a maximum separation between positive Pearson’s linear correlation coefficient statistics (corresponding to images identified as detected or labelled 1) and negative Pearson’s linear correlation coefficient statistics (corresponding to images identified as undetected or labelled 0) for the images in the plurality of images.

[0071] Training a deep neural network also includes outputting the reference image at 312. This reference image may be output to a display device and / or a memory component of a computing device, e.g., for use in subsequent object detection processes performed by template matching against the reference image, e.g., as shown in FIG. 2 described above. For example, the reference image output from 312 may be used as input in step 204 in method 200 show n in FIG. 2.

[0072] In some examples, method 300 may be used to generate a plurality of reference images via the deep neural network. When generating a plurality of images, the method may comprise calculating a Pearson’s linear correlation coefficient between each image in the plurality of images and each of the reference images; refining the reference images based on the calculated Pearson’s linear correlation coefficients; aggregating the references images to generate a single reference image; and outputting the reference image.

[0073] FIG. 4 shows an example method 400 of using a deep neural network (DNN) for template reference generation. As in FIG. 3, the method shown in FIG. 4 may be used to generate optimal reference images that may be used in template matching object detection in an image. As in methodDocket No. ARL25305PCT300 described above, the reference image output(s) of method 400 may be used as the reference image in method 200 shown in FIG. 2 described above for object detection in an image. Method 300 may be carried out by a computing device, such as the computing device 110 of FIG. 1 (e.g.. by executing instructions stored in memory of the computing device).

[0074] At 402. method 400 includes receiving a set of input images of cellular structures to be used to train the DNN. Each image in the set of images (i.e., the plurality of images) may actually comprise an ensemble of images packed as respective channels of the input image. Batches of these images may comprise sets of images with these multiple channels. For example, for a batch of B of these input images with C channels, the input may comprise effectively B*C images with B respective output images to be generated. These B*C images may be the “sub-images” defined by bounding boxes described above.

[0075] Method 400 then proceed train the DNN based on the input set of images. It should be understood that, although the example method 400 shown in FIG. 4 uses the example of a DNN, any suitable machine learning or artificial intelligence algorithm or process may be used and trained on the input images so as to be configmed to output optimal reference images.

[0076] In this example, training the DNN based on the input set of images may comprise Applying 32 convolutional filters of size 3x3 with rectified liner unit (ReLU) activation to the input images, i.e. calculating Conv2D (32, 3x3, rein) of the DNN at 404. Performing max pooling with a 2x2 pool size to reduce the spatial dimensions, i.e., performing MaxPooling2D (2x2) of the DNN at 406. Applying 64 convolutional filters of size 3x3 with ReLU activation, i.e., calculating Conv2D (64, 3x3, rein) of the DNN at 408. Performing another max pooling with a 2x2 pool size, i.e., performing MaxPooling2D (2x2) of the DNN at 410. Applying another set of 64 convolutional filters of size 3x3 with ReLU activation, i.e., calculating Conv2D (64, 3x3, relu) at 412. Flattening the data into a one-dimensional array at 414. Passing the flattened data through a dense layer with 128 units and ReLU activation, i.e., calculating Dense (128, relu) of the DNN at 416. Applying dropout with a rate of 0.5 to prevent overfitting, i.e., applying Dropout (0.5) of the DNN at 418. Processing the data through another dense layer with linear activation, i.e.. performing Dense (linear) of the DNN at 420. In some examples, method 400 may also include reshaping the output to desired dimensions, i.e., performing Reshape (height, width, 1) of the DNN.

[0077] In FIG. 4, outputs are shown in 422 and include an output image 424 that is output from the DNN and which is fed into a correlation layer 426. In the correlation layer 426 the Pearson’s linear correlation coefficient r is calculated between each of the C input respective channel images and the output image. As shown in FIG. 4 it may be advantageous to include the correlation layer 426 as its own separate module, which is configured to be executed independently from the steps of the DNN training (e.g., steps 404 to 420). The Pearson's linear correlation coefficient r calculation may be calculated within a layer of the deep neural network: It is embedded in the model. This is meant toDocket No. ARL25305PCTinsure that training and inference correlations match (vs including the r calculation as part of a complex custom loss function). The model includes the calculation of r, and thresholding all the respective input image r values to get respective positive and negative class values (0 or 1) and these binary values are what get output and checked against the input image annotations as ground truth. This allows the very simple and generic loss functions for binary classification to be used. Training stability may be much simpler in practice using such an approach, by not calculating r as a separate operation outside the model. Thus, the r calculation may be included inside the model and used to classify the ensemble of input images. The ensemble of input images may comprise many images input, with r calculations for all against a single output (all the above may be multiplied by B for batches, for example).

[0078] At 428, method 400 includes applying a trainable threshold to refine the output. The threshold may be applied to the respective C r values per output image to create C classifications as detected (1) or undetected (0). Method 400 then includes producing the final output image maximizing r for the ensemble of C channel images. Additionally, method 400 may include assigning predicted labels of the DNN. which are the classifications per output image, (the 1’s and 0’s for detection) based on the r values and the threshold.

[0079] FIG. 5 shows an example method 500 for generating a reference template image. As above, the method shown in FIG. 5 may be used to generate optimal reference images that may be used in template matching object detection in an image. For example, the reference image output(s) of method 500 may be used as the reference image in method 200 shown in FIG. 2 described above for object detection in an image. Method 500 may be carried out by a computing device, such as the computing device 110 of FIG. 1 (e.g., by executing instructions stored in memory of the computing device).

[0080] At 502. method 500 includes inputting an initial reference template image for template matching. FIG. 7 shows an initial template match image example, IrefO, which may be manually selected from images of nuclei. These images of nuclei may comprise fluorescent images (for example reference template input and output). It should be understood that any suitable cellular structure may be used and an initial template image of that structure may be selected in this step. Method 500 may include various other inputs as well. For example, additional inputs may include FOV image(s) of fluorescence channel with nuclear seed (stain) for ground truth for nuclei / cell locations and respective ground truth "bounding boxes,” and FOV image(s) of brightfield channel for applying ground truth bounding boxes to the respective cells within each respective brightfield FOV. For example, by applying ground truth bounding boxes to these brightfield FOVs, sub-images can be created for positive class input images to the DNN. Applying spatial offsets to these ground truth bounding boxes and then applying these shifted ground truth bounding boxes to the same respective brightfield FOVs may provide negative class input images for the DNN.

[0081] At 504, method 500 includes obtaining a first set of bounding boxes (bboxes) via template matching using the initial reference template image IrefO. In particular, the initial reference templateDocket No. ARL25305PCTimage may be used to detect target objects in a field of view image using Pearson's cross correlation coefficient r local maxima to obtain the initial set of respective bounding boxes. The bounding boxes may be obtained from die nuclear stained fluorescence FOV image(s). Then the resulting fluorescent template reference image may be used to re-measure the same nuclear stained fluorescence FOV image(s) to (generally) obtain a more accurate detection result set, including a more accurate set of bounding boxes. For obtaining brightfield reference templates, this step of obtaining a superior fluorescent template reference image is not typically required, but is recommended to insure the best accuracy of the ground truth bounding boxes used for creating the training (positive and negative) subimage set (via the "generator" as is common practice for DNN training). For example, FIG. 6 shows 1) creating a better fluorescent nuclear stain based template reference image, 2) then use that to create a more accurate bbox set for use for generating images for training a second DNN for generating a brightfield template reference image (as described below).

[0082] At 506, method 500 includes recording locations and sizes of each bounding box, i.e., documenting the locations and sizes of each bounding box. For example, FIG. 8 shows an example heatmap of bounding box (bbox) count per well using a reference template image for correlation coefficient based template matching. At 508, method 500 may include auditing bounding boxes. For example, the bounding boxes may be overlaid on the input images for auditing purposes. For example, FIG. 9 shows an example of bounding boxes overlaid on a nuclear channel after detecting nuclei with the initial reference template image, IrefO.

[0083] At 510, method 500 includes creating or obtaining positive class input for the DNN. Positive class input images may be generated by creating sub-images defined by the bounding boxes. For example, FIG. 10 shows example sub-images used as a “positive” class for generating a reference template image. The example sub-images shown in FIG. 10 comprise fluorescence stained nuclear images for generating a reference template image.

[0084] At 512. method 500 includes creating or obtaining negative class input for the DNN. Negative class input images may be generated by spatially offsetting the bounding boxes to intentionally miss the target object. For example, similar to FIG. 10, sub-images used as a “negative” class may be used for generating a reference template image. These sub-images may comprise fluorescence stained nuclear images for generating a reference template image.

[0085] At 514. method 500 includes training the untrained DNN, which results in a usable reference template image. At 516, method 500 includes using the trained DNN to produce an output template image from a single or batch of input images. At 518, method 500 includes aggregating the output images to create a single reference template image. This reference template image may comprise a nuclear stain channel generated reference template image, Irefl.

[0086] FIG. 6 shows an example method 600 for generating a brightfield nuclear reference template image given nuclear seed / stain images and brightfield images. Method 600 is similar toDocket No. ARL25305PCTmethod 500 described above but the results are cascaded to create a brightfield reference template image. As above, the method shown in FIG. 6 may be used to generate optimal reference images that may be used in template matching object detection in an image. For example, the reference image output(s) of method 600 may be used as the reference image in method 200 shown in FIG. 2 described above for object detection in an image. Method 600 may be carried out by a computing device, such as the computing device 110 of FIG. 1 (e.g., by executing instructions stored inmemory of the computing device).

[0087] At 602. FIG 6 shows various inputs which may include an initial reference template image (IrefO) 604, fluorescent nuclear stained images 606 and brightfield images 608. Various outputs 612 are shown in FIG. 6, including nuclear stained reference template image (Irefl) 614 and brightfield nuclei reference template image (lref2) 618.

[0088] Method 600 uses an initial reference template image (Ireffl) 604 to generate a first generated template reference image (Irefl) at 610. The first reference template image is generated from the initial reference template image (IrefO) and fluorescent nuclear stained images 606 using a DNN trained on those images. This process outputs a nuclear stained reference template image 614.

[0089] Method 600 includes using the first reference template image (Irefl) 612 to generate a second generated template reference image (lref2) 618. The second generated template reference image (lref2) 618 is generated from a DNN with inputs comprising nuclear stained reference template image 614 and brightfield images 608. This process outputs brightfield nuclei reference template image 618.

[0090] As in method 500 above, method 600 includes similar steps to generate these different template images. For example, method 600 may include recording locations and sizes of each bounding box. For example, similar to as shown in FIG. 8. a heatmap of bounding box (bbox) count per well using a brightfield reference template image for correlation coefficient based template matching may be generated. Method 600 may also include auditing bounding boxes by overlaying bounding boxes on the input images. For example, similar to as shown in FIG. 9, bounding boxes may be overlaid on a nuclear channel after detecting nuclei with an initial brightfield reference template image (Irefl). The bounding boxes may be overlaid on a brightfield channel after detecting nuclei with a generated reference template image (Irefl).

[0091] Like method 500, method 600 also includes creating or obtaining positive class input for the DNN. The positive class input images may be generated by creating sub-images defined by the bounding boxes. Method 600 also includes creating or obtaining negative class input for the DNN. The negative class input images may be generated by spatially offsetting the bounding boxes to intentionally miss the target object.

[0092] FIG. 11 shows an example nuclear stain channel derived bounding box set applied to brightfield FOVs (ground truth) based generated brightfield reference template image that may be output by method 600. For example, the nuclear stain channel may be used to get the bounding boxesDocket No. ARL25305PCTto create the training dataset by applying them to brightfield FOV images. This template image output by method 600 may be used as the reference image in method 200 shown in FIG. 2 described above for object detection in an image.

[0093] FIG. 12 shows an example plot of accuracy versus image quality. In particular, FIG. 12 shows high accuracy in template match based detection using the generated reference template in brightfield. Each point (dot) of FIG. 12 represents the aggregate match accuracy for a single FOV, where each FOV has a plurality (typically in the hundreds) of cells. FIG. 12 illustrates the efficiency, speed and accuracy of the systems and method disclosed herein. In FIG. 12, each point represents the bbox (bounding box) match accuracy = (tp + tn) / (tp + tn + fp + fn) in brightfield matched to each respective nuclear stained fluorescent channel bbox. The matched bbox counts are tp for true positive, tn for true negative, fp for false positive and fn for false negative. FIG. 12 shows that for good image quality, the brightfield template matching using the optimized reference template image of this method results in very high accuracy (matches each object detected by the nuclear stain channel) for cases where image quality is decent.

[0094] The following claims particularly point out certain combinations and sub-combinations regarded as novel and non-obvious. These claims may refer to “an” element or “a first” element or the equivalent thereof. Such claims should be understood to include incorporation of one or more such elements, neither requiring nor excluding two or more such elements. Other combinations and subcombinations of the disclosed features, functions, elements, and / or properties may be claimed through amendment of the present claims or through presentation of new claims in this or a related application. Such claims, whether broader, narrower, equal, or different in scope to the original claims, also are regarded as included within the subject matter of the present disclosure.

Claims

Docket No. ARL25305PCTCLAIMS1. A method for generating a reference image of a cellular structure for object detection in an image, comprising:obtaining a plurality of images of a cellular structure;training a deep neural network based on the plurality of images;generating a reference image via the deep neural network;calculating a Pearson’s linear correlation coefficient between each image in the plurality of images and the reference image;refining the reference image based on the calculated Pearson’s linear correlation coefficients; andoutputting the reference image.

2. The method of claim 1, w herein refining the reference image comprises classifying each image in the plurality of images relative to the reference image based on a threshold applied to the Pearson’s linear correlation coefficient calculated between the image and the reference image.

3. The method of claim 2, wherein classifying each image in the plurality of images relative to the reference image comprises assigning a classification of detected or undetected to the image based on the Pearson’s linear correlation coefficient calculated between the image and the reference image.

4. The method of claim 2, wherein refining the reference image further comprises updating the reference image based on the threshold applied to the Pearson’s linear correlation coefficient calculated between tire image and the reference image.

5. The method of claim 1, w herein refining the reference image comprises generating a reference image that has a maximum separation betw een positive Pearson’s linear correlation coefficient statistics and negative Pearson’s linear correlation coefficient statistics for the images in the plurality of images.

6. The method of claim 1, wherein training the deep neural network comprises comparing each Pearson’s linear correlation coefficient between each image in the plurality of images and the reference image with a respective ground truth label of 1 or 0.

7. The method of claim 1. wherein the plurality of images of a cellular structure comprises images of an ensemble of target objects wherein the target objects have already been identified and the method further comprises applying bounding boxes centered on and offset from each target object and labelingDocket No. ARL25305PCTbounding boxes centered on an object with a first indicium and labelling bounding boxes offset from an object by a threshold amount with a second indicium.

8. The method of claim 7, wherein the target object are nuclei and the threshold amount is an average diameter of a nucleus.

9. The method of claim 1, wherein the plurality of images of a cellular structure comprises an ensemble of images packed as respective channels of an input image.

10. The method of claim 1, wherein the cellular structure comprises nuclei and the plurality of images of a cellular structure have been identified via fluorescent nuclear stained images.

11. The method of claim 1, further comprising:generating a plurality of reference images via the deep neural network; calculating a Pearson’s linear correlation coefficient between each image in the plurality of images and each of the reference images;refining the reference images based on the calculated Pearson’s linear correlation coefficients;aggregating the references images to generate a single reference image; and outputting the reference image.

12. The method of claim 1. wherein the cellular structure comprises a cell nucleus structure.

13. A system, comprising:memory storing instructions; andone or more processors configured to execute the instructions to:obtain a plurality of images of a cellular structure;train a deep neural network based on the plurality of images;generate a reference image via the deep neural network;calculate a Pearson’s linear correlation coefficient betw een each image in the plurality of images and the reference image;update the reference image based on the calculated Pearson’s linear correlation coefficients; andoutput the reference image.Docket No. ARL25305PCT14. The system of claim 13, wherein updating the reference image comprises classifying each image in the plurality of images relative to the reference image based on a threshold applied to the Pearson’s linear correlation coefficient calculated between the image and the reference image.

15. The system of claim 14, wherein the cellular structure is a nucleus and tire threshold amount is an average diameter of a nucleus.

Images